AI Developer Digest

Sun, Jul 12, 2026

8 signals that cleared the gate43 scanned27 min read

The Signal — start here

July 12 is a migration day for inference infrastructure. Two simultaneous forced-migration events hit production stacks at once: Google's Gemini 2.5 Flash, Flash Lite, and Pro began returning 404 "no longer available" errors as of July 9 — three months before the documented October 16 shutdown date — with no official explanation and no resolution as of this digest. Separately, vLLM v0.25.0 (released July 11) removes PagedAttention entirely and makes Model Runner V2 the default for all dense models, requiring developers who depended on the legacy attention backend to migrate. On the same day, Terry Tao (Fields Medal, UCLA) published a practitioner post documenting how he used Claude Code and modern coding agents to port 27-year-old Java 1.0 math visualizations to JavaScript in hours — and found two bugs his code had silently carried for nearly three decades. The llama.cpp rapid-release cadence adds a DeepSeek V3.2/V4 lightning indexer and a multimodal tool result fix, while LiteLLM v1.92.0 ships per-MCP-server rate limiting.

Must-reads today

Gemini 2.5 Flash/Pro unexpected 404s — models returning "no longer available" since July 9 despite Oct 16 shutdown date; migrate to Gemini 3.5 Flash or Gemini 3.1 Pro immediately

vLLM v0.25.0: PagedAttention removed — if you run vLLM, this is a required migration; MRv2 is now the only execution path for dense models

Terry Tao on coding agents — the #1 HN story July 12; concrete first-person account of what production agentic coding actually does on real scientific codebases

Breaking Changes

●Breaking

Gemini 2.5 Flash, Flash Lite, and Pro — 404 "no longer available" since July 9, three months before official shutdown date

What changed

Starting July 9, 2026 (~11:50 AM Pacific), API calls to gemini-2.5-flash, gemini-2.5-flash-lite, and gemini-2.5-pro on generativelanguage.googleapis.com began returning HTTP 404 NOT_FOUND with the message: "This model models/gemini-2.5-flash is no longer available. Please update your code to use a newer model for the latest features and improvements." The models still appear in ListModels responses. The Google Gemini API deprecation page (updated July 2, 2026) had listed October 16, 2026 as the shutdown date for gemini-2.5-flash — making the current 404s three months ahead of schedule with no official announcement or changelog entry.

TL;DR

gemini-2.5-flash, gemini-2.5-flash-lite, and gemini-2.5-pro are returning 404 on the Gemini API as of July 9, 2026, three months before their documented October 16 deprecation date; no Google explanation has been published as of July 12.

Developer signal

Any production service calling gemini-2.5-flash, gemini-2.5-flash-lite, or gemini-2.5-pro directly via generativelanguage.googleapis.com is currently broken. The required action: (1) Flash/Flash Lite users: migrate to gemini-3.5-flash (GA since Google I/O May 20, 2026; $1.50/1M input, $9/1M output; same speed tier). (2) Pro users: migrate to gemini-3.1-pro (the documented replacement). Note: this is NOT a planned upgrade with advance notice — the models went offline unexpectedly. Whether this is an accidental early retirement or an intentional undocumented accelerated shutdown is not confirmed by Google as of this digest. Check GET https://generativelanguage.googleapis.com/v1beta/models/{model} to verify if a model is still responding before deciding on migration path. Separately, GitHub Copilot announced July 2 that it will deprecate Gemini 2.5 Pro and Gemini 3 Flash from all Copilot experiences on July 31, 2026 — an independent but thematically related change for developers using GitHub Copilot model selection.

Affects you ifYou call gemini-2.5-flash, gemini-2.5-flash-lite, or gemini-2.5-pro via the Gemini API in any production or development environment; you have hard-coded model strings referencing the 2.5 model family.EffortSignificant — update model strings and retest; behavior differences between 2.5 Flash and 3.5 Flash may require prompt adjustments.

Google AI Developer Forum (multiple threads) | Date: Started July 9, 2026; ongoing July 12 | Link: https://discuss.ai.google.dev/t/gemini-2-5-flash-deprecated-without-warning-earlier-than-shutdown-date/174217https://discuss.ai.google.dev/t/gemini-2-5-flash-deprecated-without-warning-earlier-than-shutdown-date/174217 (direct fetch blocked at time of review; confirmed via multiple independent search results citing the same error message and thread content)

●Breaking

vLLM v0.25.0 — PagedAttention removed; Model Runner V2 now the only execution path for dense models; multiple models removed

What changed

vLLM v0.25.0 removes PagedAttention entirely (#47361) — the legacy attention implementation is deleted, not deprecated. Model Runner V2 (MRv2) is now the default for all dense models (previously default only for Llama, Mistral, and Qwen3). Additionally, several models are removed entirely from the codebase: Baichuan, Aquila, Grok (open-weight), Tarsier/Tarsier2, AyaVision/MusicFlamingo, and Mantis. The legacy api_server.py is moved to the examples directory.

TL;DR

vLLM v0.25.0 removes PagedAttention and the V0 backend with no fallback — all inference now runs through MRv2; any configuration, benchmark, or custom code that relied on PagedAttention behavior is broken on upgrade.

Developer signal

This release requires an explicit migration audit before upgrading. (1) PagedAttention removal: If your configuration referenced V0-specific settings (e.g., attention backend flags that forced the old path), they no longer exist. MRv2 is now the only path. The VLLM_USE_V2_MODEL_RUNNER=1 flag is now irrelevant — MRv2 is the default and there is no opt-out back to PagedAttention. Run your existing test suite on v0.25.0 before deploying to production. (2) Model removals: If you are serving Baichuan, Aquila, Tarsier, AyaVision, MusicFlamingo, or Mantis models, they are not supported in v0.25.0 — pin to v0.24.x for now. (3) api_server.py moved: If you directly imported or ran the legacy API server from its old location, update your launch scripts to use vllm serve instead. (4) gptq_marlin removed from ROCm quantization: If you are running quantized models on AMD GPUs with gptq_marlin, this scheme is no longer available on ROCm; check alternative quantization options. (5) Image decompression-bomb fix: vLLM v0.25.0 adds a DoS prevention fix for malformed images — a security improvement, but if you tested against specifically crafted test images, behavior may differ. MRv2 supports the same model classes as the removed PagedAttention backend (Llama, Mistral, Qwen, Gemma, Falcon, Phi, and all other dense transformer architectures).

Affects you ifYou run vLLM in production; you serve any of the removed models (Baichuan, Aquila, Grok open-weight, Tarsier, AyaVision, Mantis); you use AMD ROCm with gptq_marlin quantization; you import or script api_server.py directly; you have integration tests that verify PagedAttention-specific memory behavior.EffortModerate to Significant — depends on which removed features/models you use; pure users of major models (Llama, Mistral, Qwen) likely face Quick-to-Moderate migration.

vllm-project/vllm GitHub Releases | Date: July 11, 2026 | Link: https://github.com/vllm-project/vllm/releases/tag/v0.25.0https://github.com/vllm-project/vllm/releases/tag/v0.25.0

Model Releases

No new model releases in the last 24 hours. (Gemini 3.5 Flash launched May 20, 2026; not new today.)

API & SDK Changes

No new API or SDK changes beyond the Breaking Changes section above.

Research

Nothing cleared the quality bar this period. No papers from cs.AI, cs.CL, or cs.LG new submissions for July 11-12, 2026 combined recognized-lab authorship, an associated code repository, and measurable benchmark numbers at the required threshold within the scan window.

Tooling

High

vLLM v0.25.0 — 558 commits: MRv2 default, new Streaming Parser Engine, MiniMax-M3 with NVFP4, Rust frontend HTTPS/mTLS, security hardening

What changed

v0.25.0 is vLLM's largest release since the V1 rewrite — 558 commits from 232 contributors (64 new). Beyond the breaking changes (PagedAttention removal, model removals — see Breaking Changes), the headline additions are: Model Runner V2 expanded capabilities including EVS, real-time embeddings, prefix caching for Mamba hybrid models, and multimodal prefix bidirectional attention; a new Streaming Parser Engine for unified tool-call and reasoning parsing; MiniMax-M3 support with pipeline parallelism and NVFP4 precision; and significant new model support including LLaVA-OneVision-2, Unlimited OCR, MOSS-Transcribe-Diarize, openai/privacy-filter, Hy3, GLM-5/DeepSeek-V3.2 in the model zoo with GLM-5.2 tuning.

TL;DR

vLLM v0.25.0 ships 558 commits that make MRv2 the sole execution path, add a Streaming Parser Engine for tool-call/reasoning, bring in MiniMax-M3 (NVFP4), LLaVA-OneVision-2, and Unlimited OCR model support, and harden against image decompression-bomb DoS attacks.

Developer signal

Beyond migration (see Breaking Changes section), the new capabilities to act on: (1) Streaming Parser Engine: Tool-call and reasoning parsing is now handled by a unified parser framework — this matters if you use vLLM as a backend for agentic frameworks and previously had to handle inconsistent tool-call parsing. Kimi k2.5/k2.6/k2.7 is now supported in the parser. (2) MiniMax-M3 with NVFP4: If you have H100/H200 hardware that supports NVFP4 precision, MiniMax-M3 is now natively supported with pipeline parallelism — relevant for high-throughput MoE inference. (3) Universal speculative decoding: Heterogeneous vocabulary support means speculative decoding now works across more model combinations. The new DSpark and DFlash drafters are available. (4) Rust frontend: Now supports HTTPS/mTLS — if you're running vLLM behind a reverse proxy that requires mTLS, this removes a layer of complexity. (5) Sleep mode pluggable backend: For distributed inference setups, the new sleep-mode abstraction allows custom backends to handle idle-state resource reclamation. (6) Bounded tokenizer with explicit truncation: Previously, extremely long inputs could cause unbounded tokenizer work; now explicit truncation settings are enforced — relevant if you process untrusted user input.

Affects you ifYou run vLLM as an inference backend for any production LLM application; you use speculative decoding; you use tool-calling via vLLM's API; you serve MiniMax-M3 or other newly added models.EffortModerate — review Breaking Changes section first; core users of major models (Llama, Mistral, Qwen, Gemma) should upgrade and test; users of removed models or removed quantization schemes need workarounds.

vllm-project/vllm GitHub Releases | Date: July 11, 2026 | Link: https://github.com/vllm-project/vllm/releases/tag/v0.25.0https://github.com/vllm-project/vllm/releases/tag/v0.25.0

Notable

LiteLLM v1.92.0 — per-MCP-server RPM rate limiting, Granian ASGI option, Gemini 3.5 Flash + Opus 4.8 support, Postgres partitioning for SpendLogs

What changed

v1.92.0 (July 12) is the stable successor to v1.91.3 (July 11). Key additions over v1.91.x: per-MCP-server RPM rate limiting for team and API keys; Granian ASGI server as a deployment option; Claude Opus 4.8 and Fable 5 added across Anthropic, Bedrock, Vertex, and Azure providers with adaptive thinking routing for Opus 4.8; Gemini managed agents support; per-token-type detail metrics in Prometheus; budget enforcement for dynamically added deployment models; OAuth2 M2M for Databricks Apps; Postgres partitioning opt-in for SpendLogs retention.

TL;DR

LiteLLM v1.92.0 adds per-MCP-server RPM rate limiting, Claude Opus 4.8/Fable 5 across all providers, adaptive thinking routing, and a Granian ASGI deployment option for improved throughput stability.

Developer signal

Three changes worth acting on: (1) Per-MCP-server RPM limiting: If you run LiteLLM as an MCP gateway proxy with multiple MCP servers behind it, you can now enforce per-server rate limits at the key or team level — useful for preventing a single MCP tool from starving other tools under load. (2) Granian ASGI option: Granian is an async-first Rust-based ASGI server; switching from Uvicorn to Granian can reduce throughput variance under sustained load. To enable: litellm --server granian (experimental flag — test before promoting to production). (3) Adaptive thinking routing for Opus 4.8: LiteLLM's routing layer now automatically identifies requests that benefit from Opus 4.8's extended thinking mode and adjusts request parameters accordingly — this is useful if you use LiteLLM to abstract model selection and want extended thinking to activate on complex requests without per-call instrumentation. (4) Postgres partitioning for SpendLogs: If your LiteLLM SpendLogs table is growing large (common in high-volume proxy deployments), the new opt-in partitioning reduces query latency on retention scans. Enable in config.

Affects you ifYou use LiteLLM as an MCP gateway proxy with multiple MCP servers; you run LiteLLM under sustained high concurrency; you use LiteLLM's routing layer with Claude Opus 4.8 or Fable 5; you have a large LiteLLM SpendLogs table.EffortQuick — drop-in update; Granian and Postgres partitioning require explicit opt-in.

BerriAI/litellm GitHub Releases | Date: July 12, 2026 | Link: https://github.com/BerriAI/litellm/releases/tag/v1.92.0https://github.com/BerriAI/litellm/releases/tag/v1.92.0

Notable

llama.cpp b9968–b9977 — DeepSeek V3.2/V4 Lightning Indexer (CPU), Vulkan Adreno matmul fix, multimodal tool_result image block fix

What changed

10 builds in 18 hours (b9968 through b9977), picking up from b9966 covered in the July 11 digest. Three changes stand out:

| Build | UTC | Key change |

|-------|-----|------------|

| b9977 | 16:18 | server: fix image blocks in tool_result silently dropped during Anthropic→OpenAI format conversion |

| b9974 | 14:16 | cuda: prevent crash when querying memory on device with zero free memory |

| b9973 | 13:40 | DeepSeek V4: clear cache only for current sequence, not full context |

| b9972 | 13:07 | server: stream: true now supported for exec_shell_command |

| b9971 | 12:36 | server: server_stream refactor with improved API documentation |

| b9970 | 12:03 | ggml: add GGML_OP_LIGHTNING_INDEXER for DeepSeek V3.2/V4 with CPU support and test coverage |

| b9969 | 05:30 | Vulkan: route large matmuls to medium tile on Adreno — fixes llama-cli breakage with longer prompts |

| b9968 | 01:12 | opencl: int8 dp4 dense and MoE prefill optimization for Adreno GPUs |

TL;DR

b9977 fixes a silent data loss bug (image blocks dropped in Anthropic→OpenAI multimodal conversion); b9970 adds GGML_OP_LIGHTNING_INDEXER for DeepSeek V3.2/V4 CPU inference; b9969 fixes Vulkan/Adreno prompt-length crashes.

Developer signal

(1) b9977 — multimodal tool_result image fix: If you use the llama.cpp server with models that return images in tool_result content blocks (e.g., models running with Anthropic-format tool calls and you proxy to OpenAI format), the image blocks were previously silently dropped during format conversion. This is now fixed. If you saw truncated or missing visual outputs from tool calls in your llama.cpp server pipeline, upgrade to b9977. (2) b9970 — LIGHTNING_INDEXER for DeepSeek V3.2/V4 (CPU): DeepSeek V3.2 and V4 use a "lightning indexer" architecture for efficient token routing. This GGML operation adds CPU-path support, enabling CPU inference for these model variants in llama.cpp. If you run DeepSeek V3.2 or V4 GGUF models on CPU (or in hybrid CPU+GPU setups), upgrade to b9970 or later. GGUF weights compatible with the new op are required — check the model card. (3) b9969 — Vulkan/Adreno large matmul fix: The previous Vulkan backend routed large matrix multiplications to the large tile path on Adreno GPUs, which exceeded shared memory limits and caused llama-cli crashes with prompts above a threshold length. Now routes to medium tile. If you run llama.cpp on Qualcomm Adreno GPUs via Vulkan and hit crashes on longer prompts, upgrade to b9969. (4) b9968 — OpenCL int8 dp4 MoE optimization: For Adreno GPU inference via OpenCL (not Vulkan), dense and MoE prefill performance improves via int8 dp4 operations. Expect measurable prefill speed gains on Qualcomm devices running OpenCL backend.

Affects you ifYou use the llama.cpp server with multimodal tool calls in Anthropic→OpenAI proxy mode; you run DeepSeek V3.2 or V4 GGUF models on CPU; you run llama.cpp on Qualcomm Adreno via Vulkan or OpenCL.EffortQuick — binary upgrade; b9970 requires GGUF weights compatible with LIGHTNING_INDEXER op.

ggml-org/llama.cpp GitHub Releases | Date: July 12, 2026 | Link: https://github.com/ggml-org/llama.cpp/releaseshttps://github.com/ggml-org/llama.cpp/releases

Benchmarks & Leaderboards

No leaderboard movements in the last 24 hours above the quality gate threshold. LMArena added seedream-5.0-pro to the Image Arena on July 11 — insufficient vote accumulation for score stability as of this writing; see Quick Hits.

Trends & Emerging Tech

Gemini 2.5 unexpected shutdown signals accelerating model lifecycle compression

What's happening

The Gemini 2.5 Flash family going 404 three months before the documented shutdown date is not an isolated accident — it's consistent with a pattern across the ecosystem of accelerated model cycling. Google separately announced July 31 as the end-of-life date for Gemini 2.5 Pro in GitHub Copilot. The gemini-embedding-001 EOL date (July 14) is already documented. The question of whether the 2.5 Flash shutdown is intentional or a configuration error hasn't been addressed publicly, but production breakage has been running for 4+ days without reversal — suggesting it's unlikely to be reverted.

Why watch this

For any team running Gemini 2.5 Flash as a cost-efficient workhorse (it was the recommended default for most AI Studio workloads post-May), this is a forced migration moment. The pattern across Google's 2026 model lifecycle — rapidly superseding 2.x-generation models with 3.x equivalents — suggests teams should plan for 6-month active lifetimes on any non-LTS Gemini model version, not the 12+ months previously implied by deprecation policy. Build model aliasing into your Gemini integration layer now rather than hardcoding version strings.

Google AI Developer Forum | Date: July 9–12, 2026 | Link: https://discuss.ai.google.dev/t/gemini-2-5-flash-deprecated-without-warning-earlier-than-shutdown-date/174217

vLLM MRv2-only signals the inference ecosystem's PagedAttention era is over

What's happening

PagedAttention — the original vLLM innovation from 2023 that demonstrated OS-style memory paging for KV cache — is now fully removed as a code path. This isn't a deprecation with fallback; the implementation is deleted. vLLM's MRv2 (Model Runner V2) backend uses a fundamentally different approach: hardware-aware scheduling rather than virtual memory management, with better support for speculative decoding, heterogeneous vocabularies, and modern attention variants like GQA and MLA. The removal reflects a 3-year maturation arc in inference engineering.

Why watch this

The broader implication for developers: benchmarks, tutorials, and blog posts from 2023-2024 that describe PagedAttention as a core component of vLLM-based inference are now describing removed infrastructure. Teams evaluating or teaching vLLM need to update their mental model. More practically: if your team built custom extensions or profiling tooling that hooked into PagedAttention's memory management layer, those hooks are gone. The migration is not as simple as swapping a flag — it requires understanding that MRv2 handles memory differently and testing that your throughput and latency characteristics are preserved (they often improve, but not always for all batch sizes and sequence lengths).

vllm-project/vllm GitHub Releases | Date: July 11, 2026 | Link: https://github.com/vllm-project/vllm/releases/tag/v0.25.0

Technical Discussions

Terry Tao uses Claude Code to rescue 27-year-old Java 1.0 math code — finds two bugs he had missed

Terence Tao (terrytao.wordpress.com) | Date: July 11, 2026 | Link: https://terrytao.wordpress.com/2026/07/11/old-and-new-apps-via-modern-coding-agents/ | HN: https://news.ycombinator.com/item?id=48880170 (#1 story July 12)

Quick Hits

gemini-embedding-001 shutdown: 2 days remaining — hard deadline is July 14, 2026 (tomorrow); migrate to models/gemini-embedding-2 now. Full migration guide was in the July 10 digest Breaking Changes section.
LMArena: seedream-5.0-pro added to Image Arena — July 11, 2026; insufficient vote accumulation for stable ELO score as of July 12. Watch for score stabilization in the next few days. https://huggingface.co/spaces/lmarena-ai/arena-leaderboard
GitHub Copilot: Gemini 2.5 Pro + Gemini 3 Flash deprecated July 31 — separate from the API 404 issue; update any Copilot model selection settings before July 31. https://github.blog/changelog/2026-07-02-upcoming-deprecation-of-gemini-2-5-pro-and-gemini-3-flash/

Worth Watching (Announced, Not Yet Shipped)

Item	ETA	Notes
gemini-embedding-001 shutdown	July 14, 2026 — tomorrow	Migrate to `gemini-embedding-2` immediately
Gemini 3.5 Pro	Unconfirmed; GA expected July 2026	Google has confirmed it's coming but hasn't published an API endpoint or model card as of July 12 — especially relevant now that Gemini 2.5 Pro is returning 404
Claude Opus 4.7 fast mode removal	July 24, 2026	Requests to `claude-opus-4-7` with `speed: "fast"` will error after this date; migrate to Opus 4.8 fast mode
GPT-Live-1 and GPT-Live-1 mini API	"Coming soon"	No pricing, endpoint shape, or timeline published
Claude Opus 4.1 retirement	August 5, 2026	Migrate to Opus 4.8
Imagen 4 shutdown	August 17, 2026	Migrate to Imagen 4 Ultra or Veo 3

Filtered from 30+ primary sources against a published quality rubric. No press releases, no fluff — only what changes what you build.