llama.cpp v0.8.0 and b11972: CLI Now Requires --model or -hf

Key changes #

Start with the CLI flag requirement, since it changes what existing invocations must contain. The rows cover both release notes.

v0.8.0: LFM3 Models and Realtime Audio on the Server #

The v0.8.0 part covers a CLI requirement, several MoE and ggml behaviour changes, and the LFM3 family with realtime audio on the server. Start with --model or -hf in every CLI call, because it affects existing invocations directly.

Required --model or -hf in the CLI #

The CLI now requires one of two flags, --model or -hf. Any invocation that supplies neither is affected. The change is listed in the v0.8.0 release notes as #31242, titled "common : --model or -hf is required". Search shell scripts, Makefiles and CI definitions for CLI calls, and confirm that each one passes a model source before it runs against the new build.

MoE amax rescaling, -sm tensor and laguna #

The release enables MoE amax rescaling for laguna and denies -sm tensor on architectures that use amax rescaling (#31147, #31081), both listed in the v0.8.0 release notes. This suggests that a laguna deployment passing -sm tensor is affected, so its launch configuration should be reviewed first. The same notes list a 50/50 SLRU cache for host-resident MoE experts (#30637), which replaces a plain LRU cache.

The -1 expert index in ggml v0.28.0 #

The ggml bump to v0.28.0 is listed in the v0.8.0 notes with this change:

-1 is now a valid expert index in MUL_MAT_ID / ADD_ID : the slot is skipped and the output row zeroed

A graph that reads the output row for a -1 slot will now see zeros rather than an invalid index. This suggests that custom ggml graphs and backends must be reviewed for how they use that row. The same bump adds GGML_OP_TOP_K_MASK, which the notes describe as a fused sparse-attention mask.

DRY sampler sequence-breaker cap #

The new LLAMA_DRY_MAX_SEQ_BREAKERS constant is 64 and caps distinct sequence breakers, because each one costs O(n_vocab) at init (#31035). Count the distinct sequence breakers in your DRY configuration against that limit. The related fix, titled "llama: fix exhausting CPU and MEM caused by dry sequence breaker", is listed in the v0.8.0 release notes.

Grammar recursion and JSON nesting limits #

Grammar recursion depth is now limited and enforced to prevent stack overflow (#29755). The chat API rejects JSON nested more than 128 levels before walking it (#30242). The notes do not give a numeric grammar recursion limit, so send your deepest grammar through the server after the upgrade and check whether it is accepted.

Proxy variables for model downloads #

The common HTTP client honors HTTP_PROXY and HTTPS_PROXY from the environment (#21752). Downloads can now run concurrently across processes (#29312), and model resolution is pinned on real HF repo layouts (#30548). Jobs that download models through a proxy should be checked for these variables. To list the proxy variables visible in a shell, run:

env | grep -i proxy

lightonocr defaults to native resolution #

lightonocr now uses native resolution by default instead of forcing a 1540 longest edge (#30600). This suggests that OCR output may differ from runs at the 1540 longest edge. Compare both settings on a sample set of your documents before you accept the new default.

Realtime and TTS endpoints on the server #

The server adds /v1/realtime, which implements the OpenAI realtime protocol for conversation and transcription (#31269, #30636). It sends response.done after the committed audio drains (#30994), and it accepts tool calls (#31287). Clients built for that protocol should wait for response.done before they read the reply. A separate POST /tts endpoint serves text-to-speech models (#26603), so speech clients should call it with a TTS model loaded.

Truncated GGUF tensor data in split, export-lora and merge #

The GGUF tools reject truncated tensor data in split, export-lora and merge (#30497, #31044), both listed in the v0.8.0 notes. Before you run these tools on a file copied or downloaded in pieces, verify that the file is complete, since a truncated input is now rejected.

LFM3 family and streaming audio in mtmd #

The LFM3 family ships four checkpoints: LFM3-1.7B for text, LFM3-VL-2B for vision, and LFM3-Audio-2B and LFM3-Omni-2B (#31252, #31257). LFM3-1.7B and LFM3-VL-2B are verified against the reference HF implementation. LFM3-Audio-2B and LFM3-Omni-2B provide ASR, TTS and voice chat from one model, with streaming audio input and output. Video input for the family follows in #31283.

For code that calls the multimodal API, the experimental mtmd_audio_stream API turns PCM fed as it arrives into decoder-ready embedding rows (#30629). Audio generation is now stateful: mtmd_gen_state replaces the raw state_data buffers of the audio gen API (#30902), so code that reads those buffers must move to the state object. The mtmd_caps structure gains gen_audio, the new TTS types are SOPRANO and LFM3A, and a repetition penalty applies to sampled codes (#28607, #31257).

Also in this release #

• New models: DeepSeek V4.1 Flash, SenseNova U1, Beam (Reflection), Kolibri-1 (Aleph-Alpha), Flint, Muse Spark Open, Granite Switch Decision with its 256-way decision head, Nemotron-3(.5)-Embed, H2O-Lightning-4B, MiniCPM-OCR 1.0 and MiniCPM-OCR-Jev, and LiquidAI d1-2B and d1-omni-2B.
• Multimodal: Soprano TTS, LFM2 tiling and layout matched to transformers, Qwen-VL resize, and mtmd_helper_get_capabilities.
• Core and memory: 50/50 SLRU cache for host-resident MoE experts, fused QSA mask op, no_alloc respected in all memory caches, obsolete n_pad removed.
• Tokenizer and conversion: clean_spaces and metaspace prepend-scheme GGUF keys, Tekken pre-tokenizer for Pixtral, Qwen tensor merge on convert, Granite speech tensor fixes.
• Speculative decoding and LoRA: persisted dynamic ngram cache, probabilistic DFlash2 drafts, optional DFlash FC input scale, prism.hadamard folded LoRA weights.
• Server API: typed rerank input, reasoning effort levels exposed, deferred tools and tool_reference parts, logprobs with speculative decoding, Anthropic /v1/messages sampling parameters and id_slot.
• Server operations: --connect for p2p access outside the LAN, tensor parallel on 4 to 10 GPUs for stepfun and laguna, router device memory margin, SWA checkpoint invalidation fix.
• Fixes: usage.completion_tokens with n > 1, missing output_index in Responses API SSE events.
• UI: model discovery and download from HF Hub, conversation search and compaction, models manager route, OGG attachments converted to WAV, "Show tool call in progress" setting.
• ggml v0.28.0: GGML_OP_TOP_K_MASK, SCALE on non-contiguous views, SUM on non-contiguous inputs, CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, restructured Hexagon HTP backend, KleidiAI SVE kernels.

b11972: Bounded Control Vector Layers and M-RoPE Position Loops #

The b11972 build changes two write paths, one in control vector loading and one in multimodal M-RoPE position filling. Its release notes cover only these two changes. Check first that control vector layer indices above the model layer count are rejected.

Control vector layer indices #

The b11972 notes describe the layer index as parsed from the tensor name with no upper bound, while the buffer size and write offset were computed in int. They describe a large index as overflowing both, so the accumulation writes before the start of the buffer. The fix passes the model layer count to the loader and computes the sizes in size_t:

reject indices above it like the existing zero index check

Anyone who loads control vectors should confirm that every direction tensor's layer index is below the model's layer count.

M-RoPE positions filled by whole merge tiles #

The position loops of the Qwen-VL, GLM, PaddleOCR and Qwen2.5-VL encoders hardcoded a 2x2 merge inside a stride of n_merge. According to the b11972 notes, a patch grid that is not a multiple of the merge wrote past the positions buffer. The loops now iterate over whole merge tiles with n_merge inner steps, which bounds the writes by the patch count and gives the same positions on aligned grids. PaddleOCR now requires a merge-aligned image at graph build, as the Qwen-VL family already does, so PaddleOCR users should supply images whose patch grid is a multiple of the merge.

Upgrade checklist #

1. Before upgrading, search scripts, Makefiles and CI for CLI calls and confirm that each one passes --model or -hf.
2. Before upgrading, find -sm tensor in launch configurations for laguna and for any other architecture with MoE amax rescaling.
3. Before upgrading, count distinct DRY sequence breakers against LLAMA_DRY_MAX_SEQ_BREAKERS (64).
4. Before upgrading, run env | grep -i proxy in the environment of model download jobs and check HTTP_PROXY and HTTPS_PROXY.
5. Before upgrading, compare lightonocr output at native resolution with output at a 1540 longest edge.
6. Upgrade to v0.8.0 or b11972 as your deployment requires. The two b11972 changes appear only in the b11972 notes, so review them separately if you run that build.
7. After upgrading, send grammars and JSON nested beyond 128 levels through the chat API and confirm how they are handled.
8. After upgrading, test /v1/realtime clients for response.done and POST /tts clients with a TTS model loaded.
9. After upgrading, for custom ggml backends, confirm that -1 expert slots produce zeroed output rows where that is intended.
10. After upgrading, confirm that PaddleOCR inputs are merge-aligned and that control vector layer indices are below the model layer count.
11. After upgrading, verify that GGUF inputs to split, export-lora and merge are complete.