
Release b12549 Isolates Inkling 2 Attention and Fixes Hybrid Graph Reuse
The maintainers of ggml-org/llama.cpp published pre-release b12549 (commit 8af68d4) on November 19, 2026, shipping pull request #32429 (llama: fixes for Inkling 2 attention mask) alongside cross-platform binary packages and build attestations. Rather than introducing new user-facing commands or model families, the update resolves structural mismatches in how llama.cpp builds attention masks and manages graph reuse for hybrid interleaved sliding-window attention (iswa) and recurrent memory architectures.
Specifically, the five-commit series refactors how the Inkling 2 model graph handles relative-position attention masks and per-query attention sinks, while repairing a graph-reuse defect affecting hybrid iswa models—explicitly naming LFM2, LFM3, and Inkling2—when running under a unified KV cache with flash attention enabled. It also relaxes load-time parameter checks in src/models/inkling2.cpp so zero-valued expert weight scales and final logit softcaps are skipped instead of rejected.
Engineering Architecture: Relative-Position Masks and Hybrid Memory Composition
Previously, the Inkling 2 graph constructed its flash attention mask from the hybrid iswa input and forwarded it to build_attn through a dedicated kq_mask argument. That design allowed a caller to pair a dense attention mask with the kv_rows of llm_graph_input_attn_kv_iswa, whose mask tracks per-sequence cell lists whenever kv_rows is active. Release b12549 replaces that coupling with llm_graph_input_attn_rel_pos, a dedicated input structure that owns both dense KQ masks and query-to-cell distances, populating both from the same cells. Its build_attn overload applies per-head bias directly to its own mask and never passes kv_rows, allowing the standard iswa build_attn signature to return to its earlier form backed by a shared static helper, build_attn_iswa_impl.
To remove a model-specific branch from build_attn_mha, Inkling 2 now folds its per-query sink directly into the attention bias regardless of whether flash attention is active. Meanwhile, the release corrects a mask-generation mismatch in llm_graph_input_mem_hybrid_iswa. Its can_reuse_attn_mask check expected kv_rows masks whenever enabled, yet the class built dense masks instead; under a unified cache and flash attention, that check consistently failed and prevented LFM2, LFM3, and Inkling2 from reusing compute graphs. Building those masks via build_attn_inp_attn_mask aligns hybrid iswa behavior with the plain iswa input. For Inkling 2, review iterations moved llm_graph_input_mem_hybrid_iswa_rel from inheritance to composition so it owns both its attention and recurrent inputs as a single composite memory input.
The b12549 artifact matrix spans desktop, mobile, and server accelerators: macOS Apple Silicon (arm64), macOS Intel (x64), and iOS XCFramework; Ubuntu x64 and arm64 builds for CPU and Vulkan; Ubuntu 24.04 and 26.04 s390x CPU plus 26.04 zDNN; Ubuntu CUDA 12 (12.8 libraries), CUDA 13 (13.5 libraries), ROCm 10.0, OpenVINO, and SYCL (FP32 and FP16); Linux and Android arm64 Snapdragon targets (CPU, Adreno GPU, Hexagon NPU); and Windows x64 and arm64 builds across CPU, OpenCL Adreno, CUDA 12.4, CUDA 13.5, Vulkan, OpenVINO, SYCL, and ROCm 10.0.
Workflow Impact: Cleaner GGUF Loading and Restored Graph Reuse
For developers deploying hybrid models on llama.cpp, the most immediate operational improvement is optional metadata handling in src/models/inkling2.cpp. By treating the expert weight scale and final logit softcap keys as optional and skipping them when zero, Inkling 2 now matches sibling model loaders rather than failing initialization on GGUF exports where those fields are zeroed. Restoring can_reuse_attn_mask compatibility for LFM2, LFM3, and Inkling2 under unified KV caches and flash attention also removes a structural blocker to graph reuse in iterative inference loops, though the release notes provide no timing or memory measurements quantifying runtime gains.
Maintenance clarity also improves through formatting updates in src/models/inkling2.cpp, which reorganize key and tensor loading into aligned groups, add descriptive per-layer names and section comments, and assign cb() names to intermediate tensors without changing the underlying compute graph. Notably, all five commits in #32429 include an Assisted-By: Claude Opus 5.5 attribution alongside code review revisions that steered llm_graph_input_mem_hybrid_iswa_rel toward explicit composition.
Limitations, Build Exclusions, and Deployment Caveats
Several technical and packaging caveats remain in b12549. First, the tag is distributed as a GitHub pre-release (signed with GPG key B5690EEEBB952194 and accompanied by build attestations), and the release page includes no benchmark results, accuracy checks, or independent performance verification. Second, llm_graph_input_mem_hybrid_iswa_rel still constructs dense masks only: its build_attn explicitly asserts that no kv_rows are present, and the commit log records an open TODO: a kv_rows variant, leaving sparse kv_rows mask support unimplemented for Inkling 2.
Third, not every target in the release checklist shipped in this build. Five configurations are explicitly marked DISABLED: macOS Apple Silicon with KleidiAI enabled, plus all four openEuler targets (x86 and aarch64 builds for 310p and 910b ACL Graph). Finally, the llama.cpp release entry documents runtime engineering changes only and specifies no model-weight licensing or deployment terms for Inkling 2, LFM2, or LFM3.
EXPLORE THE SOURCE
Projects in this story
GitHub repositories
- ggml-org/llama.cpp ↗View project details
Automated Vertex AI (gemini-4-hydrogen) primary-source report; reviewed 2026-11-19.



