Skip to content
ignisignis

ignis v0.7.1

Released 2026-10-18. The container image, the archives and the notes as written: the release on GitHub.

Changes

Section titled “Changes”
  • Flash-Next on a 96 GB card keeps its experts only in VRAM. A resident load (the VRAM plan’s kv_pool_policy: resident) reads every expert from the artifact straight into its slot and keeps no pinned copy in RAM: on an RTX PRO 6000 the pinned memory went from 41.1 to 3.3 GB and the host plan from 69.9 to 36.8 GB, with the same decode, TTFT and greedy text. A device-only load reserves no prefill staging ring (the KV pool takes its ~1.5 GB) and charges its 4.7 GB of layer buffers only while the experts load (#329 items 2, 3; ADR 0054)
  • The rtxpro6000 profile, measured on an RTX PRO 6000 Blackwell Server Edition: 1,048,576 tokens of context for both models, 8 Flash-Next lanes, MTP with 3 draft tokens (1.85x speculation off on code and 1.44x on prose at one lane), DFlash2 7 on the 27B (1.6-3.3x), and ngram.hot_bytes: auto, the whole n-gram table in RAM where the host plan leaves room (1.24x decode, 2.3x prefill on a cloud volume). Inside a Linux container on cgroup v2 the host plan reads its available RAM capped by the container’s memory.max, so a load that does not fit is refused, not OOM-killed (#329 items 7, 8). Re-verified on the card with 0a810c0b, this release’s base before its final merges (the pinned hot set and the row budget did not run there): rates within ~2.1%, a 299K-token prompt answered
  • A pinned hot set of Flash-Next experts: vram.pinned_expert_bytes (--vram-pinned-expert-bytes) keeps the calibration-hottest experts in VRAM for the whole load, outside the cache’s LRU, with no copy in RAM. Off by default; 2 GiB under the rtx5090 profile: 2.1 GB less RAM for at most ~1% of three-lane decode. A pin that cannot leave every K class the slots of the widest decode step is refused at start, naming the most it can pin (#329 item 12; ADR 0055)
  • spec.draft_rows up to 32 (--spec-draft-rows, default 0 = 8): a wider verify round lets five to eight lanes draft with MTP. Past 8 rows the round is held numerically equivalent to speculation off, not token for token: greedy output may differ where two tokens are nearly tied. Measured slower than 8 on both the RTX 5090 (0.74-0.80x) and the RTX PRO 6000 (0.42-0.84x: a round past 8 rows takes 64-80 ms against ~19.5 ms), so no profile sets it (#329 item 5; ADR 0056)
  • Probes and stop, for containers and Knative. /health is liveness and /ready readiness, and the listener answers from the moment the server starts: /health 200 and /ready 503 through a cold load. SIGTERM during a download or a load exits at once and frees the GPU; a stop while serving drains in-flight requests and does not wait for a load (a switch, a reload, the load after an idle unload), answering its waiters 503 server_stopping; a second signal ends the process (#327; #329 items 17, 18)
  • An idle Responses WebSocket is closed after server.socket_idle_close seconds, 300 by default, with a close frame that says why; the Playground closes its own sockets a minute after its tab is hidden, and the Monitor scrapes nothing from a hidden tab, so an open tab no longer keeps a scale-to-zero service awake (#329 items 10, 11)
  • Admission tells a client when to come back: a full engine’s 503 engine_full carries Retry-After from when a lane is expected to free (an overdue estimate waits at most 15 s), and every 503 while a model loads or switches says Retry-After: 5. The non-streaming timeout is 120 s (was 30). A non-streaming answer nobody waits for (a 504 or a client gone) is cancelled instead of holding its lane, and ignis.request.cancelled says why, how long and what it generated (#327)
  • The loop guard on /metrics and in the Monitor: loops by outcome, watched and looped requests, rewound tokens, the margin to the threshold (ignis_loop_guard_*) (#330)
  • The CUDA build links no NVIDIA driver library: ignis-server model download runs on a machine with no GPU and no driver (a Kubernetes CPU node, the published image included). A GPU load without a driver is refused by name, and a driver older than the CUDA runtime is named as such (#329 item 9)
  • ignis-bench replay: --as-recorded sends a trace’s recorded chat body (its messages and tools), --retry-engine-full waits out engine_full (#329 item 14)

Fixes

Section titled “Fixes”
  • A Flash-Next MTP round cut near the end of a request’s reservation no longer kills the request (and the other lanes of its round) a few tokens before its cap: such a round runs as a plain step (#329 items 4, 6)
  • PATCH /v1/config with spec.backend: off drops a live draft count it does not name, in one step, instead of refusing it (#329 item 15)

Upgrade notes

Section titled “Upgrade notes”
  • The rtx5090 profile now pins 2 GiB of Flash-Next’s experts: Flash-Next needs ~2.1 GB less RAM, every default load pays ~2 s to read the pin, and the Monitor’s expert slots (state="capacity") are the LRU’s, lower by the pinned ones. --vram-pinned-expert-bytes 0 is the previous behaviour.
  • An MTP load splits its expert cache differently: every K class first gets the slots of the widest decode step, under MTP a verify round’s lanes x (drafts + 1) rows rather than the lanes, at any pin (a correctness fix). A small cache with MTP on may give the cold classes more of it.
  • The non-streaming request timeout is 120 s (server.request_timeout, was 30 s).
  • Probes (Kubernetes, Knative): point liveness at /health and readiness at /ready. A liveness delay sized to outlast a cold load is no longer needed, since /health answers during the load. An idle Responses WebSocket now closes after 300 s; a client opens a new socket for its next request.
  • A resident Flash-Next load reads its experts through the page cache: with ngram.hot_bytes: auto holding the whole n-gram table on a 64 GiB host, a reload read them in ~31 s, slower than the previous build’s warm 12.5-12.8 s (measured on another node type), and in 5 s only when the cache held them all.
  • spec.draft_rows past 8 changes greedy output at near ties (ADR 0056); no profile sets it.

Work in progress (issues still open)

Section titled “Work in progress (issues still open)”
  • #327: the anvil-serving qualification’s remaining acceptance criteria (5-7)
  • #329 is closed with this release; its item 1 (a device-resident n-gram table) is deferred by the owner
  • #331: a native wide MoE decode route for verify rounds past 8 rows, spec flash-next/08; Phase 0 (a profile with a pre-registered go / no-go) comes first

Decisions

Section titled “Decisions”
  • ADR 0054 — a resident Flash-Next load keeps its experts only in VRAM
  • ADR 0055 — a pinned hot set of experts: a field off by default, 2 GiB in the rtx5090 profile
  • ADR 0056 — speculation past 8 verify rows is numerically equivalent, not bit-exact (amended after acceptance: the 16-ulp bound for a moved pick and the 0.1 acceptance band were set by the review from the measured data, not by the owner)
  • Amended: ADR 0017, 0030, 0032, 0045, 0052

Container image (linux/amd64)

Section titled “Container image (linux/amd64)”
podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.7.1

(docker: --gpus all in place of --device.) The image carries the CUDA runtime but no driver — the host’s NVIDIA driver is injected by the container runtime. It is built from the same compile as the Linux tarball below, so the two always carry the same binaries.

Binaries

Section titled “Binaries”

The kernel is compiled for SM120a (RTX 5090 / Blackwell consumer) only, and the engine needs a .ninfer artifact, which is not part of this release. The Linux tarball needs the CUDA 13 runtime on the host; the Windows zip carries cudart64_*.dll. Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep their own terms (NOTICE-kernel).

Full Changelog: https://github.com/gpillon/ignis/compare/v0.7.0...v0.7.1