Skip to content
ignisignis

ignis v0.7.3

Released 2026-10-22. The container image, the archives and the notes as written: the release on GitHub.

Changes

Section titled “Changes”
  • The 27B’s prefill attention reads the KV codec, with no prompt planes (ADR 0061, #335). Each hq-e8-2b prefill chunk now attends to the prompt straight from the codec, so the per-load prompt-plane buffers are gone, on every card. The default band is 8-bit (INT8 Q·Kᵀ, E4M3 P·V). --model-prefill-attention fp8|fp16 (config model.prefill_attention, per model) picks it; fp16 is the FP16-accumulate fallback. Flash-Next ignores the option.
  • The 27B’s token embedding lives in pinned host memory (ADR 0062, #335). That takes 1,350,860,800 bytes off the VRAM weights line, and the KV pool gets them. On the 5090, decode is -0.1% at one lane and about -0.4% at eight, cold prefill -0.15% to -0.7% from 14K to 105K, greedy text identical. The 27B now plans its host memory the way Flash-Next does, against model.host_margin_bytes. A start that does not fit is refused, and the message names the 27B’s own flags.
  • rtx5090 profile: the 27B’s context goes from 786,432 to 917,504 tokens, with vision, DFlash2 and 1,200 MiB headroom (#335). auto rope scaling follows, so its YaRN factor goes from 3 to 3.5.
  • The 27B prefills 2,048 tokens a chunk under the rtx5090 and rtxpro6000 profiles, up from 1,024 (#341). On the 5090: cold prefill +1.9% / +2.2% at 46K / 105K, unchanged at 8K / 14K. 4K KLD 0.04916 against 0.05221. A decoding lane’s longest pause under a 32.7K prefill goes from 184 to 307 ms. Flash-Next and a start without a profile keep their widths. A start whose chunk takes VRAM from the KV pool says what 1,024 would free.
  • The 27B’s BF16 exception tensors can be held in FP8, opt-in (ADR 0065, proposed; #341). --model-exception-weights fp8 (config model.exception_weights) converts the nine BF16 matrices to FP8 row-scale at load. On the 5090: cold prefill +4.1% / +3.5% / +2.8% at 14K / 46K / 105K, the load just as long, 534 MB less VRAM, 4K KLD unchanged within its interval. Two pre-registered decision fixtures read differently, so bf16 stays the default.
  • GET /v1/models lists every model a request can be served by (#343). The loaded model comes first (active: true, data[0]). Then come the catalog entries whose artifact is downloaded, then the known models whose artifact file exists. With switch.allow_implicit: false only the loaded model is listed. A request naming a known model whose artifact is gone is now a 404 model_not_found, and no switch starts. It used to tear the loaded model down for a load that then failed.
  • Responses: a response id is resp_ plus 32 random hex digits. It used to be the request id. A Playground socket that outlived a model load (an image switch, the idle unload) could continue one session on another session’s history through previous_response_id. Item and call ids share the new suffix, and a queued response gets its id when it is created (no more resp_q<n>).
  • Request ids carry on from load to load. A switch, a reload and the idle unload’s next load all draw from one count, so a request_id in the log names one request for the life of the process.
  • Playground:
    • The chat’s sessions and the Decide tab’s decisions are kept in the browser’s store (IndexedDB), and they come back after a reload (#171).
    • A made picture opens full screen on a click, and its prompt shows whole on request. Its Mark and Image view buttons sit over the picture.
  • Build and tests: the Linux build’s warnings are gone, and two loop-guard metrics tests no longer race the mock engine on a slow runner.

Upgrade notes

Section titled “Upgrade notes”
  • The 27B’s outputs change under the rtx5090 profile. That covers short prompts too: YaRN is 3.5 because the context is 917,504, the prefill attention has no prompt planes, and prefill chunks are 2,048 tokens. Under rtxpro6000, the chunk and the prefill attention change.
  • The 27B pins about 1.35 GB more host memory for its token embedding. A host short of RAM now refuses the start with the host plan’s message.
  • GET /v1/models can list more than one model. A client that reads data[0] still gets the loaded model. One that sends whatever id it finds in the list now triggers an implicit switch, unless switch.allow_implicit is false.
  • Response ids and item ids have a new format. resp_q<n> ids are gone.

Decisions

Section titled “Decisions”
  • ADR 0060: the 27B gets a second body format, trellis, for the 16 GB card class
  • ADR 0061: prefill attention reads the hq-e8-2b codec, with no prompt planes, on every card
  • ADR 0062: the 27B’s token embedding lives in pinned host memory, on every card
  • ADR 0063: the profile follows the card class, and its defaults adapt to the budget
  • ADR 0064: a catalog entry states the smallest card class it loads on; aliases resolve within a family; a switch refuses before teardown
  • ADR 0065 (proposed): the 27B’s BF16 exception tensors can be held in FP8, converted at load

Work in progress (issues still open)

Section titled “Work in progress (issues still open)”
  • #334: the 27B on 16 GB cards, phase 1 (#336-#340). ADRs 0060, 0063 and 0064 are accepted, but the 16 GB artifact and its kernels are not in this release.
  • #342: the 27B’s prefill on the 5090, continued (backbone GEMMs and GDN, a GPTQ NVFP4 encoding)
  • #171: the Playground’s chat history; the owner’s live check is still open
  • #333: Flash-Next prefix reuse does not engage at ~500K and 1M prompts

Container image (linux/amd64)

Section titled “Container image (linux/amd64)”
podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.7.3

(docker: --gpus all in place of --device.) The image carries the CUDA runtime but no driver — the host’s NVIDIA driver is injected by the container runtime. It is built from the same compile as the Linux tarball below, so the two always carry the same binaries.

Binaries

Section titled “Binaries”

The kernel is compiled for SM120a (RTX 5090 / Blackwell consumer) only, and the engine needs a .ninfer artifact, which is not part of this release. The Linux tarball needs the CUDA 13 runtime on the host; the Windows zip carries cudart64_*.dll. Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep their own terms (NOTICE-kernel).

Full Changelog: https://github.com/gpillon/ignis/compare/v0.7.2...v0.7.3