Skip to content
ignisignis

ignis v0.8.0

Released 2026-10-24. The container image, the archives and the notes as written: the release on GitHub.

The 27B on a 16 GB card

Section titled “The 27B on a 16 GB card”

This release runs Qwen3.8-27B on the 16 GB card class: RTX 5060 Ti 16 GB, RTX 5070 Ti, RTX 5080 (#334, #336-#340).

  • A second artifact for the 27B, qwen3.8-27b-16g (gpillon/Qwen3.8-27B-trellis-3bpw-ignis, 12.46 GiB). Its linear layers use a trellis code at about 3 bits per weight, a bit-for-bit repack of turboderp’s public EXL3 SC_3.00bpw_H4 quant. The token embedding, the DFlash2 drafter and the rest come from the 32 GB NVFP4 artifact (ADR 0060, #338).
  • Dense trellis kernels of ignis’s own for decode, verify, prefill and the output head (#337). The 27B program binds whichever body the artifact holds, and a load never mixes the two (#339).
  • On a 16 GB card a bare ignis-server needs no flags. --profile auto picks rtx50-16gb (262,144 tokens of context, two decode lanes, DFlash2 with 7 drafts, vision off). The default id qwen3.8-27b resolves to the 16 GB entry, which a start with no model on disk offers to fetch (ADRs 0063, 0064, #336).

How it was measured. On an RTX 5090 (PCIe Gen 3), not on a 16 GB card. The speeds, the capacity, DFlash2 and the loops come from the load a 16 GB card makes, --profile rtx50-16gb --vram-budget-bytes 17179869184; KLD and MMLU-Pro come from the same build’s engine logits, with the same KV format and prefill chunk.

  • Quality, four gates registered before the run, all passed (finding), each against the 32 GB NVFP4 artifact or BF16:
    • KLD from BF16 is 0.0403, 0.77x the NVFP4’s at the same prefill chunk and 0.82x against the NVFP4 at its served chunk (gate: at most 1.2x);
    • MMLU-Pro is 65.84% against BF16’s 67.97% on 281 questions, exact McNemar p 0.31 (gate: p > 0.05);
    • DFlash2 commits 0.996x the NVFP4’s tokens a round over 200 recorded agent turns, teacher-forced (gate: at least 0.90x);
    • reasoning loops hit 9 of 32 turns against the NVFP4’s 14 of 32 (gate: not significantly more at one-sided Fisher p < 0.05; p 0.94).
  • Capacity (finding):
    • a 16 GiB and a 15 GiB budget keep 262,144 tokens on two lanes with nothing adapted;
    • a 13 GiB budget adapts the context to 65,536;
    • on a real card the budget is its free memory less the profile’s 1,200 MiB headroom.
  • Speed on the 5090 held to 16 GiB (finding): one lane decodes 416 tok/s on coding prompts at 8K of context and 216 at 253K; two lanes reach 684 together at 8K. Agent turns accept fewer drafts a round, which would scale these by about 0.58.
  • Speed on the 16 GB cards, an estimate, not a measurement: one lane at 8K, RTX 5080 243-307 tok/s, RTX 5070 Ti 204-270, RTX 5060 Ti 16 GB 103-153.
  • Prefill is the gap: the trellis body prefills 2.5x slower than the NVFP4 at 130K on the 5090. That work is #345.

If you have a 16 GB card, ignis-bench small-card --endpoint http://127.0.0.1:8000 --out small-card.json writes one record of its load and speed. A GitHub issue with that record is how rtx50-16gb gets measured numbers (guide).

Changes

Section titled “Changes”

  • --profile auto is the default (ADR 0063, #336). The start reads the card’s memory from NVML and takes the built-in profile of its class: rtx50-16gb under 20 GiB, rtx5090 from 28 to 40 GiB, rtxpro6000 from 88 to 104 GiB. The ignis.config.source log line names the profile and the total that chose it.

  • The 27B’s load adapts before it refuses (ADR 0063, #336). When the VRAM plan cannot hold a profile’s or a default’s values in the memory the start finds free, the load lowers them one at a time: the drafter’s head, then the context in steps of model.adapt_context_step (65,536) until the pool keeps vram.adapt_spare_tokens (65,536) spare, then the lanes. A value you name yourself never adapts. The ignis.runtime.vram_plan event lists every step taken.

  • Decode lanes are set per load for both families (spec.decode_lanes, 1 to 8 for the 27B), and the kernel buffers are sized by them (#336).

  • A catalog entry states its card class (ADR 0064, #336). card_class is the smallest class the entry loads on. aliases resolve within a family, so qwen3.8-27b is served by qwen3.8-27b-16g on a 16 GB card. A switch to an entry above the card’s class is refused with 409 model_above_card_class before anything is unloaded, and model download refuses one before it fetches anything.

  • GET /v1/models says what the load runs at (#340): data[0] of a text model carries decode_lanes and adapted, which lists every adaptation step and is "" when the load took none.

  • ignis-bench small-card (#340): one command against a running server writes one pasteable JSON record. It holds the card, the VRAM plan, what the load adapted, decode tok/s on coding prompts and agent turns, and TTFT at 8K and 64K.

  • The user guide is a site, gpillon.github.io/ignis (ADR 0066, #347, #348):

    • a Why ignis section and the Playground view by view, with screenshots taken from a real server;
    • every config field, profile, catalog entry, metric and route, generated from the code;
    • a Changelog section written from these release notes.

    It is built from the release’s tag and published once the release is out.

Upgrade notes

Section titled “Upgrade notes”
  • The default profile was rtx5090 and is now auto.
    • On a 32 GB RTX 5090 nothing changes.
    • On a 96 GB RTX PRO 6000 a start with no --profile now takes rtxpro6000.
    • A card of any other size is refused under auto with a message naming --profile.
    • --profile rtx5090 keeps the old behaviour.
  • A 27B start that used to be refused for VRAM may now load at a smaller context. This happens when the desktop holds part of the card and the context came from the profile. auto rope scaling follows the context the load runs, so the YaRN factor, and with it the outputs, change with it. Name --model-max-context to pin the context: a start that does not fit it is then refused, as before.
  • On a 16 GB card, a request naming qwen3.8-27b is answered by qwen3.8-27b-16g, and the response’s model field says so.

Decisions

Section titled “Decisions”
  • ADR 0066: the user guide is a Starlight site built from docs/user/ alone, published when a release is published
  • Amended: ADR 0030 (VRAM reserved at load), 0046 (one declaration per config field), 0047 (the model catalog), 0063 (the profile follows the card class), 0064 (a catalog entry’s card class)

Work in progress (issues still open)

Section titled “Work in progress (issues still open)”
  • #345: the dense trellis prefill at the tensor peak, the 16 GB artifact’s prefill gap
  • #334: the 27B on 16 GB cards, phase 1; its last ticket is this release and the artifact’s Hugging Face card (#346)
  • #342: the 27B’s prefill on the 5090, continued
  • #333: Flash-Next prefix reuse does not engage at ~500K and 1M prompts

Container image (linux/amd64)

Section titled “Container image (linux/amd64)”
podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.8.0

(docker: --gpus all in place of --device.) The image carries the CUDA runtime but no driver — the host’s NVIDIA driver is injected by the container runtime. It is built from the same compile as the Linux tarball below, so the two always carry the same binaries.

Binaries

Section titled “Binaries”

The kernel is compiled for SM120a (RTX 5090 / Blackwell consumer) only, and the engine needs a .ninfer artifact, which is not part of this release. The Linux tarball needs the CUDA 13 runtime on the host; the Windows zip carries cudart64_*.dll. Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep their own terms (NOTICE-kernel).

Full Changelog: https://github.com/gpillon/ignis/compare/v0.7.3...v0.8.0