ignis v0.6.0

Released 2026-10-10. The container image, the archives and the notes as written: the release on GitHub.

Changes

Qwen3.8-Flash-Next served by ignis: its own .ninfer artifact and converter, trellis-coded MoE experts with a bit width per expert on our own kernels, expert residency between VRAM and pinned RAM, the forward with QSA and the n-gram embedding, prompt reuse, and MTP speculative decoding (off unless --spec-backend mtp) (#298-#304, #307; ADR 0043, 0044)
Runtime model switch between the 27B and Flash-Next without a restart: POST /v1/models/switch, or a request whose model names another known model (#305)
Config v2: every option declared once, with a flag --<group>-<field>, an env var IGNIS_<GROUP>_<FIELD> and a key in a JSON/YAML config file, plus per-family values (--qwen38-…, --qwen38flashnext-…). Also help --fields, config generate|print|patch, GET/PATCH /v1/config, hardware profiles (rtx5090, measured, the default) and config-file auto-discovery (#311; ADR 0046)
Model catalog: ignis-server model download [<id>…] | --all and model list. The built-in catalog covers the 27B, the 27B abliterated and Flash-Next with its MTP companion, every file pinned at a fixed revision. An operator catalog file serves private or air-gapped setups, and --download-endpoint / --download-token point it at a mirror. Every file, sidecars included, is verified before use (#313; ADR 0047)
KV pool sized by whether the model is resident, page-wise reservations, a default max_tokens, and a disk tier for evicted sequence state (#309, #310; ADR 0045)
tool_choice: "required" and a named function force the call on chat completions and the Responses API (#286)
--vision-max-tokens shrinks an over-large image instead of refusing it (#248). --max-context is validated against the GQA attention envelope (#228)
/v1 answers 503 server_not_ready until the warm-up traversal has run (#129)
Flash-Next decode fusion (+6.5% decode) and prefill work: sixteen n-gram readers, decoding lanes kept at ~31 tok/s during a prefill (#306)

Upgrade notes

Every flag and env var was renamed into its group, for example --kv-host-pool-bytes → --reuse-kv-host-pool-bytes, IGNIS_ARTIFACT → IGNIS_MODEL_ARTIFACT, --model → --model-id, --no-model-download → --download-enabled false. The old names are refused. ignis-server help --fields lists the new ones.
The rtx5090 profile is applied by default. A start with no flags now runs the 27B with 768K context, vision and dflash2 (draft 7), and Flash-Next with 589,824 tokens of context; both use yarn:4. --profile none gives the plain hardcoded defaults.
make UNCENSORED=1 serves qwen3.8-27b-abliterated, no longer qwen3.8-27b.
A start-time download now fetches the whole catalog entry. A sidecar served with a digest other than the pinned one is refused.
Decode share defaults to 25 on both models. The hardcoded VRAM headroom is 1.5 GiB; the rtx5090 profile lowers the 27B’s to 1200 MiB.

Work in progress (issues still open)

Flash-Next performance and polish follow-ups (#306)
KV pool / disk tier follow-ups (#309, #310)

Decisions

ADR 0043 — a second model, Qwen3.8-Flash-Next, served without a ninfer reference
ADR 0044 — Flash-Next’s experts are trellis-coded with a bit width per expert, on our own kernels
ADR 0045 — the KV pool is sized by whether the model is resident, and evicted state goes down to disk
ADR 0046 — one declaration per config field: flag, env, file key, default, validator
ADR 0047 — a model catalog the operator can extend, and a model command to fetch from it
Amended: ADR 0017, 0018, 0023, 0024, 0029, 0030, 0032, 0033

Container image (linux/amd64)

podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.6.0

(docker: --gpus all in place of --device.) The image carries the CUDA runtime but no driver — the host’s NVIDIA driver is injected by the container runtime. It is built from the same compile as the Linux tarball below, so the two always carry the same binaries.

Binaries

The kernel is compiled for SM120a (RTX 5090 / Blackwell consumer) only, and the engine needs a .ninfer artifact, which is not part of this release. The Linux tarball needs the CUDA 13 runtime on the host; the Windows zip carries cudart64_*.dll. Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep their own terms (NOTICE-kernel).

Full Changelog: https://github.com/gpillon/ignis/compare/v0.5.0...v0.6.0