# ignis

> A CUDA inference engine for Qwen3.8 on RTX 50-series and RTX PRO 6000 cards: install, configure, call and extend it.

Each link is a page's Markdown source; the rendered guide is https://gpillon.github.io/ignis/ and the code https://github.com/gpillon/ignis. An AI agent working for a user (asked what ignis is, to install it, or to connect a client to it) starts at the agent path, under Reading paths: each task with its checks and the choices that are the user's.

## Start here

- [What ignis is](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/start/index.md): What ignis serves, on which cards and operating systems, what comes with it, and what it deliberately does not do.
- [Install](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/start/install.md): Get ignis: the native binaries for Windows and Linux, or the container image, and what each needs on the host.
- [Quick start](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/start/quick-start.md): From nothing to a first answer: run the container, let it fetch the default model, open the Playground and send one request.
- [Connect a client](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/start/connect-a-client.md): The base URL, the API key and how to point the OpenAI SDKs, curl, opencode and chat front ends at ignis.
- [Install with a coding agent](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/start/install-with-an-agent.md): Let the coding agent you already use install ignis: what to tell it, what it checks, what it asks you, and what it leaves to you.

## Why ignis

- [Why ignis](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/index.mdx): What sets ignis apart: one model family on one class of card, built for the load an agent makes, and every claim measured.
- [One round for every lane](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/decode-round.md): Eight lanes share one pass over the weights, replayed from a CUDA graph per batch width, so a decode round runs at the card's memory bandwidth.
- [A drafter inside the artifact](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/dflash2.md): DFlash2 proposes up to seven tokens a round and the model verifies them in the same pass, so the output stays the model's own.
- [A 125B MoE on a 32 GB card](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/flash-next.md): Flash-Next's experts in a 2.5-bit trellis code, the hot ones in VRAM and the rest streamed from pinned RAM, at about 100 tok/s on one lane.
- [The 27B on a 16 GB card](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/sixteen-gb-cards.md): A 3-bit trellis body, a profile chosen by the card's memory, and four quality gates passed: the 27B on an RTX 5060 Ti 16 GB, 5070 Ti or 5080.
- [KV at 9 KB a token](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/kv-in-bytes.md): The hq-e8-2b KV format holds 7.11 times the tokens of BF16, and the pool is budgeted in bytes, not in sequences.
- [State that outlives its request](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/state-that-outlives.md): A repeated prompt resumes from a copy held in VRAM, pinned RAM or on disk, and the response says how many tokens it reused.
- [Every byte planned at load](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/every-byte-planned.md): The server reserves and prints its whole VRAM plan before the first request, and refuses a start it cannot hold instead of failing at minute forty.
- [Lanes that know who is asking](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/lanes.md): A request says whether it is the conversation or a subagent, so a burst of subagents fills idle lanes and yields first when memory is short.
- [The Responses API over a WebSocket](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/responses-websocket.md): One socket per agent, only new items per turn, a real prewarm and a queue instead of a 503.
- [Reasoning that finishes](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/reasoning-that-finishes.md): A reasoning loop is caught while it streams, cut back and nudged in the same response; a turn that ends inside its thinking is closed, not returned empty.
- [System One decisions, without decoding](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/decisions.md): A System One-compatible API (POST /v1/systemone, POST /v1/decide) that reads a yes/no, a choice or a score from one position of the model's distribution: one prefill, no tokens generated.
- [Images as evidence](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/images-as-evidence.md): The vision tower's output is kept past the request, so several questions over one screenshot encode it once.
- [Pictures from the same card](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/image-generation.md): Qwen-Image-2.1-Turbo serves OpenAI's image routes from the same card, with kernels written for the 5090: a 2048x2048 image in about 7.9 s end to end.
- [Models that come and go](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/models-that-come-and-go.md): A request that names another model switches the server to it and waits; an idle model is unloaded and the card is free.
- [Context that follows the card](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/context-follows-the-card.md): Rope scaling picks itself from the context: none up to 262,144 tokens, YaRN at the exact ratio beyond.
- [Everything in one binary](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/batteries.md): A Playground, a live Monitor, Prometheus, a verified model download and a configuration API ship in one server.
- [How these numbers are made](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/why-ignis/how-we-measure.md): What a finding is, how gates are registered before a run, why greedy and sampled speculation differ, and which negative results are kept.

## Playground

- [The Playground](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/index.md): The browser UI that ships inside ignis-server: every feature of the engine, one view at a time.
- [Chat](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/chat.md): The Playground's chat: a reply with its reasoning and figures, the prompt box, the model picker, the context meter and the session log.
- [Settings](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/settings.md): The chat's settings panel: the system prompt, thinking and its budget, sampling, the lane tag, parallel sessions, the transport and the tools.
- [Tools and agents](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/tools-and-agents.md): The chat's tools: subagents on agent lanes, web search, questions to you, JavaScript, a plan, memory, files, pictures and attachments.
- [Sessions and storage](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/sessions.md): The chat's sessions, what the Playground keeps in this browser, and what a reload forgets.
- [Decide](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/decide.md): The Decide view: many questions over one piece of evidence, answered from one prefill with the whole distribution behind each answer.
- [Image studio](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/image-studio.md): The Image view: make and edit pictures with the image model, step by step, with marks, masks and seeds.
- [Monitor](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/monitor.md): The Monitor: the server's own metrics, scraped by the browser and drawn live, from the verdict in words to the VRAM plan.
- [Server](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/playground/server.md): The Server view: the loaded model, the models a switch can go to, and the running configuration as a form you can apply.

## Reading paths

- [The user path](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/user.md): From nothing to a chat: install ignis, fetch a model, open the Playground, connect your tools. No configuration needed.
- [The advanced path](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/advanced.md): Tune ignis: how a request is served, the configuration model, VRAM and the KV pool, reuse, speculation, host RAM, models, and running it in production.
- [The developer path](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/developer.md): Build on ignis's API: the OpenAI-compatible endpoints, the Responses API, streaming and tools, deadlines, model switching, images, decisions.
- [The contributor path](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/contributor.md): Work on ignis itself: the ground rules, building from source, the tests and the GPU discipline, make, the records a change leaves, releases.
- [The agent path](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/paths/agent.md): For an AI agent working for a user: what ignis is in one paragraph, the checks before installing it, the choices that are the user's, starting it and seeing it serve, connecting a client, and where each question is answered.

## Guides

- [Get a model](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/models.md): The catalog, `model download`, letting the server fetch its model, the endpoint and token, and pointing at an artifact you already have.
- [Your own models: an operator catalog](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/operator-catalog.md): Add models to the catalog with a YAML or JSON file in the built-in catalog's format.
- [Air-gapped machines](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/air-gapped.md): Carry the files in, or point the server at a mirror.
- [The uncensored variant](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/uncensored.md): qwen3.8-27b-abliterated: the 27B with the huihui-ai abliteration, and what it changes.
- [Measure a 16 GB card](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/measure-a-16gb-card.md): Run `ignis-bench small-card` against your server and paste the record into a GitHub issue: the card, the VRAM plan, what the load adapted, decode speed and time to first token.
- [Telemetry and metrics](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/telemetry.md): Logs, OpenTelemetry traces and the Prometheus metrics listener.
- [Run behind Knative](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/knative.md): Scale-to-zero and connection draining: the socket idle close and the Monitor's polling.
- [Why not DGX Spark?](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/guides/dgx-spark.md): Why ignis does not target the DGX Spark (GB10, SM121).

## Concepts

- [How ignis serves a request](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/how-ignis-serves.md): The path of a request through the server: admission, prefill, decode lanes, speculation and the stream back, and how the Rust server and the CUDA kernel leaf divide the work.
- [Models and artifacts](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/models-and-artifacts.md): What a .ninfer artifact is, the model families ignis serves, how a served id maps to a catalog entry, and how each family uses VRAM and host RAM.
- [The KV cache and prompt reuse](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/kv-cache-and-prompt-reuse.md): How the KV pool, retained state and the host and disk tiers let a request skip work an earlier request did, what that saves, and when it does not apply.
- [The VRAM plan](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/vram-plan.md): How a load decides every byte of device memory before the first request, why a start is refused instead of running out of memory later, and what Windows does differently.
- [Speculative decoding](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/speculative-decoding.md): How DFlash2 and Flash-Next's MTP head draft tokens that the model then verifies, what acceptance means, how the row budget is shared across lanes, and why MTP is off on the RTX 5090.
- [Reasoning and the loop guard](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/reasoning.md): How thinking is bounded and closed, what the server does when a turn ends inside open reasoning, and how a reasoning loop is detected, taken back and nudged.
- [Model switching and idle unload](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/switching-and-idle-unload.md): Why ignis holds one model at a time, what a switch and an idle unload do to the card, how a request can trigger either, and what a client sees while a model loads.
- [Profiles and defaults](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/concepts/profiles.md): What a hardware profile is, where it sits between the config file and the built-in defaults, what the rtx5090 and rtxpro6000 profiles choose and why, and how to define your own.

## Configuration

- [Configuration](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/index.md): Fields, groups, flags, env vars and the config file; precedence, per-model scopes, profiles and live changes.
- [Model and server](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/model-and-server.md): The artifact, the served id, the bind address, keys and the server's own switches.
- [Prompting](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/prompting.md): Sampling defaults, reasoning and the chat template's switches.
- [Context, KV and speculation](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/context-and-speculation.md): Context length, KV format, decode lanes and DFlash2 speculative decoding.
- [Flash-Next speculation (MTP)](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/flash-next-mtp.md): Flash-Next's multi-token-prediction head: when it pays, and the row budget.
- [VRAM budget and the KV pool](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/vram-and-kv-pool.md): How the VRAM plan sizes the KV pool and retained state, and the knobs that move it.
- [KV-disk](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/kv-disk.md): The disk tier under the host KV pool for Flash-Next's long contexts.
- [Flash-Next n-gram startup cache](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/ngram-cache.md): The n-gram hot rows: where they are kept and what a start costs without them.
- [Host RAM](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/host-ram.md): What each model needs in host memory, and how the host plan refuses a start that does not fit.
- [Vision](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/vision.md): The 27B's vision tower: on, off, and its token budget.
- [Playground, metrics, access](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/configuration/access.md): The Playground, the metrics listener, API keys, CORS origins and public exposure.

## API

- [The API](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/index.md): The OpenAI-compatible HTTP surface: every endpoint, probes, keys and errors.
- [Chat completions](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/chat-completions.md): POST /v1/chat/completions: fields, streaming, tools, reasoning and the agent lane.
- [The Responses API](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/responses.md): POST /v1/responses and its WebSocket mode.
- [Image generation](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/images.md): The image model: generating and editing images over the API and in the Playground.
- [Switching models](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/model-switching.md): Explicit and implicit switches between the served models, and what a request sees during one.
- [Idle unload](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/idle-unload.md): The server unloads an idle model to free the GPU, and loads it again on the next request.
- [Deadlines](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/deadlines.md): How long a request may take, what happens when it runs over, and how a client states its own deadline.
- [Counting a prompt](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/count.md): POST /v1/tokenize: how many tokens a chat body prefills, without serving it; and /v1/detokenize.
- [Examples](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/examples.md): curl calls for the common requests.
- [Decisions: a System One-compatible API](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/api/decide.md): POST /v1/decide and POST /v1/systemone: the System One API's typed decisions, answered from one prefill of the model's distribution, plus locating a line or an item.

## Reference

- [Reference](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/index.md): The lists the code writes about itself — config fields, profiles, models, metrics, routes, commands — and the glossary.
- [Every config field](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/configuration.md): Every field ignis-server takes, by group: its flag, env var, default, rule, the models it applies to and whether it changes live.
- [Built-in profiles](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/profiles.md): The hardware profiles built into ignis-server and every value each one sets, general and per model.
- [Built-in models](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/models.md): The catalog compiled into ignis-server: every model it can fetch, the repository and revision it is pinned at, and every file's size and SHA-256.
- [Metrics](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/metrics.md): Every Prometheus metric ignis-server exports: its name, type, meaning, and when it appears.
- [HTTP routes](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/http-routes.md): Every route of the HTTP surface, from the OpenAPI document the server builds from its handlers.
- [Command line](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/cli.md): ignis-server's commands and subcommands, as its --help prints them.
- [Glossary](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/reference/glossary.md): The terms the ignis guide uses, in alphabetical order, each with a short definition and a link to the page that explains it.

## Troubleshooting

- [Troubleshooting](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/troubleshooting.md): Symptoms and error messages, grouped by when they appear, with the cause and the fix for each.

## Contributing

- [Contributing](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/index.md): How ignis is worked on: the ground rules, the repository's layout, and where each kind of work is recorded.
- [Building from source](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/building.md): The kernel leaf, the Rust workspace, the tests and the canary suite.
- [Testing](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/testing.md): The CPU-only default suite, the explicit GPU profile, the Playground's tests and the guide's build: what each covers and when it is due.
- [Working with make](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/make.md): The make targets a developer and an agent use, and how a run is guarded.
- [How work is tracked](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/conventions.md): GitHub issues and their triage labels, specs, ADRs, findings, the guide, worktrees and commit messages.
- [Cutting a release](https://raw.githubusercontent.com/gpillon/ignis/main/docs/user/contributing/releases.md): Version bump, changelog, tag, and what the release workflow publishes.

## Optional

- [Releases](https://github.com/gpillon/ignis/releases): Every release's notes, which the guide's Changelog is written from.
- [Architecture decisions](https://github.com/gpillon/ignis/tree/main/docs/adr): Why ignis is built the way it is, one record per decision.
- [AGENTS.md](https://raw.githubusercontent.com/gpillon/ignis/main/AGENTS.md): The rules for agents working on ignis itself, not for using it.