--model-max-context <tokens> |
IGNIS_MODEL_MAX_CONTEXT |
40960 |
Max per-sequence context (prompt + generation). The KV pool must be able to hold one of them or the load is refused. On the 27B the attention bounds it too: at most 524,288 on bf16, 1,048,576 on hq-e8-2b; past that the start is refused. Flash-Next has no such bound. A 27B context the profile or the default gave adapts to the free memory (Adaptation); a named one never does. |
--model-adapt-context-step <tokens> |
IGNIS_MODEL_ADAPT_CONTEXT_STEP |
65536 |
The 27B only: when its VRAM plan refuses a context the profile or the default gave, the load lowers it this many tokens at a time, never below this many; a start whose plan does not hold one step is refused. A whole number of KV pages (a nonzero multiple of 64). A Flash-Next start refuses it. |
--model-default-max-tokens <n|0> |
IGNIS_MODEL_DEFAULT_MAX_TOKENS |
38912 |
The max_tokens of a request that sends none (see Chat completions), its reasoning included, on both models. Never more than what the prompt leaves of --model-max-context, so a long prompt is never refused for it. An explicit cap always wins. 0 = none: such a request may generate to the end of the context, as before. (make knob DEFAULT_MAX_TOKENS.) |
--model-prefill-chunk <tokens> |
IGNIS_MODEL_PREFILL_CHUNK |
1024; 2048 for the 27B under the built-in profiles |
The prefill chunk width, a nonzero multiple of 128. The program’s prefill scratch is reserved for it at load. |
--spec-decode-lanes <n> |
IGNIS_SPEC_DECODE_LANES |
the model’s own: 8 on the 27B, 3 on Flash-Next |
1..=8, both models: the sequences decoded at once. Scoped like any field both models take (--qwen38-spec-decode-lanes, --qwen38flashnext-spec-decode-lanes); a general value is both models’ on a switch. On the 27B each lane reserves its state (~233 MB with DFlash2 under hq-e8-2b) and its share of the round buffers (~35 MB at a 7-token draft window: sampling, the decode and verify rounds, the drafter’s), its decode and verify graphs are captured at widths up to the lanes, and the default host retained slots are two per lane: 2 frees ~1.6 GB of VRAM for the KV pool against 8. On Flash-Next the lanes share the KV pool (The KV pool): min(524,288 tokens, lanes × --model-max-context), never below one --model-max-context sequence and a page per retained slot. At make’s 262,144 tokens that is one context at one lane and 524,288 tokens from two lanes on, so one lane leaves the expert cache about 1 GiB more, and further lanes cost only their state; at 131,072 tokens three lanes get 393,216. make MODEL=flash-next runs 3 lanes at up to 262,144 tokens each (the checkpoint’s trained positions); the make knob is LANES, passed in the started model’s own section. |
--model-decode-share <percent> |
IGNIS_MODEL_DECODE_SHARE |
25 on both models |
The part of the time decoding lanes keep while a prompt prefills, 0-99: after a chunk that took t, the next chunk of the same prompt waits t * s / (1 - s) of wall time while they decode, and nothing waits when no lane decodes or after the prompt’s last chunk. It trades the prefilling request’s TTFT (about x1.33 at 25, x2 at 50, only while lanes decode) for the lanes’ rate (15.6-18.3 tok/s on Flash-Next instead of ~1; why 25: ADR 0018). --model-decode-share 0 restores one decode round per chunk, the pacing before #92; 50 splits time evenly. A prompt alone is unaffected. |
--model-kv-format <fmt> |
IGNIS_MODEL_KV_FORMAT |
hq-e8-2b |
hq-e8-2b (serving) or bf16 (retained; the format every correctness oracle runs against — ADR 0022). Decides what a pool byte budget is worth in tokens. |
--model-prefill-attention <precision> |
IGNIS_MODEL_PREFILL_ATTENTION |
fp8 |
The 27B only: the precision of the prompt attention’s band pass over an hq-e8-2b prefill chunk. fp8 runs it in 8 bits (Q·Kᵀ in INT8, P·V in E4M3); fp16 is the FP16-accumulate fallback. Fixed for the life of the load. Flash-Next’s attention is QSA, with no such pass: a Flash-Next start refuses an explicit value. |
--model-exception-weights <format> |
IGNIS_MODEL_EXCEPTION_WEIGHTS |
bf16 |
The 27B only: how its nine BF16 exception tensors (the attention projections of the early GQA layers, the GDN output of layer 4) are held on the GPU. bf16 keeps them as the artifact stores them. fp8 converts them at load to FP8 row-scale: 0.53 GB less VRAM, cold prefill ~3% faster at 14K-105K, the same 4K KLD, but two decision fixtures read differently (finding); they are multiplied on the FP8 tensor cores in prefill and in any call past 8 rows (a verify round of two lanes or more), with BF16 activations in a decode round and a one-lane verify round. Fixed for the life of the load. Flash-Next has none: a Flash-Next start refuses an explicit value. |
--vram-kv-pool-bytes <bytes|tokens> |
IGNIS_VRAM_KV_POOL_BYTES |
the KV pool policy’s |
The KV pool, on both models: a byte count (K/M/G suffixes) or a token count, <n>tok, <n>Ktok or <n>Mtok with binary multipliers (512Ktok is 524,288 tokens: the same context on either model and either format, where a byte count is worth a different one on each). It replaces the policy’s size (The KV pool). Smaller than one --model-max-context sequence plus a page per retained slot, or past the VRAM budget, fails the load by name. (make knob KV_POOL_BYTES.) |
--reuse-kv-host-pool-bytes <bytes> |
IGNIS_REUSE_KV_HOST_POOL_BYTES |
2G |
The KV-RAM host tier. Page-locked whole at start and held for the life of the load, so it is RAM the process holds even idle — and the figure Windows reports as shared GPU memory. 0 disables the host tier. (make sets 8G, and Flash-Next’s own 2G as --qwen38flashnext-reuse-kv-host-pool-bytes: knob FN_KV_HOST_POOL_BYTES.) |
--spec-backend <backend> |
IGNIS_SPEC_BACKEND |
unset |
Speculative decoding backend: dflash2 (Qwen3.8-27B), mtp (Qwen3.8-Flash-Next, see Flash-Next speculation (MTP)), off, or unset for none. |
--spec-draft-tokens <n> |
IGNIS_SPEC_DRAFT_TOKENS |
— |
1..7. Required with --spec-backend dflash2; with --spec-backend mtp the most drafts a lane verifies per round (default 2). |
--spec-draft-rows <n> |
IGNIS_SPEC_DRAFT_ROWS |
0 (= 8) |
Flash-Next MTP only: the rows one verify round may take across all lanes, 0 or 2..32. Each lane drafts min(draft tokens, rows / lanes - 1), so 3 drafts at one lane only, and past 8 five to eight lanes draft too (16: 3 drafts up to four lanes, 2 at five, 1 from six to eight). A budget past 8 reserves the wider rounds’ VRAM, taken from the expert cache: about 50 MB at 16 rows. Past 8 the round is held numerically equivalent to speculation off, not token for token: greedy output may differ from it where two tokens are nearly tied (ADR 0056). Measured slower than 8 on both the RTX 5090 and the RTX PRO 6000 (Flash-Next speculation (MTP)). (make knob DRAFT_ROWS.) |
--spec-draft-head <head> |
IGNIS_SPEC_DRAFT_HEAD |
full |
Needs --spec-backend. The head the drafter proposes with: full (the target’s output head), shortlist (the artifact’s Q4 head over the 131,072 most frequent tokens, ~357 MB more VRAM: it is loaded beside the full head) or auto (full on every card, the rtx50-16gb profile’s value; the load logs draft_head auto -> full). What a round accepts is still the target’s choice, so the text changes only at near ties; measured on coding prompts shortlist accepts 7% fewer tokens per round and is slower overall (finding). (make knob DRAFT_HEAD.) |
--model-rope-scaling <spec> |
IGNIS_MODEL_ROPE_SCALING |
auto |
auto decides from the model’s --model-max-context: no scaling up to the checkpoint’s trained 262,144 positions, else YaRN at max_context / 262,144 (524,288 is 2, 786,432 is 3, 1,048,576 is 4); past 64x (16,777,216 tokens) the start is refused. none is the linear table whatever the context; yarn:F[,t=<c>][,bf=<n>][,bs=<n>] rescales the envelope by F in (1, 64]. An explicit value overrides auto. Both models apply it; on Flash-Next one table serves attention, its sparse-attention indexer and the MTP head, with YaRN’s attention factor on queries and keys alike, as the model’s reference implementations apply it (ADR 0059). It scales the text table only: the DFlash2 drafter keeps its own unscaled table at window-local positions, and vision its 2-D one. The load log (ignis.model.loaded) names the table run (rope_scaling) beside the setting (rope_scaling_setting). (make knob ROPE_SCALING.) |
