Built-in profiles

Note

Generated from the code by make docs-reference (crates/server/src/user_docs.rs); an edit here is overwritten, and cargo test fails while this page and the code disagree.

A profile is a layer of defaults for one kind of card, between the config file and the built-in defaults: any flag, env var or file value still wins over it. An unset --profile is auto, and auto takes the built-in profile of the card’s class, read from its total memory; --profile none skips the layer. A config file’s profiles: section defines more, or shadows these (Profiles and defaults).

rtx50-16gb

Field	Model	Value
model.max_context	Qwen3.8-27B	262144
spec.backend	Qwen3.8-27B	dflash2
spec.decode_lanes	Qwen3.8-27B	2
spec.draft_head	Qwen3.8-27B	auto
spec.draft_tokens	Qwen3.8-27B	7
vision.enabled	every model	true
vision.max_tokens	every model	4096
vision.residency	every model	host
vram.headroom_bytes	Qwen3.8-27B	1200M

rtx5090

Field	Model	Value
model.max_context	Qwen3.8-27B	917504
model.max_context	Qwen3.8-Flash-Next	589824
model.prefill_chunk	Qwen3.8-27B	2048
spec.backend	Qwen3.8-27B	dflash2
spec.decode_lanes	Qwen3.8-Flash-Next	3
spec.draft_tokens	Qwen3.8-27B	7
vision.enabled	every model	true
vision.residency	every model	host
vram.headroom_bytes	Qwen3.8-27B	1200M
vram.headroom_bytes	every model	1536M
vram.pinned_expert_bytes	every model	2G

rtxpro6000

Field	Model	Value
model.max_context	Qwen3.8-27B	1048576
model.max_context	Qwen3.8-Flash-Next	1048576
model.prefill_chunk	Qwen3.8-27B	2048
ngram.hot_bytes	every model	auto
spec.backend	Qwen3.8-27B	dflash2
spec.backend	Qwen3.8-Flash-Next	mtp
spec.decode_lanes	Qwen3.8-Flash-Next	8
spec.draft_tokens	Qwen3.8-27B	7
spec.draft_tokens	Qwen3.8-Flash-Next	3
vision.enabled	every model	true
vision.residency	every model	device