services.tabbyapi.settings.draft_model.draft_cache_mode
NixOS option
Cache mode for draft models to save VRAM. Unlike the model’s cache_mode, this does not accept a k_bits,v_bits pair.
type: one of "FP16", "Q8", "Q6", "Q4"Default
"FP16"
Example
declared in: nixos/modules/services/web-apps/tabbyapi.nixView source on NixOS/nixpkgs →"Q8"