// TOPIC
#quantization
3 articles
◆◆IntermediateNVIDIAHugging Face
01Choosing a serving engine: vLLM, SGLang, TensorRT-LLM, and Ollama in 2026
A decision framework for picking the right LLM inference engine based on workload, hardware, and ops complexity — grounded in 2025-2026 benchmarks.
#inference#serving#quantization
15 min◆◆IntermediateNVIDIAHugging Face
02Quantization Demystified: GPTQ, AWQ, GGUF, and FP8 With Real Tradeoffs
A practical comparison of GPTQ, AWQ, GGUF, and FP8 quantization formats — memory math, quality loss, hardware targets, and when each format actually wins in production.
#quantization#inference#serving
16 min◆◆IntermediateHugging FaceMeta
03LoRA, QLoRA, and the PEFT Landscape
How low-rank adapters let you fine-tune a 70B model on a single GPU, with the rank, alpha, and target-module settings that actually matter in production.
#fine-tuning#post-training#quantization
14 min