Tag: quantization
-
TorchAO vs ONNX Runtime: 8-bit Quantization Benchmark
Compare TorchAO vs ONNX Runtime 8-bit quantization performance. Benchmark results reveal surprising differences in speed, accuracy, and memory usage.
-
QAT vs PTQ: When 3% Accuracy Drop Kills Your Model
Compare QAT vs PTQ to find when that 3% accuracy gap destroys real-world performanceโand which quantization method saves your model.
-
QLoRA Review: 4-Bit Finetuning for 65B Models on One GPU
QLoRA fits a 65B model on one 48GB GPU with zero quality loss. Here's exactly how NF4 quantization, double quant, and LoRA combine โ with code.