Abstract: Post-training quantization (PTQ) for vision transformers (ViTs) has received increasing attention from both academic and industrial communities due to its minimal data needs and high time ...

To achieve a better balance between performance and complexity in SCL decoders, non-uniform quantization (NUQ) is commonly employed. NUQ strategically adjusts the quantization steps to improve the ...

Vector Post-Training Quantization (VPTQ) is a novel Post-Training Quantization method that leverages Vector Quantization to high accuracy on LLMs at an extremely low bit-width (<2-bit). VPTQ can ...

[2024/01] We provide experimental support for GGUF q4_0 and q4_1 formats. [2024/11] We provide experimental support for VLM quantization, please check out the README [2024/11] We provide some tips and ...