Libraries & frameworks
llm-compressor
vllm-project/llm-compressor
A library for quantizing and compressing large language models for fast production inference with vLLM.
Topics
Related projects
aimet2.7kA toolkit for quantizing and compressing trained neural networks for faster efficient edge deployment.
PaddleSlim1.6kAn open-source Paddle toolkit for model compression via quantization, pruning, distillation, sparsity, and neural architecture search.
nncf1.2kA Python framework for compressing neural networks with quantization, pruning, and sparsity for faster OpenVINO inference.
faster-whisper26.7kA fast Whisper transcription library built on CTranslate2, with quantization, batching, VAD filtering, and word timestamps.
turbovec17.5kAn in-process vector index library in Rust with Python bindings, compressing embeddings via TurboQuant for fast, low-memory similarity search.
lz412.7kExtremely fast lossless compression algorithm with a BSD-licensed C library, high-compression variant, CLI tools, and documented block frame formats.