Agent infrastructure
whisper.cpp
ggml-org/whisper.cpp
A dependency-free, high-performance C/C++ port of Whisper for on-device speech recognition, with quantization, broad hardware acceleration, and many language bindings.
Topics
Related projects
faster-whisper26.7kA fast Whisper transcription library built on CTranslate2, with quantization, batching, VAD filtering, and word timestamps.
whisper-ctranslate21.4kA faster, OpenAI-compatible Whisper CLI client built on CTranslate2, adding speaker diarization, VAD filtering, live microphone transcription, and Docker images.
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
freetoken16.8kFreeToken is an edge-native Mixture-of-Experts inference engine that runs large open-weight models on consumer hardware, with a desktop app, CLI, and OpenAI/Anthropic-compatible APIs.
nano-vllm16.3kA lightweight vLLM reimplementation in about 1,200 lines of Python delivering fast, readable offline LLM inference.
inference9.6kA unified model-serving library to deploy LLMs and multimodal models locally or in the cloud via OpenAI-compatible API.