Agent infrastructure
willow-inference-server
toverainc/willow-inference-server
Willow Inference Server is a self-hosted, CUDA-optimized inference server exposing Whisper speech recognition, text-to-speech and LLM inference over REST, WebRTC and WebSocket APIs.
Topics
Related projects
speech-to-text-benchmark699A minimalist, extensible Python framework for benchmarking speech-to-text engines, comparing accuracy, latency, memory and compute across local and cloud providers on standard datasets.
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
faster-whisper26.7kA fast Whisper transcription library built on CTranslate2, with quantization, batching, VAD filtering, and word timestamps.
kaldi15.5kkaldi-asr/kaldi is the official location of the Kaldi project.
vosk-api15.2kAn offline speech recognition toolkit with small models and bindings for many languages, built for apps and devices.
whisper.cpp55.1kA dependency-free, high-performance C/C++ port of Whisper for on-device speech recognition, with quantization, broad hardware acceleration, and many language bindings.