Agent infrastructure
tensorsharp
zhongkaifu/tensorsharp
Native .NET inference engine for GGUF text and multimodal LLMs, providing a console app, web chatbot interface, and Ollama/OpenAI-compatible APIs with GPU support across Windows, macOS, and Linux.
Topics
Related projects
sglang38.3kA fast open-source inference framework for serving large language and multimodal models at scale.
nano-vllm16.2kA lightweight vLLM reimplementation in about 1,200 lines of Python delivering fast, readable offline LLM inference.
freetoken14.8kFreeToken is an edge-native Mixture-of-Experts inference engine that runs large open-weight models on consumer hardware, with a desktop app, CLI, and OpenAI/Anthropic-compatible APIs.
LMCache12.2kA vendor-neutral KV cache management layer for LLM inference that reuses cached context to cut latency and boost throughput.
inference9.6kA unified model-serving library to deploy LLMs and multimodal models locally or in the cloud via OpenAI-compatible API.
turbo-fieldfare6.9kAn efficient Gemma 4 inference runtime for Apple-silicon Macs, streaming experts under low-memory budgets and bundling a demo app with evaluation tooling.