Agent infrastructure
qwen38-27b-rtx3090
syv-ai/qwen38-27b-rtx3090
An optimized vLLM serving stack for running large Qwen models fast on a single consumer GPU.
Topics
Related projects
beellama.cpp1.2kllama.cpp fork focused on local LLM inference efficiency, using KVarN, KV-cache precision tailoring, and low-bit quantization to support longer context with better precision in the same VRAM.
project-maya2.6kProject Maya is a local LLM inference engine that runs large GLM, DeepSeek, and MiniMax models on your own GPUs, with quantized weights, a web dashboard, and OpenAI/Anthropic-compatible APIs.
mlx-dspark736A native MLX speculative-decoding inference engine that speeds up local LLMs on Apple Silicon.
pooled647Pooled runs large open models across browsers by splitting model layers among devices in a room over WebRTC and WebGPU. Users open a link to chat or use Code mode, and the room can expose an OpenAI/Anthropic-compatible API to other tools.
little-coder2.7kWraps the Pi agent framework as a lightweight coding harness bundled with extensions and skills, optimizing cold startup and usability for smaller local language models.
mtplx2.7kA native Mac app and server that runs large language models locally with chat, dashboards, and compatible APIs.