Agent infrastructure
beellama.cpp
anbeeld/beellama.cpp
llama.cpp fork focused on local LLM inference efficiency, using KVarN, KV-cache precision tailoring, and low-bit quantization to support longer context with better precision in the same VRAM.
Topics
Related projects
LMCache12.3kA vendor-neutral KV cache management layer for LLM inference that reuses cached context to cut latency and boost throughput.
qwen38-27b-rtx30902.6kAn optimized vLLM serving stack for running large Qwen models fast on a single consumer GPU.
bigmoeonedge709An inference engine built on stock llama.cpp that runs oversized MoE models on memory-constrained phones by streaming only routed experts from flash without quality loss.
llm-action25.3kA curated Chinese knowledge base sharing LLM training, fine-tuning, inference, and deployment tutorials with hands-on code.
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
nano-vllm16.3kA lightweight vLLM reimplementation in about 1,200 lines of Python delivering fast, readable offline LLM inference.