InferSSD is a local inference engine built on llama.cpp that streams mixture-of-experts weights from an SSD, letting you run very large GGUF models with browser chat and an OpenAI-compatible API.
1226cuda,gguf,llama-cppInferSSD is a local inference engine built on llama.cpp that streams mixture-of-experts weights from an SSD, letting you run very large GGUF models with browser chat and an OpenAI-compatible API.