smg-project/smg

smg

✅ Recently maintained

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

698Stars
205Forks
+137-day stars

Rust · Apache-2.0 · Last push 2026-11-03

Official repository & installation ↗
anthropicclaudegeminiinference-gatewayllmopenairoutingsglangvllmtensorrtllm

GitValue · Value evidence

Unscored · Evidence coverage 0%

Provisional rubric; documentation and runtime testing are separate. Missing evidence does not imply low value.

Star history

2025-11-172026-11-03
View historical data
DateStars
2026-10-05552
2026-10-06553
2026-10-07558
2026-10-08560
2026-10-09562
2026-10-10562
2026-10-11565
2026-10-12569
2026-10-13571
2026-10-14576
2026-10-15583
2026-10-16603
2026-10-17610
2026-10-18617
2026-10-19643
2026-10-20653
2026-10-21658
2026-10-22663
2026-10-23667
2026-10-24670
2026-10-25673
2026-10-26684
2026-10-27687
2026-10-28689
2026-10-29691
2026-10-30693
2026-10-31693
2026-11-01696
2026-11-02698
2026-11-03698

Get the project

git clone https://github.com/smg-project/smg.git

Cloning downloads the source. For dependencies and installation, follow the project README.

Related projects & alternative candidates

Related domains do not imply feature parity. Review official documentation and licenses.

litellm

Call 100+ LLM APIs using the OpenAI input/output format.

★ 62,604vllm

High-throughput, memory-efficient LLM inference and serving engine that lets developers deploy models faster without building serving infrastructure.

★ 96,180sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

★ 38,386