flashinfer-ai/flashinfer

flashinfer

✅ Recently maintained

FlashInfer: Kernel Library for LLM Serving

6,842Stars
1,688Forks
+507-day stars

Cuda · Apache-2.0 · Last push 2026-11-03

Official repository & installation ↗
gpularge-large-modelscudapytorchllm-inferencejitattentionnvidiadistributed-inferencemoe

GitValue · Value evidence

Unscored · Evidence coverage 0%

Provisional rubric; documentation and runtime testing are separate. Missing evidence does not imply low value.

Star history

2025-09-292026-11-03
View historical data
DateStars
2026-10-056,539
2026-10-066,543
2026-10-076,550
2026-10-086,564
2026-10-096,570
2026-10-106,577
2026-10-116,582
2026-10-126,595
2026-10-136,605
2026-10-146,610
2026-10-156,622
2026-10-166,647
2026-10-176,670
2026-10-186,689
2026-10-196,705
2026-10-206,721
2026-10-216,731
2026-10-226,746
2026-10-236,757
2026-10-246,770
2026-10-256,782
2026-10-266,789
2026-10-276,798
2026-10-286,811
2026-10-296,819
2026-10-306,824
2026-10-316,826
2026-11-016,829
2026-11-026,832
2026-11-036,842

Get the project

git clone https://github.com/flashinfer-ai/flashinfer.git

Cloning downloads the source. For dependencies and installation, follow the project README.

Related projects & alternative candidates

Related domains do not imply feature parity. Review official documentation and licenses.

private-gpt

Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference serve

★ 57,626