microsoft/MInference

MInference

✅ Recently maintained

[NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

1,231Stars
84Forks
+27-day stars

Python · MIT · Last push 2026-10-10

Official repository & installation ↗

Star history

2025-09-292026-11-02
View historical data
DateStars
2026-10-041,227
2026-10-051,227
2026-10-061,227
2026-10-071,227
2026-10-081,227
2026-10-091,228
2026-10-101,228
2026-10-111,228
2026-10-121,228
2026-10-131,228
2026-10-141,228
2026-10-151,228
2026-10-161,228
2026-10-171,228
2026-10-181,228
2026-10-191,228
2026-10-201,228
2026-10-211,228
2026-10-221,228
2026-10-231,228
2026-10-241,229
2026-10-251,229
2026-10-261,229
2026-10-271,229
2026-10-281,229
2026-10-291,231
2026-10-301,231
2026-10-311,231
2026-11-011,231
2026-11-021,231

Get the project

git clone https://github.com/microsoft/MInference.git

Related projects & alternative candidates

Related domains do not imply feature parity. Review official documentation and licenses.

rea

Use AI agents to reverse engineer app behavior and native binaries with tool integrations.

★ 419,787openclaw

Cross-platform open-source personal AI assistant focused on data ownership, written in TypeScript for self-hosting.

★ 393,671deepseek-harness

DeepSeek's open-source agent framework where every capability is built as a plugin.

★ 268,697