jundot/omlx

omlx

✅ Recently maintained

LLM inference server for Apple Silicon with continuous batching and SSD caching, managed from the macOS menu bar.

23,304Stars
2,088Forks
+2137-day stars

Python · Apache-2.0 · Last push 2026-11-03

Official repository & installation ↗
apple-siliconinference-serverllmmacosmlxopenai-api

GitValue · Value evidence

Unscored · Evidence coverage 0%

Provisional rubric; documentation and runtime testing are separate. Missing evidence does not imply low value.

Star history

2026-02-092026-11-03
View historical data
DateStars
2026-10-0522,489
2026-10-0622,534
2026-10-0722,567
2026-10-0822,593
2026-10-0922,625
2026-10-1022,653
2026-10-1122,676
2026-10-1222,702
2026-10-1322,741
2026-10-1422,779
2026-10-1522,815
2026-10-1622,840
2026-10-1722,857
2026-10-1822,882
2026-10-1922,914
2026-10-2022,944
2026-10-2122,958
2026-10-2222,981
2026-10-2323,004
2026-10-2423,021
2026-10-2523,052
2026-10-2623,078
2026-10-2723,114
2026-10-2823,138
2026-10-2923,157
2026-10-3023,207
2026-10-3123,235
2026-11-0123,268
2026-11-0223,282
2026-11-0323,304

Get the project

git clone https://github.com/jundot/omlx.git

Cloning downloads the source. For dependencies and installation, follow the project README.

Related projects & alternative candidates

Related domains do not imply feature parity. Review official documentation and licenses.

vllm

High-throughput, memory-efficient LLM inference and serving engine that lets developers deploy models faster without building serving infrastructure.

★ 96,180litellm

Call 100+ LLM APIs using the OpenAI input/output format.

★ 62,604sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

★ 38,386