huggingface/datatrove

datatrove

✅ Recently maintained

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

3,414Stars
328Forks
+77-day stars

Python · Apache-2.0 · Last push 2026-11-03

Official repository & installation ↗

GitValue · Value evidence

Unscored · Evidence coverage 0%

Provisional rubric; documentation and runtime testing are separate. Missing evidence does not imply low value.

Star history

2025-09-292026-11-03
View historical data
DateStars
2026-10-053,372
2026-10-063,373
2026-10-073,374
2026-10-083,377
2026-10-093,378
2026-10-103,379
2026-10-113,379
2026-10-123,381
2026-10-133,383
2026-10-143,387
2026-10-153,389
2026-10-163,390
2026-10-173,390
2026-10-183,393
2026-10-193,393
2026-10-203,395
2026-10-213,397
2026-10-223,398
2026-10-233,400
2026-10-243,402
2026-10-253,405
2026-10-263,406
2026-10-273,409
2026-10-283,409
2026-10-293,410
2026-10-303,412
2026-10-313,413
2026-11-013,414
2026-11-023,414
2026-11-033,414

Get the project

git clone https://github.com/huggingface/datatrove.git

Cloning downloads the source. For dependencies and installation, follow the project README.

Related projects & alternative candidates

Related domains do not imply feature parity. Review official documentation and licenses.