jev-curate: sift pretraining datasets at 1,500+ rows/sec

A high-throughput sifter for synthetic and pretraining datasets: stream Parquet and JSONL, filter and score every row with Jev.

Notes

1,500+ rows per second per the README.

source-backed — Public repository, docs or live artifact. About this label

Destinations

Filed under

• #dataset
• #training
• #batch
• #filtering

In collections

• Search, ranking & queues →