Needle 3 vs Jev: an 'imposter Jev' gateway benchmark

An LLM gateway that mimics TypeSafe's structured output so local models can be scored against Jev; Needle 3 on an M4 Pro came out worse than Jev, Qwen3.8 27B and OSS 120B.

Notes

Author: worse quality and much slower locally; 'maybe I did something wrong on implementation'.

source-backed — Public repository, docs or live artifact. About this label

Destinations

Filed under

• #benchmark
• #local
• #comparison

In collections

• Benchmarks & independent tests →