TypeAR: Jev-inspired type-safe decoding

An open-sourced decoding engine for autoregressive LLMs inspired by Jev: one output token per decision, no out-of-schema outputs, KV-cache reuse; demoed on SGLang with Qwen 27B.

Notes

Categorical decisions cost one output token; numbers use tokenizer-native constrained decoding; 5.8x relative throughput batched versus sequential on Qwen3.8-27B. Repository now under the TypeAR-AI org.

source-backed — Public repository, docs or live artifact. About this label

Destinations

Also linked

• Original repository (redirects) ↗

Filed under

• #replica
• #decoding
• #sglang

In collections

• Local & replica models →