Jev Alternatives for .NET: What Actually Replaces It, and What Doesn't

I wrote two posts about Jev recently — what it is (a "System One" model that returns a typed value from your options instead of writing JSON token-by-token) and how to test it and reproduce it in C#. The reaction was immediate and it was the right question: do I actually need Jev, or can I get this from things I already run?

Mostly the latter. Let me be blunt about why, and then give you the menu.

The reframe: separate the technique from the model

Jev's pitch bundles two very different things:

1. A technique — force a model's output to be one of your enumerated options, so it can never be malformed. This is constrained decoding, and in 2026 it is thoroughly commoditized. XGrammar is now the default structured-generation backend for vLLM, SGLang, and TensorRT-LLM, running under ~40µs per token. This part you can get everywhere.
2. A model — one purpose-built and tuned so that a single forward pass over your options yields a calibrated probability, fast and cheap, at 88% accuracy on their benchmark. This part is genuinely harder to reproduce.

Almost everything sold as "you need Jev" is really point 1, which you already own. So the useful question isn't "what's another System One model," it's "which constrained-output tool fits where I deploy — and how close does it get to point 2?"

The centerpiece: there is an open-source Jev

Before the menu, the exhibit that settles the argument. open-jev (its browser demo is called SemIf) is an MIT-licensed reproduction of Jev's actual mechanism, running Gemma 3 4B on Apple Silicon via MLX. It does exactly what Jev does: prefill the context once, expand the KV cache across the option batch, and score every option in one padded forward pass — the score is the log-probability of each option's tokens given the context, and a softmax over those scores gives a probability per option. On an M5 Pro it scores 8 options in 0.17 seconds, and it exposes a dead-simple server:

curl -s localhost:8000/score -H 'content-type: application/json' -d '{
  "context": "Customer: my order arrived broken. Agent:",
  "options": [" send a replacement", " read the returns policy"],
  "norm": "mean"
}'
# → {"best":" send a replacement","probability":[0.95,0.05], ...}

It even ships a trained calibration head (ECE 0.027, versus 0.724 for a shuffled control), which is the one hard part starting to fall too. The honest gap: on TypeSafe's own test set the small open model lands around 63.7% accuracy vs Jev's 88.3%. So open-jev proves the mechanism is free and reproducible; the accuracy is where a purpose-built, well-trained model still earns its keep.

That single repo is why I say you probably don't need Jev. Now, the full menu — and because most of these tools are Python-world, I'll flag what a .NET developer can actually reach.

A. Constrained-decoding libraries (the harness itself)

These do point 1 directly — mask the next-token distribution so only legal tokens survive:

• Outlines — the "structured generation" flagship (regex / JSON-schema / CFG).
• XGrammar and llguidance — CFG engines fast enough to often beat unconstrained decoding, because when the grammar forces the next token they skip sampling entirely.
• lm-format-enforcer, and GBNF grammars built into llama.cpp.

.NET reach: these are Python/Rust, so you consume them through a server (below) or, in-process, via LLamaSharp's GBNF support — which is exactly what my LocalDecider uses.

B. Serving engines with guided decoding built in

Here the harness is just a request field:

• vLLM — guided_choice, guided_json, guided_regex (auto-selecting XGrammar/Outlines).
• SGLang, TGI, Ollama (format / JSON schema), LM Studio (structured output).

.NET reach: this is the sweet spot. All of them speak the OpenAI-compatible /v1/chat/completions, so one C# class with response_format: json_schema reaches the whole set. You point BaseAddress at localhost:11434 for Ollama, or a vLLM box, or LM Studio — same code.

C. Cloud frontier models, constrained

If you're already calling a big model, you don't need a second service:

• OpenAI Structured Outputs — response_format: json_schema with strict: true is now the production default; the model cannot emit a value outside your enum. (Plain "JSON mode" is legacy — it only guarantees valid syntax, not your schema.)
• Azure OpenAI, Gemini (responseSchema + enum), Anthropic tool-use.

.NET reach: identical to category B — same json_schema call, just a cloud base URL and a key. One class covers B and C.

D. Don't use an LLM at all

The alternative people forget, and often the right one for pure classification — cheaper and faster than Jev:

• Embeddings + a tiny classifier (logistic regression / kNN over the vectors).
• SetFit, zero-shot DeBERTa, or fastText.

If your decision is "route this ticket into one of five buckets" and you have a few hundred labeled examples, an embedding classifier is a rounding error in cost and latency, and it's actually calibrated. Reach for an LLM-shaped decider only when you don't have labels or the classes keep changing.

The .NET payoff: same interface, swap the alternative

The reason I keep coming back to one IDecider interface is that this whole menu collapses into configuration. I extended github.com/egarim/systemone-deciders so it now has five engines behind that one contract:

IDecider decider = engine switch
{
    "jev"     => new JevDecider(new HttpClient()),                  // TypeSafe cloud
    "openjev" => new OpenJevDecider(new HttpClient{ BaseAddress = new("http://localhost:8000") }),  // the OSS Jev
    "openai"  => new OpenAiCompatDecider(http, model: "gpt-4o-mini"),  // OpenAI/Azure/Ollama/LM Studio/vLLM
    "local"   => new LocalDecider(http),                            // llama.cpp + GBNF
    _         => FakeDecider.Always("technical", 0.82),             // no model, for tests
};

var d = await decider.ChooseAsync(ticket, "Which team?", teams, abstainBelow: 0.6);

OpenJevDecider wraps that /score endpoint and hands you a real per-option distribution in one pass. OpenAiCompatDecider uses strict json_schema and hands you a guaranteed-valid value — but, honestly, not a calibrated probability; it reports confidence 1.0 because the schema can't lie about the shape, only about which valid option was right. That difference is the whole story of this post, encoded in two classes. 22 tests, all hermetic, no key and no GPU required to run them.

The honest verdict

• Want a self-hosted Jev with a real distribution? open-jev. It is the alternative — MIT, runs on your Mac, same mechanism.
• Already calling OpenAI/Azure, or running Ollama/vLLM/LM Studio? Strict json_schema structured outputs. You get bounded, never-malformed values today, from one C# class. You give up calibrated confidence.
• Pure, stable classification with some labels? Skip the LLM — an embedding classifier is cheaper, faster, and calibrated.
• Actually need Jev specifically? Only if you need both calibrated confidence and sub-second latency at a scale your current stack can't hit — and even the calibration you can approximate with temperature scaling on logprobs, or a trained head like open-jev's.

"System One" was never a model you had to buy. It's a way of calling a model — constrain the output space, read the distribution — and in 2026 that capability is lying around in half the tools you already run. Jev packaged it well and tuned it hard, and that tuning is real. But the moment you can point one IDecider at five different engines and watch your app not care which one answered, you've already priced the moat.

Tried open-jev, or wired structured outputs into a .NET service? Tell me how it went via the links on the about page.