An evidence survey · Updated 7 Oct 2026

Jev Survey

Every answer type-checks. The evidence decides the rest.

TypeSafe’s Jev returns typed answers with a probability for every option, and open projects now copy its shape. This survey reads the studies, audits the open resources and traces every number to its source: what the probabilities mean, when software should act on them, and where typed decisions fail.

state

“Shoes arrived in the wrong size. Can someone call me?”

choice team

  • returns0.81
  • billing0.15
  • other0.04

score anger

  • 0.12calm
  • 0.70upset
  • 0.18furious

noul wants a person

0.92
Illustrative values: one state, three typed questions, a distribution for each — no generated text.
Read the findings Search the evidence Survey (PDF) GitHub
  1. 01Decision contractWhat is asked?F1F10
  2. 02Inference & readoutHow is a distribution made?F5F9
  3. 03Probability & calibrationWhat does it mean?F2F8
  4. 04Selective controlAct, ask or escalate?F3
  5. 05Downstream useDoes the workflow improve?F4F6F7

By

02 The object

A function call that returns a distribution

Jev is a hosted model from TypeSafe AI, released on 15 September 2026 as its first “System One” model. Software sends a text state and typed questions; the model answers from the declared options with probabilities and never writes free text.

Request shapeabridged from the documentation
POST /v1/systemone
{
  "model": "jev-1.13.0",
  "state": { "message": "Shoes arrived in the wrong size…" },
  "questions": {
    "team":  { "type": "choice", "criteria": { "returns": "…", "billing": "…" } },
    "anger": { "type": "score",  "criteria": [ "calm", "upset", "furious" ] },
    "human": { "type": "noul",   "instructions": "Wants a person?" }
  }
}

Questions are evaluated independently against one shared state. Option names and their descriptions reach the model.

choice

Choice

Returns
The chosen option, a probability for every option and a confidence value.
Semantics
A distribution over named options, each defined by a description. The launch post states support for up to 255 options.
Watch
The model reads option names and descriptions; a short label can outweigh its definition, and the vendor notes a lean toward the first option. F1 · Valid ≠ correct
Documentation ↗
score

Score

Returns
A position on ordered levels (it may fall between levels), probabilities per level and confidence.
Semantics
Two to ten described levels. Ordinal, not a regression: the vendor warns against interpolating exact magnitudes.
Watch
Middle levels can absorb errors, and decisions can use only part of the scale. F1 · Valid ≠ correct
Documentation ↗
noul

Noul

Returns
The probability that the answer is yes. No separate confidence field.
Semantics
A proposition probability, optionally with descriptions of what counts as true and false.
Watch
A yes/no question and a two-option Choice over the same proposition need not agree; a statement and its negation need not sum to one. F8 · Calibrated ≠ coherent
Documentation ↗
A

The commercial model

TypeSafe’s hosted Jev, version jev-1.13.0. Its interface, prices, limits and known weak spots are documented; its architecture, training data and confidence statistic are not.

Evidence: black-box evaluations

B

The paradigm

Jev-like typed decision models: encoders, encoder–decoders, frozen or fine-tuned decoders, diffusion reads and adapters that reproduce the request and response shape with other mechanisms.

Evidence: open code and weights, of uneven completeness

C

The systems

Judges, agents, search engines, databases, robots and annotation pipelines whose results depend on typed decisions — and on much else.

Evidence: end-to-end outcomes that need attribution

Documented by the vendor

  • Input is text only: a string, JSON object or array of text. Images, audio and video must be converted first.
  • Versioned IDs; the aliases jev-latest and jev-preview point to jev-1.13.0.
  • confidence is a statistic of the returned distribution; yes/no answers carry none.
  • Calibration is measured over groups of predictions and does not certify an individual answer.
  • A published list of known weak spots ↗: literal reading, math and numbers, dates, indirection, large states, adversarial text and Choice option order.

Vendor-reported, not independently tested

03 The map

Five places a claim can live

Each study is placed where its claim sits on the path from a declared question to a workflow outcome. The map is a frame for comparing evidence, not an architecture.

  1. 01

    Decision contract

    What exactly is being asked, and what may the answer be?

    State, typed question, option names, rubrics, levels, cardinality, abstain options.

    Evidence to look forType validity; name and order invariance; closed-set failure when the right option is missing.

    F1 · Valid ≠ correctF10 · Typed ≠ secure
    Studies at this stage
  2. 02

    Inference & readout

    How is a distribution over the declared options produced?

    Hosted service, encoder heads, frozen or fine-tuned decoder readouts, diffusion reads, generative adapters.

    Evidence to look forSame-backbone ablations; readout geometry; what is and is not disclosed.

    F5 · Compatible ≠ equivalentF9 · Reads, doesn’t compute
    Studies at this stage
  3. 03

    Probability & calibration

    What does a returned probability or confidence mean, and for which population?

    Native vs verbalised probabilities, confidence statistics, ECE/Brier/NLL, recalibration, quantisation.

    Evidence to look forPer-task reliability, high-confidence errors, recalibration on held-out labels.

    F2 · Measurable ≠ calibratedF8 · Calibrated ≠ coherent
    Studies at this stage
  4. 04

    Selective control

    When should software act, ask, escalate or abstain?

    Thresholds, risk–coverage, cascades to stronger models, human review budgets.

    Evidence to look forCoverage at fixed risk; escalation rate; thresholds chosen on a separate selection set.

    F3 · First pass, then escalate
    Studies at this stage
  5. 05

    Downstream use

    Does the whole workflow get better, cheaper or faster?

    End-task success, end-to-end latency, cost per correct completion, derived quantities.

    Evidence to look forSystem-level baselines; module attribution; matched measurement scope.

    F4 · Fast is not one numberF6 · Attribute the gainF7 · Young, clustered evidence
    Studies at this stage

Six research questions

RQ1

Definition & lineage

How do runtime-defined finite-option decisions relate to classifiers, entailment-based labelling, rerankers, reward models and constrained generation?

RQ2

Probability quality

What do returned probabilities and confidence mean, for which tasks, languages and distributions, and are they coherent across related questions?

RQ3

Efficiency attribution

How much of a speed or cost gain comes from the readout, and how much from model size, batching, prefix sharing, caching, pruning or the network?

RQ4

Selective control

Which decisions can be delegated, and when should software escalate, abstain or let code compute — judged by end outcome and total cost?

RQ5

Robustness & security

How do option names, order, definitions, missing options, language, injected text and poisoned checkpoints change decisions?

RQ6

Openness & reproducibility

What do alternatives release — code, weights, data, predictions — and how much of the literature survives a common protocol?

The literature arrived within weeks of launch

First-submission dates of the core studies on arXiv, coloured by their primary relationship to Jev.

Evaluates hosted JevBuilds an independent modelUses Jev inside a system15 Sep: Jev released
    15Sep
      16
        17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          1Oct
          2
          3
          4
          5
          6

          04 Findings

          Ten findings, each with its counter-evidence

          Every finding is a synthesis across studies. The highlighted number comes from one of them; the lists name the evidence that supports or limits the claim. Values are as reported by their sources.

          F1 Valid ≠ correct · Decision contract

          A type-valid answer can still be the wrong answer.

          99% → 7%

          right when the answer is on the menu; correct rejections when it is missing, an explicit “None” notwithstanding

          Rejection Bottleneck

          Typed interfaces remove parse and schema failures by construction. They do not ensure that the model reads the options the way the author meant: short labels can outweigh written definitions, rebinding names to rubrics can reverse a ranking, a middle option can absorb errors, and when the right answer is missing the model usually picks a wrong option instead of the exit. Order effects can change many individual answers while aggregate accuracy stays flat. The type-error rate stays at zero throughout.

          Supported by

          • Hosted Jev AUROC .8146 → .5806 after name–rubric swap; 0% type errorsAuthor-reported experiment
          • The label wins over the definition — and the cause is one rendering lineAuthor-reported experiment
          • ANLI: Neutral absorbs 51% of errors, at every positionAuthor-reported experiment
          More supporting evidence
          • Hosted Jev answers 63% of evidence-removed items and loses track after three stepsAuthor-reported experiment
          • ABSTAIN catches invalid menus but costs valid answers; order flips 46 of 120 picksAuthor-reported experiment
          • Swap the two arms and a third of answers keep their directionAuthor-reported experiment
          • Order invariance by construction; sequential heads still flip 2–3%Author-reported experiment
          • Typed outputs, hijacked inputs: up to 100% injection successAuthor-reported experiment
          • Missing answer noticed 24.8% of the time (Jev) — Laya over-rejectsAuthor-reported experiment
          • Rejection thresholds do not travel: 69% false rejection after transferAuthor-reported experiment
          • More candidates: Laya rejects half of covered queries, Jev misses a quarter of out-of-scope onesAuthor-reported experiment
          • Definitions move 28% of answers — without reliably improving themAuthor-reported experiment
          • Hosted beats open on 9 of 11 harness decisions; routing at chance for bothAuthor-reported experiment
          • Evaluates what is described; does not simulate what is notAuthor-reported experiment
          • Isolate candidates: order flips 18.2% → 1.2%Author-reported experiment
          • Order flips on a third of 20-option items; rotation averaging halves themAuthor-reported experiment
          • Threshold for 5% risk still admits 31% of out-of-scope requestsAuthor-reported experiment
          • Yes/no swap flips half of the hosted model’s answersAuthor-reported experiment
          • Ordinal scales compressed: decisions use 67–76% of gold supportAuthor-reported experiment
          • Cacheable scoring loses rule sensitivity (recall@1 1.00 → 0.24)Author-reported experiment
          • Persona shifts reproduce 87% (English) vs 62% (Arabic) of a human differenceAuthor-reported experiment
          • Jev’s argument scores move with the stated sourceAuthor-reported experiment
          • Picks “I don’t know” on 10.5% of unanswerable questionsAuthor-reported experiment
          • Option rotation changes 37% of answers; aggregate accuracy hides itAuthor-reported experiment
          • Natural context flips 61% of correct decisions (vs 2% neutral)Author-reported experiment
          • 23/30 anchors correct in all 12 responses; 5 consistently wrongAuthor-reported experiment
          • Test–retest floor ≤1.33% answer flips; swap = 24× the floorAuthor-reported experiment
          • Open marker head: AUROC .9376 → .2315 (inversion), 76.92% flipsAuthor-reported experiment
          • Type-error rate 0% in every condition, by constructionAuthor-reported experiment
          • Multi-way: neutral renaming flips 52.42%; accuracy .5637 → .2782Author-reported experiment
          • Reliability depends on action-set size and near-valid alternativesAuthor-reported experiment
          • 7 wrong selections changed counts but not the final labelAuthor-reported experiment
          • Abstain removed: acc 0.95 → 0; Noul vs Choice differ by 0.125jujumilk3/jev-calibration-audit ↗Community report (repository)
          • ECE 0.0204 (CLINC150) vs 0.0936 (Banking77); 0 type errorsjourdanlabs/assay-001 ↗Community report (repository)
          • Noul vs Choice framing reverses the comparison with GPT-4.1jjd-lab/jev-synthetic-survey ↗Community report (repository)
          • Rule checks on agent edits: 10 of 39 flags confirmedcoldteadotai/abide ↗Community report (repository)
          • More rows per request, fewer correct: 100% → 77–94%realZachi/pg-jev ↗Community report (repository)
          • Vendor-listed failure modes for jev-1.13TypeSafe: Jev 1.13 jaggedness ↗Vendor documentation

          Limited or qualified by

          • Random-string names return flips to the neutral regime (2–12%)Author-reported experiment
          • No type errors by construction; plotted 0% is analyticalTypeSafe: Introducing System One Models & Jev ↗Vendor documentation
          • Open recipe holds up; extra terms buy robustness, not accuracyAuthor-reported experiment
          • Open contrastive decision model judges near chance, but ignores answer orderAuthor-reported experiment

          F2 Measurable ≠ calibrated · Probability & calibration

          A native probability makes calibration measurable, not guaranteed.

          +0.21–0.33

          confidence above accuracy on news past the model’s knowledge boundary, even after recalibration

          Beyond Answer Confidence

          Every call returns a distribution, so reliability can be audited on each run. Measured calibration varies by task, construct and language; probabilities can rank well yet sit badly against a fixed threshold; temperatures fitted on a few options transfer poorly to many; and confidence stays high where the model lacks the knowledge. Thresholds and recalibration fitted on separate data help, but only for the workload they were fitted on.

          Supported by

          • Median ECE 0.157 vs 0.066 for the best (verbalised) LLMAuthor-reported experiment
          • Binary probabilities rank well; fixed 0.5 threshold does notAuthor-reported experiment
          • Threshold for 5% risk still admits 31% of out-of-scope requestsAuthor-reported experiment
          More supporting evidence
          • Calibrated confidence prices query plans; overconfidence hides output errorsAuthor-reported experiment
          • Jev confidence supports selective review; Laya’s barely ranks errorsAuthor-reported experiment
          • Looping a small decision model: +13.5 points at 7.7× computeAuthor-reported experiment
          • Extra passes buy calibration, not accuracy; escalate and curate insteadAuthor-reported experiment
          • Calibrated after one temperature, yet the gate escalates 92–100%Author-reported experiment
          • Registered test passed: ECE 0.017, macro-F1 0.92 against 0.87 for a 31B modelAuthor-reported experiment
          • One calibrated pass against a relaxation: as good or better at a thirtieth of the costAuthor-reported experiment
          • Registered prospective test: stable fraction forecast within 2.1 pointsAuthor-reported experiment
          • Better calibrated than a commercial LLM, not than an open oneAuthor-reported experiment
          • Task-specialised open model beats Jev on rewriting, trails on routingAuthor-reported experiment
          • Right mode, wrong mass: 89% placed where the truth is 36%Author-reported experiment
          • Rejection thresholds do not travel: 69% false rejection after transferAuthor-reported experiment
          • Open RLCD recipe trails plain cross-entropy by 2.5–3.0 pointsAuthor-reported experiment
          • Turkish typed decisions: Jev fourth of 16 at a tenth of the leader’s timeAuthor-reported experiment
          • Ask the model a targeted question instead of reading its confidenceAuthor-reported experiment
          • Same backbone, decision-native output: calibration error down up to 10×Author-reported experiment
          • Threshold from development data: rejection 7% → 79%Author-reported experiment
          • Open RLCD: best selective prediction among SFT, STaR, GRPOAuthor-reported experiment
          • Ahead of a 27B open LLM on 27/37 datasets; pooled Choice ECE 0.028Author-reported experiment
          • Language-specific open encoder ≈ hosted Jev in Chinese, better calibratedAuthor-reported experiment
          • Known-probability test: Noul 0.92, Score 0.89, Choice 0.76Author-reported experiment
          • SemEval ABSA: best regression error with 488 fitted coefficientsAuthor-reported experiment
          • Picks “I don’t know” on 10.5% of unanswerable questionsAuthor-reported experiment
          • Open Laya checkpoint under-confident (gap −0.214); one temperature fixes itAuthor-reported experiment
          • Confidence = chance-normalised max probability; pooled ECE 0.034Author-reported experiment
          • Ranks well, thresholds badly: median ECE 0.168 vs null 0.074Author-reported experiment
          • Raw ECE 0.12; isotonic recalibration → 0.008 on holdoutAuthor-reported experiment
          • Shared discrepancy: 93% / 96% coverage vs 29% / 36% sampling-onlyAuthor-reported experiment
          • Confidence ≥ 0.9: tied with GPT-6 (94.7% vs 94.4%)Author-reported experiment
          • Empathy: confident near chance; task ECE 0.538Author-reported experiment
          • Recalibration on held-out labels cut calibration error ~3.3×Author-reported experiment
          • Two-decimal probabilities set a calibration-error floorAuthor-reported experiment
          • JevLite ensemble AUROC .974, ECE .052; no false alarmsAuthor-reported experiment
          • Abstain removed: acc 0.95 → 0; Noul vs Choice differ by 0.125jujumilk3/jev-calibration-audit ↗Community report (repository)
          • ECE 0.0204 (CLINC150) vs 0.0936 (Banking77); 0 type errorsjourdanlabs/assay-001 ↗Community report (repository)
          • Spanish state: −3.0 to −6.4 accuracy points on 4 datasetsmarcosmartinez/jev-acento ↗Community report (repository)
          • Unknowable-rule task: temperature ≈3.4 needed; 0.01 quantisationscienthoon/jev-ood-calibration ↗Community report (repository)
          • Frozen Qwen3-8B: order flips 0.230 → 0.073; auto-decidable 7.7% → 52%nokia-applied-research/AnyJev ↗Community report (repository)
          • Overconfident where 100 annotators disagree (0.81 vs 0.47)GautamTalksDev/jevbench ↗Community report (repository)
          • Open 9B readout edges Jev on accuracy and calibration in the author’s benchmarkuspraveen/Jevify ↗Community report (repository)
          • Calibration is a group property, not a per-answer guaranteeTypeSafe: System One ↗Vendor documentation

          Limited or qualified by

          • Confidence = a statistic of the returned distributionTypeSafe: Confidence ↗Vendor documentation
          • Safe automation coverage: 7.6% (symmetric) → 47% (separate thresholds)Author-reported experiment
          • Independent boards: top tier on yes/no, second on intents, no model beats guessing on helpfulnessJevals/jevals-data ↗Community report (repository)

          F3 First pass, then escalate · Selective control

          The most consistent use is a bounded first pass with confidence-gated escalation.

          41%

          of GPT-6’s fee for a frozen accept-or-escalate cascade that beat GPT-6 by 0.9 points

          JEV-as-a-Judge

          Across judging, annotation, moderation, intrusion detection, infrastructure triage and agents, the typed model is usually cheaper and faster but not the most accurate component. Gains appear when uncertain cases go to a stronger model or a person and thresholds are frozen on separate data; the strongest designs fix them in advance and test on new workloads. Savings shrink once pre-screen costs, held-out misses and cheap trained classifiers are counted, so those belong in the comparison.

          Supported by

          • Pre-registered gate: a third of SCADA reviews offloaded, accuracy keptAuthor-reported experiment
          • Pre-specified live test: matched GPT-6 exactly, 74% escalatedAuthor-reported experiment
          • Low-confidence routing matches the LLM at ¼–½ of its costAuthor-reported experiment
          More supporting evidence
          • Laya coordinates agents with 78–94% fewer GPT tokens; random choices trail by 1.8 pointsAuthor-reported experiment
          • Jev confidence supports selective review; Laya’s barely ranks errorsAuthor-reported experiment
          • Extra passes buy calibration, not accuracy; escalate and curate insteadAuthor-reported experiment
          • One calibrated pass against a relaxation: as good or better at a thirtieth of the costAuthor-reported experiment
          • Search agent: 3.7–4.7× faster, accuracy 45% → 54%Author-reported experiment
          • As a guard: ROC-AUC 0.95–1.0 on eight detection benchmarksAuthor-reported experiment
          • Missing answer noticed 24.8% of the time (Jev) — Laya over-rejectsAuthor-reported experiment
          • Within 1.6 macro-F1 of the best LLM at ~97% lower costAuthor-reported experiment
          • No labels needed: beats a supervised classifier on unseen subtypesAuthor-reported experiment
          • Semantic SQL: pruning + typed decisions finish 540K-pair joinsAuthor-reported experiment
          • Intrusion pilot: F1 0.859, 4.8× faster than an LLMAuthor-reported experiment
          • Ask the model a targeted question instead of reading its confidenceAuthor-reported experiment
          • Classifier first, Jev on escalation: same accuracy at 0.43 of the costAuthor-reported experiment
          • Population-scale reading audited by blind coders: sensitivity 0.999Author-reported experiment
          • Threshold from development data: rejection 7% → 79%Author-reported experiment
          • Binary probabilities rank well; fixed 0.5 threshold does notAuthor-reported experiment
          • Memory agent: LoCoMo 91.7% with Jev judging evidence (AUC 0.94)Author-reported experiment
          • SemEval ABSA: best regression error with 488 fitted coefficientsAuthor-reported experiment
          • CWE triage: Top-1 0.46 vs 0.50 (frontier) at 1/56 of the costAuthor-reported experiment
          • Trace security: F1 77.8 vs 74.1 for the best LLM judgeAuthor-reported experiment
          • Medicine: parity on abstracts, −22 points on diagnostic casesAuthor-reported experiment
          • Zero-shot Jev 0.67 on rule-defined classes vs 0.90–0.92 for specialistsAuthor-reported experiment
          • Rescoring: WER 7.5% vs 7.8% for 7B LLMs, no GPU neededAuthor-reported experiment
          • Safe automation coverage: 7.6% (symmetric) → 47% (separate thresholds)Author-reported experiment
          • Learned “when to ask” policy ≈ oracle (AUC 0.80 vs 0.80)Author-reported experiment
          • AndroidWorld 79% (vs 84% step-wise) at 73% lower API costAuthor-reported experiment
          • Parity on most rubric criteria at 1/16–1/325 of the costAuthor-reported experiment
          • Zero-shot failure detection: median AUROC 0.886 at 1/63 of judge costAuthor-reported experiment
          • Accuracy 77.38% vs 78.96–83.21% for hosted LLMsAuthor-reported experiment
          • Kendall τ 0.573 / 0.398 with expert error counts; beats open NLIAuthor-reported experiment
          • 1.7% flagship anchors cut effect error 41%; decision gain 0.27 → 0.39Author-reported experiment
          • Passes content gates in 15/15 new countries; fails intervention gatesAuthor-reported experiment
          • Jev 0.422 s median; $0.15 of $3.71 per gameAuthor-reported experiment
          • Within 3 points of GPT-6 where the verdict can be read off the textAuthor-reported experiment
          • Confidence ≥ 0.9: tied with GPT-6 (94.7% vs 94.4%)Author-reported experiment
          • 95% success with 1.12 strong calls/task vs 88% and 4.10 (−72.7%)Author-reported experiment
          • Full semantic correctness; median 0.335 s, p95 0.442 sAuthor-reported experiment
          • Behind best LLM on 14/15 tasks (−11.6 F1) at 44× lower costAuthor-reported experiment
          • F1 0.908 vs 2,416 human judgments; one frontier LLM +0.059Author-reported experiment
          • Fees per correct decision −60% to −81% on four-field contractsAuthor-reported experiment
          • Frozen Qwen3-8B: order flips 0.230 → 0.073; auto-decidable 7.7% → 52%nokia-applied-research/AnyJev ↗Community report (repository)
          • Reranking: ties Cohere Rerank 4 Pro at half the latencyanessbelbati/jev-rerank-bench ↗Community report (repository)
          • Zero-shot spam: matches a trained baseline; catches more new phishingbitnovus/jev-spam-eval ↗Community report (repository)
          • Blind security run: 96.5% on prompt injection, 89% on code pairsGaurav-Gosain/jev-sec-bench ↗Community report (repository)
          • Independent boards: top tier on yes/no, second on intents, no model beats guessing on helpfulnessJevals/jevals-data ↗Community report (repository)
          • 10,000 comments: 4× faster than a chat model, five points less agreementNanmiCoder/jev-arena ↗Community report (repository)
          • Grading Jev with Jev’s own labels inflates its gainzhuyansen/jev-search-rerank-eval ↗Community report (repository)
          • Vendor-listed failure modes for jev-1.13TypeSafe: Jev 1.13 jaggedness ↗Vendor documentation
          • Vendor dashboard: 67.8% agreement at $0.0004 and 0.4 s per caseTypeSafe: Workflow evals ↗Vendor-reported result

          Limited or qualified by

          • BFCL / τ-style: limited edge over a cheap generative cascadeAuthor-reported experiment
          • Reliability depends on action-set size and near-valid alternativesAuthor-reported experiment
          • Correlation with MOS 0.879 vs P.1204.1 0.898Author-reported experiment
          • Judges share Jev’s confident errors (96%); cascades gain ≤2.7 pointsAuthor-reported experiment
          • An appended opinion flips 12% and drains 38% of confident answers below the gateAuthor-reported experiment
          • Packet numbers in, 28% accuracy out — trees trained on 8k records reach 70%Author-reported experiment
          • Self-audit: a 23.9% saving was really 4.3%Author-reported experiment
          • Same accuracy, no cost advantage at batch pricesAuthor-reported experiment
          • Rule checks on agent edits: 10 of 39 flags confirmedcoldteadotai/abide ↗Community report (repository)
          • A week as a skill router: 28 of 539 suggestions used, then removedshimo4228/jev-skill-router ↗Community report (repository)
          • Calibrated after one temperature, yet the gate escalates 92–100%Author-reported experiment
          • Swap the two arms and a third of answers keep their directionAuthor-reported experiment
          • Agent-built artifacts: 94–97% of Jev’s quality at 2–25% of its projected costAuthor-reported experiment

          F4 Fast is not one number · Downstream use

          Speed and cost claims decompose into different mechanisms and scopes.

          1.2–3.3×

          the input tokens billed per decision against a batch-priced LLM, leaving no cost advantage

          Political Science Replications

          Single-request latency, per-question amortized time, batch throughput, end-to-end task time, deadline attainment under load, simulated timing and API bills are different quantities. Prefix sharing, batching, caching and pruning can explain as much as the readout does; billing every option as input can erase the price advantage against batch-priced LLMs; and a faster step can make a slower agent. None of the figures can be pooled into a single leaderboard.

          Supported by

          • With caching, latencies converge: the gain is in fresh decisionsAuthor-reported experiment
          • Hosted Jev inside the agent: better on 3 of 8 tasks, 1.4–2.1× slowerAuthor-reported experiment
          • Self-audit: a 23.9% saving was really 4.3%Author-reported experiment
          More supporting evidence
          • Agent-built artifacts: 94–97% of Jev’s quality at 2–25% of its projected costAuthor-reported experiment
          • Laya coordinates agents with 78–94% fewer GPT tokens; random choices trail by 1.8 pointsAuthor-reported experiment
          • Wireless control: 3.5–8.5× faster than LLMs; quality depends on the taskAuthor-reported experiment
          • Faster steps, slower agent: success 77.6% → 66.1%Author-reported experiment
          • Within 1.6 macro-F1 of the best LLM at ~97% lower costAuthor-reported experiment
          • Semantic SQL: pruning + typed decisions finish 540K-pair joinsAuthor-reported experiment
          • Intrusion pilot: F1 0.859, 4.8× faster than an LLMAuthor-reported experiment
          • 0.75 s vs 6.0 s per request; accuracy difference not establishedAuthor-reported experiment
          • Classifier first, Jev on escalation: same accuracy at 0.43 of the costAuthor-reported experiment
          • Strong reranking quality; latency between local models and pointwise LLMsAuthor-reported experiment
          • Open encoder–decoder: 78.05% on DecisionBench, 32 questions in 221 msAuthor-reported experiment
          • Single-token LLM classifier: 85.5% at 0.53 s, flat across label spacesAuthor-reported experiment
          • Navigation: 27% SR at 0.65 s/step; components, not just Jev, carry the gainAuthor-reported experiment
          • CWE triage: Top-1 0.46 vs 0.50 (frontier) at 1/56 of the costAuthor-reported experiment
          • Trace security: F1 77.8 vs 74.1 for the best LLM judgeAuthor-reported experiment
          • One Jev selection call ≈ LLM-extraction memory at 1/3,061 the write costAuthor-reported experiment
          • Hosted Jev never meets 100 ms near-RT deadlines (warm p50 281 ms)Author-reported experiment
          • Rescoring: WER 7.5% vs 7.8% for 7B LLMs, no GPU neededAuthor-reported experiment
          • Parity on most rubric criteria at 1/16–1/325 of the costAuthor-reported experiment
          • Zero-shot failure detection: median AUROC 0.886 at 1/63 of judge costAuthor-reported experiment
          • Emulated router saves 13–21% of coding-agent spendAuthor-reported experiment
          • $0.000228 and 1.24 s per contract — lowest of ten modelsAuthor-reported experiment
          • Under $0.03 per 100 report pairs (judgment calls only)Author-reported experiment
          • $0.0227 per 1,000 predictions; median 213 msAuthor-reported experiment
          • Jev 0.422 s median; $0.15 of $3.71 per gameAuthor-reported experiment
          • 0.36% of GPT-6’s fee at 0.15 s medianAuthor-reported experiment
          • Frozen cascade: +0.9 points over GPT-6 at 41% of its feeAuthor-reported experiment
          • 8.9× vs serial, 3.4× vs no-reuse batch; 5.7 ms/question amortizedAuthor-reported experiment
          • Typed head: no consistent gain over LM-head readoutAuthor-reported experiment
          • Full semantic correctness; median 0.335 s, p95 0.442 sAuthor-reported experiment
          • 499,500 narratives screened for $25.23; 195,857 fully codedAuthor-reported experiment
          • 64.5 ms/decision; 4.9× faster than same backbone generatingAuthor-reported experiment
          • 30.9 ms per decision on one laptop GPU; 32 decisions/sAuthor-reported experiment
          • $0.000217 electricity per suite pass vs $10.636 for a hosted modelAuthor-reported experiment
          • Meets the 1 s RIC budget on 99.8% of calls (LLMs: 17.9%, 0%)Author-reported experiment
          • Under load, slow interpreters saturate — a small local JSON model keeps up tooAuthor-reported experiment
          • Median decision latency −22.7% to −64.5% vs the fastest LLMAuthor-reported experiment
          • Fees per correct decision −60% to −81% on four-field contractsAuthor-reported experiment
          • Live admission: 0.91–0.95 exact and on time where LLMs fall below 0.1Author-reported experiment
          • Local 4B readout: 76% on public JevBench at 48 msYinsongxu/LLM2Jev ↗Community report (repository)
          • Reranking: ties Cohere Rerank 4 Pro at half the latencyanessbelbati/jev-rerank-bench ↗Community report (repository)
          • 10,000 comments: 4× faster than a chat model, five points less agreementNanmiCoder/jev-arena ↗Community report (repository)
          • More rows per request, fewer correct: 100% → 77–94%realZachi/pg-jev ↗Community report (repository)
          • Routing subagents with Jev: cheaper than one baseline, dearer than anothersuenot/codex-jev-router ↗Community report (repository)
          • Vendor dashboard: 67.8% agreement at $0.0004 and 0.4 s per caseTypeSafe: Workflow evals ↗Vendor-reported result

          Limited or qualified by

          F5 Compatible ≠ equivalent · Inference & readout

          Interface compatibility is not mechanism equivalence.

          9 of 11

          agent-harness decision points where hosted Jev beat an open Jev-like model on identical inputs

          Fast Models, Slow Evidence

          Open projects reproduce the request and response shape with encoders, encoder–decoders, frozen or fine-tuned decoders, diffusion reads and task-specialised models, and a frozen instruction-tuned LLM read through option identifiers can match community Jev-style models built on the same backbone. Their probabilities come from different mechanisms, they differ in what is released and in how reproducible they are, and none establishes the architecture of the commercial model. Rankings between hosted and open models change from task to task.

          Supported by

          • A frozen 4B LLM already matches community Jev-style modelsAuthor-reported experiment
          • Hosted answers drift day to day (0.5%); open model reproduces exactlyAuthor-reported experiment
          • Native typed-decision head: 86.2% vs Jev 85.0% on JevBenchAuthor-reported experiment
          More supporting evidence
          • Precedents help Jev pick the right rule (+5 to +22 points); Laya barely movesAuthor-reported experiment
          • Jev confidence supports selective review; Laya’s barely ranks errorsAuthor-reported experiment
          • Looping a small decision model: +13.5 points at 7.7× computeAuthor-reported experiment
          • Menu changes are predictable offline for decision models, not for generative LLMsAuthor-reported experiment
          • Open contrastive decision model judges near chance, but ignores answer orderAuthor-reported experiment
          • Registered test passed: ECE 0.017, macro-F1 0.92 against 0.87 for a 31B modelAuthor-reported experiment
          • Order invariance by construction; sequential heads still flip 2–3%Author-reported experiment
          • Task-specialised open model beats Jev on rewriting, trails on routingAuthor-reported experiment
          • More candidates: Laya rejects half of covered queries, Jev misses a quarter of out-of-scope onesAuthor-reported experiment
          • Security-specialised 0.8B beats general 9B by 20.5 pointsAuthor-reported experiment
          • The label wins over the definition — and the cause is one rendering lineAuthor-reported experiment
          • Isolate candidates: order flips 18.2% → 1.2%Author-reported experiment
          • Order flips on a third of 20-option items; rotation averaging halves themAuthor-reported experiment
          • Same backbone, decision-native output: calibration error down up to 10×Author-reported experiment
          • Open encoder–decoder: 78.05% on DecisionBench, 32 questions in 221 msAuthor-reported experiment
          • Open RLCD: best selective prediction among SFT, STaR, GRPOAuthor-reported experiment
          • Cacheable scoring loses rule sensitivity (recall@1 1.00 → 0.24)Author-reported experiment
          • Ahead of a 27B open LLM on 27/37 datasets; pooled Choice ECE 0.028Author-reported experiment
          • Language-specific open encoder ≈ hosted Jev in Chinese, better calibratedAuthor-reported experiment
          • Single-token LLM classifier: 85.5% at 0.53 s, flat across label spacesAuthor-reported experiment
          • Open recipe holds up; extra terms buy robustness, not accuracyAuthor-reported experiment
          • Typed-question features route LLMs: 72.1% vs 69.2% best single modelAuthor-reported experiment
          • Local likelihood readout 85.1% vs Jev 89.1% on MMLUAuthor-reported experiment
          • Open Laya checkpoint under-confident (gap −0.214); one temperature fixes itAuthor-reported experiment
          • Adapted open guard: AUC 0.83 → 0.997, with an OOD “alignment tax”Author-reported experiment
          • Open Jev-like models: Laya AUROC 0.53 vs Jev 0.96 on R-JudgeAuthor-reported experiment
          • Learned “when to ask” policy ≈ oracle (AUC 0.80 vs 0.80)Author-reported experiment
          • Typed readout and generation gain equally after adaptationAuthor-reported experiment
          • Open marker head: AUROC .9376 → .2315 (inversion), 76.92% flipsAuthor-reported experiment
          • Typed head: no consistent gain over LM-head readoutAuthor-reported experiment
          • JevLite ensemble AUROC .974, ECE .052; no false alarmsAuthor-reported experiment
          • 64.5 ms/decision; 4.9× faster than same backbone generatingAuthor-reported experiment
          • 68 questions: 0.941 (Brier .042) vs Jev 0.765 (.133); gap = 12 itemsAuthor-reported experiment
          • 2,250 questions: 0.844 vs Jev 0.803 — on the open model’s training shapesAuthor-reported experiment
          • Under load, slow interpreters saturate — a small local JSON model keeps up tooAuthor-reported experiment
          • Local 4B readout: 76% on public JevBench at 48 msYinsongxu/LLM2Jev ↗Community report (repository)
          • Frozen Qwen3-8B: order flips 0.230 → 0.073; auto-decidable 7.7% → 52%nokia-applied-research/AnyJev ↗Community report (repository)
          • Browser tasks: specialised open model 38.5% vs Jev 16.7%; single steps alikelexmount/WebJev ↗Community report (repository)
          • Open 9B readout edges Jev on accuracy and calibration in the author’s benchmarkuspraveen/Jevify ↗Community report (repository)
          • Same weights for all accounts; architecture undisclosedTypeSafe: Models (Jev 1.13) ↗Vendor documentation
          • Wire-compatible, not a logit read: probabilities are generatedgithubnext/localjev ↗Code/README inspection
          • Reproduces the interface pattern, not Jev’s model or trainingTheoLeeCJ/SemIf ↗Code/README inspection
          • Architecture follows third-party speculation, not a disclosurejaredpalmer/kev ↗Code/README inspection

          Limited or qualified by

          • Text-only input; English is the primary training languageTypeSafe: Models (Jev 1.13) ↗Vendor documentation
          • Same evidence, different readout: +18 AP on one dataset, none on the otherAuthor-reported experiment
          • Open RLCD recipe trails plain cross-entropy by 2.5–3.0 pointsAuthor-reported experiment

          F6 Attribute the gain · Downstream use

          System-level gains need module-level attribution and the right reference.

          31% → 87%

          ALFWorld games solved once code supplies the lookahead the model cannot simulate

          Code Owns the Simulation

          Memory systems, query engines, search agents, orchestration pipelines and scientific workflows improve for many reasons at once. Attribution requires ablations, matched baselines and references that measure what the workflow reuses. When code owns pruning, lookahead or simulation, the typed model can serve as a strong evaluator; when one call must both simulate and evaluate, or step errors compound over a long trajectory, the system gets worse even as each step gets faster.

          Supported by

          • Faster steps, slower agent: success 77.6% → 66.1%Author-reported experiment
          • Semantic SQL: pruning + typed decisions finish 540K-pair joinsAuthor-reported experiment
          • Memory agent: LoCoMo 91.7% with Jev judging evidence (AUC 0.94)Author-reported experiment
          More supporting evidence
          • Laya coordinates agents with 78–94% fewer GPT tokens; random choices trail by 1.8 pointsAuthor-reported experiment
          • Calibrated confidence prices query plans; overconfidence hides output errorsAuthor-reported experiment
          • Precedents help Jev pick the right rule (+5 to +22 points); Laya barely movesAuthor-reported experiment
          • Node-local candidates: 113/120 correct decisions against 90/120Author-reported experiment
          • Search agent: 3.7–4.7× faster, accuracy 45% → 54%Author-reported experiment
          • Pre-registered gate: a third of SCADA reviews offloaded, accuracy keptAuthor-reported experiment
          • Agent with spawned finite actions: Maze 0.96 vs 0.52Author-reported experiment
          • Hosted Jev inside the agent: better on 3 of 8 tasks, 1.4–2.1× slowerAuthor-reported experiment
          • Population-scale reading audited by blind coders: sensitivity 0.999Author-reported experiment
          • Navigation: 27% SR at 0.65 s/step; components, not just Jev, carry the gainAuthor-reported experiment
          • Typed-question features route LLMs: 72.1% vs 69.2% best single modelAuthor-reported experiment
          • One Jev selection call ≈ LLM-extraction memory at 1/3,061 the write costAuthor-reported experiment
          • Jev-routed LoRA composition: +0.2 to +1.2 pointsAuthor-reported experiment
          • AndroidWorld 79% (vs 84% step-wise) at 73% lower API costAuthor-reported experiment
          • Emulated router saves 13–21% of coding-agent spendAuthor-reported experiment
          • 1.7% flagship anchors cut effect error 41%; decision gain 0.27 → 0.39Author-reported experiment
          • 4/4 full-game wins incl. two vs Lv7 with GPT-6 plans; Jev-only never expandedAuthor-reported experiment
          • Enemy elimination 16.5% → 37.7%; wins 3 → 7 of 105Author-reported experiment
          • Macro accuracy 0.706 → 0.761 after post-training (seen families)Author-reported experiment
          • 7 wrong selections changed counts but not the final labelAuthor-reported experiment
          • PLCC vs VMAF: 0.737 (metadata) → 0.824 (+bitstream, pixel)Author-reported experiment
          • Coded fields understate narrative fidelity by 0.26 kappaAuthor-reported experiment
          • LoCoMo judge 0.777 (+11.0%); build 158 s; 0.93 s/queryAuthor-reported experiment
          • Radio conditions outweigh the interpreter; no LLM gain over JevAuthor-reported experiment
          • Live admission: 0.91–0.95 exact and on time where LLMs fall below 0.1Author-reported experiment
          • A week as a skill router: 28 of 539 suggestions used, then removedshimo4228/jev-skill-router ↗Community report (repository)
          • Browser tasks: specialised open model 38.5% vs Jev 16.7%; single steps alikelexmount/WebJev ↗Community report (repository)
          • Routing subagents with Jev: cheaper than one baseline, dearer than anothersuenot/codex-jev-router ↗Community report (repository)
          • Chess: random from the raw board, competent once code supplies the factswondertwins/jev-benchmark ↗Community report (repository)

          F7 Young, clustered evidence · Downstream use

          The evidence base is young, clustered and unreplicated.

          23.9% → 4.3%

          a reported saving after the authors audited their own analysis pipeline

          Fast Models, Slow Evidence

          The core preprints appeared within weeks of the launch, mostly as a single version, and several were substantially revised soon after. Several author groups contribute more than one study, some benchmarks are built by teams that also rank their own models, many community results are self-reported, and one study’s audit of its own pipeline corrected its headline deployment numbers. Hosted answers drift from day to day. None of the core results has been independently reproduced.

          Supported by

          • Hosted answers drift day to day (0.5%); open model reproduces exactlyAuthor-reported experiment
          • Identical re-run changes the Score value on 38.5% of Score questionsAuthor-reported experiment
          • Turkish typed decisions: Jev fourth of 16 at a tenth of the leader’s timeAuthor-reported experiment
          More supporting evidence
          • Ecosystem: judgment 77%, scoring 52%, action selection 31% of projectsAuthor-reported experiment
          • One run each: 5 min faster, different severities — not a controlled testAuthor-reported experiment
          • Test–retest floor ≤1.33% answer flips; swap = 24× the floorAuthor-reported experiment
          • Self-reported 81.65% on public JevBench items at 7.08 msAuthor-reported experiment
          • “0% bypass” with a 95% interval up to 27.8%Author-reported experiment
          • 68 questions: 0.941 (Brier .042) vs Jev 0.765 (.133); gap = 12 itemsAuthor-reported experiment
          • 2,250 questions: 0.844 vs Jev 0.803 — on the open model’s training shapesAuthor-reported experiment
          • Unknowable-rule task: temperature ≈3.4 needed; 0.01 quantisationscienthoon/jev-ood-calibration ↗Community report (repository)
          • Grading Jev with Jev’s own labels inflates its gainzhuyansen/jev-search-rerank-eval ↗Community report (repository)
          • 193.6× faster, 444.6× cheaper on vendor workflow evalsTypeSafe: Introducing System One Models & Jev ↗Vendor-reported result

          Limited or qualified by

          F8 Calibrated ≠ coherent · Probability & calibration

          A calibrated probability is not a coherent belief.

          89% vs 36%

          probability placed on the most likely option, against its true chance

          JEVal and InnerJev

          Probabilities from the same model can break basic rules across related questions (a statement and its negation, single labels and their union), change the decision when the same question is asked through another interface, and pile mass on the most likely option far beyond its true chance. Average calibration certifies none of this. Check coherence across complementary questions and interfaces before treating outputs as beliefs, and prefer uses that only need the ranking.

          Supported by

          • P(X) + P(not X) misses 1 by 0.064 (open LLM readout: 0.293)Author-reported experiment
          • Two interfaces, same question: different decision on 32.8% of pairsAuthor-reported experiment
          • Confidence ≠ knowing: +0.21–0.33 overconfidence past the knowledge boundaryAuthor-reported experiment
          More supporting evidence
          • Menu changes are predictable offline for decision models, not for generative LLMsAuthor-reported experiment
          • Known-probability test: Noul 0.92, Score 0.89, Choice 0.76Author-reported experiment
          • Staged decisions disagree with direct ones; Jev loses 22.9 points on CLINC150Author-reported experiment
          • Three single-label probabilities sum to 1.14Author-reported experiment
          • Identical re-run changes the Score value on 38.5% of Score questionsAuthor-reported experiment
          • Overconfident where 100 annotators disagree (0.81 vs 0.47)GautamTalksDev/jevbench ↗Community report (repository)
          • Open 9B readout edges Jev on accuracy and calibration in the author’s benchmarkuspraveen/Jevify ↗Community report (repository)

          F9 Reads, doesn’t compute · Inference & readout

          Typed decisions read what the input states; they do not reliably compute what it implies.

          97% vs 44%

          answers within one of an exact number, against answers exactly right

          JEVal and InnerJev

          Studies of arithmetic menus, exact numbers, graph structure, packet and sensor values, games and planning meet the same boundary: answers are strong when the needed fact is written in the state and weak when it must be counted, calculated or simulated. The vendor’s own guidance is to count, convert and compare in code. Letting code compute or simulate and asking the model to evaluate the result works better. Calculation-heavy benchmark items can still look easy when they may have been seen in training.

          Supported by

          • Present 99%, rejected when absent 7% — though Boolean checks get 99%Author-reported experiment
          • Evaluates what is described; does not simulate what is notAuthor-reported experiment
          • Reads every edge; counts degree right 68% of the timeAuthor-reported experiment
          More supporting evidence
          • Hosted Jev answers 63% of evidence-removed items and loses track after three stepsAuthor-reported experiment
          • Wireless control: 3.5–8.5× faster than LLMs; quality depends on the taskAuthor-reported experiment
          • Let code simulate: ALFWorld solved 31% → 87%Author-reported experiment
          • Packet numbers in, 28% accuracy out — trees trained on 8k records reach 70%Author-reported experiment
          • Sensor activity recognition from text: macro-F1 ≤ 0.12Author-reported experiment
          • Medicine: parity on abstracts, −22 points on diagnostic casesAuthor-reported experiment
          • Zero-shot Jev 0.67 on rule-defined classes vs 0.90–0.92 for specialistsAuthor-reported experiment
          • Range refinement: 83.4% within 5% vs 80.5% direct choiceAuthor-reported experiment
          • Within 3 points of GPT-6 where the verdict can be read off the textAuthor-reported experiment
          • Chess: random from the raw board, competent once code supplies the factswondertwins/jev-benchmark ↗Community report (repository)

          Limited or qualified by

          • Calculation-heavy MMLU easier for Jev (94% vs 91%) — memorisation not ruled outAuthor-reported experiment

          F10 Typed ≠ secure · Decision contract

          A typed output does not make a decision safe from its inputs.

          100%

          success of injections that slip false facts into the state, on one decision task

          Hidden Risks of Jev

          Because the answer depends on the state, text placed in the state can move it. Injected text rarely reached an attacker’s target in one study, while in another both context-ignoring injections and injections that supply false facts redirected nearly every decision; natural context and appended opinions flip confident answers; poisoned open checkpoints carry backdoors with clean accuracy intact; and confidence gates can themselves be attacked to drain capacity. Constrained outputs limit what an attacker can make the model say, not what it decides.

          Supported by

          • Injection shifts probabilities (+0.043) but rarely selects the target (1.8%)Author-reported experiment
          • Natural context flips 61% of correct decisions (vs 2% neutral)Author-reported experiment
          • Poisoned open checkpoints: >89% targeted redirection, clean accuracy intactAuthor-reported experiment
          More supporting evidence
          • An appended opinion flips 12% and drains 38% of confident answers below the gateAuthor-reported experiment
          • Score feedback doubles attacker probability; success 1.8% → 3.5%Author-reported experiment

          Where each study lands

          Teal: the study supports the finding. Amber: it qualifies or limits it. Blank: no claim. Select a column to bring its studies to the top.

          family19 Sep 2026supportssupportssupports
          family19 Sep 2026supportssupportssupports
          20 Sep 2026supportssupportssupports
          21 Sep 2026supportssupportssupports
          21 Sep 2026supports
          family21 Sep 2026supportssupportssupportssupports
          21 Sep 2026qualifiessupports
          21 Sep 2026supportssupports
          family21 Sep 2026supportssupports
          21 Sep 2026supportssupportssupportssupports
          21 Sep 2026supports
          22 Sep 2026supportssupportssupports
          family22 Sep 2026supportssupports
          22 Sep 2026supportssupportssupportssupports
          22 Sep 2026supportssupportssupports
          23 Sep 2026supportssupportssupports
          23 Sep 2026supportssupportssupportssupports
          23 Sep 2026supportssupportssupports
          23 Sep 2026supportssupportssupports
          23 Sep 2026supports
          family23 Sep 2026supports
          24 Sep 2026supportssupports
          24 Sep 2026supports
          family24 Sep 2026supports
          24 Sep 2026supportssupportssupports
          24 Sep 2026supportssupports
          24 Sep 2026supportssupports
          24 Sep 2026supports
          24 Sep 2026supportssupports
          24 Sep 2026supportssupports
          25 Sep 2026supportssupports
          25 Sep 2026supports
          25 Sep 2026qualifiessupportssupportssupports
          26 Sep 2026qualifiessupports
          27 Sep 2026supports
          27 Sep 2026qualifiessupportssupports
          27 Sep 2026supportssupports
          27 Sep 2026supports
          27 Sep 2026supportssupportssupports
          family27 Sep 2026supportssupports
          27 Sep 2026supports
          27 Sep 2026supports
          27 Sep 2026supports
          27 Sep 2026supportssupportssupportssupports
          28 Sep 2026qualifies
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supportssupports
          28 Sep 2026supports
          28 Sep 2026supports
          28 Sep 2026supports
          29 Sep 2026supportssupports
          29 Sep 2026supportssupportssupportsqualifies
          29 Sep 2026supportssupports
          family29 Sep 2026supportssupportssupportssupports
          30 Sep 2026supports
          30 Sep 2026supportssupports
          30 Sep 2026supportssupports
          30 Sep 2026qualifiessupportssupports
          30 Sep 2026supportssupports
          30 Sep 2026supportssupportssupportssupports
          30 Sep 2026supportssupports
          30 Sep 2026supports
          30 Sep 2026supportssupports
          1 Oct 2026supportssupportssupports
          1 Oct 2026supportssupports
          1 Oct 2026supportsqualifiessupportssupportssupports
          1 Oct 2026supportssupports
          1 Oct 2026supportssupportssupports
          family1 Oct 2026supportssupports
          1 Oct 2026supportssupportssupports
          1 Oct 2026supportssupports
          1 Oct 2026supports
          1 Oct 2026supportsqualifies
          1 Oct 2026supportssupports
          2 Oct 2026supports
          2 Oct 2026supportssupportssupports
          2 Oct 2026supportssupportssupportssupports
          2 Oct 2026supportssupportssupportssupportssupports
          3 Oct 2026supportssupports
          4 Oct 2026supportssupportssupports
          4 Oct 2026supportssupports
          4 Oct 2026supportssupportssupportssupports
          4 Oct 2026supportssupportsqualifiessupports
          family5 Oct 2026supports
          5 Oct 2026supportsqualifiessupports
          family5 Oct 2026supportssupports
          family5 Oct 2026qualifiessupportsqualifiessupports
          5 Oct 2026supportssupports
          6 Oct 2026supportssupportssupportssupports
          6 Oct 2026supportssupportssupportssupports
          6 Oct 2026supportssupportssupportssupports
          6 Oct 2026supportssupports
          6 Oct 2026supportssupportssupports
          6 Oct 2026qualifiessupports

          Numbers that cannot share a leaderboard

          Speed and cost figures grouped by what they measure. Figures in different groups are not comparable.

          Single request

          Latency or cost of one independent decision request.

          Amortized per question

          Batch time divided by the number of questions sharing a context.

          • 8.9× vs serial, 3.4× vs no-reuse batch; 5.7 ms/question amortizedVisual Jev

          End to end

          Includes network, queueing, retries or downstream execution.

          Author estimate

          Derived or modelled figures (e.g. electricity-only cost).

          05 Failure modes

          Where a valid answer goes wrong

          How reported failures arise. Illustrative panels use hand-made numbers; Reported panels show published values with their source.

          Illustrative

          Keep the rubric. Move the name.

          Each option is a name plus a rubric that defines it. The study behind this panel changes only which name sits in front of which rubric; question, state, rubric wording and the set of names stay byte-identical.

          noThe request does not meet the refund policy.
          yesThe request meets the refund policy.

          Names and rubrics as the task ships them.

          Reported

          What the swap did (n = 1,200)

          AUROC as shippedAUROC after name–rubric swap
          AUROC before and after the name–rubric swapHosted Jev falls from 0.8146 to 0.5806; an open marker-readout head falls from 0.9376 to 0.2315. n = 1,200 items (2609.26758, §4.6 and §4.2).0.00.51.0chanceHosted Jev (TypeSafe)Hosted Jev (TypeSafe): aligned AUROC 0.8146Hosted Jev (TypeSafe): swapped AUROC 0.58060.81460.5806Open marker-readout headOpen marker-readout head: aligned AUROC 0.9376Open marker-readout head: swapped AUROC 0.23150.93760.2315
          • 32.50% of hosted answers flipped against about 2% under neutral names: 24× the test–retest floor.
          • Type-error rate: 0% in every arm — by construction.
          • In open models, the short label beat the written definition, and deleting every definition changed nothing.
          • Order is a different test: one community audit saw no flips on hosted Jev, while four rotations changed 37% of answers in another study.

          Type-Safe Is Not Error-Free · §4.6; Table 6 ↗Type-Safe Is Not Error-Free · Abstract; §§4.1–4.2 ↗Labels Override Definitions · Abstract; §§5.2–5.3 ↗Biosecurity Audit · §IV; Figure 2 ↗

          Failure-mode catalogue

          Decision contract

          Option names outweigh the rubric

          Rebinding which rubric sits behind “no/yes” moves answers far more than the same swap behind neutral names, and a yes/no swap flipped half of the hosted model’s answers in another study; a 0% type-error rate hides both.

          Mitigation Neutral identifiers with meaning carried in the rubric; name-invariance checks next to type-error rates.

          Decision contract

          Labels outweigh definitions

          In open typed-decision models the answer followed the short option label rather than its written definition, and supplying a dataset’s definition moved up to 28% of predictions without reliably improving them.

          Mitigation Test with neutral labels and with labels that contradict their definitions; render options so the definition, not the label, carries the rule.

          Decision contract

          Order flips hide in aggregates

          One audit found no order flips on hosted Jev, yet four cyclic rotations changed 37% of answers item by item in another study; open readouts flipped on a third of 20-option items, and the vendor now lists option order among Jev’s known weaknesses.

          Mitigation Report per-item flip rates under rotation next to aggregate accuracy; average over rotations or isolate candidates.

          Decision contract

          The primitive changes the answer

          A Noul and a two-option Choice over the same question can disagree, the choice of primitive can reverse a comparison with an LLM baseline, and asking the same probability through two interfaces changed the act/defer decision on a third of pairs.

          Mitigation Fix the primitive in a protocol; test both when a result depends on it.

          Decision contract

          Missing answers go unnoticed

          When the right answer is absent the model usually picks a wrong option rather than the exit: 7% correct rejection of arithmetic menus with an explicit “None”, 24.8% detection of a removed reference label, a hard answer on 63% of items whose deciding evidence was removed, few “I don’t know” choices on unanswerable questions.

          Mitigation Keep an explicit exit, verify candidates with separate yes/no questions, and fit a rejection threshold on held-out answer-absent cases.

          Decision contract

          Middle options absorb errors

          On ANLI the Neutral option took 51% of errors at every candidate position, and across 36 ordinal datasets decisions used only about two-thirds to three-quarters of the gold scale.

          Mitigation Check how much of the label scale predictions use; compare ordinal and nominal framings of the same items.

          Decision contract

          Injected state redirects decisions

          Text placed in the state moved decisions — from rarely reaching an attacker’s target with instruction-style injections to near-certain redirection when the injection supplies false facts; natural context flipped 61% of correct decisions in another study.

          Mitigation Separate trusted from untrusted state, verify facts in code, and red-team with false-state injections, not only instructions.

          Decision contract

          Vendor-listed jagged edges

          The vendor lists literal reading, math and numbers, date comparison, indirection, large irrelevant state, adversarial content, contradictory criteria, option order and generation as known weak spots of jev-1.13.

          Mitigation Keep arithmetic and date logic in code; filter state; split indirect questions; re-check answers under reordered options.

          Decision contract

          Consistent is not correct

          A model can return the same wrong label under every request condition and repeat; agreement across repeats then hides a stable error.

          Mitigation Score every repeat against gold labels and report stable errors separately from changing ones.

          Inference & readout

          Same request, different answer

          Repeated identical requests to the hosted model do not always return the same answer — a re-run changed the Score value on 38.5% of Score questions in one study and 0.5% of decisions a day later in another — while an open checkpoint reproduced bit-for-bit.

          Mitigation Measure a test–retest floor and report effects relative to it.

          Inference & readout

          Cacheable scoring loses the rules

          Encoding candidates independently so they can be cached cut rule-sensitive recall from 1.00 to 0.24; isolating candidates removes order effects only if the rules stay in view.

          Mitigation Keep the state and rules in every candidate’s view; re-measure rule sensitivity after any caching change.

          Inference & readout

          Poisoned open checkpoints

          Poisoning 5% of an open model’s training records implanted backdoors that redirected most targeted inputs while clean accuracy stayed above 90%.

          Mitigation Pin checkpoints by hash and source, and prefer releases whose training data and recipe are documented.

          Probability & calibration

          Probabilities that do not add up

          P(X) and P(not X) missed 1 by 0.064 on average, three single-label probabilities summed to 1.14, and the most likely option received 89% of the mass where its true chance was 36%.

          Mitigation Ask complementary questions and check sums; use rankings rather than raw magnitudes where coherence was not tested.

          Probability & calibration

          Confident beyond what it knows

          With no answer-relevant information the model still put up to 0.80 on a salient option, and on news past its knowledge boundary confidence exceeded accuracy by 0.21–0.33 even after recalibration on earlier months.

          Mitigation Ask targeted yes/no questions about whether the evidence suffices; do not read low confidence as missing knowledge.

          Probability & calibration

          Unknowable rules and distribution shift

          On labels not recoverable from the text, probabilities stayed concentrated; a large post-hoc temperature was needed to make them honest.

          Mitigation Hold out domains and rule changes; recalibrate on target-domain labels.

          Probability & calibration

          Language shift costs accuracy

          A Spanish state with English instructions lost accuracy on every tested dataset, calibration worsened on the hardest tasks, and persona shifts reproduced a smaller share of human differences in Arabic than in English.

          Mitigation Evaluate in the deployment language; pin the model version.

          Probability & calibration

          Two-decimal probabilities

          Probabilities are returned on a 0.01 grid, which puts a floor under calibration error for rare events and can place exactly zero on a correct answer.

          Mitigation Treat rare-event probabilities with care; smooth before log-loss.

          Selective control

          Thresholds that do not travel

          A threshold set for 5% in-scope risk still admitted 31% of out-of-scope requests, a rejection threshold fitted on one task rejected 69% of covered inputs on another, binary probabilities ranked well but sat badly against 0.5, and in-sample thresholds aimed at 5% misses gave up to 17% on held-out data.

          Mitigation Fit thresholds on held-out data from the deployment distribution, per option count, and re-check after any version change.

          Downstream use

          Asked to compute, it estimates

          Exact numbers, counts, graph degrees, packet statistics, chains of statements and opponents’ moves must be computed or simulated, and typed answers land near rather than on them: 44% exact on repeating division (97% within one), 68% on vertex degree, 28% on packet-based traffic classes, chance on liar chains from six steps.

          Mitigation Compute in code and ask the model to judge the result; supply simulations or lookahead explicitly.

          Downstream use

          Fast steps, slower agent

          Replacing an agent’s action selection with a typed model cut median episode time but lowered success from 77.6% to 66.1% and lengthened the slowest runs as step errors compounded.

          Mitigation Evaluate at the episode level, including tail latency and success, not per step.

          Downstream use

          Right label, wrong quantity

          A wrong intermediate selection can change a derived count while leaving the final label correct.

          Mitigation Score intermediate relations and derived quantities, not only final labels.

          06 Applications & selective control

          A first pass, not a replacement

          Across domains the typed model is usually the cheapest and fastest component and rarely the most accurate. Reported gains come from routing uncertain cases elsewhere, with thresholds fixed on separate data, and from letting code compute what the model would only estimate.

          Illustrative · synthetic scores

          How a threshold spends the escalation budget

          A cheap typed model answers every item and reports a confidence; items below the threshold τ go to a strong model assumed correct 95% of the time and 50× more expensive. The data are simulated to show the trade-off, not measured.

          Model test · includes a study family

          Workflow decisions

          The typed model decidesGates, approvals and rule applications inside business workflows: route, accept, reject or escalate.

          • Classifier first, Jev on escalation: same accuracy at 0.43 of the cost
          • Threshold for 5% risk still admits 31% of out-of-scope requests
          • Present 99%, rejected when absent 7% — though Boolean checks get 99%
          • Cacheable scoring loses rule sensitivity (recall@1 1.00 → 0.24)

          Caution Trained classifiers win when labels exist; missing answers and many-option thresholds are weak spots.

          Model test · includes a study family

          Judging & evaluation

          The typed model decidesWhich of two responses is better, whether an answer is supported, how a rubric criterion is met.

          • Frozen cascade: +0.9 points over GPT-6 at 41% of its fee
          • Pre-specified live test: matched GPT-6 exactly, 74% escalated
          • Parity on most rubric criteria at 1/16–1/325 of the cost
          • Kendall τ 0.573 / 0.398 with expert error counts; beats open NLI
          • Open contrastive decision model judges near chance, but ignores answer order

          Caution Falls behind where the verdict must be derived (math, code, logic); judges can share its confident errors.

          Model test · System test · includes a study family

          Annotation & social science

          The typed model decidesLabels for social-science text, political replications and crash narratives, with a probability per label.

          • Behind best LLM on 14/15 tasks (−11.6 F1) at 44× lower cost
          • Low-confidence routing matches the LLM at ¼–½ of its cost
          • Same accuracy, no cost advantage at batch prices
          • Population-scale reading audited by blind coders: sensitivity 0.999

          Caution Per-construct failures; no cost advantage against batch-priced LLMs; one author group contributes several studies.

          Model test · includes a study family

          Security & fraud

          The typed model decidesIntrusion, injection, trace-safety and vulnerability triage decisions — and the security of the decision layer itself.

          • Intrusion pilot: F1 0.859, 4.8× faster than an LLM
          • Trace security: F1 77.8 vs 74.1 for the best LLM judge
          • As a guard: ROC-AUC 0.95–1.0 on eight detection benchmarks
          • Typed outputs, hijacked inputs: up to 100% injection success

          Caution A strong detector whose own decisions can be redirected through the state.

          Model test

          Safety & moderation

          The typed model decidesHate-speech moderation and failure detection against written policies.

          • Within 1.6 macro-F1 of the best LLM at ~97% lower cost
          • Definitions move 28% of answers — without reliably improving them
          • Zero-shot failure detection: median AUROC 0.886 at 1/63 of judge cost
          • Precedents help Jev pick the right rule (+5 to +22 points); Laya barely moves
          • Jev confidence supports selective review; Laya’s barely ranks errors

          Caution Supplied definitions move many answers without reliably improving them; retrieved precedents help Jev more than open models.

          Model test · System test · includes a study family

          Agents & tool use

          The typed model decidesWhich tool, action or branch an agent takes next, escalating to a stronger model when unsure.

          • 95% success with 1.12 strong calls/task vs 88% and 4.10 (−72.7%)
          • Hosted beats open on 9 of 11 harness decisions; routing at chance for both
          • Faster steps, slower agent: success 77.6% → 66.1%
          • Let code simulate: ALFWorld solved 31% → 87%
          • Laya coordinates agents with 78–94% fewer GPT tokens; random choices trail by 1.8 points

          Caution Faster steps can make a slower, less successful agent; let code supply lookahead and simulation.

          Model test

          Search, retrieval & ranking

          The typed model decidesRelevance, evidence sufficiency and next search actions; reranking retrieved candidates.

          • Search agent: 3.7–4.7× faster, accuracy 45% → 54%
          • Task-specialised open model beats Jev on rewriting, trails on routing
          • Strong reranking quality; latency between local models and pointwise LLMs

          Caution Specialised open models beat Jev on some search decisions; hosted latency includes network overhead.

          System test

          Agent memory

          The typed model decidesHow memories are typed, selected and retrieved, and when retrieval stops.

          • LoCoMo judge 0.777 (+11.0%); build 158 s; 0.93 s/query
          • One Jev selection call ≈ LLM-extraction memory at 1/3,061 the write cost
          • Memory agent: LoCoMo 91.7% with Jev judging evidence (AUC 0.94)

          Caution System-level gains measured against different baselines.

          Hybrid · Model test · System test · includes a study family

          Networks & infrastructure

          The typed model decidesIntent fields, A1 policies, traffic classes and alarm causes in networks and water infrastructure.

          • Meets the 1 s RIC budget on 99.8% of calls (LLMs: 17.9%, 0%)
          • Live admission: 0.91–0.95 exact and on time where LLMs fall below 0.1
          • Pre-registered gate: a third of SCADA reviews offloaded, accuracy kept
          • Packet numbers in, 28% accuracy out — trees trained on 8k records reach 70%
          • Hosted Jev never meets 100 ms near-RT deadlines (warm p50 281 ms)

          Caution Meets second-scale control deadlines, not 100 ms ones; raw numeric features defeat it; one author group contributes three studies.

          Model test · includes a study family

          Health & medicine

          The typed model decidesMedical questions, surveillance from narratives, imaging decisions and speech rescoring.

          • Medicine: parity on abstracts, −22 points on diagnostic cases
          • Population-scale reading audited by blind coders: sensitivity 0.999
          • Same backbone, decision-native output: calibration error down up to 10×
          • Rescoring: WER 7.5% vs 7.8% for 7B LLMs, no GPU needed
          • Registered test passed: ECE 0.017, macro-F1 0.92 against 0.87 for a 31B model

          Caution Parity on abstracts but large gaps on diagnostic cases; none of this is clinical validation.

          Model test · Simulation · System test · includes a study family

          Data systems & graphs

          The typed model decidesSemantic filters, joins and rankings inside SQL; structural questions about graphs.

          • Semantic SQL: pruning + typed decisions finish 540K-pair joins
          • Reads every edge; counts degree right 68% of the time
          • Calibrated confidence prices query plans; overconfidence hides output errors

          Caution Pruning carries much of the gain; graph structure has to be computed, not read.

          Model test · includes a study family

          Perception & multimodal

          The typed model decidesImage questions with an independent visual model, and decisions from video, pixel or sensor features rendered as text.

          • Macro accuracy 0.706 → 0.761 after post-training (seen families)
          • PLCC vs VMAF: 0.737 (metadata) → 0.824 (+bitstream, pixel)
          • Typed readout and generation gain equally after adaptation
          • Sensor activity recognition from text: macro-F1 ≤ 0.12

          Caution Hosted Jev is text-only; numeric features rendered as text lose to trained models.

          System test

          Embodied control & games

          The typed model decidesMoves in games, navigation steps and robot skills chosen from short candidate lists.

          • 4/4 full-game wins incl. two vs Lv7 with GPT-6 plans; Jev-only never expanded
          • Navigation: 27% SR at 0.65 s/step; components, not just Jev, carry the gain
          • Evaluates what is described; does not simulate what is not
          • Node-local candidates: 113/120 correct decisions against 90/120

          Caution Plays as if the opponent were random unless the opponent’s move is stated.

          System test

          Population simulation

          The typed model decidesThe distribution of a described respondent’s survey answers, once per unique state.

          • 1.7% flagship anchors cut effect error 41%; decision gain 0.27 → 0.39
          • Passes content gates in 15/15 new countries; fails intervention gates

          Caution Fails intervention-sensitivity gates without flagship anchors; two human-referenced tests.

          Model test

          Legal review

          The typed model decidesEntailment, contradiction or no mention for each hypothesis about one contract.

          • $0.000228 and 1.24 s per contract — lowest of ten models
          • Accuracy 77.38% vs 78.96–83.21% for hosted LLMs
          • 23/30 anchors correct in all 12 responses; 5 consistently wrong

          Caution Cheapest and fastest, but below every hosted LLM on accuracy; small stability panel.

          Model test · System test

          Scientific workflows

          The typed model decidesWhich scientific relation governs a calculation that code then performs.

          • Full semantic correctness; median 0.335 s, p95 0.442 s
          • 7 wrong selections changed counts but not the final label
          • One calibrated pass against a relaxation: as good or better at a thirtieth of the cost

          Caution One study covers twenty Choices in ten cases; the materials-discovery layer is calibrated on in-domain labels and is not a language model.

          07 The method atlas

          Six ways to return a typed answer

          Implementations share the request and response shape, not the mechanism behind the probabilities. Select a family to see how it works, what is public and where its evidence stops.

          A · Hosted System One service

          1. InputText state + typed questions
          2. BackboneUndisclosed hosted model
          3. ReadoutUndisclosed; distribution over the declared options
          4. OutputAnswer + probabilities + confidence

          Hosted System One service

          A closed, hosted model returns a distribution over declared options for each typed question about a shared text state.

          What is public
          Interface, primitives, pricing, limits, versioning and known failure modes are documented; architecture, training data and the exact confidence statistic are not.
          Where evidence stops
          Black-box evaluation only; the served model can change behind aliases; text-only input.
          Representatives
          • TypeSafe Jev (jev-1.13.0)

          Studies in this family

          B · Encoder decision heads

          1. InputState ⊕ question ⊕ options
          2. BackboneBidirectional encoder or encoder–decoder
          3. ReadoutPer-option score → softmax over the legal set
          4. OutputDistribution over options

          Encoder decision heads

          A bidirectional encoder, or an encoder–decoder, reads state, question and options; a trained head scores each option and a softmax is taken over the legal set.

          What is public
          Weights and inference code for most members; several release training code; the readout geometry (marker token, span mean, blind per-option scoring) changes name and order sensitivity.
          Where evidence stops
          Short context windows; option labels can dominate written definitions; adjacent to GLiClass, SetFit and NLI zero-shot classifiers.
          Representatives
          • Laya (ModernBERT-large, mmBERT-base)
          • Bongard (T5Gemma 2 encoder–decoder, 2609.39111)
          • language-specific encoders: Chinese-Jev, sokudan (Japanese), ufakzeka-karar (Turkish)
          • SBERT2S1 (biomedical sentence encoders)
          • WaterSheep, Verdict (ModernBERT)
          Open models and readouts in this family

          Studies in this family

          C · Frozen decoder option readout

          1. InputPrompt listing options with short identifiers
          2. BackboneFrozen causal LM
          3. ReadoutNext-token logits of the identifiers, renormalised; optional debiasing
          4. OutputDistribution over options

          Frozen decoder option readout

          An unmodified causal LM is prompted once and the next-token logits of short option identifiers are renormalised over the declared options, optionally debiased over label priors and option rotations.

          What is public
          Fully open and training-free; a frozen 4B model read this way matches community fine-tunes on public JevBench items — the natural baseline for any trained decision model.
          Where evidence stops
          Raw logits are order- and label-sensitive; one prefill cannot reason; rotation averaging multiplies cost by the number of rotations.
          Representatives
          • SemIf (formerly OpenJev)
          • AnyJev (prior division and rotation averaging, 2610.00831)
          • LLM-as-Jev (bracketed numeric identifiers, 2610.02076)
          • LLM2Jev, LitJev, simple-jev
          Open models and readouts in this family

          Studies in this family

          D · Fine-tuned decoder decision models

          1. InputPrompt with declared options
          2. BackboneDecoder + LoRA or full fine-tune
          3. ReadoutRestricted softmax at a designated position or pointer head
          4. OutputDistribution (+ fitted temperature)

          Fine-tuned decoder decision models

          A decoder is adapted (LoRA or full fine-tuning) so that a designated position or pointer head yields calibrated option probabilities in one pass.

          What is public
          Many release weights, training code and evaluation data; lineages exist (this-that-model is adapted from decider-2b; SecJev builds on Kev); task-specialised variants beat general ones on their own tasks.
          Where evidence stops
          Gains may come from task coverage in training; benchmarks are often built by the same teams; derived checkpoints are not independent methods.
          Representatives
          • Kev
          • decider
          • this-that-model-1.0
          • NanoJev
          • Bespoke Nimble
          • Open-Jev
          • OneJev (multimodal)
          • WebJev (browser actions)
          • SecJev (security)
          • SearchJev (search agents)
          • InnerJev (self-distilled, 2610.03935)
          • JevLite (2609.23959)
          • Visual Jev (2609.25845)
          Open models and readouts in this family

          Studies in this family

          E · Diffusion structured readout

          1. InputCanvas seeded with the answer template
          2. BackboneDiscrete diffusion LM
          3. ReadoutOne denoising step; read each answer slot
          4. OutputDistribution + project-defined confidence

          Diffusion structured readout

          A discrete diffusion LM is seeded with the answer template; after one denoising step, the distribution at each answer slot is read directly.

          What is public
          Open servers with a Jev-compatible wire API; depends on request extensions proposed in an unmerged vLLM PR.
          Where evidence stops
          A different mechanism from last-token decoder readout; custom entropy-based confidence.
          Representatives
          • djev (vLLM example server)
          • OpenJev on DiffusionGemma (razorback16)
          Open models and readouts in this family

          Studies in this family

          No core study uses this family as its main object.

          F · Generative & constrained adapters

          1. InputPrompt + JSON schema
          2. BackboneGeneral LLM API
          3. ReadoutGenerated (verbalised) probabilities, validated and retried
          4. OutputJev-shaped response

          Generative & constrained adapters

          A general LLM is asked (often under JSON-schema constraints) to emit a decision and probabilities, which are validated and returned in Jev-shaped responses.

          What is public
          Wire compatibility for controlled comparisons; probabilities are generated or verbalised, not read from logits.
          Where evidence stops
          Interface compatibility is not mechanism equivalence; retries, parsing and self-reported probabilities remain.
          Representatives
          • system-one-adapter-python (official, for comparisons)
          • LocalJev
          Open models and readouts in this family

          Studies in this family

          A compatible interface is not an equivalent mechanism. No project here establishes the architecture of the commercial model; several say so explicitly.

          08 The open ecosystem

          “Open” is several different things

          Code, weights, training recipes, evaluation material, data and raw predictions are released separately. Availability follows each project’s own README; nothing here was run, and “not located” does not mean absent.

          Open-weight decision models, readouts that turn open LLMs into typed deciders, and compatible servers.

          ResourceWeightsTrainingInferenceEvaluationDataPredictionsLicence
          alperiox/audio-jevlikeEncoder decision heads · Whisper / WavLM audio encoders

          Prosodia: an audio-native Jev-shaped model in which speech is encoded once and several typed questions branch over the encoding, without transcription.

          		○●●●○○—
          Contrastive-LM/CLMEncoder decision heads · Qwen3-8B (frozen encoder)

          Contrastive language models (CLM): a System One model with state and action heads on a frozen Qwen3-8B encoder, trained with a bidirectional InfoNCE loss and served behind a Jev-compatible API (Choice, Noul, Score) with cached state and action embeddings.

          		●◐●◐○◐Apache-2.0
          Heman10x-NGU/Verdict-open-jevEncoder decision heads · ModernBERT (151M, GLiClass lineage)

          Verdict: a 151M ModernBERT-based non-autoregressive decision model with RLCD-style calibration and an in-browser WebGPU demo.

          		●●●●○○NOASSERTION
          hiroki-abe-58/sokudanEncoder decision heads · sbintuitions/modernbert-ja-310m

          sokudan: a Japanese System One model on a 310M ModernBERT-ja backbone with a Jev-compatible API, Japanese and English benchmarks and a Laya position-bias reproduction.

          		●●●●○○Apache-2.0
          mizorewww/laya-mlxEncoder decision heads · Laya checkpoints (ModernBERT / mmBERT)

          Native MLX runtime for the open Laya typed-decision checkpoints on Apple Silicon, with no PyTorch, text generation or cloud API.

          		●○●●○○Apache-2.0
          NandhaKishorM/layaEncoder decision heads · ModernBERT-large; mmBERT-base

          Open, non-autoregressive encoder decision engine: ModernBERT-large (421M) and mmBERT-base (322M) checkpoints with a per-request router; reports 33 ms for one question on a T4.

          		●○●◐○○Apache-2.0
          olanotolu/jevbetterEncoder decision heads · From scratch (hashed n-gram encoder)

          One-pass option scorer trained from scratch with a hashed n-gram encoder, rival-aware attention and temperature scaling, benchmarked against a starter design.

          		○●●●○○MIT
          SamratDuttaOfficial/WaterSheepEncoder decision heads · answerdotai/ModernBERT-base

          WaterSheep: a ModernBERT-base decision model for yes/no, choice, score and multi-label questions with a probability per option and a Jev-compatible local server.

          		●●●●○○Apache-2.0
          daseinlabs/open-jevFrozen decoder option readout · Gemma 3 4B

          One-pass option scoring with a local Gemma 3 4B: the context is prefilled once and every option is scored by its log-probability, with per-task fine-tuning recipes.

          		–●●◐○○MIT
          ekzhang/openjev-sglangFrozen decoder option readout · Qwen3.6-35B-A3B

          Early Jev-compatible server that answers decisions with prefill-only option scoring of Qwen3.6-35B-A3B on SGLang.

          		–○●◐○○—
          featherless-ai/simple-jevFrozen decoder option readout · Open instruction-tuned models (e.g. Gemma 4 26B-A4B)

          Turns open Hugging Face models into Jev-style endpoints by reading next-token logits for each question and building the response from the scores.

          		–○●●○◐Apache-2.0
          nokia-applied-research/AnyJevFrozen decoder option readout · any causal LM (Qwen3-8B in README)

          Turns any open LLM into a Jev-style decider from one prefill of its next-token distribution, with label-free permutation calibration (L0) and label-based calibration (L1).

          		–○●◐○○Apache-2.0
          rorshopping/jev-on-a-laptopFrozen decoder option readout · Stock 1.5B–8B open models

          Unofficial study of Jev-style parallel typed decisions on stock 1.5B–8B models on an Apple Silicon laptop: one prefill, a broadcast KV cache and every field in one forward pass.

          		–○●●○○NOASSERTION
          TheoLeeCJ/SemIfFrozen decoder option readout · frozen Qwen3.5-4B and others

          Semantic ifs from frozen open models (formerly “OpenJev”): reads option-token probabilities in one pass, with prefix reuse, MLX/MPS/llama.cpp backends and per-workload temperature calibration.

          		–○●◐○○MIT
          Yinsongxu/LLM2JevFrozen decoder option readout · Qwen3.5 (e.g. 4B) and other local LLMs

          LLM2Jev: turns local language models into Jev-style decision models from text and images with prefill alone, judging each option separately and normalising across options.

          		–○●●○○Apache-2.0
          zhengxuyu/litjevFrozen decoder option readout · Qwen model family

          LitJev: reads typed choices and probability distributions directly from the output head of off-the-shelf Qwen checkpoints, with no training and no generated text.

          		–○●○○○Apache-2.0
          zhihz/openjevFrozen decoder option readout · Qwen3-4B-Instruct-2507 (frozen)

          Open JEV: local bilingual probability decisions over candidate answers, currently read from a frozen Qwen3-4B-Instruct-2507 backend.

          		–○●●○○NOASSERTION
          bespokelabsai/nimbleFine-tuned decoder decision models · Qwen3.5-9B (LoRA)

          Open recipe and Bespoke-Nimble-9B (LoRA on Qwen3.5-9B) for typed choice and yes/no decisions with candidate scoring; training (2,676) and held-out (324) examples published; temperature refitted on 2026-09-22.

          		●●●○●○NOASSERTION
          FLock-io/this-that-modelFine-tuned decoder decision models · decider-2b (Qwen3.5-2B lineage)

          this-that-model-1.0 (~1.9–2B): restricted-softmax readout at a designated position; weights on Hugging Face (MIT), inference package and scripts said to reproduce every paper measurement.

          		●○●●◐○MIT
          getainode/jebadiahFine-tuned decoder decision models · Qwen3.8-27B, Qwen3.5-9B, Qwen3.5-4B (LoRA)

          Jebadiah: open decision models (4B–27B LoRA on Qwen bases) with the trainer, data builders, evaluation harnesses and a result record for every run.

          		●●●●●●Apache-2.0
          gitchw/LCTFine-tuned decoder decision models · Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen3-8B

          Jev-LCT: Qwen-based decision models that loop only the top two layers before a typed readout, read confidence from the loop trajectory, exit early on clear cases and serve a Jev-compatible API.

          		●●●●●◐Apache-2.0
          guanxuyu-sv/Visual-JevFine-tuned decoder decision models · Qwen3-VL-4B

          Project page and result JSON for Visual Jev; figures are generated from the result files behind the paper tables.

          		–○–○○◐NOASSERTION
          iapp-technology/openthai-systemoneFine-tuned decoder decision models · Qwen3.5-0.8B (continued pre-training on Thai)

          OpenThai-SystemOne: a Thai–English decision model whose Qwen3.5-0.8B language-model head is replaced by a 256-way slot softmax, with a Jev-compatible server.

          		●●●○○○Apache-2.0
          jaredpalmer/kevFine-tuned decoder decision models · Qwen3.5 (0.8B, 4B, 9B)

          Small Jev-like decision models (0.8B/4B/9B) on Qwen3.5 with LoRA and an option-pointer readout; training code, frozen evaluation suites and a Jev-compatible local server.

          		●●●●●○Apache-2.0
          lexmount/WebJevFine-tuned decoder decision models · Qwen3.5-35B-A3B-Base

          WebJev: Jev-like decision models specialised for browser use that pick the next action on a live page in one forward pass, with training, evaluation and real-website task runs.

          		●●●●●○Apache-2.0
          Mapika/deciderFine-tuned decoder decision models · Qwen3.5-2B/4B/35B-A3B

          One-pass typed decisions (Choice 2–255 options, Score 2–10 levels, Noul) from decoders built on Qwen3.5-2B/4B/35B-A3B; training mixture of public data plus labels from a local Qwen3.5-27B teacher.

          		●◐●○○○Apache-2.0
          nico-martin/open-jevFine-tuned decoder decision models · Kev and DeBERTa-v3 ONNX exports

          Browser TypeScript library that runs open Jev-shaped checkpoints on-device with Transformers.js (WebGPU or WebAssembly).

          		●○●○○○MIT
          OmniJev/OneJevFine-tuned decoder decision models · Qwen3.5-0.8B to Qwen3.8-27B (vision tower kept)

          OneJev: multimodal System One models (0.8B–27B) that answer typed questions about screenshots, photos, video and text in one forward pass.

          		●●●●●○Apache-2.0
          OmniJev/PlayJevFine-tuned decoder decision models · Qwen3.5-0.8B-Base

          PlayJev: Qwen3.5-0.8B fine-tuned to play ten browser games from raw pixels, one frame in and one move out per forward pass.

          		●●●○●○Apache-2.0
          PsiACE/dohnutsFine-tuned decoder decision models · 0.8B multimodal base (LoRA)

          Dohnuts: a 0.8B multimodal decision model that chooses, judges or scores messages, documents and images, with toolkits for training, evaluation and agent use.

          		●●●●○○Apache-2.0
          Rizzo-AI-Academy/rizzo-flowFine-tuned decoder decision models · Spark-X2.5 4B and 1.7B (LoRA)

          Rizzo Flow: local typed decisions from small LLMs without generating tokens, with LoRA fine-tunes of Spark-X2.5 for typed decisions and the training recipe.

          		●●●●○○Apache-2.0
          shamazharikh/qwen-rlcdFine-tuned decoder decision models · Qwen3.5-0.8B

          Prototype of a Jev-style decision model on Qwen3.5-0.8B that forks one state encoding into many question branches.

          		○◐●○○○—
          TianyuCodings/NanoJevFine-tuned decoder decision models · 0.6B decoder

          A 0.6B parallel decision model (“nano replica”) with public weights and dataset, trained on game-control decisions (ViZDoom, maze, Snake).

          		●○●◐●○MIT
          uspraveen/JevifyFine-tuned decoder decision models · Gemma 4 E4B, Qwen3.5-2B/9B, Qwen3-VL-2B

          Jevify: turns open LLMs into calibrated Jev-style decision models with fine-tuned readouts, released checkpoints, the jev-bench dataset and a Jev baseline with a label audit.

          		●●●●●◐NOASSERTION
          Zefan-Cai/Open-JevFine-tuned decoder decision models · Open-Jev 2B, 9B and 27B checkpoints

          Open-Jev: open decision models (2B–27B) that return typed probabilities for supplied candidates, with the Open-Jev training data and release reports.

          		●●●●●○MIT
          JoshuaSP/open-jevDiffusion structured readout · DiffusionGemma 26B-A4B

          Typed JSON decisions from DiffusionGemma without training: denoise, then pick the most likely allowed tokens from the final logits.

          		–○●●○○MIT
          mmastrac/djevDiffusion structured readout · DiffusionGemma

          Example server for Jev-style structured reads on DiffusionGemma: seeds the answer template, pins positions and reads slot distributions after one denoising step.

          		–○●○○○Apache-2.0
          razorback16/openjevDiffusion structured readout · DiffusionGemma 26B-A4B

          Open Jev-compatible decision server on DiffusionGemma 26B-A4B via vLLM or MLX, with optional image questions and its own confidence definition.

          		–○●○○○Apache-2.0
          githubnext/localjevGenerative & constrained adapters · DiffusionGemma via chat completions

          Local Jev-compatible API in TypeScript that asks a model for JSON probabilities, validates, retries and normalises them.

          		–○●○○○MIT
          typesafe-ai/system-one-adapter-pythonGenerative & constrained adapters

          Official drop-in replacement for the SDK’s system_one call that routes typed questions to OpenAI, Anthropic, Gemini or compatible LLM APIs, for cost, speed and quality comparisons.

          		–○●○○○MIT
          feder-cr/jevNot classified

          jevos: a laptop-CPU decision model behind a Jev-compatible API that answers yes/no, choice and score questions in tens of milliseconds.

          		○○●●○○MIT
          PAI-CUHK/MEDJEVNot classified · Qwen3 backbones (e.g. Qwen3-0.6B)

          Research implementation of typed decisions over clinical evidence: a record, a statement and a runtime candidate set (supported, contradicted, unresolved) scored on open Qwen backbones.

          		○●●◐○○MIT
          PAI-CUHK/SLEEPJEVNot classified · Polysomnography signal encoder

          Typed Choice, yes/no and Score decisions over overnight polysomnography, built on a reusable whole-night encoding and sparse temporal retrieval.

          		○●●●○○MIT
          pCwOrM/werrNot classified

          Jev wire-compatible “zero-memory” decision runtime accompanying the peripheral paper 2609.25498.

          		–○–○○○NOASSERTION
          vinnylarouge/jevlikeNot classified

          Starter for training small one-pass option scorers that take text and a changing list of options and return one probability per option.

          		○●●●○○MIT
          ● Available◐ Partial◑ Restricted◔ Project page only○ Not located– Not applicableLicence is the field GitHub detects; NOASSERTION means “not identified”, not “no licence”.

          Study family

          Intent at RIC Timescales · Replacing LLMs with Jev at the Edge

          Same author group (Li, Wang, Gong, Lang, Yu); both papers were substantially revised after first posting. The group also wrote the network-control SoK (2610.06425). Synthesise as one family, not independent replications.

          Study family

          Laya Reproduction · CLM-as-a-Judge

          Same author: a reproduction of Laya and a preregistered judge evaluation of CLM that reuses its protocol; synthesise as one family, not independent replications.

          Study family

          HakemBench · ufakzeka-karar

          Same author: the HakemBench benchmark and the ufakzeka-karar model it ranks; the model’s benchmark numbers are flagged by the author as not blind.

          Study family

          PixelJev · GraphDecide

          Shared authors (Yang, Zhao): PixelJev and GraphDecide.

          Study family

          REFLEX with Jev · Decision Hijacking

          Same two authors (Wu, Lim): an agent architecture built on Jev and an attack study of Jev decision layers.

          Study family

          Calibrated Decisions at Scale · Pregnancy in Crash Narratives · Decision Gates Benchmark

          Same two authors (Rafe, Das): crash-narrative coding, pregnancy surveillance in crash narratives and a decision-gate benchmark; synthesise as one family, not independent replications.

          Derived from

          FLock-io/this-that-model → Mapika/decider

          this-that-model-1.0 is adapted from decider-2b (Apache-2.0), per the paper’s Availability section.

          Uses resources from

          this-that-model-1.0 → TianyuCodings/NanoJev

          Environment simulator and the recorded hosted-service cohort (68 questions) come from NanoJev (MIT).

          Compatible, not equivalent

          githubnext/localjev → razorback16/openjev

          LocalJev states it is wire-compatible but not mathematically equivalent to OpenJev’s logit read.

          Same name, different thing

          “RLCD”

          TypeSafe’s “Reinforcement Learning for Calibrated Decisions” is unrelated to “RLCD: Reinforcement Learning from Contrastive Distillation” (2023); 2609.24574 makes the same distinction. Neither adjacency implies the same algorithm or copying.

          Same name, different thing

          “OpenJev”

          At least eight unrelated projects share the name: SemIf (formerly OpenJev), daseinlabs/open-jev and zhihz/openjev (frozen-decoder readouts); razorback16/openjev and JoshuaSP/open-jev (DiffusionGemma readouts); Zefan-Cai/Open-Jev (fine-tuned decision models); nico-martin/open-jev (an in-browser runtime for open checkpoints); and the “Open-Jev” of the title of 2609.23959, whose model is JevLite.

          Same name, different thing

          “REFLEX”

          The agent architecture of 2609.26532 is not any community project named “reflex”.

          Same name, different thing

          “Visual Jev”

          An independent research model (2609.25845), not a TypeSafe release; hosted Jev is text-only.

          Same name, different thing

          “System One / System 1”

          TypeSafe’s product category borrows Kahneman’s System 1 metaphor; System-1/2 reasoning literature (e.g. 2502.17419) predates and is broader than the product.

          Same author group

          SoK: Semantic Decision Engines in Network Control Loops · Replacing LLMs with Jev at the Edge · Intent at RIC Timescales

          The SoK on semantic decision engines in network control loops comes from the authors of the two edge/RAN studies.

          09 The reading room

          Search the evidence

          Filter by stage, method family, relationship to Jev, topic or openness, and open any record for its evidence, versions and limitations.

          All references

          1. CoreBOTTLED

            Agent in a Bottle: Can LLM Agents Turn Their Capabilities Into Cheap, Scalable Artifacts?

            Ankit Sonthalia, Haritz Puerto, Alexander Rubinstein et al.

            Strong zero-shot models did not reliably bottle (48 of 60 runs fell below their model’s zero-shot interval), but bottling could be far cheaper; against zero-shot Jev on 1,000-item samples, an Opus 5 solution reached 94% of Jev’s ESCI macro-F1 (0.499 against 0.531) at about a quarter of Jev’s projected full-workload cost, and a Gemini 3.1 Pro solution 97% of Jev’s RAID AUROC (0.822 against 0.846) at about 2% of its cost.

            arXiv preprint6 Oct 2026Code

          2. CoreSystem One Moderation

            Benchmarking System One Models in Online Moderation

            Federico Mazzoni, Andrea Failla

            With the full rule inventory held fixed, retrieved precedents raised Jev’s exact policy-selection accuracy by 12.35, 5.20 and 22.20 points on three evaluations while Laya gained little or lost; precedents raised Jev’s AUROC on OLID (.789 → .816), HateXplain (.749 → .755) and ModerationBench (.893 → .936 with Choice), and its confidence supported selective review (71.9% of ModerationBench decisions at confidence ≥ 0.8, with 94.3% accuracy), although it was not consistently calibrated.

            arXiv preprint6 Oct 2026Code

          3. CoreReadout Stability

            Readout Stability in Prefill-Only Decision Models:Zero-Label Prediction and Inference-Time Compute Allocation

            Ran Li, Lei Chen

            Across seven open decision-model families, accuracy after a menu-only change was predicted from the first pass within 4.2 points (exactly for one family), while same-scale generative models erred by 1.6–15.8 points; extra order-permuted passes cut calibration error by 67% but added almost no accuracy, a confidence cascade to a larger sibling beat every re-ask scheme at matched cost, and a 0.8B model on a curated five-candidate menu reached 95.4% on CLINC150 against 80.0% for a 4B model on all 150 labels.

            arXiv preprint6 Oct 2026Code

          4. CoreSanSi

            SanSi: A Looped Typed Decision Model for System 1.5 Thinking

            Shuyu Gan, Young-Jun Lee, Dongyeop Kang

            Eight loops raised accuracy from 58.4% to 72.0% at fixed shape, data and recipe (+13.5 points), within 1.8 points of a single-pass model three times larger, and extended solvable reasoning depth beyond training; hosted Jev, not trained on these data, scored 78.9% overall but gave a hard answer to 62.8% of items whose deciding evidence had been removed and fell from 100% to chance on liar chains as the number of steps grew.

            arXiv preprint6 Oct 2026Code not located

          5. CoreS1-MAS

            Token-Efficient Multi-Agent Collaboration via System One-Guided Computational Division of Labor

            Zihan Zhou, Xinzhe Hu, Hanxu Yang et al.

            S1-MAS reached the highest mean accuracy on six benchmarks (macro-average 93.97%, 0.85 points above DyLAN) while cutting GPT-4o tokens by 78.3–94.0% and latency by 58.2–91.7% against five multi-agent baselines; in ablation, Laya’s coordination decisions beat random choices from the same legal menus by only 1.83 points but matched a GPT-4o-mini controller’s accuracy with 75.5% fewer GPT tokens.

            arXiv preprint6 Oct 2026Code not located

          6. CorePlans Change Answers

            When Plans Change Answers: Formalizing Cost-Accuracy Optimization for Semantic Queries

            Kyoungmin Kim

            Weighting each calibrated decision’s expected error by its contribution to the output predicts a plan’s output quality without labels; in simulation, plans for the same query differed by up to three orders of magnitude in cost, selection pushdown with bands calibrated on the plan’s own candidates met the output target on 62% of instances against 96% for pull-up, per-tuple prices met the target at about a fifth of the cost of the best operator-level band, and an overconfident model predicted output precision 0.981 where 0.872 was realised.

            arXiv preprint6 Oct 2026Code not located

          7. CoreCLM-as-a-Judge

            CLM-as-a-Judge: Evaluating an Open Contrastive Decision Model on Public Judge Benchmarks

            Gowthamkumar Nandakishore

            Zero-shot CLM scored 0.351–0.593 (at chance on RM-Bench and JudgeBench, one constant label on HaluEval) while a same-size reward model reached 0.764–0.976; raw confidences were overconfident by up to +0.401 but one pooled temperature brought ECE to at most 0.062, and order flips were 0.0002 against 0.2188 for the generative judge, yet a confidence-gated cascade escalated 92.3–100% of items to the strong judge.

            arXiv preprint5 Oct 2026CodePredictions

          8. CoreGraphDecide

            GraphDecide: Benchmarking System One Models on Graph Tasks

            Xianliang Yang, Yapu Zhang, Li Zhao

            Jev answered every adjacency query correctly and did well on cycle, connectivity and distance, yet reached only 67.80% on exact degree and 61.25% on articulation points; joint graph-text input did not consistently improve prediction, and feasible constructions did not imply good solutions.

            arXiv preprint5 Oct 2026CodePredictions (partial)

          9. CorePolitical Science Replications

            JEV versus LLMs: Accuracy, Cost and Calibration on Seven Political Science Replications

            Steven Denney, Matthew DiGiuseppe

            Jev matched or came close to both LLMs across tasks, but had no cost advantage over GPT-6 Luna at batch prices because it billed 1.2–3.3 times as many input tokens per decision; asked once, its probabilities were better calibrated than GPT-6 Luna’s token probabilities but not consistently better than Qwen3.8-27B’s.

            arXiv preprint5 Oct 2026Code not located

          10. CoreSharedKV-BT

            SharedKV-BT: Node-Local Typed Decisions for Behavior-Tree Agents

            Naoki Wake, Justin Wagle

            Exposing only the current node’s candidates raised correct joint typed decisions from 90/120 to 113/120 and closed-loop four-object completion from 0/30 to 18/30; prefix reuse alone cut decision latency 1.29–2.09-fold, and stage gating with external postconditions prevented out-of-order actions and premature completion, but an explicit ABSTAIN option rejected 28/30 invalid menus while choosing correctly in only 17/30 valid ones.

            arXiv preprint5 Oct 2026Code not located

          11. Coreufakzeka-karar

            ufakzeka-karar: An Open Turkish Typed-Decision Model with Order-Invariant Option Scoring

            Sait Furkan Teke

            Scoring options blind to each other made answers independent of order by construction, while a sequential head trained on shuffled options changed 2.3–2.8% of answers when only the order changed; REINFORCE lost 0.102 macro-F1 to cross-entropy. The released model ranks 7th of 16 on HakemBench (composite 0.660) at a median of 102.5 ms on one CPU thread, and temperature scaling raised held-out calibration error (0.027 → 0.045).

            arXiv preprint5 Oct 2026CodeWeights

          12. CoreFiorillo

            Calibrated Answers About Randomized Trials From a 4-Billion-Parameter Open Model: A Registered Test and a License-Clean Release

            Johann Emmanuel Li

            All four criteria registered before test predictions passed: ECE 0.0168 (limit 0.05), log loss below Gemma 4 31B-it by 0.1829 and macro-F1 0.9248 against 0.8668; reading checks showed macro-F1 0.4384 without the article, a gain from the title alone, and only 66.5% of direction answers reversed when intervention and comparator were exchanged.

            arXiv preprint4 Oct 2026CodeWeightsPredictions (partial)

          13. CoreCrystalJev

            CrystalJev: thinking fast and slow with atomistic foundation models for materials discovery

            Peng Kang, Zhen Li, Yu Liu et al.

            For MACE-MP-0 and CHGNet, one calibrated unrelaxed pass decided stability better than recalibrated relax-and-threshold (log-loss 0.191 against 0.247 and 0.196 against 0.266) at about a thirtieth of the cost, and slightly worse for four newer potentials; in a registered prospective test, single-pass forecasts calibrated only on existing data over-stated the stable fraction of unseen candidates (5.8%) by 0.8–2.1 points, with AUROC 0.965–0.982.

            arXiv preprint4 Oct 2026Code not located

          14. CoreHidden Risks of Jev

            Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

            Shang Wang, Tianqing Zhu, Huajie Chen et al.

            Context-ignoring injection reached 96.5% attack success against Jev on support routing and Jev-specific injections reached 100% on subscription cancellation and at least 96.8% on issue severity; backdoors implanted by poisoning NanoJev succeeded on over 89% of targeted inputs while clean accuracy stayed at 90.5–94.0%; as a detector Jev reached ROC-AUC 0.95–1.0 on all eight benchmarks, and the same capability supported jailbreak planning and model extraction.

            arXiv preprint4 Oct 2026Code

          15. CoreSearchJev

            SearchJev: A Fast and Calibrated System-1 Model for Search Agents

            Congfeng Cao, Lipeng Zuo, Konstantinos Papakostas et al.

            SearchJev beat same-size Qwen3.5 autoregressive models in decision quality while deciding 5.2–5.3 times faster with 41–74% lower average ECE; on held-out data it exceeded hosted Jev on query rewriting (and the 4B model on relevance) while Jev stayed stronger on routing and verification; in BrowseComp-Plus the dual-system agents cut median active search time 3.7–4.7-fold and raised answer accuracy from 45% to up to 54%.

            arXiv preprint4 Oct 2026CodeWeights

          16. CoreWireless Decision-Making

            System One Models for Wireless Decision-Making:Applications and Performance Evaluation

            Masoud Rahimi, S. M. Matin Alemohammad, Hamid Behroozi et al.

            In receive-antenna selection Jev cut median response latency by up to 8.5 times against the evaluated LLMs but lost decision quality to stronger task-specific methods; in intent-conditioned RAN slicing it reached utility comparable to the LLMs with more than 3.5 times lower median latency. Faster decisions did not necessarily mean lower end-to-end service latency.

            arXiv preprint3 Oct 2026Code not located

          17. CoreCandidate Coverage and Transfer

            Benchmarking candidate coverage and rejection policy transfer in typed decision models

            Jiawen Lu, Tongtong Wu

            Rejection thresholds calibrated on one task often failed on another: with 80 calibration pairs, none-score policies exceeded 5% target false rejection in six of twelve transfers for both Laya and Jev, a Jev policy fitted on DBpedia rejected 69.3% of covered Emotion inputs and an Emotion-fitted policy disabled rejection; with retrieved CLINC150 menus, more candidates raised Laya’s out-of-scope detection to 99.0% at 48.8% false rejection while Jev kept about 3% false rejection but detected 75.0%.

            arXiv preprint2 Oct 2026CodePredictions

          18. CoreJEVal and InnerJev

            General Decision Models: Benchmarking and Insights Beyond Jev

            Feiyu Duan, Jiayu Lin, Jia Wang et al.

            The strongest decision models came within three points of leading thinking LLMs overall but fell behind on specialist knowledge; Jev picked the mode on all 100 prior-probability items yet put 89.35% of the mass on it against 35.60% in the true distribution; in τ-bench, Jev action selection cut median episode time from 267 s to 161 s but lowered success from 77.6% to 66.1%; InnerJev-27B performed on par with Jev at about 0.1 s per query.

            arXiv preprint2 Oct 2026Code

          19. CoreSecJev

            SecJev: Bringing Security Expertise to System One Decision Models

            Zheng Chen, Fei Yu, Haohao Huang et al.

            Security specialisation improved every model in the family: SecJev-0.8B gained 39.40 points of task-macro accuracy over Kev-0.8B and beat general Kev-9B by 20.51 points, with accuracy and latency close to answer-only generative fine-tuning at lower peak memory; gains over Kev held on new prompt-injection and traffic sources, with capture-dependent false alarms.

            arXiv preprint2 Oct 2026CodeWeights

          20. CoreHateDecide

            To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

            Demetris Paschalides, George Pallis, Marios D. Dikaiakos

            Commercial LLMs significantly beat all decision models on only one of four datasets (by 4.1–5.9 macro-F1), and supervised classifiers lost up to 54 points out of distribution; supplying definitions changed up to 28% of predictions but improved agreement in only 29% of comparisons, decomposition helped in 20% and hurt in 37%, and the best hosted decision model came within 1.6 points of the best commercial LLM at about 97% lower inference cost.

            arXiv preprint2 Oct 2026Code not located

          10 About

          How the evidence was gathered, and how to cite it

          What counts as a study here, how each one is read, related reviews, release notes and the data behind every section.

          Scope & sources

          • Core studies evaluate Jev, build a Jev-like typed decision model, or depend on one inside a system.
          • Background references cover adjacent and earlier work: classifiers, calibration, selective prediction, structured output, routing, judging and related reviews.
          • Studies come from repeated arXiv searches; repositories from GitHub searches and curated lists, each checked against its own README. Queries and screening decisions are published with the data.

          Reading the evidence

          • Core studies are read in full, and revised versions are re-read; every evidence record points to the section or table it comes from.
          • Values are as reported by authors, the vendor or community repositories; they were not re-run.
          • Openness is recorded field by field — code, weights, data, predictions — because each can be released on its own.

          Details: methodology · limitations · evidence audit

          What each one covers, as stated in its own text. A blank cell means the property is not described there.

          Review or catalogueScopeUnit of analysisWhat is recordedReleased materialVersioning
          This surveyStudies, repositories and vendor documents on Jev and Jev-like models, 15 Sep 2026 to 7 Oct 2026Studies, evidence records, repositoriesClaim-level evidence with locators, sample sizes and versions; openness field by field; failure modesJSON, CSV, BibTeX, website, manuscriptDated, versioned releases
          Decisions, Not Tokens ↗Working draft on GitHubMachine-native decision models, from classical classifiers to JevLiteratureNarrative survey of the model classManuscript source on GitHubWorking draft dated 21 Sep 2026
          Typed Decision Models: An Early Evidence Audit ↗arXiv preprint28 papers posted 19–24 September 2026PapersRecurring weaknesses of the early studies; a 14-item evaluation checklistPer-paper corpus sheet (arXiv ancillary file)arXiv v1
          SoK: Semantic Decision Engines in Network Control Loops ↗arXiv preprint139 paper families on semantic decision engines in network control loopsPaper familiesClassification by decision interface, execution path and check ownership; a minimum reporting recordPaperarXiv v1
          OmniJev / awesome-jev-gallery ↗Curated list on GitHubOpen models, projects, benchmarks, commentary and context around JevRepositories, posts and papersOne-line descriptions with linksREADME listContinuously edited
          OmniJev / awesome-jev-papers ↗Curated list on GitHubResearch papers on Jev and open models built in its shapePapersThematic list with linksREADME listContinuously edited

          News

          • Version 0.3.1: studies from the 7 October arXiv listing, a revised candidate-coverage study, and two more open models.

          • Version 0.3.0: studies posted through early October; findings on probability coherence, computation and security; an audited open ecosystem of models, benchmarks and applications; a redesigned site.

          • Version 0.2.0: four new core studies from the 24 September arXiv listing, new application areas and a new failure mode.

          • Version 0.1.0: first public release of the survey, its data and this site.

          Cite

          A versioned working draft without a DOI; it has not been peer-reviewed.

          @misc{luo2026jevsurvey,
  title        = {Jev and Typed Decision Models: An Empirical Survey of Calibration,
                  Selective Control, and Open Implementations},
  author       = {Luo, Meng},
  year         = {2026},
  howpublished = {\url{https://eurekaleo.github.io/awesome-jev-survey/}},
  note         = {Version 0.3.1, updated 7 Oct 2026. Working draft}
}

          Evidence record

          Loading…