ezjev reads a piece of state — an email, a contract, a ticket, a tool list — and answers choice, noul and score questions with a probability for every option. One prefill, no generated tokens, served through the Jev-style /v1/systemone API.
ezjev 读入一段状态（邮件、合同、工单、工具列表……），回答 choice、noul、score 三类问题，给每个选项一个概率。只做一次 prefill，不生成文字，接口是 Jev 风格的 /v1/systemone。
// request { "state": "I was charged twice. Please refund the duplicate.", "questions": { "refund": { "type": "noul", "instructions": "Does the user request a refund?" }, "dept": { "type": "choice", "instructions": "Which department?", "criteria": { "billing": "Payments and refunds", "technical": "Software bugs" } } } } // response — numbers illustrative { "answers": { "refund": { "type": "noul", "noul": 0.98 }, "dept": { "type": "choice", "choice": "billing", "probabilities": { "billing": 0.97, "technical": 0.03 } } } }
ezjev has been submitted to two leaderboards. Decision Index has published its result; JevBench is still in the queue — status as of 2026-10-07.ezjev 已经提交到两个榜单。Decision Index 已经公布成绩，JevBench 还在排队——以下是 2026-10-07 的状态。
Submitted 2026-10-04 (PR #55) · listed 2026-10-06 · #35 of 1112026-10-04 提交（PR #55）· 2026-10-06 上榜 · 111 个中第 35
Best of all models ≤ 5B — every entry ranked above it has at least 9B parameters. Decision Index 0.3 scores 20% public benchmarks (ezjev: 50.82), 50% private tests of the same skills (46.22) and 30% private tasks from new domains (40.73), so most of the score comes from tests nobody can train on. See the leaderboard.在 5B 以下的所有模型中排第一——排在它前面的模型都至少有 9B 参数。Decision Index 0.3 的总分由三部分组成：20% 公开 benchmark（ezjev 50.82）、50% 同类技能的私有测试（46.22）、30% 新领域的私有任务（40.73），大部分分数来自谁都没法拿来训练的题。详见榜单。
Submitted 2026-10-07 · ref 7B97B020 · model ezjev-4b-s3 · free queue, position #322026-10-07 提交 · 编号 7B97B020 · 模型 ezjev-4b-s3 · 免费队列第 32 位
Self-test on the public items only. The official JevBench Score (Intelligence, Calibration, Speed, Cost, incl. sealed items) is measured by the maintainer; we submitted through the official form (the earlier GitHub request #193 was closed at the maintainer's request), and the result will appear on the live board.这里只是公开题的自测。正式的 JevBench 总分（准确率、校准、速度、成本四项，包括封存题）由维护者测量；我们已通过官方表单提交（之前在 GitHub 上的报名 #193 应维护者要求已关闭），结果会出现在榜单上。
Every question comes back as a probability distribution over its options, so you can set thresholds, route on confidence, or abstain.每个问题返回的都是选项上的概率分布，可以按置信度设阈值、分流，或者选择不回答。
Routing, intent, tool selection, policy outcomes. Named options with descriptions; up to hundreds of options.路由、意图识别、工具选择、政策判定。选项带名称和说明，最多可以有几百个。
Returns P(true): phishing or not, supported by the evidence or not, did the answer meet every requirement.返回 P(true)：是不是钓鱼邮件、证据是否支持、回答是否满足了所有要求。
Severity, urgency, quality — a distribution over ordered levels, not just a single number.严重程度、紧急程度、质量——给出各个等级上的分布，而不只是一个数字。
Both are full merged BF16 weights (~9.1 GB) on Hugging Face. Use s3 unless you need to reproduce the Decision Index run.两个版本都是合并好的完整 BF16 权重（约 9.1 GB），放在 Hugging Face 上。除非要复现 Decision Index 的成绩，否则用 s3。
Latest. s2 plus ~20k code-generated long-document and date/number reasoning decisions. Submitted to JevBench.最新版。在 s2 基础上加了约 2 万道代码生成的长文档、日期和数字推理题。已报名 JevBench。
hf download everettjf/ezjev-4b-s3 --local-dir ezjev-4b-s3
The checkpoint listed on Jev Decision Index 0.3 — the best model ≤ 5B on that board.上榜 Jev Decision Index 0.3 的版本，是该榜单上 5B 以下最好的模型。
hf download everettjf/ezjev-4b-s2 --local-dir ezjev-4b-s2
About 10 minutes from a fresh GPU machine to your first decision. Two processes run side by side: vLLM loads the model and computes option logprobs, and llm2jev turns them into the /v1/systemone API your code calls.从一台新的 GPU 机器到拿到第一个决策结果，大约 10 分钟。需要同时运行两个进程：vLLM 加载模型、计算各选项的 logprob；llm2jev 把结果包装成 /v1/systemone 接口，供你的代码调用。
Use a fresh virtual environment; these are the exact versions we test with.建议用一个新的虚拟环境；下面是我们测试时用的确切版本。
python -m venv ezjev-env && source ezjev-env/bin/activate pip install "vllm==0.30.0" "llm2jev==0.6.1"
The first start downloads the weights from Hugging Face (9.1 GB). Leave it running.第一次启动会从 Hugging Face 下载权重（9.1 GB）。启动后让它一直运行。
vllm serve everettjf/ezjev-4b-s3 --port 8000 \ --max-logprobs 256 --return-tokens-as-token-ids --max-model-len 32768
Ready when curl -s localhost:8000/health returns HTTP 200. --max-logprobs and --return-tokens-as-token-ids are required — without them every request fails.就绪的标志：curl -s localhost:8000/health 返回 HTTP 200。--max-logprobs 和 --return-tokens-as-token-ids 两个参数必须加，否则每个请求都会失败。
--model must match the model vLLM serves; --temperature must match the model (1.34 for s3).--model 要和 vLLM 加载的模型一致；--temperature 要和模型对应（s3 用 1.34）。
llm2jev --model everettjf/ezjev-4b-s3 --backend vllm \ --url http://127.0.0.1:8000 --port 8080 --prompt chat --temperature 1.34
Ready when curl -s localhost:8080/health returns {"status": "ok", ...}.就绪的标志：curl -s localhost:8080/health 返回 {"status": "ok", ...}。
Send a state (any text, or a JSON object) and one or more named questions. Write criteria as short, mutually exclusive descriptions — they are what the model chooses between.发送一段 state（任意文字，也可以是 JSON 对象），加上一个或多个带名字的问题。criteria 写成简短、互斥的描述——模型就是在这些描述之间做选择。
curl -s http://127.0.0.1:8080/v1/systemone -d '{
"state": "I was charged twice. Please refund the duplicate.",
"questions": {
"refund": {"type": "noul",
"instructions": "Does the user request a refund?"},
"dept": {"type": "choice", "instructions": "Which department?",
"criteria": {"billing": "Payments and refunds",
"technical": "Software bugs"}}
}
}'import requests
API = "http://127.0.0.1:8080/v1/systemone"
r = requests.post(API, json={
"state": "Order 1182 arrived with a cracked screen.",
"questions": {
"urgency": {"type": "score",
"instructions": "How urgent is this?",
"criteria": ["routine", "soon", "urgent"]},
"replace": {"type": "noul",
"instructions": "Is a refund or replacement due?"},
},
}, timeout=30)
r.raise_for_status()
a = r.json()["answers"]
print(a["urgency"]["probabilities"]) # {"0": .., "1": .., "2": ..}
print(a["replace"]["noul"]) # P(true)| Type题型 | What you send请求里写什么 | What you get back返回什么 |
|---|---|---|
| noul | instructions (a yes/no statement); optional criteria {"true": …, "false": …}instructions（一个是非判断）；可选 criteria {"true": …, "false": …} | "noul": P(true), between 0 and 1"noul"：P(true)，0 到 1 之间 |
| choice | criteria: {name: description}criteria：{名称: 描述} | "choice" (the most likely name) and "probabilities" per name"choice"（概率最高的选项名）和每个选项的 "probabilities" |
| score | criteria: list of levels, lowest firstcriteria：等级列表，从低到高 | "probabilities" keyed by level index "0", "1", …"probabilities"，键是等级下标 "0"、"1"…… |
Using the probabilities: act automatically above a threshold (e.g. 0.9) and send the rest to a human or a bigger model. On the 231 JevBench public items, s3 was ≥ 0.9 confident on 132 of them and got 97% of those right (all 93 answers at ≥ 0.95 were right).怎么用这些概率：高于某个阈值（比如 0.9）就自动处理，其余交给人工或更大的模型。在 JevBench 公开题上，s3 在 231 道 JevBench 公开题里，有 132 道的置信度 ≥ 0.9，其中 97% 答对；置信度 ≥ 0.95 的 93 道全部答对。
--max-logprobs 256 --return-tokens-as-token-ids.每个请求都返回 422 / 400 → 启动 vLLM 时漏了 --max-logprobs 256 --return-tokens-as-token-ids。nvcc, e.g. slim containers) → add VLLM_USE_FLASHINFER_SAMPLER=0 and --additional-config '{"gdn_prefill_backend": "triton"}'. Neither changes the logprobs.vLLM 编译 kernel 失败（没有 nvcc，比如精简容器）→ 加上 VLLM_USE_FLASHINFER_SAMPLER=0 和 --additional-config '{"gdn_prefill_backend": "triton"}'，两者都不影响 logprob。--max-model-len (e.g. 16384) or add --gpu-memory-utilization 0.85.显存不够 → 调小 --max-model-len（比如 16384），或加 --gpu-memory-utilization 0.85。--served-model-name) → pass the same name to llm2jev with --served-model-name.vLLM 里的模型名不同（用了本地路径或 --served-model-name）→ 给 llm2jev 也传同样的 --served-model-name。Download once at a pinned revision, then serve from the local folder — nothing needs the network afterwards.按固定版本下载一次，之后从本地目录启动，不再需要联网。
hf download everettjf/ezjev-4b-s3 \ --revision 36d180dcdf09ddb7168b06c41f88f1919c20981d --local-dir ./ezjev-4b-s3 vllm serve ./ezjev-4b-s3 --served-model-name ezjev-4b-s3 --port 8000 \ --max-logprobs 256 --return-tokens-as-token-ids --max-model-len 32768 llm2jev --model ./ezjev-4b-s3 --served-model-name ezjev-4b-s3 --backend vllm \ --url http://127.0.0.1:8000 --port 8080 --prompt chat --temperature 1.34
Only the vLLM backend is validated for ezjev. llm2jev also offers SGLang, transformers and MLX backends (untested with this model).我们只在 vLLM 后端上验证过 ezjev。llm2jev 还支持 SGLang、transformers 和 MLX 后端（没有用这个模型测试过）。
No special head and no generation. The question and all options go into one chat prompt; the model reads the logits of the option letters right after Answer:.没有特殊结构，也不生成文字。问题和所有选项放进同一个对话提示词，模型在 Answer: 之后读出各个选项字母的 logit。
LoRA r=16 on every linear layer, including Qwen3.5's Gated-DeltaNet linear attention, merged into full weights.LoRA rank 16，覆盖所有线性层，包括 Qwen3.5 的 Gated-DeltaNet 线性注意力层，训完合并成完整权重。
Cross-entropy + Brier over the option letters only — the same prompt as inference.只看选项字母的 logit：交叉熵加 Brier。训练和推理用同一套提示词。
One global temperature fitted on a held-out dev split after training (1.34 for s3).训练后在留出的开发集上拟合一个全局温度（s3 是 1.34）。
Submission status is above. Two leaderboards with different philosophies. Decision Index rewards breadth (37 public benchmarks plus private tests of the same skills and new domains); JevBench weighs accuracy, calibration, speed and cost equally.提交状态见上方。两个思路不同的榜单：Decision Index 考的是覆盖面（37 个公开 benchmark，加上同类技能和新领域的私有测试）；JevBench 把准确率、校准、速度和成本四项同等看待。
Official result, compared with the next-best 4B entry and with Jev itself. Skill scores, 0 = chance:正式成绩，与第二好的 4B 模型以及 Jev 本身对比。技能分，0 = 随机水平：
Public items answered correctly, by tier (official typesafe adapter, serial requests):各档公开题答对的数量（官方 typesafe adapter，串行请求）：
| ECE | p50 | p95 | $ / 1k每千次 | |
|---|---|---|---|---|
| s2 | 0.059 | 34 ms | 98 ms | $0.027 |
| s3 | 0.044 | 34 ms | 99 ms | $0.027 |
Three stages, all reproducible from the repository on Hugging Face Jobs.一共三个阶段，都能用代码仓库在 Hugging Face Jobs 上复现。
~85k samples / 106k questions from the train splits of public datasets, matched to Decision Index task formats (ContractNLI, RAGTruth, ToolACE/Glaive, When2Call, ESCI, SGD …) plus code-generated BBH / CRUXEval / GSM8K-style items.约 8.5 万条样本、10.6 万个问题，来自公开数据集的 train 部分，按 Decision Index 的题型格式整理（ContractNLI、RAGTruth、ToolACE/Glaive、When2Call、ESCI、SGD……），另加代码生成的 BBH / CRUXEval / GSM8K 风格题。
Low-LR pass on weak tasks (RAGTruth, ESCI, CRUXEval, MMLU, SATA, NLI4CT) plus new phishing-email and claim-verification data, with 40% replay.用较低学习率针对弱项再训一轮（RAGTruth、ESCI、CRUXEval、MMLU、SATA、NLI4CT），另加新写的钓鱼邮件和说法核查数据，40% 回放。
~20k code-generated decisions: multi-clause policies with amendments, business-day / time-zone / leap-year deadlines, pro-rating, alias → record → rule lookups, answer judging, “cannot determine” cases. Gold labels are computed by code.约 2 万道代码生成的决策题：带修订条款的多条件政策、营业日 / 时区 / 闰年期限、按比例计算、别名 → 主数据 → 规则的多跳查表、答案评判、“无法判断”的情况。答案都由程序算出。