Models

Every model measured

Jev scores 0.753 on the same benchmark. wemm-4b-4bit and minicpm5-2b are built in; any other is one command away, jul models add <name> --repo <repository>, which fits it on the dev sets.

+ autotune is a head on the vectors (features="vector"). With a hybrid head (vectors + TF-IDF, see features=), e5-small — an encoder, see Micro models — reaches 0.790, above Jev's 0.753 (AG News 0.95, Banking77 0.88, Emotion 0.54), at 6 ms per text on an M4 Pro. The other models were not measured with a hybrid head.

Adding a model

jul models add my-model --repo org/Some-Instruct-3B            # on the default backend
jul models add minicpm5-2b --backend torch                     # a known preset, on another backend

One command runs the protocol that produced the built-in presets, on the dev datasets only (never on the Jev benchmark):

1. checks — the model loads, the prefix cache leaves the vectors unchanged, a call does not disturb the next one, the letters reading (Noul, Score) has single-token markers;
2. extraction — 4 dev sets x 50 examples and 200 generic texts, every layer of the upper half read in the same pass;
3. choice — layers and center by dev accuracy averaged over neighbouring layers (a plateau, not a lucky peak), then tau by pooled NLL;
4. output — ~/.jul/presets/<name>@<backend>.json and its generic center, used from then on by --model <name> on that backend. jul models lists it with what was measured.

The calibration data is downloaded once from BTZSC into ~/.jul/calibration-data (needs pip install "jul[calibrate]"), or taken from --data <dir>. The dev accuracy it reports comes with its standard error (±3.5 points at n=200): it orients, it does not rank close models. Measure on the Jev bench separately, once.

Micro models: encoders

An encoder (BERT, XLM-R, multilingual-e5…) is a backbone like any other: jul models add, autotune and jul pack run on it unchanged, on the torch and onnx backends. JuL recognizes one by its model_type, or by use_bidirectional_attention in its config (a decoder turned into an embedding model, as EmbeddingGemma; torch backend only for now), and reads it as it was trained, not as a decoder (lib/jul/encoder.py):

• the vector is the mean of the layer over the whole sequence (the sentence embedding e5 was trained to produce), not a last token; zero-shot is plain embedding similarity between state and options;
• the prompts are the model's own input convention (query: {state} for e5), no chat template and no "in one word" cue (Backbone.templates). A model declares it in its config_sentence_transformers.json ("prompts": {"query": "query: "}); for repos that do not, the official e5 ones included, JuL reads it from a list by repo name, lib/jul/assets/text_prefixes.json, which $JUL_HOME/text_prefixes.json extends or overrides;
• attention is bidirectional, so no prefix can be cached: the prefix runs again with each query, and a sequence is cut to the model's positions (512), the input first, then the prefix;
• no logits: the letters reading and decision models need a decoder.

python -m jul.backends.onnx_export intfloat/multilingual-e5-small models/e5-small-onnx
python -m jul.backends.onnx_export models/e5-small-onnx models/e5-small-onnx-w8 --int8   # 86 MB
jul models add e5-small --repo models/e5-small-onnx-w8 --backend onnx

Why bother: on a support-triage task (jul-lambda, 2026-09-25), multilingual-e5-small (21 M parameters outside its embedding) with a hybrid head matched Harrier 0.6B (440 M) — emotion 0.787 against 0.791, Banking77 0.890 against 0.890 — in 4 ms per message against 37 ms on an M4 Pro, and 17 ms on a 1,769 MB AWS Lambda ($0.59 per million calls, cold start 2.4 s). Zero-shot, on the calibration dev sets, it scored 0.590 against 0.475 for Harrier 0.6B. The heads carry it: alone, zero-shot, a small encoder is no match for a 4B embedding model.

Embeddings APIs (--backend api)

Any embeddings endpoint is read like an encoder: the state's vector is the embedding of the prompt that holds it, the options' are the embeddings of the options. Nothing is downloaded; jul models add fits the center and tau on the calibration sets, as for a local model.

jul models add qwen3-emb --repo ollama:qwen3-embedding:0.6b --backend api      # Ollama, no key
OPENAI_API_KEY=... jul models add oai-small --repo openai:text-embedding-3-small --backend api
jul models add my-emb --repo http://localhost:8000/v1#bge-m3 --backend api     # any OpenAI-compatible server
jul ask choice "Which team?" -o billing -o technical --state "I was charged twice" --model qwen3-emb --backend api

Against a local model: one layer (the API's output), no prefix cache (each state is one call per formulation; the options are embedded once per question), no logits (Noul and Score are read as vectors, their default; letters and pointer need the weights), and usage counts characters. With a hosted provider the state leaves the machine.

Texts are sent in requests of JUL_API_BATCH (16), each cut to JUL_API_MAX_CHARS (8000) characters with a warning, with a timeout of JUL_API_TIMEOUT (300 s). Only Ollama has been run for real; OpenAI, Mistral and Voyage use the same OpenAI-format request (Voyage's input_type is not sent; Mistral caps a request's size).

Cross models: reading the question and the text together

The vector reading encodes the text and each option apart: the model never sees both at once. That is what makes it fast and cacheable, and also why a question about how two things relate — is this a paraphrase of that sentence, does it follow that…, does the request mention a place and not a date — is answered near chance by an embedding model of any size. A cross model is a second small encoder, fine-tuned on pairs: <s> query: question </s></s> text </s> goes through it once, and the mean of its last layer feeds a head chosen by the question type (lib/jul/cross.py):

• Noul: one pass, three logits (yes, no, unknown); noul = p(yes) + p(unknown) / 2. Its own descriptions of true and false, when given, are appended to the question;
• Score: one pass per level, one logit each;
• Choice: supported, not routed by default (the vector reading classifies as well and reads each option once for every call).

It sits next to a vector preset, not in place of it. jul models add attaches one; the types its cross.json declares (Noul and Score) are then read by it, and everything else by the vectors. A question with a tuned head or a calibration from autotune keeps the vector reading those were fitted on, and method="vector" or method="cross" forces one reading for a call.

jul models add jul-decision-e5-small --repo usejul/jul-decision-e5-small-onnx --backend onnx   # cross/ attached
# a cross model of your own: export it like any encoder, then attach it
python -m jul.backends.onnx_export <cross model> models/cross-onnx --layers 11
python -m jul.backends.onnx_export models/cross-onnx models/cross-onnx-w8 --int8 --embedding-bits 4   # 98 MB
cp <cross model>/cross.json <cross model>/cross_heads.npz models/cross-onnx-w8/
jul models add jul-decision-e5-small --backend onnx --cross models/cross-onnx-w8

cross.json holds the input prefix, the length the pairs are cut to (as in training), the layer, the separator between the two segments and the types; jul.cross.write_spec writes it from a trained model.

Which reading answers what:

autotune judges a head for a Noul or Score against the cross model's zero-shot answers on the same examples, not against the vectors' (which are weaker there): if the head wins, the question is answered by it; otherwise it stays with the cross model. Choice stays on vectors because they carry to labels the models never saw — read through a cross model trained on the same data, tool routing went from 0.54 to 0.76 and classification from 0.82 to 0.86, but the zero-shot Jev bench fell from 0.557 to 0.460 (Banking77, 77 intents, 0.59 to 0.35) — and because a vector reading is what heads are trained on. Its own vectors do not survive the cross training (0.61 to 0.39 on the dev sets), which is why the cross model is a second set of weights.

jul pack ships only what its questions need: a bundle of Noul and Score questions (with no tuned head) needs the cross model alone, a bundle of Choice questions the vector model alone (bundle.json, models). On onnx, JUL_ONNX_CROSS_MODEL points the cross graph at S3 as JUL_ONNX_MODEL does the vector one.

Measured on jul-decision-e5-small (its vectors and its cross model, both ONNX 8-bit, M4 Pro), on Kev's typed-decision questions it never trained on (transfer-v9 development split, clean questions):

Paraphrase goes from 0.50 to 0.79, QNLI from 0.55 to 0.70, offensive posts from 0.725 to 0.80. The cross model was trained on relational yes/no (paraphrase, inference, compositions, dates), single-text decisions and questions about the writer's tone: "is the customer angry?" gets 0.84–0.99 on angry support messages and 0.00–0.02 on calm ones. What it does not do well yet: urgency (no training data under a commercial license), hard but polite complaints read as offensive, sentences with the same words in another order, and knowledge questions (MMLU does not move — that needs a larger model, not another reading).

A cross model on the preset's own weights (LoRA)

A cross model can also be a set of adapters on the vector model itself instead of a second encoder: one model in memory for both readings. Its cross.json says "method": "lora" and names LoRA adapters (A and B for each Linear layer, by module path, as transformers and mlx-lm name them) and the heads. The engine attaches them to the backbone it already holds and switches them on only while it reads a Noul or a Score, so the vector reading is unchanged: the features are the same with the adapters attached (tests/test_cross.py). The prompt is Text: "<text>", then Question: <question> (and, for a Score, Candidate answer: <level>), then Verdict:, read on the last token of the last layer after the final norm: one pass for a Noul, one per level for a Score. MLX and PyTorch.

# a directory with cross.json, adapter.npz and cross_heads.npz, on top of the default model
jul models add wemm-4b-4bit --backend mlx --cross models/jul-decision-wemm-4b-4bit
jul ask noul "Was it paid on time?" --state "Invoice due May 9; paid May 3."

jul-decision-wemm-4b-4bit is WeMM-Embedding-4B with such a cross model (rank-16 adapters on its 248 Linear layers: 32.5M parameters, 65 MB in float16; trained on relational yes/no, single-text decisions and scores). Measured on the same Kev questions as above, never trained on:

Paraphrase goes from 0.69 to 0.93 and QNLI from 0.79 to 0.87 (MLX), and Noul's calibration error from 0.126 to 0.043. On date questions worded freely (warranties, trials, bookings, ages, deadlines: 390 cases in English and French), comparing two dates is solved — which comes first, same month, on time: 1.00, against 0.62–0.90 for the vectors — but a gap to compute (an age, a warranty in months, a trial in days) stays near chance, with or without the adapters. That is also why Score barely moves in 4-bit: its questions here are deadlines, close calls that 4-bit weights flip. Choice questions keep the vectors, so the Jev benchmark (0.857 zero-shot) is untouched by construction.

A LoRA cross model can also read a Choice, when its cross.json has a choice entry: the text, the question and every option in one prompt, one pass for any number of options.

Text: "<text>"
Question: <question>
Options:
- <option 1>
- <option 2>
Answer:

Each option is read on the newline that closes it (it has seen the text, the question and the options before it), the answer on Answer:; a bilinear head (choice_q, choice_k in the heads file) scores each option against the answer. jul adds that score to the vector reading, log p = log p_vector + mix · log p_listwise (mix in cross.json, 3 for the default model), because each fixes the other's errors: on the four label sets of the dev, never trained on, the vectors score 0.707, the listwise reading alone 0.723 and both 0.730.

jul-decision-wemm-4b, the default model, is WeMM-Embedding-4B with such adapters for Noul and Choice (usejul/jul-decision-wemm-4b, 65 MB); Score is read by the vectors. They were trained on the bf16 weights and are attached on PyTorch; on MLX the preset reads as wemm-4b-4bit until they are measured in 4-bit. Decision bench, PyTorch on an A10G (2,108 questions, 12 task families, English and French):

Per family: README. The decision bench keeps the questions and label sets of the adapters' training data (only its texts are new). On a bench of 300 typed questions written from scratch for it (sentiment, finance, support, agent routing, moderation), Score read by the adapters fell behind the vectors of the same weights, so since adapter 1525e34 their cross.json no longer lists score:

Decision models

A decision model is a model trained to answer questions about a state, rather than to write text. It reads the state and the options through delimiter tokens it learned, and a small head scores each option against the question. jul runs one with no code of its own: everything that model needs sits next to its weights, in a decision.json (delimiters, layout, readout, head file, temperature, and the longest state and question it was trained on).

jul models add jul-decision-minicpm5-2b --repo usejul/jul-decision-minicpm5-2b-mlx-4bit   # MLX, 1.4 GB
jul models add jul-decision-minicpm5-2b --repo usejul/jul-decision-minicpm5-2b --backend torch
jul ask choice "Which team should handle this ticket?" -o billing -o shipping -o access \
    --state "I was charged twice for order 4411" --model jul-decision-minicpm5-2b

jul-decision-minicpm5-2b is built in (alias fast): TypeSafeClient(model="fast") needs nothing else. Another repo (or a local directory) holding a decision.json brings its own format: no layer to choose and no tau for the pointer head. jul models add still fits the vector reading of the same weights, which long questions (routing by option count) and the types the model routes (routing.types) are read with; --no-routing skips it. The API is the same as for any other model.

jul-decision-minicpm5-2b is MiniCPM5-2B with a merged LoRA and a pointer head (Kev's architecture), trained on human-written typed decisions; its pointer head reads Choice and Noul, and its decision.json routes Score to the vectors, which read it better. On a bench of 300 typed questions written from scratch for it (sentiment, finance, support, agent routing, moderation), PyTorch on an A10G:

The older versions stay on the Hub as revision="v1.0" and "v1.1"; the old repo names redirect. The state is encoded once per call and every question continues from it, so questions never see each other.

The state is paid once per call: a ticket with four questions (two Choice, a Noul and a Score) answers in 180 ms on an M4 Pro, against 65 ms for the first question alone. What costs is the options — they are re-read on every request — so a three-option question runs in 64 ms where a fifty-nine-option one takes 596 ms (v1, M4 Pro). Weights: usejul/jul-decision-minicpm5-2b-mlx-4bit (MLX, 1.4 GB) and usejul/jul-decision-minicpm5-2b (PyTorch, bf16).

autotune(...) works on a decision model through that same vector reading. The head is trained on its vectors, and a question with an active head is read through the vectors plus the head. Questions without a head keep the pointer head. The safety net judges the head against what the question gets without one: the pointer head for Choice and Noul, the vector reading for a routed question (Score on jul-decision-minicpm5-2b). A head that does not beat it stays inactive. A decision model added with --no-routing has no vector reading, and autotune says so.

A state longer than the limit in its decision.json is truncated rather than stretched.

Strands Decider (pointer head at <answer>)

Strands Decider checkpoints (Apache-2.0) are a LoRA on Qwen/Qwen3.5-4B plus a pointer head, released as an archive rather than as a model repo. Their prompt is plain text (<state>…</state><question type="…">…<options>1. name — description…</options></question><answer>), the query is the last token of <answer> and each option is read at the last token of its line. Convert the archive once, then add the directory like any decision model:

python scripts/convert_strands_decider.py strands-decider-4B-hobson-v22.tar ./strands-v22 --sha256 7305d520...
jul models add strands-v22 --repo ./strands-v22 --backend torch

The converter checks the archive, merges the LoRA into the pinned base tensor by tensor (the model is never loaded: 21 s and 2.8 GB of memory for the 4B), and writes decision.json with the four options this format needs: layout: "text" (positions from character spans), a LayerNorm in the head, a temperature per type, and a JSON state rendered as indented JSON. Their defaults are the Kev format's behaviour, so other decision models do not change. Text questions only (no image path). An unofficial candidate: its JevBench numbers are the author's estimate, not ours; scripts/strands_parity.py compares jul with Strands Decider's own code.

Letter-readout decision models (JevK5, Plumb, Quyet, spark-s1, Decision 3.0)

Other open decision models answer with a letter: a causal LM with a merged LoRA, trained to write the letter of an option after its own fixed prompt, read as the softmax of the option letters' logits divided by a calibration temperature (SemIf's protocol). jul's own letters reading uses jul's prompt, which they were not trained on, so jul models add recognises them and reads them the way their own runtime does (lib/jul/letter_models.py). Nothing is fitted:

jul models add jevk5 --repo alibiserikbay/JevK5                 # --backend torch or mlx
jul models add plumb-4b --repo crh225/plumb-4b
jul models add quyet-medium --repo chinhnc/Quyet-1.0-Medium
jul models add spark-s1-4b --repo abhishek085/spark-s1-4b-v6
jul models add d3-edge --repo vllm-sr/d3-edge
jul ask noul "Does the customer ask for money back?" --state "I was billed twice, please refund" --model jevk5

The spec is stored in the preset (letters): the prompt format, the answer letters, how many options one pass reads, the temperature per question type and the input limits. A repo can also ship it as a decision.json with "method": "letters" (same keys: format, letters, max_options, temperature, many_options, prompt_version, limits), which wins over the runtime configs. Each format keeps its runtime's input rules: SemIf refuses inputs over 16,384 tokens, Quyet cuts the state to its token limit (the head kept, the tail for a list), spark-s1 cuts it to 12,000 characters (head and tail kept).

Parity with the runtimes themselves, on the examples of each README (scripts/letters_parity.py, float32 on CPU, same weights): the probabilities agree to within 5e-5 on all four models (13 questions: Choice, Noul, Score, and a 20-option Choice through the knockout). The prompts are the runtimes' token for token; the questions of one call share the state, which runs once as a cached prefix.

Backends. PyTorch and MLX (--backend mlx, the weights read in bf16 as published). On MLX, against the runtimes in float32 (same examples, scripts/letters_parity.py --backend mlx --reference):

Largest probability gap per model, same answer everywhere. MLX is never further from the runtime than torch at the same dtype: the gap is bf16's, not the MLX path's. Quyet's prompts (state cut included) are token-identical with mlx-lm's tokenizer and with transformers'. MLX in float32 was not measured (4B in float32 is about 18 GB).

What they do not do: no vector reading is fitted on their weights, so method= cannot pick another reading, autotune has nothing to train a head on, jul pack refuses them; plumb's JevBench v1.5 settings (yes/no answers moved out of the 0.2–0.8 band, a score temperature of 1.2) are output transforms of its own server for that benchmark, not part of the model, and are not applied. onnx and api are refused.

A state too long for the model's own prompt rule is cut as its runtime cuts it: Quyet keeps the head of the state (the tail of a list) within max_state_tokens, cut once for the whole request so that every question reads the same text; open-spark-jev keeps the head and tail of a state over max_state_chars characters. Like every cut (input limits), it is logged (input cut: <model> letters …), counted in usage.truncated_tokens (open-spark-jev's in the tokens of the dropped middle), and refused by on_long="error". JevK5 and Plumb refuse a prompt over max_tokens (16384), like their runtime: ValueError, HTTP 400 from jul serve. open-spark-jev's abstain is not read: its gateway does not add it on /v1/systemone either.

Decision 3.0 (d3). vLLM Semantic Router's decision models (collection, Apache-2.0) ship a Qwen3.5 backbone, a decision_config.json (format d3-code-readout-v1) and their own runtime as remote code (d3_runtime.py). jul reads the weights without that code. The prompt is d3_format.py's: a fixed system prompt, then State: / Question: / Options: with one line per option under a code (A..Z, AA..: 255 single-token codes, listed in the config), and a closing line. A Choice option without a description shows its key alone, else key: description; a Noul lists false then true, described by the question's criteria or No / false / Yes / true; a Score lists its levels (2 to 10, as the runtime accepts). The reading differs from the other formats in two ways: the hidden state at the last position (after the final norm) is multiplied by the model's readout.safetensors matrix instead of the LM head, and the full-attention layers see the whole prompt (no causal mask; attention_mode: noncausal_full_attention), the Gated DeltaNet layers staying causal. Each question runs its whole prompt (without the causal mask, no prefix can be shared). The checkpoint is the bare multimodal Qwen3_5Model: jul loads its text weights only (the vision tower is not read; images and video, permutation_average and the runtime's AMD kernels are out of scope). jul does not pin Hub revisions: to read the commit measured below, pass the snapshot directory as --repo.

Parity with d3_runtime.D3 (scripts/d3_parity.py, runtime on CPU in float32), on the README's text requests, a 30-option Choice (codes past Z) and 50 decision-bench questions (Choice of 2 to 21 options, Noul, Score, states up to ~4,000 tokens); every prompt has the runtime's token count:

Largest probability gap over 62 questions (argmax agreement in brackets). In float32 jul is the runtime to within float noise on both backends. In bf16 (the weights as published, MLX's default) the gap is bf16's: the runtime in bf16, its own default, is as far from its float32 reading. Against that bf16 runtime, torch bf16 gives the same probabilities (1.5e-7 on d3-edge) and MLX bf16 is 2.7e-2 (d3-edge) / 3.0e-2 (d3-lite) away, the same answer on 61/62 and 62/62 questions. MLX float32 is measured with the weights cast to float32 as they are read (mlx-lm adds 1 to Qwen3.5's norm weights while loading, in the stored dtype). The script's default tolerance follows the dtype: 5e-3 in float32, 5e-2 in bf16.

Laya

Laya (ConvAI Innovations, Apache-2.0) is a decision model with its own format and its own runtime, the laya package (PyTorch). JuL does not read its weights: it hands the questions to Laya and returns its answers as JuL's, so Laya takes the same calls, the same response and the same commands as any other model (jul ask, run, serve, bench, an Escalation tier).

pip install "jul[laya]"            # or: jul setup --model laya
jul ask choice "Which team should handle this ticket?" -o billing -o technical \
    --state "I was charged twice" --model laya

Laya picks the device (CUDA, MPS, then CPU) and pins the Hub revision it downloads; the first call downloads the checkpoint. What JuL adds on its own readings does not apply: no method, no heads or calibration from a context, no autotune, no pack (each raises a ValueError saying so), and --backend other than torch is refused. The answers are Laya's own: its Choice and Score confidence is calibrated by Laya, not the top probability JuL's readings report, so a bar set for one model in an Escalation does not carry over to the other.

Contrastive heads (any backbone; CLM-8B)

A contrastive preset reads a frozen backbone through two small projection heads: the state (with the question's instructions after a blank line) goes through the state head, each option through the action head, the score is scale * cosine and a softmax gives the answer. It is a way of reading a model, not a model: the backbone is any repo jul loads (MLX, torch or onnx, decoders and encoders), and contrastive.json says how it is read:

The heads only mean something on the backbone they were trained on. Train them on any backbone from labeled rows ({state, type, question, options, answer}, or autotune's {state, answers} with --questions); the backbone stays frozen and torch is needed for the training only:

jul models add my-heads --repo Qwen/Qwen3-0.6B --backend torch --train-heads rows.jsonl
python -m jul.contrastive train Qwen/Qwen3-0.6B rows.jsonl out/   # same, without a preset

The loss is CLM's InfoNCE (the question's own options, plus the batch's other options as negatives), the epoch kept is the best on held-out rows, and the plain cosine of the same embeddings is reported next to it (training.json).

Measured once (decision-bench v1, 2,108 scored items, torch CPU): heads trained on a frozen Qwen3-0.6B from 5,974 rows of the decision dataset's train split score 0.378, against 0.337 for the plain cosine of the same embeddings (choice 0.218 / 0.151, noul 0.570 / 0.542, score 0.294 / 0.297), 542 ms p50. That is a real gain over the cosine but far below the fitted presets (wemm-4b-4bit 0.682): the mode works on any backbone, it has not yet been measured on an embedding model or with more data.

Known limits:

• Reading without last_hidden: default_reading probes last_hidden on one token. A backend that does not expose the final-norm hidden states (onnx) or returns something unusable is read at its last layer before the final norm, with a warning in the jul.contrastive log. Heads trained that way only fit that reading; heads that need "final" on such a backend fail with an error naming the backend.
• In-batch negatives: the other questions' options in a batch count as negatives with weight 0.5 (train_heads(in_batch=...)), CLM's choice. Equivalent texts across questions become false negatives; the weight is not exposed in the CLI.
• --train-heads with --cross: the cross model is not attached to the trained preset.
• Overwrite: ~/.jul/heads/<name> is replaced without warning when the same name is trained again.

CLM-8B

CLM-8B is not a model of its own: two small projection heads (19 M parameters, 75 MB) trained with InfoNCE on top of a frozen Qwen3-8B, read at its last token. The state, with the question's instructions after a blank line, goes through the state head; each option through the action head; the score is scale * cosine and a softmax gives the answer. The heads mean nothing without that exact backbone, so jul runs Qwen3-8B itself and reads the heads in numpy (no torch at inference, no vLLM server):

jul models add clm-8b --repo Contrastive-LM/CLM-v0.1-8B    # converts the .pt once (needs torch)
jul ask choice "Which team should handle this?" -o "billing:Charges, invoices, refunds" \
    -o "technical:Bugs and outages" --state "my invoice was charged twice" --model clm-8b

jul models add recognises a CLM repo by its config.json ("model_type": "clm"), converts the checkpoint into ~/.jul/heads/<name>/ (contrastive.json + heads.npz) and writes a preset whose repos are the backbone's: Qwen/Qwen3-8B (bf16, 16 GB) on torch, mlx-community/Qwen3-8B-8bit (8.7 GB) on MLX. python -m jul.contrastive convert <.pt or repo> <dir> --backbone-mlx <repo> picks another backbone; jul models add <name> --repo <dir> then registers that directory.

The texts are CLM's own (schema.build_pairs), not jul's formulations: objects rendered as key: value fields, options verbatim, a Noul without descriptions read as CLM's "true: Yes. This is true: <question>". On CLM's reference requests the MLX 8-bit backbone answers as its bf16 vLLM server does (M1 Pro):

4-bit keeps every argmax but moves the Noul by 0.13, hence 8-bit by default. A warm call with the options cached takes ~210 ms on an M1 Pro: every question embeds state + instructions once, since the state head reads the question too.

autotune works: the head is trained on the encoder embedding of state + instructions (the vector the state head reads), judged against the CLM heads' own answers, and a question keeps CLM's answer when the head does not beat it. jul pack does not apply (vector method only).

On the Jev bench (scripts/bench_jul.py clm-8b zero-shot,tuned, M1 Pro, MLX 8-bit):

Zero-shot it collapses onto a few labels (Banking77's 77 options, Emotion), as it does behind its own vLLM server (0.53 / 0.08 / 0.23 measured there, torch bf16 on CPU): the port is faithful, the heads are what they are. Jev scores 0.753, wemm-4b-4bit 0.857.

The built-in presets

wemm-4b-4bit is the most accurate vector reading: 10 points above Jev with no training. The default, jul-decision-wemm-4b, is the same preset plus LoRA adapters that read Noul and Choice with the question and the text together; they are attached on PyTorch, and on MLX it reads exactly as wemm-4b-4bit until they are measured in 4-bit. On the same M4 Pro it is 2.3 times slower than minicpm5-2b.

A third option does not read a general model at all: jul-decision-minicpm5-2b is MiniCPM5-2B trained to answer typed questions (a merged LoRA and a pointer head), built in as the alias fast: as accurate as the default on our 300-question bench (0.680 against 0.677) at half its size. Its pointer head brings its own format; Score goes to the vector reading of the same weights, shipped fitted per backend (the layers and tau in the table are that reading's, on MLX). autotune(...) trains its heads on that same vector reading: a question with an active head is read through it, the others keep the pointer head (see Decision models).

Input limits

Every reading takes a bounded input. Past the limit the end of the text is dropped: an answer that sits there is never seen by that reading. The cut is logged each time on the jul.truncation logger (jul serve prints it), naming the reading, its limit and the tokens dropped, and usage.truncated_tokens in the response is the largest cut of the call (0 when nothing was cut).

A state longer than its cross.json limit is therefore answered from its beginning only on jul-decision-wemm-4b for Noul and Choice, while Score (vector reading) sees it whole. The limits come from the files next to the weights, so they follow the Hub: read them there rather than from this page.

on_long decides what a cut of the state does, per call (system_one(..., on_long=), the on_long field of a jul serve request), per client (TypeSafeClient(on_long=)) or by default (JUL_ON_LONG):

Only the state is refused: an option description cut to a cross model's max_option is logged and counted in usage.truncated_tokens, the call is answered. error refuses after the call was read, so a refused call costs as much as an answered one. The refusal is a jul.truncation.InputTooLong, a ValueError.

jul bench --on-long error counts the rows a model refuses in refused_rows, and computes accuracy (and n) over the rows it answered. With an escalation (--escalate-to), a refusal by the local tier is raised, not sent to the next tier: error means the call is refused.

How it answers

For each formulation of the preset, the state and every option go through the same prompt; the answer is the option whose vector is closest (cosine, after subtracting a center). The scores of the two formulations are averaged, then softmax(cosine / tau).

1.  This text: "{state}" means in one word: "
2.  {instructions}\nPossible answers: {options}.\nText: "{state}"\nIn one word, the answer is: "

The model never writes anything. Everything state-independent — the prompt prefix, the option vectors, the center — is computed once, so a call only pays for its own tokens.

Every reading, and every setting

Five ways to read a model. A preset picks one; a call may override it.

A decision model may also route by option count: below the threshold the pointer head answers, above it the vector reading does. The pointer reads every option on every call, so its cost grows with the option list (65 ms at 3 options, 868 ms at 59) while the vector reading is flat; past ~30 options it stops earning that latency. Routing is per question, so one call can mix both. Measured on massive, 59 options, same weights: 868 ms → 77 ms at equal accuracy.

Routing trades accuracy for speed, and the trade is not free. Measured on massive by subsampling one set's own options — so the option count is not confounded with the task — the pointer head is better at every count, by about 5 points, while costing 105 ms at 3 options and 832 ms at 59. There is no count above which it stops earning its answer; there is only a count above which its speed stops being worth those points.

So jul models add measures the threshold rather than guessing one: it takes the smallest option count at which the vector reading is three times faster, and records in the preset what that costs in accuracy. A model where that never happens gets no routing at all. --route-above N sets it by hand and --no-routing skips the whole fitting; the model's own decision.json may also carry a threshold.

A decision model may also route by question type: its decision.json lists in routing.types the types its pointer head reads worse than the vector reading of the same weights, and those questions go to the vectors at any option count (route_above=0 does not change that). The fallback is the one jul models add fits anyway.

For maximum accuracy, pass route_above=0 on the call: every question then goes to the pointer head, whatever its option count, and you pay the latency in the table above. The default is a compromise, and jul models add prints exactly what it costs on the dev set it measured.