macrocosm-os/iota-decide-4b-v0.4

iota-decide-4b v0.4

A 4B open-weights decision model by Macrocosmos. Give it a state (a ticket, an agent trace, a policy plus a request, any JSON or text) and a set of typed questions. It returns a calibrated probability for every option, with no text generation: one forward pass per question.

• 78.6 on JevBench, measured by Benchmark Heaven: ahead of every decision model from OpenAI, Microsoft, Google, Cloudflare, Perplexity and Liquid AI on the board.
• Fast and cheap: about 33 ms per decision on one GPU (Benchmark Heaven's measurement), 8.4 GB of bf16 weights.
• Calibrated: every answer comes with a probability you can threshold on.
• Drop-in: requests and responses use the SystemOne /v1/systemone format.
• Trained on disaggregated compute: 16 consumer GPUs (RTX A4000) kept in sync over the internet, no GPU cluster.

Quickstart

pip install torch "transformers>=5" accelerate safetensors huggingface_hub

import sys
from huggingface_hub import hf_hub_download

# the loader ships in this repo
sys.path.insert(0, hf_hub_download("macrocosm-os/iota-decide-4b-v0.4", "letter_decide.py").rsplit("/", 1)[0])
from letter_decide import load_release_model, systemone

model, tokenizer, temperature = load_release_model("macrocosm-os/iota-decide-4b-v0.4")
response = systemone(model, tokenizer, temperature, {
    "model": "iota-decide-4b",
    "state": {
        "ticket": "Customer asks for a refund 45 days after purchase.",
        "policy": "Refunds are allowed within 30 days of purchase.",
    },
    "questions": {
        "eligible": {"type": "noul", "instructions": "Is the customer eligible for a refund under the policy?"},
        "route": {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"refunds": "Refunds and billing team", "tech": "Technical support", "sales": "Sales"}},
        "urgency": {"type": "score", "instructions": "How urgent is this ticket?",
                    "criteria": {"1": "Low", "2": "Medium", "3": "High"}},
    },
})
print(response["answers"])

Question type	What you get
noul	noul: the probability that the statement is true
choice	choice and confidence, plus a probability for every option
score	score (expected level), confidence, and a probability for every level

Yes/no answers are reported with at least 0.801 on the chosen side (JevBench counts 0.2 < P(yes) < 0.8 as an abstention); the answer itself is unchanged. Choice lists take 2 to 26 options. Questions in one request are answered independently, one forward pass each.

Serving: use the bundled loader

v0.4 uses two-way attention: its full-attention layers read the whole prompt in both directions (the linear-attention layers stay causal), so the state is read in the light of the question. letter_decide.py reads "bidirectional": true from joint_head_config.json and builds the attention masks. Plain causal inference (model.generate, stock vLLM or SGLang, llama.cpp) runs the model differently from how it was trained and scores lower, so there is no GGUF build of v0.4. Tested with transformers 5.15.

Results

JevBench (Benchmark Heaven, v1.8.1)

System	Composite score
Torchcast Decision 4B v1.3	80.5
iota-decide-4b v0.4	78.6
H2O Lightning 4B v1.1	77.9
Liquid AI d1-ultra (API)	76.0
Perplexity Decider v1.1 27B	74.9
iota-decide-4b v0.3	72.8
Microsoft Decision-1 (API)	69.1
Cloudflare Clef 27B	69.0
OpenAI Decisions (API)	62.4

The composite is JevBench's official headline: Intelligence, Calibration, Speed and Cost weighted equally. v0.4's axes: Intelligence 60.2, Calibration 87.8, Speed 92.2, Cost 83.0. Benchmark Heaven measured and independently recomputed v0.4 on 4 Nov 2026; its board listing is pending (it was measured on a newer question draw that the board has not yet equated). Other systems' scores are from the v1.8.1 board.

Our own evaluations

Set	v0.3	v0.4	Torchcast Decision 4B v1.3
Cross-domain skill: 15 Decision Index benchmarks, chance-corrected (9,012 questions)	55.4	63.0	65.6
Rotten Tomatoes / TweetEval offensive (human gold, never trained on)	85.1 / 76.4	89.3 / 76.6	90.7 / 79.5
JevBench public split (231 questions)*	188	195	211

* The v0.4 checkpoint was chosen on this split (snapshots every 50 steps), so it is not an independent estimate; the Benchmark Heaven score above is.

Test overlap. We checked the training data against every evaluation set with an 8-word-span overlap test. 387 of the 9,012 cross-domain questions (4.3%, mostly WinoGrande, ANLI, MMLU and ARC) share text with training records. Scoring v0.4 without them gives 62.9 instead of 63.0, so the overlap does not account for the result.

Training

Base	Qwen/Qwen3.5-4B (instruct)
Method	LoRA rank 64 / alpha 128 on every linear layer of the language model, merged into these weights; answers read from the option letters at the answer position
Attention	two-way in the full-attention layers, causal in the linear-attention layers
Objective	soft-target cross-entropy on teacher probabilities, learned temperature
Schedule	LR 1e-4, 2% warmup, cosine, 750 steps
Compute	streaming DiLoCo across 16 RTX A4000 machines (4 fragments, staggered merges, no synchronous waits)
Checkpoint	chosen on the JevBench public split among snapshots every 50 steps

Data. 879k records after de-duplication:

• the v0.3 mix: human-labelled decision rows, 24 public human-labelled datasets (science, medicine, law, math, reading, topics, emotion, paraphrase, entailment, ethics, commonsense) and generated decision states with verified labels;
• a synthetic dataset of LLM-generated decision states and reasoning questions, with LLM-labelled answers.

How it was trained

iota-decide-4b was trained on disaggregated compute: 16 independent consumer GPUs with no GPU cluster, kept in sync over the internet with streaming DiLoCo. It is built with Macrocosmos's IOTA SDK.

Limitations

• Multi-step arithmetic and quantitative reasoning (date windows, pro-rated amounts, word problems) is the weakest area. The model answers in one pass and does not compute step by step.
• Causal and multi-step narrative reasoning (CLadder, MuSR in our cross-domain set) is well below its other skills, as is humour judgement.
• Text only. The vision encoder of the base model is removed.
• Needs the bundled loader for correct results (see Serving).
• English is the only language we evaluated.
• It is decision support. Keep a human in the loop for high-stakes medical, legal, credit or hiring decisions.

Licence

Apache-2.0. The training data includes third-party datasets under their own licences (among them RACE, and MNLI, SNLI and BoolQ under CC BY-SA), used under their respective terms.

Revisions

• Text-only (current): the 333.5M-parameter vision encoder was removed; every language-model tensor is byte-identical to the first v0.4 upload (all 426 checked), and readout logits and answers are bit-identical on JevBench items. The model now loads as Qwen3_5ForCausalLM (8.41 GB instead of 9.08 GB). The first upload, with the vision encoder, stays available at its earlier revision of this repo.