Umpire 35B-A3B, MLX 4-bit

This is the MLX affine 4-bit checkpoint used to build the Umpire Splash package, a decision fine-tune of Ornith-1.5-35B-A3B.

To run Umpire, use the Splash package. Its card covers the model, results, API example and requirements. This repository is for reproducing weight sharing: serving the fine-tune beside its base without duplicating unchanged weights. That needs MLX checkpoints: a packed Splash package loads as it is, so two different packages never share files, even where their bytes match.

Vision needs our Splash fork. Since 2026-10-02 this repository carries Ornith's vision tower. Image questions on /v1/systemone (an images field) exist only in our fork, and we have served vision only there. On upstream Splash, serve with --language-only. Image decisions are smoke-tested only, not evaluated.

This is an independent derivative, not endorsed by Ornith AI, the OpenJev authors or Inco AI. Licence: CC BY-NC 4.0, non-commercial use only (see NOTICE).

What differs from the base

Compared with ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit:

Component Comparison
Changed attention (self_attn.{q,k,v,o}_proj), linear attention (linear_attn.{in_proj_qkv,in_proj_z,out_proj}) and the shared expert
Byte-identical 1,007 of the language model's 1,757 tensors, 18.72 of 19.51 GB: every routed expert, the router, embedding, head and norms
Layout the base's: MLX affine 4-bit, group size 64, with the router and shared-expert gate at 8 bits
Vision tower added 2026-10-02: model-vision.safetensors (333 BF16 tensors, 0.89 GB) and preprocessor_config.json, byte-identical to ezoushen/ornith-1.5-35b-a3b-mlx-4bit's. The fine-tune did not touch the tower. Serve vision on our fork
Not included the draft (Splash fetches it)
chat_template.jinja renders a system message after the first as a system turn instead of raising, the one-line change incoai's Qwen3.6 Splash package makes (since 2026-09-30)

config.json is the base's with text_config.rope_parameters.rope_type: "default" and Ornith's vision_config added. Splash 1.1.0 requires rope_type; Ornith's config uses type (reported upstream).

Reproducing weight sharing

The upstream run holds two full copies of the weights, about 37 GiB. We ran on a 128 GB Mac.

# The fork; for the "before" run, use https://github.com/incoai/splash.git (main) instead
git clone -b shared-expert-weights https://github.com/ezoushen/splash.git && cd splash && make -j4

# Splash 1.1.0 wants rope_parameters.rope_type, which Ornith's config spells "type"
# (see the note on incoai/splash#38). Patch the cached config once, before the first serve:
CFG=$(hf download ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit config.json --quiet)
sed 's/"type": "default"/"rope_type": "default", "type": "default"/' "$CFG" > "$CFG.new" && mv "$CFG.new" "$CFG"

# Both servers use one fresh weight cache
export SPLASH_WEIGHT_CACHE=$HOME/splash-share-test
./splash serve --model ornith-ai/Ornith-1.5-35B-A3B-MLX-4bit --language-only --max-memory 32G --port 8001
# second terminal, same SPLASH_WEIGHT_CACHE, once the first is ready:
./splash serve --model ezoushen/umpire-35b-a3b-mlx-4bit --language-only --max-memory 32G --port 8002

Our run of these steps on an M5 Max, at the same commits, with this repository's files and ornith-ai's at 19504d9, served from a local Hub cache (language only, before the vision tower was added):

Umpire started beside the base upstream c64a578 fork 830d93b
Files it prepares 42 (embedding, head, 40 layers): 18.20 GiB 40 layer files: 0.79 GiB
Base files it maps the draft only: 0.45 GiB 17.86 GiB (experts, embedding, head, draft)
Weight cache on disk 36.8 GiB 19.4 GiB
Host memory added (one reading) 18.4 GiB 3.7 GiB

The memory figures are single readings of wired + anonymous + compressor pages on a busy machine. Our measured runs, three fresh starts each under a quiet-machine protocol, gave 18.54 GiB upstream against 3.00 GiB with the fork.

Licence

The fine-tuned weights are CC BY-NC 4.0 (LICENSE). The unchanged Ornith weights, vision tower and tokenizer are Ornith AI's, under the MIT License. See NOTICE for attribution and training-data licences.

Downloads last month
109
Safetensors
Model size
35B params
Tensor type
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ezoushen/umpire-35b-a3b-mlx-4bit

Finetuned
(40)
this model

Dataset used to train ezoushen/umpire-35b-a3b-mlx-4bit

Collection including ezoushen/umpire-35b-a3b-mlx-4bit