Instructions to use GEHIRNS/magi-1.3-1.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use GEHIRNS/magi-1.3-1.7B with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS # Run inference directly in the terminal: llama cli -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS # Run inference directly in the terminal: llama cli -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS # Run inference directly in the terminal: ./build/bin/llama cli -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Use Docker
docker model run hf.co/GEHIRNS/magi-1.3-1.7B:IQ2_XXS
- Llama new
- LM Studio
- Jan
- vLLM
How to use GEHIRNS/magi-1.3-1.7B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "GEHIRNS/magi-1.3-1.7B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "GEHIRNS/magi-1.3-1.7B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/GEHIRNS/magi-1.3-1.7B:IQ2_XXS
- Ollama
How to use GEHIRNS/magi-1.3-1.7B with Ollama:
ollama run hf.co/GEHIRNS/magi-1.3-1.7B:IQ2_XXS
- Unsloth Desktop
- Pi
How to use GEHIRNS/magi-1.3-1.7B with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "GEHIRNS/magi-1.3-1.7B:IQ2_XXS" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use GEHIRNS/magi-1.3-1.7B with Docker Model Runner:
docker model run hf.co/GEHIRNS/magi-1.3-1.7B:IQ2_XXS
- Lemonade
How to use GEHIRNS/magi-1.3-1.7B with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Run and chat with the model
lemonade run user.magi-1.3-1.7B-IQ2_XXS
List all available models
lemonade list
- Hermes Agent
How to use GEHIRNS/magi-1.3-1.7B with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use GEHIRNS/magi-1.3-1.7B with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf GEHIRNS/magi-1.3-1.7B:IQ2_XXS
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "GEHIRNS/magi-1.3-1.7B:IQ2_XXS" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download README.md from GEHIRNS/magi-1.3-1.7B: direct link, hf CLI and curl.
- Browser
- Download file 26.4 kB
-
https://huggingface.co/GEHIRNS/magi-1.3-1.7B/resolve/main/README.md
- Command line
-
hf download hf://GEHIRNS/magi-1.3-1.7B/README.md
-
curl -L -o README.md https://huggingface.co/GEHIRNS/magi-1.3-1.7B/resolve/main/README.md
license: apache-2.0
license_link: https://huggingface.co/LiquidAI/LFM3-1.7B/blob/main/LICENSE
base_model: LiquidAI/LFM3-1.7B
base_model_relation: finetune
library_name: gguf
pipeline_tag: text-generation
language:
- en
tags:
- gguf
- lfm3
- magi
- mixed-precision
- quantized
- llama.cpp
- ollama
- reasoning
- conv
- dpo
metrics:
- type: index
value: 72.45
name: Full2313 Macro Index @863 MiB
- type: id
value: 2.207
name: Information Density
model-index:
- name: magi-1.3-1.7B
results:
- task:
type: full2313
name: Full Bench 2313
dataset:
type: custom
name: full2313 12 suites think 4096
metrics:
- type: index
value: 72.45
- type: id
value: 2.207
- task:
type: screen250
name: Screen 250
dataset:
type: custom
name: screen250
metrics:
- type: index
value: 73.73
- type: id
value: 2.288
Paper: Where the Bits Go (Sun, 2026) · Recipe: GEHIRNS/magi · True params 1728801536 - GGUF base IQ2_XXS with 167-type mixed override · DOI 10.5281/zenodo.23630199
Introduction
magi-1.3-1.7B is LFM3-1.7B in 863 MiB at 32K context, highest capability density we have measured: rho=2.207 on full 2313-task bench, vs 1.305 for magi-1.2-2B, 1.149 for magi-1.1-2B and 0.385 for Qwen3.5-2B BF16, all same settings. Scores 72.45% on 2313 tasks in 0.84 GiB, 5.36 points higher than last gen 2B in 400 MiB less memory, thinks 41% less than magi-1.2-2B and 57% less than magi-1.1-2B.
LFM3-1.7B base is LiquidAI hybrid: 24 layers = 13 conv shortconv.conv q8_0, 6 full_attn 2,6,10,14,18,22 q4_k, 5 sliding w256, hidden 2048, 16 heads 4 kv, intermediate 7680, vocab 128k, max 262k, KV 113MB @32K q4_0 28MB @8192 per slot. No layers pruned after imatrix — all 24 blk intact, imatrix only quant importance.
Two steps:
- SFT commonsense-heavy: QLoRA r16 a32 on w1,w2,w3,q_proj,k_proj,v_proj,o_proj,gate_proj,in_proj,out_proj, 1696 traces 88% commonsense (hellaswag 1400 winogrande 600 obqa 400 sciq 400 arc 400 mmlu 300 gsm8k 100), loss 0.668->0.389, adapter 63MiB.
- DPO v1 best: beta0.1 + NLL1.0 lr2e-5 grad_accum8, 607 pairs hellaswag 409 mmlu88 obqa78 gsm8k48 sciq27 arc25, 76 steps loss 13->7 ref logps precomputed. Screen 73.73% ID2.288 delta+1.92 vs SFT 71.81% W20 L15. v2/v3 regressed 72.5%.
Capability density
rho = -log2(1-score)/GiB. Measured same way llama.cpp b11800 q4_0 KV -ub 128 8192 ctx per slot, memory at 32K from fit.log: 863 MiB = 700 model+100 ctx+63 compute. fit.log: CUDA0 RTX 5070 12226=10965+(863=700+100+63)+... Host 0.
Benchmarks
Protocol: 2313 tasks full + 250 screen, 12 suites, think 4096 greedy, q4_0 KV flash-attn ub128 num_ctx 8192 per slot, 6 lanes max efficiency 99 overall TPS @2.2GB (282 eval), 1 lane 6.3s/task 325 eval TPS 648 overall for stability, 7 lanes 2.5-2.9GB 65 overall 218 eval, 8 lanes deadlock 0 tasks @2698MiB 82% util (parallel scheduler circular wait). Base LFM3 BF16 3.3GB 3298MiB 16bpw, Q4_K_M 1.1GB, magi IQ2_XXS 871MiB 4.19bpw; Qwen3.5-2B BF16 4.72 GiB; magi-1.2-2B 826MB Ollama 789M GGUF 1.23GiB; magi-1.1-2B 1.36GiB.
| Benchmark | Qwen3.5-2B BF16 4.72 GiB | LFM3-1.7B BF16 3.22 GiB | magi-1.1-2B 1.36 GiB | magi-1.2-2B 1.23 GiB | magi-1.3-1.7B 0.84 GiB |
|---|---|---|---|---|---|
| Efficiency | |||||
| Memory at 32K (GiB) | 4.72 | 3.22 | 1.36 | 1.23 | 0.84 |
| Full2313 score | 71.66% | ~74.5% scr | 66.06% | 67.09% | 72.45% |
| Screen250 score | ~72% est | 74.54% | 66.1% est | ~67% est | 73.73% |
| Capability density rho | 0.385 | 0.613 | 1.149 | 1.305 | 2.207 |
| Decode speed (tok/s) eval | - | ~300 | 325 | 333 | 325 |
| Mean think tokens | 3,018 | ~1,600 | 3,409 | 2,499 | 1,476 |
| Hit 4K budget | 62.4% | ~35% | 80.7% | 48.2% | 13.5% 312/2313 |
| Traces ending loop | 49.2% | ~20% | 77.4% | 38.8% | ~8% est |
| Coding | |||||
| HumanEval+ | 56.10 | ~86% scr | 46.95 | 50.61 | 86.6 |
| MBPP+ | 55.64 | ~68% | 44.61 | 44.86 | 68.2 |
| LiveCodeBench | 11.33 | ~25% | 1.33 | 8.00 | 23.3 |
| Math / Science | |||||
| GSM8K | 82.67 | ~91% | 78.00 | 76.67 | 90.7 |
| ARC-Challenge | 94.67 | ~90% | 93.33 | 96.00 | 88.7 |
| MMLU | 74.50 | ~60% | 64.00 | 65.50 | 59.0 |
vs last gen 2B: +5.36 pts vs magi-1.2-2B, +6.39 vs 1.1-2B, +~9 vs 1.0-2B, 0.39 GiB less, 1.69x density. vs base IFM LFM3: 74.54% BF16 ->72.45% =97% kept 3.8x less memory 3.6x density. vs Qwen base: 71.66%->72.45% +0.79 pts 5.6x less 5.7x density. LFM3 conv wins coding HE 86.6% vs 50.6% for 1.2-2B.
Extra graphs — TPS + thinking efficiency
Thinking tokens are mean over 2313 tasks: Qwen 3018, LFM3 ~1600, 1.1-2B 3409, 1.2-2B 2499, 1.3-1.7B 1476 — 41% less than 1.2-2B, 57% less than 1.1-2B, due to SFT on short correct traces + DPO + LFM3 conv architecture (shortconv 7 L-cache reduces loop). Cap hits 13.5% vs 48.2% vs 80.7% — DPO loop fix works. TPS: 1 lane 6.3s/task 325 eval, 6 lanes 2.2GB 99 overall 282 eval max efficiency (per 9GB ceiling), 7 lanes 2.5-2.9GB 65 overall 218 eval, 8 lanes deadlock 0 tasks 2698MiB — Ollama parallel scheduler circular wait all slots prompt-eval. Must set OLLAMA_NUM_PARALLEL=6 before serve.
Recipe
Per-tensor types LFM3-1.7B 24 layers: 13 conv, 6 full_attn 2,6,10,14,18,22, 5 sliding w256.
| Tensor group | Types | Share | bpw | vs 2B last gen 187 types | vs 1.2-4B 249 types |
|---|---|---|---|---|---|
| FFN down/gate/up 24 layers | IQ3_XXS x82 IQ4_XS x15 | 48% | 3.3 | Same 82 iq3_xxs as 2B +2 iq4_xs vs 2B | -49 iq3_xxs vs 4B -4 iq4_xs |
| Token embd | Q4_K x1 | 27% | 4.5 | vs q3_k in 2B (q3_k->q4_k) | Same |
| Full-attn q,k,v,o+gate 6 layers | Q4_K x31 | 12% | 4.5 | vs 37 q4_k +11 q5_k in 2B simpler no q5_k | vs 32 q4_k +8 q5_k in 4B |
| ShortConv conv/in/out 13 layers | Q8_0 x39 | <1% | 8.5 | vs 36 q8_0 ssm in 2B +3 | vs 48 q8_0 in 4B -9 |
| Norms | F32 | <0.1% | 32 | Same | Same |
Quantizer llama-quantize --pure b4706 base IQ2_XXS override lfm3-magi.types.txt 167 (vs 187 for 2B, 249 for 4B). Imatrix imatrix-lfm3-own.gguf 3.5MB PPL2.3983 calib-4b.txt 22x8192.
No layers pruned: all 24 blk intact. Imatrix only quant importance.
Training
SFT LoRA r16 a32 w1,w2,w3,q_proj,k_proj,v_proj,o_proj,gate_proj,in_proj,out_proj 1696 commonsense-biased loss 0.668->0.389 adapter 63MiB chunked CE 128 QLoRA nf4 double quant bf16 checkpointing. DPO v1 beta0.1 NLL1.0 lr2e-5 grad_accum8 607 pairs 76 steps loss 13->7 ref logps precomputed. v2/v3 673/676 rows regressed.
How to run
llama.cpp: llama-server -m magi-1.3-1.7B-IQ2_XXS.gguf -c 32768 --jinja --flash-attn on --cache-type-k q4_0 --cache-type-v q4_0 -ub 128 -np 6
Ollama: $env:OLLAMA_NUM_PARALLEL='6'; $env:OLLAMA_MAX_QUEUE='12'; ollama serve; ollama create magi-1.3-1.7B -f Modelfile.lfm3-r36-dpo-v1-full; ollama run magi-1.3-1.7B
Limitations
LFM3 not Qwen: conv vs Gated DeltaNet cross-arch shows conv wins coding HE 86.6% vs 50.6% for 1.2-2B and GSM8K 90.7% vs 76.7%. Below BF16 screen 74.54%->72.45% 97% kept 2.1 pts gap hellaswag -10 winogrande -4.5, LCB 23.3% actually better than Qwen BF16? DPO helps. Still loops much less 13.5% vs 48%. Selected on screen250 v1 best confirmed full2313. Narrow eval one runtime.
Quant structure summary
| Version | Arch | Layers | Types | GGUF | BPW | True Params | Notable |
|---|---|---|---|---|---|---|---|
| magi-1.0-2B | Qwen2.5 hybrid | ~28 | 187 (82 iq3_xxs 37 q4_k 36 q8_0 13 iq4_xs 11 q5_k 7 iq2_s 1 q3_k) | 789M | ~3.4 | 1.88B | q3_k token embd |
| magi-1.1-2B | Qwen3.5-2B | 24 | 187 same | 895M | ~3.5 | 1.88B | +DPO loop |
| magi-1.2-2B | Qwen3.5-2B | 24 | 187 same breakdown 2B-fitted map | 826M Ollama 789M | ~3.4 | 1.88B | 39 groups moved |
| magi-1.2-4B | Qwen3.5-4B | ~48 | 249 (131 iq3_xxs 48 q8_0 32 q4_k 19 iq4_xs 11 iq2_s 8 q5_k) | 1.8G | ~3.4 | ~4.2B | +62 types |
| magi-1.3-1.7B | LFM3-1.7B conv | 24 (13c 6f 5s) | 167 (82 iq3_xxs 39 q8_0 31 q4_k 15 iq4_xs) | 871M 913MB Ollama | 4.19 | 1728801536 | No iq2_s/q5_k/q3_k simpler higher BPW 863MiB fit |
Related
GEHIRNS/magi-1.2-2B last gen 2B 67.09% @1.23 GiB rho1.305 187 types 826MB, magi-1.1-2B 66.06% @1.36 GiB rho1.149, magi-1.0-2B recipe alone, magi-1.2-4B 4B 249 types 1.8G M64x map, LiquidAI/LFM3-1.7B base 24 layers hybrid true 1728801536 128k vocab 262k max, Qwen/Qwen3.5-2B Qwen base 1.88B true, Paper Where the Bits Go DOI 10.5281/zenodo.23630199
License
Apache 2.0 inherited from LiquidAI/LFM3-1.7B. See base LICENSE.




