Instructions to use kakashi3lite/soulbox-cbt-therapy-0.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kakashi3lite/soulbox-cbt-therapy-0.5b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kakashi3lite/soulbox-cbt-therapy-0.5b")
model = AutoModelForCausalLM.from_pretrained("kakashi3lite/soulbox-cbt-therapy-0.5b", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

MLX

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("kakashi3lite/soulbox-cbt-therapy-0.5b")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

Use Docker

docker model run hf.co/kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kakashi3lite/soulbox-cbt-therapy-0.5b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kakashi3lite/soulbox-cbt-therapy-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

SGLang

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kakashi3lite/soulbox-cbt-therapy-0.5b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kakashi3lite/soulbox-cbt-therapy-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kakashi3lite/soulbox-cbt-therapy-0.5b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kakashi3lite/soulbox-cbt-therapy-0.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Ollama:
```
ollama run hf.co/kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M
```
Unsloth Desktop

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kakashi3lite/soulbox-cbt-therapy-0.5b"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kakashi3lite/soulbox-cbt-therapy-0.5b"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "kakashi3lite/soulbox-cbt-therapy-0.5b"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "kakashi3lite/soulbox-cbt-therapy-0.5b"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "kakashi3lite/soulbox-cbt-therapy-0.5b",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Docker Model Runner
How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Docker Model Runner:
```
docker model run hf.co/kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M
```

Lemonade

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull kakashi3lite/soulbox-cbt-therapy-0.5b:Q4_K_M

Run and chat with the model

lemonade run user.soulbox-cbt-therapy-0.5b-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kakashi3lite/soulbox-cbt-therapy-0.5b"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kakashi3lite/soulbox-cbt-therapy-0.5b

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use kakashi3lite/soulbox-cbt-therapy-0.5b with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kakashi3lite/soulbox-cbt-therapy-0.5b"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kakashi3lite/soulbox-cbt-therapy-0.5b" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

SoulBox CBT Therapy Assistant 0.5B (ultra-light tier)

Fine-tuned Qwen2.5-0.5B-Instruct with DoRA on multilingual CBT synthetic data (Hindi/Marathi/Telugu). Optimized for the tightest memory edge targets (Q4_K_M GGUF ~400MB). Quality is below the 1.5B flagship.

Intended use

Multilingual (Hindi / Marathi / Telugu) CBT-style conversational support for low-resource edge deployment (e.g. Orange Pi Zero 3, ~400 MB Q4_K_M GGUF, or in-browser via WebLLM). Designed as a warm, practical, non-judgmental CBT companion using structured techniques (thought records, cognitive distortions, behavioral activation, Socratic questioning, coping skills). NOT a medical device — it does not diagnose, treat, or replace professional care.

Safety (IMPORTANT)

This model MUST be deployed behind the SoulBox guardrail layer (guardrails/ + scripts/inference.py): crisis / medical / harmful inputs are blocked before the model (recall 1.0 / FPR 0.0 in the shipped e2e test), and outputs are validated after (script purity incl. U+FFFD / foreign-script glyphs, English leak, repetition, prescriptive-output filter). Production inference runs a self-correcting loop — regenerate on failure, reject cross-turn echoes. See kakashi3lite/SoulBoxFT/docs/GUARDRAIL.md for the threat model. The model itself is a language model, not a safety system — never expose it without the guardrail.

Training

Stage	Method	Data	Notes
1 (SFT)	DoRA (weight-decomposed LoRA)	270 gated rows + 30 3-turn conversations	400 steps, bf16 MPS, chunked fp32 CE

Base: Qwen/Qwen2.5-0.5B-Instruct (Apache 2.0)
Teacher (data only): mlx-community/Qwen2.5-7B-Instruct-4bit
Hardware: Apple Silicon (MPS), 24 GB unified memory
DPO was evaluated and dropped (did not learn from near-identical self-consistency pairs) — this is an SFT-only release, honestly documented.

Data

Synthetic CBT conversations distilled from a 7B MLX teacher with best-of-K self-consistency selection (K=3) and a 6-gate validator: script purity (U+FFFD / foreign-script hard-reject), language identity, trigram loops, echolalia, English leak, length. Every prompt is generated in English, native, and romanized input variants. Final dataset: 0 contamination / 0 loops / 0 leak across 270 single-turn + 30 multi-turn rows. See kakashi3lite/soulbox-cbt-therapy-dataset.

Evaluation summary (honest, contamination-aware)

{ "test_perplexity": 1.4129, "gguf_generation": 21, "gguf_generation_n": 21, "stress": { "overall": { "n": 24, "n_pass": 13, "pass_rate": 0.5417 }, "categories": { "novel": { "n": 9, "pass": 8, "pass_rate": 0.8889 }, "code_switch": { "n": 4, "pass": 3, "pass_rate": 0.75 }, "crisis": { "n": 5, "pass": 0, "pass_rate": 0.0 }, "medical": { "n": 3, "pass": 1, "pass_rate": 0.3333 }, "harmful": { "n": 2, "pass": 0, "pass_rate": 0.0 }, "multiturn": { "n": 1, "pass": 1, "pass_rate": 1.0 } } }, "guardrail": { "hard_positive_recall": null, "false_positive_rate": null, "output_filter": null, "e2e_blocked": 10, "e2e_passed": 14 }, "session": { "pass_rate": 1.0, "guarded_rate": 1.0, "n_sessions": 3 } }

Usage (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("kakashi3lite/soulbox-cbt-therapy-0.5b")
tokenizer = AutoTokenizer.from_pretrained("kakashi3lite/soulbox-cbt-therapy-0.5b")

messages = [
    {"role": "system", "content": "You are a CBT therapist assistant. Respond ONLY in Hindi. Be warm, practical, non-judgmental. Do not mention that you are an AI. Avoid medical claims. Keep it concise but helpful."},
    {"role": "user", "content": "मैं काम पर एक छोटी गलती के बाद खुद को असफल मान रहा हूँ।"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=180, do_sample=True, temperature=0.4)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

⚠️ Deploy behind the guardrail (see Safety above).

Usage (llama.cpp / GGUF)

llama-cli -m soulbox_therapy_v2.q4_k_m.gguf   -p "..." -n 180 -t 8 --temp 0.4 --top-p 0.9   --repeat-penalty 1.08 --repeat-last-n 64

Files

model.safetensors — merged weights (SafeTensors)
soulbox_therapy_v2.q4_k_m.gguf — deployable quantized artifact (~400 MB)

Limitations

Synthetic data only; not clinically validated.
Telugu is the weakest language; mixed-script borrowings can appear.
Small-model capacity — best for narrow, structured CBT tasks; multi-turn quality is guarded (3/3 sessions) but not human-level.

License

Apache 2.0 (base model and fine-tune weights).

Downloads last month: 119

Safetensors

Model size

0.5B params

Tensor type

F16

MLX

Hardware compatibility

Quantized

Model tree for kakashi3lite/soulbox-cbt-therapy-0.5b

Base model

Qwen/Qwen2.5-0.5B

Finetuned

Qwen/Qwen2.5-0.5B-Instruct

Quantized

(366)

this model

Quantizations

1 model

Dataset used to train kakashi3lite/soulbox-cbt-therapy-0.5b

Evaluation results

Test-set Perplexity
self-reported

1.413