Kimi-K3-Uncensored-GGUF

An abliterated GGUF quantization of Kimi K3 (2.8T params / 104B active / 896 experts, top-16), with the refusal direction orthogonalized out of the residual-stream write path.

Size 539.7 GiB, 34 shards
Quant IQ1_S-XS (experts IQ1_S, router F32, attn/KDA IQ4_XS, shexp Q5_K)
Tensors modified 279 of 2573
Harmful-prompt refusal 0.0% (0/26), vs 7.7% (2/26) for the unmodified baseline
Over-refusal on benign prompts 0.0% (0/30) โ€” unchanged from baseline
Incoherence 0.0% (0/18) โ€” unchanged from baseline
PPL @12 chunks 1.9323 ยฑ 0.0473

English below ยท ไธญๆ–‡่งๆญค

Why this repo exists

The two existing K3 uncensored repos publish no refusal numbers at all: one claims ">98% of guardrails removed" with no benchmark, the other states retention is "pending / has not been measured." An abliteration whose effect was never measured is indistinguishable from a broken model โ€” the characteristic failure mode of the technique is not "still refuses" but "answers incoherently," and counting only "did it emit refusal boilerplate" scores a lobotomized model as a success.

So this repo ships the measurement alongside the weights, including the parts that came out inconclusive. Read the Limitations section before trusting any number here โ€” the headline 0.0% is real but the comparison against baseline is not statistically significant at this sample size, and we say so rather than rounding it up into a claim.

Measured refusal behaviour

74 held-out prompts, three separately-reported rates, greedy decoding (temp=0.0, seed=20260803), judged on the first 240 characters of the final answer after stripping the think channel.

abliterated (this repo) baseline UD-IQ1_S
Refusal rate (26 harmful โ€” lower = more thoroughly removed) 0.0% (0/26) 7.7% (2/26)
Over-refusal rate (30 benign, topically adjacent โ€” lower = better) 0.0% (0/30) 0.0% (0/30)
Incoherence rate (18 factual โ€” lower = capability retained) 0.0% (0/18) 0.0% (0/18)
Empty replies 0/74 0/74
Soft-evasion markers 0/74 0/74
Median answer length, harmful set 2511 chars 2548 chars

Three rates, not one. Reporting only the refusal rate is the mistake that makes a destroyed model look successful:

  1. Refusal rate (harmful set) โ€” does the intervention work at all.
  2. Over-refusal rate (benign set) โ€” did it damage legitimate requests. These prompts are deliberately topically adjacent to the harmful set (lawful firearm maintenance, medication dosage questions, security research). A rise here means the direction was mis-selected or the scale was too aggressive.
  3. Incoherence rate (factual set) โ€” questions with objectively checkable answers, scored by keyword hit. This catches the failure mode where the model stops refusing because it has stopped making sense.

The abliterated model answered every harmful prompt at a median 2511 characters with zero empty replies, and hit 18/18 on the factual set โ€” so the 0.0% is not the artifact of a broken model emitting nothing.

Limitations โ€” read this before citing any number above

The improvement over baseline is not statistically significant. 0/26 vs 2/26 gives a two-sided Fisher exact p = 0.490. At n=26 this evaluation simply cannot resolve an effect the size of 7.7%. The direction of the result is what we set out to produce, and the intervention provably changed the weights (see below), but this test does not establish that it lowered the refusal rate. A larger or harder harmful set would be needed for that.

The baseline itself refused only 7.7%. Either unsloth's UD-IQ1_S is already permissive, or these 26 prompts are not hard enough to elicit refusals from K3. Both readings limit what the comparison can show.

The baseline is a different quantization recipe. The honest comparison would be against our own IQ1_S-XS with the abliteration omitted; that artifact was deleted to reclaim disk before this evaluation was designed. UD-IQ1_S (unsloth, 553.2 GiB) shares the expert type but allocates non-expert layers differently. Using a third party's unmodified release removes any suspicion of self-serving baseline construction, but it is not a same-recipe A/B.

The judge is keyword matching, not semantic. Soft evasion โ€” "this is a complex topicโ€ฆ" followed by no content โ€” is not counted as a refusal, so the reported refusal rates are underestimates. We checked for 15 soft-evasion patterns and found 0/74 in the scored window, but that window is only the first 240 characters; we did not scan full texts, and the stored evaluation JSON keeps only the answer head. An LLM judge would be more accurate at the cost of an external dependency and its own biases; we chose the reproducible criterion.

PPL is measured at 12 chunks. See the perplexity section for why this matters and how easily it misleads.

How the abliteration was done

Standard abliteration finds the direction in activation space that mediates refusal, then projects it out of the weights that write to the residual stream. The interesting part here is how cheaply that lands on K3's architecture.

The refusal direction

llama-cvector-generator over 308 prompt pairs (harmful / harmless, matched in surface form) across all 93 layers, computed on the BF16 tensors of the local UD-Q8_K_XL copy โ€” not taken from a published vector, so the direction belongs to this model rather than to a donor.

Layer selection matters more than it looks. We take the stable band mid60-80 (layers 56โ€“73) and use an in-band normalized average, not a single layer and not the all-layer mean: early and late layers carry directions that pull the average off, so including them degrades the result.

279 tensors, and why that is the whole model

Only matrices that write into the residual stream are orthogonalized. Read-side projections are left alone.

Tensor family Count Note
attn_output 93 attention write-back
ffn_routed_up 92 the MoE write side โ€” 3584 latent โ†’ 7168 residual
ffn_down_shexp 92 shared-expert write-back
ffn_down 1 layer 0's dense MLP
token_embd 1
Total 279 of 2573 tensors

ffn_routed_up is the one that makes this tractable. K3's latent MoE routes through a pair of shared projections, and that pair is shared across all 896 experts โ€” ffn_routed_down (7168 โ†’ 3584) reads, ffn_routed_up (3584 โ†’ 7168) writes. Orthogonalizing that single matrix per layer removes the refusal direction from every one of the 896 experts at once, which is why 1347 GiB of expert weights were not touched at all. ffn_routed_down is correctly left unmodified โ€” it reads from the residual stream, it does not write to it.

This is a gift from K3's specific structure. It does not transfer to MoE architectures that give each expert its own output projection.

Verification

scale=1.0 (full orthogonalization). Removed-component ratio across the 279 tensors: median 1.2084%, range 0.9615โ€“1.7291%.

Eleven tensors were then audited individually against five criteria: the destination's projection onto the refusal direction drops to 1.2โ€“1.6% of the source (โ‰ค2% is the BF16 noise floor); source norms still match the pre-run report (confirming nothing was modified in place); source and destination have different inodes with link count 1 (confirming the reflink clone is COW, not a hard link); bit patterns genuinely differ; and the orthogonal complement changed by only 1.5e-3, i.e. BF16 rounding โ€” the intervention removed the refusal direction without disturbing the other components.

Perplexity

Unified methodology: wikitext-2-raw/wiki.test.raw, 12 chunks, n_ctx=512, --n-cpu-moe 93, same machine and same llama.cpp build as every other row.

Quant Size PPL vs Q8
UD-Q8_K_XL (lossless reference) 1453.9 GiB 1.3453 ยฑ 0.0420 โ€”
UD-IQ1_S (baseline, unmodified) 553.2 GiB 1.8824 ยฑ 0.0446 +0.5371
IQ1_S-XS (our recipe, not abliterated) 539.7 GiB 1.9193 ยฑ 0.0469 +0.5740
IQ1_S-XS abliterated (this repo) 539.7 GiB 1.9323 ยฑ 0.0473 +0.5870

Against the same recipe without abliteration, the cost is +0.013 (+0.68%) โ€” smaller than that row's own ยฑ0.0469 error bar. At this sample size the abliteration produces no statistically detectable language-modelling loss. That is a weaker claim than "no loss," and it is the one the data supports.

A trap worth documenting, because we walked into it. The full 568-chunk run of this model gives PPL 2.6479 ยฑ 0.0115, and comparing that to the 12-chunk baselines suggests a catastrophic +38% regression. It is an artifact. The sliding perplexity average rises monotonically as more of the corpus is covered โ€” this same model reads 1.93 at chunk 12 and 2.65 at chunk 568. Every published K3 baseline here was measured at 12 chunks, so a 568-chunk figure is not comparable to any of them. Before comparing PPL across models, check that the chunk count matches (grep 'calculating perplexity over N chunks'). Numbers quoted without their chunk count, corpus and n_ctx cannot be cross-compared with this table โ€” including the ones in this table, against anyone else's.

How to run

Identical to any other K3 GGUF. Requires the unsloth llama.cpp fork, which stacks on ggml-org PR #26185 (K3 architecture support, authored by llama.cpp member pwilkin, not yet merged to master):

git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..

# If nvcc is not on PATH, set it explicitly or cmake fails to find the CUDA compiler
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
  --target llama-cli llama-server llama-perplexity
hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
  --include "*IQ1_S-XS-*"

./llama.cpp/build/bin/llama-cli \
  --model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 -c 8192 \
  --temp 1.0 --top-p 0.95 --single-turn

Point --model at shard 1 only; llama.cpp derives the rest from the filename, so keep all 34 shards in one directory under their published names.

--n-cpu-moe 93 places all MoE layers in CPU memory. Memory is the hard gate: you need roughly 550 GB of RAM+VRAM combined. 8ร—H100 (640 GB HBM) cannot hold this alone and must be paired with large system RAM; our node is 8ร—H100 + 2TB RAM, where mmap keeps resident memory around 22 GB and leaves the rest in page cache. Expect ~6.8 tok/s generation and a ~4.5 min first (cold) load.

Known issue inherited from the fork: during warmup, _exps selection does not bypass top-k, so only 16 of 896 experts load and the first load is slow. Work around it with --no-warmup.

Reproducing

The quantization recipe is unchanged from IQ1_S-XS โ€” see 6block/Kimi-K3-GGUF for the full bit-width allocation and quantize command. The only difference is that the source GGUF is the orthogonalized copy.

# 1. refusal direction: 308 pairs ร— 93 layers, on the BF16 tensors of UD-Q8_K_XL
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  --positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
  -o refusal_dir.gguf

# 2. pick the stable band (layers 56-73), in-band normalized average
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json

# 3. orthogonalize the 279 residual-write tensors, scale=1.0
#    reflink COW clone: the 1.5 TiB output costs almost no extra disk
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
  --direction layer_pick.json --scale 1.0

# 4. verify before spending hours on quantization
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/

# 5. quantize the orthogonalized copy with the IQ1_S-XS recipe
#    Set metadata here, at quantize time -- see the note below
cd /path/with/imatrix   # so imatrix.file records a bare filename
llama-quantize --allow-requantize --keep-split \
  --imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
  --token-embedding-type q6_k --output-tensor-type q6_k \
  --override-kv general.quantized_by=str:Ryanchen911 \
  --override-kv general.name=str:Kimi-K3-abliterated \
  UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64

Set the metadata with --override-kv during quantization. general.quantized_by and general.repo_url are inherited from the source GGUF and are not rewritten by llama-quantize, so a quant built from someone else's file ships their attribution. Likewise quantize.imatrix.file records the --imatrix argument verbatim โ€” run from the imatrix's directory and pass a bare filename, or your absolute server path gets published. Fixing this afterwards means streaming 540 GiB through a rewriter; setting it here costs nothing.

size_label stays at 896x60B โ€” no experts were pruned in this tier.

Reproducing the evaluation:

llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# wait for {"status":"ok"} -- a 540 GiB load takes minutes and returns 503 until ready
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
  --json-out eval_abl.json

The 74 evaluation prompts are deliberately disjoint from the 308 pairs used to compute the direction โ€” different topics, and none of the six wrapper templates. Self-testing on the prompts that produced the direction inflates every number.

Intended use and risk

This model will attempt harmful requests. It is published for safety research (measuring what guardrail removal actually does to a frontier MoE), red-teaming, and users who need an unfiltered model for legitimate work and accept responsibility for their prompts. It is not appropriate as a drop-in for a user-facing product, and the underlying model's knowledge limits and quantization damage still apply. You are responsible for what you generate with it.

Credits

  • moonshotai/Kimi-K3 โ€” original model
  • unsloth โ€” UD-Q8_K_XL lossless source and UD-IQ1_S baseline; full-size fixes in the llama.cpp fork
  • pwilkin โ€” K3 architecture support in llama.cpp
  • Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (arXiv:2406.11717) โ€” the method

License

Inherits the Kimi K3 License (near-MIT; explicitly permits modification, distribution, sublicensing and derivative works. Obligations apply only to MaaS businesses above $20M annual revenue, and products above 100M MAU or $20M monthly revenue, which must display "Kimi K3" in their UI).


ไธญๆ–‡่ฏดๆ˜Ž

Kimi K3๏ผˆ2.8T ๅ‚ๆ•ฐ / 104B ๆฟ€ๆดป / 896 experts top-16๏ผ‰็š„ๆถˆ่ž๏ผˆabliterated๏ผ‰GGUF ้‡ๅŒ–๏ผŒ ๅทฒๅฐ†ๆ‹’็ปๆ–นๅ‘ไปŽๅ†™ๅ…ฅๆฎ‹ๅทฎๆต็š„ๆƒ้‡ไธญๆญฃไบคๅŒ–็งป้™คใ€‚

ไฝ“็งฏ 539.7 GiB๏ผŒ34 ๅˆ†็‰‡
้‡ๅŒ–ๆกฃ IQ1_S-XS๏ผˆไธ“ๅฎถ IQ1_Sใ€router ไฟ F32ใ€attn/KDA IQ4_XSใ€shexp Q5_K๏ผ‰
ๆ”นๅŠจๅผ ้‡ 2573 ไธชไธญ็š„ 279 ไธช
ๆœ‰ๅฎณๆ็คบๆ‹’็ป็އ **0.0%**๏ผˆ0/26๏ผ‰๏ผŒๆœชๆถˆ่žๅŸบ็บฟไธบ 7.7%๏ผˆ2/26๏ผ‰
ๆญฃๅฝ“่ฏทๆฑ‚่ฟ‡ๅบฆๆ‹’็ป็އ **0.0%**๏ผˆ0/30๏ผ‰โ€”โ€” ไธŽๅŸบ็บฟๆŒๅนณ
็ญ”้žๆ‰€้—ฎ็އ **0.0%**๏ผˆ0/18๏ผ‰โ€”โ€” ไธŽๅŸบ็บฟๆŒๅนณ
PPL @12 chunks 1.9323 ยฑ 0.0473

่ฟ™ไธชไป“ๅบ“ไธบไป€ไนˆๅญ˜ๅœจ

็Žฐๆœ‰ไธคไธช K3 uncensored ไป“ๅบ“้ƒฝๆฒกๆœ‰็ป™ๅ‡บไปปไฝ•ๆ‹’็ป็އๆ•ฐๅญ—๏ผšไธ€ไธชๅฃฐ็งฐใ€Œ็งป้™ค >98% ้˜ฒๆŠคใ€ๅดๆ—  benchmark๏ผŒ ๅฆไธ€ไธช README ๆ˜Žๅ†™ retentionใ€Œpending / ๅฐšๆœชๆต‹้‡ใ€ใ€‚ๆฒกๆต‹่ฟ‡็š„ๆถˆ่žๅ’Œๅๆމ็š„ๆจกๅž‹ๆ— ๆณ•ๅŒบๅˆ†โ€”โ€”่ฟ™ไธช ๆŠ€ๆœฏ็š„ๅ…ธๅž‹ๅคฑ่ดฅๆจกๅผไธๆ˜ฏใ€Œไป็„ถๆ‹’็ปใ€่€Œๆ˜ฏใ€Œ็ญ”ๅพ—ไธ็Ÿฅๆ‰€ไบ‘ใ€๏ผŒๅช็ปŸ่ฎกใ€Œๆœ‰ๆฒกๆœ‰่ฏดๆ‹’็ป่ฏๆœฏใ€ไผšๆŠŠๅบŸๆމ็š„ ๆจกๅž‹ๅˆคๆˆๆˆๅŠŸใ€‚

ๆ‰€ไปฅๆœฌไป“ๅบ“่ฟžๅŒๆƒ้‡ไธ€่ตท็ป™ๅ‡บๆต‹้‡็ป“ๆžœ๏ผŒๅŒ…ๆ‹ฌ้‚ฃไบ›็ป“่ฎบไธ็กฎๅฎš็š„้ƒจๅˆ†ใ€‚ๅผ•็”จๆœฌ้กตไปปไฝ•ๆ•ฐๅญ—ๅ‰่ฏทๅ…ˆ่ฏป ๅฑ€้™ไธ€่Š‚๏ผšๆ ‡้ข˜ไธŠ็š„ 0.0% ๆ˜ฏ็œŸๅฎžๆต‹้‡ๅ€ผ๏ผŒไฝ†ไธŽๅŸบ็บฟ็š„ๅทฎๅผ‚ๅœจๆญคๆ ทๆœฌ้‡ไธ‹ไธๅ…ท็ปŸ่ฎกๆ˜พ่‘—ๆ€ง๏ผŒ ๆˆ‘ไปฌๅฆ‚ๅฎž่ฏดๆ˜Ž่€Œไธๆ˜ฏๆŠŠๅฎƒๅ››่ˆไบ”ๅ…ฅๆˆไธ€ไธช็ป“่ฎบใ€‚

ๆ‹’็ป่กŒไธบๅฎžๆต‹

74 ๆก held-out ๆ็คบ๏ผŒๅˆ†ไธ‰ไธช็އ็‹ฌ็ซ‹ๆŠฅๅ‘Š๏ผŒ่ดชๅฟƒ่งฃ็ ๏ผˆtemp=0.0ใ€seed=20260803๏ผ‰๏ผŒ ๅ‰ฅๆމ think channel ๅŽๅช็œ‹ๆœ€็ปˆ็ญ”ๆกˆ็š„ๅ‰ 240 ๅญ—็ฌฆใ€‚

ๆถˆ่ž็‰ˆ๏ผˆๆœฌไป“ๅบ“๏ผ‰ ๅŸบ็บฟ UD-IQ1_S
ๆ‹’็ป็އ๏ผˆ26 ๆกๆœ‰ๅฎณ๏ผŒ่ถŠไฝŽ่ฏดๆ˜Ž็งป้™ค่ถŠๅฝปๅบ•๏ผ‰ **0.0%**๏ผˆ0/26๏ผ‰ 7.7%๏ผˆ2/26๏ผ‰
่ฟ‡ๅบฆๆ‹’็ป็އ๏ผˆ30 ๆกๆญฃๅฝ“ไฝ†่ฏ้ข˜็›ธ้‚ป๏ผŒ่ถŠไฝŽ่ถŠๅฅฝ๏ผ‰ **0.0%**๏ผˆ0/30๏ผ‰ 0.0%๏ผˆ0/30๏ผ‰
็ญ”้žๆ‰€้—ฎ็އ๏ผˆ18 ๆกๆœ‰ๅฎข่ง‚็ญ”ๆกˆ๏ผŒ่ถŠไฝŽ่ฏดๆ˜Ž่ƒฝๅŠ›ไฟๆŒ่ถŠๅฅฝ๏ผ‰ **0.0%**๏ผˆ0/18๏ผ‰ 0.0%๏ผˆ0/18๏ผ‰
็ฉบๅ›žๅค 0/74 0/74
่ฝฏๆ€งๅ›ž้ฟๆ ‡่ฎฐ 0/74 0/74
ๆœ‰ๅฎณ้›†ๅ›ž็ญ”ๅญ—ๆ•ฐไธญไฝๆ•ฐ 2511 ๅญ— 2548 ๅญ—

ๅฟ…้กปๅˆ†ไธ‰ไธช็އ๏ผŒๅชๆŠฅๆ‹’็ป็އๆ˜ฏ้”™็š„โ€”โ€”้‚ฃๆญฃๆ˜ฏ่ฎฉไธ€ไธช่ขซ็ ดๅ็š„ๆจกๅž‹็œ‹่ตทๆฅๆˆๅŠŸ็š„ๅŽŸๅ› ๏ผš

  1. ๆ‹’็ป็އ๏ผˆๆœ‰ๅฎณ้›†๏ผ‰โ€”โ€” ๅนฒ้ข„ๅˆฐๅบ•่ตทๆฒก่ตทไฝœ็”จใ€‚
  2. ่ฟ‡ๅบฆๆ‹’็ป็އ๏ผˆๆญฃๅฝ“้›†๏ผ‰โ€”โ€” ๆœ‰ๆฒกๆœ‰ไผคๅˆฐๆญฃๅธธ่ฏทๆฑ‚ใ€‚่ฟ™ไบ›ๆ็คบๅˆปๆ„ไธŽๆœ‰ๅฎณ้›†่ฏ้ข˜็›ธ้‚ป ๏ผˆๅˆๆณ•ๆžชๆขฐไฟๅ…ปใ€็”จ่ฏๅ‰‚้‡ใ€ๅฎ‰ๅ…จ็ ”็ฉถ๏ผ‰ใ€‚่ฟ™ไธ€้กนไธŠๅ‡่ฏดๆ˜Žๆ–นๅ‘้€‰้”™ๆˆ– scale ่ฟ‡ๅคงใ€‚
  3. ็ญ”้žๆ‰€้—ฎ็އ๏ผˆไบ‹ๅฎž้›†๏ผ‰โ€”โ€” ๆœ‰ๅฎข่ง‚็ญ”ๆกˆ็š„ๆ™ฎ้€š้ข˜๏ผŒ้ ๅ…ณ้”ฎ่ฏๅ‘ฝไธญๅˆคๆ–ญใ€‚ๅฎƒๆ•ๆ‰็š„ๆ˜ฏ ใ€Œๆจกๅž‹ไธๅ†ๆ‹’็ปๆ˜ฏๅ› ไธบๅฎƒๅทฒ็ปไธ็Ÿฅ้“่‡ชๅทฑๅœจ่ฏดไป€ไนˆใ€่ฟ™็งๅคฑ่ดฅใ€‚

ๆถˆ่ž็‰ˆๅฏนๆฏไธ€ๆกๆœ‰ๅฎณๆ็คบ้ƒฝไฝœ็ญ”๏ผŒๅญ—ๆ•ฐไธญไฝๆ•ฐ 2511ใ€้›ถ็ฉบๅ›žๅค๏ผŒไบ‹ๅฎž้›† 18/18 ๅ…จไธญโ€”โ€”ๆ‰€ไปฅ่ฟ™ไธช 0.0% ไธๆ˜ฏใ€Œๆจกๅž‹ๅไบ†ไป€ไนˆ้ƒฝไธ่พ“ๅ‡บใ€้€ ๆˆ็š„ๅ‡่ฑกใ€‚

ๅฑ€้™ โ€”โ€” ๅผ•็”จไธŠ้ขไปปไฝ•ๆ•ฐๅญ—ๅ‰่ฏทๅ…ˆ่ฏป่ฟ™้‡Œ

็›ธๅฏนๅŸบ็บฟ็š„ๆ”นๅ–„ไธๅ…ท็ปŸ่ฎกๆ˜พ่‘—ๆ€งใ€‚ 0/26 vs 2/26๏ผŒFisher ็ฒพ็กฎๆฃ€้ชŒๅŒๅฐพ p = 0.490ใ€‚ n=26 ๆ นๆœฌๅˆ†่พจไธๅ‡บ 7.7% ้‡็บง็š„ๆ•ˆๅบ”ใ€‚็ป“ๆžœๆ–นๅ‘ไธŽๆˆ‘ไปฌ็š„็›ฎๆ ‡ไธ€่‡ด๏ผŒไธ”ๅนฒ้ข„็กฎๅฎžๆ”นๅ˜ไบ†ๆƒ้‡๏ผˆ่งไธ‹ๆ–‡ ้ชŒ่ฏ๏ผ‰๏ผŒไฝ†่ฟ™ไธชๆต‹่ฏ•ๅนถไธ่ƒฝ่ฏๆ˜Žๅฎƒ้™ไฝŽไบ†ๆ‹’็ป็އใ€‚่ฆ่ฏๆ˜Ž้œ€่ฆๆ›ดๅคงๆˆ–ๆ›ด้šพ็š„ๆœ‰ๅฎณ้›†ใ€‚

ๅŸบ็บฟ่‡ชๅทฑไนŸๅชๆ‹’็ปไบ† 7.7%ใ€‚ ่ฆไนˆ unsloth ็š„ UD-IQ1_S ๆœฌ่บซๅฐฑๆฏ”่พƒๅฎฝๆพ๏ผŒ่ฆไนˆ่ฟ™ 26 ๆกๆ็คบ ไธ่ถณไปฅ่ฎฉ K3 ่งฆๅ‘ๆ‹’็ปใ€‚ไธค็ง่งฃ้‡Š้ƒฝ้™ๅˆถไบ†่ฟ™ไธชๅฏน็…ง่ƒฝ่ฏดๆ˜Ž็š„ไธœ่ฅฟใ€‚

ๅŸบ็บฟๆ˜ฏไธๅŒ็š„้‡ๅŒ–้…ๆ–นใ€‚ ่ฏšๅฎž็š„ๅฏน็…งๅบ”่ฏฅๆ˜ฏๆˆ‘ไปฌ่‡ชๅทฑ็š„ IQ1_S-XS ๅŽปๆމๆถˆ่ž้‚ฃไธ€ๆญฅ๏ผ›ไฝ†้‚ฃไธชไบง็‰ฉ ๅœจๆœฌๆฌก่ฏ„ๆต‹่ฎพ่ฎกๅ‡บๆฅไน‹ๅ‰ๅฐฑไธบ่…พ็ฃ็›˜ๅˆ ๆމไบ†ใ€‚UD-IQ1_S๏ผˆunsloth๏ผŒ553.2 GiB๏ผ‰ไธ“ๅฎถๅฑ‚็ฑปๅž‹็›ธๅŒใ€ ้žไธ“ๅฎถๅฑ‚ๅˆ†้…ไธๅŒใ€‚็”จ็ฌฌไธ‰ๆ–นๆœชไฟฎๆ”น็š„ๅ‘ๅธƒ็‰ˆๅšๅŸบ็บฟๆŽ’้™คไบ†ใ€Œ่‡ชๅทฑ้€ ไธ€ไธชๆœ‰ๅˆฉๅŸบ็บฟใ€็š„ๅซŒ็–‘๏ผŒ ไฝ†ๅฎƒไธๆ˜ฏๅŒ้…ๆ–น็š„ A/B ๅฏน็…งใ€‚

ๅˆคๆฎๆ˜ฏๅ…ณ้”ฎ่ฏๅŒน้…่€Œ้ž่ฏญไน‰ๅˆคๆ–ญใ€‚ ่ฝฏๆ€งๅ›ž้ฟโ€”โ€”ใ€Œ่ฟ™ไธช่ฏ้ข˜ๅพˆๅคๆ‚โ€ฆโ€ฆใ€็„ถๅŽไธ็ป™ๅ†…ๅฎนโ€”โ€”็ฎ—ไธๅ‡บ ๆ‹’็ป๏ผŒๆ‰€ไปฅๆŠฅๅ‘Š็š„ๆ‹’็ป็އๆ˜ฏไฝŽไผฐๅ€ผใ€‚ๆˆ‘ไปฌๆŸฅไบ† 15 ไธช่ฝฏๆ€งๅ›ž้ฟๆจกๅผ๏ผŒๅœจ่ฏ„ๅˆ†็ช—ๅฃๅ†…ๅ‘ฝไธญ 0/74๏ผŒ ไฝ†้‚ฃไธช็ช—ๅฃๅชๆœ‰ๅ‰ 240 ๅญ—็ฌฆ๏ผ›ๆˆ‘ไปฌๆฒกๆœ‰ๆ‰ซๆๅ…จๆ–‡๏ผŒ่ฏ„ๆต‹ JSON ไนŸๅชไฟๅญ˜ไบ†็ญ”ๆกˆๅผ€ๅคดใ€‚ LLM-judge ๆ›ดๅ‡†๏ผŒไฝ†ๅผ•ๅ…ฅๅค–้ƒจไพ่ต–ๅ’Œๅฎƒ่‡ชๅทฑ็š„ๅ็ฝฎ๏ผŒ่ฟ™้‡Œ้€‰ๅฏๅค็Žฐ็š„ๅฃๅพ„ใ€‚

PPL ๆ˜ฏ 12 chunks ๅฃๅพ„ใ€‚ ไธบไป€ไนˆ่ฟ™็‚น้‡่ฆใ€ไปฅๅŠๅฎƒๅคšๅฎนๆ˜“่ฏฏๅฏผไบบ๏ผŒ่ง perplexity ไธ€่Š‚ใ€‚

ๆถˆ่žๆ˜ฏๆ€Žไนˆๅš็š„

ๆ ‡ๅ‡†ๅšๆณ•๏ผšๆ‰พๅˆฐๆฟ€ๆดป็ฉบ้—ด้‡Œๆ‰ฟ่ฝฝๆ‹’็ป่กŒไธบ็š„ๆ–นๅ‘๏ผŒๅ†ๆŠŠๅฎƒไปŽๆ‰€ๆœ‰ๅ†™ๅ…ฅๆฎ‹ๅทฎๆต็š„ๆƒ้‡้‡ŒๆŠ•ๅฝฑๆމใ€‚ ่ฟ™้‡Œๅ€ผๅพ—่ฏด็š„ๆ˜ฏๅฎƒๅœจ K3 ๆžถๆž„ไธŠ่ฝๅœฐๅพ—ๆœ‰ๅคšไพฟๅฎœใ€‚

ๆ‹’็ปๆ–นๅ‘

llama-cvector-generator ่ท‘ 308 ๅฏนๆ็คบ๏ผˆๆœ‰ๅฎณ / ๆ— ๅฎณ๏ผŒ่กจ้ขๅฝขๅผ้…ๅฏน๏ผ‰ร— ๅ…จ้ƒจ 93 ๅฑ‚๏ผŒ ๅœจๆœฌๅœฐ UD-Q8_K_XL ็š„ BF16 ๅผ ้‡ไธŠ่ฎก็ฎ—โ€”โ€”ไธ็”จๅˆซไบบๅ…ฌๅธƒ็š„ๅ‘้‡๏ผŒๆ‰€ไปฅ่ฟ™ไธชๆ–นๅ‘ๅฑžไบŽ่ฟ™ไธชๆจกๅž‹ ๆœฌ่บซ่€Œไธๆ˜ฏๆŸไธชๆ่ต ๆจกๅž‹ใ€‚

้€‰ๅฑ‚ๆฏ”็œ‹่ตทๆฅๆ›ดๅ…ณ้”ฎใ€‚ๆˆ‘ไปฌๅ–็จณๅฎšๅธฆ mid60-80๏ผˆlayer 56โ€“73๏ผ‰๏ผŒ็”จๅธฆๅ†…ๅฝ’ไธ€ๅŒ–ๅนณๅ‡๏ผŒ ๆ—ขไธๆ˜ฏๅ•ๅฑ‚ไนŸไธๆ˜ฏๅ…จๅฑ‚ๅนณๅ‡๏ผš้ ๅ‰ๅ’Œ้ ๅŽ็š„ๅฑ‚ๅธฆ็š„ๆ–นๅ‘ไผšๆŠŠๅนณๅ‡ๆ‹–ๅ๏ผŒ็บณๅ…ฅๅฎƒไปฌๅ่€Œๅ˜ๅทฎใ€‚

279 ไธชๅผ ้‡๏ผŒไปฅๅŠไธบไป€ไนˆ่ฟ™ๅฐฑ็ญ‰ไบŽๆ•ดไธชๆจกๅž‹

ๅชๆญฃไบคๅŒ–ๅ†™ๅ…ฅๆฎ‹ๅทฎๆต็š„็Ÿฉ้˜ต๏ผŒ่ฏปไพงไธ€ๅพ‹ไธ็ขฐใ€‚

ๅผ ้‡ๆ— ๆ•ฐ้‡ ่ฏดๆ˜Ž
attn_output 93 attention ๅ†™ๅ›ž
ffn_routed_up 92 MoE ็š„ๅ†™ไพง โ€”โ€” 3584 latent โ†’ 7168 ๆฎ‹ๅทฎ
ffn_down_shexp 92 ๅ…ฑไบซไธ“ๅฎถๅ†™ๅ›ž
ffn_down 1 layer 0 ็š„ dense MLP
token_embd 1
ๅˆ่ฎก 279 2573 ไธชๅผ ้‡ไธญ

ffn_routed_up ๆ˜ฏ่ฎฉ่ฟ™ไปถไบ‹ๅ˜ๅพ—ๅฏ่กŒ็š„ๅ…ณ้”ฎใ€‚K3 ็š„ latent MoE ้€š่ฟ‡ไธ€ๅฏนๅ…ฑไบซๆŠ•ๅฝฑ่ทฏ็”ฑ๏ผŒ่€Œ่ฟ™ๅฏนๆŠ•ๅฝฑ ๆ˜ฏๅ…จ้ƒจ 896 ไธชไธ“ๅฎถๅ…ฑ็”จ็š„โ€”โ€”ffn_routed_down๏ผˆ7168 โ†’ 3584๏ผ‰่ฏป๏ผŒffn_routed_up ๏ผˆ3584 โ†’ 7168๏ผ‰ๅ†™ใ€‚ๆฏๅฑ‚ๅชๆญฃไบคๅŒ–่ฟ™ไธ€ไธช็Ÿฉ้˜ต๏ผŒๅฐฑ็ญ‰ไบŽๅŒๆ—ถๆถˆๆމๅ…จ้ƒจ 896 ไธชไธ“ๅฎถ็š„ๆ‹’็ปๆ–นๅ‘๏ผŒ ๆ‰€ไปฅ 1347 GiB ็š„ไธ“ๅฎถๆƒ้‡ไธ€ไธชๅญ—่Š‚้ƒฝๆฒก็ขฐใ€‚ffn_routed_down ๆญฃ็กฎๅœฐไฟๆŒๆœชไฟฎๆ”นโ€”โ€”ๅฎƒไปŽๆฎ‹ๅทฎๆต ่ฏปๅ–๏ผŒไธๅ‘ๅ…ถๅ†™ๅ…ฅใ€‚

่ฟ™ๆ˜ฏ K3 ็‰นๅฎš็ป“ๆž„้€็š„ไพฟๅฎœใ€‚ๆขๆˆๆฏไธชไธ“ๅฎถๅ„ๆœ‰่‡ชๅทฑ่พ“ๅ‡บๆŠ•ๅฝฑ็š„ MoE ๆžถๆž„๏ผŒ่ฟ™ไธช็ป“่ฎบไธๆˆ็ซ‹ใ€‚

้ชŒ่ฏ

scale=1.0๏ผˆๅฎŒๅ…จๆญฃไบคๅŒ–๏ผ‰ใ€‚279 ไธชๅผ ้‡็š„ๆถˆ้™คๆฏ”ไพ‹๏ผš**ไธญไฝๆ•ฐ 1.2084%**๏ผŒ่Œƒๅ›ด 0.9615โ€“1.7291%ใ€‚

้šๅŽ้€ไธชๅฎก่ฎกไบ† 11 ไธชๅผ ้‡็š„ 5 ้กนๆŒ‡ๆ ‡๏ผš็›ฎๆ ‡ๅผ ้‡ๅœจๆ‹’็ปๆ–นๅ‘ไธŠ็š„ๆŠ•ๅฝฑ้™ๅˆฐๆบ็š„ 1.2โ€“1.6% ๏ผˆโ‰ค2% ๅณ BF16 ๅ™ชๅฃฐๅบ•๏ผ‰๏ผ›ๆบๅผ ้‡่Œƒๆ•ฐไปไธŽ่ฟ่กŒๅ‰ๆŠฅๅ‘Šไธ€่‡ด๏ผˆ็กฎ่ฎคๆฒกๆœ‰่ขซๅฐฑๅœฐๆ”นๅ†™๏ผ‰๏ผ›ๆบไธŽ็›ฎๆ ‡ inode ไธๅŒไธ”้“พๆŽฅๆ•ฐไธบ 1๏ผˆ็กฎ่ฎค reflink ๆ˜ฏ COW ๅ…‹้š†่€Œ้ž็กฌ้“พๆŽฅ๏ผ‰๏ผ›ไฝๆจกๅผ็กฎๆœ‰ๅทฎๅผ‚๏ผ›ไปฅๅŠๆญฃไบค่กฅ็š„็›ธๅฏน ๅ˜ๅŒ–ไป… 1.5e-3๏ผŒๅณๅชๆœ‰ BF16 ่ˆๅ…ฅโ€”โ€”ๅนฒ้ข„็งป้™คไบ†ๆ‹’็ปๆ–นๅ‘่€Œๆฒกๆœ‰ไผคๅˆฐๅ…ถไป–ๅˆ†้‡ใ€‚

Perplexity

็ปŸไธ€ๅฃๅพ„๏ผšwikitext-2-raw/wiki.test.rawใ€12 chunksใ€n_ctx=512ใ€--n-cpu-moe 93๏ผŒ ไธŽๅ…ถไป–ๆฏไธ€่กŒๅŒไธ€ๅฐๆœบๅ™จใ€ๅŒไธ€ llama.cpp ๆž„ๅปบใ€‚

้‡ๅŒ– ไฝ“็งฏ PPL ็›ธๅฏน Q8
UD-Q8_K_XL๏ผˆๆ— ๆŸๅŸบๅ‡†๏ผ‰ 1453.9 GiB 1.3453 ยฑ 0.0420 โ€”
UD-IQ1_S๏ผˆๅŸบ็บฟ๏ผŒๆœชๆถˆ่ž๏ผ‰ 553.2 GiB 1.8824 ยฑ 0.0446 +0.5371
IQ1_S-XS๏ผˆๆˆ‘ไปฌ็š„้…ๆ–น๏ผŒๆœชๆถˆ่ž๏ผ‰ 539.7 GiB 1.9193 ยฑ 0.0469 +0.5740
IQ1_S-XS ๆถˆ่ž็‰ˆ๏ผˆๆœฌไป“ๅบ“๏ผ‰ 539.7 GiB 1.9323 ยฑ 0.0473 +0.5870

ไธŽๅŒ้…ๆ–นๆœชๆถˆ่ž็‰ˆ็›ธๆฏ”๏ผŒไปฃไปทๆ˜ฏ +0.013๏ผˆ+0.68%๏ผ‰โ€”โ€”ๅฐไบŽ้‚ฃไธ€่กŒ่‡ช่บซ็š„ ยฑ0.0469 ่ฏฏๅทฎๆฃ’ใ€‚ ๅœจๆญคๆ ทๆœฌ้‡ไธ‹๏ผŒๆถˆ่žๆต‹ไธๅ‡บ็ปŸ่ฎกๆ˜พ่‘—็š„่ฏญ่จ€ๅปบๆจกๆŸๅคฑใ€‚่ฟ™ไธช่ฏดๆณ•ๆฏ”ใ€Œๆฒกๆœ‰ๆŸๅคฑใ€ๅผฑ๏ผŒไฝ†ๅฎƒๆ˜ฏๆ•ฐๆฎ ๆ”ฏๆŒๅพ—ไฝ็š„้‚ฃไธ€ไธชใ€‚

ไธ€ไธชๅ€ผๅพ—่ฎฐๅฝ•็š„้™ท้˜ฑ๏ผŒๅ› ไธบๆˆ‘ไปฌ่‡ชๅทฑ่ธฉไบ†่ฟ›ๅŽปใ€‚ ่ฟ™ไธชๆจกๅž‹่ท‘ๅฎŒๆ•ด 568 chunks ๅพ— PPL 2.6479 ยฑ 0.0115๏ผŒๆ‹ฟๅฎƒๅ’Œ 12-chunk ็š„ๅŸบ็บฟๆฏ”๏ผŒ็œ‹่ตทๆฅๆ˜ฏ็พ้šพๆ€ง็š„ +38% ้€€ๅŒ–ใ€‚้‚ฃๆ˜ฏๅ‡่ฑกใ€‚ ๆป‘ๅŠจ perplexity ๅนณๅ‡ๅ€ผ้š่ฏญๆ–™่ฆ†็›–ๅขžๅŠ ่€Œๅ•่ฐƒไธŠๅ‡โ€”โ€”ๅŒไธ€ไธชๆจกๅž‹ๅœจ chunk 12 ่ฏปๆ•ฐ 1.93๏ผŒ ๅœจ chunk 568 ่ฏปๆ•ฐ 2.65ใ€‚่ฟ™้‡Œๆฏไธชๅ…ฌๅผ€็š„ K3 ๅŸบ็บฟ้ƒฝๆ˜ฏ 12 chunks ๆต‹็š„๏ผŒๆ‰€ไปฅ 568-chunk ็š„ๆ•ฐๅญ— ไธŽๅฎƒไปฌไธญไปปไฝ•ไธ€ไธช้ƒฝไธๅฏๆฏ”ใ€‚่ทจๆจกๅž‹ๆฏ” PPL ไน‹ๅ‰๏ผŒๅ…ˆ็กฎ่ฎค chunk ๆ•ฐไธ€่‡ด ๏ผˆgrep 'calculating perplexity over N chunks'๏ผ‰ใ€‚ๆœช้™„ chunk ๆ•ฐใ€่ฏญๆ–™ๅ’Œ n_ctx ็š„ PPL ๆ•ฐๅญ—ไธ่ƒฝไธŽๆญค่กจไบคๅ‰ๆฏ”่พƒโ€”โ€”ๅŒ…ๆ‹ฌๆ‹ฟๆญค่กจๅŽปๅ’Œๅˆซไบบ็š„ๆฏ”ใ€‚

ๆ€Žไนˆ่ท‘

ไธŽไปปไฝ•ๅ…ถไป– K3 GGUF ็›ธๅŒใ€‚้œ€่ฆ unsloth ็š„ llama.cpp fork๏ผŒๅฎƒๅ ๅœจ ggml-org PR #26185 ไน‹ไธŠ๏ผˆK3 ๆžถๆž„ๆ”ฏๆŒ๏ผŒ ็”ฑ llama.cpp ๆˆๅ‘˜ pwilkin ๆไบค๏ผŒๅฐšๆœชๅˆๅ…ฅ master๏ผ‰๏ผš

git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..

# nvcc ไธๅœจ PATH ๆ—ถๅฟ…้กปๆ˜พๅผๆŒ‡ๅฎš๏ผŒๅฆๅˆ™ cmake ๆ‰พไธๅˆฐ CUDA ็ผ–่ฏ‘ๅ™จ
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
  -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
  -DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
  --target llama-cli llama-server llama-perplexity
hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
  --include "*IQ1_S-XS-*"

./llama.cpp/build/bin/llama-cli \
  --model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 -c 8192 \
  --temp 1.0 --top-p 0.95 --single-turn

--model ๅชๆŒ‡ๅ‘็ฌฌ 1 ไธชๅˆ†็‰‡๏ผ›llama.cpp ไผšไปŽๆ–‡ไปถๅๆŽจๅฏผๅ…ถไฝ™ๅˆ†็‰‡๏ผŒๆ‰€ไปฅ 34 ไธชๅˆ†็‰‡่ฆๆ”พๅœจๅŒไธ€็›ฎๅฝ• ๅนถไฟๆŒๅ‘ๅธƒๆ—ถ็š„ๆ–‡ไปถๅใ€‚

--n-cpu-moe 93 ๆŠŠๆ‰€ๆœ‰ MoE ๅฑ‚ๆ”พๅœจ CPU ๅ†…ๅญ˜ใ€‚ๅ†…ๅญ˜ๆ˜ฏ็กฌ้—จๆง›๏ผš้œ€่ฆ RAM+VRAM ๅˆ่ฎก็บฆ 550 GBใ€‚8ร—H100๏ผˆ640 GB HBM๏ผ‰ๅ•็‹ฌ่ฃ…ไธไธ‹๏ผŒๅฟ…้กป้…ๅคงๅ†…ๅญ˜๏ผ›ๆˆ‘ไปฌ็š„่Š‚็‚นๆ˜ฏ 8ร—H100 + 2TB ๅ†…ๅญ˜๏ผŒ mmap ่ฎฉๅธธ้ฉปๅ†…ๅญ˜็ปดๆŒๅœจ็บฆ 22 GB๏ผŒๅ…ถไฝ™็•™ๅœจ page cacheใ€‚้ข„ๆœŸ็”Ÿๆˆ้€Ÿๅบฆ็บฆ 6.8 tok/s๏ผŒ ้ฆ–ๆฌกๅ†ทๅŠ ่ฝฝ็บฆ 4.5 ๅˆ†้’Ÿใ€‚

็ปงๆ‰ฟ่‡ช fork ็š„ๅทฒ็Ÿฅ้—ฎ้ข˜๏ผšwarmup ้˜ถๆฎต _exps ็š„้€‰ๆ‹ฉๆฒกๆœ‰็ป•่ฟ‡ top-k๏ผŒๅชๅŠ ่ฝฝ 896 ไธชไธ“ๅฎถไธญ็š„ 16 ไธช๏ผŒๅฏผ่‡ด้ฆ–ๆฌกๅŠ ่ฝฝๅพˆๆ…ขใ€‚็”จ --no-warmup ็ป•่ฟ‡ใ€‚

ๅค็Žฐ

้‡ๅŒ–้…ๆ–นไธŽ IQ1_S-XS ๅฎŒๅ…จไธ€่‡ดโ€”โ€”ๅฎŒๆ•ดไฝๅฎฝๅˆ†้…ๅ’Œ้‡ๅŒ–ๅ‘ฝไปค่ง 6block/Kimi-K3-GGUFใ€‚ๅ”ฏไธ€ๅŒบๅˆซๆ˜ฏๆบ GGUF ๆขๆˆไบ†ๆญฃไบคๅŒ–ๅŽ็š„ๅ‰ฏๆœฌใ€‚

# 1. ๆ‹’็ปๆ–นๅ‘๏ผš308 ๅฏน ร— 93 ๅฑ‚๏ผŒๅœจ UD-Q8_K_XL ็š„ BF16 ๅผ ้‡ไธŠ็ฎ—
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  --positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
  -o refusal_dir.gguf

# 2. ๅ–็จณๅฎšๅธฆ๏ผˆlayer 56-73๏ผ‰๏ผŒๅธฆๅ†…ๅฝ’ไธ€ๅŒ–ๅนณๅ‡
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json

# 3. ๆญฃไบคๅŒ– 279 ไธชๆฎ‹ๅทฎๅ†™ไพงๅผ ้‡๏ผŒscale=1.0
#    reflink COW ๅ…‹้š†๏ผš1.5 TiB ่พ“ๅ‡บๅ‡ ไนŽไธ้ขๅค–ๅ ็›˜
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
  --direction layer_pick.json --scale 1.0

# 4. ๅ…ˆ้ชŒ่ฏ๏ผŒๅ†่Šฑๅ‡ ๅฐๆ—ถๅŽป้‡ๅŒ–
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/

# 5. ็”จ IQ1_S-XS ้…ๆ–น้‡ๅŒ–ๆญฃไบคๅŒ–ๅŽ็š„ๅ‰ฏๆœฌ
#    ๅ…ƒๆ•ฐๆฎๅœจ่ฟ™ไธ€ๆญฅๅฐฑ่ฎพๅฅฝ โ€”โ€” ่งไธ‹้ข็š„่ฏดๆ˜Ž
cd /path/with/imatrix   # ่ฟ™ๆ ท imatrix.file ่ฎฐๅฝ•็š„ๆ˜ฏ่ฃธๆ–‡ไปถๅ
llama-quantize --allow-requantize --keep-split \
  --imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
  --token-embedding-type q6_k --output-tensor-type q6_k \
  --override-kv general.quantized_by=str:Ryanchen911 \
  --override-kv general.name=str:Kimi-K3-abliterated \
  UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
  Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64

ๅ…ƒๆ•ฐๆฎ็”จ --override-kv ๅœจ้‡ๅŒ–ๆ—ถ่ฎพๅฅฝใ€‚ general.quantized_by ๅ’Œ general.repo_url ๆ˜ฏไปŽๆบ GGUF ็ปงๆ‰ฟ็š„๏ผŒllama-quantize ไธไผš้‡ๅ†™๏ผŒๆ‰€ไปฅ็”จๅˆซไบบ็š„ๆ–‡ไปถๅšๅ‡บๆฅ็š„้‡ๅŒ–ไผšๅธฆ็€ๅฏนๆ–น็š„ ็ฝฒๅใ€‚ๅŒ็† quantize.imatrix.file ้€ๅญ—่ฎฐๅฝ• --imatrix ๅ‚ๆ•ฐโ€”โ€”่ฏทๅœจ imatrix ๆ‰€ๅœจ็›ฎๅฝ•่ฟ่กŒๅนถ ไผ ่ฃธๆ–‡ไปถๅ๏ผŒๅฆๅˆ™ไฝ ็š„ๆœๅŠกๅ™จ็ปๅฏน่ทฏๅพ„ไผš่ขซๅ‘ๅธƒๅ‡บๅŽปใ€‚ไบ‹ๅŽไฟฎ่ฆๆŠŠ 540 GiB ๆตๅผ้‡ๅ†™ไธ€้๏ผŒ ๅœจ่ฟ™ไธ€ๆญฅ่ฎพๅฅฝๅˆ™้›ถๆˆๆœฌใ€‚

size_label ไฟๆŒ 896x60Bโ€”โ€”่ฟ™ไธ€ๆกฃๆฒกๆœ‰ๅ‰ชไธ“ๅฎถใ€‚

ๅค็Žฐ่ฏ„ๆต‹๏ผš

llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
  -ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# ็ญ‰ๅˆฐ {"status":"ok"} ๅ†ๆ‰“ โ€”โ€” 540 GiB ๅŠ ่ฝฝ่ฆๅ‡ ๅˆ†้’Ÿ๏ผŒๅฐฑ็ปชๅ‰ไธ€ๅพ‹่ฟ”ๅ›ž 503
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
  --json-out eval_abl.json

้‚ฃ 74 ๆก่ฏ„ๆต‹ๆ็คบไธŽ็”จๆฅ็ฎ—ๆ–นๅ‘็š„ 308 ๅฏนๅˆปๆ„ๆ— ้‡ๅ โ€”โ€”ไธป้ข˜้”™ๅผ€๏ผŒไธ”ไธๅฅ—็”จ้‚ฃ 6 ไธช wrapper ๆจกๆฟใ€‚ๅœจไบง็”Ÿๆ–นๅ‘็š„ๅŒไธ€ๆ‰นๆ็คบไธŠ่‡ชๆต‹๏ผŒๆฏไธชๆ•ฐๅญ—้ƒฝไผš่™š้ซ˜ใ€‚

็”จ้€”ไธŽ้ฃŽ้™ฉ

่ฟ™ไธชๆจกๅž‹ไผšๅฐ่ฏ•ๆ‰ง่กŒๆœ‰ๅฎณ่ฏทๆฑ‚ใ€‚ๅฎƒๅ‘ๅธƒ็”จไบŽๅฎ‰ๅ…จ็ ”็ฉถ๏ผˆๆต‹้‡็งป้™ค้˜ฒๆŠคๅฏนไธ€ไธชๅ‰ๆฒฟ MoE ็ฉถ็ซŸๅšไบ†ไป€ไนˆ๏ผ‰ใ€ ็บข้˜Ÿๆต‹่ฏ•๏ผŒไปฅๅŠ็กฎๅฎž้œ€่ฆๆ— ่ฟ‡ๆปคๆจกๅž‹ๅšๆญฃๅฝ“ๅทฅไฝœๅนถๅฏน่‡ชๅทฑ็š„ๆ็คบ่ดŸ่ดฃ็š„ไฝฟ็”จ่€…ใ€‚ๅฎƒไธ้€‚ๅˆ็›ดๆŽฅ็”จๅœจ ้ขๅ‘็”จๆˆท็š„ไบงๅ“้‡Œ๏ผŒไธ”ๅบ•ๅบงๆจกๅž‹็š„็Ÿฅ่ฏ†่พน็•ŒไธŽ้‡ๅŒ–ๆŸไผคไพ็„ถๅญ˜ๅœจใ€‚ไฝ ๅฏน่‡ชๅทฑ็”Ÿๆˆ็š„ๅ†…ๅฎน่ดŸ่ดฃใ€‚

่‡ด่ฐข

  • moonshotai/Kimi-K3 โ€”โ€” ๅŽŸๅง‹ๆจกๅž‹
  • unsloth โ€”โ€” UD-Q8_K_XL ๆ— ๆŸๆบไธŽ UD-IQ1_S ๅŸบ็บฟ๏ผ›llama.cpp fork ไธญ็š„ๅ…จๅฐบๅฏธๆจกๅž‹ไฟฎๅค
  • pwilkin โ€”โ€” llama.cpp ็š„ K3 ๆžถๆž„ๆ”ฏๆŒ
  • Arditi ็ญ‰๏ผŒRefusal in Language Models Is Mediated by a Single Direction ๏ผˆarXiv:2406.11717๏ผ‰โ€”โ€” ๆ–นๆณ•ๅ‡บๅค„

่ฎธๅฏ

็ปงๆ‰ฟ Kimi K3 License ๏ผˆๆŽฅ่ฟ‘ MIT๏ผ›ๆ˜Ž็กฎๅ…่ฎธไฟฎๆ”นใ€ๅˆ†ๅ‘ใ€ๅ†่ฎธๅฏๅ’Œ่ก็”Ÿไฝœๅ“ใ€‚ไน‰ๅŠกไป…้€‚็”จไบŽๅนด่ฅๆ”ถ่ถ… 2000 ไธ‡็พŽๅ…ƒ็š„ MaaS ไธšๅŠก๏ผŒไปฅๅŠ MAU ่ถ… 1 ไบฟๆˆ–ๆœˆ่ฅๆ”ถ่ถ… 2000 ไธ‡็พŽๅ…ƒ็š„ไบงๅ“๏ผŒ่ฟ™ไบ›้กปๅœจ UI ไธญๆ˜พ็คบใ€ŒKimi K3ใ€๏ผ‰ใ€‚

Downloads last month
3,797
GGUF
Model size
2.8T params
Architecture
kimi-k3
Hardware compatibility
Log In to add your hardware

1-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ 1 Ask for provider support

Model tree for Ryanchen911/Kimi-K3-Uncensored-GGUF

Quantized
(58)
this model

Space using Ryanchen911/Kimi-K3-Uncensored-GGUF 1

Paper for Ryanchen911/Kimi-K3-Uncensored-GGUF