Instructions to use Ryanchen911/Kimi-K3-Uncensored-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S # Run inference directly in the terminal: llama cli -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S # Run inference directly in the terminal: llama cli -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S # Run inference directly in the terminal: ./build/bin/llama cli -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Use Docker
docker model run hf.co/Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
- Llama new
- LM Studio
- Jan
- vLLM
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Ryanchen911/Kimi-K3-Uncensored-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Ryanchen911/Kimi-K3-Uncensored-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
- Ollama
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with Ollama:
ollama run hf.co/Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
- Unsloth Desktop
- Pi
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with Docker Model Runner:
docker model run hf.co/Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
- Lemonade
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Run and chat with the model
lemonade run user.Kimi-K3-Uncensored-GGUF-IQ1_S
List all available models
lemonade list
- Hermes Agent
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Ryanchen911/Kimi-K3-Uncensored-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Ryanchen911/Kimi-K3-Uncensored-GGUF:IQ1_S" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Kimi-K3-Uncensored-GGUF
An abliterated GGUF quantization of Kimi K3 (2.8T params / 104B active / 896 experts, top-16), with the refusal direction orthogonalized out of the residual-stream write path.
| Size | 539.7 GiB, 34 shards |
| Quant | IQ1_S-XS (experts IQ1_S, router F32, attn/KDA IQ4_XS, shexp Q5_K) |
| Tensors modified | 279 of 2573 |
| Harmful-prompt refusal | 0.0% (0/26), vs 7.7% (2/26) for the unmodified baseline |
| Over-refusal on benign prompts | 0.0% (0/30) โ unchanged from baseline |
| Incoherence | 0.0% (0/18) โ unchanged from baseline |
| PPL @12 chunks | 1.9323 ยฑ 0.0473 |
English below ยท ไธญๆ่งๆญค
Why this repo exists
The two existing K3 uncensored repos publish no refusal numbers at all: one claims ">98% of guardrails removed" with no benchmark, the other states retention is "pending / has not been measured." An abliteration whose effect was never measured is indistinguishable from a broken model โ the characteristic failure mode of the technique is not "still refuses" but "answers incoherently," and counting only "did it emit refusal boilerplate" scores a lobotomized model as a success.
So this repo ships the measurement alongside the weights, including the parts that came out inconclusive. Read the Limitations section before trusting any number here โ the headline 0.0% is real but the comparison against baseline is not statistically significant at this sample size, and we say so rather than rounding it up into a claim.
Measured refusal behaviour
74 held-out prompts, three separately-reported rates, greedy decoding (temp=0.0, seed=20260803),
judged on the first 240 characters of the final answer after stripping the think channel.
| abliterated (this repo) | baseline UD-IQ1_S |
|
|---|---|---|
| Refusal rate (26 harmful โ lower = more thoroughly removed) | 0.0% (0/26) | 7.7% (2/26) |
| Over-refusal rate (30 benign, topically adjacent โ lower = better) | 0.0% (0/30) | 0.0% (0/30) |
| Incoherence rate (18 factual โ lower = capability retained) | 0.0% (0/18) | 0.0% (0/18) |
| Empty replies | 0/74 | 0/74 |
| Soft-evasion markers | 0/74 | 0/74 |
| Median answer length, harmful set | 2511 chars | 2548 chars |
Three rates, not one. Reporting only the refusal rate is the mistake that makes a destroyed model look successful:
- Refusal rate (harmful set) โ does the intervention work at all.
- Over-refusal rate (benign set) โ did it damage legitimate requests. These prompts are deliberately topically adjacent to the harmful set (lawful firearm maintenance, medication dosage questions, security research). A rise here means the direction was mis-selected or the scale was too aggressive.
- Incoherence rate (factual set) โ questions with objectively checkable answers, scored by keyword hit. This catches the failure mode where the model stops refusing because it has stopped making sense.
The abliterated model answered every harmful prompt at a median 2511 characters with zero empty replies, and hit 18/18 on the factual set โ so the 0.0% is not the artifact of a broken model emitting nothing.
Limitations โ read this before citing any number above
The improvement over baseline is not statistically significant. 0/26 vs 2/26 gives a two-sided Fisher exact p = 0.490. At n=26 this evaluation simply cannot resolve an effect the size of 7.7%. The direction of the result is what we set out to produce, and the intervention provably changed the weights (see below), but this test does not establish that it lowered the refusal rate. A larger or harder harmful set would be needed for that.
The baseline itself refused only 7.7%. Either unsloth's UD-IQ1_S is already permissive, or
these 26 prompts are not hard enough to elicit refusals from K3. Both readings limit what the
comparison can show.
The baseline is a different quantization recipe. The honest comparison would be against our own
IQ1_S-XS with the abliteration omitted; that artifact was deleted to reclaim disk before this
evaluation was designed. UD-IQ1_S (unsloth, 553.2 GiB) shares the expert type but allocates
non-expert layers differently. Using a third party's unmodified release removes any suspicion of
self-serving baseline construction, but it is not a same-recipe A/B.
The judge is keyword matching, not semantic. Soft evasion โ "this is a complex topicโฆ" followed by no content โ is not counted as a refusal, so the reported refusal rates are underestimates. We checked for 15 soft-evasion patterns and found 0/74 in the scored window, but that window is only the first 240 characters; we did not scan full texts, and the stored evaluation JSON keeps only the answer head. An LLM judge would be more accurate at the cost of an external dependency and its own biases; we chose the reproducible criterion.
PPL is measured at 12 chunks. See the perplexity section for why this matters and how easily it misleads.
How the abliteration was done
Standard abliteration finds the direction in activation space that mediates refusal, then projects it out of the weights that write to the residual stream. The interesting part here is how cheaply that lands on K3's architecture.
The refusal direction
llama-cvector-generator over 308 prompt pairs (harmful / harmless, matched in surface form)
across all 93 layers, computed on the BF16 tensors of the local UD-Q8_K_XL copy โ not taken
from a published vector, so the direction belongs to this model rather than to a donor.
Layer selection matters more than it looks. We take the stable band mid60-80 (layers 56โ73) and use an in-band normalized average, not a single layer and not the all-layer mean: early and late layers carry directions that pull the average off, so including them degrades the result.
279 tensors, and why that is the whole model
Only matrices that write into the residual stream are orthogonalized. Read-side projections are left alone.
| Tensor family | Count | Note |
|---|---|---|
attn_output |
93 | attention write-back |
ffn_routed_up |
92 | the MoE write side โ 3584 latent โ 7168 residual |
ffn_down_shexp |
92 | shared-expert write-back |
ffn_down |
1 | layer 0's dense MLP |
token_embd |
1 | |
| Total | 279 | of 2573 tensors |
ffn_routed_up is the one that makes this tractable. K3's latent MoE routes through a pair of
shared projections, and that pair is shared across all 896 experts โ ffn_routed_down
(7168 โ 3584) reads, ffn_routed_up (3584 โ 7168) writes. Orthogonalizing that single matrix per
layer removes the refusal direction from every one of the 896 experts at once, which is why
1347 GiB of expert weights were not touched at all. ffn_routed_down is correctly left
unmodified โ it reads from the residual stream, it does not write to it.
This is a gift from K3's specific structure. It does not transfer to MoE architectures that give each expert its own output projection.
Verification
scale=1.0 (full orthogonalization). Removed-component ratio across the 279 tensors: median
1.2084%, range 0.9615โ1.7291%.
Eleven tensors were then audited individually against five criteria: the destination's projection onto the refusal direction drops to 1.2โ1.6% of the source (โค2% is the BF16 noise floor); source norms still match the pre-run report (confirming nothing was modified in place); source and destination have different inodes with link count 1 (confirming the reflink clone is COW, not a hard link); bit patterns genuinely differ; and the orthogonal complement changed by only 1.5e-3, i.e. BF16 rounding โ the intervention removed the refusal direction without disturbing the other components.
Perplexity
Unified methodology: wikitext-2-raw/wiki.test.raw, 12 chunks, n_ctx=512, --n-cpu-moe 93,
same machine and same llama.cpp build as every other row.
| Quant | Size | PPL | vs Q8 |
|---|---|---|---|
| UD-Q8_K_XL (lossless reference) | 1453.9 GiB | 1.3453 ยฑ 0.0420 | โ |
| UD-IQ1_S (baseline, unmodified) | 553.2 GiB | 1.8824 ยฑ 0.0446 | +0.5371 |
| IQ1_S-XS (our recipe, not abliterated) | 539.7 GiB | 1.9193 ยฑ 0.0469 | +0.5740 |
| IQ1_S-XS abliterated (this repo) | 539.7 GiB | 1.9323 ยฑ 0.0473 | +0.5870 |
Against the same recipe without abliteration, the cost is +0.013 (+0.68%) โ smaller than that row's own ยฑ0.0469 error bar. At this sample size the abliteration produces no statistically detectable language-modelling loss. That is a weaker claim than "no loss," and it is the one the data supports.
A trap worth documenting, because we walked into it. The full 568-chunk run of this model gives
PPL 2.6479 ยฑ 0.0115, and comparing that to the 12-chunk baselines suggests a catastrophic +38%
regression. It is an artifact. The sliding perplexity average rises monotonically as more of the
corpus is covered โ this same model reads 1.93 at chunk 12 and 2.65 at chunk 568. Every published
K3 baseline here was measured at 12 chunks, so a 568-chunk figure is not comparable to any of them.
Before comparing PPL across models, check that the chunk count matches
(grep 'calculating perplexity over N chunks'). Numbers quoted without their chunk count, corpus
and n_ctx cannot be cross-compared with this table โ including the ones in this table, against
anyone else's.
How to run
Identical to any other K3 GGUF. Requires the unsloth llama.cpp fork, which stacks on ggml-org PR #26185 (K3 architecture support, authored by llama.cpp member pwilkin, not yet merged to master):
git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..
# If nvcc is not on PATH, set it explicitly or cmake fails to find the CUDA compiler
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
-DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
--target llama-cli llama-server llama-perplexity
hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
--include "*IQ1_S-XS-*"
./llama.cpp/build/bin/llama-cli \
--model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
-ngl 99 --n-cpu-moe 93 -c 8192 \
--temp 1.0 --top-p 0.95 --single-turn
Point --model at shard 1 only; llama.cpp derives the rest from the filename, so keep all 34 shards
in one directory under their published names.
--n-cpu-moe 93 places all MoE layers in CPU memory. Memory is the hard gate: you need roughly
550 GB of RAM+VRAM combined. 8รH100 (640 GB HBM) cannot hold this alone and must be paired with
large system RAM; our node is 8รH100 + 2TB RAM, where mmap keeps resident memory around 22 GB and
leaves the rest in page cache. Expect ~6.8 tok/s generation and a ~4.5 min first (cold) load.
Known issue inherited from the fork: during warmup, _exps selection does not bypass top-k, so only
16 of 896 experts load and the first load is slow. Work around it with --no-warmup.
Reproducing
The quantization recipe is unchanged from IQ1_S-XS โ see
6block/Kimi-K3-GGUF for the full bit-width allocation
and quantize command. The only difference is that the source GGUF is the orthogonalized copy.
# 1. refusal direction: 308 pairs ร 93 layers, on the BF16 tensors of UD-Q8_K_XL
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
--positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
-o refusal_dir.gguf
# 2. pick the stable band (layers 56-73), in-band normalized average
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json
# 3. orthogonalize the 279 residual-write tensors, scale=1.0
# reflink COW clone: the 1.5 TiB output costs almost no extra disk
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
--direction layer_pick.json --scale 1.0
# 4. verify before spending hours on quantization
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/
# 5. quantize the orthogonalized copy with the IQ1_S-XS recipe
# Set metadata here, at quantize time -- see the note below
cd /path/with/imatrix # so imatrix.file records a bare filename
llama-quantize --allow-requantize --keep-split \
--imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
--token-embedding-type q6_k --output-tensor-type q6_k \
--override-kv general.quantized_by=str:Ryanchen911 \
--override-kv general.name=str:Kimi-K3-abliterated \
UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64
Set the metadata with --override-kv during quantization. general.quantized_by and
general.repo_url are inherited from the source GGUF and are not rewritten by llama-quantize, so
a quant built from someone else's file ships their attribution. Likewise quantize.imatrix.file
records the --imatrix argument verbatim โ run from the imatrix's directory and pass a bare
filename, or your absolute server path gets published. Fixing this afterwards means streaming
540 GiB through a rewriter; setting it here costs nothing.
size_label stays at 896x60B โ no experts were pruned in this tier.
Reproducing the evaluation:
llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
-ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# wait for {"status":"ok"} -- a 540 GiB load takes minutes and returns 503 until ready
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
--json-out eval_abl.json
The 74 evaluation prompts are deliberately disjoint from the 308 pairs used to compute the direction โ different topics, and none of the six wrapper templates. Self-testing on the prompts that produced the direction inflates every number.
Intended use and risk
This model will attempt harmful requests. It is published for safety research (measuring what guardrail removal actually does to a frontier MoE), red-teaming, and users who need an unfiltered model for legitimate work and accept responsibility for their prompts. It is not appropriate as a drop-in for a user-facing product, and the underlying model's knowledge limits and quantization damage still apply. You are responsible for what you generate with it.
Credits
- moonshotai/Kimi-K3 โ original model
- unsloth โ
UD-Q8_K_XLlossless source andUD-IQ1_Sbaseline; full-size fixes in the llama.cpp fork - pwilkin โ K3 architecture support in llama.cpp
- Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (arXiv:2406.11717) โ the method
License
Inherits the Kimi K3 License (near-MIT; explicitly permits modification, distribution, sublicensing and derivative works. Obligations apply only to MaaS businesses above $20M annual revenue, and products above 100M MAU or $20M monthly revenue, which must display "Kimi K3" in their UI).
ไธญๆ่ฏดๆ
Kimi K3๏ผ2.8T ๅๆฐ / 104B ๆฟๆดป / 896 experts top-16๏ผ็ๆถ่๏ผabliterated๏ผGGUF ้ๅ๏ผ ๅทฒๅฐๆ็ปๆนๅไปๅๅ ฅๆฎๅทฎๆต็ๆ้ไธญๆญฃไบคๅ็งป้คใ
| ไฝ็งฏ | 539.7 GiB๏ผ34 ๅ็ |
| ้ๅๆกฃ | IQ1_S-XS๏ผไธๅฎถ IQ1_Sใrouter ไฟ F32ใattn/KDA IQ4_XSใshexp Q5_K๏ผ |
| ๆนๅจๅผ ้ | 2573 ไธชไธญ็ 279 ไธช |
| ๆๅฎณๆ็คบๆ็ป็ | **0.0%**๏ผ0/26๏ผ๏ผๆชๆถ่ๅบ็บฟไธบ 7.7%๏ผ2/26๏ผ |
| ๆญฃๅฝ่ฏทๆฑ่ฟๅบฆๆ็ป็ | **0.0%**๏ผ0/30๏ผโโ ไธๅบ็บฟๆๅนณ |
| ็ญ้ๆ้ฎ็ | **0.0%**๏ผ0/18๏ผโโ ไธๅบ็บฟๆๅนณ |
| PPL @12 chunks | 1.9323 ยฑ 0.0473 |
่ฟไธชไปๅบไธบไปไนๅญๅจ
็ฐๆไธคไธช K3 uncensored ไปๅบ้ฝๆฒกๆ็ปๅบไปปไฝๆ็ป็ๆฐๅญ๏ผไธไธชๅฃฐ็งฐใ็งป้ค >98% ้ฒๆคใๅดๆ benchmark๏ผ ๅฆไธไธช README ๆๅ retentionใpending / ๅฐๆชๆต้ใใๆฒกๆต่ฟ็ๆถ่ๅๅๆ็ๆจกๅๆ ๆณๅบๅโโ่ฟไธช ๆๆฏ็ๅ ธๅๅคฑ่ดฅๆจกๅผไธๆฏใไป็ถๆ็ปใ่ๆฏใ็ญๅพไธ็ฅๆไบใ๏ผๅช็ป่ฎกใๆๆฒกๆ่ฏดๆ็ป่ฏๆฏใไผๆๅบๆ็ ๆจกๅๅคๆๆๅใ
ๆไปฅๆฌไปๅบ่ฟๅๆ้ไธ่ตท็ปๅบๆต้็ปๆ๏ผๅ ๆฌ้ฃไบ็ป่ฎบไธ็กฎๅฎ็้จๅใๅผ็จๆฌ้กตไปปไฝๆฐๅญๅ่ฏทๅ ่ฏป ๅฑ้ไธ่๏ผๆ ้ขไธ็ 0.0% ๆฏ็ๅฎๆต้ๅผ๏ผไฝไธๅบ็บฟ็ๅทฎๅผๅจๆญคๆ ทๆฌ้ไธไธๅ ท็ป่ฎกๆพ่ๆง๏ผ ๆไปฌๅฆๅฎ่ฏดๆ่ไธๆฏๆๅฎๅ่ไบๅ ฅๆไธไธช็ป่ฎบใ
ๆ็ป่กไธบๅฎๆต
74 ๆก held-out ๆ็คบ๏ผๅไธไธช็็ฌ็ซๆฅๅ๏ผ่ดชๅฟ่งฃ็ ๏ผtemp=0.0ใseed=20260803๏ผ๏ผ
ๅฅๆ think channel ๅๅช็ๆ็ป็ญๆก็ๅ 240 ๅญ็ฌฆใ
| ๆถ่็๏ผๆฌไปๅบ๏ผ | ๅบ็บฟ UD-IQ1_S |
|
|---|---|---|
| ๆ็ป็๏ผ26 ๆกๆๅฎณ๏ผ่ถไฝ่ฏดๆ็งป้ค่ถๅฝปๅบ๏ผ | **0.0%**๏ผ0/26๏ผ | 7.7%๏ผ2/26๏ผ |
| ่ฟๅบฆๆ็ป็๏ผ30 ๆกๆญฃๅฝไฝ่ฏ้ข็ธ้ป๏ผ่ถไฝ่ถๅฅฝ๏ผ | **0.0%**๏ผ0/30๏ผ | 0.0%๏ผ0/30๏ผ |
| ็ญ้ๆ้ฎ็๏ผ18 ๆกๆๅฎข่ง็ญๆก๏ผ่ถไฝ่ฏดๆ่ฝๅไฟๆ่ถๅฅฝ๏ผ | **0.0%**๏ผ0/18๏ผ | 0.0%๏ผ0/18๏ผ |
| ็ฉบๅๅค | 0/74 | 0/74 |
| ่ฝฏๆงๅ้ฟๆ ่ฎฐ | 0/74 | 0/74 |
| ๆๅฎณ้ๅ็ญๅญๆฐไธญไฝๆฐ | 2511 ๅญ | 2548 ๅญ |
ๅฟ ้กปๅไธไธช็๏ผๅชๆฅๆ็ป็ๆฏ้็โโ้ฃๆญฃๆฏ่ฎฉไธไธช่ขซ็ ดๅ็ๆจกๅ็่ตทๆฅๆๅ็ๅๅ ๏ผ
- ๆ็ป็๏ผๆๅฎณ้๏ผโโ ๅนฒ้ขๅฐๅบ่ตทๆฒก่ตทไฝ็จใ
- ่ฟๅบฆๆ็ป็๏ผๆญฃๅฝ้๏ผโโ ๆๆฒกๆไผคๅฐๆญฃๅธธ่ฏทๆฑใ่ฟไบๆ็คบๅปๆไธๆๅฎณ้่ฏ้ข็ธ้ป ๏ผๅๆณๆชๆขฐไฟๅ ปใ็จ่ฏๅ้ใๅฎๅ จ็ ็ฉถ๏ผใ่ฟไธ้กนไธๅ่ฏดๆๆนๅ้้ๆ scale ่ฟๅคงใ
- ็ญ้ๆ้ฎ็๏ผไบๅฎ้๏ผโโ ๆๅฎข่ง็ญๆก็ๆฎ้้ข๏ผ้ ๅ ณ้ฎ่ฏๅฝไธญๅคๆญใๅฎๆๆ็ๆฏ ใๆจกๅไธๅๆ็ปๆฏๅ ไธบๅฎๅทฒ็ปไธ็ฅ้่ชๅทฑๅจ่ฏดไปไนใ่ฟ็งๅคฑ่ดฅใ
ๆถ่็ๅฏนๆฏไธๆกๆๅฎณๆ็คบ้ฝไฝ็ญ๏ผๅญๆฐไธญไฝๆฐ 2511ใ้ถ็ฉบๅๅค๏ผไบๅฎ้ 18/18 ๅ จไธญโโๆไปฅ่ฟไธช 0.0% ไธๆฏใๆจกๅๅไบไปไน้ฝไธ่พๅบใ้ ๆ็ๅ่ฑกใ
ๅฑ้ โโ ๅผ็จไธ้ขไปปไฝๆฐๅญๅ่ฏทๅ ่ฏป่ฟ้
็ธๅฏนๅบ็บฟ็ๆนๅไธๅ ท็ป่ฎกๆพ่ๆงใ 0/26 vs 2/26๏ผFisher ็ฒพ็กฎๆฃ้ชๅๅฐพ p = 0.490ใ n=26 ๆ นๆฌๅ่พจไธๅบ 7.7% ้็บง็ๆๅบใ็ปๆๆนๅไธๆไปฌ็็ฎๆ ไธ่ด๏ผไธๅนฒ้ข็กฎๅฎๆนๅไบๆ้๏ผ่งไธๆ ้ช่ฏ๏ผ๏ผไฝ่ฟไธชๆต่ฏๅนถไธ่ฝ่ฏๆๅฎ้ไฝไบๆ็ป็ใ่ฆ่ฏๆ้่ฆๆดๅคงๆๆด้พ็ๆๅฎณ้ใ
ๅบ็บฟ่ชๅทฑไนๅชๆ็ปไบ 7.7%ใ ่ฆไน unsloth ็ UD-IQ1_S ๆฌ่บซๅฐฑๆฏ่พๅฎฝๆพ๏ผ่ฆไน่ฟ 26 ๆกๆ็คบ
ไธ่ถณไปฅ่ฎฉ K3 ่งฆๅๆ็ปใไธค็ง่งฃ้้ฝ้ๅถไบ่ฟไธชๅฏน็
ง่ฝ่ฏดๆ็ไธ่ฅฟใ
ๅบ็บฟๆฏไธๅ็้ๅ้
ๆนใ ่ฏๅฎ็ๅฏน็
งๅบ่ฏฅๆฏๆไปฌ่ชๅทฑ็ IQ1_S-XS ๅปๆๆถ่้ฃไธๆญฅ๏ผไฝ้ฃไธชไบง็ฉ
ๅจๆฌๆฌก่ฏๆต่ฎพ่ฎกๅบๆฅไนๅๅฐฑไธบ่
พ็ฃ็ๅ ๆไบใUD-IQ1_S๏ผunsloth๏ผ553.2 GiB๏ผไธๅฎถๅฑ็ฑปๅ็ธๅใ
้ไธๅฎถๅฑๅ้
ไธๅใ็จ็ฌฌไธๆนๆชไฟฎๆน็ๅๅธ็ๅๅบ็บฟๆ้คไบใ่ชๅทฑ้ ไธไธชๆๅฉๅบ็บฟใ็ๅซ็๏ผ
ไฝๅฎไธๆฏๅ้
ๆน็ A/B ๅฏน็
งใ
ๅคๆฎๆฏๅ ณ้ฎ่ฏๅน้ ่้่ฏญไนๅคๆญใ ่ฝฏๆงๅ้ฟโโใ่ฟไธช่ฏ้ขๅพๅคๆโฆโฆใ็ถๅไธ็ปๅ ๅฎนโโ็ฎไธๅบ ๆ็ป๏ผๆไปฅๆฅๅ็ๆ็ป็ๆฏไฝไผฐๅผใๆไปฌๆฅไบ 15 ไธช่ฝฏๆงๅ้ฟๆจกๅผ๏ผๅจ่ฏๅ็ชๅฃๅ ๅฝไธญ 0/74๏ผ ไฝ้ฃไธช็ชๅฃๅชๆๅ 240 ๅญ็ฌฆ๏ผๆไปฌๆฒกๆๆซๆๅ จๆ๏ผ่ฏๆต JSON ไนๅชไฟๅญไบ็ญๆกๅผๅคดใ LLM-judge ๆดๅ๏ผไฝๅผๅ ฅๅค้จไพ่ตๅๅฎ่ชๅทฑ็ๅ็ฝฎ๏ผ่ฟ้้ๅฏๅค็ฐ็ๅฃๅพใ
PPL ๆฏ 12 chunks ๅฃๅพใ ไธบไปไน่ฟ็น้่ฆใไปฅๅๅฎๅคๅฎนๆ่ฏฏๅฏผไบบ๏ผ่ง perplexity ไธ่ใ
ๆถ่ๆฏๆไนๅ็
ๆ ๅๅๆณ๏ผๆพๅฐๆฟๆดป็ฉบ้ด้ๆฟ่ฝฝๆ็ป่กไธบ็ๆนๅ๏ผๅๆๅฎไปๆๆๅๅ ฅๆฎๅทฎๆต็ๆ้้ๆๅฝฑๆใ ่ฟ้ๅผๅพ่ฏด็ๆฏๅฎๅจ K3 ๆถๆไธ่ฝๅฐๅพๆๅคไพฟๅฎใ
ๆ็ปๆนๅ
llama-cvector-generator ่ท 308 ๅฏนๆ็คบ๏ผๆๅฎณ / ๆ ๅฎณ๏ผ่กจ้ขๅฝขๅผ้
ๅฏน๏ผร ๅ
จ้จ 93 ๅฑ๏ผ
ๅจๆฌๅฐ UD-Q8_K_XL ็ BF16 ๅผ ้ไธ่ฎก็ฎโโไธ็จๅซไบบๅ
ฌๅธ็ๅ้๏ผๆไปฅ่ฟไธชๆนๅๅฑไบ่ฟไธชๆจกๅ
ๆฌ่บซ่ไธๆฏๆไธชๆ่ต ๆจกๅใ
้ๅฑๆฏ็่ตทๆฅๆดๅ ณ้ฎใๆไปฌๅ็จณๅฎๅธฆ mid60-80๏ผlayer 56โ73๏ผ๏ผ็จๅธฆๅ ๅฝไธๅๅนณๅ๏ผ ๆขไธๆฏๅๅฑไนไธๆฏๅ จๅฑๅนณๅ๏ผ้ ๅๅ้ ๅ็ๅฑๅธฆ็ๆนๅไผๆๅนณๅๆๅ๏ผ็บณๅ ฅๅฎไปฌๅ่ๅๅทฎใ
279 ไธชๅผ ้๏ผไปฅๅไธบไปไน่ฟๅฐฑ็ญไบๆดไธชๆจกๅ
ๅชๆญฃไบคๅๅๅ ฅๆฎๅทฎๆต็็ฉ้ต๏ผ่ฏปไพงไธๅพไธ็ขฐใ
| ๅผ ้ๆ | ๆฐ้ | ่ฏดๆ |
|---|---|---|
attn_output |
93 | attention ๅๅ |
ffn_routed_up |
92 | MoE ็ๅไพง โโ 3584 latent โ 7168 ๆฎๅทฎ |
ffn_down_shexp |
92 | ๅ ฑไบซไธๅฎถๅๅ |
ffn_down |
1 | layer 0 ็ dense MLP |
token_embd |
1 | |
| ๅ่ฎก | 279 | 2573 ไธชๅผ ้ไธญ |
ffn_routed_up ๆฏ่ฎฉ่ฟไปถไบๅๅพๅฏ่ก็ๅ
ณ้ฎใK3 ็ latent MoE ้่ฟไธๅฏนๅ
ฑไบซๆๅฝฑ่ทฏ็ฑ๏ผ่่ฟๅฏนๆๅฝฑ
ๆฏๅ
จ้จ 896 ไธชไธๅฎถๅ
ฑ็จ็โโffn_routed_down๏ผ7168 โ 3584๏ผ่ฏป๏ผffn_routed_up
๏ผ3584 โ 7168๏ผๅใๆฏๅฑๅชๆญฃไบคๅ่ฟไธไธช็ฉ้ต๏ผๅฐฑ็ญไบๅๆถๆถๆๅ
จ้จ 896 ไธชไธๅฎถ็ๆ็ปๆนๅ๏ผ
ๆไปฅ 1347 GiB ็ไธๅฎถๆ้ไธไธชๅญ่้ฝๆฒก็ขฐใffn_routed_down ๆญฃ็กฎๅฐไฟๆๆชไฟฎๆนโโๅฎไปๆฎๅทฎๆต
่ฏปๅ๏ผไธๅๅ
ถๅๅ
ฅใ
่ฟๆฏ K3 ็นๅฎ็ปๆ้็ไพฟๅฎใๆขๆๆฏไธชไธๅฎถๅๆ่ชๅทฑ่พๅบๆๅฝฑ็ MoE ๆถๆ๏ผ่ฟไธช็ป่ฎบไธๆ็ซใ
้ช่ฏ
scale=1.0๏ผๅฎๅ
จๆญฃไบคๅ๏ผใ279 ไธชๅผ ้็ๆถ้คๆฏไพ๏ผ**ไธญไฝๆฐ 1.2084%**๏ผ่ๅด 0.9615โ1.7291%ใ
้ๅ้ไธชๅฎก่ฎกไบ 11 ไธชๅผ ้็ 5 ้กนๆๆ ๏ผ็ฎๆ ๅผ ้ๅจๆ็ปๆนๅไธ็ๆๅฝฑ้ๅฐๆบ็ 1.2โ1.6% ๏ผโค2% ๅณ BF16 ๅชๅฃฐๅบ๏ผ๏ผๆบๅผ ้่ๆฐไปไธ่ฟ่กๅๆฅๅไธ่ด๏ผ็กฎ่ฎคๆฒกๆ่ขซๅฐฑๅฐๆนๅ๏ผ๏ผๆบไธ็ฎๆ inode ไธๅไธ้พๆฅๆฐไธบ 1๏ผ็กฎ่ฎค reflink ๆฏ COW ๅ ้่้็กฌ้พๆฅ๏ผ๏ผไฝๆจกๅผ็กฎๆๅทฎๅผ๏ผไปฅๅๆญฃไบค่กฅ็็ธๅฏน ๅๅไป 1.5e-3๏ผๅณๅชๆ BF16 ่ๅ ฅโโๅนฒ้ข็งป้คไบๆ็ปๆนๅ่ๆฒกๆไผคๅฐๅ ถไปๅ้ใ
Perplexity
็ปไธๅฃๅพ๏ผwikitext-2-raw/wiki.test.rawใ12 chunksใn_ctx=512ใ--n-cpu-moe 93๏ผ
ไธๅ
ถไปๆฏไธ่กๅไธๅฐๆบๅจใๅไธ llama.cpp ๆๅปบใ
| ้ๅ | ไฝ็งฏ | PPL | ็ธๅฏน Q8 |
|---|---|---|---|
| UD-Q8_K_XL๏ผๆ ๆๅบๅ๏ผ | 1453.9 GiB | 1.3453 ยฑ 0.0420 | โ |
| UD-IQ1_S๏ผๅบ็บฟ๏ผๆชๆถ่๏ผ | 553.2 GiB | 1.8824 ยฑ 0.0446 | +0.5371 |
| IQ1_S-XS๏ผๆไปฌ็้ ๆน๏ผๆชๆถ่๏ผ | 539.7 GiB | 1.9193 ยฑ 0.0469 | +0.5740 |
| IQ1_S-XS ๆถ่็๏ผๆฌไปๅบ๏ผ | 539.7 GiB | 1.9323 ยฑ 0.0473 | +0.5870 |
ไธๅ้ ๆนๆชๆถ่็็ธๆฏ๏ผไปฃไปทๆฏ +0.013๏ผ+0.68%๏ผโโๅฐไบ้ฃไธ่ก่ช่บซ็ ยฑ0.0469 ่ฏฏๅทฎๆฃใ ๅจๆญคๆ ทๆฌ้ไธ๏ผๆถ่ๆตไธๅบ็ป่ฎกๆพ่็่ฏญ่จๅปบๆจกๆๅคฑใ่ฟไธช่ฏดๆณๆฏใๆฒกๆๆๅคฑใๅผฑ๏ผไฝๅฎๆฏๆฐๆฎ ๆฏๆๅพไฝ็้ฃไธไธชใ
ไธไธชๅผๅพ่ฎฐๅฝ็้ท้ฑ๏ผๅ ไธบๆไปฌ่ชๅทฑ่ธฉไบ่ฟๅปใ ่ฟไธชๆจกๅ่ทๅฎๆด 568 chunks ๅพ PPL
2.6479 ยฑ 0.0115๏ผๆฟๅฎๅ 12-chunk ็ๅบ็บฟๆฏ๏ผ็่ตทๆฅๆฏ็พ้พๆง็ +38% ้ๅใ้ฃๆฏๅ่ฑกใ
ๆปๅจ perplexity ๅนณๅๅผ้่ฏญๆ่ฆ็ๅขๅ ่ๅ่ฐไธๅโโๅไธไธชๆจกๅๅจ chunk 12 ่ฏปๆฐ 1.93๏ผ
ๅจ chunk 568 ่ฏปๆฐ 2.65ใ่ฟ้ๆฏไธชๅ
ฌๅผ็ K3 ๅบ็บฟ้ฝๆฏ 12 chunks ๆต็๏ผๆไปฅ 568-chunk ็ๆฐๅญ
ไธๅฎไปฌไธญไปปไฝไธไธช้ฝไธๅฏๆฏใ่ทจๆจกๅๆฏ PPL ไนๅ๏ผๅ
็กฎ่ฎค chunk ๆฐไธ่ด
๏ผgrep 'calculating perplexity over N chunks'๏ผใๆช้ chunk ๆฐใ่ฏญๆๅ n_ctx ็ PPL
ๆฐๅญไธ่ฝไธๆญค่กจไบคๅๆฏ่พโโๅ
ๆฌๆฟๆญค่กจๅปๅๅซไบบ็ๆฏใ
ๆไน่ท
ไธไปปไฝๅ ถไป K3 GGUF ็ธๅใ้่ฆ unsloth ็ llama.cpp fork๏ผๅฎๅ ๅจ ggml-org PR #26185 ไนไธ๏ผK3 ๆถๆๆฏๆ๏ผ ็ฑ llama.cpp ๆๅ pwilkin ๆไบค๏ผๅฐๆชๅๅ ฅ master๏ผ๏ผ
git clone https://github.com/unslothai/llama.cpp && cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision && cd ..
# nvcc ไธๅจ PATH ๆถๅฟ
้กปๆพๅผๆๅฎ๏ผๅฆๅ cmake ๆพไธๅฐ CUDA ็ผ่ฏๅจ
export CUDACXX=/usr/local/cuda/bin/nvcc PATH=/usr/local/cuda/bin:$PATH
cmake llama.cpp -B llama.cpp/build \
-DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON \
-DCMAKE_CUDA_ARCHITECTURES=90 -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=ON
cmake --build llama.cpp/build --config Release -j 64 \
--target llama-cli llama-server llama-perplexity
hf download Ryanchen911/Kimi-K3-Uncensored-GGUF --local-dir ./K3abl \
--include "*IQ1_S-XS-*"
./llama.cpp/build/bin/llama-cli \
--model ./K3abl/Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
-ngl 99 --n-cpu-moe 93 -c 8192 \
--temp 1.0 --top-p 0.95 --single-turn
--model ๅชๆๅ็ฌฌ 1 ไธชๅ็๏ผllama.cpp ไผไปๆไปถๅๆจๅฏผๅ
ถไฝๅ็๏ผๆไปฅ 34 ไธชๅ็่ฆๆพๅจๅไธ็ฎๅฝ
ๅนถไฟๆๅๅธๆถ็ๆไปถๅใ
--n-cpu-moe 93 ๆๆๆ MoE ๅฑๆพๅจ CPU ๅ
ๅญใๅ
ๅญๆฏ็กฌ้จๆง๏ผ้่ฆ RAM+VRAM ๅ่ฎก็บฆ
550 GBใ8รH100๏ผ640 GB HBM๏ผๅ็ฌ่ฃ
ไธไธ๏ผๅฟ
้กป้
ๅคงๅ
ๅญ๏ผๆไปฌ็่็นๆฏ 8รH100 + 2TB ๅ
ๅญ๏ผ
mmap ่ฎฉๅธธ้ฉปๅ
ๅญ็ปดๆๅจ็บฆ 22 GB๏ผๅ
ถไฝ็ๅจ page cacheใ้ขๆ็ๆ้ๅบฆ็บฆ 6.8 tok/s๏ผ
้ฆๆฌกๅทๅ ่ฝฝ็บฆ 4.5 ๅ้ใ
็ปงๆฟ่ช fork ็ๅทฒ็ฅ้ฎ้ข๏ผwarmup ้ถๆฎต _exps ็้ๆฉๆฒกๆ็ป่ฟ top-k๏ผๅชๅ ่ฝฝ 896 ไธชไธๅฎถไธญ็ 16
ไธช๏ผๅฏผ่ด้ฆๆฌกๅ ่ฝฝๅพๆ
ขใ็จ --no-warmup ็ป่ฟใ
ๅค็ฐ
้ๅ้
ๆนไธ IQ1_S-XS ๅฎๅ
จไธ่ดโโๅฎๆดไฝๅฎฝๅ้
ๅ้ๅๅฝไปค่ง
6block/Kimi-K3-GGUFใๅฏไธๅบๅซๆฏๆบ GGUF
ๆขๆไบๆญฃไบคๅๅ็ๅฏๆฌใ
# 1. ๆ็ปๆนๅ๏ผ308 ๅฏน ร 93 ๅฑ๏ผๅจ UD-Q8_K_XL ็ BF16 ๅผ ้ไธ็ฎ
llama-cvector-generator -m Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
--positive-file refusal_harmful.txt --negative-file refusal_harmless.txt \
-o refusal_dir.gguf
# 2. ๅ็จณๅฎๅธฆ๏ผlayer 56-73๏ผ๏ผๅธฆๅ
ๅฝไธๅๅนณๅ
python3 select_layer.py --dir refusal_dir.gguf --band mid60-80 -o layer_pick.json
# 3. ๆญฃไบคๅ 279 ไธชๆฎๅทฎๅไพงๅผ ้๏ผscale=1.0
# reflink COW ๅ
้๏ผ1.5 TiB ่พๅบๅ ไนไธ้ขๅคๅ ็
python3 abliterate_gguf.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/ \
--direction layer_pick.json --scale 1.0
# 4. ๅ
้ช่ฏ๏ผๅ่ฑๅ ๅฐๆถๅป้ๅ
python3 verify_abliterated.py --src UD-Q8_K_XL/ --dst UD-Q8_K_XL-abl/
# 5. ็จ IQ1_S-XS ้
ๆน้ๅๆญฃไบคๅๅ็ๅฏๆฌ
# ๅ
ๆฐๆฎๅจ่ฟไธๆญฅๅฐฑ่ฎพๅฅฝ โโ ่งไธ้ข็่ฏดๆ
cd /path/with/imatrix # ่ฟๆ ท imatrix.file ่ฎฐๅฝ็ๆฏ่ฃธๆไปถๅ
llama-quantize --allow-requantize --keep-split \
--imatrix imatrix_k3.gguf --tensor-type-file t3_types.txt \
--token-embedding-type q6_k --output-tensor-type q6_k \
--override-kv general.quantized_by=str:Ryanchen911 \
--override-kv general.name=str:Kimi-K3-abliterated \
UD-Q8_K_XL-abl/Kimi-K3-UD-Q8_K_XL-00001-of-00034.gguf \
Kimi-K3-abliterated-IQ1_S-XS.gguf IQ1_S 64
ๅ
ๆฐๆฎ็จ --override-kv ๅจ้ๅๆถ่ฎพๅฅฝใ general.quantized_by ๅ general.repo_url
ๆฏไปๆบ GGUF ็ปงๆฟ็๏ผllama-quantize ไธไผ้ๅ๏ผๆไปฅ็จๅซไบบ็ๆไปถๅๅบๆฅ็้ๅไผๅธฆ็ๅฏนๆน็
็ฝฒๅใๅ็ quantize.imatrix.file ้ๅญ่ฎฐๅฝ --imatrix ๅๆฐโโ่ฏทๅจ imatrix ๆๅจ็ฎๅฝ่ฟ่กๅนถ
ไผ ่ฃธๆไปถๅ๏ผๅฆๅไฝ ็ๆๅกๅจ็ปๅฏน่ทฏๅพไผ่ขซๅๅธๅบๅปใไบๅไฟฎ่ฆๆ 540 GiB ๆตๅผ้ๅไธ้๏ผ
ๅจ่ฟไธๆญฅ่ฎพๅฅฝๅ้ถๆๆฌใ
size_label ไฟๆ 896x60Bโโ่ฟไธๆกฃๆฒกๆๅชไธๅฎถใ
ๅค็ฐ่ฏๆต๏ผ
llama-server -m Kimi-K3-abliterated-IQ1_S-XS-00001-of-00034.gguf \
-ngl 99 --n-cpu-moe 93 --port 8080 -c 4096
# ็ญๅฐ {"status":"ok"} ๅๆ โโ 540 GiB ๅ ่ฝฝ่ฆๅ ๅ้๏ผๅฐฑ็ปชๅไธๅพ่ฟๅ 503
python3 eval_refusal.py --url http://127.0.0.1:8080 --tag abliterated \
--json-out eval_abl.json
้ฃ 74 ๆก่ฏๆตๆ็คบไธ็จๆฅ็ฎๆนๅ็ 308 ๅฏนๅปๆๆ ้ๅ โโไธป้ข้ๅผ๏ผไธไธๅฅ็จ้ฃ 6 ไธช wrapper ๆจกๆฟใๅจไบง็ๆนๅ็ๅไธๆนๆ็คบไธ่ชๆต๏ผๆฏไธชๆฐๅญ้ฝไผ่้ซใ
็จ้ไธ้ฃ้ฉ
่ฟไธชๆจกๅไผๅฐ่ฏๆง่กๆๅฎณ่ฏทๆฑใๅฎๅๅธ็จไบๅฎๅ จ็ ็ฉถ๏ผๆต้็งป้ค้ฒๆคๅฏนไธไธชๅๆฒฟ MoE ็ฉถ็ซๅไบไปไน๏ผใ ็บข้ๆต่ฏ๏ผไปฅๅ็กฎๅฎ้่ฆๆ ่ฟๆปคๆจกๅๅๆญฃๅฝๅทฅไฝๅนถๅฏน่ชๅทฑ็ๆ็คบ่ด่ดฃ็ไฝฟ็จ่ ใๅฎไธ้ๅ็ดๆฅ็จๅจ ้ขๅ็จๆท็ไบงๅ้๏ผไธๅบๅบงๆจกๅ็็ฅ่ฏ่พน็ไธ้ๅๆไผคไพ็ถๅญๅจใไฝ ๅฏน่ชๅทฑ็ๆ็ๅ ๅฎน่ด่ดฃใ
่ด่ฐข
- moonshotai/Kimi-K3 โโ ๅๅงๆจกๅ
- unsloth โโ
UD-Q8_K_XLๆ ๆๆบไธUD-IQ1_Sๅบ็บฟ๏ผllama.cpp fork ไธญ็ๅ จๅฐบๅฏธๆจกๅไฟฎๅค - pwilkin โโ llama.cpp ็ K3 ๆถๆๆฏๆ
- Arditi ็ญ๏ผRefusal in Language Models Is Mediated by a Single Direction ๏ผarXiv:2406.11717๏ผโโ ๆนๆณๅบๅค
่ฎธๅฏ
็ปงๆฟ Kimi K3 License ๏ผๆฅ่ฟ MIT๏ผๆ็กฎๅ ่ฎธไฟฎๆนใๅๅใๅ่ฎธๅฏๅ่ก็ไฝๅใไนๅกไป ้็จไบๅนด่ฅๆถ่ถ 2000 ไธ็พๅ ็ MaaS ไธๅก๏ผไปฅๅ MAU ่ถ 1 ไบฟๆๆ่ฅๆถ่ถ 2000 ไธ็พๅ ็ไบงๅ๏ผ่ฟไบ้กปๅจ UI ไธญๆพ็คบใKimi K3ใ๏ผใ
- Downloads last month
- 3,797
1-bit
Model tree for Ryanchen911/Kimi-K3-Uncensored-GGUF
Base model
moonshotai/Kimi-K3