Instructions to use pfnet/plamo-3-translate-8b-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use pfnet/plamo-3-translate-8b-MLX-4bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download pfnet/plamo-3-translate-8b-MLX-4bit --local-dir plamo-3-translate-8b-MLX-4bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Download serving_memory.py from pfnet/plamo-3-translate-8b-MLX-4bit: direct link, hf CLI and curl.
- Browser
- Download file 916 Bytes
-
https://huggingface.co/pfnet/plamo-3-translate-8b-MLX-4bit/resolve/main/serving_memory.py
- Command line
-
hf download hf://pfnet/plamo-3-translate-8b-MLX-4bit/serving_memory.py
-
curl -L -o serving_memory.py https://huggingface.co/pfnet/plamo-3-translate-8b-MLX-4bit/resolve/main/serving_memory.py
916 Bytes
| """Materialize extracted prompt caches before sharing them with queued requests. | |
| In mlx-lm 0.32.0, extract() leaves contiguous/slice operations lazy. A pending | |
| request can therefore retain the entire old generation batch through its small | |
| prefix cache, even after the LRU evicts that entry. Evaluate at the ownership | |
| boundary so the queue holds the extracted tensors, not the extraction graph. | |
| """ | |
| import mlx.core as mx | |
| from mlx_lm.models.cache import LRUPromptCache | |
| def install(): | |
| original = LRUPromptCache.insert_cache | |
| if getattr(original, "_plamo_materialized", False): | |
| return | |
| def insert_cache(self, model, tokens, prompt_cache, *, cache_type="assistant"): | |
| mx.eval([cache.state for cache in prompt_cache]) | |
| return original(self, model, tokens, prompt_cache, cache_type=cache_type) | |
| insert_cache._plamo_materialized = True | |
| LRUPromptCache.insert_cache = insert_cache | |