Instructions to use luluw/Qwen3-MedEmbed-0.6B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use luluw/Qwen3-MedEmbed-0.6B-GGUF with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("luluw/Qwen3-MedEmbed-0.6B-GGUF") sentences = [ "Liver cancer treatments options", "The patient underwent mesohepatectomy with total caudate lobectomy of the liver.", "The patient was treated empirically with amikacin and ceftazidime, which was later replaced with meropenem and piperacillin-tazobactam. From August 3 to August 11, the patient was treated with colistin.", "A porto-systemic shunt was created to increase blood flow. Transarterial chemoembolization (TACE) was performed using 5-fluorouracil (5-FU), lobaplatin, and pirarubicin to treat tumor thrombi in the Right IntraHepatic Vessels (RIPV)." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use luluw/Qwen3-MedEmbed-0.6B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible embeddings server: llama serve -hf luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M --embeddings
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible embeddings server: llama serve -hf luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M --embeddings
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible embeddings server: ./build/bin/llama serve -hf luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M --embeddings
- Llama new
- LM Studio
- Jan
- Ollama
How to use luluw/Qwen3-MedEmbed-0.6B-GGUF with Ollama:
ollama run hf.co/luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use luluw/Qwen3-MedEmbed-0.6B-GGUF with Docker Model Runner:
docker model run hf.co/luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M
- Lemonade
How to use luluw/Qwen3-MedEmbed-0.6B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull luluw/Qwen3-MedEmbed-0.6B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Qwen3-MedEmbed-0.6B-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Qwen3-MedEmbed-0.6B
This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B on the med_embed-training-triplets-v1 dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: Qwen/Qwen3-Embedding-0.6B
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 1024 dimensions
- Similarity Function: Cosine Similarity
- Training Dataset:
- Language: en
- License: apache-2.0
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
(2): Normalize()
)
Evaluation
Metrics
Triplet
- Dataset:
triplet-eval - Evaluated with
TripletEvaluator
| Metric | Value |
|---|---|
| cosine_accuracy | 0.9743 |
Information Retrieval
The dataset was splitted in 90:10 ratio. Eval set had 23269 samples. Extra 23269 negative samples were added for information retrieval.
- Datasets:
eval-set-23269kq-46538kdandeval-set-23269kq-23269kd - Evaluated with
InformationRetrievalEvaluator
| Metric | eval-set-23kq-46kd | eval-set-23kq-23kd |
|---|---|---|
| cosine_accuracy@1 | 0.4359 | 0.5438 |
| cosine_accuracy@3 | 0.8113 | 0.8997 |
| cosine_accuracy@5 | 0.9103 | 0.953 |
| cosine_accuracy@10 | 0.9586 | 0.9703 |
| cosine_precision@1 | 0.4359 | 0.5438 |
| cosine_precision@3 | 0.2704 | 0.2999 |
| cosine_precision@5 | 0.1821 | 0.1906 |
| cosine_precision@10 | 0.0959 | 0.097 |
| cosine_recall@1 | 0.4359 | 0.5438 |
| cosine_recall@3 | 0.8113 | 0.8997 |
| cosine_recall@5 | 0.9103 | 0.953 |
| cosine_recall@10 | 0.9586 | 0.9703 |
| cosine_ndcg@10 | 0.7131 | 0.7837 |
| cosine_mrr@10 | 0.632 | 0.7206 |
| cosine_map@100 | 0.6331 | 0.7209 |
- Downloads last month
- 148
4-bit
6-bit
8-bit
Model tree for luluw/Qwen3-MedEmbed-0.6B-GGUF
Dataset used to train luluw/Qwen3-MedEmbed-0.6B-GGUF
Evaluation results
- Cosine Accuracy on triplet evalself-reported0.974
- Cosine Accuracy@1 on eval set 23kq 46kdself-reported0.436
- Cosine Accuracy@10 on eval set 23kq 46kdself-reported0.959
- Cosine Precision@1 on eval set 23kq 46kdself-reported0.436
- Cosine Recall@10 on eval set 23kq 46kdself-reported0.959
- Cosine NDCG@10 on eval set 23kq 46kdself-reported0.713
- Cosine MRR@10 on eval set 23kq 46kdself-reported0.632
- Cosine MAP@100 on eval set 23kq 46kdself-reported0.633
