meta-llama/Llama-3.1-8B-Instruct
Text Generation • 8B • Updated • 8.91M • • 5.68k
HIGGS
Models from inference-optimization org containing quantization bit info
-
-
RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic
Text Generation • 8B • Updated • 27.4k • 9 -
RedHatAI/Llama-3.1-8B-Instruct-NVFP4
Text Generation • 5B • Updated • 19.5k • 1 -
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_hybrid
6B • Updated • 29 -
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_noise
6B • Updated • 35 -
inference-optimization/Llama-3.1-8B-Instruct_5_bits_mode_heuristic
6B • Updated • 28 -
inference-optimization/Llama-3.1-8B-Instruct_5.5_bits_mode_hybrid
6B • Updated • 38 -
inference-optimization/Llama-3.1-8B-Instruct_5.5_bits_mode_noise
6B • Updated • 29 -
inference-optimization/Llama-3.1-8B-Instruct_5.5_bits_mode_heuristic
6B • Updated • 26 -
inference-optimization/Llama-3.1-8B-Instruct_6_bits_mode_hybrid
6B • Updated • 30 -
inference-optimization/Llama-3.1-8B-Instruct_6_bits_mode_noise
6B • Updated • 29 -
inference-optimization/Llama-3.1-8B-Instruct_6_bits_mode_heuristic
6B • Updated • 32 -
inference-optimization/Llama-3.1-8B-Instruct_6.5_bits_mode_hybrid
7B • Updated • 31 -
inference-optimization/Llama-3.1-8B-Instruct_6.5_bits_mode_noise
7B • Updated • 28 -
inference-optimization/Llama-3.1-8B-Instruct_6.5_bits_mode_heuristic
7B • Updated • 40 -
inference-optimization/Llama-3.1-8B-Instruct_7_bits_mode_hybrid
7B • Updated • 36 -
inference-optimization/Llama-3.1-8B-Instruct_7_bits_mode_noise
7B • Updated • 27 -
inference-optimization/Llama-3.1-8B-Instruct_7_bits_mode_heuristic
7B • Updated • 26 -
meta-llama/Llama-3.2-1B-Instruct
Text Generation • 1B • Updated • 4.16M • • 1.36k -
inference-optimization/Llama-3.2-1B-Instruct-FP8-Dynamic
1B • Updated • 17 -
inference-optimization/Llama-3.2-1B-Instruct-NVFP4
0.8B • Updated • 18 -
inference-optimization/Llama-3.2-1B-Instruct_5_bits_mode_hybrid
1B • Updated • 36 -
inference-optimization/Llama-3.2-1B-Instruct_5_bits_mode_noise
1B • Updated • 40 -
inference-optimization/Llama-3.2-1B-Instruct_5_bits_mode_heuristic
1B • Updated • 38 -
inference-optimization/Llama-3.2-1B-Instruct_5.5_bits_mode_hybrid
1B • Updated • 41 -
inference-optimization/Llama-3.2-1B-Instruct_5.5_bits_mode_noise
1B • Updated • 46 -
inference-optimization/Llama-3.2-1B-Instruct_5.5_bits_mode_heuristic
1B • Updated • 42 -
inference-optimization/Llama-3.2-1B-Instruct_6_bits_mode_hybrid
1B • Updated • 44 -
inference-optimization/Llama-3.2-1B-Instruct_6_bits_mode_noise
1B • Updated • 40 -
inference-optimization/Llama-3.2-1B-Instruct_6_bits_mode_heuristic
1B • Updated • 32 -
inference-optimization/Llama-3.2-1B-Instruct_6.5_bits_mode_hybrid
1B • Updated • 45 -
inference-optimization/Llama-3.2-1B-Instruct_6.5_bits_mode_noise
1B • Updated • 40 -
inference-optimization/Llama-3.2-1B-Instruct_6.5_bits_mode_heuristic
1B • Updated • 29 -
inference-optimization/Llama-3.2-1B-Instruct_7_bits_mode_hybrid
1B • Updated • 30 -
inference-optimization/Llama-3.2-1B-Instruct_7_bits_mode_noise
1B • Updated • 40 -
inference-optimization/Llama-3.2-1B-Instruct_7_bits_mode_heuristic
1B • Updated • 28 -
meta-llama/Llama-3.2-3B-Instruct
Text Generation • 3B • Updated • 5.86M • 2.09k -
inference-optimization/Llama-3.2-3B-Instruct-FP8-Dynamic
3B • Updated • 12 -
inference-optimization/Llama-3.2-3B-Instruct-NVFP4
2B • Updated • 16 -
inference-optimization/Llama-3.2-3B-Instruct_5_bits_mode_hybrid
3B • Updated • 23 -
inference-optimization/Llama-3.2-3B-Instruct_5_bits_mode_noise
3B • Updated • 25 -
inference-optimization/Llama-3.2-3B-Instruct_5_bits_mode_heuristic
3B • Updated • 25 -
inference-optimization/Llama-3.2-3B-Instruct_5.5_bits_mode_hybrid
3B • Updated • 26 -
inference-optimization/Llama-3.2-3B-Instruct_5.5_bits_mode_noise
3B • Updated • 24 -
inference-optimization/Llama-3.2-3B-Instruct_5.5_bits_mode_heuristic
3B • Updated • 24 -
inference-optimization/Llama-3.2-3B-Instruct_6_bits_mode_hybrid
3B • Updated • 22 -
inference-optimization/Llama-3.2-3B-Instruct_6_bits_mode_noise
3B • Updated • 25 -
inference-optimization/Llama-3.2-3B-Instruct_6_bits_mode_heuristic
3B • Updated • 23 -
inference-optimization/Llama-3.2-3B-Instruct_6.5_bits_mode_hybrid
3B • Updated • 22 -
inference-optimization/Llama-3.2-3B-Instruct_6.5_bits_mode_noise
3B • Updated • 25 -
inference-optimization/Llama-3.2-3B-Instruct_6.5_bits_mode_heuristic
3B • Updated • 24 -
inference-optimization/Llama-3.2-3B-Instruct_7_bits_mode_hybrid
3B • Updated • 24 -
inference-optimization/Llama-3.2-3B-Instruct_7_bits_mode_noise
3B • Updated • 21 -
inference-optimization/Llama-3.2-3B-Instruct_7_bits_mode_heuristic
3B • Updated • 23 -
Qwen/Qwen3-8B
Text Generation • 8B • Updated • 8.78M • • 1.04k -
RedHatAI/Qwen3-8B-FP8-dynamic
Text Generation • 8B • Updated • 20.3k • 12 -
RedHatAI/Qwen3-8B-NVFP4
Text Generation • 5B • Updated • 1.24k • 1 -
inference-optimization/Qwen3-8B_5_bits_mode_hybrid
6B • Updated • 39 -
inference-optimization/Qwen3-8B_5_bits_mode_noise
6B • Updated • 36 -
inference-optimization/Qwen3-8B_5_bits_mode_heuristic
6B • Updated • 31 -
inference-optimization/Qwen3-8B_5.5_bits_mode_hybrid
6B • Updated • 31 -
inference-optimization/Qwen3-8B_5.5_bits_mode_noise
6B • Updated • 32 -
inference-optimization/Qwen3-8B_5.5_bits_mode_heuristic
6B • Updated • 36 -
inference-optimization/Qwen3-8B_6_bits_mode_hybrid
7B • Updated • 80 -
inference-optimization/Qwen3-8B_6_bits_mode_noise
7B • Updated • 30 -
inference-optimization/Qwen3-8B_6_bits_mode_heuristic
7B • Updated • 31 -
inference-optimization/Qwen3-8B_6.5_bits_mode_hybrid
7B • Updated • 33 -
inference-optimization/Qwen3-8B_6.5_bits_mode_noise
7B • Updated • 32 -
inference-optimization/Qwen3-8B_6.5_bits_mode_heuristic
7B • Updated • 28 -
inference-optimization/Qwen3-8B_7_bits_mode_hybrid
7B • Updated • 32 -
inference-optimization/Qwen3-8B_7_bits_mode_noise
7B • Updated • 32 -
inference-optimization/Qwen3-8B_7_bits_mode_heuristic
7B • Updated • 32 -
Qwen/Qwen3-30B-A3B
Text Generation • Updated • 1.59M • • 874 -
RedHatAI/Qwen3-30B-A3B-FP8-dynamic
Text Generation • 31B • Updated • 4.09k • 3 -
RedHatAI/Qwen3-30B-A3B-NVFP4
Text Generation • 17B • Updated • 26.8k • 2 -
inference-optimization/Qwen3-30B-A3B_5.0_bits_mode_hybrid
20B • Updated • 23 -
inference-optimization/Qwen3-30B-A3B_5.0_bits_mode_noise
20B • Updated • 26 -
inference-optimization/Qwen3-30B-A3B_5.0_bits_mode_heuristic
20B • Updated • 26 -
inference-optimization/Qwen3-30B-A3B_5.5_bits_mode_hybrid
22B • Updated • 27 -
inference-optimization/Qwen3-30B-A3B_5.5_bits_mode_noise
22B • Updated • 30 • 1 -
inference-optimization/Qwen3-30B-A3B_5.5_bits_mode_heuristic
22B • Updated • 26 -
inference-optimization/Qwen3-30B-A3B_6.0_bits_mode_hybrid
23B • Updated • 25 -
inference-optimization/Qwen3-30B-A3B_6.0_bits_mode_noise
24B • Updated • 20 -
inference-optimization/Qwen3-30B-A3B_6.0_bits_mode_heuristic
23B • Updated • 26 -
inference-optimization/Qwen3-30B-A3B_6.5_bits_mode_hybrid
24B • Updated • 19 -
inference-optimization/Qwen3-30B-A3B_6.5_bits_mode_noise
25B • Updated • 28 -
inference-optimization/Qwen3-30B-A3B_6.5_bits_mode_heuristic
25B • Updated • 26 -
inference-optimization/Qwen3-30B-A3B_7.0_bits_mode_hybrid
25B • Updated • 24 -
inference-optimization/Qwen3-30B-A3B_7.0_bits_mode_noise
27B • Updated • 29 -
inference-optimization/Qwen3-30B-A3B_7.0_bits_mode_heuristic
27B • Updated • 27 -
Qwen/Qwen3-30B-A3B-Instruct-2507
Text Generation • Updated • 1.15M • • 798 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507-FP8-Dynamic
31B • Updated • 15 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507-NVFP4
17B • Updated • 18 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.0_bits_mode_hybrid
20B • Updated • 28 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.0_bits_mode_noise
20B • Updated • 23 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.0_bits_mode_heuristic
20B • Updated • 27 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.5_bits_mode_hybrid
22B • Updated • 25 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.5_bits_mode_noise
22B • Updated • 24 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_5.5_bits_mode_heuristic
22B • Updated • 25 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.0_bits_mode_hybrid
23B • Updated • 28 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.0_bits_mode_noise
23B • Updated • 23 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.0_bits_mode_heuristic
23B • Updated • 31 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.5_bits_mode_hybrid
25B • Updated • 29 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.5_bits_mode_noise
25B • Updated • 25 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_6.5_bits_mode_heuristic
25B • Updated • 29 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_7.0_bits_mode_hybrid
26B • Updated • 34 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_7.0_bits_mode_noise
26B • Updated • 30 -
inference-optimization/Qwen3-30B-A3B-Instruct-2507_7.0_bits_mode_heuristic
27B • Updated • 28