🪿 RWKV7 RWKV7 models 🪿 fla-hub/rwkv7-7.2B-g0a Text Generation • 7B • Updated Aug 30, 2025 • 20 • 3 fla-hub/rwkv7-7.2B-g0 Text Generation • 7B • Updated Aug 6, 2025 • 14 • 3 fla-hub/rwkv7-2.9B-g1 Text Generation • 3B • Updated Aug 6, 2025 • 26 • 3 fla-hub/rwkv7-2.9B-world Text Generation • 3B • Updated May 7, 2025 • 62 • 4
GSA fla-hub/gsa-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 21 fla-hub/gsa-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 15 Gated Slot Attention for Efficient Linear-Time Sequence Modeling Paper • 2409.07146 • Published Sep 11, 2024 • 20
Gated Slot Attention for Efficient Linear-Time Sequence Modeling Paper • 2409.07146 • Published Sep 11, 2024 • 20
GLA fla-hub/gla-1.3B-100B Text Generation • 1B • Updated Sep 9, 2025 • 434 • 1 fla-hub/gla-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 243 Gated Linear Attention Transformers with Hardware-Efficient Training Paper • 2312.06635 • Published Dec 11, 2023 • 9
Gated Linear Attention Transformers with Hardware-Efficient Training Paper • 2312.06635 • Published Dec 11, 2023 • 9
RetNet fla-hub/retnet-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 232 • 1 fla-hub/retnet-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 48 • 1 Retentive Network: A Successor to Transformer for Large Language Models Paper • 2307.08621 • Published Jul 17, 2023 • 172
Retentive Network: A Successor to Transformer for Large Language Models Paper • 2307.08621 • Published Jul 17, 2023 • 172
HGRN fla-hub/hgrn-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 13 fla-hub/hgrn-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 15 Hierarchically Gated Recurrent Neural Network for Sequence Modeling Paper • 2311.04823 • Published Nov 8, 2023 • 2
Hierarchically Gated Recurrent Neural Network for Sequence Modeling Paper • 2311.04823 • Published Nov 8, 2023 • 2
Qwen2.5 fla-hub/transformer-1.5B-qwen2.5 2B • Updated Feb 13, 2025 • 17 • 1 fla-hub/transformer-1.5B-qwen2.5-instruct 2B • Updated Feb 13, 2025 • 9 fla-hub/transformer-3B-qwen2.5 3B • Updated Feb 13, 2025 • 7 fla-hub/transformer-3B-qwen2.5-instruct 3B • Updated Feb 13, 2025 • 11
DeltaNet fla-hub/delta_net-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 459 fla-hub/delta_net-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 14 • 1 Parallelizing Linear Transformers with the Delta Rule over Sequence Length Paper • 2406.06484 • Published Jun 10, 2024 • 4
Parallelizing Linear Transformers with the Delta Rule over Sequence Length Paper • 2406.06484 • Published Jun 10, 2024 • 4
HGRN2 fla-hub/hgrn2-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 191 fla-hub/hgrn2-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 146 HGRN2: Gated Linear RNNs with State Expansion Paper • 2404.07904 • Published Apr 11, 2024 • 20
RWKV6 fla-hub/rwkv6-7B-finch Text Generation • 8B • Updated Jun 13, 2024 • 70 • 1 fla-hub/rwkv6-1.6B-finch Text Generation • 2B • Updated Jun 13, 2024 • 262 • 1
Mamba fla-hub/mamba-1.3B-100B Text Generation • 1B • Updated Aug 31, 2024 • 17 fla-hub/mamba-2.7B-100B Text Generation • 3B • Updated Oct 1, 2024 • 9 • 2
Transformer++ fla-hub/transformer-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 996 fla-hub/transformer-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 13 fla-hub/transformer-7B-mistral Text Generation • 7B • Updated Feb 9, 2025 • 15
🔥 flame A collection of baselines trained by 🔥 flame fla-hub/transformer-340M-4K-0.5B-20480-lr3e-4-cosine 0.4B • Updated Mar 14, 2025 • 9 • 1 fla-hub/transformer-340M-4K-0.5B-20480-lr3e-4-decay0.1-sqrt 0.4B • Updated Mar 14, 2025 • 23
🪿 RWKV7 RWKV7 models 🪿 fla-hub/rwkv7-7.2B-g0a Text Generation • 7B • Updated Aug 30, 2025 • 20 • 3 fla-hub/rwkv7-7.2B-g0 Text Generation • 7B • Updated Aug 6, 2025 • 14 • 3 fla-hub/rwkv7-2.9B-g1 Text Generation • 3B • Updated Aug 6, 2025 • 26 • 3 fla-hub/rwkv7-2.9B-world Text Generation • 3B • Updated May 7, 2025 • 62 • 4
DeltaNet fla-hub/delta_net-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 459 fla-hub/delta_net-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 14 • 1 Parallelizing Linear Transformers with the Delta Rule over Sequence Length Paper • 2406.06484 • Published Jun 10, 2024 • 4
Parallelizing Linear Transformers with the Delta Rule over Sequence Length Paper • 2406.06484 • Published Jun 10, 2024 • 4
GSA fla-hub/gsa-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 21 fla-hub/gsa-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 15 Gated Slot Attention for Efficient Linear-Time Sequence Modeling Paper • 2409.07146 • Published Sep 11, 2024 • 20
Gated Slot Attention for Efficient Linear-Time Sequence Modeling Paper • 2409.07146 • Published Sep 11, 2024 • 20
HGRN2 fla-hub/hgrn2-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 191 fla-hub/hgrn2-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 146 HGRN2: Gated Linear RNNs with State Expansion Paper • 2404.07904 • Published Apr 11, 2024 • 20
GLA fla-hub/gla-1.3B-100B Text Generation • 1B • Updated Sep 9, 2025 • 434 • 1 fla-hub/gla-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 243 Gated Linear Attention Transformers with Hardware-Efficient Training Paper • 2312.06635 • Published Dec 11, 2023 • 9
Gated Linear Attention Transformers with Hardware-Efficient Training Paper • 2312.06635 • Published Dec 11, 2023 • 9
RWKV6 fla-hub/rwkv6-7B-finch Text Generation • 8B • Updated Jun 13, 2024 • 70 • 1 fla-hub/rwkv6-1.6B-finch Text Generation • 2B • Updated Jun 13, 2024 • 262 • 1
RetNet fla-hub/retnet-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 232 • 1 fla-hub/retnet-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 48 • 1 Retentive Network: A Successor to Transformer for Large Language Models Paper • 2307.08621 • Published Jul 17, 2023 • 172
Retentive Network: A Successor to Transformer for Large Language Models Paper • 2307.08621 • Published Jul 17, 2023 • 172
Mamba fla-hub/mamba-1.3B-100B Text Generation • 1B • Updated Aug 31, 2024 • 17 fla-hub/mamba-2.7B-100B Text Generation • 3B • Updated Oct 1, 2024 • 9 • 2
HGRN fla-hub/hgrn-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 13 fla-hub/hgrn-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 15 Hierarchically Gated Recurrent Neural Network for Sequence Modeling Paper • 2311.04823 • Published Nov 8, 2023 • 2
Hierarchically Gated Recurrent Neural Network for Sequence Modeling Paper • 2311.04823 • Published Nov 8, 2023 • 2
Transformer++ fla-hub/transformer-1.3B-100B Text Generation • 1B • Updated Feb 9, 2025 • 996 fla-hub/transformer-2.7B-100B Text Generation • 3B • Updated Feb 9, 2025 • 13 fla-hub/transformer-7B-mistral Text Generation • 7B • Updated Feb 9, 2025 • 15
Qwen2.5 fla-hub/transformer-1.5B-qwen2.5 2B • Updated Feb 13, 2025 • 17 • 1 fla-hub/transformer-1.5B-qwen2.5-instruct 2B • Updated Feb 13, 2025 • 9 fla-hub/transformer-3B-qwen2.5 3B • Updated Feb 13, 2025 • 7 fla-hub/transformer-3B-qwen2.5-instruct 3B • Updated Feb 13, 2025 • 11
🔥 flame A collection of baselines trained by 🔥 flame fla-hub/transformer-340M-4K-0.5B-20480-lr3e-4-cosine 0.4B • Updated Mar 14, 2025 • 9 • 1 fla-hub/transformer-340M-4K-0.5B-20480-lr3e-4-decay0.1-sqrt 0.4B • Updated Mar 14, 2025 • 23
