Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss
Paper • 2608.03796 • Published • 16
Iñigo Pikabea
inigopm
AI & ML interests
None yet
Recent Activity
upvoted an article 15 days ago
Making Knowledge Distillation Cheap Enough to Run at Scale updated a dataset 4 months ago
inigopm/RealWorldQA-IberOrganizations
None yet
upvoted a paper 15 days ago
upvoted an article 15 days ago
Article
Making Knowledge Distillation Cheap Enough to Run at Scale
MultiverseComputingCAI
• • 38 upvoted a paper about 1 year ago
upvoted an article about 1 year ago
Article
SmolLM3: smol, multilingual, long-context reasoner
- +21
eliebak, cmpatino, anton-l, edbeeching, m-ric, nouamanetazi, akseljoonas, guipenedo, hynky, clefourrier, SaylorTwift, kashif, qgallouedec, hlarcher, glutamatt, Xenova, reach-vb, ngxson, craffel, lewtun, loubnabnl, lvwerra, thomwolf
• • 790 upvoted a paper over 1 year ago
upvoted an article over 1 year ago
Article
Open-R1: a fully open reproduction of DeepSeek-R1
- +1
eliebak, lvwerra, lewtun
• • 891 upvoted a collection over 1 year ago