(( QTRL Catalog ))
Collection
35 items • Updated
MindQTRL
mindqtrl
AI & ML interests
None yet
Recent Activity
updated a collection about 5 hours ago
(( QTRL Catalog )) updated a collection 11 days ago
(( QTRL Catalog )) updated a collection 11 days ago
(( SMART ))Organizations
None yet
updated a
collection about 5 hours ago
updated 2
collections 11 days ago
updated 2
models 24 days ago
published 2
models 27 days ago
updated 2
models 27 days ago
published 2
models about 1 month ago
updated a
collection about 1 month ago
updated a
collection 2 months ago
liked a
model 2 months ago
updated a
collection 2 months ago
reactedto codelion's post with 🔥 2 months ago
Post
3241
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
- 1 reply
updated 3
collections 2 months ago