Article
There is no such thing as a tokenizer-free lunch
catherinearnett
• • 98Catherine Arnett
catherinearnett
AI & ML interests
multilingual NLP, tokenization
Recent Activity
published a dataset about 9 hours ago
catherinearnett/trilingual-tokenizers published a dataset about 9 hours ago
catherinearnett/trilingual-tokenizer-data updated a model 7 days ago
catherinearnett/afr_Latn_als_Latn_9_91_bpe_nowhitespace_16384Organizations
published an article 8 months ago
published an article 8 months ago
Article
An Analysis of Multilingual Models on Hugging Face
catherinearnett
• • 5 published an article about 1 year ago
Article
Best Practices for Open Multilingual LLM Evaluation
catherinearnett
• • 6 published an article over 1 year ago
Article
They Said It Couldn’t Be Done
Pclanglais
• • 91 published an article over 1 year ago
Article
Releasing the largest multilingual open pretraining dataset
Pclanglais
• • 107 published an article over 1 year ago
Article
Detoxifying the Commons
catherinearnett
• • 6 published an article over 1 year ago
Article
wHy DoNt YoU jUsT uSe ThE lLaMa ToKeNiZeR??
catherinearnett
• • 54