Why Language Models Still Can't Spell: The Case for Morphologically-Aware Tokenization2024-09-03T13:47:30+01:00•38 min read#nlp#tokenization#morphology#linguistics#ai#language-modelsLanguage models can write poetry but struggle with basic spelling. Discover why current tokenization breaks language, and how morphology-aware approaches fix it.