#mixture of experts

Engram: How DeepSeek Added a Second Brain to Their LLM

A technical deep dive into DeepSeek's Engram architecture, which introduces conditional memory as a new axis of sparsity for large language models.