#Deep Learning

6 posts tagged with "Deep Learning"

Engram: How DeepSeek Added a Second Brain to Their LLM

2026-01-13T11:00:00Z•18 min read

#deep learning #llm architecture #memory #mixture of experts #deepseek #sparse computation

A technical deep dive into DeepSeek's Engram architecture, which introduces conditional memory as a new axis of sparsity for large language models.

VL-JEPA: Why Predicting Embeddings Beats Generating Tokens for Vision-Language AI

2025-12-30T12:00:00Z•15 min read

#vision language #jepa #deep learning #multimodal ai #self-supervised learning

VL-JEPA achieves 50% parameter reduction and 2.85x faster decoding by predicting embeddings instead of generating tokens, offering a compelling alternative to autoregressive vision-language models.

Why 1000-Layer Networks Finally Work for Reinforcement Learning

2025-12-18T12:00:00Z•10 min read

#reinforcement learning #deep learning #goal-conditioned rl #network architecture #self-supervised learning

Recent research shows 1024-layer networks achieve 2x to 50x improvements in goal-conditioned RL. Here's why extreme depth works now, and when you should consider it for your own agents.

Tensor Logic: One Equation to Rule Them All

2025-12-08•12 min read

#tensor logic #neuro-symbolic ai #einstein notation #transformers #deep learning #machine learning

Pedro Domingos proposes that neural networks and symbolic AI are the same mathematical operation - a logical rule can be equivalently written as a tensor equation in Einstein summation notation. If true, we've been building separate tools for problems that share identical structure.

When Machines Design Their Own Learning Algorithms

2025-12-08•13 min read

#reinforcement learning #meta-learning #algorithm discovery #deep learning #ai research #machine learning

A machine trained on simple grid worlds beat every hand-designed RL algorithm on Atari. DeepMind's DiscoRL discovers algorithms through meta-learning that outperform DQN, PPO, and A3C - methods humans spent decades developing.

How Transformers Reprogram Themselves: Understanding In-Context Learning

2024-10-28T14:44:32Z•11 min read

#ai #machine learning #transformers #in-context learning #deep learning

When you provide GPT-4 with examples, it adapts without training. This post reveals the mechanism: ghost fine-tuning through implicit weight updates.