#Transformers

5 posts tagged with "Transformers"

Tensor Logic: One Equation to Rule Them All

2025-12-08•12 min read

#tensor logic #neuro-symbolic ai #einstein notation #transformers #deep learning #machine learning

Pedro Domingos proposes that neural networks and symbolic AI are the same mathematical operation - a logical rule can be equivalently written as a tensor equation in Einstein summation notation. If true, we've been building separate tools for problems that share identical structure.

Why Your LLM Only Uses 10-20% of Its Context Window (And How TITANS Fixes It)

2025-12-08•15 min read

#ai #machine learning #transformers #memory architectures #long context #titans #miras #neural networks

GPT-4's 128K context window? It only uses about 10% effectively. Google's TITANS architecture introduces test-time memory learning that outperforms GPT-4 on long-context tasks with 70x fewer parameters.

Nested Learning: How Your Neural Network Already Learns at Multiple Timescales

2025-11-10T18:00:00+00:00•17 min read

#deep-learning #neural-networks #optimization #memory-consolidation #language-models #transformers #continual-learning

Nested Learning: The Illusion of Deep Learning Architectures - A comprehensive guide to the arXiv paper revealing how neural networks learn at multiple timescales through hierarchical optimization.

How Transformers Reprogram Themselves: Understanding In-Context Learning

2024-10-28T14:44:32Z•11 min read

#ai #machine learning #transformers #in-context learning #deep learning

When you provide GPT-4 with examples, it adapts without training. This post reveals the mechanism: ghost fine-tuning through implicit weight updates.

Using Embedding Models to Predict Sentence Complexity

2024-10-24T22:41:07+01:00•14 min read

#nlp #embeddings #machine-learning #readability #bert #transformers

Traditional readability formulas miss the mark. Modern embedding models can capture semantic nuance and syntactic structure, but do they actually predict complexity better?