#Transformers

8 posts tagged with "Transformers"

End-to-End Test-Time Training: Making Long Context Work Without the Memory Tax

2026-01-15T11:00:00Z•16 min read

#llm #long context #test-time training #machine learning #transformers #inference optimization

How TTT-E2E achieves constant inference latency regardless of context length by treating long context as a learning problem rather than an architecture problem.

ChemBERTa: When Language Models Learned to Speak Chemistry

2026-01-08T11:00:00Z•20 min read

#machine learning #chemistry #transformers #drug discovery #molecular property prediction

How the same transformer architecture powering GPT learned to predict molecular properties by treating chemistry as a language problem

MolBERT: Teaching Transformers to Read the Language of Chemistry

2026-01-08T10:00:00Z•21 min read

#machine learning #drug discovery #transformers #chemistry #smiles #bert

How researchers adapted BERT for molecular property prediction, turning SMILES strings into drug discovery insights

Tensor Logic: One Equation to Rule Them All

2025-12-08•12 min read

#tensor logic #neuro-symbolic ai #einstein notation #transformers #deep learning #machine learning

Pedro Domingos proposes that neural networks and symbolic AI are the same mathematical operation - a logical rule can be equivalently written as a tensor equation in Einstein summation notation. If true, we've been building separate tools for problems that share identical structure.

Why Your LLM Only Uses 10-20% of Its Context Window (And How TITANS Fixes It)

2025-12-08•15 min read

#ai #machine learning #transformers #memory architectures #long context #titans #miras #neural networks

GPT-4's 128K context window? It only uses about 10% effectively. Google's TITANS architecture introduces test-time memory learning that outperforms GPT-4 on long-context tasks with 70x fewer parameters.

Nested Learning: How Your Neural Network Already Learns at Multiple Timescales

2025-11-10T18:00:00+00:00•17 min read

#deep-learning #neural-networks #optimization #memory-consolidation #language-models #transformers #continual-learning

Nested Learning: The Illusion of Deep Learning Architectures - A comprehensive guide to the arXiv paper revealing how neural networks learn at multiple timescales through hierarchical optimization.

How Transformers Reprogram Themselves: Understanding In-Context Learning

2024-10-28T14:44:32Z•11 min read

#ai #machine learning #transformers #in-context learning #deep learning

When you provide GPT-4 with examples, it adapts without training. This post reveals the mechanism: ghost fine-tuning through implicit weight updates.

Using Embedding Models to Predict Sentence Complexity

2024-10-24T22:41:07+01:00•14 min read

#nlp #embeddings #machine-learning #readability #bert #transformers

Traditional readability formulas miss the mark. Modern embedding models can capture semantic nuance and syntactic structure, but do they actually predict complexity better?