Building the Automated AGI Lab: Core Automation's Jerry Tworek and Rohan Anil

Key takeaways

Transformers have powered recent AI advances but are reaching their fundamental limits, especially in continual, test-time learning.
The future of AI development involves designing new architectures that can learn lifelong, adaptively, and more efficiently, rather than solely scaling existing models.
Integrating pre-training with reinforcement learning into a seamless, end-to-end training process could lead to more adaptable and intelligent systems.
Hardware and low-level optimization, including kernels and custom hardware, are critical bottlenecks and opportunities for accelerating AI progress.
Core Automation envisions building an autonomous, highly automated AI research lab that emphasizes rapid experimentation, hardware-efficient architectures, and the discovery of next-generation AI models.
Achieving true AGI will require systems capable of self-improvement and adaptation without human input, a state that current AI technology has yet to reach.

Summary

The Limits of Transformers and the Need for New Architectures

The discussion highlights that while transformers have been revolutionary and economically valuable, they now face fundamental limitations, particularly in continual learning. Speakers argue that transformers are incapable of truly adapting or learning from deployment in a continuous manner, which is essential for progress toward AGI. The conversation emphasizes that the future of AI architecture should move beyond transformers to models designed explicitly for lifelong, test-time learning, capable of adapting to new information without retraining from scratch.

Reevaluating Machine Learning Paradigms: From Scale to Architecture

Both speakers challenge the current paradigm that bigger models and more compute are the primary paths to advanced intelligence. They discuss that recent successes have been driven by scaling, but to achieve smarter, more adaptable systems, the focus must shift to developing innovative architectures that are more expressive and better suited for real-world deployment. The dialogue stresses the importance of architectural research at scales large enough to uncover breakthroughs, which most big labs are reluctant to pursue due to economic or strategic reasons, thus opening opportunities for alternative approaches.

The Future of Learning Algorithms and End-to-End Training

The discussion dives into the need for integrating pre-training with reinforcement learning, emphasizing that current practices treat these as separate phases. Speakers advocate for end-to-end optimization of models that learn continuously from deployment and experience, reducing inefficiencies and enabling models to adapt on the fly. They highlight that current training methods, like in-context learning and fine-tuning, have limitations such as data inefficiency and catastrophic forgetting, thereby underscoring the importance of developing new algorithms designed explicitly for test-time adaptation.

Innovations in Hardware and Kernel Optimization for AI

A major bottleneck in advancing AI models is computational efficiency, especially hardware utilization. The speakers discuss efforts to improve kernel performance and hardware-specific optimizations, asserting that these are crucial for enabling larger, more capable models and architectures. They cite examples like the QR kernel optimization, which can drastically accelerate matrix operations, making a profound difference in training and inference costs. The conversation underscores that breakthroughs in low-level efficiency are key to unlocking next-generation AI systems.

Automating Scientific Research and Building an Autonomous Lab

Core Automation aims to create a highly automated AI research lab capable of rapidly experimenting with new architectures, training methods, and optimization techniques. The vision includes maximizing human agency, enabling researchers to iterate much faster, and eventually scaling this automation to the point where the lab can independently discover and improve architectures. This approach is driven by the belief that persistent iteration, hardware/software advancements, and kernel-level optimizations are necessary to surpass current limitations and accelerate progress towards truly intelligent systems.

Defining and Recognizing AGI in Practice

The participants explore the concept of AGI, emphasizing that a true AGI would be a system capable of self-improvement or self-augmentation without human intervention. They discuss that current AI systems are still heavily dependent on human-in-the-loop processes. The conversation concludes that achieving AGI would require models that can learn and adapt continuously in real-time, with minimal external input, a milestone that remains far from current capabilities.

Show notes

Jerry Tworek led reasoning at OpenAI, convinced that scaling reinforcement learning was the path to AGI. Rohan Anil co-led Gemini pre-training and built the Shampoo optimizer. Now they've teamed up at Core Automation on a contrarian premise: the transformer has carried us as far as it can, and the bottleneck to smarter systems is no longer scale — it's the architecture itself. The missing capability is continual learning, models that adapt at test time, which transformers can't do. In-context learning taps out fast (Codex needs compacting after ~20 minutes) and fine-tuning invites catastrophic forgetting. Rohan argues pre-training and RL should be optimized end-to-end, and that transformers spend computation inefficiently. They lay out why the largest labs won't chase alternatives while locked in the coding-agent race, and why building the world's most automated lab starts with automating kernel generation—the one place frontier models still lose to a high-taste human.

Hosted by Sonya Huang and Pat Grady, Sequoia Capital