Invest Like the Best with Patrick O'Shaughnessy
Etched - Building AI Hardware to Make Inference Faster and Cheaper - [Invest Like the Best, EP.480]
· Summarized
Key takeaways
- Disruptive innovation in AI hardware hinges on focusing on critical design principles such as low-voltage operation, high inter-chip bandwidth, and extreme vertical integration.
- Achieving scale in AI inference requires not only advanced chip architecture but also manufacturing supply chain mastery, especially with partners like TSMC.
- A culture of relentless speed, naive belief in the possible, and aggressive risk-taking are crucial for breakthrough technological developments.
- Hardware design choices directly shape the future capabilities of AI models, enabling faster, more dynamic, and more context-aware systems.
- Continual iteration, even with high failure rates in experiments, is necessary to solve complex engineering challenges and push physical and technological limits.
Summary
Innovations in AI Hardware Design by Etched
This episode delves into the groundbreaking work of Gavin Uberti and Rob Wachen, founders of Etched, who are building hardware to accelerate AI inference at a lower cost. They challenge industry norms with their first-in-kind chip and system architecture, emphasizing their technical bets around low-voltage inference and high inter-chip bandwidth. The founders discuss their approach to full vertical integration in hardware, their focus on scalability, and how their younger team members and veteran industry legends collaborate to solve complex engineering problems, like clock phase alignment at a picosecond level, to achieve performance breakthroughs.
Market Potential and Industry Impact of AI Chips
The conversation explores why inference could become one of the largest markets in the world, emphasizing how scalable and efficient hardware can democratize AI usage globally. The founders describe their ambition to produce trillions of tokens, reduce environmental impact per unit of compute, and support the proliferation of AI-powered agents functioning at a planetary scale. They discuss their strategic focus on supply chain resilience, partnership with TSMC, and targeted hardware design principles that prioritize mass production to meet soaring demand.
Origins, Team Building, and Overcoming Challenges
Gavin and Rob share their personal stories, including overcoming severe health challenges and their early entry into tech entrepreneurship. They discuss their recruitment philosophy—focusing on industry legends paired with young, naive talent—and strategies for attracting top-tier talent willing to take significant risks. The episode highlights their intense, hands-on project execution, from Bangalore hardware sprints to wafer-level testing, along with managing supply chain constraints and navigating early funding hurdles, including raising a record-breaking Series A of $800 million amidst skepticism.
Technical and Business Lessons Learned
The founders reflect on key lessons, such as the importance of speed, vertical integration, and making disruptive technical bets. They emphasize the need to assume the impossible is possible, handle failure as a learning process, and relentlessly iterate from experiments—often with a high failure rate—to finally succeed. They also discuss their strategy to stay ahead by controlling the entire hardware and system stack, focusing on simplicity, scalability, and supply chain mastery, to stay competitive in the rapidly evolving AI hardware ecosystem.
Future of AI and Hardware Infrastructure
Looking ahead, they forecast exponential growth in inference-driven productivity, AI agents, and the economic impacts of scaled AI infrastructure. Gavin and Rob see trillion-dollar data centers and AI systems transforming civilization, with AI playing a central role in global GDP, workforce automation, and scientific discovery. They foresee the development of multimodal, dynamic, and highly efficient hardware architectures capable of supporting complex, long-term tasks, and the building of mega-scale AI factories to meet global demand.
Philosophy and Culture of Building a Tech Company in Defense of Skepticism and Naivete
Throughout the discussion, the importance of having a ‘sick in the head’ mentality—embracing risk, naivete, and relentless belief in seemingly impossible goals—is highlighted. The founders describe their culture of extreme aggressiveness, vertical integration, and rapid decision-making as critical to their success. They recount their focused recruitment, including targeting legendary industry veterans, and their intense work ethic—such as 12-hour days in Bangalore—showing how passion, conviction, and fearless problem-solving drive breakthrough innovations.
Capacities and Limitations of Hardware and Model Development
Gavin and Rob discuss how hardware design influences the evolution of AI models—making them faster, more dynamic, and capable of handling larger contexts with less energy. They highlight fundamental physical limits like the speed of light, but emphasize that continuous innovation will push these boundaries. They also explore how specialized hardware architectures will enable next-gen models to utilize huge parameters efficiently, powering agents that can perform complex, multi-year scientific, educational, and societal tasks.
Sponsors
- Ramp · Ramp is the only platform built to make your finance team leaner, faster, and better, saving businesses 5% annually on average so you can stay focused on growth. Ramp customers grew revenue 3.2 times faster than the average American business. Visa, Fersel, Cursor, Stripe, Notion, 11-Mibes, Shopify, and 70,000 other businesses all run on Ramp. Learn more at ramp.com slash invest.
- Vanta · Vanta automate security and compliance for over 16,000 fast-moving companies, like Ramp, Kersher, and Harvey, keeping them all ready around the clock. It helps companies watch for risks that show up between audits across vendors and AI tools, acting like a 24-7 GRC engineer in the background. It can cut vendor assessment time by up to 50%. Invest like the best listeners get a special offer for $1,000 off at vanta.com slash invest.
- Colossus · Become a Colossus member to get our quarterly print magazine and private audio experience, including exclusive profiles and early access to episodes. Subscribe at colossus.com/subscribe.
- rigo · Rogo is the AI platform for finance, building agents for Wall Street that understand how bankers and investors work—from diligence and modeling to analysis and deliverables. Learn more at rogo.ai/invest.
- Ridgeline · Ridgeline has built a modern operating system for investment managers that handles trading, portfolio management, compliance, customer reporting, and more through a unified, real-time cloud platform. Visit ridgelineapps.com to see what they can unlock for your firm.
Show notes
My guests today are Gavin Uberti and Rob Wachen, the founders of Etched.
A few years ago, when they set out to build a better AI chip than the largest companies in the world, almost everyone I called told me it could not be done. They have since done it, taping out a working chip on their first attempt and becoming the first hardware company founded after ChatGPT to do so.
They already have more than a billion dollars of customer demand for their first product, and have raised eight hundred million dollars to build it.
Etched builds chips and systems designed to run AI models faster and at lower cost. They started the company in 2023, and that product is a complete rack for inference, the chip along with the boards, the power delivery, the interconnects, and the manufacturing to produce it all.
We talk about the technical bets behind their architecture, how they hired industry legends and paired them with elite 22 year-olds, and why they believe inference will become one of the largest markets in the world.
I think you will find the story of what they have built hard to forget.
Please enjoy my conversation with Gavin and Rob.
For the full show notes, transcript, and links to mentioned content, check out the episode page here.
-----
Become a Colossus member to get our quarterly print magazine and private audio experience, including exclusive profiles and early access to select episodes. Subscribe at colossus.com/subscribe.
-----
Ramp’s mission is to help companies manage their spend in a way that reduces expenses and frees up time for teams to work on more valuable projects. Go to ramp.com/invest to sign up for free and get a $250 welcome bonus.
-----
Trusted by thousands of businesses, Vanta continuously monitors your security posture and streamlines audits so you can win enterprise deals and build customer trust without the traditional overhead. Invest Like the Best listeners get a special offer of $1,000 off Vanta when you go to vanta.com/invest.
-----
WorkOS is the infrastructure B2B and AI-native companies use to sell to enterprise. It covers everything enterprise security requires: SSO, SCIM, RBAC, Audit Logs, AI governance, and more. Trusted by 2,000+ fast-growing companies, including OpenAI, Anthropic, Cursor, and Vercel.
-----
Rogo is the AI platform for finance. They're building agents for Wall Street that are trained to understand how bankers and investors actually do work: from diligence and modeling, to turning analysis into deliverables. To learn more, visit rogo.ai/invest.
-----
Ridgeline has built a complete, real-time, modern operating system for investment managers. It handles trading, portfolio management, compliance, customer reporting, and much more through an all-in-one real-time cloud platform. Visit ridgelineapps.com.
-----
Editing and post-production work for this episode was provided by The Podcast Consultant.
Timestamps:
(00:00:00) Welcome to Invest Like The Best
(00:02:07) Gavin Uberti and Rob Wachen
(00:03:54) Two 21-Year-Olds Taking on NVIDIA
(00:07:52) The Two Technical Bets Behind Their Architecture
(00:14:15) Why Inference Becomes the Biggest Market
(00:20:23) Rob and Gavin's Origins Stories
(00:28:38) How They Recruit Industry Legends
(00:36:30) Moving a Dozen Engineers to Bangalore for Six Months
(00:38:01) Speed Wins
(00:43:58) Getting More Concurrency Out of Every Megawatt
(00:52:44) Vertical Integration
(00:57:43) Hardest Obstacles to Overcome
(01:01:09) Raising The Largest AI Chip Series A Ever
(01:06:29) TSMC
(01:13:20) Designing Gen 2 for Gigawatt-Scale Production
(01:16:42) Why Machines Don't Think Like People
(01:20:03) A Year of Compute Compressed Into a Month
(01:23:44) The Trillion-Dollar Data Center
(01:26:19) The Kindest Thing
Follow this show on AudioBrevity and new episodes get summarized for you. Create a free account