Building the Cloud for an Agentic World | AWS CEO Matt Garman

In this opening segment, host Raghu Raghuram introduces AWS CEO Matt Garman, highlighting the rapid growth of AWS from its inception in 2006 to $169 billion in revenue today. Garman reflects on AWS's early strategy of focusing on startups, emphasizing their importance as a source of innovation and future enterprise clients. He discusses AWS's ongoing commitment to supporting startups by reserving capacity, particularly for GPU-intensive AI workloads, citing their plan to purchase 2 million GPUs over the next two years. The episode underscores the massive demand for AI hardware, with graphics card values reaching around $220 billion, fueling AWS's substantial capital investments despite capacity constraints caused by supply chain issues and infrastructure bottlenecks.

Garman recounts his early involvement with AWS as an intern in 2005, noting that startups have always been central to AWS’s ecosystem, accounting for a significant portion of its revenue—an estimated 30-40%. He describes how startups today are larger and more ambitious from inception, often valued at over a billion dollars with hundreds of millions in funding. Despite their growth, their core needs around architecture, security, and scalability remain consistent with earlier days. A shift is noted in the ways teams use AWS, moving toward automation, agents, and API-driven interactions, which AWS is actively optimizing for, supporting the trend of coding agents that can manage tasks automatically and swiftly.

This segment explores how AWS is adapting its services for agent-driven workflows and automation. Garman details ongoing efforts to build new foundational tools such as sandboxing, fine-grained permissions, and virtualization technologies (like Firecracker micro VMs), tailored for agents that deploy and operate with minimal overhead. A significant focus is on simplifying onboarding, with new onboarding capabilities allowing customers to create full AWS accounts in less than 30 seconds without complex initial setup. AWS is also working on building blocks like AWS Context, which help agents access data across different repositories efficiently, enabling faster, more scalable interactions while maintaining performance and security—one of the reasons agent workflows perform better on AWS.

Garman addresses the challenge of balancing the immense demand for GPU resources from frontier labs and startups with AWS’s capacity to supply hardware. AWS plans to buy 2 million GPUs over the next few years, highlighting the scale of their investment. He explains that their CapEx has grown significantly—possibly exceeding $220 billion annually—and that supply chain constraints, such as power, construction, and component shortages, are major bottlenecks. To manage this, AWS allocates GPU resources thoughtfully, prioritizing high-growth startups and enterprise customers, and avoiding over-concentration of capacity with a single customer. The company maintains a diversified portfolio of customers and invests substantially to keep pace with demand, viewing their capital expenditures as essential to fueling AI growth.

In this section, Garman emphasizes AWS’s commitment to community and environmental sustainability, citing efforts to bring renewable energy to data centers and reduce water usage. He discusses their strategic planning for capacity expansion, which now involves multi-year forecasts and power investments, including behind-the-meter projects. Garman argues that data centers offer substantial economic benefits to communities such as lower taxes and high-paying jobs, and AWS is advocating for more transparency around these contributions. He notes that industry-wide, many data centers operate responsibly, but a few do not, damaging the reputation of the ecosystem. AWS’s approach combines robust infrastructure investments with community engagement and sustainability efforts.

Garman describes AWS’s journey into custom hardware, including chips like Graviton and Trainium. Beginning around 10-15 years ago, AWS acquired Annapurna Labs to develop network and storage offload cards, which led to the creation of Firecracker micro VMs, improving performance and security. They later transitioned to designing their own chips, starting with Graviton, which offers 20% lower cost and higher performance than commodity alternatives and is used by over 90% of their top 100 customers. Their Trainium chips, tailored for AI training, are in high demand, with capacity expected to last through next year. Garman explains that AWS’s innovative hardware approach is central to controlling costs and performance across their cloud offerings, and that they plan to continue investing heavily in this domain.