Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO

Erik Bernhardsson describes Modal as an AI infrastructure company that provides computer and storage platforms for customers including Decagon, Suno, Runway, and Cognition. Its SDK lets developers deploy and scale inference, sandboxes, training and fine-tuning jobs, batch workloads, and other AI applications without managing the underlying capacity. He says Modal has grown by at least 50× in revenue and users since his earlier appearance on the show. Before Modal, Bernhardsson spent roughly seven years at Spotify, joining around 2008 or 2009 and building the first version of its music-recommendation system from scratch. He created a small team and launched features including Related Artists and Discover Weekly, using matrix factorization and substantial computing resources. He says the experience exposed how poor infrastructure could be for building AI applications, which became the original motivation for Modal. His five years as CTO of mortgage company Better.com contributed more to his management, hiring, developer-productivity, and full-stack experience: he grew the technology organization from roughly 10 people to 200–400. After leaving Better around 2020 or 2021, he took a pandemic-era sabbatical, experimented with several ideas, and eventually returned to infrastructure and machine learning.

Bernhardsson says he initially considered a web-scraping product, then returned to workflow orchestration, an area he had explored through Luigi, the open-source framework he released while at Spotify. While hacking on ways to run workflows and code in the cloud, he concluded that the harder and more fundamental problem was not merely scheduling compute but executing it efficiently. Modal therefore began as a general-purpose way to run code in the cloud, including GPU, machine-learning, and data workloads. The company went deep into containers, filesystems, scaling, and developer experience rather than simply wrapping existing cloud tools. Stable Diffusion, released around the same period as ChatGPT in late 2022, became the early “killer app.” Modal had only four or five people, but users began arriving to run Stable Diffusion models, often with custom workflows, adapters, and inference optimizations. Bernhardsson attributes much of the early growth to his own “founder-led marketing”: compulsive tweeting, blogging, and being “terminally online.” He self-funded Modal for a long time, including through its first employees, and raised a seed round in early 2022. Meaningful revenue arrived only in late 2022: approximately $100 per month in November, $2,000 the following month, and $30,000 in January 2023. The $30,000 month largely came from one Newark-based Stable Diffusion company, Mage, which at one point represented about 89% of revenue. Revenue later diversified, with Suno becoming a major customer in late 2023.

Bernhardsson argues that existing infrastructure tools were designed for a different era. Kubernetes and Docker, for example, generally support relatively steady backend workloads, whereas generative AI involves expensive GPUs, large models that must move through networks and memory, rapidly changing demand, and a much greater need to maximize utilization. Modal therefore built its own filesystem, container runtime, autoscaler, scheduler, and capacity-management systems, along with CPU and GPU snapshotting. The company went multi-region early and treats all of its capacity as one worldwide pool rather than as isolated regional clusters. Bernhardsson calls that a consequential decision because it lets Modal aggregate capacity across hundreds of regions and absorb preemptions or local failures. Modal has had bugs and outages, but he says it has recently reached roughly four or five nines of uptime. The infrastructure challenge became much larger as the company grew and GPU markets tightened. Modal originally relied primarily on AWS and other hyperscalers, using mostly spot and on-demand capacity, later adding GCP, Azure, Oracle, and reserved capacity. By the time of the interview it worked with almost 25 clouds. Obtaining capacity now requires three-plus-year agreements, substantial upfront payments, and a dedicated procurement function of about four people. Another engineering team of roughly 10 manages bare-metal deployments around the world, including network problems, failed drives, and GPUs “falling off the bus.” Modal rents rather than owns the hardware, though Bernhardsson remains open to purchasing or using joint ventures if the economics justify it.

The host asks whether OpenAI and Anthropic are buying all available GPUs. Bernhardsson agrees that their purchases push prices upward, but says Nvidia is shipping several hundred thousand Blackwell systems per month and may prefer a more diversified customer base rather than allocating everything to a few large buyers. Modal can also use smaller, less-established clouds that offer fragmented capacity—for example, 32 nodes in Armenia or 128 nodes in Indonesia—because it is built to aggregate capacity from many locations. Modal evaluates vendors using detailed specifications for IOPS, CPU performance, network bandwidth, ingress, and egress. It may accept one node for testing before making a commitment, but sometimes signs tens or hundreds of millions of dollars in contracts with vendors it does not yet know well. Reliability matters, but so do responsiveness, support quality, and how quickly a supplier fixes failures. Bernhardsson says Modal has developed an internal sense of which providers are consistently dependable. He is aggressively buying capacity, but does not want to “corner the market.” If demand, prices, and the resale market all fell at once, an enormous inventory could become dangerous. He compares the risk to the Hunt Brothers’ failed attempt to corner silver. In the long run, hardware prices usually decline, so owning or reserving expensive assets can expose a company to buying at the top. He expects Modal may combine renting, owning, joint ventures, and other financing structures, but prefers the capital efficiency and focus of being a software company. He also expects alternative accelerators such as Google TPUs, Cerebras, and Groq to become more important, perhaps by 2027, although Nvidia remains dominant.

Modal’s original product, Modal Functions, launched in 2021 as an easy-to-deploy, easy-to-scale serverless platform. It became especially effective for inference and GPU workloads because Modal could pool demand from thousands of customers and manage bursty capacity efficiently. Functions represented about two-thirds of revenue at the time of the interview, although Bernhardsson says it was never intended to be a rectangular GPU-rental or neocloud product. Modal launched Sandboxes in 2023 after noticing that large language models were increasingly generating code that needed to run somewhere safely. Since Modal already had container-management and isolation primitives, including gVisor, Bernhardsson built an initial prototype in roughly one or two days. It initially generated little revenue, and the company even considered shutting it down. The product began accelerating as models improved at generating complete applications, companies adopted “vibe coding,” and customers used sandboxes for background agents and reinforcement learning. Sandboxes had become roughly one-third of revenue and were still growing about 30% month over month. Bernhardsson says Modal’s existing strengths—scaling, reliability, and operating hundreds of thousands or millions of sandboxes—helped it gain a large market share. It has since added full virtual machines for workloads needing Linux compatibility, better interactive data transfer and filesystem primitives, and snapshotting. His longer-term vision is for ephemeral environments that can start in about 100 milliseconds, run a command, pause, resume, clone, or migrate. That could let agents control many sandboxes simultaneously and allow CI systems to run every test in its own sandbox instead of paying minutes of setup overhead for every shard.

Bernhardsson says Modal is becoming the multi-product infrastructure toolkit he originally envisioned. The company is working on training, batch processing, low-latency inference, speech-to-speech and voice workloads, agent infrastructure, storage, data warehousing, feature stores, and real-time streaming. He is especially bullish on smaller and mid-sized training runs, which he believes are underserved compared with enormous training clusters. Customers in biotechnology and traditional machine learning may need only 64 GPUs rather than 100,000, but setting up a reliable training cluster remains difficult. He also describes a batch customer that needed to transcribe 5,000 years of audio across millions of files; Modal used its available capacity to complete the work in a few days. On fine-tuning, he says the market has changed. A few years earlier, the argument was that a smaller open-source model could be fine-tuned to approach frontier quality. Now models such as Kimi K3, DeepSeek 4.1, and GLM 5.3 are good enough out of the box that fine-tuning is more optional, though Modal still sees demand for supervised fine-tuning and reinforcement learning. He frames Modal as a provider of composable, general-purpose building blocks rather than a company trying to predict one narrow application. The same infrastructure may serve coding agents, computational biology, drug discovery, materials science, or other applications where models receive verifiable feedback. He does not believe intelligence has necessarily plateaued, but expects humans to remain competitive longer in areas with diffuse targets and slow or difficult feedback loops.

The host asks about an incident in which agents exploited a vulnerability in an application running on Modal during an attack involving Hugging Face. Bernhardsson stresses that Modal itself was not hacked: an application deployed on Modal had a vulnerability that agents used as a springboard to obtain broader internet access. He compares this to an application on AWS being compromised without meaning that AWS was compromised. Still, Modal is considering more restrictive security defaults, similar to how AWS made accidentally public S3 buckets harder to create. Modal is also using agents defensively. The company puts agents in sandboxes and asks them to try to escape or find vulnerabilities across systems. Bernhardsson believes agents will improve cybersecurity overall, even though they can also be used offensively. He does not think major cybersecurity companies are automatically “cooked,” because their enterprise relationships, sales teams, and ability to take responsibility still matter. He argues that developer experience remains important because good documentation, descriptive errors, and fast feedback loops help both humans and agents. One new challenge is that agents rely on old examples in training data, making deprecations harder; Modal has had to update online examples more carefully. The company has about 24 forward-deployed engineers, up from four earlier in the year, who tune customer models, help migrate workloads, reduce overprovisioning, and turn recurring customer-specific fixes into product features. Modal uses code reviews, but considers them optional when automated testing and deployment controls are strong. Bernhardsson says agents are now good at writing tests, making continuous integration and deployment more important than requiring a human to inspect every change.

Bernhardsson identifies three major priorities. First is the financial and operational complexity of capacity: deciding how to price and package GPU access, how much to rent or own, whether to move toward a neocloud model, and how to make long-term commitments without being trapped by falling hardware prices. He says capacity prepayments are likely to be the best use of additional capital; salaries, offices, and other operating expenses are comparatively a rounding error because the GPUs cost much more than the 200 employees. Second is product strategy: finding the next infrastructure primitives before customers broadly demand them. He sees Modal’s core value as “duration transformation”—turning large, inflexible three-year GPU rectangles into granular, flexible access. Most workloads scale up and down or arrive in bursts, while the market commonly sells capacity as a flat block beginning three to six months in the future. Because Modal aggregates thousands of customers, it can make demand smoother and allocate capacity more flexibly. Third is company building. Modal had crossed 200 employees, expected to reach about 250 by year-end, and might reach roughly 700 the following year, with most hires in systems engineering, GPU performance, networking, Linux, filesystems, CUDA, and inference optimization. Bernhardsson warns that fourfold annual growth causes cultural drift, isolated working styles, and speculative projects. He wants managers and individual contributors to have equal prestige, and says the company is laying down the cultural foundations needed for a 250-person organization. He closes by arguing that Modal’s enduring advantage will combine excellent developer experience with a large, intelligently pooled compute system.