⚡️ Which GPU Clouds Are Actually Good? | ClusterMAX 3.0

Jordan says he joined SemiAnalysis in June after spending 10 years at Hewlett Packard Enterprise, where he designed hardware systems purchased by emerging GPU cloud providers. He had followed SemiAnalysis closely, participated in industry group chats, and saw his own opinions reflected in the original ClusterMAX report. His route into the company was characteristically indirect: he criticized something SemiAnalysis had done, and Dylan hired him to fix it. Jordan explains that ClusterMAX 1 covered only about 24 providers, including several major companies that had not been tested because the team was small. It also focused heavily on Slurm, while he believed Kubernetes needed to be tested natively as well. The market view at the time tracked roughly 120 providers, including 111 emerging “NeoClouds.” ClusterMAX 3 now tracks 323 providers, and providers continue contacting the team with new GPU capacity they want tested. The testing itself has changed dramatically. The first edition relied on manually running benchmark scripts. The team has since built a repository containing industry-standard benchmarks, custom performance tests, reliability and burn-in tests, and automated checks. Coding agents can now launch tests, diagnose problems, and fix some issues autonomously, increasing both the breadth and depth of coverage.

The speakers define a NeoCloud as a company that sells access to compute, but distinguish newer AI-focused providers from legacy clouds such as AWS, Google Cloud, Azure, Oracle, Rackspace, DigitalOcean, Cloudflare, and Akamai. NeoClouds typically deploy GPUs or other accelerators for training, inference, and increasingly reinforcement learning. ClusterMAX evaluates providers across 10 criteria, including performance, security, reliability, ease of use, pricing, and availability. The report was created because companies raising tens or hundreds of millions of dollars repeatedly asked SemiAnalysis which of several providers they should use. For a startup, committing $10–15 million of a $20 million seed round to a cloud is a consequential decision. Rather than answer the same question individually, SemiAnalysis published a coarse ranking and then offered more detailed diligence and consulting for specific requirements. The team emphasizes that ClusterMAX is not a universal ranking of every cloud business. It primarily evaluates managed clusters—managed Slurm, managed Kubernetes, and related operations—not bare-metal data-center construction, raw bare-metal leasing, or inference APIs. Crusoe, for example, may be among the strongest data-center builders while receiving a weaker managed-cluster assessment; Together may have strong inference endpoints without necessarily being strong in managed services.

Dylan says security was already poor in the first ClusterMAX and remains a major problem. Providers had misconfigured network controls, allowing testers to see other customers’ jobs, data, and storage. In one first-generation case, SemiAnalysis escalated the issue to the provider and NVIDIA, ranked the company poorly, and strongly advised customers not to use it. The third-generation testing again uncovered major weaknesses. The team says it was able to see national-intelligence and military-style information belonging to certain countries on one provider’s cluster—far beyond what should have been visible during an ordinary cluster test. The provider eventually fixed the issue, but Dylan argues that customers should not have to rely on an outside ranking company to discover whether their cloud is secure. Jordan says many checks are basic rather than exotic: software and driver versions should be current, known vulnerabilities should be patched, and networking controls such as InfiniBand M- and P-keys should be correctly configured. Some providers were running software two or three years out of date despite publicly documented CVEs. SemiAnalysis also released a free ClusterMAX CLI that checks cluster software versions and points users toward upgrades. The speakers’ conclusion is blunt: the industry’s security baseline is still shockingly low.

The speakers say ClusterMAX’s standards have become progressively stricter, so providers can fall in the rankings even if they have improved somewhat. Google is one of the clearest positive examples: it moved from Bronze to Silver across editions, although the speakers correct themselves that it is currently Silver rather than Gold. Other providers stalled while the evaluation criteria became harder, resulting in downgrades. A major current dividing line is support for NVIDIA’s GB300 NVL72 systems, which are highly sought after by AI labs and customers that have raised roughly $100 million to spend on compute. Deploying them is not a routine GPU refresh. Providers must handle a new network, a 72-GPU scale-up domain, ARM CPUs, Blackwell GPUs, 800-gigabit networking, direct liquid cooling, and corresponding changes to operating-system, driver, and network management. The team rewards providers that deploy large GB300 clusters, make them work reliably, respond to feedback quickly, and fix problems in about an hour rather than two weeks. Providers that avoid the latest systems, deliver broken deployments, have unresolved security problems, or suffer major reliability failures move down. Reliability is described as especially important to top-tier customers. The underlying warning is that refusing each new architecture may leave a provider unprepared when Vera Rubin arrives.

When the host asks whether renting compute has improved, Dylan answers, “Good fucking luck.” He says it is harder than ever to rent GPUs. OpenAI and Anthropic have been taking smaller clusters than they once did, but inference companies have become much more profitable and are competing for the same capacity. Base10, Fireworks, Together, Morph, and others can reportedly achieve gross margins around 60% on open-model inference through optimizations such as vLLM-style serving and other efficiency techniques; that is very different from an earlier period when some providers were operating at roughly 10% margins or losing money. Because almost anyone who secures GPUs can now monetize them immediately, a new lab is competing not only with money-losing frontier-model companies but also with profitable inference businesses. The compression is visible throughout the market: large labs once rented clusters of around 8,000 GPUs, while some now take clusters as small as 1,000 GPUs. Inference providers may be content with four nodes, and even single-GPU spot capacity has become difficult to find. The speakers conclude that the entire supply chain is constrained. A customer seeking to train a new model is competing with companies that can turn the same hardware into revenue immediately, so raising capital no longer guarantees access to compute.

OpenAI and Anthropic are identified as the leading buyers of compute. OpenAI is a major driver of CoreWeave’s business, works with Microsoft and Oracle, and is exploring other hyperscaler and self-built arrangements. Anthropic has its AWS partnership around Rainier, uses Google Cloud TPUs, and has agreements with CoreWeave and other providers. The speakers infer that their rapidly growing revenue must be supported by correspondingly large compute commitments. Google still owns more total compute for now, but much of it is allocated to other products or sold externally. The speakers estimate that OpenAI and Anthropic already have as much or slightly more compute dedicated to research and development than DeepMind, and that by year-end each could have more R&D compute than DeepMind. Google retains a cost advantage because it designs its own TPUs, but its leadership has allowed researchers and capacity to support competitors. They question the strategy of Google funding or enabling researchers who leave and then buy Google Cloud capacity. Anthropic’s early access to Google infrastructure helped it mature, but Anthropic is now renting from more than 10 clouds, building or leasing its own data centers, deploying TPUs with Fluidstack, and developing its own infrastructure. The speakers argue that AI weakens traditional cloud lock-in: models and workloads can be migrated more readily, and GPUs are relatively fungible across providers. Google’s data and organizational advantages may still matter, but the company faces a strategic reset under new leadership after Demis Hassabis stepped away from DeepMind’s leadership role to focus on Isomorphic Labs.

The conversation turns to Poolside, which the speakers describe as an unusually infrastructure-capable AI lab. Poolside began building parts of its own infrastructure because existing NeoClouds were inadequate, gradually moved down the stack, and reportedly operated its own systems before OpenAI and Anthropic were seriously doing so. The company built the second-best American open model, according to the discussion, and developed strong systems for cluster health checks and reliable frontier-model training. The speakers correct the headline numbers around NVIDIA’s Poolside transaction, mentioning figures such as $7 billion, $12 billion, employee compensation, and a separate investment, but agree that the precise structure matters less than the strategic effect. NVIDIA can use the deal to acquire talent for Nemotron, keep experienced infrastructure builders away from competitors, and potentially create another large NeoCloud customer. If Poolside had $7 billion in cash, raised more, and borrowed against it at a 75% loan-to-value ratio, the speakers speculate it could deploy roughly $40 billion toward a gigawatt-scale cluster. They describe the Hugging Face acquisition similarly as defensive. Hugging Face is critical open-source infrastructure, and NVIDIA would not want another chip company to own it and use its ecosystem influence against NVIDIA. The team’s talent and ecosystem position are valuable even if the company continues supporting other chips. The broader prediction is that NVIDIA will keep buying teams of 100–200 exceptional people before rivals can acquire them.

Dylan and the host address criticism that ClusterMAX is a paid or overly broad ranking. Dylan says SemiAnalysis does not accept payment for rankings, and argues that doing so without disclosure would be illegal and would destroy the company’s credibility. He acknowledges personal investments in companies such as Crusoe and Fluidstack, but says those companies are not automatically placed in the top five. The ranking team has editorial independence and may rate a company poorly even when other SemiAnalysis businesses have commercial relationships with it. The main substantive criticism is that ClusterMAX can be misunderstood. It evaluates managed clusters, not a company’s entire business, stock prospects, data-center construction, bare-metal service, or inference endpoints. Some readers want more data and longer explanations, while others find the roughly 30,000-word report too long. SemiAnalysis’s answer is to offer its Cloud TCO Model, a subscription research product with weekly notes, a dashboard, and detailed performance data. Future products will separately evaluate inference endpoints, bare metal, sandboxes, and hosted reinforcement-learning providers. “InferenceX” currently focuses on chip and system performance, while a planned “EndpointX” will assess endpoint cost, reliability, security, monitoring, and operational quality. The final technical topic is heterogeneous computing. Vera Rubin systems will bring more non-NVIDIA components, while AMD, Cerebras, SambaNova, Positron, Edge, MatX, Trainium, Huawei, and other accelerators develop. Huawei’s Ascend generations are improving rapidly, although China still produces far fewer chips than NVIDIA, Google, Amazon, or AMD. The speakers expect heterogeneous silicon to matter especially for reinforcement learning, where different chips can handle forward passes and updates, but warn that numerical correctness, synchronization, networking, and the speed of data movement make multi-chip RL difficult.