Leaving the Cloud: The Finale

Key takeaways

Moving away from cloud infrastructure can be accomplished quickly and efficiently with meticulous planning and the right technology stack.
A structured approach like a criticality ladder helps prioritize applications, reduces risk, and allows for phased migration.
Most enterprises and startups can manage their own hardware if they have the knowledge, resources, and proper facilities, contrary to the myth that only hyperscalers do this.
Cloud providers employ manual labor and open source software behind the scenes, and managing your own infrastructure can be more cost-effective and transparent.
Building or renting data centers is feasible; most large-scale companies rent space rather than build their own facilities due to cost and complexity.
Running your own infrastructure requires expertise but offers control, cost savings, and less surprise costs—a compelling choice for large, resource-equipped teams.

Summary

The Rapid Transition Out of the Cloud

The podcast opens with the excitement and surprise of the 37signals team about how quickly they completed their move away from cloud hosting—just six months, far less than the anticipated years. Co-founder David Heinemeyer-Hansson discusses the planning process, technological hurdles, and the strategic decision to build their infrastructure from the ground up using open source tools like Docker, Merck, and KBM. They successfully migrated seven major applications, including their flagship email service, while maintaining zero downtime, which was a significant achievement and a product-like launch for their internal processes.

Strategic Approach and Methodology

A key theme is their meticulous, methodical approach, particularly the use of a criticality ladder approach to prioritize applications based on importance and complexity. This method allowed them to test and execute migrations gradually—starting with less critical applications and moving towards more complex services like Basecamp Classic and their large file storage system. They also leveraged their geographical proximity to cloud data centers to perform piecemeal moves efficiently, minimizing risk and allowing for rollback if needed.

Technology and Infrastructure Insights

The team built and relied on robust infrastructure tools, creating custom solutions like Merck for zero-downtime deployments. They emphasized that their own hardware could be as reliable as cloud services, provided the proper management and expertise are in place. The discussion delves into the realities of data centers, highlighting that most large-scale operations rent space in existing facilities rather than building their own, which is prohibitively expensive and complex. The team also shared their experience of negotiating cloud costs, leveraging influence and long-term contracts to optimize expenses, but ultimately found that the financial logic of running their own infrastructure was more aligned with their needs.

Debunking Cloud Mystique and Addressing Common Questions

A significant part of the episode challenges the misconception that cloud providers are extraordinarily sophisticated and that managing infrastructure is beyond the reach of smaller teams. The hosts assert that large cloud providers employ many manual operations staff and rely heavily on open source software, just like any infrastructure team. They point out that the cloud marketing often oversells the complexity and that managing your own hardware is entirely feasible and often more cost-effective for large operations. The discussion also covers practical questions about backups, physical resets, and data management, illustrating that a well-planned on-prem setup can be resilient and manageable.

Future Steps and Lessons Learned

The podcast concludes with reflections on the remaining tasks after the migration, such as moving their search cluster and large logging systems, and planning for the eventual re-migration of their stored data (8 petabytes in S3). They emphasize the importance of taking time to consolidate and optimize their setup post-move and highlight the lessons learned—namely, that owning and managing their own infrastructure is a viable option with the right expertise and resources. The episode aims to inspire listeners to reconsider assumptions about cloud dependency and to explore their own infrastructure strategies.

Show notes

You’ve no doubt heard the 37signals team talking about leaving the cloud. Well, now the transition is complete!

In this episode of Rework, 37signals co-founder David Heinemeier Hansson and Director of Operations Eron Nicholson sit down with host Kimberly Rhodes to discuss the unexpected speed of the move, the decisions, the hurdles they faced, and the behind the scenes of the meticulous process of moving their major applications, including those that had never before been run outside the cloud.

Tune in as they share the secrets behind their successful approach and the unexpected trick that allowed them to tackle individual components without risking major disruptions. Plus, they address questions from listeners, covering topics such as backups, physical resets, and the future of their infrastructure.

Listen in for an eye-opening conversation that challenges the prevailing narratives of the cloud era and delves into the untapped potential of running your own infrastructure.

Show Notes:

[00:00] - Kimberly sits down with 37signals co-founder and CTO David Heinemeier Hansson, and director of operations, Eron Nicholson to discuss 37signals move away from the cloud.

[00:39] - David shares their surprise at the quick completion of the move and the challenges they faced in planning and technology decisions.

[01:26] - Out in six months: how the team, led by Eron, tackled the various aspects such as logging, failover, and managing two data centers while resolving numerous open questions along the way.

[03:06] - An opportunity to question existing principles and processes, resulting in a novel approach. It felt like a product launch!

[05:04] - Eron reflects on the accelerated timeline and how the entire ops and SIP team worked towards the same goal.

[06:32] - How the criticality ladder approach allowed for smoother progress to more complex applications like Basecamp Classic.

[08:47] - Eron explains the logistics and the unexpected trick that helped the team tackle individual components without risking significant disruptions.

[10:52] - Moving HEY, 37signals most critical and complex app, that had never before been run outside the cloud.

[11:23] - Kimberly opens the floor to a few user questions from Twitter, the first one from Pedro: "Did your company buy or already own your own data centers, or are you renting space in existing data centers? Is that even an option?"

[11:50] - Running your own data center requires a gargantuan scale and enormous investment, but renting space in data centers makes it easy and capital-efficient.

[14:29] - Amnesia of the pre-cloud era: using data centers is more accessible than most people think.

[15:38] - Eron shares how to make the data centers work for you and your company.

[16:39] - Kimberly shares a question from Moshi on Twitter: "Congrats on the move. Before deciding to move, did you try negotiating with any of the large clouds?"

[16:54] - David shares their unique advantage in cloud pricing negotiations.

[19:46] - Why the cloud math doesn't work: the fundamental misalignment (and huge surprise costs) that led 37signals to leave the cloud and how running your own infrastructure makes those financial surprises disappear.

[21:11] - Kimberly shares a telling comment made by someone on David’s Twitter account.

[22:26] - How the cloud marketing campaign has successfully convinced people they're dumber than they really are.

[24:49] - "If you possess the know-how and resources, it absolutely makes sense to manage your own infrastructure." Eron shares the advantage that made the transition less daunting.

[25:32] - Kim