Rewriting in Rust + Being a Learning Machine with AJ Stuyvenberg

Key takeaways

Rewriting existing code, such as Datadog's Lambda extension in Rust, requires careful consideration of performance and reliability.
Understanding the ecosystems of programming languages deeply, including library code, can significantly enhance a developer's skill set.
AWS services have various pitfalls, including complex free tiers and annoying account setup processes that can hinder user experience.
Going deeper into how technologies work and actively engaging with the community can help developers progress in their careers.

Summary

The Rewrite Journey: Rust Over Go

AJ Stuyvenberg discusses the challenges faced in rewriting Datadog's Lambda extension, transitioning from Go to Rust. The motivation for the rewrite stems from performance issues with the Go runtime, which became significant at scale. Rust was chosen due to its memory management capabilities and performance benefits, particularly important in the Lambda environment where cold starts can significantly affect user experience. AJ reflects on the complexities of the rewrite process, emphasizing the need to balance performance and reliability when handling variable workloads.

Thoughts on the Rust Ecosystem

AJ shares his experiences with the Rust ecosystem, noting its maturity and the vibrant community supporting it. While he acknowledges Go's advantages, he finds Rust's challenge lies in its steep learning curve and the need for a deeper understanding of programming concepts. The discussion highlights the difference in ecosystems and the importance of immersing oneself in library code to understand idiomatic practices.

Navigating AWS: Services & Annoyances

The conversation covers a range of AWS-related topics, including the complexities of the AWS Free Tier and the challenges associated with setting up a new account. AJ touches on the limitations of services that require explicit approvals for higher usage levels, such as SES, and how these can stifle developer experience. The frustration around inter-AZ network costs is raised, highlighting how it can unexpectedly become a significant part of project budgets.

Career Advice for Developers

AJ offers valuable career advice based on his own experiences. He emphasizes the importance of delving deeper into technology—understanding protocols like HTTP, TCP, and the foundational components of programming. He encourages younger developers to actively engage with coding communities and seek out hands-on learning experiences, asserting that this will help them navigate their careers more effectively.

Show notes

Today's guest is AJ Stuyvenberg, a Staff Engineer at Datadog working on their Serverless observability project. He had a great article recently about how they rewrote their AWS Lambda extension in Rust. It's a really interesting look at a big, hard project, from thinking about when it's a good idea to do a rewrite to talking about their focus on performance and reliability above all else and what he thinks about the Rust ecosystem.

Beyond that, AJ is just a learning machine, so I got his thoughts on all kinds of software development topics, from underrated AWS services and our favorite databases to the AWS Free Tier and the annoyances of a new AWS account.

Finally, AJ dishes out some career advice for curious, ambitious developers.