Bringing up Cosmo

Key takeaways

The importance of learning from past projects: The experiences from Gimlet were crucial in shaping the design and bring-up strategies for Cosmo.
Effective communication within the engineering team is vital for troubleshooting and successfully overcoming hardware challenges.
Using robust and standardized tools is essential for hardware design and can significantly improve the surface for encountering fewer issues.
Incremental testing of components and systems helps in identifying weaknesses early in the bring-up process.
Understanding and effectively managing the electrical characteristics of components, such as resistors on buses, is crucial for reliable system behavior.

Summary

Overview of Cosmo Bring-Up

In this episode of Oxide, Bryan, Adam, and the Oxide team discuss the intricate and surprisingly smooth bring-up process of their next-generation sled, Cosmo, which incorporates an AMD Turin CPU. The conversation delves into the team's handling of adversity, the necessity of re-work, and the satisfaction of triumph alongside the complexities of product development.

Lessons Learned from Gimlet to Cosmo

The Oxide team reflects on their experiences from Gimlet, their previous generation sled, to inform and improve the bring-up processes of Cosmo. They highlight how past challenges have driven design decisions, including how to better manage component connections and the role of tools used for PCB design, such as Altium and Cadence.

Challenges and Triumphs in Hardware Development

Discussions revolve around the hardware issues faced, including problems with sequencing and power management. The team shares insights about the importance of proper design and communication within the hardware circuits, particularly mentioning the significance of resistors in the I2C bus for DIMM communication. They recount troubleshooting sessions that led to the successful powering up of the sled.

Future Steps and Ongoing Development

As the episode concludes, the Oxide team discusses the next steps after achieving successful Cosmo bring-up. They highlight the need for further testing and validation, such as DIMM margin testing and PCIe link training, and express excitement about the potential of their advances with their new system.

Show notes

Oxide is bringing up its next generation server. To discuss the (amazingly smooth) bringup process, Bryan and Adam were joined by members of the oxide team. Tales of adversity, re-work, un-re-work, and triumph!

In addition to Bryan Cantrill and Adam Leventhal, we were joined by Oxide colleagues Nathanael Huffman, Ian Sobering, Matt Keeter, and Aaron Hartwig.

We mentioned quite a few terms! Here's a helpful guide:

Cosmo - Oxide’s next-generation sled (currently in development) with an AMD Turin CPU
Gimlet - Oxide’s current-generation sled with an AMD Milan CPU
Turin - AMD Epyc 9005 Series
Milan - AMD Epyc 7003 Series
Genoa - AMD Epyc 9004 Series (Oxide chose to skip this generation)
Sequencing - the precise control of when power rails are energized throughout a PCB
Sled - One of the (max 32) computers in an Oxide rack; a custom form-factor optimized for power and cooling efficiency
IBC - Intermediate Bus Converter (Our 54VDC -> 12VDC converter)
RoT - Root of Trust
SP - Service Processor, the small computer (running Hubris) that allows for low-level control
Ignition - An even lower-level control network for power management (including power of the SP)
Ruby - The AMD reference platform (Oxide has used this to prepare Cosmo software in advance of bringup)
DC-SCM - https://www.opencompute.org/documents/ocp-dc-scm-spec-rev-1-0-pdf and OpenCompute standard form factor.
Grapefruit - OCP DC-SCM form-factor board with our SP, RoT, and FPGA on it, used to replace the OCP DC-SCM baseboard management controller in the Ruby platform.
Cadence - Software Oxide previously used for PCB design
Altium - Software Oxide now uses for PCB design
Hubris - Oxide’s embedded operating system, run on the SP and RoT
Humility - The Hubris debugger
PLM - Product Lifecycle Management – a class of software used for managing hardware BOMs
BOM - Bill of Materials – the components required to build a hardware product
RFK - Our colleague, Robert Keith (to distinguish him from our other colleague, Robert, and our former colleague, Keith)
FPGA - Field Programmable Gate Array – Also referred to as “soft logic” – effectively programmable hardware
ILA - Integrated Logic Analyzer
JTAG - A debugging interface for various processors
UART - A serial port or connection

For previous tales from the bringup lab:

If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!