AI Can Write the Code. Can It Run the System? | Grepr CEO Jad Naous

The episode begins with a discussion on AI's evolution, starting from simple autocomplete systems and advancing to modeling complex software systems. Martín Casado questions whether the same can be applied to operational systems, which are constrained by data and complexity. The hosts highlight that while AI can now generate significant production code—making engineers' work more efficient—the challenge remains in operational management, especially in handling unpredictable workloads, failures, and system states.

Jad Naous describes how his small team uses AI-driven coding agents to accelerate development processes, reducing project timelines from months to days. They utilize specific models like Opus 5.5 and implement control over agents through command line interactions, managing multiple sessions to enhance productivity. The conversation emphasizes that experienced engineers leverage AI effectively by understanding model nuances, prompts, and workflows, which has led to dramatic productivity gains in their development cycles.

The discussion shifts to the difficulties of applying AI to operational systems, such as distributed clusters and complex production environments. The hosts debate whether AI, given its current capabilities, can learn to troubleshoot by detecting changes, forming hypotheses, and running experiments. They acknowledge that because production systems are inherently unpredictable, involve rare failure states, and require understanding nuanced environmental factors, AI’s ability to operate autonomously is limited but potentially improvable by grounding models in operational data.

Martín and Jad explore whether AI can handle the astronomical complexity of operational system states, which grow exponentially as systems scale. Jad points out that current models struggle with such heavy-tailed problems, especially in distributed algorithms, where the space of possible states and failure modes is vast. They argue that historically, systems managing distributed clusters rely heavily on human oversight due to the unpredictability and rarity of certain failures, and training AI for such tasks remains a significant challenge.

The conversation broadens to the idea that AI's strength lies in its ability to operate within constrained, well-understood domains where data distribution matches the training set. For real-world, unconstrained systems, progress may depend on the ability of AI to plan and simulate actions to gather information, rather than direct prediction. They touch on how AI might influence world-changing systems by modeling or even altering environments to fit computational capacities, highlighting both the promise and limitations of current technology.

Martín and Jad reflect on how historically, system design has favored overprovisioning and simplicity rather than relying solely on predictive models. They discuss how AI can complement human expertise, especially by assisting in debugging, hypothesis generation, and managing complex workflows. The episode concludes with an optimistic note that, despite the enormity of the challenges, incremental advances in AI and better grounding in operational data are paving the way toward more autonomous, resilient systems.