AI Can Write Code. Why Isn’t Software Better?

The episode opens with Diogo Almeida expressing his frustration about the lack of meaningful automation despite AI's advanced capabilities. He highlights how AI can generate code or support programming, yet the actual software produced remains limited and traditional in nature. Almeida emphasizes that current coding agents mainly speed up the existing process without fundamentally expanding what software can do. He advocates for smarter software that incorporates intelligence to interpret intent and make decisions probabilistically, rather than just automate regular tasks.

Almeida introduces TypeSafe and their product Jev, which aims to revolutionize software development by integrating an 'inside-out' intelligence layer. Unlike traditional code generation tools like Codex or Cloud Code, Jev seeks to broaden the vocabulary of programming by allowing natural language intent to be directly translated into decisions that the software can use autonomously. Almeida envisions a primitive that includes probabilistic reasoning, enabling programs to handle uncertainty and context more effectively, thereby creating more valuable and flexible applications.

The conversation shifts to the critical role of reliability in AI-powered software. Almeida stresses that making AI reliable is more important than impressive demos, advocating for models that are consistently trustworthy and behave predictably. He discusses the concept of reliability in terms of availability, consistency, and robustness, emphasizing that trust is essential for AI to become genuinely programmable. Almeida highlights that probabilistic programming and systems that understand and manage uncertainty will be vital in advancing AI's usefulness and safety.

Almeida shares his background as a mathematician turned AI researcher, recounting his early success in math competitions and his shift into computer science. He describes how winning a Kaggle competition with automation inspired him into AI and how working at Google Brain and OpenAI shaped his vision. Almeida expresses a strong belief that AI is fun and powerful, contrasting his optimistic outlook with the more dystopian narratives common in the field. He advocates for building AI systems that improve society and create new jobs, reflecting his broad and positive view of AI's future.

The episode explores Almeida's ideas about how AI will alter software and systems at multiple levels. He argues that current AI tools mostly automate small parts of software without improving overall capabilities, but the potential lies in embedding more intelligence directly into programs. Almeida discusses the significance of reliability and probabilistic reasoning in enabling AI to 'do what I mean' in a dependable way. He envisions a future where software systems are rebuilt around these primitives, enabling entirely new functionalities and making complex tasks automatic, with implications spanning from SaaS applications to critical infrastructure.

Towards the end, Almeida voices pragmatic caution about scaling AI. He acknowledges the limitations of current systems in areas like state consistency and guarantees, emphasizing the need for reliable AI at system levels. He dismisses biologically inspired approaches as impractical and focuses on probabilistic programming and system-level trade-offs. Almeida discusses the importance of robustness, deterministic behavior, and trustworthiness, highlighting that future systems will likely leverage these principles to perform safer, more predictable operations, eventually transforming software infrastructure itself.