Episode 116: [Value Boost] What Data Scientists Need to Know Before the AI Free Ride Ends

Key takeaways

Relying solely on cloud AI models for the future is a strategic error due to expected cost increases once AI companies go public.
Organizations should consider a hybrid approach, combining on-premise hardware with cloud models, to mitigate future risks and costs.
Initial investments in affordable hardware for experimentation can scale into significant cost savings and operational independence.
Understanding AI agent orchestration, hardware parameters, and security protocols is essential for data professionals to influence AI infrastructure decisions effectively.

Summary

The Economic Dynamics of AI and Future Costs

This section delves into the current economic landscape of AI technology, highlighting that AI providers like OpenAI are subsidizing the cost of their models, incurring billions in losses annually. The discussion emphasizes that these subsidized prices are temporary; once AI companies go public through IPOs, the cost of using these models is expected to rise significantly. The hosts warn that organizations relying solely on cloud-based frontier models may face unexpected and steep increases in costs, necessitating strategic planning to contain expenses.

Risks of Over-Reliance on Cloud AI Models and the Need for On-Premise Infrastructure

Here, the conversation focuses on the strategic mistake of relying entirely on cloud-based AI models. Nick elaborates on the importance of owning infrastructure, especially considering regulatory requirements, such as HIPAA in healthcare and finance, which could mandate on-premises data handling. The risks include escalating costs when prices increase post-IPO and potential security and compliance issues. The discussion advocates for a hybrid approach—using on-premise hardware and models to reduce dependence on external vendors, control costs, and ensure data security.

Hardware Considerations, Costs, and Investment Strategies for AI Infrastructure

This section covers practical advice on building AI capabilities through hardware investments. Nick discusses options starting from affordable devices like NUCs and small GPU systems for experimentation, to more substantial investments in GPU racks costing upwards of $50,000 to $200,000 for production environments. He stresses evaluating total cost of ownership, including hardware and software, rather than just initial hardware costs. The importance of balancing experimentation, infrastructure investment, and operational needs within a hybrid framework is highlighted, with an emphasis on preparing for future cost escalations.

Guidance for Data Professionals on AI Infrastructure and Skills

The final section offers strategic advice for data professionals aiming to influence AI infrastructure decisions within their organizations. Key recommendations include understanding agent orchestration tools like OpenClai, learning to manage and deploy models, and developing hands-on experience with hardware and AI agent development. Nick emphasizes the importance of self-education through experimentation, suggesting starting small and gradually expanding hardware capabilities. The overarching message is for data professionals to proactively build expertise to become the strategic voice in their organization’s AI infrastructure choices.

Show notes

Right now, AI is cheap. But the companies providing it are losing billions of dollars a year and are expected to go public within the next 12 months. When they do, the price of intelligence is going up - and any organisation that has built its AI future entirely on rented intelligence is going to face a very unpleasant surprise.

In this Value Boost episode, Nicholas Kelly joins Dr Genevieve Hayes to explore why organisations shouldn't be building their AI future entirely on frontier models, and what data professionals need to understand to be the strategic voice in the AI infrastructure conversation.

You'll discover:
1. Why relying entirely on frontier AI models is a strategic error [02:00]
2. How the economics of AI are about to change [03:34]
3. The case for owning at least some of your AI infrastructure [07:53]
4. What data professionals need to know to become the strategic voice in AI infrastructure decisions [11:02]

Guest Bio

Nicholas Kelly is the co-founder and Chief AI Architect of Delivering Data Analytics, a consultancy that helps organisations turn data, BI, analytics, and AI into confident decisions people actually act on. He is also the author of Delivering Data Analytics, How to Interpret Data and the recently released The AI-Driven Data Team.

Links

Nicholas's Website
Connect with Nicholas on LinkedIn
Connect with Genevieve on LinkedIn
Be among the first to hear about the release of each new podcast episode by signing up HERE