The Right Way to Launch an AI Initiative

Yavur Bozhinov discusses why artificial intelligence projects tend to have a failure rate of around 80%. Unlike traditional IT projects, AI projects are probabilistic and not deterministic, which introduces significant complexity and uncertainty. Challenges include poor project choice, low algorithm accuracy, bias, and lack of trust among users, even when the AI performs well technically. He shares a personal anecdote about a successful AI product that was underused due to trust issues, underscoring the importance of user acceptance.

The episode emphasizes the critical first step of choosing the right AI project or use case. Bozhinov points out that many organizations make mistakes due to misalignment between data science interests and business impact. He advocates for focusing on impact first, then feasibility, including ethical considerations like privacy, fairness, and transparency. Understanding the intended audience—internal employees or external customers—is crucial, along with involving users early to ensure the project addresses real needs and builds trust.

Bozhinov discusses the importance of experimentation to improve AI products, citing Etsy’s example of testing a new interface feature. The key is formulating clear hypotheses before testing, rather than viewing MVPs as one-off solutions. Simplifying experiments—like changing just one parameter instead of overhauling systems—can save time and resources. Continuous learning and refining based on experiments ensure AI initiatives evolve effectively and efficiently.

Evaluation isn’t a one-time step but an ongoing process. Bozhinov highlights that AI impacts extend beyond initial predictions due to complex ecosystems and long-term unintended consequences. Regular monitoring and audits, such as LinkedIn’s example where an algorithm affected users’ career opportunities, are necessary. These audits can reveal hidden effects and help organizations adapt their strategies accordingly, maintaining trust and maximizing value.

Companies face the challenge of moving quickly with AI while maintaining ethical standards and protecting brand reputation. Bozhinov recommends building infrastructure to enable rapid experimentation in a safe and responsible way, such as pilot programs and opt-in testing. He stresses the importance of understanding the hypotheses behind experiments to make informed decisions about discarding or advancing AI initiatives, fostering both agility and responsibility.