LLM Search, UI/UX challenges, Context Engineering and the 80/20 of Eval

Key takeaways

Context Engineering is more crucial than Prompt Engineering; relevant context enhances AI performance.
Building AI agents requires a focus on understanding user intent and preferences for personalized experiences.
Search functionality must evolve beyond keyword-based to incorporate contextual understanding to accurately capture user needs.
User interface changes should present clear, immediate benefits to encourage adoption and reduce friction.
Regular evaluations (evals) are necessary to ensure AI agents maintain performance, ideally utilizing user data and LLMs for assessments.

Summary

Context Engineering vs. Prompt Engineering

Nishikant Dhanuka emphasizes the importance of 'context engineering' over 'prompt engineering' when developing AI agents. He discusses how having relevant context, such as user intent and real-time data, plays a crucial role in the effectiveness of AI applications. Nishikant explains that while the AI model capabilities are critical, the appropriateness and richness of context determine performance more than the choice between two state-of-the-art models.

Building Effective AI Agents

The podcast dives into two main categories of AI agents being built: productivity tools and e-commerce assistants. Nishikant shares experiences from creating a productivity tool for internal use at Prosus Group called 'Token' and discusses the design of shopping assistants, like the one developed for OLX. He elaborates on how these assistants need to take into account user history and preferences for a seamless experience.

Search Functionality and User Context

The episode highlights the challenges of search within AI applications, specifically e-commerce agents. Nishikant explains that traditional keyword-based search often fails to capture user intent. To improve relevance, he's adopted semantic search techniques and emphasizes the need for a hybrid model that combines keyword and semantic search while also understanding the user's context holistically.

UI Challenges and User Adoption

Nishikant discusses UI problems encountered when launching AI tools. He notes user adoption doesn't keep pace with technological advancements, leading to friction in usage. Insights from user research suggest that AI implementations need to solve real problems and provide instant perceived value. A mix of conversational AI and UI elements might help in improving engagement.

Eval Metrics for AI Performance

Eval metrics are crucial for determining the success of AI agents. Nishikant stresses that waiting until after launch to develop evals is a mistake. Effective evaluation is an iterative process where AI's performance is tested against real user interactions. He shares practical advice on utilizing both human input and LLMs for assessing AI functionality and performance.

Show notes

AI Conversations Powered by Prosus Group

Nishikant Dhanuka talks about what it really takes to make AI agents useful—especially in e-commerce and productivity. From making them smarter with context (like user history and real-time data) to mixing chat and UI for smoother interactions, he breaks down what’s working and what’s not. He also shares why evals matter, how to test with real users, and why AI only succeeds when it actually makes life easier, not more complicated.

Guest speaker: Nishikant Dhanuka - Senior Director of AI at Prosus Group

Host: Demetrios Brinkmann - Founder of MLOps Community

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity](https://x.com/mlopscommunity) or [LinkedIn](https://go.mlops.community/linkedin)]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

#contextengineering #aiengineer #aiinfrastructure #podcast