Real LLM Success Stories: How They Actually Work // Alex Strick van Linschoten // #287

Key takeaways

The consolidation of LLM use cases provides a valuable resource for understanding diverse applications and implementations.
User experience remains a significant challenge in deploying LLMs effectively, where innovative UX designs are needed to maximize their potential.
Reliable evaluation and clear success metrics are crucial for understanding the true ROI of LLM implementations.
Community contributions play a vital role in enriching resources and collective learning within the MLOps space.
Foundational software engineering and MLOps best practices are essential for any successful LLM deployment.

Summary

Creating the LLM Use Case Database

Alex Strick van Linschoten talks about his inspiration from Evidently AI and how he consolidated various real-world use cases of Large Language Models (LLMs) into a comprehensive database. He emphasizes the significance of gathering disparate data and learning from diverse implementations, whether from small teams or large enterprises. Alex describes the importance of using LLMs for summarization and automation, which allowed him to compile the database more efficiently.

Common Use Cases and Patterns

The discussion shifts to the types of LLM use cases Alex encountered. He identifies two primary categories: common implementations like chatbots for customer service and more innovative applications tailored to unique business needs. The episode dives into how companies leverage these technologies in various contexts, including the distinct flavors of chatbots and different use cases that some companies pursue to push the boundaries of LLM functionalities.

Technological Insights and UX Challenges

Throughout the conversation, Alex shares insights into the technological challenges faced by organizations implementing LLMs. He highlights the importance of foundational practices from software engineering and MLOps that support reliable AI deployment. Moreover, they touch on user experience (UX) by discussing how current interfaces may restrict the potential of LLM applications, shrouding innovative UX designs in obscurity and standardizing interactions too much.

Evaluation and Measuring Success with LLMs

An important part of the discussion centers on evaluating the success of LLM implementations. Alex shares his observations regarding the mixed results from using LLMs as judges and the importance of measuring value through defined metrics rather than broad claims. He notes that many organizations prioritize the technical details and challenges over quantifiable business outcomes, which leads to a gap in understanding the true ROI of these initiatives.

The Future and Community Contributions

The episode concludes with a call to action for community input to further enrich the LLM use case database. Alex expresses the intention to keep the database updated and acknowledges the enthusiasm from the community to submit use cases and insights. He reiterates the ongoing exploration of LLM applications and encourages listeners to share their contributions for collective knowledge enhancement.

Show notes

A software engineer based in Delft, Alex Strick van Linschoten, recently built Ekko, an open-source framework for adding real-time infrastructure and in-transit message processing to web applications. With years of experience in Ruby, JavaScript, Go, PostgreSQL, AWS, and Docker, I bring a versatile skill set to the table. I hold a PhD in History, have authored books on Afghanistan, and currently work as an ML Engineer at ZenML.

Real LLM Success Stories: How They Actually Work // MLOps Podcast #287 with Alex Strick van Linschoten, ML Engineer at ZenML.

// Abstract

Alex Strick van Linschoten, a machine learning engineer at ZenML, joins the MLOps Community podcast to discuss his comprehensive database of real-world LLM use cases. Drawing inspiration from Evidently AI, Alex created the database to organize fragmented information on LLM usage, covering everything from common chatbot implementations to innovative applications across sectors. They discuss the technical challenges and successes in deploying LLMs, emphasizing the importance of foundational MLOps practices. The episode concludes with a call for community contributions to further enrich the database and collective knowledge of LLM applications.

// Bio

Alex is a Software Engineer based in the Netherlands, working as a Machine Learning Engineer at ZenML. He was previously awarded a PhD in History (specialism: War Studies) from King's College London and has authored several critically acclaimed books based on his research work in Afghanistan.

// MLOps Swag/Merch

https://shop.mlops.community/

// Related Links

Website: https://mlops.systems

https://www.zenml.io/llmops-database https://www.zenml.io/llmops-database

https://www.zenml.io/blog/llmops-in-production-457-case-studies-of-what-actually-works

https://www.zenml.io/blog/llmops-lessons-learned-navigating-the-wild-west-of-production-llms

https://www.zenml.io/blog/demystifying-llmops-a-practical-database-of-real-world-generative-ai-implementations

https://huggingface.co/datasets/zenml/llmops-database

--------------- ✌️Connect With Us ✌️ -------------

Join our Slack community: https://go.mlops.community/slack

Follow us on Twitter: @mlopscommunity