Kubernetes, AI Gateways, and the Future of MLOps // Alexa Griffith // #294

Kubernetes, AI Gateways, and the Future of MLOps // MLOps Podcast #294 with Alexa Griffith, Senior Software Engineer at Bloomberg.

Join the Community: https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

// Abstract

Alexa shares her journey into software engineering, from early struggles with Airflow and Kubernetes to leading open-source projects like the Envoy AI Gateway. She and Demetrios discuss AI model deployment, tooling differences across tech roles, and the importance of abstraction. They highlight aligning technical work with business goals and improving cross-team communication, offering key insights into MLOps and AI infrastructure.

// Bio

Alexa Griffith is a Senior Software Engineer at Bloomberg, where she builds scalable inference platforms for machine learning workflows and contributes to open-source projects like KServe. She began her career at Bluecore, working in data science infrastructure, and holds an honors degree in Chemistry from the University of Tennessee, Knoxville. She shares her insights through her podcast, Alexa’s Input (AI), technical blogs, and active engagement with the tech community at conferences and meetups.

// Related Links

Website: https://alexagriffith.com/

Kubecon Keynote about Envoy AI Gateway https://www.youtube.com/watch?v=do1viOk8nok

~~~~~~~~ ✌️Connect With Us ✌️ ~~~~~~~

Catch all episodes, blogs, newsletters, and more: https://go.mlops.community/TYExplore

Join our Slack community [https://go.mlops.community/slack]

Follow us on X/Twitter [@mlopscommunity][https://x.com/mlopscommunity] or LinkedIn [https://go.mlops.community/linkedin]

Sign up for the next meetup: [https://go.mlops.community/register]

MLOps Swag/Merch: [https://shop.mlops.community/]

Connect with Demetrios on LinkedIn: /dpbrinkm

Connect with Alexa on LinkedIn: /alexa-griffith

Timestamps:

[00:00] Alexa's preferred coffee

[00:13] Takeaways

[02:39] Airflow Kubernetes Pain Points

[07:35] Pipelining Tools Landscape

[12:11] KubeCon AI Gateway Keynote

[14:21] Envoy vs Envoy AI Gateway

[20:53] AI Model Routing Flexibility

[25:15] Celery and Airflow Integration

[26:25] Open Source Contribution Tips

[32:03] ML Platform Adoption Tips

[37:20] Simplicity Over Complexity

[40:42] Engaging Non-Technical Stakeholders

[44:19] ML vs LLM Platform Divergence

[48:27] LLM vs ML Models

[49:47] Wrap up