First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason Droege · Lenny's Podcast: Product | Career | Growth summary

Key takeaways

The future of AI lies in models that can perform tasks rather than just possess knowledge.
Human experts remain vital in improving AI models, especially as their tasks grow in complexity.
Relationships and collaboration are key to navigating challenges in data ethics and labeling.
Establishing what 'good' looks like in AI outputs through expert evals is critical for development.
Learning from past entrepreneurial experiences can guide better decision-making in new ventures.

Summary

The Evolution of AI and Human Involvement

In the podcast, Jason Droege, CEO of Scale AI, discusses the significant transition in AI development, emphasizing the shift from models that merely 'know' to models that 'do.' He highlights the necessity of human experts in enhancing AI learning, especially amidst the evolving landscape where AI models interact with complex datasets and nuanced topics. Jason explains how Scale AI pioneers expert data labeling and the changing dynamics in training data needs, underscoring the pivotal role human judgment still plays in refining AI capabilities.

Insights from Scale AI's Partnership with Meta

Jason elaborates on the recent $14 billion investment from Meta into Scale AI, clarifying that Scale remains an independent company. He describes the implications of this investment, including the ongoing collaboration with Meta while retaining autonomy, which enables Scale to continue innovating and expanding without preferential ties that might hinder partnerships with other entities.

Navigating Challenges in Data Labeling

The conversation pivots to the challenges of finding qualified experts for data labeling at Scale AI. Jason explains the proactive measures taken, such as referral programs and campus outreach, to build a robust network of subject matter experts. The discussion outlines how essential this network is for bridging the knowledge gaps that AI currently faces and ensuring quality outputs in model training.

The Role of Expert Evals in AI

Jason highlights the growing importance of expert evals in establishing benchmarks for AI performance. He discusses how these evaluations inform what constitutes 'good' AI outputs, allowing models to evolve from basic functions to more sophisticated, accurate decision-making tools. The conversation points out how expert insights not only guide AI training but also enhance its practical applicability across various industries.

Business Lessons from Building Uber Eats

Drawing from his experience at Uber, Jason shares insights into creating Uber Eats. He underscores the value of understanding restaurant economics and the challenge of convincing stakeholders of a new model's viability. Key lessons include the importance of addressing urgent daily problems for customers and the necessity of independent thinking when evaluating new business opportunities.

Establishing High Standards in New Ventures

As he reflects on building new businesses, Jason emphasizes the importance of setting high standards. He explains how having an understanding of business models, market dynamics, and the capacity to pivot is crucial for success in a competitive landscape. This section also touches on the balance between expertise and potential growth when building teams.

Show notes

Jason Droege is the CEO of Scale AI, a company that provides foundational training data to every major AI lab. He previously co-founded Scour with Travis Kalanick and built Uber Eats from idea to $20 billion in revenue. In this conversation, Jason shares lessons from getting sued for $250 billion, discovering restaurant economics by weighing sandwich ingredients, and over 25 years of launching transformative technology businesses.

What you’ll learn:

What actually happened with Meta’s $14 billion investment in Scale AI

Why AI models still need human experts to improve, and how that relationship is evolving

How AI models learn from experts building websites and debugging code

The business lessons from building Uber Eats from zero to $20 billion

Why most enterprise data is useless for AI models today

Why urgent daily problems beat super-valuable occasional problems when building products

How to think independently when building new products and businesses

—

Brought to you by:

Merge—The fastest way to ship 220+ integrations: http://merge.dev/lenny

Figma Make—A prompt-to-code tool for making ideas real: https://www.figma.com/lenny/

Mercury—The art of simplified finances: https://mercury.com/

—

Transcript: https://www.lennysnewsletter.com/p/first-interview-with-scale-ais-ceo-jason-droege

—

My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/174979621/my-biggest-takeaways-from-this-conversation

—

Where to find Jason Droege:

• X: https://x.com/jdroege

• LinkedIn: https://www.linkedin.com/in/jasondroege/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Jason Droege

(06:01) Jason’s early career and lessons learned

(10:27) The current state of Scale AI

(12:37) The shift to expert data labeling

(17:02) Challenges and strategies in finding experts

(18:48) Reinforcement learning and AI environments

(28:18) The future of AI and human involvement

(31:21) The role of evals

(35:25) What AI models will look like in the next few years

(41:43) Building Uber Eats and understanding customer needs

(48:19) The importance of independent thinking

(50:45) Setting high standards for new businesses

(53:03) Exploring and selecting business ideas

(57:07) The McDonald’s story

(01:00:13) The role of gross margins in business feasibility

(01:04:49) Why Jason says, “Not losing is a precursor to winning”

(01:09:12) Hiring and building teams

(01:12:11) AI corner

(01:14:47) Lightning round and final thoughts

—

Referenced:

• Travis Kalanick on X: https://x.com/travisk

• Scour: https://en.wikipedia.org/wiki/Scour_Inc.

• Scale: https://scale.com/

• Alexandr Wang on X: https://x.com/alexandr_wang

• Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor): https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody

• Brendan Foody’s post on X about knowledge work changing: https://x.com/BrendanFoody/status/1970163503702188048

• MIT Finds 95% of GenAI Pilots Fail Because Companies Avoid Friction: https://www.forbes.com/sites/jasonsnyder/2025/08/26/mit-finds-95-of-genai-pilots-fail-because-companies-avoid-friction/

• Uber Eats: https://www.ubereats.com/

• Stephen Chau on X: https://x.com/thestephenchau

• a16z Podcast: https://a16z.com/podcasts/a16z-podcast/

• F1: The Movie: https://www.imdb.com/title/tt16311594/

• V03: https://v03ai.com/

• Careers at Scale: https://scale.com/careers

—

Recommended books:

• The Selfish Gene: https://www.amazon.com/Selfish-Gene-Anniversary-Introduction/dp/0199291152

• The Road Less Traveled: A New Psychology of Love, Traditional Values, and Spiritual Growth: https://www.amazon.com/Road-Less-Traveled-Timeless-Traditional/dp/0743243153/

• Good to Great: Why Some Companies Make the Leap . . . And Others Don’t: https://www.amazon.com/Good-Great-Some-Companies-Others/dp/0066620996

• Thinking, Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

—

Lenny may be an investor in the companies discussed.

To hear more, visit www.lennysnewsletter.com

First interview with Scale AI’s CEO: $14B Meta deal, what’s working in enterprise AI, and what frontier labs are building next | Jason Droege