Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann · Lenny's Podcast: Product | Career | Growth summary

Key takeaways

The departure from OpenAI by Mann and his team underscores a strong commitment to prioritizing AI safety and ethics.
There is a heated competition for AI talent, exemplified by substantial financial offers, but meaningful work remains a key factor for many researchers.
AI advancements are accelerating, and understanding this exponential growth is crucial as misconceptions about plateauing persist.
The economic implications of AGI will be profound, with potential increases in unemployment rates and significant shifts in job markets.
Implementing AI safety practices and frameworks like Constitutional AI is essential for ensuring the ethical deployment of AI technologies.
Fostering curiosity and creativity in children is crucial to prepare them for an AI-influenced future.

Summary

Leaving OpenAI to Focus on AI Safety

Benjamin Mann, co-founder of Anthropic, shares his journey from OpenAI to starting Anthropic with the entire safety team, motivated by a desire to prioritize AI safety and alignment. He reflects on the challenges faced during this transition and how the team aims to address the existential risks associated with AI development.

The $100M Talent War and AI Progress

Mann discusses the competitive landscape of AI talent acquisition, highlighting Meta's offers of $100 million for top researchers. He contrasts this talent war with Anthropic's mission-driven approach, where employees prioritize meaningful work over financial incentives, emphasizing the intrinsic value of contributing to the future of humanity.

Misconceptions About AI Progress

Mann addresses common misconceptions surrounding AI progress, asserting that advancements are accelerating rather than plateauing. He explains the importance of understanding scaling laws and the exponential nature of AI development, encouraging listeners to recognize the signs of rapid advancements in the field.

Defining AGI and its Implications

Mann introduces his concept of the 'economic Turing test' for assessing AGI, focusing on the transformative potential of AI across various sectors. He predicts significant changes in employment, projecting a possible 20% unemployment rate due to AI advancements and urges society to prepare for these shifts.

AI Safety Measures and Constitutional AI

The conversation pivots to AI safety, with Mann outlining the practices Anthropic implements to ensure their AI systems are both capable and safe. He introduces 'Constitutional AI' as a framework for aligning AI models with human values, emphasizing the importance of embedding ethical considerations into AI systems.

Personal Reflections on AI and Parenting

Towards the end of the podcast, Mann shares insights on how his work in AI influences his parenting approach. He emphasizes the significance of curiosity and creativity in children's education and contemplates the broader implications of AI on society, urging the next generation to adapt and thrive in a rapidly changing world.

Show notes

Benjamin Mann is a co-founder of Anthropic, an AI startup dedicated to building aligned, safety-first AI systems. Prior to Anthropic, Ben was one of the architects of GPT-3 at OpenAI. He left OpenAI driven by the mission to ensure that AI benefits humanity. In this episode, Ben opens up about the accelerating progress in AI and the urgent need to steer it responsibly.

In this conversation, we discuss:

1. The inside story of leaving OpenAI with the entire safety team to start Anthropic

2. How Meta’s $100M offers reveal the true market price of top AI talent

3. Why AI progress is still accelerating (not plateauing), and how most people misjudge the exponential

4. Ben’s “economic Turing test” for knowing when we’ve achieved AGI—and why it’s likely coming by 2027-2028

5. Why he believes 20% unemployment is inevitable

6. The AI nightmare scenarios that concern him most—and how he believes we can still avoid them

7. How focusing on AI safety created Claude’s beloved personality

8. What three skills he’s teaching his kids instead of traditional academics

—

Brought to you by:

Sauce—Turn customer pain into product revenue: https://sauce.app/lenny

LucidLink—Real-time cloud storage for teams: https://www.lucidlink.com/lenny

Fin—The #1 AI agent for customer service: https://fin.ai/lenny

—

Transcript: https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann

—

My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/168107911/my-biggest-takeaways-from-this-conversation

—

Where to find Ben Mann:

• X: https://x.com/8enmann

• LinkedIn: https://www.linkedin.com/in/benjamin-mann/

• Website: https://benjmann.net/

—

Where to find Lenny:

• Newsletter: https://www.lennysnewsletter.com

• X: https://twitter.com/lennysan

• LinkedIn: https://www.linkedin.com/in/lennyrachitsky/

—

In this episode, we cover:

(00:00) Introduction to Benjamin

(04:43) The AI talent war

(06:28) AI progress and scaling laws

(10:50) Defining AGI and the economic Turing test

(12:26) The impact of AI on jobs

(17:45) Preparing for an AI future

(24:05) Founding Anthropic

(27:06) Balancing AI safety and progress

(29:10) Constitutional AI and model alignment

(34:21) The importance of AI safety

(43:40) The risks of autonomous agents

(45:40) Forecasting superintelligence

(48:36) How hard is it to align AI?

(53:19) Reinforcement learning from AI feedback (RLAIF)

(57:03) AI's biggest bottlenecks

(01:00:11) Personal reflections on responsibilities

(01:02:36) Anthropic’s growth and innovations

(01:07:48) Lightning round and final thoughts

—

Referenced:

• Dario Amodei on LinkedIn: https://www.linkedin.com/in/dario-amodei-3934934/

• Anthropic CEO: AI Could Wipe Out 50% of Entry-Level White Collar Jobs: https://www.marketingaiinstitute.com/blog/dario-amodei-ai-entry-level-jobs

• Alexa+: https://www.amazon.com/dp/B0DCCNHWV5

• Azure: https://azure.microsoft.com/

• Sam Altman on X: https://x.com/sama

• Opus 3: https://www.anthropic.com/news/claude-3-family

• Claude’s Constitution: https://www.anthropic.com/news/claudes-constitution

• Greg Brockman on X: https://x.com/gdb

• Anthropic’s Responsible Scaling Policy: https://www.anthropic.com/news/anthropics-responsible-scaling-policy

• Agentic Misalignment: How LLMs could be insider threats: https://www.anthropic.com/research/agentic-misalignment

• Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next

• AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt): https://www.lennysnewsletter.com/p/ai-prompt-engineering-in-2025-sander-schulhoff

• Unitree: https://www.unitree.com/

• Arthur C. Clarke: https://en.wikipedia.org/wiki/Arthur_C._Clarke

• How Reinforcement Learning from AI Feedback Works: https://www.assemblyai.com/blog/how-reinforcement-learning-from-ai-feedback-works

• RLHF: https://en.wikipedia.org/wiki/Reinforcement_learning_from_human_feedback

• Jared Kaplan on LinkedIn: https://www.linkedin.com/in/jared-kaplan-645843213/

• Moore’s law: https://en.wikipedia.org/wiki/Moore%27s_law

• Machine Intelligence Research Institute: https://intelligence.org/

• Raph Lee on LinkedIn: https://www.linkedin.com/in/raphaeltlee/

• “The Last Question”: https://en.wikipedia.org/wiki/The_Last_Question

• Beth Barnes on LinkedIn: https://www.linkedin.com/in/elizabethmbarnes/

• “The Last Question”: https://en.wikipedia.org/wiki/The_Last_Question

• Good Strategy, Bad Strategy | Richard Rumelt: https://www.lennysnewsletter.com/p/good-strategy-bad-strategy-richard

• Pantheon on Netflix: https://www.netflix.com/title/81937398

• Ted Lasso on AppleTV+: https://tv.apple.com/us/show/ted-lasso/umc.cmc.vtoh0mn0xn7t3c643xqonfzy

• Kurzgesagt—In a Nutshell: https://www.youtube.com/channel/UCsXVk37bltHxD1rDPwtNM8Q

• 5 tips to poop like a champion: https://8enmann.medium.com/5-tips-to-poop-like-a-champion-3292481a9651

—

Recommended books:

• Superintelligence: Paths, Dangers, Strategies: https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834

• The Hacker and the State: Cyber Attacks and the New Normal of Geopolitics: https://www.amazon.com/Hacker-State-Attacks-Normal-Geopolitics/dp/0674987551

• Replacing Guilt: Minding Our Way: https://www.amazon.com/Replacing-Guilt-Minding-Our-Way/dp/B086FTSB3Q

• Good Strategy/Bad Strategy: The Difference and Why It Matters: https://www.amazon.com/Good-Strategy-Bad-Difference-Matters/dp/0307886239

• The Alignment Problem: Machine Learning and Human Values: https://www.amazon.com/Alignment-Problem-Machine-Learning-Values/dp/0393635821

—

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.

Lenny may be an investor in the companies discussed.

To hear more, visit www.lennysnewsletter.com

Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann