Decoder with Nilay Patel
How AI safety took a backseat to military money
· Summarized
Key takeaways
- AI companies are increasingly moving towards military applications, raising ethical and safety concerns.
- Traditional safety standards are being diluted by AI firms to facilitate rapid deployment of AI technologies in high-risk situations.
- The risks of deploying AI in military settings—especially with chemical and nuclear applications—are profound and poorly understood.
- Financial motivations are driving AI companies towards defense contracts, sometimes at the expense of rigorous safety measures.
Summary
AI in Military Applications
In a candid discussion, Heidi Klaaf and Hayden Field explore the significant shift within AI companies towards military applications and how this impacts AI safety. Historically, companies like OpenAI had stringent bans on military use, but recent developments, such as OpenAI's $200 million defense contract and partnerships with companies like Anthony, signify a troubling pivot in this stance. This transition raises questions about the motivations behind these partnerships and the implications for long-standing safety practices regarding AI.
Safety Concerns and Standards
Heidi emphasizes that the shift in how AI companies define safety is concerning. Traditional definitions prioritize human safety, particularly in defense-critical scenarios. However, current AI frameworks often develop what she calls 'safety revisionism,' where original safety measures are diluted to facilitate quicker deployments of AI technologies into military contexts. This raises environmental and human safety risks, particularly given that many commercial AI systems do not meet the rigorous standards previously upheld in defense.
Potential Risks of AI Systems
Heidi expresses a critical viewpoint on the inherent risks of deploying AI technologies in military settings, especially regarding the development of capabilities for chemical, biological, radiological, and nuclear (CBRN) applications. The conversation delves into how AI systems are ill-equipped to handle these sensitive operations given their current accuracy rates and the potential misuse by bad actors. Heidi warns that relying on AI for decisions within military operations, particularly involving lethal force, is not only untested but also fraught with safety concerns.
AI Industry's Financial Motivation
The conversation also touches on the financial aspects driving AI companies towards defense contracts. With many firms being 'pre-profit' and burning through capital, military contracts provide lucrative opportunities unlike traditional commercial engagements, assuring significant funding for their ongoing operations. This raises ethical questions about prioritizing profit over safety and thorough risk assessments.
Sponsors
- ServiceNow · AI is only as powerful as the platform it's built into. That's why it's no surprise that more than 85% of the Fortune 500 use the service now AI platform. While other platforms duct tape tools together, service now seamlessly unifies people, data, workflows, and AI, connecting every corner of your business. Learn how service now puts AI to work for people at servicenow.com.
- Warp · Most AI coding tools generate sloppy code that doesn't understand your setup. Warp is different. Warp understands your machine, stack, and codebase. It's built for the entire software lifecycle from prompt to production. With the powers of a terminal and the interactivity of an IDE, Warp gives you a tight feedback loop with agents so you can prompt, review, edit, and ship production-ready code.
- Regulator · Regulator is my brand new newsletter covering the Trump administration and the latest battles between big tech and big government. Tech and politics have always been at odds, and now they're locked in an existential fight that affects everything from crypto to AI. Regulator is your essential source covering that delicate balance of power and how it could impact you and your business.
- Al Nilem · Living with disease can be draining. But an innovative class of medicines, pioneered by Al Nilem Pharmaceuticals, targets disease at the source. With RNA interference, we can disrupt the production of unwanted proteins to silence the drip, drip, drip of disease. Learn more about RNAI therapeutics and Al Nilem Science at silencedise.com.
Show notes
This is Hayden Field, senior AI reporter at The Verge — and your Thursday episode guest host. I have another couple of shows for you while Nilay is out on parental leave, and we’re going to be spending more time diving into some of the unforeseen consequences of the generative AI boom.
Today, I’m talking with Heidy Khlaaf, who is chief AI scientist at the AI Now Institute, about the tech industry’s shift toward AI military applications. I wanted to know what’s motivated this shift, and why Heidy thinks leading AI firms are being far too cavalier about deploying generative AI in high-risk scenarios.
Links:
OpenAI is softening its stance on military use | The Verge
OpenAI awarded $200 million US defense contract | The Verge
OpenAI is partnering with defense tech company Anduril | The Verge
Anthropic launches new Claude service for military and intelligence use | The Verge
Anthropic, Palantir, Amazon team up on defense AI | Axios
Google scraps promise not to develop AI weapons | The Verge
Microsoft employees occupy headquarters in protest of Israel contracts | The Verge
Microsoft’s employee protests have reached a boiling point | The Verge
Credits:
Decoder is a production of The Verge and part of the Vox Media Podcast Network.
Our producers are Kate Cox and Nick Statt. Our editor is Ursa Wright.
The Decoder music is by Breakmaster Cylinder.
Learn more about your ad choices. Visit podcastchoices.com/adchoices
Follow this show on AudioBrevity and new episodes get summarized for you. Create a free account