Computer & browser use in Codex (5 real examples)

Key takeaways

AI tools like Codex and GPT models have advanced to the point where they can control browsers and computers to automate complex workflows.
Practical use cases include QA testing, persona impersonation for user insights, managing LinkedIn, automating online shopping, and routine administrative tasks.
Prompt engineering plays a crucial role in optimizing AI performance; under-prompting can sometimes yield better results than overly detailed instructions.
While powerful, these AI tools can be slow and occasionally glitchy, making human oversight still essential in many scenarios.
The future of AI automation is promising, with potential to drastically reduce digital toil and enhance productivity across personal and professional spheres.

Summary

Introduction to Browser and Computer Use in AI

The host introduces the core theme of the episode: leveraging AI to control and automate computer and browser tasks through tools like Codex and GPT models, especially highlighting the capabilities of the ChatGPT desktop app combined with Chrome extensions. The emphasis is on how these tools can act as virtual coworkers with the ability to navigate websites, control the mouse and keyboard, and assist in various workflows, both professional and personal.

Use Cases for Browser and Computer Use

The episode details multiple practical applications of AI-controlled browser and computer use, starting with QA testing of websites and onboarding flows, where AI performs exhaustive testing for responsiveness, error states, and accessibility, providing screenshots and issues in Google Sheets. Additional use cases include persona impersonation to gather insights, automating LinkedIn message management, online shopping automation to curate clothing picks, testing mobile responsiveness, filling out complex forms, managing iPhone mirroring for remote device control, and automating routine tasks like document creation and workflow management.

Benefits and Challenges of AI Automation Tools

The host discusses the advantages, such as thorough testing, time-saving automation, uncovering edge cases, and managing tedious tasks effortlessly. There is also a frank acknowledgment of limitations, such as slow processing times and occasional glitches, which can be valuable feedback for improving AI tools. The importance of prompt engineering—under-prompting for better results—is highlighted, along with the idea that sometimes AI needs human verification, especially in complex or nuanced tasks.

Personal Insights and Future Potential

Throughout the episode, the host shares personal experiences and creative ideas, like using AI for shopping by curating items in online stores or managing out-of-state tech support. The discussion emphasizes the transformative potential of these tools in everyday life, from work workflows to leisure activities. The episode concludes with encouragement for listeners to experiment with AI automation, highlighting its ability to both augment human effort and, at times, require human oversight.

Show notes

Today I’m walking you through one of my absolute favorite AI features right now: browser and computer use via Codex (the ChatGPT desktop app). I use this every single day, personally and professionally, and I wanted to share the specific workflows I’ve built, the moments that surprised me, and the mental model that makes it actually click.

What you’ll learn:

How browser use and computer use work, and why the Codex desktop app plus Chrome extension is the combo I rely on
How I use Codex to QA my onboarding flow, including exhaustive mobile testing I would never do manually
Why under-prompting frontier models gets better results than detailed step-by-step instructions
How my husband EJ Lawless’s persona-impersonation trick surfaces friction points I can’t see as the builder
How I use browser use to get through my LinkedIn inbox without touching it myself
How I had Codex shop Free People’s sale and add 10 medium items to my cart (breastfeeding-friendly and Hawaii-ready)
How computer use can control iPhone mirroring so your Mac can technically operate your phone
Three more computer-use shortcuts: filling annoying forms, creating Google Sheets mid-workflow, and managing router

—

Brought to you by:

Runway—The creative AI platform for images, video and more

Hyperagent—Deploy fleets of agents that handle real work

—

In this episode, we cover:

(00:00) Intro

(01:46) What browser use and computer use actually are

(03:08) Why I use Codex specifically and how the desktop app plus Chrome extension works

(04:15) Use case 1: QA testing my onboarding flow

(10:41) Results: 11 issues, one high-severity blocker, one Google Sheet with screenshots

(12:10) Use case 2: persona testing