Dyna-2: General Purpose Physical Agents with Hour-Long Autonomy — Jason Ma, Dyna Robotics

Jason Mah, co-founder and chief scientist of Dyna Robotics, explains that Dyna-2 is a “world action model” trained on more than one million hours of data. A conventional world model predicts how the future state of the world or future video will evolve; Dyna-2 also uses that prediction to produce actions a robot can execute. Mah compares this to human reasoning: people implicitly anticipate how the world will change as a result of their actions, and the model gives robots a similar predictive mechanism. Dyna’s tests suggest this makes robots more dexterous, robust, and capable of generalizing to environments and situations they have not seen before. The main input is a sequence of recent camera frames, supplemented by the robot’s own joint-angle and actuator information. Language instructions can also be provided so the robot can follow a task description. The output is an action trajectory, but the model can also generate a representation of future video or future world states. Mah describes this visual-and-physical prediction as analogous to chain-of-thought: the model reasons in a visual and physics space before producing motor commands. This distinguishes Dyna-2 from systems that only map an image and instruction directly to an action.

Mah says Dyna-2 is built as a diffusion transformer. The transformer supplies the sequence-modeling and data-ingestion capabilities associated with language models, while diffusion is useful for generating high-dimensional, fine-grained outputs. In image generation, diffusion can refine random noise into sharp edges and detailed structure; in robotics, the analogous benefit is smooth, precise motion that can be executed by hardware. The host connects this design to flow matching, which Dyna uses to make diffusion more practical. Mah describes flow matching as a simpler special case of diffusion: rather than heavily tuning a scheduler that converts noise into the final output over perhaps 40 to 100 steps, flow matching can reduce the process to roughly four steps in suitable settings. The host notes that this is particularly useful for robotics because the model can be reevaluated continuously in real time, whereas image generation is often effectively a one-shot operation. Mah agrees that the underlying concept is still diffusion, but flow matching is easier to tune and has become common in recent image and video generation.

Dyna’s strategy is to pretrain a general foundation model and then adapt it to specific tasks with relatively few demonstrations. Mah compares this with human capability: people possess broad physical knowledge but still need practice to become experts at a particular job. Dyna-2 starts with more than one million hours of human data, then can learn tasks such as removing drinks from a refrigerator after a small amount of task-specific training; the hosts saw examples that took about 45 minutes of mapped data. Mah calls data the central problem in robotics because, unlike language models, robotics has no internet-scale repository that can simply be downloaded. Dyna collects data internally, examines existing sources, and buys or partners for additional data. Egocentric human video is valuable because it covers an unusually broad range of physical activity and gives the model a prior for how the world changes around an embodied actor. However, a human body and a robot are not identical, so human video cannot directly provide all the motor commands a robot needs. Robot data is more expensive but teaches low-level reflexes, dexterity, and the hardware-specific adaptation needed to turn the human prior into reliable robot behavior. Mah says Dyna is seeking high-quality egocentric, robot, and other useful physical-world video through data partnerships. He gives the company’s spoken contact address as data partnerships at Dyna.co, without describing a specific vendor as preferred.

The hosts ask whether frontier models can generate simulated environments and data for robotics. Mah says Dyna is actively exploring this and believes synthetic data may become the next major dimension of scaling, alongside egocentric and robot data. He cites his doctoral work, including the Eureka project, which used GPT-4 to generate simulation environments and reward functions for reinforcement learning. Its notable demonstration involved a robot spinning pencils, but the earlier approach was limited because the models had weak visual and three-dimensional understanding and mostly generated code from sparse textual feedback. Mah says current frontier models have much stronger vision capabilities, including real-to-sim conversion, 3D reconstruction, and related tools. That makes synthetic environments much more useful for generating tailored training data. Dyna can also use its own world-action model as part of a data flywheel: because the model predicts video as well as actions, it can generate additional world data for future training. Mah presents this as an early opportunity rather than a solved method; even one million hours is only a beginning. Inference creates a separate constraint. Unlike a chatbot, a robot cannot wait several seconds while holding a knife or performing a dynamic action. Dyna therefore builds its own inference stack and expects deployed models to run locally or on-device with low latency, although the largest training model can later be distilled into a smaller deployment model.

Dyna builds the robot, actuators, model, data pipeline, and software stack rather than treating the robot as a generic platform. Mah says the company buys mature components such as lidar and some cameras when their interfaces are well understood, but intends to own components that materially affect system performance. Actuators are especially important because their quality and mechanical design determine how well the robot can be modeled in simulation and controlled in practice. The team also designed the robot’s proportions to resemble real people, improving transfer from human data. The robot is a semi-humanoid: its upper body has human-like shoulders and arms, plus four degrees of freedom in the waist, while its lower body uses wheels. Mah says wheels are currently more practical than legs because legs consume more energy, are less stable, and make safety harder to guarantee. The host highlights the robot’s ability to lean into a machine during laundry work, rather than merely moving an elevator-like torso up and down. Mah argues that humans constantly use their torso and waist without consciously noticing; removing those degrees of freedom would make physical tasks awkward and weaken transfer from human demonstrations. The discussion also covers manufacturing. Even companies that assemble robots in the United States still source raw materials and components globally. Dyna is evaluating ways to increase U.S. production while retaining a global supply chain, because building every part domestically from zero would be slow and difficult. The speakers identify magnets and other supply-chain elements as potential bottlenecks.

For a new task, Dyna teleoperates the robot to collect demonstrations, fine-tunes the model, deploys it on the real hardware, and observes the result. Some tasks need only tens of minutes of data; others take a few hours. The host cites a chopping example that took 45 minutes, while Mah describes the process as an iterative robotics version of reinforcement learning from human feedback: a rollout can be scored as good, bad, or partially successful, after which the team collects targeted data and retrains. A model that is 70% or 80% successful can often be improved by identifying exactly where it fails. The broader objective is hour-long, unsupervised workflows rather than isolated one-minute demonstrations. Mah says Dyna’s laundry system can already perform a complete workflow lasting about an hour. The key limits for more complex jobs are data and hardware. Tasks such as setting up a filming studio may require a dexterous human-like hand rather than a parallel gripper, but current dexterous hands are not durable enough and can overheat quickly. Because every improvement ultimately has to be evaluated on physical hardware, unreliable hardware also slows data collection and research progress. Mah adds that current robot data mostly captures video and joint or hand motion, while humans also use sound, touch, and smell. Those missing modalities may become essential for tasks that cannot be solved from vision alone. He says the next major alternative to a vision-only world model could be a vision-plus-tactile system that is dramatically more data- and compute-efficient.

Dyna has worked with Best Western in hotel laundry and with Din Tai Fung restaurants. Mah says the hotel use case involves folding towels, while Dyna’s largest scaled customer is Din Tai Fung, where robots fold thousands of napkins across multiple U.S. locations. The napkins must meet a tight quality standard: the fold and seam need to be positioned consistently so chopsticks can fit into the intended pocket. Mah argues that deformable objects such as napkins and towels are difficult even for specialized machines because they have effectively unlimited possible configurations. The hosts contrast a pilot in one location with a customer deploying robots across many sites. Mah says the latter is the more meaningful test because it shows that a customer believes the robot creates enough value to scale. Din Tai Fung initially used the robots for napkins and is now considering additional work, including cutting food and vegetables. The hosts joke that making dumplings would be the “final boss.” The robot’s consistency is a major benefit: restaurants in Los Angeles, New York, and Seattle may fold napkins differently when people do the work, while a shared model can produce the same behavior everywhere and be updated across the fleet. Mah’s “bitter lesson” for robotics is that scaling the right data should eventually produce better generalization, rather than relying on increasingly elaborate hand-designed priors. The challenge is identifying the right sensors and assembling enough physical-world data. Dyna’s approach differs from companies that only provide models: Mah believes controlling the full system is necessary to discover the real bottlenecks and build something customers will actually use.

Mah traces his interest in AI to growing up in Japan and China and playing Go. When AlphaGo appeared during his senior year of high school, he was struck that reinforcement learning had solved a game people had considered exceptionally difficult. At Penn, he pursued reinforcement learning for robotics, studying how to connect reinforcement learning with human data and how to make the theory rigorous. At NVIDIA, he worked with Jim Fan on dexterous tasks such as pen spinning. He realized that designing a simulation and reward function for pen spinning by hand was extremely difficult, which led to Eureka: using a language model to generate code and rewards for reinforcement learning in simulation. At Google DeepMind, Mah explored using Gemini for robotics, but he eventually wanted research to produce systems used by large numbers of people rather than laboratory demonstrations with self-designed evaluations. He and co-founder Lynn started Dyna near the end of 2024. The company now has about 30% to 40% of its staff in hardware, roughly ten people working on models, and additional software, robotics, data, and business teams. Mah describes Dyna as both a research and product company. The team identifies capabilities required for deployment—language following, whole-body control, dexterity, generalization, and rapid adaptation—then engineers create targeted internal tests, including playful ones such as manipulating four highlighters held together with rubber bands. The name Dyna refers both to a Richard Sutton paper about a self-improving agent using a world model and to Ultraman Dyna, a Japanese television character from the year Mah was born. The long-term goal is to put intelligent robots at thousands of customers and eventually operate millions of robots in the physical world.