Robot makers are increasingly handing control to generative AI models, but these systems aren't predictable the way traditional algorithms are. Dr. Ding Zhao, who directs the Safe AI lab at Carnegie Mellon University, co-founded Safeworld with Kyle Wong and Simo Rachidi to tackle this problem. Zhao says the challenge combines probabilistic evals , meaning ways to underwrite the risk of a probabilistic system, with earning trust. He says both are needed to deploy a robot.
Safeworld came out of stealth with a seed round of more than $12 million, led by Shine Capital and a16z Speedrun. Box Group, the Carnegie Mellon University Endowment, Innovation Endeavors and SV Angel also invested. a16z Speedrun partner Jonathan Lai said now is the time to build an industry safety standard, while robots are still being designed and deployed. He warned that waiting until robots are colliding with children in homes would be far too late.
Safeworld tests robot control systems in simulations filled with realistic human models, much as Tesla and Wayve test their vehicles against surprising road events. Zhao says robots are harder to test because they work in unstructured environments and every facility has its own safety standards. For a scenario like a blind corner in a factory, Safeworld builds a digital copy in a simulator such as MuJoCo or Genesis. It then runs the robot's real software through thousands of encounters with simulated people, including people tripping and falling.
Safeworld's platform resembles tools robot builders use internally. The founders believe, however, that companies will want an outside party to check their safety cases and share them between competitors. Zhao warned that many people underestimate how hard edge cases will be to solve. He said the real concern is robots deployed at scale around people who may never have operated one, not robots in demos.
Gritt Robotics builds the AI brain for robots that help install photovoltaic panels at solar farms, and it is partnering with Safeworld. CTO Vishal Dugar said these systems are very hard to prove safe with formal verification and must instead be tested empirically. His robots work alongside people, so they must respond to workers who are kneeling, crouching, running or falling, and to differences in clothing, size, height and skin color.
It's still early days for both Safeworld and generative AI in robotics. The company hasn't yet decided between offering a platform for outside users and a services-based approach. Zhao is confident the team has chosen the right problem and predicts Safeworld will probably be the first profitable company in the field, because anyone who wants to deploy robots will need to pay for safety validation.