Summary:MIT’s ‘speech-to-reality’ system uses AI and a robotic arm to turn spoken requests into physical objects—like furniture—in under five minutes.
Generative AI and robotics are inching closer to a future where verbal requests for objects can be fulfilled in minutes. MIT researchers have developed a "speech-to-reality" system—an AI-driven workflow that enables robotic arms to "speak objects into existence," fabricating items like furniture in as little as five minutes.
The system operates via a table-mounted robotic arm that processes spoken commands (e.g., "I want a simple stool") and constructs objects using modular components. To date, it has successfully created stools, shelves, chairs, small tables, and even decorative pieces like a dog statue. "We’re connecting natural language processing, 3D generative AI, and robotic assembly," notes Alexander Htet Kyaw, MIT graduate student and Morningside Academy for Design fellow. "These are rapidly advancing areas of research that haven’t been brought together before in a way that you can actually make physical objects just from a simple speech prompt."
Speech to Reality: On-Demand Production using 3D Generative AI, and Discrete Robotic Assembly
Kyaw conceived the system while taking Professor Neil Gershenfeld’s course "How to Make Almost Anything," later advancing it at MIT’s Center for Bits and Atoms (CBA) with collaborators Se Hwan Jeon (Mechanical Engineering) and Miana Smith (CBA). The workflow begins with a large language model processing speech input, followed by 3D generative AI creating a digital mesh of the object. A voxelization algorithm breaks this mesh into assembly components, which are then adjusted via geometric processing to account for real-world constraints like component quantity and structural connectivity. Finally, the system generates an assembly sequence and robotic path plan.
Compared to 3D printing (which takes hours or days), the system offers rapid fabrication and lowers barriers for non-experts by eliminating the need for 3D modeling or robotics programming skills. "This project is an interface between humans, AI, and robots to co-create the world around us," Kyaw says. "Imagine a scenario where you say ‘I want a chair,’ and within five minutes a physical chair materializes in front of you."
Future improvements include enhancing furniture weight-bearing capacity by replacing magnetic connections with more robust alternatives. Smith adds, "We’ve also developed pipelines for converting voxel structures into feasible assembly sequences for small, distributed mobile robots, which could help translate this work to structures at any size scale." Modular components enable sustainability via disassembly and reassembly into new objects. Kyaw, drawing inspiration from "Star Trek" replicators and "Big Hero 6" robots, aims for "a future where the very essence of matter is truly in your control. One where reality can be generated on demand."
The team presented their research, titled "Speech to Reality: On-Demand Production using Natural Language, 3D Generative AI, and Discrete Robotic Assembly," at ACM’s Symposium on Computational Fabrication (SCF ’25) at MIT on Nov. 21.
This material is obtained from online sources.
