VR-Doh Brings Hands-On 3D Modeling to Virtual Reality
Marylee WilliamsThursday, January 15, 2026Print this page.
The Breakdown
- The CMU-created VR-Doh lets beginners model in virtual reality using just their hands.
- Objects respond naturally to pinches, squishes and twists.
- Physics and visuals work independently, so creating feels realistic without lag.
***
Researchers in Carnegie Mellon University's School of Computer Science developed a 3D-modeling system that lets users intuitively shape objects with hand gestures in virtual reality.
Cheekily named for virtual reality Play-Doh, VR-Doh is designed for beginners who are just learning about modeling in virtual environments. It enables more precise manipulation, better visualizations and improved compute usage compared to existing modeling tools.
"Modeling with VR-Doh can be as easy as using your hand to interact with physical objects," said Zhaofeng Luo, a Ph.D. student in the Computer Science Department (CSD). "VR-Doh is about being intuitive. You don't need to understand how a traditional VR modeling system, like Blender, works, which requires users to enter the geometry, curves, shapes and a lot of other information. In VR-Doh, users with no experience can model and learn along the way."
VR-Doh treats objects as if they are physically real in VR space. It uses a physics-based simulation rather than geometry, which traditional systems rely on. Users don't need to know how to precisely manipulate the mesh or vertices of these digital objects. Instead, they can use their hands to model and edit objects in VR as if they were clay.
Researchers developed VR-Doh by customizing the material point method, which uses sets of points to represent materials. In VR-Doh, objects are made up of many tiny points that have set positions in space. When a user pinches, squishes or twists an object with their hands, the system calculates how each particle should move in response, creating lifelike, precise changes. The simulation focuses on the hand or object doing the action. This localization removed the need to simulate a whole scene in real time, focusing instead on just the area being changed.
The user's hands become what Luo called a boundary condition. This means when a user's finger touches the clay, the clay doesn't go into the finger. Instead, the clay responds how it would in the real world.
"The idea is that in VR somebody who's trying to model — or trying to learn how to sculpt clay — their hands are the input and the object is the thing they're colliding with," Luo said.
Along with Luo, the research team includes Zhitong Cui, an SCS visiting researcher and a Ph.D. student at Zhejiang University; Shijian Luo, a professor at Zhejiang University; Mengyu Chu, an assistant professor at Peking University; and Minchen Li, an assistant professor in CSD.
VR-Doh is open-source, and anyone with a virtual reality headset can try the system.
Aaron Aupperlee | 412-268-9068 | aaupperlee@cmu.edu