Meta’s haptic gloves to offer real-world touch experience in AR, VR

In a bid to build confidence in the Metaverse, Meta (formerly known as Facebook) has been showcasing various devices and projects including augmented reality (AR) glasses that they are working on and that would play an important role in the fruition of the Metaverse.

Another device that would improve the interaction experience of users in AR and VR and as the company claims could be the future of human-computer interaction is the haptic gloves.  

Meta's Reality Labs is building soft and lightweight haptic gloves that would allow the computer to accurately understand and reflect the wearer’s hand movements, in addition to reproducing a range of complex sensations such as pressure, texture and vibration.  

All this would help make a virtual object feel real in the hands of the wearer.  

These gloves can be paired with a VR headset and also used for playing a multiplayer online game in the Metaverse.  

Reality Labs is working to ensure the gloves also work with AR glasses. 

Meta already has a wide portfolio of VR headsets under the Oculus brand and also previewed its mixed reality (MR) glasses codenamed Project Cambria.  

"The value of hands to solving the interaction problem in AR and VR is immense,” Sean Keller, Research Director, Reality Labs said in a blog post.  

According to Keller, to develop the gloves, Reality Labs is learning how people perceive the sensations of touch and how they complete tasks.  

He also said the labs will leverage soft robotics, micro fluids, instrumented tracking systems and will invent entirely new soft materials to build the gloves.  

One of the challenges that Keller and his team faced was creating believable haptic sensations on the hand.  

To do that, hundreds of mechanical actuators (tiny motors) were planted all over the hand. These actuators moved together in a way that makes the wearer feel like they’re touching a virtual object. However, wearing a glove with that many actuators created too much heat. It also made the gloves big and added to the cost.  

Keller's team has decided to replace mechanical actuators with soft, pliable ones that could change shape in response to the wearer’s movements.  The problem is that such material doesn't exist right now and they have to build it.  

Further, to build an accurate picture of the virtual environment using the gloves and to know when and where to deliver the right sensations, the Reality Labs will use hand-tracking technology that would help the computer know precisely where the wearer's hand is in a virtual scene and what they are touching and how they are holding a virtual object.   

For this, they are also working to develop a rendering software that could send the right instructions to the actuators on the gloves at the right time.  

Another challenge that the researchers faced was mimicking real-world experience by combining auditory, visual, and haptic feedback. For instance, haptic gloves can’t stop the wearer's fingers from closing during attempts to grasp a virtual object or stop the hands from passing through a virtual table. For this, the researchers are leveraging perceptual science and multisensory integration, which is the study of how human senses work together to create an understanding of the world.  

“Our brains are really good at taking a little bit of haptic signal, a little bit of visual signal, a little bit of auditory signal, and fusing it all together to really feel the sensation and be convinced that there’s an object that exists in your hand,” Sophie Kim, UX Research Science Manager, Reality Labs added. 

Also read: What's so unique about Meta? Mixing reality with an Alphabet soup