Article – From the horse’s mouth
Microsoft is to be doing further work on headphones that follow your gestures
Microsoft Research
Thinking beyond audio: Augmenting headphones for everyday digital interactions – Microsoft Research
My Comments
As part of the “Headset Wars” where headset manufacturers are putting research in to and marketing the best-value Bluetooth active-noise-cancelling headset, there is research in how to improve the headphones’ user interface.
Currently, there are some over-the-ear headphones that use touch-based gestures for basic control routines like starting or stopping multimedia, taking or ending calls and adjusting volume. As well, some active-noise-cancelling headphones recognise you cupping our had over the earpiece as a way to override ANC, amplify the outside world and pause multimedia so you can engage in conversation with someone near you.
Why
One issue that is bringing Microsoft in to the headphone innovation space is the intense competition between headphone manufacturers when it comes to high-quality Bluetooth active-noise-cancelling headphones and earbuds. These manufacturers are seeing their headsets being used at work, at home and while travelling for entertainment and personal communications.
As well, the drive towards videocalls and video conferencing during COVID and the associated work-from-home orders has had us want to see a better way to engage in that Zoom videoconference. The various goals that came about include providing each user with a sense of dignity and privacy during these calls, especially if they need to get up to use the bathroom or answer the door.
Then there is the ability to effectively “talk to” a particular individual in a multiparty video conference when you direct your gaze at that particular person in the default multi-screen view.
It is also being seen as part of augmented and virtual reality such as having an avatar mimic you. This is even a feature that is becoming part of video conferencing and allied use cases where participants see themselves as though they are in a virtual space.
What is being brought out through the research
Here, it is about using the headphones to follow your head movements or recognising when you put your headphones on or took them off. There is also the idea of using mid-air hand gestures as a control mechanism with this being based on LiDAR technology to recognise these gestures.
The use of head gestures and hand gestures recognised by headphones is being seen as beyond call control for a videoconference. It includes allowing avatars that represent a user in a videoconference or video game to mimic the user they represent.
It also allows for head movement to be recognised as input for videogames. This would come in to its own with games that have the first person as their primary point of view like driving games, first-person shooters or role-playing / adventure games. I would see this also come in to its own as another input for augmented and virtual reality.
How could this be delivered
The Microsoft Research augmented headphone technology could be delivered as an head-worn attachment which may please people who use headset styles other than over-the-head headsets. Or it could be a clip-on attachment that clips to the headband of an existing over-the-head headset. Headphone manufacturers could even integrate this functionality in to their headphone products, especially if it is to be a product differentiator.
As well, a Bluetooth or USB connection would be used to transmit the head movements and had gestures to the host computing device. Here, it may be based on USB Human Interface Device specifications that relate to augmented and virtual reality use cases. If this functionality is to be delivered as an accessory for a Bluetooth headset that has a USB-C socket like just about all of the Bluetooth active-noise-cancelling headsets, there could be USB-Bluetooth forwarding. Here, the Bluetooth headset would “catch” all the head movements and gestures from the USB-connected accessory and forward them to the Bluetooth-connected host device using the one link.
But what needs to happen is for the head movements and gestures to be recognised in a standard form like a USB HID Device Class. This can allow for operating system support and avoid the need for headset manufacturers and software developers to reinvent the wheel every time they want head-movement support in their setup.
Conclusion
What Microsoft is doing is to extend recognition of head movements and gestures beyond virtual reality and augmented reality to regular computing and communications tasks using ordinary headsets.
