Microsoft Moves Beyond keyboard and Mouse :Looks into Gesture-Based Computer Interfaces


Microsoft Gesture research , Jamie Shotton and team at the Microsoft Centre, 21 Station Road, Cambridge. CB1 2FB  13 June 2016

Microsoft researchers are looking at a number of ways in which technology can start to recognise detailed hand motion — and engineers can put those breakthroughs to use in a wide variety of fields.

With the ultimate goal to allow us interact with technology in more natural ways than ever before; Microsoft has sketched some projects that deal with hand tracking, haptic feedback and gesture input.

“How do we interact with things in the real world? Well, we pick them up, we touch them with our fingers, we manipulate them,” said Shotton, a Principal Researcher in computer vision at Microsoft’s Cambridge, UK, research lab. “We should be able to do exactly the same thing with virtual objects. We should be able to reach out and touch them.”

This kind of technology is still evolving. But the computer scientists and engineers who are working on these projects say they believe they are on the cusp of making hand and gesture recognition tools practical enough for mainstream use, much like many people now use speech recognition to dictate texts or computer vision to recognise faces in photos.

That is a key step in Microsoft’s broader goal to provide more personal computing experiences by creating technology that can adapt to how people move, speak and see, rather than asking people to adapt to how computers work.

“If we can make vision work reliably, speech work reliably and gesture work reliably, then people designing things like TVs, coffee machines or any of the Internet of Things gadgets will have a range of interaction possibilities,” said Andrew Fitzgibbon, a Principal Researcher with the computer vision group at the UK lab.

That will be especially important as computing becomes more ubiquitous and increasingly anticipates our needs, as opposed to responding to our commands. To make these kinds of ambient computing systems truly work well, experts say, they must be able to combine all our senses, allowing us to easily communicate with gadgets using speech, vision and body language together – just like we do when communicating with each other.