French Engineers Teach Kinect Sign Language

There are no lightsabers or oversized lady bumps in this Kinect hack, just an earnest attempt at aiding communication.

Students at the École Supérieure d’Informatique Électronique Automatique, or ESIEA, in Paris have started to put together a program which allows Microsoft’s motion controller to understand sign language. The capacity to recognize sign language was noted in the patent for Kinect, but Microsoft has yet to officially implement the feature.

According to the video, the program currently only knows two words: sorry, and hello. However, the hard part, which is getting the system to work in the first place, is apparently complete, and now it’s just a matter of teaching it more words. The hack uses a “neural network,” which means that inputting new words is just a matter of tying the word to the gesture and doesn’t require any modification of the program. Once it has a more full vocabulary, it could serve as a teaching for people to learn sign language, or give deaf people who have sign language as their first language another avenue to talk to people online beyond just typing.

It’s hard not to notice though, that the gestures that the system does recognize are fairly large, and it’s not clear how it will handle smaller, finer gestures. Still, considering we’re talking about a videogame motion controller being hacked for a purpose it was never intended for, I’d say that the results are still quite promising.

Source: Kinect Hacks via Kotaku