Navigation
Mobile gestures
human-computer interaction, data mining, machine learning, July 2007 - May 2008
Nowadays, we interact with mobile phones via 15-20 small buttons, which makes it extremely hard to reach certain functions. To write the word "hello" needs 15 button presses on an average mobile. We also need long keystrokes to find someone in the contact list with hundreds of names. Voice dialling has been available for many years, but it isn't reliable at noisy places, and we just don't feel confortable to talk to our phones when others are listening. More and more mobile phones appear with in-built accelerometers, which may provide a new user interface.
In my application, long keystrokes can be substituted by gestures. The accelerometer sends the signal to a MacBook nearby via Bluetooth connection. The algorithm running on the MacBook recognizes gestures and sends an imitated keystroke to the mobile phone. It works with supported Sony-Ericsson mobiles, and it is very easy to set up.
We can define gestures by recording it a few times and selecting an action. Gestures are recognized by the computer and the action is sent to the mobile. It recognizes gestures with around 97% accuracy, using 10 different gestures.
This project was partly done in Kitchen Budapest.
Publications
Zoltán Prekopcsák, Péter Halácsy, and Csaba Gáspár-Papanek (2008) Design and development of an everyday hand gesture interface in MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. Amsterdam, the Netherlands, September 2008. [PDF]
Zoltán Prekopcsák (2008) Accelerometer Based Real-Time Gesture Recognition in POSTER 2008: Proceedings of the 12th International Student Conference on Electrical Engineering. Prague, Czech Republic, May 2008. [PDF]