A second advantage is how simple the commands are to learn. Eldridge says it took him five minutes to associate sounds with directions and that now, "I don't even really think about the vowel sounds I'm making."
Other attempts have been made to control cursors without speech commands. One such method is by using an eye-tracking device, where a camera focuses upon your eyes and translates eye movement into cursor movement. However, Bilmes notes, "You're eyes are really meant for receiving information, not for specifying information. Oftentimes the mouse cursor can get in the way of what you're looking at, say when you're reading an article on the web."
Jon Malkin, a University of Washington graduate student assisting in the project, has been testing the joystick in other applications. In one demonstration he's using the vocal joystick to control a small robotic arm. Since the arm is more complex than a cursor, additional sounds are needed for functions like turning the robotic arm's wrist or opening and closing its claw. Bilmes notes this is the first time vocal commands were used to control a three-dimensional object. Bilmes and Malkin presented this novel use of vocal commands at the October 2007 Assets Conference on Computers and Accessibility.
Bilmes has allowed several hundred people to try out the joystick. Before actually using the vocal joystick, a new user must spend about two minutes saying the various sounds so that the computer can "learn" the user's voice.
While it's working well, he hopes for improvement, noting "We're not as accurate, nor as fast as an existing mouse now." He says the program challenge is keeping the joystick both fast and accurate. He says, "There's sort of a trade off with accuracy and speed. So, with the vocal joystick you can be as accurate as you want if you're willing to be slower." Someday, he hopes the device will be as accurate as a mouse.
This research was presented at the October 2007 Assets Conference on Computers and Accessibility and was funded by the National Science Foundation.