The ultimate mobile user interface: brain implants!?

The ultimate mobile device might be one implanted in your head, according to a couple of recent pronouncements. An Intel-commissioned white paper from consulting firm Booz Allen Hamilton on the future of mobile technology was released May 2. It concludes that connected devices will inevitably interface with the human brain directly: “As convergence continues across device types, functions, and capabilities, the melding of mobile technologies directly into the human body becomes the logical next step.” The June 2 issue of the Wall Street Journal had a feature article, “Bionic Brains and Beyond: High-tech implants will soon be commonplace under our skin and inside our skulls, making us stronger and smarter,” by Daniel H. Wilson, a science-fiction writer.
Continue reading

Major themes at the Mobile Voice Conference

I was co-organizer with AVIOS and prepared the program for the Mobile Voice Conference, held in San Francisco, March 12-14. This third year of the conference gained momentum from the launch of Apple’s Siri, which was mentioned frequently in talks. The Mobile Voice conference theme covered specific developments in the use of speech recognition and other speech technology in mobile phones, where the small devices are motivating increasing use of voice interaction, as well as the implications of the growing acceptance of speech technology in other areas (e.g., customer service), driven in part by its acceptance on mobile devices.

It is impossible to summarize a full conference briefly. This note is thus intended to convey my impressions of the key messages of the conference. Talk titles and speaker names are still available at

Continue reading

What does it take for mobile personal assistants to “understand” us?

Discussions of computers understanding human language always seem to evoke “artificial intelligence,” along with science fiction images of powerful computers challenging humans. Apple’s Siri interprets speech to try to understand the goal of the mobile phone owner, and does a remarkable job in some contexts. By interpreting speech, rather than just displaying the words spoken, it goes beyond pure speech recognition. What does the understanding? How far can this technology evolve?

Continue reading

Voice control of your TV: Is it listening to everything you say?

In a press release on December 4, Microsoft announced agreements with content providers that expand entertainment and TV options on the Xbox 360 through Xbox Live. In the same press release, Microsoft pointed out the role of the Kinect peripheral for the Xbox in making this content accessible through speech, “turning your voice into the ultimate remote control.” One can search for games, movies, TV shows, and music on Xbox LIVE “by simply saying what you’re searching for,” the release explains. One can presumably just lean back on the couch and say what one wants, since Kinect speech recognition works at a distance–a no-remote remote.

Continue reading

Siri-style mobile phone Assistants will change Internet marketing and customer service

While the speech recognition in Apple’s Siri voice assistant has received the most attention, a key innovation in Apple’s Siri is its dealing with a user request as completely as possible. Siri shortcuts many navigation and data entry steps through tight integration with apps on the iPhone and with selected Web sites. Apple has the most elegant and integrated solution that uses this approach—in part because they tie it to a specific hardware platform and applications shipped with that platform—but other vendors such as Microsoft, Google, Nuance, and Vlingo have “voice assistant” models that move in this direction. If a text option were available (type what you would otherwise say when speaking isn’t an option), the “assistant” model can become the dominant user interface style, at least on mobile devices. (See “Where Apple missed the mark with Siri.”)

The evolution to an Assistant model is inevitable, and it could revolutionize mobile marketing and search. The change isn’t evolutionary—It’s a fundamental shift in control of user access to information and resources. Continue reading

Are Patents on the Mobile User Experience in the Public Interest?

Some of the patent battles that are emerging could affect the usability of your smartphone. Apple’s suit against mobile phone manufacturer HTC, for example, is widely regarded as an attack on Google’s Android operating system, supported by the fact that Google transferred four patents to HTC that HTC is using to counter-sue Apple. Continue reading