O'Reilly FYI

News from within O'Reilly

Communicating Interactive Gestures

By Kathryn Barrett
February 5, 2009 | Comments: 1

Nintendo's Wii, Apple's iPhone and iPod touch have made gestural interfaces wildly popular, but have created a new set of challenges for designers who want to move beyond traditional interface design. Dan Saffer's Designing Gestural Interfaces covers the subject exhaustively, but also shows how carefully we need to look at our assumptions about the way people interact with devices. In this excerpt, Dan covers communicating interactive gestures as well as the three zones of engagement designers need to be aware of.

"The perception of what a thing is and the perception of what it means are not separate, either."
—James Gibson
When Antenna Design set out to redesign New York City's Metropolitan Transit Authority's (MTA) ticket vending machines,* it initially assumed that everyone would realize the machines had touchscreens. After all, they figured, everyone had used ATMs and touchscreen kiosks at airports. But they found that, because the MTA serves literally tens of millions of people from all walks of life, a large percentage of its users had never done those things. They didn't know what it was like to use an ATM because they didn't even have a bank account.

*As related by Antenna Design partner, Sigi Moeslinger, at Interaction08. Watch the video at http://interaction08.ixda.org/Sigi_Moeslinger.php.

Thus, when it came time to design the screen for the MetroCard Vending Machines, Antenna Design needed to provide an extremely obvious visual cue that the kiosk was a touchscreen.


Figure 7-1. Even though you can touch anywhere on the screen to begin, Antenna Design wanted to make sure the instructions were dead simple, with two written cues and an animated hand to remove all ambiguity, even for tourists who do not read English. Courtesy Antenna Design.

Especially with free-form interactive gestures but also with touchscreens, it increasingly isn't enough to simply install a product and hope for the best; the best gestural interfaces need to be discoverable. Users need to be made aware of their presence and, after that, their affordances. What can I do here? How can I activate and engage with this? What are the controls?

Designer Clive Grinyer relates a humorous anecdote* that brings the need for communication into stark relief:

"I work in Paris, in a large converted telephone exchange where we have recently installed new light switches that save energy by turning off when they don't detect movement. When you go to the loo, you don't move much, you might move bits of you, or you might grimace a bit, but it's not movement as such, certainly not detectable by the infrared monitor. So, after 20 seconds, the light goes off.

"Someone, somewhere, made the decision that after 20 seconds the light would go off. It might have been the facilities manager. It was more likely the person who set the default, probably the kind-hearted engineer who programmed these switches. He (and it was almost certainly a He) went home feeling good. He may have even told his kids that he had saved the world a few kilowatts, that he was doing his bit against George Bush (especially if he was French), and felt happy and satisfied when he went to sleep that night.

"But the experience I am having of his decision is that I am sitting on a loo in a foreign country waving my arms about because I think there is probably a sensor somewhere, if only I could see it, which will eventually see me and turn the light back on."

*Read the whole post, "Technology doesn't work," at http://blog.clivegrinyer.com/blog/_archives/2007/2/13/2733052.html.

Although clearly this is an example of bad design (20 seconds is an awfully short period of time), this story amply indicates that communicating interactive gestures means communicating two pieces of information:


Alert potential users that a gestural system is available to them.


Teach users how to engage with the system. This includes the most basic instruction, namely, how do I turn this thing on (or off)?

These two things can be delivered in a variety of ways over a variety of distances from the product.

Three Zones of Engagement

Most gestural interfaces have three zones of engagement, which happen in space relative to the size of the device (this framework was suggested by Darren David and Nathan Moody of Stimulant):


Figure 7-2. The three zones of engagement. The nearer to the product the user is, the more variety of communication methods can be employed to engage the user.


A person spots the product or output from the product, such as a sound, and is (hopefully) interested and intrigued by it. This typically happens from a distance—for large environmental displays, this could be from very far away. Often the attraction is triggered by environmental cues such as signage, sound, or the hardware of the product itself, or it could simply be that the person notices someone using the system. The gestures themselves, if broad enough, could also be enough to attract attention.


Figure 7-3. This touchscreen kiosk for HP's Photosmart Express is wrapped in signage to attract users in locations such as supermarkets and pharmacies. Note how the signs peek above the height of the shelves, and there is even a mat on the aisle floor. Courtesy Hewlett-Packard.

From midrange, a person is able to see more detail about the product and the gestures involved in engaging with it. At this point, environmental cues such as signage are crucial. Signage can both instruct and engage at this distance. It's also from this distance that users can demonstrate to others how a product works. Observers learn the UI conventions of the product just by watching and asking questions.


Figure 7-4. From other parts of the room, one can watch others playing with the Wii and thus learn how to play the game. Courtesy Dru Kelly.

From up close (within a meter/yard of most devices), the person can become a user, directly interacting with the product. The instructions and affordances here are likely on the product itself, meant to be seen and read from very nearby.


Figure 7-5. Samsung's Anycall Haptic mobile phone's controls (such as those for all small, personal devices) are meant to be viewed and controlled only by someone holding the device. Courtesy Samsung.

When designing a gestural system, it is good to keep these three zones in mind so that proper communication channels can be established and designed early, and the correct communication methods are used.

You might also be interested in:


I was inspired and energized by Bill DeRouchey and Dan Saffer at interaction 09 in Vancouver. Many of these ideas and others were discussed. In addition, we did exercises for developing touch screen interfaces.
What I'd really like to see would be additional collaboration on a library of gestures to help developers design and communicate the interfaces of the future.


Popular Topics

Browse Books


Or, visit our complete archives.

FYI Topics

Recommended for You

Got a Question?