Multi-Touch Human Interfaces Have Arrived

Multi-touch human interfaces have arrived, but soarbitrary. What doesn't make sense is the
far they are like surprise visitors: we may becomeback of the button, being used as a modifier
happy to see them, but we're not ready to letto the three-finger gestures, even though the
them move-in for good. For those of us who aretouchpad is large and has the ability to sense lots
engineers and work every day to realize theof fingers. Also, the multi-touch equivalent of the
dreams of future products, multi-touch offers anmouse-click+ -drag to select seems to have
opportunity to change the world, in much thedisappeared. Anyone else have trouble completely
same way the first graphical user interfaces did.adopting a Palm(TM) PDA because Grafitti(TM)
The race has just begun, but make no mistake, itnever became second nature?
will be won soon. Personally, I'm not convinced weIntuitive gestures equal easy adoption
have a winner yet, and if we don't step up toLikewise, Microsoft with Surface(TM) seems to be
make multi-touch a great technology, we will livestumbling in the dark when it comes to using
with it being marginal for a long time.more than two-fingers. For instance, they have a
What is multi-touch?virtual air-hockey game demo that uses one
The world applauded Apple's first-generationfinger for each player to grab and move the
interpretation of a multi-touch interface in thecontroller (there is another version that uses a
iPhone 1.0. The new user interface worked, itphysical controller like a traditional air-hockey table,
offered appropriate functionality, and they didn'tbut that isn't a multi-touch interface). When I play
forget to make phone-calling easy. Apple iPhonereal air-hockey, I would have been stupid to use
added gestures to our vocabulary, offeringone finger, and the true-to-life nature of Surface
two-finger pinch to zoom-in, two-finger open todoes reflect this behavior as you can see the
zoom-out, and two-finger rotate; all intuitive anduser's controller slip-and-slide around as if he were
useful on a device with a camera and smallusing only one finger to control a large disk. There
screen.are other problems too, such as the controllers
Not many people realize that Apple began theirsometimes switch players when they get too
foray into multi-touch with the two-fingerclose. This commentary is not intended to slam
touchpad sensing implemented in their largeApple's MacBooks or Microsoft's Surface but
touchpad notebooks that was utilized by therather to highlight the fact that the multitouch field
operating system and every application. Evenis wide open. The technical capabilities available
before the iPhone, an Apple notebook user couldtoday do offer a much more natural and intuitive
scroll, both vertical and horizontal, using twouser interface IF AND ONLY IF we as engineers
fingers on the touchpad. Apple even made theirharness the power and direct it to the greater
stubborn single-button-touchpad choice lookgood.
prescient by enabling right-mouse buttonThe software development kits and software
functionality by putting two fingers on thedevelopment tools for Microsoft, Apple, and Linux
touchpad and clicking the button (eliminating theall provide built-in, standard support for keyboards
annoying -click). Again, it's hard to call theseand X-Y pointing devices (mouse and track-pad
choices anything but useful and welcome. Justbuttons as well). Anyone today can target any
recently, multi-touch offerings or announcementsoperating system and as long as the "input" is
from many different vendors highlight thetranslated into one or more keyboard keys and
"final-frontier" opportunity today for multi-touchX-Y position, any application can use that input.
user interface technology. Apple introduced theirFor example, someone could develop, using
next-generation MacBooks on October 9, 2008standard offerings in any operating system, a
showing the familiar two-finger gestures of iPhone10'-by-10' room as a trackpad replacement, where
with new - and less intuitive - three- andone runs around on the floor, jumps up and down,
four-finger gestures. Microsoft began shipping itsand throws one's body against a wall to select
multi-touch brain-child called Surface that comesand move icons around a PC screen. In the
with a hefty $5,000 to $15,000 price tag makingmulti-touch future, what do we as a development
it clearly aimed at money-making businessescommunity need to do to secure a similar level of
(early adopters are AT&T and Harrah'sfreedom to develop input devices and the
Entertainment). At the October 2008 Microsoftprograms that interact with them? What do we
Professional Developer's conference, Microsoftneed to demand as a standard set of provided
unveiled its upcoming Windows 7 and promisedcapabilities so program developers do not have to
multi-touch support. The Microsoft idea ofworry about the input device and input device
multi-touch as shown with Surface provides somemakers do not have to worry about the
gestures, like the 2-finger gestures to zoom andprograms? While the ultimate answers are up to
rotate, but seems more targeted at a multi-userall of us, the answers being developed today could
experience. So the question remains, what isseriously affect our lives for the future.
multi-touch? The impact of multi-touch will be asThe future of Multi-touch is in our hands
far-reaching as the computer mouse and the"Standards" for multi-touch interfaces are being
engineering community needs to step in to ensuredeveloped whether we like it or not and now is
that multi-touch fares better in the future thanthe time to get involved, make noise, ad shape
the graphics pad (remember the digitizing, pen-likethe future of multi-touch. Let me put forward
computer input device that none of us has on oursome of my suggestions for a better multi-touch
notebooks?)future. Read them, use them to come up with
Multi-touch technology todaybetter ideas, and then get involved by either
In contrast, single-touch interfaces are most oftenworking with the standards-setters or by
based on the old "touchscreen" technologies mostimplementing a multi-touch device and putting it
of us experienced on our cell phones and PDAsinto people's hands so we can mobilize more
were resistive, where the stylus position yieldedtroops in this fight for a great multi-touch
two voltages, one representing the "X-axis"future.First, we need a few standard, intuitive
position and the other representing the "Y-axis".gestures and second, we need a standard data
Single-touch interfaces result in the same X-Yinterface to provide position data for up to 10
data that a traditional track-pad and mousefingers. Standard gestures should cover the most
provides today. For their multi-touch interfaces,common computer/information device operations,
Apple uses capacitive sensing and Microsoftlike scroll, zoom, select, move, and
Surface uses cameras. Capacitive sensing is alsograb-and-move, as well as all the new functions
the technology typically used in single-touchmulti-touch will enable. What we do not need are
trackpads in notebook computers. There aremultiple company-patented sets of gestures.
several module, screen and individual siliconRather, let the innovators patent better and
vendors supporting multi-touch, so there aresmarter techniques of determining the gestures.
many options available on the market, makingNote that we do not need a long list of standard
selection a potentially confusing process.gestures, because if that is what we get, users
Depending upon the technology, it is possible towill need to print out the list and paste it to the
simultaneously sense the position of all 10 fingersback of their devices, just like we all did with
on a display. Certainly the capability to supportGrafitti on our Palm PDAs before we stopped
multi-touch interfaces in a myriad of ways isusing them. Also, gesture detection cannot be
available. The rub is that there is no one standardforced exclusively upon the operating system nor
way to use this multi-touch data in a computercan it be forced exclusively upon the input device.
application.The best standard solution would accommodate
To see where multi-touch can go, let's return toboth for maximum flexibility. However, if a choice
how the two easiest-to-review implementationsbetweenoperating system or input device is
of multi-touch interpret what the users of theforced, the operating system must be allowed to
future (that's you and me) need. The Applewin.
iPhone implements what is referred to as "Do not stop with standard gestures (or the
Multi-Touch Gestures " where two fingers arecorollary: Do not define everything as a gesture).
sensed and their relative motions translated into aDefine a data standard for multitouch input
gesture that a program can react to (i.e., rotate,devices for tracking up to ten independent inputs.
zoom, select and move). At a minimum, theseRealize that the use of this data for quite some
gestures need to appear quickly in the lexicon oftime to come will be application-specific. That said,
every laptop, but that means every program andand over time the best behaviors (we hope) will
operating systems needs to change tobe adopted into the operating systems, Why ten
accommodate them. Beyond the 2-fingerand not more? Most devices are primarily
gestures, the picture blurs quickly. "Multi-Touch Allsingle-user or have multiple users doing simpler
Point" technology enables many simultaneousactions on a relatively small screen. Let the
inputs from the touchscreen or touchpad. Whatspecial-purpose large-format devices like Surface
to do with these inputs, now, is the question.explore what to do with more than ten inputs,
There are huge opportunities here, but theand when something proves itself useful, it can
current examples of Apple and Microsoft are nottrickle down. The multi-touch train is leaving the
exploiting them to the fullest.station. For those who do not like everything they
Apple has included three- and four-finger gestureshave seen so far, get involved and push the
on the new MacBook, but only Apple applicationsenvelope further. Use multi-touch in new and
use these gesturesinteresting ways and let others see and hear
(unlike the scrolling and left clicking two-fingerabout your successes and failures (especially the
gestures). Additionally, the thought process behindguys in Cupertino and Redmond). The industry can
which functions should take three fingers andmake multi-touch great, but only if we
which four fingers appears to be somewhatworktogether.