This white paper is an ongoing project. If you have any comments or suggestions, please contact the author at wearables-at-edkeyes-dot-org.
Introduction and Motivation
The Archetype Wearable
Design Goals
User Interface Guidelines
Implementation Issues
Conclusions
Introduction and Motivation
Okay, you've finally gotten your wearable computer all built and running
happily. Now the question arises of what software you're going to use
with it. Are you going to load up Windows 98 and try to manipulate
your Excel spreadsheets while walking down the hall? Or are you going
to try out Linux? ... And if so are you going to run it in command-line
mode or try to deal with Netscape under X Windows?
Basically, you rapidly come to the conclusion that a decent user interface for wearables simply does not exist yet. Why is this? The main answer lies in the human end of things, actually. All the desktop user interfaces have an underlying assumption that you're not doing anything else except using the computer. They are designed assuming that 100% of your cognitive load is being used to navigate their interface and get the computer to do work for you, work that has your total attention.
This completely breaks down in a wearable environment, where the whole point is to have your computer augment your capabilities while you're doing other things. You ought to be able to take notes and look up new information in the middle of a conversation. You ought to be able to review a talk outline while you're walking down the street. You ought to be able to totally ignore your wearable until it needs to remind you of an imminent appointment. And current user interfaces simply are not built with these needs in mind.
As mentioned before, there are two great user interface paradigms out there, the command-line interface (CLI) and the desktop-metaphor graphical user interface (WIMP: windows, icons, menus, pointers). Neither one is really appropriate for wearable use.
The CLI fails because it requires too much cognitive load to navigate. Imagine trying to compose a complicated recursive `find' command line while walking down the street: bringing up man pages and trying to memorize the options you need while simultaneously working out what the chord combination is for an ampersand. You'd get hit by a bus! The whole point of wearables is to augment your own capabilities. If you're using up 90% of your brainpower just to work the wearable, it's not an augmentation but a tremendous handicap.
Likewise the WIMP fails because it is too inefficient. On a desktop, where you have a nice mouse and a big monitor, it makes sense to devote computing resources like screen space to interface elements. It's fast to mouse around, and friendly to have interface objects to directly manipulate. Unfortunately, on a wearable, pointing devices are much less efficient: moving a mouse cursor requires the attention of both the hand and the eye simultaneously, so it's nearly impossible to devote much attention to other tasks while navigating the interface. The situation isn't helped by the replacement of a mouse by a thumbpoint or tilt sensor. In a similar manner, screen space in a heads-up display is much more precious than on a 21" monitor: the wearable should use such space for relaying information, not interface.
Thus, we need a new user interface for wearable computers. This is unfortunate, since it means writing new applications from scratch, but it really is justified. People interact with their wearables in fundamentally different ways than they do with their desktop systems, so just as mainframe-era batch processing interfaces needed to be supplanted by interactive personal computer interfaces, we need to set aside the assumptions of desktop metaphors and design something new that's suited to the tasks at hand.
Fortunately, the computing industry has decades of experience in making user interfaces, and we're free to take the best of the CLI and WIMP worlds. So let's just jump right in and specify what sort of system we have in mind and what our high-level design goals are. Then we can move on and lay out our wearable UI and get to work on implementing it.
The Archetype Wearable
The sort of wearable computer we have in mind is fairly minimal. We
expect our wearable to have the computing power of a desktop machine
of a couple of generations back, say in the 10-100 MIPS range, with
8-32 MB of RAM and 50-500 MB of permanent storage. This is sandwiched
right between the capabilities of a PDA and the capabilities of a
modern desktop system. These statistics are well-met by the current
crop of wearable computers, many of which are based around 486 or
low-end Pentium chips. Since power equals weight through the wearable's
reliance on batteries, it's appropriate to have a system powerful enough
for the applications we have in mind, but no more so.
Wearables in our class have enough horsepower to run a decent GUI and crunch through text with ease, but not enough to tackle serious computational challenges such as voice recognition or intelligent vision systems like augmented reality: those unfortunately are still the domain of the `research wearable' rather than consumer-level devices. Of course, there's nothing to prevent research wearables from using our interface as well, but we won't be assuming that we have access to any of their advanced capabilities to use in the interface (such as using video finger-tracking as a pointing device).
We assume our wearable has a heads-up display. While many wearables have been built to use voice interfaces or handheld display panels, we will only be concerned with heads-up displays, since the others have their own user interface issues which are quite distinct. A voice interface is completely alien to a graphical one, and the existence of a separate screen with a touch panel makes pointing devices suitable for interfaces again, unlike our own situation.
The heads-up display is not assumed to be superb. Current consumer-level technology on this front is frankly pretty bad: you can get something that's good, something that's cheap, or something that's being sold today. Thus we'll assume our wearable is equipped with a HUD that's equivalent to the early generations of home computers which hooked up to television sets. A resolution of perhaps 240x160 in monochrome would be a rough minimal statistic, with a reasonable extension of 320x240 in color. A VGA display would be considered extravagant for our purposes.
Our wearable has a keyboard. One of the primary tasks we're devoted to optimizing is text processing: note-taking, messaging, database lookups, etc. Thus it is necessary that our wearable has an input device suited to slamming out freeform text at reasonable speeds: say 30 words per minute or so. We'll also be using the keyboard as an interface device, so one that has a good number of `extra' keys would be nice. The archetype we have in mind is the Twiddler, which is capable of decent text speed as well as a large number of customizable control characters. A standard PC keyboard is also okay if portability issues have already been worked out.
That's about it. We don't assume that any pointing device is available, although we would recommend that some means of analog input is available if you happen to have applications that require it. The user interface itself won't need it, however.
Design Goals
Perhaps as important as what our design goals are, is what they aren't.
We'll tackle these caveats first to get them out of the way.
Our interface will not necessarily be efficient for novices. Instead, we want an interface that is optimized for the experienced user, since over the lifetime of the wearable, the user will only be a novice for a short time. This is not to say that the interface won't be accessible to a novice, only that it won't be efficient. The main thing we want to avoid is making an interface so friendly that it hampers the advanced user from taking shortcuts. The eventual goal is for users to be able to navigate programs while carrying on other tasks: overly user-friendly interfaces tend to get in the way of low-overhead navigation.
The model that we're using is that of another piece of great wearable technology: the digital wristwatch. For a novice, the combination of button presses needed for multiple lap timing or whatnot can be very intimidating. But the functions you actually use from day to day rapidly become natural (when's the last time you had to consult your watch's user manual?) and are an efficient use of very limited input / output capabilities.
Our interface will not try to do everything. The Macintosh has a complete graphical user interface: there's no command line you need to drop back to for system maintenance. We won't try to do this. Our interface is only intended for everyday tasks. For special-purpose needs such as system configuration, it is perfectly acceptable to have the user quit our interface and drop back down to a command line. Why can we get away with this? Because such specialty tasks are not something you typically need to do while walking down the street. The wearable has the user's complete attention, so it's perfectly okay to switch to a desktop user interface which is already optimized for dealing with fully attentive humans.
Our interface should require no more than a one-page instruction manual to describe to a user familiar with modern GUIs. In other words, there must not be many `hidden' elements to the user interface which are not accessible in the normal course of exploration. A good example of this is the `hold down the shift key for multiple selection' rule in the Macintosh user interface: it's a great idea to have this be standardized in the interface, but if the secret is not told to the user at some point, it is likely that they will never discover it on their own.
A corollary to the above rule is that the user is expected to fully absorb everything on the single page of instructions. One of the elements on that page might be, for instance, a secret key combination to bring up the on-line help. If the user misses that, then the normal process of interface exploration will be severely hampered. The main thing we want to avoid is the dreadful interface example of `vi', where it is impossible even to quit the program without secret interface knowledge. Thus we're trying to be explicit about the amount of secret knowledge required. It might also be a good idea to lock the user into a `safe' mode when a wearable is first booted which takes the user through this one-page tutorial (easily bypassed by an interface expert, of course).
Our interface should be able to adjust to differing levels of attention. One of the things that separates a wearable from a traditional desktop environment is that the user is rarely giving the wearable his full attention. We want to be explicit about this at the lowest levels of the user interface so applications will be written to adjust themselves to different levels of cognitive load: a date book program might expand out into a monthly planner when the user is interacting with it, but then collapse down into just a big clock display when the user is otherwise engaged. Likewise, if an appointment alarm is going off, the reminder might be a discreet note at the bottom of the screen if the user is attending to the display, or a garish flashing of the entire HUD if the user's attention needs to be drawn back to the wearable from the real world.
Our interface should be efficient at mode changes. The wearable needs to keep up with rapidly-varying user needs. In less than 30 seconds, a user might need a face-recognition program to figure out who he's talking to, an address book to remind him of the person's wife's name, a notebook to jot down a reminder to send a card for her upcoming birthday, and a remembrance agent to jog his memory of the subject matter of the last conversation he had with the person.
In an ideal world, the wearable would be context-sensitive enough to do all this automatically, but this is not yet feasible, so the user still needs to maintain control of the wearable's functionality (hopefully augmented by helpful agents). To that end, the interface needs to be flexible enough to keep up with rapid mode changes. The user ought to be able to bring up common applications almost instantly to a useful state: waiting a minute for Netscape to boot itself up is simply not an option. The model here is the PalmPilot, which while computationally limited still projects an image of speed by simply being very responsive to the user in things like application switches.
A corollary to the above is that our interface should have very few modal elements. The user might be interrupted with pressing information needs at any point due to interactions in the real world, so it's a very bad idea to lock the user into a multistep process which is difficult to cleanly cancel out of. The user ought to be able to instantly table ongoing applications, look something up, and then come back to them later.
Our interface needs to be scriptable. We are designing for the advanced user, and aiming for an interface which is navigable with minimal cognitive load. To that end, we simply must give the user power to create and use macros. If we can save a single keypress with common tasks, that's another million neurons that the user can bring to bear on his real work. Likewise, agents will eventually be a large part of the wearable environment, so we want to give them the power to control as many aspects of the wearable programmatically as possible. Therefore we can't lock our interface into a mode where only the user can initiate actions, and that means scriptability. Keep in mind, though, that this opens up a good deal of security issues which will have to be paid attention to in our implementation phase.