By Joe Welinske
To all User Assistance developers,
This is my report to you on the most recent edition of the CHI Conference on Human Factors in Computing Systems, which took place March 31 to April 5 in Seattle. The conference offered a broad overview of the research being conducted in the areas of usability, accessibility, user interface design, evaluation methods, wireless computing, and many others. I attended the technical sessions on Tuesday, Wednesday, and Thursday. The conference content was very stimulating and centered around foundational research. While the information was not directly applicable to user assistance design, there were many interesting findings. Here is a brief sampling of the sessions I found of interest from a technical writer's perspective.
 
Joe Welinske
President, WinWriters
Opening Keynote: Bill Gates, Microsoft
A predictably huge crowd of about 2,800 was on hand to listen to the world's (second?) richest person. The title of the session was Sharpening Our Vision Through Research. It wasn't as much about research as a showcase of Microsoft latest initiatives including Reader, the Tablet PC, and the Notification Platform Project.
Bill Gates began with a brief review of the evolution of software applications over the last twenty years. He described how the proliferation of different screen sizes and input devices is frustrating developers. He also explained the company's new mission to support a software development model where a product is constantly updated, distributed, and sold as a service. He also talked about the new era where XML makes it possible to easily move data between different types of machines and mobile devices.
According to Bill, in 1988 the number of people officially assigned to usability testing at Microsoft was two and they did three studies. Today Microsoft employs 142 usability engineers (I'm not sure why they are called engineers!) who in 2000 performed 920 studies with 10,000+ participants at labs in ten locations.
The Reader portion of the presentation was conducted by MS researcher Bill Hill. He reminded us why printed books are so much more comfortable than online reading. He presented the encyclopedia as an example of a document type that's much richer and more accessible in online form than it was in print form. Then he went into a somewhat odd theory of how the human visual pattern recognition system, developed during hunter-gathering days, affects how we read. The recognition system is always running in the background of our consciousness. When we talk on a cell phone while driving a car, we are constantly seeing all kinds of things, but after we're parked we don't even remember how we got to where we are.
Next, Bert Keely presented a demo of the Tablet PC. Looking like a thick notebook, Tablet PC is built to hold in one hand. It is a full-featured PC, which uses a writing stylus as an input device. Bert demonstrated writing on the pad, then highlighting his scribblings, and pasting that data directly into a Word document as character text. An interesting comment from Bert was that the way you write something down on paper provides some future context. Underlining, bold strokes, and arrows all provide depth of meaning for your notes when you refer to them later. Preserving the scribblings preserves that context.
Finally, Eric Horvitz gave a future shock demo of something called the Notification Platform Project. The idea (I think) is that all of the information you collect about you, your daily tasks, appointments, pages, phone calls—all your interaction with the world—can be prioritized and programmed to operate automatically and remotely. In his demo, Eric first showed how he could have his incoming email automatically prioritized based on attributes previously assigned to senders and subjects. His Outlook mailbox also was programmed to forward messages to his cell phone, if he hadn't checked his mail in three hours. At that moment his cell phone rang and supposedly it was giving him the mail that had come for him since breakfast. At one point he had eight different windows open that were all automatically and dynamically tracking his actions including where he was standing and what he was looking at.
Read the complete keynote transcript. (Web page)
Technical Sessions
This is a round-up of the technical sessions that I attended. I felt these particular sessions could have some application to user assistance development. The session paper and slide resources included here are provided with the permission of the speakers.
Testing Web Sites: Five Users Is Nowhere Near Enough
Jared Spool of User Interface Engineering (UIE) described the results of some recent research that brings into question a long-standing rule of thumb put forth by Jakob Nielsen. The point of contention is whether you can truly find the large majority of usability problems in complex web site by employing a very small number of testers. The Nielsen rule of thumb, referred to by Jared as the "parabola of optimism," states that five testers can find 85% of the problems in a web site regardless of its size. In contrast, the UIE research indicates the number of testers needed increases linearly with the size of the web site.
The UIE numbers stemmed from working with 57 test subjects on six e-commerce sites. The subjects were charged with doing compelling shopping tasks and they were provided with incentives to use those sites in a real-world manner.
UIE found that:
- The testers only overlapped in their findings by 3.2%.
- No amount of user experience will be victorious over bad design.
- No two users approached their tasks in the same way.
 Read the session paper. (PDF)
Exploding Wireless Myths: Exploring the UI Issues Underlying the Marketing Hype
A panel of three experts pointed to several commonly shared beliefs about the evolution of wireless devices and explained why they may not be accurate.
Scott Jenson of Interaction Design pointed out that currently the Web and the desktop browser are the same animal. Virtually all web sites have been designed for presentation on a large monitor and interaction is assumed to be done through a keyboard or mouse. Rather than scaling the Web "down," fundamental changes must be made from the ground up in developing wireless web-based applications.
Annette Wagner of Sun Microsystems talked about the promise of convergence of devices, for example the integration of a PDA and a cell phone or a web browser. She strongly questioned whether an all-purpose device would provide benefits to the user. Annette reminded us that the best consumer product success stories are ones where the product is "fit for purpose."
Avril Hodges of Phone.com said that most users do not have overblown expectations that their portable wireless devices can provide the same rich experience as a desktop browser. What they do expect is a simple UI that allows them to receive the appropriate content with minimal problems. It is up to the developer to determine what is reasonable given a device's capabilities and to deliver that without excuses.
View the session slides. (PowerPoint presentation)
An Integrated Method for Evaluating Interfaces
David Bishop of MAYA Design described a new approach that his organization uses to help clients form a clearer plan for addressing software design problems. A heuristic evaluation is first used to identify what is often a long list of problems. These are categorized into groups and rated according to each user's importance. For example, fixing navigation problems might be very critical, whereas fine-tuning the layout less so. The groups are then rated for difficulty in implementing fixes. A scatter plot is generated from the pairs of data points, which provides a measure of "return on investment." Bishop claims this approach helps clients focus on big-picture solutions rather than just powering through a laundry list of bugs.
Read the session paper. (PDF)
An Empirical Study of Human Web Assistants: Implications for User Support in Web Information Systems
This was a study conducted at Linköpings universitet in Sweden. The researchers hoped to provide data regarding the value of using human assistants for web-based chat support. The study consisted of adding a user support system to an existing non-profit web site. The web traffic of the test site averages over 14,000 users sessions per day and each session lasts about thirty-five minutes. A user support system gateway was provided by Help icons placed on each web page. If a user needed help they first typed a question into a text box. The natural language query was parsed and matched against a FAQ database. Matching items were presented to the user. If this was not satisfactory to the user, they then could initiate an online chat with a support person. The support staff for the study were volunteers selected from the pool of actual site users. They were chosen based on having sufficient domain knowledge about the site.
Some of the results include:
- A user will wait for an assistant but not for long.
- A human assistant provides a human touch, and users have a high level of trust in the information and in the support system.
- Users differ in conversational style and background.
- The assistants had difficulty when working simultaneously with multiple users, and it caused high stress.
- The assistants found it difficult to express some solutions in writing.
- The most important assistant skills were patience, domain knowledge, social skills, and fast typing.
- When users are waiting for an assistant they should receive updates of their queue position and the amount of expected waiting time.
- The efficiency of the system is tied to the efficiency of the assistants.
Read the session paper. (PDF)
Reading of Electronic Documents: The Usability of Linear, Fisheye, and Overview+Detail Interfaces
As people become accustomed to reading more and more online documents, the way we structure the words, sentences, and paragraphs on these electronic pages is very important. This study, conducted by researchers at the University of Copenhagen, compared three different navigational formats for reading a document. They measured efficiency (the speed to process the document), effectiveness (comprehension of the content), and satisfaction (how the test subjects felt about the experience).
The experiment presented the documents to the user through a computer window. The three different document formats were: a Linear document, a Fisheye, and an Overview+Detail. The Linear document was just a standard page of text. In the Fisheye, certain less critical portions of the document (determined by the researchers) were distorted below readable size. This made those sections look like greeked text and they acted as visual placeholders. Clicking on a block of Fisheye text expanded it to a readable size. In the Overview+Detail format, section headings and summary text was available in a narrow pane to the left of the document text—it looked like a two-pane frameset. Clicking on part of the overview text jumped the user to the appropriate place in the document.
The test subjects were asked to answer questions about the document they'd just read and also to write essays about the document. The testers used documents that would let them test both conceptual (reading to learn to do) and procedural (reading to do) information.
Some of the findings include:
- When it came to writing essays about the documents, Overview+Detail resulted in higher grades.
- The Fisheye had lower scores regarding incidental learning questions.
- Overview+Detail was preferred by the readers.
- The Fisheye was confusing. It wasn't clear to the readers why certain text was chosen over other text to be in Fisheye form.
- When it comes to question-answering speed, the Fisheye is significantly faster than the other two formats.
- Overview+Detail took 20% longer than Linear.
- Overview+Detail helps the reader to gain a better knowledge of the overall structure of the document.
- The Fisheye helps users to grasp main concepts or to quickly complete tasks.
I mentioned to the presenters that the Fisheye is very similar in operation to the expanding/collapsing text effect that can be enabled with Dynamic HTML. This study may have some bearing on the usefulness of that technique.
View the session slides. (PowerPoint presentation)
What Makes Web Sites Credible? A Report on a Large Quantitative Study
As the amount of content on the Web grows exponentially, our ability to judge the credibility of that information is becoming more and more important. This Stanford University study surveyed 1,400 people to find out what factors increase or decrease the credibility of a web site in their eyes. This included looking at aesthetic features, content variables, and technical factors. In the researchers' definition, credibility is believability. Two of the most important components of credibility are trustworthiness and expertise.
Based on the study, the researchers assembled a number of design guidelines:
- It is important to provide elements that highlight the brick and mortar nature of the organization. The top three items are: quick answers to customer questions, providing a street address, and providing a contact phone number.
- Ease of use contributes to credibility.
- When presenting informational content, include author credentials, citations, and references. This helps convey expertise.
- Credibility increases when a site recognizes an individual when they log on and interacts with them accordingly.
- The user penalizes sites which have an overly commercial flavor. Mixing ads and content is the worst thing you can do. Banner ads that are done tastefully improve credibility.
- Credibility drops with even small glitches, typos, or broken links.
- Requiring users to register or login has no effect on credibility—positive or negative.
Read the session paper. (PDF)
View the session slides. (PowerPoint presentation)
Attending to Web Pages
As with printed pages, some web pages immediately draw in the reader while others just present a confusing array of objects. Peter Faraday of Microsoft used eye-tracking software to find out where users would fixate their gaze when presented with a page. The experiment tracked the order in which the subjects fixed their gaze on page objects and the dwell time on each object. Several different web templates were used. The templates contained headings, bullets, links, paragraph text, and other typical document objects. The experiment modified the size, typeface, and page position.
Some of the findings are consistent with long established principles and common sense, such as large text dominating smaller text and bold text dominating plain text. Other findings include:
- For an image to be the primary point of fixation, it must be much larger than the surrounding text.
- Left-to-right and top-to-bottom reading order only works for substantive text. Titles, images, and links will be scanned based on size regardless of position.
Read the session paper. (Word document)
What I Need Is What I Get: Downloadable User Interfaces via Jini and Java
Two representatives from Sun Microsystems gave a demonstration of their Jini connection technology. The concept is for devices and services to become known to each other without prior collaboration. In the demo, one of the devices was a lamp. Its menu of services consisted of being turned on or off, or dimmed. The lamp was outfitted with a controller that can wirelessly broadcast its capabilities to a user device. The user device, termed a "personal accessor," might be a PDA, a cell phone, or a speech interface for a blind person. The personal accessor learns what a device can do on the fly without previously having been designed to know anything about it. This opens the way for an infinite number of disparate devices to communicate and interact with each other. Interestingly, this future scenario was also mentioned in Bill Gates' keynote presentation.
Read the session paper. (PDF)
Concurrent Vs. Post-Task Usability Test Ratings
This study from Intel focused on how usability test results differed depending on when the feedback was elicited from the test subject. The experiment had the test subjects work through a software task. Some of the test subjects were interrupted during the task and asked for their feedback. Others were questioned after the task was complete.
The ratings of the task provided by the group questioned after the task were significantly higher than the ratings from the interrupted group. However, the study also found that subjects that were interrupted during the task were able to provide richer explanations of their experiences.
Other findings include:
- There was no difference in how the testers felt about the concurrent vs. post-task questioning.
- Concurrent testing made the task take 15% longer.
Read the session paper. (PDF)
Is Ignorance Bliss?: Informed Consent Online
The issues of privacy and security are becoming increasingly important and visible as the world of consumer interaction with the Web continues to evolve. In this session, six panelists discussed several aspects of how electronic media has changed our ability to grant informed consent regarding the collection and use of our private information. The subject of this panel was especially timely, taking place the same week that Microsoft was forced to face criticism regarding the legal fine print associated with its commercial web sites.
One of the panel organizers, Batya Friedman of the University of Washington, gave a very comprehensive overview of what informed consent means and how it is being respected in some of today's prominent web sites. Being an "informed" user means receiving adequate disclosure of the pros and cons of engaging in an activity and being able to adequately comprehend what is being disclosed. The "consent" component consists of being competent to grant the consent, having a clear opportunity to accept or decline, and voluntarily providing the information without being coerced. On the last item, voluntariness, Batya pointed out that coercion to consent can occur when there is only one reasonable way for an individual to receive certain services. For example, if just one or two online payment services were the systems of choice for most web vendors, consumers would be forced to accept the terms of those payment services in order to purchase from those vendors.
The presentation from Batya also included an interesting retrospective on how cookies have been used by Internet Explorer and Navigator. She addresses the privacy problems and provides design remedies.
View the session slides. (PDF)
(The work by the World Wide Web Consortium on standards for privacy is somewhat related to the discussion above. Link to the P3P page—Platform for Privacy Preference Project.)
Tracking Mouse Movement Activity on Websites
Florian Mueller, MIT Media Lab, floyd@mediacaptain.com
The goal of this project was to analyze all mouse movements on a web page, beyond just where a mouse rested when clicked. The researchers used a JavaScript program to track all of the movements of a mouse during the time a user was reading a page.
Interesting findings include:
- 39% of people moved the mouse while reading a page
- 34% of people moved the mouse cursor and parked it in white space
|
