Problems in Navigating Online Help:
Clues from User Search Patterns

By Robert Krull, Rensselaer Polytechnic Institute & Angela Eaton, Texas Tech University

Much has been written on writing online help systems. Farkas (1999) provides a detailed explanation of the strengths and weaknesses of seven different genres of instructions—wizards, flowcharts, playscript procedures, the interface annotation model (balloon-type help), paragraph format, the rich step format, and the streamlined step format. Shroyer (2000) promotes using reader-role theory to assess the style preferred by the audience in a help system, and uses it to explain the demise of Microsoft's PaperClip feature. Pratt (1998) uses instructional design theory to inform her recommendations for writing help, including starting with a foundation of procedural information, adding background theoretical information to deepen user understanding, and including practice.

This advice, however, may not address a few problematic areas of current online help systems. In previous studies, we tested the help systems of three versions of a popular programming language and reported the quantitative results of their success in using the help system to do four tasks (Krull et al., 2001; Krull et al, 2002). Participants particularly had difficulty in finding appropriate topics within the help system. In this paper, we will examine the participants' comments and report their main themes, including the difficulties they encounter in using the help system. The lessons learned from these responses should assist help system designers and authors in accommodating users' search patterns.

Overall, the largest problem our participants had in using the help system wasn't in processing the procedural information in the help, but rather finding the correct help topic, a topic generally unaddressed in the literature on how to write a help system. Specifically, participants had difficulty in searching for topics because their terminology differed from the terminology used by the help system, and they became lost in the unclear structure of the system. Specific results and discussion are provided below.

Problem 1: Framing the Search Question

Our users embarked on a Help search as a last resort. Their behavior is not unusual—user reluctance to seek help has been documented in the literature (Grayling, 1998). By the time they use help, they have tried other tactics to solve their problem. Their difficulties with the task are likely to transfer to the Help system as well.

They may have problems with the task because they are unfamiliar with the specific vocabulary used by a computer product. For example, in our testing we found that users would not know the distinctions among seemingly similar terms such as "name", "caption", and "label." The product used these terms to refer to different concepts and users would encounter problems with the task if they didn't have the same concepts in mind.

This problem also transferred to the users' attempts to search the Help system. Since the Help system worked from product-specific terminology, it tended to offer relevant help only when users hit on the right term. Since users didn't know the correct terms (which is why they were having problems), they also wouldn't know how to construct search strings.

The Help system also was not organized in a way that let users distinguish between similar terms. It did not, for example, supply blatant links to a panel that offered definitions of such terms as name, caption, and label. Users would need to look at several panels to make such comparisons. Rather than enabling users to find a comparison panel through one search, users would have to conduct several searches successfully, to remember what they read at the end of each search, and then to deduce what the essential differences were. Since the Help system was so extensive that there could be five levels between the top level of the help hierarchy and a terminal panel that explained terminology, users would need to make navigation choices in the five-level system successfully up to 15 times to find what they needed.

A related problem was the choice and linking of terms into search strings. For our example case, should users enter specific search terms such as name or caption at the beginning of a search string (i.e. assuming that the Help system was expecting a noun as the initial search element), should enter a category that contained the search term (i.e. that they could guess that name or caption fell into the "properties" category), or use a more mundane search string modeled on an English sentence (i.e. What is the meaning of the term name?)? The only way for users to find out which search string was correct was to try them until one succeeded, which added to their frustration.

In addition, depending on the part of the Help system users accessed, one of these strategies was more likely to be successful. The net result for users was that the offerings from the Help system could appear to be remarkably random. They might get 500 hits, far too many to sort through, or none at all.

These findings suggest two guidelines for Help system development. First, help systems should allow users to enter terminology with which they are familiar. Since their terminology may not be the same as the product's, the Help system should link the two. For example, it should include a thesaurus of likely terms with a cross-linking system that offers users help panels based on more than one kind of term. The Help system also should direct users to panels that compare similar terms so that users can see the distinctions made by the product.

To help users with constructing search strings, the Help system should either have parsing of unrestricted term combinations or should offer users search systems that construct search strings from segments based on choices made by users from alternatives. Several familiar commercial Web systems such as Google and Amazon provide those systems.

The Web-based help for the product that appeared in version 6 did allow users to use terminology that was less product-specific and user performance was somewhat better as a result.

Problem 2: How Does this Search Target Relate to the Structure of the Help System?

We have already described one kind of problem for the hierarchy of a help system — seemingly similar concepts may be located in quite different places of the help hierarchy. This problem is acute because users rarely hit the correct information on a first try.

Users, at least in our studies, would search based on their best guess about how the product's terminology and help hierarchy were organized, and then would arrive at a less than ideal spot. That, by itself, would not have been particularly problematic. Their frustration rose as they tried to move from that spot to other, more relevant information. This problem takes two forms: how does one move up in the structure to more general information and down towards more specific information; and how does one move from topic to topic at the same level of generality? Users had difficulty making both kinds of movements because the product did not support them well.

Problem 3: How Do I Move Up in the Structure?

Users who navigated to terminal help panels generally did not end up with help on the topic they needed, even if they found the help panel to be of sufficient specificity and of the right type (procedural versus conceptual). Users then wanted to be able to explore help panels of the same sort, but on somewhat different topics. They had trouble navigating upward in the hierarchy to then follow another path downward; and they found no easy way to move laterally among topics of the same specificity.

If they abandoned the first help panel and returned to the top of the hierarchy, they often got lost and could not find either the path they had just followed or alternative paths in somewhat different directions. If, instead, they tried to hunt through topics at an intermediate level of generality, they could be faced with trying to distinguish the meaning of several hundred unordered search hits, shown 12 at a time in the search window.

For example, the help system did not offer a simple path to a panel that would distinguish among the terms "name", "caption", and "label." Such a panel could exist and users might a low chance of finding it.

The Help system did not offer a way for them to move easily upward or downward in the hierarchy or among conceptual nodes in the search targets. Both kinds of content-based help organization would have made the users' searches more efficient. The Web-based version of the Help system did offer a clearer structure of topics that enabled test subjects to distinguish fundamental topics from more specific topics.

Problem 4: How Do I Move Laterally to Another Topic at the Same Level?

Related to the preceding problem is users' trying to move laterally within the Help system hierarchy. Without an easy-to-use hierarchy, users may try to move among topics at the same level by clicking on See Also links. Unfortunately, this strategy presents problems of its own. First, the titles of such links do not tell users whether they are moving upward, downward or laterally within the hierarchy. Second, the links are not ordered in a way that enables uses to specify a general direction of movement — move in the direction of links like these and not like these others. Third, the links are so limited in number that they present the opposite problem of the search panels — too few rather than too many hits. Fourth, the links do not clearly signal if they will contain declarative or procedural information. Users could not tell which kind of information they would receive from link titles.

It may be tempting to solve some of these problems by tweaking the language in See Also links. That could yield unwieldy titles and could require a lot of effort from developers. An alternative could be to attach a restricted icon system to the links. Directional arrows (up, down, horizontal), plus a number (to represent the jump distance within the hierarchy), and a symbol (D for declarative versus P for procedural) could be studied as a means of signaling to users what kind of information they are likely to receive when jumping within the Help system.

Problem 5: How Do I Move to Declarative or Procedural Topics?

We have already mentioned users' difficulty in determining from panel titles or link titles whether they will see declarative or procedural information. We saw users who would say that the panel was on the correct topic, but either did not give them enough declarative information to let them be certain about the meaning of actions or did not allow them to take actions based on what they read. Two solutions to these problems could be investigated. One is to flag help information so that users could tell in advance where they could find declarative or procedural information by moving around in the help hierarchy or through See Also links.

A second system would be to allow users to access either declarative or procedural information directly from the companion panel that supplies the complementary information. Such information could appear in a bubble placed on top of the companion panel or within its own window. Perhaps users could indicate their desire for several movement options, including this one, by right-clicking on the help panel. A list of navigation options could allow users to move within the hierarchy, follow search directions by indicating the most promising content of the See Also links, or similar actions.

These qualitative data from our three waves of data collection for different versions of this programming language have enabled us to see some of the reasons why users succeeded much less than 50 per cent of the time in searching the Help system, even though they frequently knew test tasks sufficiently well to perform them correctly without help. The problems, we found, were not in these users, but in the Help system.

References

Farkas, David K. 1999. "The Logical and Rhetorical Construction of Procedural Discourse." Technical Communication 46, no. 1, 42-54.

Grayling, Trevor. 1998. "Fear and Loathing of the Help Menu: A Usability Test of Online Help." Technical Communication 45, no. 2, 168-179.

Krull, R., Friauf, J., Brown-Grant, J., & Eaton, A. 2002. "What users want from electronic performance support: Results from three waves of qualitative data." Proceedings of the Society for Technical Communication 49th Annual Conference, 311-314.

Krull, R., Friauf, J., Brown-Grant, J., & Eaton, A. 2001. "Usability trends in an online help system: User testing on three releases of help for a visual programming language." IEEE International Professional Communication Conference Proceedings, USA, 19-26.

Pratt, Jean A. 1998. "Where is the instruction in online help systems?" Technical Communication 45, no. 1, 33-37.

Shroyer, Roberta. 2000. "Actual Readers Vs. Implied Readers: Role Conflicts in Office 97." Technical Communication 47, no. 2, 238-240.

Used with permission from STC's 52nd Annual Conference Proceedings, Arlington, VA U.S.A.

Robert Krull is Professor of Communication at Rensselaer Polytechnic Institute. He conducts research on graphics and text for procedural information and teaches courses in documentation, user interface design, and instructional design.

Angela Eaton is an Assistant Professor of Technical Communication and Rhetoric at Texas Tech University. She studies professional and technical communication practice and pedagogy, especially within online environments, and teaches courses in editing and research methods.