Objectives of the study
The purpose of the study was to explore the effect of communicating orienta-tional information using lexivisual principles and techniques.
Description of the test systems
The systems deal with basic facts about our solar system. Both systems are implemented in HyperCard 1.2.5, and contain about 60 cards each. Retro-spectively, it is clear that a larger database would have been desirable. The small size of the database is due to practical limitations. However, the principles employed here might be applicable in larger systems, given certain conditions. This is discussed at the end of this chapter.
The first version of the system is a 'conventional' HyperCard design where information is presented on separate cards. The second is an enhanced version, where orientational information have been added. The factual information in both systems are identical. The only aspects altered are related to orientational information, and to browsing and navigation.
The basic structure of the system is described in figure 6.1. The first part of the system is a short introductory sequence. The base card shows an overall view of the solar system. From this card it is possible to go to all planet cards. It is also possible to go to a magnified view of the inner planets of the solar system (these are: Mercury, Venus, Earth and Mars). From this card one can go directly to a detailed view of a particular planet. Each planet card contains additional infor-ma-tion. There are three subtopics for each planet. These topics vary from planet to planet.
Orientational features of the enhanced system
In the enhanced design the lexivisual principle of relating detailed information to its context and integrating related information have been used. In summary, the main orientational features used in the enhanced design are:
Figure 6.1: The basic structure of the experimental system. In the original version it is possible to move around by following the links as indicated in the picture. From the main view of the inner planets one can go to the Sun, Mercury, Venus, the Earth, and Mars, In the enhanced version it is also possible to jump directly between the subtopics of a planet and directly from a subtopic to any other planet. Figure 6.11 should clarify this point.
The small maps show to which context detailed information belongs, and where in the overall structure the user currently is. Figure 6.9 gives an example of how this kind of map is used. Having the picture of a planet visible when displaying detailed information serves as a reminder of which context one is in (see figure 6.11). The idea is to make the presentation less fragmented by presenting information in relation to the whole, instead of presenting it in isolation.
An interesting observation is that providing orientational information in the form of active maps can alter the way the user can browse through the system. When viewing detailed informa-tion in the enhanced design (see figure 6.11), it is possible to go directly to any other subtopic, to any other planet, or return to the solar system. In the original design (see figure 6.10) this is not possible. There the only option is to leave the current card.
In figures 6.2 to 6.11 on the following pages, ideas and principles underlying the two designs are illustrated. It should be pointed out that the systems were designed for experimental purposes only. The facts contained in the systems have been chosen in an rather ad hoc fashion. It should also be made clear that neither the original system, nor the enhanced, are intended to illustrate good graphical design.
A snapshot from the introduction sequence
Figure 6.2: Original design. This is the first card which is presented to the user. It is not possible to see how long the introduction sequence is, nor is it possible to see where in the sequence one currently is. Clicking on the small arrow goes to the next card in the sequence.
Figure 6.3: Enhanced design. Note the small strip of frames at the bottom of the screen which shows how many cards there are in the introductory sequence. The present location is indicated by a black square. Navigation is accomplished by clicking at the small arrow next to the strip. It is also possible to move to a card by clicking at the corresponding frame directly.
Figure 6.4: Original design. This is the base card in the system. From here it is possible to go to any planet card by clicking, either directly on the planet, or on the name tag of the planet. The triangular up arrow returns to the introductory sequence.
Figure 6.5: Enhanced design. The miniaturized picture of the galaxy gives contextual information and shows the location of the solar system. By clicking on this picture one gets back to the initial sequence of cards.
Detailed view of the inner planets
Figure 6.6: Original design. This card shows an enlarged view of the inner planets. It is possible to go to a planet card by clicking directly on the planet or its name. Clicking on the up arrow returns to the solar system.
Figure 6.7: Enhanced design. Contextual information is provided by a miniaturized map of the solar system in the upper left corner of the screen. The small frame in the map indicates the present location. Clicking at a planet in the map displays goes to the corresponding card, clicking outside the planets returns the user to the main view of the solar system.
Figure 6.8: Original design. This picture of the planet Venus is not related to its context, the solar system. This makes it hard to know how the planet is related to the whole and to other planets. The user has to remember the context in order not to get lost. The left and right arrow goes to the previous and next planet. The up arrow returns to the main view of the solar system.
Figure 6.9: Enhanced design. Here world vision is provided by the miniaturized map of the solar system which shows the context. A small arrow shows the present location. It is possible to go to another planet in the map by clicking on it. Clicking somewhere in the empty space around the planets returns to the main view of the solar system.
Detailed information on a planet
Figure 6.10: Original design. Detailed information is presented on a new screen, in isolation from the picture of the planet. Here it is not possible to see where one is in the structure. Thus, it is more difficult to orient oneself. Clicking the button 'Hide Calvin' returns to the view of the planet.
Figure 6.11: Enhanced design. Detailed information is presented in the context of the planet and the map of the solar system. The background picture serves as a landmark. This makes it easier to recognize where one is. It is possible to move around freely by clicking on the planets in the map and on the other subjects on the planet. This makes the system less modal than the design in figure 6.10. The purpose of the cartoon character is to make the presentation more interesting, appealing and entertaining.
Evaluation procedures
One important objective of the case study was to gather ideas and suggestions proposed by the subjects. Therefore an explorative approach using qualitative, rather than quantitative, methods, has been used. Data were obtained through discussions and interviews with the subjects in conjunction with questionnaires.
In total, 18 subjects have been studied. The subjects ranged in age from 25 to 50 years. All subject were employees of the Department of Computer Science at the University of Stockholm. Their previous experience with computers vary from experienced to very experienced. All subjects have used both versions of the system. Each version was used for about 15 minutes.
A problem with this experimental set-up is that the subject tends to be biased when using the second system, since she is then more acquainted with the subject area and the general structure of the system. Therefore the order in which the systems were used was balanced; nine subjects started with the enhanced system, and nine subjects used the original system first.
All subjects were given a short introduction before they used the systems. They were informed that they were to compare two versions of a hypermedia system about the solar system, with respect to orientation and overview. The subjects were not told which system was intended to be the 'good' one. The basic prin-ciples of how to use the system were described briefly, such as where it was possible to click and the meaning of different elements on the screen. Conven-tions for screen layout and the use of buttons were also described.
The subjects were not given any predefined tasks. They were told to browse through the system at will. However, the subjects were occasionally told to find some specified information. These questions varied from time to time depending on the situation. Some examples are:
- "Go back to the solar system"
- "Go and find information on the atmosphere of Mars"
- "Go to Neptune"
During and after the experiment subjects were also asked different questions:
- "Do you know where you are?"
- "Do you feel that you have a good overall view of the system?"
- "Did you feel lost when using the system?"
- "Did the miniaturized pictures help orientation?"
- "Did you ever feel stuck or unsure on how to proceed?"
- "What did you find most confusing?"
- "Which system was the best with respect to orientation?"
Furthermore, statements made by the subjects and questions spontaneously asked were noted.
In addition to questions and observations, I used questionnaires where subjects rated different aspects concerning orientation. The results from these questionnaires are presented in the next section.
Results from the questionnaires
All 18 subjects filled in a questionnaire for each system after having used both versions. The questions ranked different aspects related to orientation on a lickert scale (Sweeney & Dillon 1987) ranging from 1 to 10. The original design ranked an overall average of 6.4, and the enhanced design ranked an overall average of 8.2. Though the number of subjects is limited these figures support observations made during the experiment. A summary of the results from the questionnaires is found in figures 6.12 to 6.15. The figures show the results from ratings made by the subjects. Each figure show the result from one of the four questions.
Figure 6.12. Question: "The overview was ..." (1 = poor, 10 = good). The average for the enhanced version was 8.6, and the average for the original version was 6.7.
Figure 6.13. Question: "I knew where I was ..." (1 = never, 10 = always). The average for the enhanced version was 8.5, and the average for the original version was 6.7.
Figure 6.14. Question: "I felt confident i what information was available ..." (1 = no, 10 = yes). The average for the enhanced version was 7.0, and the average for the original version was 5.3. This aspect ranked relatively low for both versions. This might indicate that people find it difficult to grasp the contents of a hypermedia system.
Figure 6.15. Question: "Finding information was ..." (1 = hard, 10 = easy). The average for the enhanced version was 8.7, and the average for the original version was 6.9.
Orientation and overview was better in the enhanced design
Almost all subjects agreed that the overview was better in the enhanced design, and that it was more difficult to know where one was in the original design. They stated that the miniaturized map was the critical element, and that it signi-fi-cantly helped orientation. This map was liked by all subjects. While nearly all subjects thought that the map significantly improved orientation and gave a better overview of the system, some liked the clear use of buttons and less complex layout of the original version.
Having the picture of the planet and the map visible when viewing detailed information about a planet (see figure 6.11) was liked by most subjects. Many subjects said that the picture of the planet in the background make it easier to know where you are. However, this design also caused confusion. A few subjects thought it was distracting, and that it was more difficult to find and to read information in the enhanced version than in the original one, due to the more complex layout. These problems may be caused by the graphical design, not by the principle of showing the context as such.
Some subjects did not feel very confident about which information was avail-able, not even when using the enhanced design. One user stated that "It is impossible to know how much information there is until you have seen it all". In the questionnaires the question related to this topic ranked a lower average than the other questions. This confirms one of the initial assumptions that hyper-media is more difficult to grasp than, for instance, printed media, and that tech-niques for communicating the amount of information available are necessary.
The network structure was perceived as flat
Several persons said that the enhanced system was easier to navigate since they perceived its structure as flatter than that of the original system. Having to explicitly return to the next higher level in order to proceed, as in the original version, was considered as irritating.
It is interesting to notice that the structure of the enhanced version is a rather complex network, which is not especially flat. However, since it is possible to browse with a greater degree of freedom than is possible in the original design using the miniaturized maps, the subjects seemed to think of the system as having a flat structure. One subject pointed out that the enhanced system felt less modal than the original one.
Edward & Hardman (1989) have reported that a mixed structure (hierarchy and network) can confuse users, and that hierarchical structures are preferable. By contrast, I found that most subjects was not confused by the mixed/network structure and that they liked the ability to move around more freely. Even though this experiment is not directly comparable to that of Edward & Hardman, a possible explanation of the differences is the high degree of visualization in the enhanced version, which makes it easy both to see where one currently is and to move around in the structure.
A potentially confusing issue concerns the use of the miniaturized map. Since it is possible both to go to another planet (by clicking on it) or to go back to the main view of the solar system (by clicking outside any planets), some confusion concerning the function of the map occurred. The obvious use for the map was to go to another planet. The function of returning to the solar system was much more implicit and hard to understand, even after it was explained.
An interesting observation is that in the enhanced version people seldom returned to the main view of the solar system. Instead they used the map to go to another planet. This might be because the did not find it natural to use the map for returning to the main view of the solar system. On the other hand it is quicker to go directly to a planet using the map than going via the main view.
It is difficult to know where to click
A general problem, which is related to browsing and navigation functions, is seems to be that it is hard to know which parts of a picture are clickable. A majority of the subjects complained about difficulties to know where to click, even though the use of different buttons was explained in the introduc-tion. One subject said that it would have been very difficult to know how to use the systems (both versions) without the introduction.
The subjects complained about that it was particularly difficult to know where to click in the enhanced version. The original version was appreciated for its clear, explicit buttons. Some of these buttons were left out in the enhanced design. This was a consequence from introducing clickable pictures, like the miniaturi-zed map, instead of buttons. When designing the enhanced version, it was not expected that this would cause any problems. Visualizing clickable areas seems to be of importance, at least if the user is not familiar with the system.
Some subjects seemed to find it natural to click at different parts of the picture. In the introduction sequence of the enhanced version several subjects did not use the arrows, but clicked directly at the small strip of frames (see figure 6.3). This strip was liked by most subjects, but one person stated that it would be difficult to use this technique for larger systems. Another subject thought that the strip should have been used throughout the whole system, and not only in the introduction sequence.
Other subjects tried to click on different parts of the picture of a planet. One person tried to click at the rings of Saturn. Some subjects said they would have liked this functionality implemented as an alternative to explicit buttons. Previous experience and acquaintance with hypermedia and HyperCard clearly influence how well subjects perform. Even though no data were collected on this, subjects which were familiar with HyperCard performed better than the others. However, in some cases knowledgeable HyperCard users where distracted by the absence of conventional buttons in the enhanced design.
Another aspect related to the use of buttons is that visually explicit buttons might sometimes clutter the presentation and mess up a picture. A solution to this problem is to have an option which allows for displaying clickable areas at the choice of the user. This feature is available in HyperCard, but was deliberately not used during the experiments.
Additional comments
One subject commented that visually oriented persons and text oriented persons probably would prefer different designs. The same subject also said that the task is very central to how the system should be designed.
Initially, the size of the system (60 cards) was considered to be sufficient, but in practice it was too small to cause any real problems for the users. One subject, who very much preferred the enhanced design, stated that the original design would be even worse in a larger example. Even though the test system is small, the principles suggested here might applicable to larger scale systems as well (see discussion below).
It was pointed out by one subject that a sense of visual consistency and stability is important for recognizing the present location. In the enhanced system the picture of a planet served as a stable point of reference for all information on that planet. In the original version, by contrast, the change of background was considered confusing (see figure 6.10).
A couple of subjects complained about the fact that the introduction was sepa-rate, and not so strongly related to the rest of the system. They thought that the introduction should be better integrated with the rest of the system.
The system was also tested the system with an 11-year old girl. She said that the enhanced system was better, and that the map made it easier to use. She com-plained about explicitly having to step back to the next higher level in the original version. In the enhanced version she liked the ability to use the map to go directly to another planet.
Summary of results
There are strong indications that visualized orientational information is considered as positive, and that it can help orientation.
The major advantage with the enhanced design was the overview and location indicator provided by the miniaturized map. The possibility to navigate with greater freedom was also appreciated. Using the miniaturized map for moving around the system seemed important for the usability of the system. It is interesting to make a comparison to a system with 'passive' maps.
Having the background picture visible while viewing detailed information was liked by some subjects, and disliked by others who thought it was distracting.
The biggest problems with the enhanced design were difficulties about knowing where to click, and minor confusion due to the somewhat cluttered graphical design and layout.
The original version was liked for its clear buttons. Major flaws of the original design was that is difficult to know where one is, and that it is time consuming to traverse the hierarchy.
All subjects but one stated that the enhanced system was considerably better than the original one. The one subject disliking the enhanced version complained about unclear buttons. Nearly all subjects said that they felt much more safe and comfortable with the enhanced design than with the original one.
Results from an independent study
In addition to the study described above, an independent study has been conducted at the department of Psychology at the University of Stockholm (Wikström 1990). This study compared exactly the same versions of the hypermedia system studied here. The major difference from this study is that more formal, quantitative evaluation methods have been used.
In summary this study reports that:
On the whole, Wikström's study confirms the results from this study. It is interesting to note the subjective preference for the enhanced version, even though the system was too small to show any significant difference in quantitative performance.
Applicability to large systems
As mentioned earlier, the two systems tested here use a database which is rather small. It can be discussed wether orientation techniques employed by the enhanced system are applicable to large systems too.
One problem is that it could be difficult to provide a miniaturized overview if the contents is very large. It might be practically impossible to fit all necessary information on one map. However, if the material can be structured into a modularized hierarchy, then principles discussed here could be applied to each sub-tree of the hierarchy. While an hierarchical structure can be a restriction in many cases, a simplification of a complex network structure might be beneficial for the user.
Another approach to save screen space is to use views based on the fish eye principle. That is, information on the highest level is shown in a less detailed way than the information currently in focus. In many cases it might be sufficient to see the higher levels of the system on an overall scale only. If you are walking around in the center of Stockholm, for instance, you do not need to have detailed know-ledge of the suburbs, even less of other cities in Sweden. However, if you want to go tho some place in the suburbs you must have sufficient knowledge on how to get there.
Yet another approach to cope with the problem of the miniaturized maps taking up too much screen space, is to use a technique similar to pull-down menus in the Apple Macintosh user interface. These menus have the interesting property of giving an overview of the functions by having an ever present menu bar. At the same time it is possible to quickly browse trough the menus using the mouse. A similar principle could be used for miniaturized maps; clicking an item in the map instantly pops up a sub-map which gives a detailed view.
Since these assumptions have not been tested, further research is necessary.