Current topics in Interaction Devices, Techniques and Performance Modeling (HMI758)

Lectures

16 h.

Recommended for

HMI doctoral students, but open for all graduate students.

The course was last given:

New course.

Goals

To inform about current research in human computer interaction.

Prerequisites

At least one previous course in HCI, Interactive Systems, or Experimental Psychology, plus one interaction prototyping method (broadly defined, either hardware or software).

Organization

Seminars and individual projects.

Contents

The course consists of three parts: seminars, readings, and a research project.

A. Seminars: students of the course will attend a series of one-hour research seminars, followed by a group discussion session after each seminar. The first seminar hour is open to everyone interested in HCI research.

Seminar 1. ATOMIK (Alphabetically tuned and optimized mobile interface keyboard) - an exploration of virtual keyboard design

Pervasive computing on small and mobile devices is one of the most important trends in the evolution of computing technology. One obstacle to the progress of pervasive comput-ing is text entry. The ATOMIK Keyboard (Alphabetically Tuned and Optimized Mobile Interface Keyboard) is designed to address such an issue (Zhai, Smith, Hunter, 2000, 2001). ATOMIK has three major features. First, it has the highest movement efficiency for stylus typing. This was achieved by a Metropolis optimization algorithm in which the keyboard was treated as a "molecule" and each key as an "atom". The "atomic" interactions among all of the keys drove the movement efficiency - defined by the summation of all Fitts' law movement times between every pair of keys, weighted by the statistical frequency of the corresponding pair of letters in English - towards the minimum. Second, the layout was alphabetically tuned. There is a general tendency that letters from A to Z run from the upper left corner to the lower right corner of the keyboard. This gives novice users a cue to look for letters that are not yet memorized. Third, it maximizes, without sacrificing the first two features, the letter connectivity of the most common words. Many common words or comment fragments of words, such as "the" and "ing" are totally connected. This seminar will present the analysis, design, and user studies on optimized virtual keyboards.

More information can be found at http://www.almaden.ibm.com/u/zhai/topics/virtualkeyboard.htm

Seminar 2. Beyond Fitts' law -human movement regularities and their HCI applications

Fitts' law (Fitts, 1954), a human performance model of movement regularity, has been hailed as one of the great laws in experimental psychology and gained a wide range of applications in human machine interaction (Newell, 1990). This seminar will review a few new human movement laws recently reported by Accot and Zhai (Accot & Zhai, 1997, 1999, 2001, 2002). Going beyond the Fitts' tapping task, we found movement regularities that can be robustly modeled in two new categories of tasks relevant to human computer interaction - boundary crossing and path steering. These new movement laws can be applied to designing and evaluating new user interfaces on a scientific basis.

Seminar 3. Tale of a Mouse -Research and Development of the IBM ScrollPoint

This seminar presents the research, design, and development process of the IBM ScrollPoint^TM, a computer mouse enhanced with a finger-controlled miniature joystick for two-dimensional scrolling (Zhai, Smith, Selker 1997, Zhai, Smith 1999). I will first review the intellectual and commercial history and precursors to the ScrollPoint mouse and describe the user-centered study and analysis based on early prototypes. I will then discuss the real world challenges in turning research into products, including how to maximize user's benefits at (near) zero cost through human performance guided system engineering.

Seminar 4. Eye-gaze based interaction techniques - MAGIC Pointing and EASE Chinese Input

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking has long been considered as an alternative or potentially superior pointing method for computer input. However many fundamental limitations exist with traditional gaze pointing. In particular, it is unnatural to overload a perceptual channel such as vision with a motor control task. We therefore propose an alternative approach, dubbed MAGIC (Manual And Gaze Input Cascaded) pointing (Zhai, Morimoto, Ihde, 1999). With such an approach, pointing appears to the user to be a manual task, used for fine manipulation and selection. However, a large portion of the cursor movement is eliminated by warping the cursor to the eye gaze area, which encompasses the target. Our early-stage exploration showed that the MAGIC pointing techniques might offer many advantages, including reduced physical effort and fatigue as compared to traditional manual pointing and greater accuracy and naturalness than traditional gaze pointing.

Similar to MAGIC pointing, the EASE (eye assisted selection and entry) Chinese input system helps users selecting multiple homophonic Chinese characters by using a common selection key and implicit eye-tracking (Wang, Zhai, Su, 2001).

Seminar 5. The Partial Occlusion Effect: Utilizing Semi-transparency in 3D Human-Computer Interaction

This study investigates user performance when using semi-transparent tools in interactive 3D computer graphics environments. We hypothesize that when the user moves a semi-transparent surface in a 3D graphic display, the partial occlusion effect introduced through semi-transparency acts as an effective cue in target localization -- an essential component in many 3D manipulation tasks. This hypothesis was tested in a controlled experiment in which subjects were asked to acquire dynamic 3D targets (virtual fish) with a 3D "Silk cursor" (Zhai, Buxton, Milgram, 1996, 1994). In the experiment, cursors with and without semi-transparent surfaces were compared in monoscopic and stereoscopic displays. Results show the partial occlusion cue was effectively used by subjects in both monoscopic and stereoscopic displays and it was no less effective than stereopsis for successful 3D target acquisition.

Seminar 6. Control, Navigation, and Coordination in 3D user interfaces

This seminar reviews a series of experimental studies on user performance as a function of various input control devices properties for manipulation (Zhai & Milgram 1993, Zhai 1995, Zhai, Milgram, Buxton 1996) and navigation (Zhai, Kandogan, Smith, Selker 1999). A measure of quantifying coordination and its application to evaluating 6 DOF input devices will also be presented.

B. Readings: students are expected to read research papers related to the seminars (key readings with *)

Seminar 1. Virtual Keyboards

1.* Zhai, S., M. Hunter, and B.A. Smith, Performance Optimization of Virtual Keyboards. Human-Computer Interaction, 2002. in press.

2. Zhai, S., M. Hunter, and B.A. Smith. The Metropolis Keyboard - an exploration of quantitative techniques for virtual keyboard design. in Proceedings of The 13th Annual ACM Symposium on User Interface Software and Technology (UIST). 2000. San Diego, California: ACM. p. 119-218.

3.*MacKenzie, I.S. and S.X. Zhang. The design and evaluation of a high-performance soft keyboard. in Proceedings of CHI'99: ACM Conference on Human Factors in Computing Systems. 1999. p. 25-31.

4.Smith, B.A. and S. Zhai. Optimised Virtual Keyboards with and without Alphabetical Ordering - A Novice User Study. in Proceedings of INTERACT'2001 - IFIP International Conference on Human-Computer Interaction. 2001. Tokyo, Japan: IFIP. p. 92-99.

Seminar 2. Fitts' law and performance modelling

5.* Fitts, P.M., The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 1954. 47: p. 381-391.

6.MacKenzie, I.S., Fitts' law as a research and design tool in human computer interaction. Human Computer Interaction, 1992. 7: p. 91-139.

7.*Accot, J. and S. Zhai. Beyond Fitts' Law: Models for Trajectory-Based HCI Tasks,. In Proceedings of Proceedings of CHI'97: ACM Conference on Human Factors in Computing Systems. 1997. p. p295-302.

8.Accot, J. and S. Zhai. Performance Evaluation of Input Devices in Trajectory-based Tasks: An Application of Steering Law. in Proceedings of Proceedings of CHI'99: ACM Conference on Human Factors in Computing Systems. 1999. p. 466-472.

9.Accot, J. and S. Zhai. Scale effects in steering law tasks. in Proceedings of Proc. of CHI 2001: ACM Conference on Human Factors in Computing Systems. 2001. p. 1-8.

Seminar 3. Mouse - Dual Input

10.*Zhai, S. and B.A. Smith, An Experimental Study of Document Scrolling Methods. IBM Systems Journal,, 1999. Vol 38(4): p. 642-651

11.*Buxton, W. and B.A. Myers. A study in two-handed input. in Proceedings of CHI'86: ACM Conference on Human Factors in Computing Systems. 1986. p. 321-326

12.Bier, E.A., et al. Toolglass and magic lenses: the see-through interface. in Proceedings of SIGGRAPH 93. 1993.

13.Leganchuk, A., S. Zhai, and W. Buxton, Manual and Cognitive Benefits of Two-handed Input: An Experimental Study,. ACM Transactions on Computer-Human Interaction, 1998. Vol 5(No. 4): p. 326-359.

Seminar 4. Eye tracking

14.*Zhai, S., C. Morimoto, and S. Ihde. Manual And Gaze Input Cascaded (MAGIC) Pointing. in Proceedings of CHI'99: ACM Conference on Human Factors in Computing Systems. 1999: ACM Press. p. 246-253

15.Wang, J., S. Zhai, and H. Su. Chinese Input with Keyboard and Eye Tracking - An Anatomical Study. in Proceedings of Proc. of CHI 2001 - ACM Conference on Human Factors in Computing Systems. 2001. p. 349-356

16.*Jacob, R.J.K. What You Look At is What You Get: Eye Movement-Based Interaction Techniques. in Proceedings of CHI'90: ACM Conference on Human Factors in Computing Systems. 1990: Addison-Wesley/ACM Press. p. 11-18

Seminar 5. Semi-transparency

17.*Zhai, S., B. Buxton, and P. Milgram, The partial occlusion effect: utilizing semi-transparency in 3D human-computer interaction. ACM Transactions on Computer-Human Interaction, 1996. 3(3): p. 254-284.

Seminar 6. 3D control and coordination

18.*Zhai, S., User Performance in Relation to 3D Input Device Design, Computer Graphics. Computer Graphics, 1998. 32(4): p. 50-54.

19.*Zhai, S., Milgram, P. Quantifying coordination in multiple DOF movement and its application to evaluating 6 DOF input devices. in Proceedings of CHI'98: the ACM Conference on Human Factors in Computing Systems. 1998. p. 320-327

General Reading (not required but recommended)

20.Wickens, C.D., Engineering Psychology and Human Performance. 1992: HarperCollines Publishers. (Or the revised edition by Wickens and Holland)

C. Projects

Students are required to do one project that produces a report that clearly documents what was accomplished. A list of sample topics is as follows. Actual project is mutually decided between the student and the instructor. The topic can also come from students' own thesis related research, as long as it has new material produced as a result of taking this course. Emphasis is placed on innovative ideas and methodological validity. Students are encouraged to interact with the instructor during their projects. The number of credit given depends on the number of weeks the project needs to take.

Sample Course Projects: the following open problems are candidate projects.

1. Mobile phone input - the effects of color and spatial coding
2. Visual feedback for scrolling and paging
3. Directional effects in steering law tasks
4. Fitts' law in multiple sequential targets acquisition
5. Crossing interfaces in 3D / VR
6. Steering law in 3D / VR (driving, airplane landing, circular vs. straight path)
7. Visual / texture appearance of virtual keys
8. Hits distribution as a function of the target shape
9. Pointing, Crossing and Gesture - Crossing as a bridge from pointing to gesture.
10. Tracing vs. steering, spatial vs. temporal constraints in motor
11. Selector-executor model of physical control devices.

Literature

To be announced.

Teachers

Dr. Shumin Zhai, Ph.D. Guest Professor.

Examiner

Shumin Zhai / Sture Hägglund.

Schedule

Spring 2002 (Late January to April. Seminars will be concentrated in the early part of the semester. Projects and class discussion continue till late April ).

Examination

Project and paper.

Credit

5 p expected, actual number of credits will vary according to project size.

Comment

Currently a guest professor at KTH and Linköping University, Dr. Shumin Zhai is on sabbatical from the IBM Almaden Research Center, San Jose, California, where he leads research and development of innovative input devices, interaction techniques, and theoretical modeling of human computer interaction performance. He has led the R&D of the IBM ScrollPoint Mouse; co-authored the "Steering law" (with Johnny Accot); received 7 US patents; and published over 60 papers in the area of Human Computer Interaction, Human Factors, and Automatic Control Systems. He is an associate editor of ACM Transactions on Computer Human Interaction, and a member of editorial boards of Interacting with Computers, and Personal and Ubiquitous Computing. More information and his publications can be found at
http://www.almaden.ibm.com/u/zhai/