Hide menu

729A99 Usability Testing

Literature


All course literature is available free online if accessed via Liu university network (if you are not on the local network, go to the library page to log in).

Adaption to distance education: Hints on remote user testing

Note:As long as the pandemic situation permits, you may choose to do your tests either remote or on location, or a mix (the last is educational for the comparison). Remote testing qualifies as use of a technical tool in the requirements list on the course info page.

Performing a moderated user test of a website or computer app can actually be done using Teams or Zoom or any similar tool where you can share the screen that the user sees and listen to the subject thinking aloud. Especially tools that permit recording of the sessions. Zoom does (i believe) enable you to also see the user's face, and (if close enough) note the most obvious facial expressions. Directions and user tasks can be transmitted using the chat function and starting a recording with the subject reading the informed consent form aloud and verbally agreeing to it constitutes legally valid (as far as I understand) replacement for the written consent form.
Some more helpful tips and tricks can be found here: Steven Carr on Remote user testing and here: Bryan Kern on Remote moderated user testing.

Books

  • Jeffrey Rubin and Dana Chisnell Handbook of Usability Testing, How to Plan, Design and Conduct Effective Tests, second edition, Wiley, 2008. Available as E-book via LIU library. Called HUT in this course).
  • Tom Tullis and Bill Albert Measuring the User Experience: Collecting, Analyzing and Presenting Usability Metrics. Second edition. Morgan Kaufmann, 2013. Available as E-book from Liu library (Called MTUE in this course).
  • Joseph S. Dumas and Beth A. Loring Moderating Usability Tests: Principles and Practices for Interacting, Morgan Kaufmann, 2008. Available as E-book from Liu library (Called MUT in this course).

Papers

  • Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24, 574-594. Online here
  • Bevan, N., Carter, J., Earthy, J., Geis, T. and Harker, S. (2016) New ISO Standards for Usability, Usability Reports and Usability Measures. In proceedings of HCI 2016, pp 268-278. DOI:10.1007/978-3-319-39510-4_25. Online here
  • Brockmyer, J. H., Fox, C. M., Curtiss, K. A., McBroom, E., Burkhart, K. M., & Pidruzny, J. N. (2009). The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing. Journal of Experimental Social Psychology, 45(4), 624-634. Online here
  • Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A. and Jernigan, W. (2016). GenderMag: A method for evaluating a software's gender inclusiveness, Interacting with Computers, 2016Online here.
  • Finstad, K. (2010) The Usability Metric for user Experience, Interacting with Computers, Volume 22, Issue 5, September 2010, Pages 323-327. Online here
  • Folstad, A., Law, E. & Hornbaek, K. (2012) Analysis in Practical Usability Evaluation: A Survey Study. In Proceedings of CHI, 2012. Online here
  • Frandsen-Thorlacius, O.,Hornbaek, K., Hertzum, M. & Clemmensen, T. (2009) Non-universal usability? a survey of how usability is understood by Chinese and Danish users. Proceedings CHI 09, pp 41-50, DOI=10.1145/1518701.1518708 Online here
  • Franz, R., Munteanu, C., Barbosa-Neves, B. & Baecker, R. (2015) Time to retire old methodologies? Reflecting on conducting usability evaluations with older adults. MobileHCI-15, ACM. pp 912-915. Online here
  • Fu, F. L., Su, R. C., & Yu, S. C. (2009). EGameFlow: A scale to measure learners enjoyment of e-learning games. Computers & Education, 52(1), 101-112. Online here
  • Georgiou, Y., & Kyza, E. A. (2017). The development and validation of the ARI questionnaire: An instrument for measuring immersion in location-based augmented reality settings. International Journal of Human-Computer Studies, 98, 24-37. Online here
  • Grossman, T., Fitzmaurice, G. & Attar, R. (2009) A Survey of Software Learnability: Metrics, Methodologies and Guidelines. Proceedings of CHI 2009, pp. 649-658. Online here
  • Hill, R., Dickinson, A., Arnott, J., Gregor, P. & McIver, L. (2011) Older Web Users Eye Movements: Experience Counts. Proceedings of CHI-2011, pp. 1151-1160. Online here
  • Jonsdottir Johannessen, G. H. & Hornbaek, K., (2014) "Must evaluation methods be about usability? Devising and assessing the utility inspection method", Behaviour & Information Technology, 33(2), pp. 195-206, 2014. Online here
  • Lewis, J.R. (2018) The System Usability Scale: Past, Present, and Future, International Journal of Human Computer Interaction, DOI: 10.1080/10447318.2018.1455307 Online here.
  • Lindgaard, G. & Chattratichart, J. (2007) "Usability Testing: What Have We Overlooked?", in Proceedings of CHI 2007, pp. 1415-1424, 2007. Online here.
  • Melhart, D., Azadvar, A., Canossa, A., Liapis, A., Yannakakis, G. (2019) "Your Gameplay Says It All: Modelling Motivation in Tom Clancy's The Division", in Proceedings of IEEE Conference on Games (COG) 2019, Osaka, Japan. Online here.
  • Miao, M., Pham, H. A., Friebe, J. & Weber, G. (2016) Contrasting usability evaluation methods with blind users. Universal Access in the Information Society, March 2016, vol 15, no 1, pp 63-76. Springer. Online here
  • Olmsted-Hawala, E. L., Murphy, E. D., Hawala, S. & Ashenfelter, K. T. (2010) Think-aloud protocols: a comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. Proc. CHI'2010. Online here
  • Oyugi, C., Abdelnour-Nocera,J. & Clemmensen, T. (2014) Harambee: a novel Usability Evaluation Method for Low-End Users in Kenya. Presented at NordiCHI-14, proceedings of 8th nordic conference on human-computer interaction: fun, fast, foundational. pp 179-188 Online here
  • Ragnemalm, E.L, Bang, M. & Alin-Nilsson, I. (2011) Heuristic Evaluation of persuasive systems: the case of educational programs. Presented at The sixth International Conference on Persuasive Technology (PERSUASIVE 2011), Columbus, Ohio, USA, june 2-5 2011. Online here
  • Rounding, K., Tee, K., Wu, X., Guo, C. & Tse, E.(2013) Evaluating Interfaces with children. Personal and Ubiquitous Computing, December 2013, vol 17, no 8, pp1663-1666. Springer. Online here
  • Sauro, J. (2016) The challenges and Opportunities of Measuring the User Experience. Online here.
  • Sauro, J. & Lewis, J.R. (2011) When Designing Usability Questionnaires, Does It Hurt to Be Positive? Proceedings of CHI-2011, pp. 2215-2223. Online here
  • Sim, G. & Horton, M. (2012) Investigating children's opinions of games: Fun toolkit vs This or That. In proceedings of IDC2012, 12-15th June, Bremen, Germany. pp 70-77. Online here
  • Sonderegger, A. & Sauer, J. (2013) The influence of socio-cultural background and product value in usability testing. I Applied Ergonomics, vol 44, no 3, May 2013, pp 341-349. Online here
  • Tuch, A., Trusell, R. & Hornbaek, K. (2013) "Analyzing Users' Narratives to Understand Experience with Interactive Products", in Proceedings of CHI 2013, pp. 2079-2088, 2013. Online here
  • Wiemeyer J., Nacke L., Moser C., Floyd Mueller F. (2016) Player Experience. In: Dörner R., Göbel S., Effelsberg W., Wiemeyer J. (eds) Serious Games pp 243-271. Springer, Cham. Online here
  • Åsberg, R., Hummerdal, D. & Dekker, S. (2011). There are no qualitative methods nor quantitative for that matter: the misleading rhetoric of the qualitative quantitative argument. Theoretical Issues in Ergonomics Science, 12:5, pp. 408-415. Online here

Page responsible: Eva Ragnemalm
Last updated: 2022-01-07