Heuristic Evaluation Versus Guideline Reviews: A Tale of Comparing Two Domain Usability Expert's Evaluation Methods

Sehrish Nizamani University of Sindh ; Saad Nizamani University of Sindh ; Nazish Basir University of Sindh ; Gulsher Laghari University of Sindh ; Khalil Khoumbati University of Sindh ; Sarwat Nizamani University of Sindh

Abstract

Background: The usability of university websites is important to ascertain that they serve their intended purpose. Their usability can be evaluated either by testing methods that rely on actual users or by inspection methods that rely on experts for evaluation. Heuristic evaluation and guideline reviews are two inspection methods of usability evaluation. A heuristic evaluation consists of a few general heuristics (rules), which are limited to checking general flaws in the design. A guideline review uses a much larger set of guidelines/suggestions that fit a specific business domain. Literature review: Most of the literature has equated usability studies with testing methods and has given less focus to inspection methods. Moreover, those studies have examined usability in a general sense and not in domain- and culture-specific contexts. Research questions: 1. Do domain- and culture-specific heuristic evaluation and guideline reviews work similarly in evaluating the usability of applications? 2. Which of these methods is better in terms of the nature of evaluation, time needed for evaluation, evaluation procedure, templates adopted, and evaluation results? 3. Which method is better in terms of thoroughness and reliability? Research methodology : This study uses a comparative methodology. The two inspection methods—guideline reviews and heuristic evaluation—are compared in a domain- and the culture-specific context in terms of the nature, time required, approach, templates, and results. Results: The results reflect that both methods identify similar usability issues; however, they differ in terms of the nature, time duration, evaluation procedure, templates, and results of the evaluation. Conclusion: This study contributes by providing insights for practitioners and researchers about the choice of an evaluation method for domain- and culture-specific evaluation of university websites.

Journal
IEEE Transactions on Professional Communication
Published
2022-12-01
DOI
10.1109/tpc.2022.3201732
CompPile
Open Access
Closed
Topics
Export

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (59) · 2 in this index

  1. University websites: The so-so, the bad, and the egregious
  2. 10.1080/15332861.2013.763696
  3. Best university websites of 2017: 5 things they all have in common
  4. Ergonomics of Human-System Interaction Guidance on World Wide Web User Interfaces
  5. 10.26692/sujo/2019.01.27
Show all 59 →
  1. University websites: Top 10 design guidelines
  2. Research-Based Web Design and Usability Guidelines
  3. 10.1353/jhe.2014.0024
  4. University websites
  5. University websites: Top 10 design guidelines
  6. UX of university websites: Usability issues
  7. University websites: Making the grade
  8. University website
  9. IEEE Transactions on Professional Communication
  10. 10.1145/142750.142834
  11. 10.1145/142167.142179
  12. 10.1207/S15327590IJHC1501_13
  13. Practical usability rating by experts (PURE) a pragmatic approach for scoring product usability
    Proc CHI Conf Extended Abstr Human Factors Comput Syst
  14. 10.1111/j.1365-2575.2007.00255.x
  15. 10.1016/j.intcom.2006.10.001
  16. (2019 Jan.). Best Practices in University Website Design, Clique
  17. Research-based web design & usability guidelines
  18. 10.1109/ITNG.2014.81
  19. 10.1080/0144929X.2013.783114
  20. An Exploration of Knowledge Practices Among User Experience (UX) Professionals
  21. 10.1080/10447318.2014.930311
  22. 10.1145/1362550.1362602
  23. 10.1207/s15327590ijhc0903_2
  24. 10.1207/s15327051hci1303_2
  25. 10.2307/2529786
  26. 10.1145/3359174
  27. 10.1207/S15327590IJHC1304_05
  28. A user-centric methodology to establish usability heuristics for specific domains
    Proc Int Conf Ergonom Human Factors
  29. Are two pairs of eyes better than one? A comparison of concurrent think-aloud and co-part…
    J Usability Studies
  30. 10.15388/infedu.2013.16
  31. 10.1177/0961000618773133
  32. 10.1093/jamia/ocw100
  33. 10.1016/j.ijmedinf.2019.05.001
  34. 10.1207/s15327051hci1303_1
  35. 10.1145/259963.260531
  36. 10.4018/978-1-5225-1944-7.ch015
  37. 10.1016/j.csi.2019.01.004
  38. 10.1016/j.csi.2020.103423
  39. 10.1145/1970378.1970382
  40. Ergonomic vs. domain usability of user interfaces
    Proc 6th Int Conf Human Syst Interact
  41. 10.1016/j.infsof.2011.02.007
  42. Domain usability, user's perception
    Human-Computer Systems Interaction Backgrounds and Applications 3  
  43. 10 usability heuristics for user interface design
  44. 10.1080/0144929X.2012.751708
  45. 10.1109/ICADEIS49811.2020.9277343
  46. Human computer interaction course notes
  47. 10.21015/vtcs.v8i2.382
  48. 10.12785/ijcds/040404
  49. 10.1145/259963.260531
  50. 10.1109/ITNG.2013.26
  51. Cultural issues and their relevance in designing usable websites
    International Journal of Innovative Technology and Creative Engineering
  52. Homepage Usability 50 Websites Deconstructed
  53. 10.1080/0144929X.2021.1903080
  54. IEEE Transactions on Professional Communication