IEEE Transactions on Professional Communication Dec 2022

Heuristic Evaluation Versus Guideline Reviews: A Tale of Comparing Two Domain Usability Expert's Evaluation Methods

Sehrish Nizamani University of Sindh ; Saad Nizamani University of Sindh ; Nazish Basir University of Sindh ; Gulsher Laghari University of Sindh ; Khalil Khoumbati University of Sindh ; Sarwat Nizamani University of Sindh

Abstract

Background: The usability of university websites is important to ascertain that they serve their intended purpose. Their usability can be evaluated either by testing methods that rely on actual users or by inspection methods that rely on experts for evaluation. Heuristic evaluation and guideline reviews are two inspection methods of usability evaluation. A heuristic evaluation consists of a few general heuristics (rules), which are limited to checking general flaws in the design. A guideline review uses a much larger set of guidelines/suggestions that fit a specific business domain. Literature review: Most of the literature has equated usability studies with testing methods and has given less focus to inspection methods. Moreover, those studies have examined usability in a general sense and not in domain- and culture-specific contexts. Research questions: 1. Do domain- and culture-specific heuristic evaluation and guideline reviews work similarly in evaluating the usability of applications? 2. Which of these methods is better in terms of the nature of evaluation, time needed for evaluation, evaluation procedure, templates adopted, and evaluation results? 3. Which method is better in terms of thoroughness and reliability? Research methodology : This study uses a comparative methodology. The two inspection methods—guideline reviews and heuristic evaluation—are compared in a domain- and the culture-specific context in terms of the nature, time required, approach, templates, and results. Results: The results reflect that both methods identify similar usability issues; however, they differ in terms of the nature, time duration, evaluation procedure, templates, and results of the evaluation. Conclusion: This study contributes by providing insights for practitioners and researchers about the choice of an evaluation method for domain- and culture-specific evaluation of university websites.

Journal
IEEE Transactions on Professional Communication
Published
2022-12-01
DOI
10.1109/tpc.2022.3201732
CompPile
Open Access
Closed
Topics
assessment technical communication
Export
BibTeX RIS

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (59) · 2 in this index

  1. fullick (2016)
    University websites: The so-so, the bad, and the egregious
  2. 10.1080/15332861.2013.763696
     ↗
  3. (2017)
    Best university websites of 2017: 5 things they all have in common
  4. (2008)
    Ergonomics of Human-System Interaction Guidance on World Wide Web User Interfaces
  5. 10.26692/sujo/2019.01.27
     ↗
Show all 59 →
  1. sherwin (2018)
    University websites: Top 10 design guidelines
  2. shneiderman (2006)
    Research-Based Web Design and Usability Guidelines
  3. 10.1353/jhe.2014.0024
     ↗
  4. sherwin (2016)
    University websites
  5. sherwin (2016)
    University websites: Top 10 design guidelines
  6. lang (2015)
    UX of university websites: Usability issues
  7. (2014)
    University websites: Making the grade
  8. munroe (2018)
    University website
  9. Haaksma et al. (2018)
    Users’ Personal Conceptions of Usability and User Experience of Electronic and …
    IEEE Transactions on Professional Communication
  10. 10.1145/142750.142834
     ↗
  11. 10.1145/142167.142179
     ↗
  12. 10.1207/S15327590IJHC1501_13
     ↗
  13. rohrer (2016)
    Practical usability rating by experts (PURE) a pragmatic approach for scoring product usability
    Proc CHI Conf Extended Abstr Human Factors Comput Syst
  14. 10.1111/j.1365-2575.2007.00255.x
     ↗
  15. 10.1016/j.intcom.2006.10.001
     ↗
  16. facchinello (0)
    (2019 Jan.). Best Practices in University Website Design, Clique
  17. leavitt (2021)
    Research-based web design & usability guidelines
  18. 10.1109/ITNG.2014.81
     ↗
  19. 10.1080/0144929X.2013.783114
     ↗
  20. seo (2021)
    An Exploration of Knowledge Practices Among User Experience (UX) Professionals
  21. 10.1080/10447318.2014.930311
     ↗
  22. 10.1145/1362550.1362602
     ↗
  23. 10.1207/s15327590ijhc0903_2
     ↗
  24. 10.1207/s15327051hci1303_2
     ↗
  25. 10.2307/2529786
     ↗
  26. 10.1145/3359174
     ↗
  27. 10.1207/S15327590IJHC1304_05
     ↗
  28. hermawati (2015)
    A user-centric methodology to establish usability heuristics for specific domains
    Proc Int Conf Ergonom Human Factors
  29. mayhew (2018)
    Are two pairs of eyes better than one? A comparison of concurrent think-aloud and co-part…
    J Usability Studies
  30. 10.15388/infedu.2013.16
     ↗
  31. 10.1177/0961000618773133
     ↗
  32. 10.1093/jamia/ocw100
     ↗
  33. 10.1016/j.ijmedinf.2019.05.001
     ↗
  34. 10.1207/s15327051hci1303_1
     ↗
  35. 10.1145/259963.260531
     ↗
  36. 10.4018/978-1-5225-1944-7.ch015
     ↗
  37. 10.1016/j.csi.2019.01.004
     ↗
  38. 10.1016/j.csi.2020.103423
     ↗
  39. 10.1145/1970378.1970382
     ↗
  40. ba?íková (2013)
    Ergonomic vs. domain usability of user interfaces
    Proc 6th Int Conf Human Syst Interact
  41. 10.1016/j.infsof.2011.02.007
     ↗
  42. ba?íková (2014)
    Domain usability, user's perception
    Human-Computer Systems Interaction Backgrounds and Applications 3  ↗
  43. nielsen (2020)
    10 usability heuristics for user interface design
  44. 10.1080/0144929X.2012.751708
     ↗
  45. 10.1109/ICADEIS49811.2020.9277343
     ↗
  46. andrews (2022)
    Human computer interaction course notes
  47. 10.21015/vtcs.v8i2.382
     ↗
  48. 10.12785/ijcds/040404
     ↗
  49. 10.1145/259963.260531
     ↗
  50. 10.1109/ITNG.2013.26
     ↗
  51. daniel (2011)
    Cultural issues and their relevance in designing usable websites
    International Journal of Innovative Technology and Creative Engineering
  52. nielsen (2001)
    Homepage Usability 50 Websites Deconstructed
  53. 10.1080/0144929X.2021.1903080
     ↗
  54. Heinonen et al. (2021)
    Information Design for Small Screens: Toward Smart Glass Use in Guidance for In…
    IEEE Transactions on Professional Communication
CrossRef global citation count: 6 View in citation network → Build reading path →

Related Articles

  1. Journal of Technical Writing and Communication Mar 2026
    Canon to Code: Rhetorical Rulemaking for Generative AI Content Audits and Governance ↗
    Miriam F. Williams
    classical rhetoric modern rhetorical theory rhetorical criticism assessment technical communication artificial intelligence
  2. IEEE Transactions on Professional Communication Mar 2026
    Surveys as UXR: Using Design Thinking to Shape a Survey-Based UX Assessment for Rural Audiences ↗
    Kaitlin J. Coyle; Derek G. Ross
    assessment technical communication qualitative research literacy studies
  3. Written Communication Jan 2026
    Modeling Writing Processes and Predicting Text Quality in Technical Communication ↗
    Zhijun Gao; Lin Dong; Jiangying Wang
    writing pedagogy assessment technical communication
  4. IEEE Transactions on Professional Communication Jun 2025
    Presenting and Making Relevant: Analyzing Teaching Assistant Perceptions of Writing in Statistics Using Semantic Frames ↗
    Ben Markey
    rhetorical criticism genre theory discourse analysis graduate education teacher development assessment technical communication professional writing
  5. Journal of Business and Technical Communication Jan 2025
    Technical Communication's Fight Against Extractive Large Language Modeling by Applying FAIR and CARE Principles of Data ↗
    Chris A. Lindgren; Erin Yunes; Cana Uluak Itchuaqiyaq
    assessment technical communication artificial intelligence