Abstract
Usability testing is essential for improving digital user experiences but has practical limitations in terms of cost-effectiveness. Recent advancements in multimodal Large Language Models (LLMs), like ChatGPT-4, offer new possibilities for UX evaluations. This study investigated the most effective prompt engineering techniques for identifying UX issues in digital interfaces. To achieve this, five prompt engineering techniques were carefully selected based on previous research, and the outputs generated using these techniques were analyzed based on severity assessment criteria. We discovered that Role Prompting and (Zero-Shot) Chain of Thought Prompting were highly effective. Further investigation revealed that a hybrid approach combining both techniques produced the best results. Our findings shed light on the possibility of using multimodal LLM as a UX evaluator, offering meaningful value for future advancements in LLM-based UX evaluations.
| Original language | English |
|---|---|
| Title of host publication | CHI EA 2025 - Extended Abstracts of the 2025 CHI Conference on Human Factors in Computing Systems |
| Publisher | Association for Computing Machinery |
| ISBN (Electronic) | 9798400713958 |
| DOIs | |
| Publication status | Published - 2025 Apr 26 |
| Event | 2025 CHI Conference on Human Factors in Computing Systems, CHI EA 2025 - Yokohama, Japan Duration: 2025 Apr 26 → 2025 May 1 |
Publication series
| Name | Conference on Human Factors in Computing Systems - Proceedings |
|---|
Conference
| Conference | 2025 CHI Conference on Human Factors in Computing Systems, CHI EA 2025 |
|---|---|
| Country/Territory | Japan |
| City | Yokohama |
| Period | 25/4/26 → 25/5/1 |
Bibliographical note
Publisher Copyright:© 2025 Copyright held by the owner/author(s).
All Science Journal Classification (ASJC) codes
- Human-Computer Interaction
- Computer Graphics and Computer-Aided Design
- Software
Fingerprint
Dive into the research topics of 'Can LLMs See What I See? A Study on Five Prompt Engineering Techniques for Evaluating UX on a Shopping Site'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver