Can a ‘Video-Based Conversational Chatbot’ Survey Really Work? A Pilot Study in New York City

Why Read This from an HCI Perspective

This article is especially meaningful for HCI/UX practitioners and researchers because it treats AI not as a mere automated response tool, but as an interface that elicits people’s lived experiences. In particular, the attempt to combine video with a conversational chatbot to transform surveys that rely on memory into something closer to an in-the-moment experience is an intriguing direction that helps address limitations of existing measurement approaches. Looking at both user experience and data quality also makes the work directly relevant to practice.

CIT's Commentary

The core value of this study lies in interaction design rather than model performance. The flow—watching video, making an immediate judgment, and then explaining why—serves as a strong mechanism for externalizing the sensations users already have in their minds. At the same time, it clearly reveals a trade-off: as repeated responses increase, fatigue builds and drop-off grows. That’s why, for tools like this, what matters is not simply asking more questions, but deciding when to stop, what to ask, and how to leave an intervention path. In mobility environments where safety is critical, it’s especially important how transparently the system state is presented, and how far users can trust and correct the AI’s judgments. Moreover, using LLMs not only as response generators but as part of UX measurement tools is a practical approach that expands the research methodology itself. In Korea, where contexts are often familiar with short, fast interactions—such as Naver, Kakao, and public survey environments—this kind of design may require even more concise and clearly defined feedback loops.

Questions to Consider While Reading

• Q.In a video-based chatbot survey, how should the number of questions and the way context is presented be adjusted to reduce user fatigue while still collecting enough narrative data?
• Q.When the AI presents explanations first in safety-related judgments, how can we reduce the bias that pulls users toward those explanations?
• Q.If this approach were applied to Korean mobility services or local-government complaint collection, which interaction elements would need to be localized first?