Talk2AI: A Longitudinal Dataset of Human--AI Persuasive Conversations

Why Read This from an HCI Perspective

This article is important for HCI researchers and UX practitioners because it treats LLMs not just as text generators, but as conversational interfaces that can change people’s thoughts and attitudes. In particular, since it captures not a single interaction but exchanges over multiple weeks, it allows us to examine issues such as trust formation, persuasion, and points of intervention along a time axis. It also prompts product teams to think about what actually moves people when adding AI.

CIT's Commentary

What’s interesting is that this study makes it much clearer how a conversation shapes a person’s experience than it does how to build an AI that produces ‘good answers.’ Even with the same model, persuasion effects can vary depending on the topic, an individual’s traits, and repeated interaction patterns—suggesting that simply improving model performance in real services does not automatically guarantee safe or desirable interactions. In particular, interface mechanisms such as prompting questions, encouraging rebuttals, and limiting message length can significantly change user behavior, so products need to evaluate whether these mechanisms help guide users or instead intensify manipulation. The study’s measurement of how trust and conviction evolve across repeated conversations also offers hints for designing more rigorous LLM-based UX evaluations. For example, even if you use an LLM to assist with analyzing survey responses or conversation patterns, you still need to separately validate the tool’s own biases and the consistency of its measurements.

Questions to Consider While Reading

• Q.In repeated conversations, how can we separate and verify whether the factors that increase persuasion effectiveness come from the model’s response quality or from the interface’s prompting and guidance methods?
• Q.What interaction signals could distinguish the points where users perceive AI persuasion as ‘help’ versus where they perceive it as ‘manipulation’?
• Q.If we apply this dataset to Korean services from Naver, Kakao, and startups, in which topics and user groups would the results differ the most?