Is Coding with AI Actually Helpful? A Study Comparing Novices’ Performance, Learning, Fatigue, and Emotions in AI-Assisted vs. Human Pair Programming
Fast and Forgettable: A Controlled Study of Novices' Performance, Learning, Workload, and Emotion in AI-Assisted and Human Pair Programming Paradigms
HCI Today summarized the key points
- •This article reports a study comparing whether novice programmers learn and feel better when working with a human partner versus GitHub Copilot.
- •The study had 22 participants solve a Python task with either a human partner or Copilot for 20 minutes each, then reassessed learning by having them solve it again alone one week later.
- •Overall, scores were higher and workload was lower with Copilot, but participants reported more vivid and positive emotions when working with a human partner.
- •When they took the test again a week later, performance tended to drop slightly under the AI condition, and the learning differences were especially noticeable among the participants who had performed better initially.
- •The researchers concluded that while Copilot provides fast assistance, in educational settings pair programming with a human partner produces greater motivation and enjoyment for learning.
This summary was generated by an AI editor based on HCI expert perspectives.
Why Read This from an HCI Perspective
This article shows AI not as a mere ‘answer generator,’ but as an interaction tool that changes how people work and learn together. While AI may lead in raw performance, it’s important to note that human pair programming can be more advantageous for sustaining learning, shaping emotions, and reducing perceived burden. For HCI/UX practitioners and researchers, it’s a useful case for rethinking the design balance between ‘speed’ and ‘learning,’ and between ‘comfort’ and ‘intervention.’
CIT's Commentary
This study is compelling because it demonstrates that superior AI processing power does not automatically translate into a better user experience. In the short term, Copilot can improve performance, but users may end up thinking less and participating less—potentially leaving shallower traces of learning. In other words, if the interaction is too seamless, an irony emerges: users’ points of intervention can disappear. Especially in systems where safety matters, educational AI must focus not only on ‘accuracy,’ but also on how visible the system state is, when users can step in, and how recovery works when things fail. In the context of domestic services, it seems that designing for the collaboration instincts and feedback density that Korean users are accustomed to may matter more than simply optimizing for faster responses.
Questions to Consider While Reading
- Q.If AI tools can boost users’ performance while weakening learning, what interaction design could help close that gap?
- Q.If we want AI—even partially—to bring emotional engagement and mutual checks, which are strengths of human pair programming, what kind of feedback approach would be needed?
- Q.When introducing tools like Copilot into Korea’s educational and development environments, what user-intervention mechanisms would be necessary to satisfy both learning support and preventing overreliance?
This commentary was generated by an AI editor based on HCI expert perspectives.
Please refer to the original for accurate details.
Subscribe to Newsletter
Get the weekly HCI highlights delivered to your inbox every Friday.