Building AI Agents to Resist Prompt Injection

Why Read This from an HCI Perspective

This article is important for HCI practitioners and researchers because it argues that we should think of AI not merely as a smart answer generator, but as a ‘system that takes action.’ Especially once agents can use external tools, follow human instructions, and handle sensitive information, safety cannot be guaranteed by model performance alone. What matters is how much authority users grant, where and how humans can intervene, and what failure modes can be anticipated in advance. In this sense, interaction design becomes security design—a prime example.

CIT's Commentary

The core idea of this article is to treat prompt injection not as a ‘problem of the model being fooled,’ but as a ‘problem of an interaction path being left open.’ In agent architectures where the system can call tools and access data, interfaces that prevent dangerous actions ahead of time matter more than achieving a higher accuracy rate. For example, even an email-reading and summarization feature can quickly lead to incidents if users cannot easily tell which messages are being sent externally. Ultimately, trust is built through transparency, the ability to intervene, and clear warnings about failure modes. This kind of design is especially important for domestic services. In environments where teams move fast to ship features—such as Naver, Kakao, or startups—‘convenience’ can easily overpower safety safeguards. A good question is less ‘How do we block it?’ and more ‘When do we stop and hand it back to the user?’

Questions to Consider While Reading

• Q.Before an agent takes risky actions, how much— and in what format—should it show the user about its state and intent?
• Q.To protect sensitive data, where should the system decide automatic blocking versus where should the human decide user intervention?
• Q.In real products, how much does this defensive structure reduce convenience, and what criteria can be used to evaluate it?