Holo3: A New Challenge to Break the Limits of Computer Use

Why Read This from an HCI Perspective

This piece clearly shows that as AI agents move beyond merely generating text and into the stage of actually ‘using’ real screens, the key differentiator is no longer raw performance—it’s the interface experience. In particular, handling a single app well versus moving across multiple systems while maintaining state are entirely different problems. For HCI practitioners and researchers, it’s a case that makes you rethink the kinds of errors hidden behind the convenience of autonomous execution, where users need to intervene, and how trust is formed.

CIT's Commentary

With computer-use agents like Holo3, what matters more than whether the model is ‘smart’ is how users can verify that intelligence, stop it, and correct it. As tasks get longer and the agent hops between multiple apps, even a small mistake can cascade into major outcomes—like sending the wrong email or missing an approval. That’s why, rather than designing around ‘how well it performs,’ these systems should first be designed around ‘where it shows state, where it brings humans into the loop, and how it recovers when it fails.’ The interesting part is that the evaluation framework for these agents is becoming an HCI research question itself. You can connect it to what kinds of interruption signals are needed in real work contexts, whether users can predict the AI’s next action, and even whether you can use LLMs to build the tools that measure these interactions faster.

Questions to Consider While Reading

• Q.In multi-app work, when an AI agent makes a mistake, what state information should users be able to see first?
• Q.How can you incorporate criteria for when a person should intervene into the interface?
• Q.When evaluating computer-use agents, what HCI metrics are essential beyond success rate?