Computer-Controlled AI Agents: Automation With a New Attack Surface
AI agents with computer control capabilities are systems that can see what is on a user’s screen, understand graphical interfaces, and independently click, type, and execute multi-step workflows across browsers, desktops, and mobile devices without relying on traditional APIs. This moves AI from passive text prediction into active software operation, creating both powerful automation and a new class of security risk. That shift matters: an agent that can control your computer is no longer just a tool, it is an operational actor that can help or harm depending on what it encounters on the open web. Google has now folded computer use into Gemini 3.5 Flash as a built-in tool, replacing the standalone model previously needed to build agents that see and control screens. Product leadership describes Flash as able to “see, reason about, and take action on screens,” meaning it can handle browsers, mobile devices, and desktops, click buttons, fill forms, and execute multi-step workflows without custom integrations. This is a genuine step-change in screen automation—but it also means any mistake or manipulation now happens at full GUI speed.
Hidden Web Traps: Prompt Injection Turns Websites Into Attackers
The most worrying AI agent security risks come from content that looks harmless to humans but carries malicious instructions for agents. Google highlights prompt injection as the core threat: attackers embed instructions in a webpage that trick an AI agent into unintended actions, from leaking data to approving payments. In other words, websites can become booby-trapped manuals for your AI assistant. When Gemini’s computer use feature acts “on a user’s behalf” it may encounter untrusted content on screens and make errors in executing actions. That untrusted content includes the “traps” a senior scientist warns are already being set to steal money from humans by targeting their AI agents. Site owners may need stronger bot controls and the ability to identify when hackers have hidden prompt-injection instructions on their sites, because websites are quickly becoming the battlefield from which attackers launch attacks on AI agents. Pretending this is a niche concern is naïve; the incentives for abuse are already in place.
From Perimeter Security to Screen-Level Threats
By baking Gemini screen automation into an accessible model, Google has effectively shifted automation from API calls to full desktop and browser control. Developers can now build agents that do far more than call APIs; they can automate GUI-only workflows such as testing software, filling forms, navigating dashboards, or using legacy apps that have no API at all. This is a dream for operations teams—but a headache for security. As the number of AI agents on the web proliferates, attack surfaces grow and hackers naturally turn their attention to exploiting them. Agents now operate beyond traditional security perimeters, acting inside user sessions and enterprise tools that were never designed to be driven by autonomous software. Continuous testing agents that verify functionality without human testers, or knowledge workers pulling data from dashboards and internal tools, all share a common problem: they can be tricked by content that security teams don’t yet know how to monitor or filter at the agent level. Defense-in-depth stops being theory and becomes a practical requirement.
Autonomous Agent Threats: The Attacks Are No Longer Hypothetical
We are already past the stage of purely theoretical autonomous agent threats. A senior scientist has warned that scaled AI agents create incentives “for malicious people to do malicious things,” and that malicious actors are already setting traps to steal money from humans by targeting their agents. The warning is backed by a concrete incident: this month, a cybersecurity expert experienced illicit charges made to his credit card due to an AI agent built on another provider’s system. Websites are becoming the battlefield where attackers launch prompt-injection and other agent-specific attacks. Google’s response includes targeted adversarial training against prompt injection and two enterprise safeguards: one that requires explicit user confirmation before sensitive or irreversible actions like form submissions or purchases, and another that automatically halts tasks if it detects an indirect prompt injection attempt. However, both safeguards are opt-in, not defaults. Leaving such controls optional is a strategic choice that puts the burden squarely on enterprises to recognize these risks and configure protections themselves.
What Enterprises Should Do Now: Sandboxes, Oversight, and Agent-Aware Monitoring
Enterprises that rush to deploy Gemini-powered agents without a security plan are inviting trouble. Google’s own safety guidance is candid: computer use presents unique security and operational risks because a model acting on a user’s behalf might encounter untrusted content or make execution errors. The company recommends a “defense-in-depth” approach, and that is not marketing spin—it is survival strategy. Concretely, teams should enforce human-in-the-loop controls so that when safety responses indicate require_confirmation, users must approve sensitive actions. They should run agents in secure, sandboxed environments such as virtual machines, containers, or dedicated browser profiles with limited permissions to contain damage. Input sanitization and content guardrails need to inspect user prompts, tool inputs and outputs, and agent responses for prompt injection and jailbreak attempts. Allowlists and blocklists should restrict where agents can go and what they can do, while detailed observability and logging—covering prompts, screenshots, suggested function calls, safety outputs, and executed actions—become mandatory for debugging, auditing, and incident response. If you treat an AI agent like a harmless assistant instead of an autonomous actor, you are misreading the threat.
