Screen-Operating Agents: Browser-Use / Computer-Use
The Browser-Use/Computer-Use line is technology where AI operates a browser or PC screen like a human. It has the power to automate even business systems with no API.
What It Can Do
- Auto-operate internal systems/web screens without an API
- Proxy research, form entry, data transcription
- Chaining operations across multiple apps
Risks Behind the Power
- Misoperation impact is large: touching real screens, a wrong delete/send causes real harm
- Breadth of privilege: giving logged-in state lets it operate a lot
- Prompt injection: it can be hijacked by instructions on a viewed page
Use It Safely
- Run with a privilege-narrowed dedicated account/sandbox
- Insert human approval for irreversible operations (send, payment, delete)
- Keep operation logs; verify small under supervision at first
Latest: open-weight screen-operating agents catch up
In May 2026, Microsoft Research released Fara1.5, a family of browser Computer-Use agents (4B / 9B / 27B). The 27B model scored 72% on the Online-Mind2Web benchmark, reportedly outperforming ChatGPT Agent (the old Operator), Gemini 2.5 Computer Use, and Yutori Navigator n1. The release also includes FaraGen1.5, a synthetic data pipeline that trains agents on gated sites — closing the gap between open-weight and proprietary Browser-Use agents.
Positioning
"A last resort that can automate even API-less work." But the wider the impact, the stricter the guardrails needed. The next chapter is internal operating rules.
