Agents > Computer Use

Computer Use for agents

Let agents interact with desktop GUIs in sandboxed cloud environments for automated UI testing and validation.

Computer Use enables Warp’s agents to interact with desktop environments. The agent can see what’s displayed on screen, click and drag, type text, use keyboard shortcuts, and perform other GUI interactions—all within a secure, isolated sandbox.

A key use case is testing UI changes with a self-contained feedback loop, where the agent can verify that your code changes produce the expected visual and behavioral results without requiring manual testing.

Overview

With Computer Use, agents can:

Take screenshots - Capture and analyze the current display
Record video - Capture sessions as annotated video recordings; see Testing and recordings
Interact with applications - Click buttons, fill forms, navigate interfaces
Type and control keyboard - Enter text and use keyboard shortcuts
Automate testing workflows - Test UI changes end-to-end without manual intervention
Work with browser-based interfaces - Test web apps and navigate the web

Computer Use is only available in Warp’s sandboxed cloud environments, not in local interactive terminal sessions.

In this section

Browser use - Drive the bundled Chromium browser, visually or with scripted Playwright commands.
Testing and recordings - Validate changes end-to-end and capture annotated video recordings as proof.
Screenshots and videos in pull requests - Attach captured screenshots and recordings to pull request descriptions automatically.

Enabling Computer Use

Computer Use is enabled by default for cloud agent runs on Warp’s built-in harness: when a run’s configuration doesn’t specify a Computer Use setting, the agent runs with Computer Use available. Runs on third-party harnesses (Claude Code, Gemini, Codex) default to Computer Use disabled because third-party harnesses don’t integrate with Warp’s Computer Use tooling. You can control Computer Use per run or per configuration through several entry points:

Warp app settings

Runs started from the Warp app don’t use the server default: they always follow the app’s Computer use in Cloud Agents setting (cloud_agent_computer_use_enabled), which is off by default. To control Computer Use for Cloud Agents started from the Warp app, navigate to Settings > Agents > Warp Agent > Experimental > Computer use in Cloud Agents.

Oz CLI

When running cloud agents with the Oz CLI, use flags to control Computer Use per run:

oz agent run-cloud --computer-use --prompt "<task>"
oz agent run-cloud --no-computer-use --prompt "<task>"

Oz API

When creating a cloud agent run with the Oz API, the optional config.computer_use_enabled field controls Computer Use. When omitted, it defaults to true for runs on Warp’s built-in harness and false for runs on third-party harnesses. Set it to false to disable Computer Use for the run:

{
  "prompt": "Build a button component that matches this design, then test it in the browser",
  "config": {
    "computer_use_enabled": false,
    "environment_id": "optional-environment-id"
  }
}

For full API documentation, see the Oz API & SDK reference.

Oz web app

In the Oz web app, you can enable or disable Computer Use for:

New agent runs - Configure Computer Use when starting a new agent run from the web app
Scheduled agent runs - Enable Computer Use for scheduled agents managed from the web app
Integrations - Configure Computer Use for Slack, Linear, and other integration-triggered agents

How Computer Use works

Setup and requirements

Computer Use runs in a containerized sandbox, allowing headless cloud environments to render and interact with graphical applications. The sandbox is fully isolated—it does not have access to your local machine, credentials, or sensitive data outside the sandbox environment.

Your cloud environment must include any desktop applications you want the agent to control in your environment configuration. Browser-based tasks need no extra setup: every Computer Use environment includes a bundled Chromium browser. Learn more in Browser use.

Model selection

Computer Use currently supports Anthropic Claude models. Warp uses an auto model selector to choose the best-suited model for each Computer Use task.

Recording sessions

Agents can record their Computer Use sessions as annotated video artifacts — capturing clicks, key presses, and cursor movement alongside the interaction, with idle time trimmed out. Recordings are captured entirely inside the sandbox and never include your local desktop or credentials.

For the recording pipeline, what the finished video contains, and where to find recordings, see Testing and recordings. For how recordings and screenshots land on pull requests, see Screenshots and videos in pull requests.

Security considerations

Computer Use comes with unique security considerations. These risks are heightened when interacting with the internet.

To minimize risks when using Computer Use:

Avoid sensitive data - Do not pass API keys, authentication tokens, or personal information to agents using Computer Use
Limit internet access - If your environment has internet access, consider restricting to an allowlist of known-safe domains
Require human confirmation - For tasks with real-world consequences (e.g., financial transactions, accepting legal terms), ask a human to confirm before the agent proceeds
Review agent actions - Regularly review what agents are doing on your behalf, especially when testing new workflows

Example workflows

Testing UI changes

Verify that code changes produce the expected visual results and behavior:

Build from mockups - Receive a Figma design or mockup image, build the UI, and test it matches
Visual regression testing - After code changes, verify UI renders correctly
Form and interaction testing - Test form submissions, validation, error handling
Responsive design validation - Test layout on different screen sizes

For complete end-to-end examples with demo recordings — validating UI changes, QA of critical flows, and bug reproduction — see Testing and recordings.

Browsing and web interaction

Computer Use can also help with general web tasks: browsing websites, filling out and submitting forms, and navigating multi-step workflows in web applications. See Browser use for how agents drive the browser.

Images as Context - Pass design mockups and screenshots as context
Full Terminal Use - Let agents drive interactive terminal apps, see live output, and run commands