Phones

A phone is an agent whose machine holds an Android phone: Android 13, a 720 by 1280 screen, and a browser. Like any agent it keeps everything, sleeps when idle and comes back exactly as it was, open apps and signed-in accounts included.

Phones come with paid plans. A phone uses two running slots, and always sleeps when idle.

Two ways to use one

An agent with a phone. Deploy with "phone": true. The harness inside the machine learns that it has a phone and how to use it, so you can just ask: "open this app and tell me what you see".
A phone for your own agent. Your agent runs wherever you want, and drives a Waken phone over MCP. One persistent phone per end user of your product.

Driving a phone over HTTP

Route	What it does
GET /agents/{id}/phone/screenshot	A PNG of the screen
POST /agents/{id}/phone/actions	Up to 50 actions in order, stops at the first failure
GET /agents/{id}/phone/apps	The apps that can be opened
POST /agents/{id}/phone/install	Installs an APK file from the agent's files

POST /agents/{id}/phone/actions

{ "actions": [
  { "type": "open_url", "url": "https://example.com" },
  { "type": "wait", "ms": 3000 },
  { "type": "tap", "x": 360, "y": 640 },
  { "type": "type", "text": "hello" },
  { "type": "key", "key": "enter" }
] }

Action	Fields
tap	x, y
long_press	x, y, and ms (800 by default)
swipe	x, y, to_x, to_y, and ms (300 by default). Swiping up scrolls the page down
type	text. Tap the field first
key	key: back, home, recents, enter, delete, tab, and the Android key names
open_url	url, in the browser
open_app	package, for example org.chromium.chrome
wait	ms

Coordinates are pixels on the 720 by 1280 screen. These routes need the computers scope.

Installing apps

Apps are installed from APK files. Upload the file to the agent's files, then install it:

Install an app

curl -X PUT "https://api.waken.sh/agents/$ID/fs/write?path=app.apk" \
  -H "Authorization: Bearer $WAKEN_KEY" --data-binary @app.apk
 
curl -X POST https://api.waken.sh/agents/$ID/phone/install \
  -H "Authorization: Bearer $WAKEN_KEY" -d '{"path": "app.apk"}'

Over MCP

The MCP server is the same as for desktops. Ask for a phone with os:

get_computer

{ "user_id": "user_42", "os": "android" }

Then computer and computer_batch take the phone actions above and return a screenshot. One end user can have a desktop and a phone.

What to know

There is no app store on the phone. You bring the APK files.
Apps that check the device before they run, such as banking apps and some games, will refuse to start.
Typed text is limited to letters, digits and punctuation without accents.
The phone starts in about ten seconds the first time, and wakes in about two.

Next: container images