# Observation

Screens, element trees and live views.

Screens, element trees and live views.

## observe

`GET /v1/devices/{deviceId}/observe`

The agent's look at the screen - screenshot reference plus a compact element list

The cheapest way for an agent to see a device. Elements come from the accessibility tree when the
driver has one, or from on-device-free vision (OCR and control detection) when it does not.
Secure fields and masked apps are redacted before anything leaves the engine.

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `deviceId` | path | yes | string | Stable device id such as `ios:00008120-001A2B3C4D5E6F70` or `android:R58M123ABC`. |
| `Silan-Lease` | header | no | string | Lease token when the device is claimed. |
| `screenshot` | query | no | `none`, `jpeg`, `png` |  |
| `maxLongEdge` | query | no | integer |  |
| `elements` | query | no | `none`, `interactive`, `all` |  |

| Response | Means |
|---|---|
| 200 | Observation. `Observation` |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |

MCP tools on this operation: [`observe`](/reference/mcp/observe/).

## screenshot

`GET /v1/devices/{deviceId}/screenshot`

Current screen as an image

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `deviceId` | path | yes | string | Stable device id such as `ios:00008120-001A2B3C4D5E6F70` or `android:R58M123ABC`. |
| `Silan-Lease` | header | no | string | Lease token when the device is claimed. |
| `format` | query | no | `png`, `jpeg`, `heic` |  |
| `maxLongEdge` | query | no | integer |  |
| `quality` | query | no | number |  |

| Response | Means |
|---|---|
| 200 | Image. Coordinates in later calls may use `space=image` with the returned id. |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |

MCP tools on this operation: [`screenshot`](/reference/mcp/screenshot/).

## getScreenshot

`GET /v1/screenshots/{screenshotId}`

Fetch a recent screenshot by id (kept in memory for 10 minutes unless part of a run)

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `screenshotId` | path | yes | string |  |

| Response | Means |
|---|---|
| 200 | Image. |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |

## getElements

`GET /v1/devices/{deviceId}/elements`

Full element tree

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `deviceId` | path | yes | string | Stable device id such as `ios:00008120-001A2B3C4D5E6F70` or `android:R58M123ABC`. |
| `Silan-Lease` | header | no | string | Lease token when the device is claimed. |
| `source` | query | no | `auto`, `accessibility`, `vision` |  |
| `visibleOnly` | query | no | boolean |  |

| Response | Means |
|---|---|
| 200 | Element snapshot. `ElementSnapshot` |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |

MCP tools on this operation: [`get_elements`](/reference/mcp/get-elements/).

## findElements

`POST /v1/devices/{deviceId}/elements/find`

Find elements matching a query

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `deviceId` | path | yes | string | Stable device id such as `ios:00008120-001A2B3C4D5E6F70` or `android:R58M123ABC`. |
| `Silan-Lease` | header | no | string | Lease token when the device is claimed. |

Request body: `application/json` (`ElementQuery`).

| Field | Type | What it is |
|---|---|---|
| `text` | string |  |
| `identifier` | string |  |
| `role` | `application`, `window`, `button`, `link`, `text`, `textField`, `secureTextField`, `searchField`, `image`, `icon`, `cell`, `list`, `switchControl`, `slider`, `picker`, `tab`, `tabBar`, `navigationBar`, `toolbar`, `alert`, `sheet`, `keyboard`, `keyboardKey`, `webView`, `notificationBanner`, `other` |  |
| `description` | string | Free text like "the red Checkout button"; tree first, then vision. |
| `match` | `exact`, `contains`, `regex`, `caseInsensitive` |  |
| `index` | integer |  |

| Response | Means |
|---|---|
| 200 | Matches, best first. |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |

MCP tools on this operation: [`find_elements`](/reference/mcp/find-elements/).

## streamMjpeg

`GET /v1/devices/{deviceId}/stream.mjpeg`

Simple live view for local tools (loopback only; the remote view uses WebRTC)

| Parameter | In | Required | Type | What it is |
|---|---|---|---|---|
| `deviceId` | path | yes | string | Stable device id such as `ios:00008120-001A2B3C4D5E6F70` or `android:R58M123ABC`. |
| `fps` | query | no | integer |  |

| Response | Means |
|---|---|
| 200 | multipart/x-mixed-replace stream of JPEG frames. |
| 4XX | Error (RFC 9457). |
| 5XX | Error (RFC 9457). |