Tests API

These endpoints create groups and tests, trigger runs, and read run status and step results. Authentication, error shapes and rate limits are covered in the REST API overview.

ID format: The {suite_id}, {case_id}, {run_id}, and {suite_run_id} path parameters are string public IDs (the public_id field in responses, e.g. "suite_abc123"), not the numeric id. App references in query and body fields generally take the app’s public_id as well — Create Suite additionally accepts a legacy numeric app_id. One exception on run IDs: the AI suite-run cancel endpoint takes the numeric ID.

Test Suites

A group is called a suite in the API: the routes below are /tests/suites, and a group’s id is its suite_id. The API and CLI call a group a suite.

A test can be in several suites, each with its own order. Deleting a suite never deletes its tests: they stay, and are listed with the tests that are in no suite.

List Suites

GET /api/v1/tests/suites

Auth: Bearer token required. Viewer role or above.

Query parameters:

Response:

[
  {
    "id": 1,
    "public_id": "suite_abc123",
    "name": "Login Flow",
    "description": "Tests for the login feature",
    "app_id": 1,
    "app_public_id": "app_xyz789",
    "app_name": "My App",
    "platform": "ios",
    "case_count": 5,
    "reset_strategy": "clear_data",
    "context": null,
    "variables": {},
    "last_run_status": "passed",
    "last_run_at": "2025-01-20T14:00:00",
    "created_at": "2025-01-15T10:30:00",
    "updated_at": "2025-01-20T14:00:00",
    "tags": [{"name": "profiles", "display_name": "Profiles"}]
  }
]

Create Suite

POST /api/v1/tests/suites

Auth: Bearer token required. Developer role or above.

Request body:

Response includes tags: [{name, display_name}] reflecting the suite’s current tags, and visual_regression_warn_pct / visual_regression_fail_pct.

Screenshot comparison is on by default for a new suite: it starts with visual_regression_warn_pct: 30 (warn when 30% or more of a screen’s pixels changed) and visual_regression_fail_pct: null (a changed screen never fails the run). Change either per suite with Update Suite, or set both to null to switch the comparison off. A suite imported from YAML takes the file’s values, with the same defaults for any it leaves out.

Export Suite as YAML

Downloads the suite and all its cases as a YAML file. Includes step mappings, source recordings, and device dimensions for portable replay. Step ids are left out: importing a file always gives its steps new ids.

GET /api/v1/tests/suites/{suite_id}/export-yaml

Auth: Bearer token required. Viewer role or above.

Response: YAML file download (text/yaml).

Import Suite from YAML

Creates a test suite from an uploaded YAML file. Optionally links to an app.

POST /api/v1/tests/suites/import-yaml

Auth: Bearer token required. Developer role or above.

Body: multipart/form-data with a file field containing the YAML file.

Query parameters:

If app_id is not provided but the YAML contains an appId field, the import attempts to match an existing app by package name.

Response:

{
  "public_id": "my-suite-abc123",
  "name": "My Test Suite",
  "case_count": 5,
  "warnings": []
}

warnings lists the keys the imported tests use with no value (see Test data warnings).

Get Suite

Returns the suite with all its test cases.

GET /api/v1/tests/suites/{suite_id}

Auth: Bearer token required. Viewer role or above.

Update Suite

PATCH /api/v1/tests/suites/{suite_id}

Auth: Bearer token required. Developer role or above.

Request body:

PATCH answers 409 when a test of the suite is also in another suite, and 400 when app_public_id moves the suite between a phone or tablet app and a TV app while one of its tests uses a setup that stays on the other kind of app (or is the setup of such a test).

Delete Suite

DELETE /api/v1/tests/suites/{suite_id}

Auth: Bearer token required. Developer role or above.

The suite’s tests are kept (they stay in any other suite they belong to).

Add Tests to a Suite

POST /api/v1/tests/suites/{suite_id}/members

Auth: Bearer token required. Developer role or above.

Request body: {"case_ids": [...]} — tests of the suite’s app, added at the end in the order given. The response lists the suite’s test ids in order. 404 for an unknown test, 400 for a test of another app.

Remove a Test from a Suite

DELETE /api/v1/tests/suites/{suite_id}/members/{case_id}

Auth: Bearer token required. Developer role or above. The test is kept.

Tests page reads

These read-only endpoints back the Tests page. All take the app’s public ID as {app_id}, or none for tests and groups that belong to no app. The tests, attention, runs and runs overview endpoints take scope: app (default) or family, which also covers the app’s linked apps of the same form factor (a phone app’s family scope never includes a TV app). Auth: Bearer token required. Viewer role or above.

List an app’s tests

GET /api/v1/tests/apps/{app_id}/tests

Returns the app, its groups, and every test of the app in one response (app is null for none). Filtering and sorting are up to the client. how_it_runs is recorded, learned or ai. recent lists up to the last five finished runs, newest first, each with status, visual_regression_status and run_id. setup_count is how many setup steps the test runs first, and used_by how many tests use it as a setup. avg_duration_ms is over the last 30 days and null with no counted run. attention is null, or the newest failure’s reason, its run_id and failed_runs (failures among the last 3 counted runs) when the test needs attention and isn’t quarantined. missing_test_data lists the keys the test’s steps use with no value, each with its key and message (as in Test data warnings); it is [] when there are none. last_by_platform maps each platform the test ran on (ios, android, tvos, androidtv, roku — the platform of the device, not of the test’s app) to its last counted run: outcome (passed or failed), run_id and at. Not checked, cancelled, errored and stopped runs don’t count; a platform with no counted run is absent. For a linked app, app.family is {"public_id", "name", "platforms"}, where platforms lists the product’s platforms of the app’s form factor (ios, android, tvos, androidtv, roku, in that order); it is null for an app that is not linked.

{
  "app": { "public_id": "app_abc", "name": "MyApp", "platform": "ios", "package_name": "com.example.app" },
  "groups": [
    { "public_id": "a1b2c3", "name": "Checkout", "tags": [{ "name": "smoke", "display_name": "smoke" }], "reset_strategy": "clear_data", "member_count": 4 }
  ],
  "tests": [
    {
      "public_id": "tc_123",
      "name": "Sign in",
      "step_count": 6,
      "is_reusable": false,
      "used_by": 0,
      "setup_count": 0,
      "groups": [{ "public_id": "a1b2c3", "position": 0 }],
      "how_it_runs": "recorded",
      "recent": [
        { "status": "passed", "visual_regression_status": null, "run_id": "run_789" },
        { "status": "failed", "visual_regression_status": null, "run_id": "run_456" }
      ],
      "last_run_at": "2026-10-09T14:03:11",
      "last_run_status": "passed",
      "last_by_platform": {
        "ios": { "outcome": "passed", "run_id": "run_789", "at": "2026-10-09T14:04:02" },
        "android": { "outcome": "failed", "run_id": "run_611", "at": "2026-10-08T09:12:40" }
      },
      "avg_duration_ms": 41000,
      "attention": null,
      "flaky": true,
      "missing_test_data": [],
      "quarantined_at": null
    }
  ]
}

The group objects also carry the description, notes and screenshot-check thresholds, and each test carries its creation and update times; both are trimmed here.

Needs attention

GET /api/v1/tests/apps/{app_id}/attention

Tests where one of the last 3 counted runs failed (quarantined tests are left out), then new routes waiting for review.

{
  "items": [
    {
      "kind": "failing",
      "case_public_id": "tc_123",
      "case_name": "Sign in",
      "reason": "Step 3 failed: the Sign in button was not found",
      "run_id": "run_456",
      "path_id": null,
      "failed_runs": 2,
      "at": "2026-10-09T14:03:11"
    }
  ]
}

kind is failing or route_review. A route_review item has a path_id, failed_runs is null, and run_id is the run that found the route (or null).

List an app’s runs

GET /api/v1/tests/apps/{app_id}/runs

Every run of the app’s tests, group runs and single-test runs, newest first.

{
  "items": [
    {
      "id": "run_456",
      "kind": "test",
      "name": "Sign in",
      "group_public_id": null,
      "case_public_id": "tc_123",
      "status": "failed",
      "stop_reason": null,
      "visual_regression_status": null,
      "tests_passed": 0,
      "tests_total": 1,
      "device_name": "iPhone 16",
      "device_model": "iPhone 16",
      "platform": "ios",
      "trigger": "cli",
      "duration_ms": 38000,
      "cost_micro": 12000,
      "real_cost_micro": 12000,
      "est_cost_micro": 0,
      "is_byok": false,
      "model_used": "example-model",
      "started_at": "2026-10-09T14:03:11"
    }
  ],
  "next_before": "2026-10-09T13:40:02"
}

kind is group, test or batch (a run started from the Run dialog or POST /api/v1/tests/runs, one row whatever its device count; device_name then reads, for example, 2 devices). platform is the platform of the device the run ran on (null when unknown). device_model is that device’s model (on a batch, its first device’s; null when unknown). A batch row also has platforms, the platforms of its devices in a fixed order (iOS, Android, Apple TV, Android TV, Roku); platform is set on a batch only when all its devices share one. A group run has group_public_id and case_public_id is null; a single-test run is the other way round. stop_reason is set only on a single-test run that VibeView stopped: out_of_credit, ai_paused, ai_disabled or free_test_limit. cost_micro is real_cost_micro (charged to usage credit) plus est_cost_micro (an estimate for runs on your own AI key); 1,000,000 is $1.00. next_before is null on the last page. An unknown group returns 404, and an unknown status returns 422.

Runs overview

GET /api/v1/tests/apps/{app_id}/runs/overview?tz_offset_min=0

The Runs section’s tiles and chart: the last 30 days against the 30 days before, counting only runs that tell you something about the app or the test (cancelled runs, runs stopped for usage credit and runs that ended with the device session don’t count).

{
  "window_days": 30,
  "current": { "runs": 52, "pass_rate": 0.904, "avg_duration_ms": 43000 },
  "previous": { "runs": 40, "pass_rate": 0.875, "avg_duration_ms": 47000 },
  "failing_now": { "count": 2, "first_name": "Sign in" },
  "per_day": [{ "date": "2026-10-09", "passed": 4, "failed": 1 }]
}

pass_rate and avg_duration_ms are null with no data, and per_day has one entry for each of the 30 days.

List Org Tags

GET /api/v1/tests/tags

Auth: Bearer token required. Viewer role or above.

Lists every tag in the user’s organization with usage counts. Used by autocomplete UIs. Pass ?app_id=<app public ID> to count only that app’s groups and leave out tags none of its groups use; an unknown app returns 404.

Response: 200 OK

[
  { "name": "smoke", "display_name": "smoke", "suite_count": 7 },
  { "name": "profiles", "display_name": "Profiles", "suite_count": 3 }
]

Sorted by suite_count DESC, name ASC. Tags with zero suites are included.

Test Cases

Create a Test in a Group

POST /api/v1/tests/suites/{suite_id}/cases

Auth: Bearer token required. Developer role or above.

Request body:

Steps

Every step is a plain-language instruction: a string ("Tap Sign in"), or an object whose text holds the instruction, optionally with per-step settings such as post_delay_ms or force_ai. Raw recorded actions (objects like {"type": "touch_start", "params": {...}}) are not steps: send them as source_recording, next to text steps that describe them. A request whose steps include a recorded action, an empty instruction, or an object without text is refused with 422, and the message names the first such step. A case needs at least one step: an empty steps list is refused with 422 too, since a run of it would do nothing on the device. (Updating a case to no steps of its own is allowed only while it has setup steps to run.) The same rules apply when you update a case’s steps, create a test outside any group, or import a group from YAML.

Each saved step gets an id (for example "s_k3m9q2x7ab4c"), returned with the case. Send it back when you update the steps and the step keeps its id when you reorder, insert or remove other steps. A step you send without an id is matched to an existing step with exactly the same text, in order. A step without an id whose text you changed in place, between the same neighbouring steps, keeps the step it replaces, with its visual baselines and recording screenshot. Inserted, deleted or moved steps never pass baselines on: a changed step in the same gap as an insert, a delete or a move is a new step, as is any other step with no match, or one sent with "id": null. An id that doesn’t belong to one of the case’s current steps, or isn’t in that format, is ignored, and the step is matched as if it had none. Plain-string steps are returned as objects ({"id": "...", "text": "..."}).

Response:

{
  "id": 1,
  "suite_id": 1,
  "name": "Login with valid credentials",
  "mode": "ai",
  "steps": [{"id": "s_k3m9q2x7ab4c", "text": "Tap Sign in"}, ...],
  "assertions": [],
  "step_count": 5,
  "steps_version": 3,
  "has_source_recording": true,
  "source_recording_summary": {
    "total_actions": 12,
    "gesture_counts": {"tap": 3, "swipe": 1}
  },
  "source_gesture_list": [...],
  "step_gesture_groups": [...],
  "source_device_width": 390,
  "source_device_height": 844,
  "context": null,
  "created_at": "2025-01-15T10:30:00",
  "updated_at": "2025-01-15T10:30:00"
}

assertions in the response is deprecated and always []. The response also carries warnings (see Test data warnings).

Test data warnings

Every request that saves steps — Create a Test, Create a Test in a Group, Update Test Case, Import Suite from YAML and Import Maestro flows — returns warnings, and so does Get Test Case: one entry per {{key}} the test’s steps use that has no value, or one per group of platforms when the platforms differ in which environments give it a value (see When a key has no value). The steps are saved either way; warnings is [] when every key has a value.

{
  "warnings": [
    {
      "code": "missing_test_data",
      "key": "pin",
      "message": "Steps use {{pin}}, which has no value for MyApp.",
      "platforms": ["ios"],
      "environments": []
    }
  ]
}

• key names the key and message says what is missing; neither ever holds a value.
• platforms lists the platforms with no value for the key. For a linked app it is every platform of the test’s form factor the key is missing on (a key only the Android app has is reported with ["ios"]).
• environments lists the test environments that give the key a value on those platforms, when it has one only there; the message then reads “…which has a value for MyApp only in the Staging environment.”
• A secret or one-time code counts as a value.
• Only the exact form {{key}} (letters, digits and underscores) is a key; {{ key }} or {{key|upper}} is not checked.
• The two imports add test_id and test_name to each entry.

steps_version increases every time the case’s steps or its recording change. Sending back the recording exactly as you read it is not a change.

List a Group’s Tests

A suite’s cases are returned embedded in the suite object — fetch them with Get Suite (GET /api/v1/tests/suites/{suite_id}). There is no separate list endpoint.

Get Test Case

Returns the test case. Works for every test, in a group or not. To fetch a case’s run history, use GET /api/v1/tests/cases/{case_id}/runs.

GET /api/v1/tests/cases/{case_id}

Auth: Bearer token required. Viewer role or above.

Response: The test case object, with warnings for its steps as they are now: an entry goes away once its key has a value.

Update Test Case

PATCH /api/v1/tests/cases/{case_id}

Auth: Bearer token required. Developer role or above.

Request body:

Send each step’s id back to keep the step’s identity: a step’s visual regression baselines and recording screenshot follow its id, so reordering or inserting steps never changes which references a step uses. Removing a step deletes its references. A step sent without an id keeps the references of the step it matches: one with exactly the same text, or the step it rewords in place (see Steps above). A step with no match, or sent with "id": null, is new and starts without references. See Reordering or removing steps.

Response: The test case object, with warnings for its steps as saved.

Errors:

• 409 with detail.code "steps_changed" — sent with steps_version: the case’s steps or recording changed since that version. Nothing was saved. Reload the case and apply your edit again.
• 409 with detail.code "save_in_progress" — sent with steps_version: another save of the same case was in progress at that moment. Nothing was saved and nothing changed; send the same request again after a moment.

Delete Test Case

DELETE /api/v1/tests/cases/{case_id}

Auth: Bearer token required. Developer role or above.

The test’s run history is deleted with it, including finished runs that ran only this test. A group run or a run of several tests keeps the other tests’ results and leaves the deleted test out of its rows and counts.

Copy Test Case

Copy a test case to another suite.

POST /api/v1/tests/suites/{suite_id}/cases/{case_id}/copy

Auth: Bearer token required. Developer role or above.

Request body:

The copy keeps the steps (as new steps, with their own ids), the recording and the attached setup steps (it references the same reusable cases). It keeps the original name when that name is free in the destination suite; otherwise it is named <case name> (copy), or (copy 2), (copy 3) and so on. Copying a copy doesn’t add a second suffix.

Copy a Test

Duplicate a test, or copy it to another app.

POST /api/v1/tests/cases/{case_id}/copy

Auth: Bearer token required. Developer role or above.

Request body (every field optional):

The copy keeps the steps (as new steps, with their own ids), the recording and the original’s setup steps (it references the same reusable tests). It keeps the original name when that is free in the target app; otherwise it is named <name> (copy), then (copy 2) and so on. Copying to an app of a different kind (phone or tablet to TV, or the other way round) is refused with 400. Returns the new test.

Quarantine a Test

PUT /api/v1/tests/cases/{case_id}/quarantine

Auth: Bearer token required. Developer role or above.

{ "quarantined": true }

A quarantined test keeps running and recording results, but it is left out of Needs attention and sends no run.warning.flaky notification. Setting the state it already has changes nothing.

Response: 200 OK

{ "public_id": "tc_123", "quarantined_at": "2026-10-10T09:12:00", "quarantined_by": "dev@example.com" }

Create a Test

Create a test, in no group or (with group_ids) in groups. If a test with the same name already exists for the app, if_exists decides what happens. By default an identical case is reused as it is, and a different one is never overwritten: the request is refused with 409.

The check compares the mode, context, each step’s text and settings (a setting left null counts as not set), and every other field the request sets: the recording, device size and model, and platform.

POST /api/v1/tests/cases

Auth: Bearer token required. Developer role or above.

Request body:

Response: The test case object, same shape as Create a Test in a Group (including groups and start_state), plus outcome: created, updated (overwritten with if_exists: "update") or unchanged (an identical case was reused), and warnings.

Errors:

• 400 — A setup that is missing, not reusable, has setups of its own or is for another kind of app; or an unknown start_state.
• 404 — App not found in your organization.
• 409 — A test with this name already exists for the app and differs from the request (or if_exists is fail). The message names the existing case.
• 422 — A step is a raw recorded action or has no instruction (see Steps).

Import Maestro flows

POST /api/v1/tests/import/maestro

Auth: Bearer token required. Developer role or above. Limited to 10 requests per minute.

Request: multipart/form-data

Response: the tests the flows become (key, name, steps, setup_keys, start_state, flags, importable), the setups they use, upload-level flags, and the test_data keys the steps refer to (key, suggested_value, used_by, secret, and after an import status: created, kept, needs_value or skipped). The suggested value of a secret key is masked. created is null on a dry run; after an import it is the number of tests created (setups included), each test and setup has its created_id, and warnings lists the keys the created tests use with no value. A flag has file, where, command, severity (rewrite: no step was created; check: review the step; dropped: nothing needed) and message.

Errors:

• 400 — The upload can’t be read (the message says why, for example YAML anchors, a file over the limit, or a password-protected .zip).
• 403 — Your role can’t create tests.
• 404 — App or group not found.
• 413 — Upload too large.
• 422 — Nothing to import, an invalid skip, or a group name that matches several groups.
• 429 — Too many imports; wait a minute.

List ungrouped tests

List tests that are in no group.

GET /api/v1/tests/cases

Auth: Bearer token required. Viewer role or above.

Query parameters:

Response: Array of test case objects (most recent first), each including recent_run_statuses for the last 10 runs.

App Notes and Test Data

Values a test uses as {{key}} in its steps, kept per app (see Test data, secrets and environments). Reads need the Viewer role or above; writes need Developer or above.

Keys are letters, digits and underscores, up to 128. A value is up to 2000 characters; up to 200 values per app. kind is text or totp (a base32 setup key for a one-time code, always secret; 422 if it is not valid). A secret is one line of text with at least one letter or digit, and may be any length (422 otherwise). A secret is never returned; sending [MASKED] back keeps the stored value. Secrets are masked in run results, reasoning, progress and logs, and the AI model only ever sees {{key}}.

On an app linked to other platforms (an app family), these endpoints mean “this app’s platform”:

• GET .../data returns the values this platform uses, each with "scope": "shared" or "platform", and the response has "family": {"public_id", "name"}.
• PUT .../data/{key}: a new key is shared by every linked app; a different value is this platform’s own; the shared value removes this platform’s own value.
• DELETE .../data/{key} removes this platform’s own value. A key it only shares returns 409; delete it with DELETE /api/v1/app-families/{family_id}/data/{key}.
• GET .../notes also returns shared_notes (the family’s shared notes); PUT .../notes writes this platform’s notes.
• The suite variables field follows the same rules; a PATCH that drops a shared key returns 422 naming it.

Environments

A named set of overrides for an app’s test data, called a test environment in the dashboard.

In the dashboard, manage test environments under App notes & data on the Tests page and pick one in the Run dialog (Test environment). Run requests (POST /api/v1/tests/runs and the CI run endpoints) accept an optional environment (name or ID); an unknown one returns 404. The CLI has no environment option yet.

On a linked app, creating, renaming and deleting an environment applies to every linked app (the default build stays per app), and its values follow the same rules as app data.

Test Case Runs

List Runs for a Test Case

GET /api/v1/tests/cases/{case_id}/runs

Auth: Bearer token required. Viewer role or above.

Returns all runs for the specified test case, ordered by most recent first.

Visual Regression Baselines

A baseline belongs to one step (by the step’s id) on one device: device name, OS version and, on a foldable, posture (closed, partial or open; "" on other devices). See Visual Regression Testing.

Errors from the endpoints that change baselines carry detail: {"code": "...", "message": "..."}:

List Baselines for a Test Case

GET /api/v1/tests/cases/{case_id}/baselines

Auth: Bearer token required. Viewer role or above.

Query parameters:

Returns the baselines of the steps the case has now, in step order. A step that has a recording screenshot but no baseline in the returned set gets one extra row with is_synthetic: true: the reference a device without a baseline is compared with.

screenshot_path and recording_revision (both always null), perceptual_hash and recording_step_index are deprecated: use image_url and recording_image_url.

List Baseline Devices for a Test Case

GET /api/v1/tests/cases/{case_id}/baselines/devices

Auth: Bearer token required. Viewer role or above.

One entry per device, OS version and posture that has baselines on the case (device_name, os_version, posture, step_count, last_updated_at), most recently updated first.

Get Baseline Image

GET /api/v1/tests/cases/{case_id}/baselines/step_{step_index}.jpg

Auth: Bearer token required. Viewer role or above.

Returns the current baseline of the step now at position step_index as a JPEG image. Pass device_name (and optionally os_version and posture) for one device’s baseline; without device_name, the step’s most recently updated baseline on any device is returned.

Get Visual Image

GET /api/v1/visual-images/{image_id}.jpg

Auth: Bearer token required. Viewer role or above.

Returns a stored visual regression image (a baseline, recording screenshot, run screenshot or diff) by id, as linked from the baselines list and from run results. Images never change, so responses can be cached indefinitely. Returns 404 for an unknown image or one that belongs to another organization.

Clear a Device’s Baselines

DELETE /api/v1/tests/cases/{case_id}/baselines?device_name={name}

Auth: Bearer token required. Developer role or above.

Query parameters: device_name (required; 400 when empty), os_version (omitted means ""), and posture (optional: "", closed, partial or open, 422 otherwise). Without posture, every posture of the device is cleared.

Response: {"deleted": 3, "device_name": "iPhone 16 Pro"}

Accept Baselines from a Run

POST /api/v1/tests/runs/{run_id}/set-baseline                # every step (passed runs only, 400 otherwise)
POST /api/v1/tests/runs/{run_id}/set-baseline/drifted        # warn and fail steps only
POST /api/v1/tests/runs/{run_id}/set-baseline/{step_index}   # one step, by its position in the run

Auth: Bearer token required. Developer role or above.

Each step’s screenshot becomes the baseline of the step that produced it, on the run’s device, even if the step has moved since. Works for test cases inside and outside a suite. The whole-run and drifted variants skip steps removed since the run; the per-step variant refuses them (409 step_removed). Error codes are listed under Visual Regression Baselines.

Responses:

{"count": 5, "device_name": "iPhone 16 Pro"}
{"step_indices": [2, 4], "step_ids": ["s_k3m9q2x7ab4c", "s_9f2k1m4p8q3r"], "device_name": "iPhone 16 Pro"}
{"status": "ok", "step_index": 4, "step_id": "s_9f2k1m4p8q3r", "device_name": "iPhone 16 Pro"}

(whole run, drifted, one step). step_indices are positions in the run.

Start a Run

Start a run of a test, a selection of tests, a group or every test of an app, on one or more devices at once. This is the API behind the dashboard’s Run dialog, and it takes the same fields as the CI routes. A run on one or more devices gets one id, which you use to read it, cancel it and open it.

POST /api/v1/tests/runs

Auth: Bearer token required. Developer role or above.

Each device runs the tests one after another, in group order, in a session of its own, and the devices run at the same time. A device waits in the queue when your plan’s concurrent sessions are all in use.

Response: 201 Created

{
  "id": "run_abc123",
  "batch_id": "run_abc123",
  "name": "Smoke",
  "status": "running",
  "target_kind": "group",
  "trigger": "cli",
  "open_path": "/tests/runs/run_abc123",
  "run_url": "https://vibeview.io/tests/runs/run_abc123",
  "lanes": [
    {
      "id": "lane_1",
      "status": "queued",
      "platform": "ios",
      "model": "iPhone 16",
      "os_version": "18.2",
      "queue": { "position": 1, "reason": "devices_busy", "priority": "low" },
      "runs": [{ "kind": "group", "id": "sr_1", "name": "Smoke", "status": "pending", "tests_total": 5, "tests_passed": 0, "tests_failed": 0 }]
    }
  ],
  "summary": { "tests_total": 5, "tests_passed": 0, "tests_failed": 0 }
}

lanes has one entry per device. run_url opens the run’s page. Each runs entry and the summary also carry tests_not_checked: the tests in which no step was checked, which never count in tests_passed. Each runs entry lists its tests under tests, one row per test with name, status (not_run for a test that never ran), not_run_reason (why it never ran, or null), steps_not_checked (steps the run could not check) and quarantined (whether the test was quarantined when this run started; quarantining a test later never changes an earlier run). The response also carries mode, environment (the environment’s name, or null), start_state, stop_on_first_failure, commit_sha, metadata, cancel_requested, created_at, finished_at, app_public_id and group_public_id.

Errors:

• 400 — the group or app has no tests, a test has no steps, a device is for the other form factor (a phone test on a TV, or the reverse), a device is picked twice, an app has no build to install, or mode is replay and nothing has a recording.
• 403 — AI can’t run (for example no usage credit; detail carries a message and remedy), your streaming minutes are used up, the app is restricted, or your role can’t run tests.
• 404 — an app, test, group, build or environment wasn’t found.
• 409 — the free test isn’t open to you, or your plan isn’t the free one.
• 410 — a device’s build_id names a build that has expired: detail.error is build_expired, with message and newer_build_id (see Build retention). Nothing starts.
• 422 — a field is invalid, for example target test without exactly one id.

Get a run

GET /api/v1/tests/batches/{id}

Auth: Bearer token required. The run’s current state in the shape of the start response, for polling. Each entry of lanes has its status, queue while it waits (position, reason, priority), error_message, and its runs. priority is high on paid plans and low on Free. When every device is busy, paid plans’ requests start first, so a Free request’s position can go up. 404 when the run isn’t in your organization.

Each lane’s runs entry (a test, or a group with its own tests) lists a tests row per test: test_id, name, run_id (null before it ran), status (a run status, pending while the device is still going, or not_run when it never ran, including a test skipped because an earlier test failed under stop_on_first_failure), stop_reason, duration_ms, cost_micro, is_byok and error_message. The run also carries start_state, commit_sha and commit_url (a link to the commit on GitHub when your organization has connected a repository).

Run options

GET /api/v1/tests/run-options?app_id={app_id}&group_id={group_id}

Auth: Bearer token required. What the Run dialog offers for an app: apps (the app and your same-form-factor apps with the same bundle id or package name on other platforms), builds (each app’s latest runnable build and recent ones), concurrency (limit and in_use), payer (who_pays: usage_credit, own_key, free or blocked, with the model, and a message and remedy when blocked), environments, and default_start_state for the group. schedules_allowed is false on the Free plan. family is {"public_id", "name", "first_app_id"} when the app is linked to apps on other platforms as one product (first_app_id: the product’s first app of this form factor, in the order iOS, Android, Apple TV, Android TV, Roku), else null.

Resolve a run id

GET /api/v1/tests/runs/{run_id}/resolve

Auth: Bearer token required. A run link can name a single test run, a group run or a run on one or more devices. This returns {"kind": "test_run" | "suite_run" | "batch", "id": "..."} so a client knows which detail view to read. 404 when it isn’t in your organization.

Schedules

A schedule saves a run (the same fields as Start a Run, without commit_sha and metadata) to start on chosen days at a set time or on every new build of the app, as the schedule’s owner: the person who created it. See Scheduled runs. Paid plans only.

List schedules

GET /api/v1/tests/schedules?app_id={app_id}

Auth: Bearer token required. app_id is optional. Returns {"items": [...]}, each a schedule as below.

Create a schedule

POST /api/v1/tests/schedules

Auth: Bearer token required. Developer role or above. Rate limit: 30 per minute.

Response: 201 Created

{
  "public_id": "sch_abc123",
  "name": "Smoke",
  "app_id": "app_123",
  "trigger": "nightly",
  "local_time": "02:00",
  "timezone": "Europe/Vilnius",
  "days": ["mon", "wed", "fri"],
  "target": "group",
  "group_id": "grp_123",
  "group_name": "Smoke",
  "test_ids": [],
  "family": false,
  "devices": [{ "app_id": "app_123", "platform": "ios", "model": "iPhone 16", "os_version": "18.2", "device_category": "phone", "build_id": null }],
  "mode": "hybrid",
  "start_state": null,
  "environment": null,
  "stop_on_first_failure": false,
  "max_minutes": 120,
  "owner_email": "you@example.com",
  "owner_runnable": true,
  "paused_reason": null,
  "paused_message": null,
  "skipping_unpaid": false,
  "next_run_at": "2026-10-12T23:00:00",
  "last_fired_at": null,
  "last_run": null,
  "last_error": null,
  "created_at": "2026-10-12T09:30:00"
}

Times are UTC. paused_reason is owner_gone, plan, target_gone, app_blocked, build_expired (the build it runs has expired: pick another, or turn on Keep for it, then resume), refused or manual when the schedule is paused, with paused_message saying why. owner_runnable is false when the owner can no longer run it (resuming then makes you its owner). skipping_unpaid is true while the schedule’s runs are skipped because the subscription payment is overdue. days lists the days a nightly schedule runs on, Monday first, or is null when it runs every day (also when all seven were sent); it is null for new_build. A nightly schedule runs at most once per day in its time zone: after a change of time or days, or a resume, on a day it already ran, next_run_at is its next day after that. last_run is {"id", "status", "started_at"} of the most recent run it started. family is true for an “all tests” schedule that runs every test of the product’s form factor.

Errors: 403 on the Free plan (detail has message and remedy: "upgrade"), while the subscription payment is overdue (remedy: null), or without the Developer role; 404 when a test, group, build, environment or app wasn’t found; 400 when a device is for the other form factor or the run is the free test; 409 at 50 schedules; 422 for a missing time, an unknown time zone, or days that is empty, repeats a day or names an unknown one.

Change a schedule

PATCH /api/v1/tests/schedules/{public_id}

Auth: Developer role or above. Any of name, local_time, timezone, days, max_minutes, retry_failed, paused (true pauses, false resumes), trigger and run.

• trigger switches between nightly and new_build. To nightly it needs a valid local_time and timezone (in the request or already saved; 422 otherwise) and runs every day unless the request sends days. To new_build it clears the time and days, and only builds that land after the change start it.
• run replaces the saved run, as the body of Start a Run: devices, builds, mode, start_state, environment, stop_on_first_failure and retry_failed. A saved model or provider the request leaves out is kept, and so is the device_id of a device that is still listed (same app, model and OS) and doesn’t name its own. It is checked like a new schedule (a missing test, group, environment or app is 404; a device of the other form factor is 400). What the schedule runs can’t change: an app_id, target, test_ids, group_id or family that differs from the saved one is refused (400); left out, the saved one is kept. A new_build schedule keeps watching each app it still has devices for, and a newly added app starts after its existing builds.
• The schedule’s owner never changes (the audit log names who edited it), and an edit never resumes a paused schedule.
• A run or trigger change is refused on the Free plan and while a paid plan’s payment is overdue (403, the same detail as creating a schedule). Renaming, the time limit, pausing and the time of day are not. Resuming a schedule whose owner can no longer run it makes you its owner. Resuming is refused (403) on the Free plan, (404) when something it runs was deleted, and (400) when its time zone is no longer recognised (change timezone first) or when it has no devices left after an app was removed from its product (send a run with devices first). A schedule with no devices left lists devices as []. days replaces the saved days (send all seven to run every day; left out, the saved days are kept). local_time, timezone and days on a new_build schedule are refused (400); a time that isn’t HH:MM, 24-hour, is refused (422).

Run a schedule now

POST /api/v1/tests/schedules/{public_id}/run-now

Auth: Developer role or above. Starts a run now, as the schedule’s owner. Returns {"batch_id", "open_path", "run_url"} as Start a Run does. 409 when the schedule is paused; when the run can’t start (for example no usage credit), the same status and detail (message, remedy) as Start a Run. When the build it runs has expired, it answers 410 and the schedule pauses (paused_reason: "build_expired").

Delete a schedule

DELETE /api/v1/tests/schedules/{public_id}

Auth: Developer role or above. Returns {"status": "deleted"}. The runs it started stay.

Test Runs (run-level operations)

Get Test Run

GET /api/v1/tests/runs/{run_id}

Auth: Bearer token required.

While the run waits for a device (status: "queued"), queue holds its position in line, the reason it waits (devices_busy, devices_updating or plan_limit, or null when VibeView can’t tell) and its priority. priority is high on paid plans and low on Free. When every device is busy, paid plans’ requests start first, so a Free request’s position can go up. Otherwise queue is null.

A run’s status and result are set only by VibeView as the run executes; there is no endpoint to set them.

Each entry in result has a status: passed, failed, error, or not_checked. A not_checked step is one a replay run could not check — it has no recorded actions, and replay mode never uses AI (see Checking results with Verify steps). It is neither a pass nor a failure, and its note says why. steps_not_checked counts them, so a run that passed with steps_not_checked > 0 was not fully checked.

A run in which no step was checked at all ends with status: "not_checked": neither a pass nor a failure (its error_message says why). A group run, and a run’s lanes and the run itself, end not_checked when every test in them was not checked; with at least one passed test and no failure they end passed. A test skipped because an earlier test failed under stop_on_first_failure has status: "cancelled" and an error_message starting Not run:; the dashboard shows it as Not run.

Each entry in result has missing_test_data: the keys that step used with no value in this run (null when there were none, and on runs from before this was recorded), and missing_test_data_not_typed: those of them the step tried to type, which were refused (null when none). warnings gathers them, one entry per key with code (missing_test_data), key, steps (step numbers, from 1) and message; it is [] otherwise. A run whose recorded typing would type such a key fails before its first step, and its error_message names the step and the key.

Each entry in result has started_ms: when the step started, in milliseconds after the run started (null on runs from before this was recorded). Actions inside a step may carry started_ms too. timeline holds origin_ms (when the run started, in milliseconds since the Unix epoch) and segments: the wait for the app to be ready before step 1 (kind: "readiness") and each setup that ran before the test (kind: "setup", with its name), each with started_ms and duration_ms. timeline is null on older runs.

replayed_step_count and ai_step_count say how many steps ran without and with AI; ai_spend_avoided_micro is the usage credit the replayed steps saved, in micro-credits (1,000,000 = $1.00), and ai_spend_avoided_is_estimate is true when a replayed path was learned with your own API key (the figure is then an estimate). All four are null on older runs. A replayed step counts only when it passed. learned_paths lists the paths the run saved or proposed for review (id, step_id, step_text, platform, device_category, orientation, posture, status, replaces_id, decision). Each result entry has path_source (recorded, learned or null) and learn_status (learnable or not_learnable: <reason> on steps the AI ran, set only on a run in which every step and check passed).

assertion_results and assertions_summary are deprecated and always null: test cases no longer have a separate assertions list.

The run page’s own fields:

• start_state — what the run started from: reinstall, clear_data, relaunch or none. null when it wasn’t recorded: older runs, and a run’s first attempt on a CI session’s fresh device.
• commit_sha and commit_url — the commit the run was started with, and a link to it on GitHub when your organization has connected a repository (otherwise null).
• all_ai_cost_micro — what the same run would have cost with AI on every step, in micro-credits: the run’s cost plus the AI spend avoided. Present only when at least one step was replayed and every replayed step was a learned path; otherwise null. It is an estimate when ai_spend_avoided_is_estimate is true.
• mode — the AI use the run was started with: hybrid, replay or ai.
• environment — the name of the environment the run used, or null when it ran with the app’s own test data (or the environment was deleted since).
• trigger — what started the run: dashboard, ci, cli or mcp (a coding agent); null on older runs.
• failure — why a run that didn’t pass failed; null while it is going and when it passed. It has:
  • kind — one of check_failed, screenshot_drifted, action_failed, app_crashed, app_bug, blocked_by_state, missing_test_data (a step tried to type a key with no value, or the run’s recorded typing would have; see When a key has no value), setup_failed, couldnt_start, stopped_credit, stopped_ai, infrastructure or cancelled.
  • label — the heading the run page shows, for example “Check failed”.
  • explanation — the reason, in words (or null).
  • step_index and step_id — the step that failed (null when no step did), and step_case_public_id, the test the step belongs to (a setup test’s id when the step is from a setup).
  • actions — what the run page offers: bug_report, expectation_changed, ai_misjudged (not available yet), changed_on_purpose, too_strict, test_is_wrong, run_with_smart, rerun, change_start_state, edit_app_notes, open_setup_test, remedy, top_up, add_key, add_test_data (Add a value in App notes & data), ai_settings (open AI configuration to raise the run’s AI spending limit) or contact_support.
  • remedy — for a run that couldn’t start for lack of usage credit, a key or a plan: add_key, buy_credits or upgrade.
  • setup_name — the setup test, for setup_failed on a setup step.

Get Test Run Logs

The app’s log lines from a test run, each tagged with the step it was written during.

GET /api/v1/tests/runs/{run_id}/logs?level=all&q=payment

Auth: Bearer token required. Any role can read it.

Query parameters:

• level — all (default), error, warn (or warning), info, or debug. Any other value returns 422.
• q — keep only lines whose message or tag contains this text (case-insensitive, up to 200 characters).
• include_system — true to include system lines (see below); false (default) leaves them out.

Response:

{
  "run_id": "tr7abc123x",
  "capturing": false,
  "available": true,
  "counts": { "all": 5, "error": 1, "warn": 1, "info": 2, "debug": 1 },
  "lines": [
    {
      "seq": 2,
      "ts": "2026-10-10T09:20:01.500Z",
      "ts_ms": 1760088001500,
      "offset_ms": 1500,
      "kind": "line",
      "level": "error",
      "tag": "ReactNativeJS",
      "source": "app",
      "message": "Payment failed: 502",
      "system": false,
      "step_index": 1
    }
  ],
  "lost": 0,
  "truncated": false,
  "system_lines": 3,
  "system_truncated": false,
  "related": {
    "step_index": 1,
    "lines": [
      {
        "seq": 2,
        "ts": "2026-10-10T09:20:01.500Z",
        "ts_ms": 1760088001500,
        "offset_ms": 1500,
        "kind": "line",
        "level": "error",
        "tag": "ReactNativeJS",
        "source": "app",
        "message": "Payment failed: 502",
        "system": false,
        "step_index": 1
      }
    ]
  },
  "empty_reason": null,
  "empty_message": null
}

• lines are in time order. offset_ms is the time since the run started; step_index is the step that had most recently started (null before the first step, and for a live run until its result is stored).
• A system line (system: true) was written by the operating system’s own frameworks running inside the app, not by the app’s code: on iOS and Apple TV, for example, network or launch-measurement messages; on Android and Android TV, lines from Android’s own components that run inside the app, such as media codecs, audio, graphics and the window system. Lines on Roku are never marked as system lines. System lines are left out unless include_system=true; system_lines says how many the run kept either way. Runs from before system lines were told apart show every line as the app’s.
• counts apply q and include_system but not level, so a level picker can show how many lines each choice would show.
• A line with kind: "gap" marks a point where the device’s log lost lines. It is not counted, stays visible under any filter, and no step is attributed lines across it.
• capturing is true while the run is queued, pending or running, and for a short while after it ends, until its last lines are collected. available is true once the log was collected for the run. lost counts lines that could not be captured. A run keeps up to 10,000 of the app’s lines and, separately, up to 2,000 system lines, so system lines never crowd out the app’s: truncated is true when the app’s lines hit their limit, system_truncated when the system lines hit theirs.
• related holds up to three of the app’s error and warning lines logged during the first failed step, nearest to the failure first (the run page shows them as “Logged during this step”), or null. It never holds a system line, and ignores level, q and include_system.
• empty_reason says why the run has no lines, with empty_message as a sentence to show; both are null when the run has lines or may still get them:
  • no_lines — the log was collected, and the app wrote no log lines during this run (system lines do not count).
  • expired — the run’s log was deleted: logs are kept for 30 days.
  • not_captured — no log was kept for this run (for example, it ran before VibeView kept run logs).

Errors:

• 404 — Run not found.
• 422 — Invalid level.

Get Test Run Video

The video of the device screen during a test run, or an Explore run.

GET /api/v1/tests/runs/{run_id}/video

Auth: Bearer token required. Any role can read it.

Response when the video is ready:

{
  "status": "ready",
  "message": null,
  "duration_ms": 48210,
  "run_start_ms": 1830,
  "truncated": false,
  "primary_panel_only": false,
  "size_bytes": 6120448,
  "expires_at": "2026-11-12T09:41:07",
  "segments": [
    {
      "index": 0,
      "url": "https://vibeview.io/api/v1/public/run-videos/...",
      "start_ms": 0,
      "duration_ms": 48210,
      "width": 1179,
      "height": 2556
    }
  ],
  "step_marks": [{ "step_index": 0, "at_ms": 2010 }]
}

• status is recording or processing while the run goes on and the video is saved, packaging while it is prepared for playback, then ready. It is failed when the video could not be saved, expired once it was deleted (videos are kept for 30 days), and none when the run has no video. Every status but ready returns only status and message, a sentence to show (or null).
• segments play one after another. A new segment starts when the screen changes size, for example on rotation. Each url is an MP4 file that works without a token, supports seeking, and stops working after 6 hours: call this endpoint again for fresh links.
• run_start_ms is where the run starts in the video, and step_marks say where each step started (at_ms from the start of the video).
• truncated is true when the recording stopped before the run ended, for example at its length or size limit. primary_panel_only is true on a device with two screens: the video shows the one in use.

Errors:

• 404 — Run not found.

Run Feedback

GET /api/v1/tests/runs/{run_id}/feedback
PUT /api/v1/tests/runs/{run_id}/feedback

Auth: Bearer token required. Any member of the organization can answer, Viewers included.

GET returns your own answer on the run: {"run": {"verdict": "up", "note": null, "updated_at": "..."}}, or {"run": null} when you haven’t answered.

PUT with {"verdict": "up" | "down" | null, "note": "optional, up to 2000 characters"} saves your answer to “Was the AI right?” and returns the same shape. Answering again replaces your answer; "verdict": null removes it. It changes nothing in the run.

Errors:

• 404 — Run not found.
• 409 — The run hasn’t finished yet.

Cancel a Run

Cancel any run by its id: a single test run, a group run, a run on one or more devices, or a multi-suite run from CI. A run on a device that is waiting leaves the queue at once; one that is running stops after its current step or test. For a run that belongs to a run on several devices, the id of one of its test or group runs cancels that device only; the run’s own id cancels every device.

POST /api/v1/tests/runs/{run_id}/cancel

Auth: Bearer token required. Developer role or above.

Response:

{
  "status": "cancellation_requested",
  "kind": "test_run",
  "left_queue": false
}

kind is what was cancelled. left_queue is true when the run was still waiting for a device and is already cancelled, false when it was running and stops after its current step or test.

Errors:

• 400 — Run is not active or already finished.
• 404 — Run not found.

Explore

Start an Explore run (beta): the AI uses the app on one device for a time budget and records findings. See Explore.

POST /api/v1/tests/explore

Auth: Bearer token required. Developer role or above. At most 5 starts a minute.

Response: 201 Created — {"id": "...", "batch_id": "...", "open_path": "/tests/runs/<id>", "run_url": "https://vibeview.io/tests/runs/<id>"}.

Errors: 403 with {"detail": {"message", "remedy"}} when VibeView-provided AI can’t run (out of credit, paused, no usable model); 404 app not found; 422 a budget other than 5, 10 or 20, or a field this route doesn’t take (Explore runs on one device, so a devices list is refused); 429 more than 5 starts in a minute.

GET /api/v1/tests/explore/{id}

Any role. Returns the run’s status (queued, pending, running, finished, stopped, cancelled, error), the reason it stopped (stop_reason and a readable stop_label), remedy for a stop you can fix (buy_credits, add_key, upgrade or check_key; otherwise null), the goal, budget, model, cost, the device it asked for (device) and the one it got, findings and the latest actions. Poll it while status is queued, pending or running. Explore runs are kept for 30 days.

Cancel with POST /api/v1/tests/runs/{id}/cancel.

GET /api/v1/tests/explore/{id}/findings/{finding_id}/draft

Any role. A test draft for the finding: name, steps ([{"text"}]), seedable and seed_note. Saves nothing.

POST /api/v1/tests/explore/{id}/findings/{finding_id}/seed

Developer role or above. Body {"test_id": "<test public_id>"}: after you saved the draft as a test (POST /api/v1/tests/cases), attach the route Explore took to the test’s unchanged steps. Response {"seeded": 3, "steps": 4, "reason": null}; reason says why nothing was attached when that is the case. 400 when the test is for another app.

Learned Replay Paths

See Learned replay paths. mode is auto (save automatically), ask (ask me first) or never; null means “use the app’s setting” on a test and “use the default (auto)” on an app. A path applies only to the platform, device type (phone, tablet, tv), orientation and foldable posture it was learned on.

Get a Test’s Learned Paths

GET /api/v1/tests/cases/{case_id}/learned-paths

Auth: Bearer token required.

Response: {"mode": null, "effective_mode": "auto", "app_mode": null, "default_mode": "auto", "platforms": [{"platform": "ios", "device_category": "phone", "orientation": "portrait", "posture": null, "steps": 3, "last_saved_at": "...", "source_run_id": "..."}], "pending": [ ...learned path objects... ]}

Set a Test’s Learned Path Mode

PUT /api/v1/tests/cases/{case_id}/learned-path-mode

Auth: Bearer token required. Developer role or above. Body: {"mode": "ask"} (or null). Response: as Get.

Forget a Test’s Learned Paths

DELETE /api/v1/tests/cases/{case_id}/learned-paths?platform=ios

Auth: Bearer token required. Developer role or above. platform (ios, android or roku) is optional; without it every platform is forgotten. Response: {"forgotten": 3}

Review a Route Change

POST /api/v1/tests/learned-paths/{path_id}/review

Auth: Bearer token required. Developer role or above. Body: {"decision": "save_new"}, "keep_old" or "bug". 409 when the path changed since (reload the run). Response: the learned path object: id, case_id, step_id, step_text, platform, device_category, orientation, posture, status (active, pending, superseded, rejected, forgotten), action_count, replaces_id, source_run_id, decision, device_name, created_at.

App Learned Path Setting and Figures

GET /api/v1/apps/{app_id}/learned-paths
GET /api/v1/apps/{app_id}/learned-paths?scope=family
PUT /api/v1/apps/{app_id}/learned-path-mode      # body {"mode": "ask"} or null; developer role or above

scope=family counts the figures over the tests of every linked app of the same form factor; mode is the same on every linked app. Without scope (or with scope=app) the figures are the app’s own.

Response: {"mode": null, "effective_mode": "auto", "default_mode": "auto", "steps_learned": 12, "replayed_steps_30d": 140, "ai_steps_30d": 31, "spend_avoided_micro_30d": 52000} (last 30 days; amounts in micro-credits, 1,000,000 = $1.00).

Suite Runs

Get Suite Run

Get details of a suite run including all linked test runs.

GET /api/v1/tests/suite-runs/{suite_run_id}

Auth: Bearer token required.

Response:

{
  "id": 1,
  "suite_id": 1,
  "session_id": "uuid-string",
  "status": "passed",
  "reset_strategy": "clear_data",
  "started_at": "2025-01-20T14:00:00",
  "finished_at": "2025-01-20T14:05:00",
  "duration_ms": 300000,
  "error_message": null,
  "test_runs": [
    {
      "id": 1,
      "test_case_id": 1,
      "status": "passed",
      "duration_ms": 15000,
      "error_message": null
    }
  ]
}