VibeView keeps a short summary of how your tests are doing. It is on each group and on the Runs section of the Tests page.
Which runs count
A run counts when it tells you something about your app or your test: it passed, or it failed because a check failed, the screen didn’t match, a step couldn’t be done or the app crashed. Cancelled runs, runs stopped because usage credit ran out, and runs that ended because the device session ended don’t count toward pass rates, run counts, durations, Needs attention or flaky tests.
Group summary
Opening a group shows, for the last 30 days:
- Pass rate — counted runs that passed, out of all counted runs.
- Runs — counted runs of the group’s tests, on their own or in a group run.
- Avg duration — the average time of counted runs.
- Cost — what the group’s runs cost in the window, including cancelled ones. Runs on your own AI key show an estimate.
- Tests — how many tests the group has.
- Flaky tests — how many of its tests are flaky now.
A value with no data shows —, never 0%.
Flaky tests
A test is flaky when its last 10 counted runs include both a pass and a failure. A test that passes again 10 times in a row is no longer flaky. When a run makes a test flaky, the run.warning.flaky notification is sent once; it is sent again only after the test has been stable and turns flaky again. Quarantined tests don’t send it. See Webhooks and Slack.
Needs attention
A test needs attention when one of its last 3 counted runs failed. Each one shows the first line of why the run failed, and links to that run. A quarantined test is left out.
Slow tests
Sort the test list by Average duration to find the slowest tests.
Analytics API
GET /api/v1/tests/suites/{suite_id}/analytics?days=30 returns the group summary. days is 1 to 365 and defaults to 30.
{
"suite": { "public_id": "a1b2c3", "name": "Checkout", "case_count": 4 },
"stats": {
"window_days": 30,
"total_runs": 12,
"pass_rate": 0.917,
"avg_duration_ms": 41000,
"total_cost_micro": 90000,
"real_cost_micro": 90000,
"est_cost_micro": 0,
"case_count": 4,
"flaky_count": 1
}
}The suite object also carries the group’s description, app, tags, start state and screenshot-check thresholds; it is trimmed here.
pass_rate is null when no counted run finished in the window, and avg_duration_ms is null when none had a duration. flaky_count is null until one of the group’s tests has a counted run. Costs are in micro-credits: 1,000,000 is $1.00, and est_cost_micro is the estimate for runs on your own AI key.