Run mobile tests on every pull request with GitHub Actions

Testing a mobile app in GitHub Actions usually means two compromises. The iOS half needs a macOS runner, which, on a private repository, GitHub bills at about ten times the Linux per-minute rate (GitHub’s runner pricing). The Android half boots an emulator inside the job, which adds minutes of boot time to every run and is a well-known source of flaky failures that have nothing to do with your change.

With VibeView the job only orchestrates. It builds or uploads your app, asks VibeView to run your tests on a device that is already in the cloud, and waits for the answer. Every job in the template runs on ubuntu-latest.

What each pull request gets

• One comment per platform. A row for each group, linked to the group’s run, and a list of its tests, each linked to its run page. A failed test also shows why it failed. The comment is edited in place on every push, so it never piles up, and it names the commit it tested.
• A failing job when a test fails. The run command exits 0 when every test passed, 1 when a test failed, and 2 when the run could not complete (no group with your tag on that platform, a timeout, a canceled run).
• A job summary. The workflow run also gets a job summary with the same content, including runs that aren’t on a pull request.
• Commit statuses, if you connect the repository. With the repository connected in VibeView’s GitHub integration, VibeView posts one status per group, named vibeview/<group> (<platform>), linked to its run. Make the ones that matter required in your branch protection rules, and a failing test blocks the merge.

Each test’s run page in VibeView has the step-by-step result, screenshots and the run video, so a reviewer can see what the test saw without checking out the branch.

Set it up

The smallest workflow that works

The template is long because it is built to be safe on a public repository and to produce a readable report. The core is short. This is a complete iOS-only workflow: it builds the app in the cloud, runs the tagged groups against that build, and fails when a test fails.

name: VibeView (iOS)

on:
  pull_request:

permissions:
  contents: read

concurrency:
  group: vibeview-${{ github.event.pull_request.number || github.ref }}
  cancel-in-progress: true

jobs:
  ios:
    # Pull requests from forks and from Dependabot get no secrets, so skip them.
    if: >-
      github.event.pull_request.head.repo.full_name == github.repository &&
      github.event.pull_request.user.login != 'dependabot[bot]'
    runs-on: ubuntu-latest
    timeout-minutes: 150
    env:
      VV_TAG: ${{ vars.VIBEVIEW_TAG }}
      HEAD_SHA: ${{ github.event.pull_request.head.sha }}
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: 22

      - name: Install the VibeView CLI
        run: npm install --global --no-audit --no-fund vibeview@0.58.0

      - name: Build in the cloud
        id: build
        env:
          VIBEVIEW_API_TOKEN: ${{ secrets.VIBEVIEW_API_TOKEN }}
        run: |
          vibeview build --cloud --platform ios --json > build.ndjson || true
          build_id=$(jq -rs '[.[] | select(.event == "build_finished" and .status == "succeeded")]
                             | last | .app_build_id // empty' build.ndjson)
          if [ -z "$build_id" ]; then
            jq -rj 'select(.event == "build_log") | .content' build.ndjson | tail -n 100
            echo "::error::The iOS cloud build failed."
            exit 1
          fi
          echo "build_id=$build_id" >> "$GITHUB_OUTPUT"

      - name: Run the tagged groups
        env:
          VIBEVIEW_API_TOKEN: ${{ secrets.VIBEVIEW_API_TOKEN }}
          BUILD_ID: ${{ steps.build.outputs.build_id }}
        run: |
          exec vibeview run-suites --tag "$VV_TAG" --platform ios --build "$BUILD_ID" \
            --mode hybrid --commit-sha "$HEAD_SHA" --timeout 1800

A few of these lines do more than they appear to:

• --build pins the run to the build this job just made. Without it, a run uses the app’s latest runnable build, which on a busy repository may be someone else’s.
• --commit-sha is what turns the result into a commit status on the pull request, once the repository is connected in VibeView’s GitHub integration.
• exec makes the CLI the step’s own process, so when a newer push cancels this job, the CLI is stopped and cancels the run on VibeView instead of leaving it to finish.
• --mode hybrid is Smart in the dashboard: it replays recorded and learned paths, and uses AI only where the screen changed or a step has no path yet (beta). --mode replay (No AI) never uses AI at all.
• timeout-minutes: 150 is longer than the cloud build and the CLI’s own wait combined, so the CLI gives up first and cleans up.

The template adds an Android build and test job, the report job that writes the pull request comments, and a step that cancels the cloud build if the job is canceled. It also skips docs-only pull requests and keeps the VibeView token out of every step that runs your project’s build scripts. Start from the template for a real repository.

Android, Apple TV, Android TV and Roku

Android is built by your own job in the template, so it can use whatever Gradle setup you already have. That job has no VibeView token, so a pull request’s build scripts can never read it. A second job uploads the APK with vibeview upload-app and runs the Android groups against the new build. The APK must include the x86_64 ABI, and a React Native APK must be a release build because, as on iOS, a debug build waits for Metro. To build Android in the cloud instead, give the android entry of vibeview.json a build block and use the iOS job’s steps with --platform android (with --profile preview for React Native or Expo).

Apple TV and Android TV use a copy of a platform job with --platform tvos or --platform androidtv. The CLI picks a TV device for a TV group on its own, and the report labels the comment Apple TV or Android TV. An Android TV APK needs both the x86 and x86_64 ABIs.

Roku channels are packaged in your own job (the zip you would sideload) and uploaded the same way:

# VV_TAG and HEAD_SHA come from the job's env:, as in the workflow above.
BUILD_ID=$(vibeview upload-app ./out/channel.zip --output json | jq -er '.build_id')
vibeview run-suites --tag "$VV_TAG" --platform roku --build "$BUILD_ID" --commit-sha "$HEAD_SHA"

The full instructions for each, including a macOS-runner build if you’d rather build iOS yourself, are in CI testing.

Secrets: what goes where

There is exactly one VibeView secret in GitHub: VIBEVIEW_API_TOKEN. Everything else lives in VibeView.

• Test logins stay in VibeView, not in the workflow. A test that signs in types {{password}}, not the password. You keep the value in the app’s test data and mark it secret: it is never put in the AI’s prompts or the run’s results and logs, and VibeView types it into the device for the step. If the value is a one-time code (TOTP) setup key, VibeView types the current six-digit code, so a test can get past two-factor sign-in. There is no workflow input that feeds a GitHub secret into test data, and you don’t need one. See Test data and secrets.
• Build secrets are stored once, from your machine. An API key or a signing file the iOS cloud build needs goes in with vibeview env set NAME (stored as secret by default, never readable back). A CI-only token can’t manage build secrets, so set them with your own login, not from the workflow. See Build variables and secrets.
• Forks and Dependabot are skipped. GitHub gives their pull requests no secrets, so the jobs are skipped instead of failing. Never switch the workflow to pull_request_target to test them: that runs the pull request’s code with your token. Push a reviewed fork change to a branch of your own instead.
• No ${{ }} inside run:. The template passes every value to its scripts through env:, so text from a pull request can’t inject shell commands. Test names and error messages can quote text from your app’s screen, so the template only adds them to the comment as escaped Markdown.

Already building every pull request with the GitHub App?

The VibeView GitHub App builds each pull request in the cloud with no workflow file, and its comment opens the build on a device. It doesn’t run your tests. Use this workflow when you want the test results on the pull request. If you only want the tests to run on each new build and report in VibeView, use a schedule set to On every new build instead.

What it costs

• Each group run uses a live device session, which counts toward your concurrent sessions and streaming minutes like any other session. When every session your plan allows is in use, the run waits for one (up to 30 minutes) instead of failing.
• Smart uses AI only where it’s needed (see above). Steps that use the AI agent (beta) are billed to usage credit, unless you use your own AI provider key. A No AI run uses no usage credit. Tests that pass with AI save the path they took, so the next run of the same step on the same kind of device replays it instead of asking the AI again; see Tests that stop paying for AI.
• The iOS cloud build is billed like any cloud build.
• To save minutes, the template skips pull requests that only change Markdown or docs/, and a new push cancels any run still in progress for the previous push.

Troubleshooting

• “The repository secret VIBEVIEW_API_TOKEN is not set.” The template checks the secret and the VIBEVIEW_TAG variable before doing anything. Add them under Settings > Secrets and variables > Actions. On a pull request from a fork, GitHub hides secrets, which is why the template skips forks.
• The job exits with code 2 before any test runs. Usually no group with that tag exists on that platform, or a tagged group belongs to a different app than the build. List what the tag matches with vibeview list-suites --tag smoke.
• Every test fails on a red error screen. It’s a React Native or Expo Debug build, which needs a Metro dev server. The CLI prints a warning when such a build starts a run. Build the preview profile (Release) for CI.
• The Android build won’t install. The APK must include x86_64 (and x86 for Android TV). An arm64-only APK is the usual cause.
• The run says it is queued. Every concurrent session your plan allows is busy, often because several pull requests pushed at once. The run starts when a session frees up. If streaming minutes for the month are used up and overage is off, the run is refused with the same message the dashboard shows.
• The job hit timeout-minutes before the CLI finished. Keep each job’s timeout above the CLI’s own wait (--timeout per group, times the number of groups, capped at 2 hours), plus the cloud build for iOS, so the CLI stops first and cancels the run.
• A test fails once and passes on the next push. Pass --retry to retry a failed test once, after a fresh start of the app. A test that passes on the retry passes the run and is marked flaky, so it stays visible on the Tests page.

Frequently asked questions

Can I run iOS tests in CI without a Mac?

Yes. The template builds the iOS app on VibeView’s macOS builders and runs the tests on a cloud iOS simulator, so every job runs on ubuntu-latest. If you’d rather build on your own macOS runner, the CI testing docs show the job to swap in.

Is this an alternative to running the Android emulator in GitHub Actions?

Yes. No emulator runs in your job. The tests run on an Android emulator in VibeView’s cloud, so the job needs no hardware acceleration, no system image download and no boot wait. The job only uploads the APK and waits for the result.

Do I have to write the tests in code?

No. A test is a list of plain-language steps, recorded on a device or written in a sentence and reviewed. The same test runs on iOS and Android phones (and on Apple TV, Android TV and Roku for a TV app) when your apps are linked as one product.

Does it work with GitLab CI or CircleCI?

Yes. The CLI runs on any CI that runs shell commands, and --output junit writes a report those systems display. See the GitLab CI and CircleCI pages.

Can a coding agent set this up for me?

Yes. On the Tests page, open CI and click Get the prompt to run them in CI for a prompt to give your coding agent. The agent asks which tag to run, adds this workflow and shows it to you before it commits. You add the VIBEVIEW_API_TOKEN secret yourself: the agent never writes a token into a file.