Before you start
- An app in qarunbook with a test checklist.
- For a website: its address online (a staging site is fine). For a phone app: a Limrun account, or your own computer with a simulator, an emulator or a phone plugged in.
- Money in your AI credit (Billing, AI testing). It is pay as you go: see below.
Run it
- In qarunbook
Press Run with AI
On the app's board, press Run with AI, or Run with AI on one section or one check. Pick what to test and which platforms; several platforms run side by side.
- In qarunbook
Answer what it asks, once
A platform that isn't set up asks a few questions, one at a time: where it should run, and where your app comes from. Each has Not sure which? with a plain explanation. A platform that is set up asks nothing.
- In qarunbook
Watch, or come back later
The run opens beside your board: the check it's on and every step on the left, the live screen on the right. When a check ends, its replay plays there. Results fill in on the board as each check finishes.
Where it can run
Websites
- Our browser: a fresh, private browser in our cloud, signed out, for every check. Nothing to install. It reaches addresses that are online, not ones only on your own computer (like
localhost). - Your own Chrome: your Chrome, signed in as you, for checks that need your account. Needs the helper on your computer and the QA Runbook Tester extension: download and unzip it, open
chrome://extensions, turn on Developer mode, click Load unpacked and choose the unzipped folder. The AI is told never to change your settings, password or payment details.
Phone apps
The device is yours
- Limrun: real Android phones and iPhones in the cloud, fresh for every run. Nothing to install; paste your Limrun API key once. Limrun bills you for the minutes used.
- A simulator on your computer: an iPhone simulator (needs a Mac with Xcode) or an Android emulator (any computer with Android Studio). Needs the helper.
- Your own phone: a real phone plugged into your computer by cable. Needs the helper. An Android phone needs USB debugging on; an iPhone needs a Mac and a one-time setup, below.
Your app comes from a file you upload once (an .apk, or a zipped iOS Simulator build), is built from your connected GitHub repository before each run, or is already on the phone.
Connect your computer
The helper is a small program that lets the AI tester use your computer's simulators, emulators, phones and Chrome. It works on macOS, Windows and Linux, needs no administrator password, and starts by itself when you log in.
- In qarunbook
Get your line
When a choice needs your computer, qarunbook says so and shows Show my connection line; or open App configuration, Test configuration, Connected computers, Connect a computer. Pick Mac, Windows or Linux.
- In Your terminal
Paste it
On a Mac, open Terminal; on Windows, PowerShell; on Linux, a terminal. Paste the line and press Enter. It takes about two minutes. qarunbook notices by itself and carries on.
A real iPhone
The first time, qarunbook shows a short checklist. Each step ticks itself as soon as your Mac sees it done:
- Connect your iPhone to the Mac with a cable, unlock it, and tap Trust. After that, the same Wi-Fi is enough.
- Turn on Developer Mode: Settings › Privacy & Security › Developer Mode, then let the phone restart.
- Sign in to Xcode (free from the Mac App Store): Xcode › Settings › Accounts, then + and your Apple ID. A free Apple ID works.
- Press Set up my iPhone. We put Apple's testing app on the phone, signed with your Apple account. A few minutes the first time; keep the phone unlocked. If the phone asks, trust the developer under Settings › General › VPN & Device Management.
Your app goes on the iPhone the usual way (TestFlight, the App Store or Xcode) and you pick it from the list, or you upload an .ipa built for that phone and the helper installs it.
Only your workspace
What it costs
The AI tester is pay as you go. The workspace owner adds money to the AI credit under Billing, AI testing (by card, through Stripe), and each check the AI works through takes its price from it:
- On qarunbook's AI, what the AI used for that check plus 50% — usually about 1.01¢ a check, never less than 0.5¢.
- 1¢ a check when your workspace uses its own AI key; the AI itself is then on your own provider's bill.
- API checks are free: requests to your backend need no AI to run.
A check counts when it ends passed, failed or couldn't check. A check that could not run at all takes nothing. A run that the balance cannot cover is not started, and says how much it needs. The balance never expires. Limrun minutes are billed by Limrun, not by us.
Use your own AI
By default the AI tester runs on qarunbook's AI, with nothing to set up. If your team must use its own AI account, for example because only an approved provider may see your app, connect your own key: your screenshots then go only to your provider.
- In qarunbook
Open Which AI runs your checks
In App configuration, Test configuration, choose Your own AI key. An admin or owner of the workspace can change it; it applies to every app in the workspace.
- In your provider
Get a key
OpenRouter is the easiest: one key reaches every listed model. OpenAI and Google Gemini work directly. A company provider that accepts OpenAI-style requests (such as Azure OpenAI) goes under Another provider.
- In qarunbook
Pick a model, paste the key, Check and save
The AI tester needs a model that can see screens and point at them, so pick one from the list. Before saving, qarunbook sends the model one small screenshot: a wrong key, an empty account or a model that cannot see screens is said there and then.
When your provider says no
What you get back
- Passed or Failed on the board, under “AI tester”. A fail raises an issue with what it saw, the last screen and the replay.
- Couldn't check when a step needs something the AI doesn't have, such as a real account or a second phone. It never guesses: those are left for a person.
- A replay of every check, and what each check and run took from your balance.