Turn Sentry errors into bugs — qarunbook API docs

Sentry sees what breaks in production before anyone reports it. Connect it to a live app and each new error is a bug here, in the same list as the ones people report — and the fix is checked by a person before Sentry is told it is done.

The shape of it

Sentry sees a new issue in the project you connect, and it passes your filters.
It becomes a bug on the app, raised by Sentry: its title first, then where it happened, the level, platform and environment, when it was first and last seen, how many people and events, the release, and a link back. The bug shows a Sentry · SHOP-WEB-1A link.
Someone fixes it and marks it fixed. A tester checks the fix on the live app and confirms it.
qarunbook resolves the issue in Sentry. If Sentry sees the error again, the bug reopens and the people on it are told.

It is the same bug as one raised by hand, so everything else follows: it is filed in Linear, Jira, GitHub or GitLab if the app is connected, it can be given people to fix it, Slack hears of it, and it counts towards Free's monthly allowance of issues.

Sentry is for apps in live mode, where a bug does not need a check. An app being tested before release asks which check failed, and an error in production cannot say — so switch the app to live under Settings first. If an app goes back to testing, Sentry's errors are not raised until it is live again.

Connect an app

In Sentry, go to Settings → Developer Settings → Custom Integrations → Create New Integration and choose Internal Integration. Name it qarunbook.
Give it permissions Project: Read and Issue & Event: Read & Write — write is what lets qarunbook resolve an issue. Under Webhooks, tick issue. Leave the webhook URL for now, and save.
Copy the integration's token and its client secret.
In qarunbook, open the app, choose Configure, then the Integrations tab, and find Sentry. You need to be an admin of the app.
Pick your Sentry — sentry.io in the US, sentry.io in the EU (de.sentry.io), or your own — give the organization slug and the token, choose Load projects, pick the project, paste the client secret, set the filters, and Connect.
Copy the integration webhook URL the card now shows, paste it into the internal integration's Webhook URL in Sentry, and save it again.

A user auth token works too, with the scopes project:read, event:read and event:write — but an internal integration's token belongs to the org rather than to a person, so it keeps working when people leave. The token and client secret are stored encrypted and never shown again; the card shows only their last four characters.

Or with an alert

Sentry tells an integration about an issue the moment it is created, when it has hit one person. To raise a bug only once an error has hit, say, fifty people, use an alert instead: set People affected, at least on the card, then in Sentry create an issue alert with a condition like the issue is seen by more than 50 users, and an action that sends it to qarunbook — either Send a notification via the internal integration (turn on Alert Rule Action in it), or the project's WebHooks integration pointed at the alert webhook URL on the card.

A delivery from the internal integration is signed: Sentry-Hook-Signature is an HMAC-SHA256 of the body under the client secret, and one that does not check out is refused. The WebHooks action signs nothing, so its URL carries an unguessable token instead, and Replace alert URL makes a new one at once if it gets out. However many times Sentry tells us about an issue, it becomes one bug.

What becomes a bug

Level — error and fatal by default. Warnings, info and debug messages can be let in too.
Environments — say production to leave staging out. When a list is set, an issue whose environment cannot be told is left out too.
People affected — 0 raises from the first. More needs an alert, as above, to tell qarunbook when the number is reached.
Only new issues — on by default: an issue Sentry already knew before you connected is not raised, even when an alert fires on it.

The bug's priority comes from the level: fatal and error are high, warning is medium, info and debug are low. Its platforms come from what the error ran on — a browser, a browser on a phone, iOS or Android — matched to the app's own platforms. When Sentry cannot tell, the bug affects the platforms you pick on the card, or every platform.

Keeping Sentry in step

Resolve in Sentry when the fix is confirmed — on by default. A fix is only done when someone has seen it hold on the live app.
Resolve as soon as it is marked fixed — off by default, for teams that would rather Sentry heard straight away.
Mark fixed when resolved in Sentry — on by default. Resolving the issue in Sentry marks the bug fixed here, ready for someone to confirm.
When Sentry unresolves an issue it had resolved — a regression — the bug is reopened, whoever reopened it is named, and the people on it are told. It happens once per regression, however many times Sentry says so.

Good to know

Nothing already in Sentry is copied across when you connect. Bugs come in as Sentry tells qarunbook about issues from then on.
At most 30 bugs an hour come in from Sentry for one app, so a release that breaks everything is one bad hour, not hundreds of bugs.
The card says when Sentry last got through and what came of it — raised QA-41, or why it was held back — which is the quickest way to check the setup.
The REST API returns the link on every issue as sentry (short_id and url), or null, and via is sentry.
Self-hosted Sentry must be reachable over https from the internet. Private and local addresses are refused, so the token is never sent inside a network.
Disconnecting stops new bugs at once; delete the internal integration in Sentry too. Bugs already raised keep their links, and connecting again never raises the same Sentry issue twice.