# Send test results from CI

> Add one step to any CI job so By Dawn records its test results and coverage as a test run, with saw-agent tests submit.

This page shows you how to send a CI job's test results to By Dawn. By the end, each pipeline run shows up as one test
run on its ticket and pull request, however many jobs and shards it took, and By Dawn's worker gets the failing tests
when CI is red.

The command is `saw-agent tests submit`. It runs after your tests, in the same job, and sends the report files your
test runner already wrote: JUnit XML, TRX, CTRF or `go test -json` results, and lcov, Cobertura, Go coverprofile or
coverlet coverage. By Dawn reads them itself, so you don't convert anything. Your tests still run on your CI, never on
By Dawn's servers. The command only carries what the runner wrote. (You may see it written `saw tests submit`. It's the
same command, in the `saw-agent` binary.)

If your runner doesn't write one of those files yet, [Make your test runner write reports](https://docs.bydawn.com/developers/test-reports/)
has the flag for each runner. If your repo is on GitHub and already uploads its reports as workflow artifacts, By Dawn
can [pick them up](https://docs.bydawn.com/tests-github-artifacts/) with no CI change at all. [Test results](https://docs.bydawn.com/tests/) compares the ways in.

## Before you start

You need three things:

- **An API key with the Tests access.** An owner or admin creates it in **Settings › Organization › Security**,
  on the **API keys** card, with **Access** set to **Tests — submit results** ("Sends test results from CI
  (saw-agent tests submit) — no board access."). That key's only scope is `tests:write`: it can send test runs and
  can't read or change anything else, so it's safe to hand to CI. Store it as a CI secret named `SAW_API_KEY`.
  [API keys](https://docs.bydawn.com/api-keys/) walks through creating one.
- **Your project's ID**, such as `project-3a86…`. Store it as a CI variable named `SAW_PROJECT`. The app doesn't show
  project IDs yet. List your projects with a board key in [Try it](https://docs.bydawn.com/developers/try-it/), or with
  `GET /orgs/{orgId}/projects` as in the [Quickstart](https://docs.bydawn.com/developers/quickstart/), and copy the project's `id`.
- **The repo connected to that project**, under the same name. By Dawn finds the repo by name, and by default the
  command uses your CI's repository name (`saw-ui` for `LenderCom/saw-ui`). If the repo has a different name in
  By Dawn, pass `--repo`.

Test results also have to be on for your organization. If they aren't, the upload is refused with
`entitlement_required` ([Turn on test results](https://docs.bydawn.com/tests/#turn-on-test-results)).

## Install it

The CI install is one line. It downloads the newest stable release for Linux x64 into the working directory:

```sh
curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
```

The archive holds a single `saw-agent` binary. Swap `linux_amd64` for `linux_arm64`, `darwin_amd64` or
`darwin_arm64` to match your runner. There's no Windows build of this archive, so run the step on a Linux or macOS
runner. Pairing the agent with your organization, as a [remote agent](https://docs.bydawn.com/remote-agents/) needs, isn't required: this
command needs only the key.

> **Until the next stable release:** `tests submit` is new, and the newest stable release (`v0.1.2`) doesn't have it yet, so the line above fails until
> the next stable release is out. Until then, install the newest pre-release with the GitHub CLI. In GitHub Actions,
> give the step `GH_TOKEN: ${{ github.token }}`.
> 
> ```sh
tag=$(gh release list --repo LenderCom/saw-agent-releases --limit 1 --json tagName --jq '.[0].tagName')
gh release download "$tag" --repo LenderCom/saw-agent-releases \
  --pattern 'saw-agent_*_linux_amd64.tar.gz' --output - | tar -xz
```

## Add it to your CI

Add the step after your tests. Make it run when the tests fail, too, because a red job's failures are the results
worth having. Each tab sets the key and project from your CI's secret store, and sets `SAW_API_URL`: the command
sends to `https://workspace.sawrun.com` unless you set it, and these docs use the dev workspace,
`https://workspace.dev.sawrun.com`. If By Dawn already picks up your CI's results by itself
([GitHub Actions](https://docs.bydawn.com/tests-github-artifacts/), or the others in [Set up your CI](https://docs.bydawn.com/tests-ci/)), you don't need the step
as well.

**GitHub Actions**

Add a step to the job that runs your tests. `!cancelled()` runs it whether the tests passed or failed. In a sharded
matrix, give each shard its place with `--shard`. (On GitHub Actions you can skip the key and the project ID
altogether: [sign in with GitHub's OIDC token](#sign-in-with-githubs-oidc-token) instead.)

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3]
    steps:
      - uses: actions/checkout@v4
      - run: npx vitest run --shard=${{ matrix.shard }}/3 --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml
      - name: Send test results to SAW
        if: ${{ !cancelled() }}
        env:
          SAW_API_KEY: ${{ secrets.SAW_API_KEY }}
          SAW_PROJECT: ${{ vars.SAW_PROJECT }}
          SAW_API_URL: https://workspace.dev.sawrun.com
        run: |
          curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
          ./saw-agent tests submit --results 'reports/*.xml' --shard ${{ matrix.shard }}/3
```

**GitLab CI**

`after_script` runs whether `script` passed or failed. Store `SAW_API_KEY` as a masked CI/CD variable. `parallel`
shards are detected for you:

```yaml
test:
  parallel: 3
  variables:
    SAW_API_URL: https://workspace.dev.sawrun.com
  script:
    - npx vitest run --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml
  after_script:
    - curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
    - ./saw-agent tests submit --results 'reports/*.xml'
```

**Azure Pipelines**

`succeededOrFailed()` runs the step after a failed test step. Azure Pipelines doesn't pass secret variables to
scripts on its own, so map `SAW_API_KEY` in `env`:

```yaml
- script: npx vitest run --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml
  displayName: Test
- script: |
    curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
    ./saw-agent tests submit --results 'reports/*.xml'
  displayName: Send test results to SAW
  condition: succeededOrFailed()
  env:
    SAW_API_KEY: $(SAW_API_KEY)
    SAW_PROJECT: $(SAW_PROJECT)
    SAW_API_URL: https://workspace.dev.sawrun.com
```

A job with `parallel:` is detected as shards.

**Jenkins**

In a declarative pipeline, send the results from `post { always { … } }`, with the key from a Jenkins secret text
credential:

```groovy
pipeline {
  agent any
  environment {
    SAW_PROJECT = 'project-3a86…'
    SAW_API_URL = 'https://workspace.dev.sawrun.com'
  }
  stages {
    stage('Test') {
      steps {
        sh 'npx vitest run --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml'
      }
    }
  }
  post {
    always {
      withCredentials([string(credentialsId: 'saw-api-key', variable: 'SAW_API_KEY')]) {
        sh '''
          curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
          ./saw-agent tests submit --results 'reports/*.xml' --job test
        '''
      }
    }
  }
}
```

The build's `BUILD_TAG` ties every stage of one build into one run. Name the job with `--job`, because a `post`
block isn't inside a stage.

**CircleCI**

`when: always` runs the step after a failed test step. Set `SAW_API_KEY` and `SAW_PROJECT` in the project's
environment variables or a context. `parallelism` is detected as shards:

```yaml
jobs:
  test:
    docker:
      - image: cimg/node:22.11
    parallelism: 3
    environment:
      SAW_API_URL: https://workspace.dev.sawrun.com
    steps:
      - checkout
      - run: npx vitest run --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml
      - run:
          name: Send test results to SAW
          when: always
          command: |
            curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
            ./saw-agent tests submit --results 'reports/*.xml'
```

Every job in one workflow shares the workflow's ID, so a workflow's test jobs become one run.

**Buildkite**

Buildkite steps don't share files, so send the results in the step that ran the tests. Keep the tests' exit code,
so a red test step stays red:

```yaml
steps:
  - label: "test"
    parallelism: 3
    env:
      SAW_API_URL: https://workspace.dev.sawrun.com
    command: |
      status=0
      npx vitest run --shard=$((BUILDKITE_PARALLEL_JOB + 1))/$BUILDKITE_PARALLEL_JOB_COUNT --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml || status=$?
      curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
      ./saw-agent tests submit --results 'reports/*.xml' --allow-missing
      exit $status
```

Set `SAW_API_KEY` and `SAW_PROJECT` in the agent's environment hook or your secrets plugin. `parallelism` is
detected as shards, and a retried job is sent as a new attempt.

When it works, the step prints one line for each run it made:

```text
saw-agent tests submit: run trun-7c1e… group tgrp-41d0… state=ready tests=412 passed=409 failed=2 skipped=1 flaky=1 reports=r0=parsed
```

`state` is `ready` once By Dawn has read the reports, `ingesting` while it still is, or `failed` if no report could be
read. `flaky` counts tests that failed and then passed on a retry, and they're counted in `passed` too. Each part
after `reports=` is one file you sent: `r0`, `r1` and so on for results, `c0` and on for coverage. A part By Dawn
couldn't read says `rejected`, and the run's other files still count.

## Sign in with GitHub's OIDC token

On GitHub Actions the command can send results with no By Dawn key stored anywhere. GitHub gives every job that asks
for one a signed OIDC token: a short-lived JSON Web Token naming the job's repository, branch and run. When
`SAW_API_KEY` isn't set and the job has `permissions: id-token: write`, the command asks GitHub for a token for the
`sawrun` audience and exchanges it at `POST /auth/token/github-oidc`. By Dawn checks GitHub's signature, then looks for a
**GitHub Actions trust** your organization added for that repository. If one matches, By Dawn issues the same 15-minute
access token a Tests API key would get.

An owner or admin adds the trust in [API keys](https://docs.bydawn.com/api-keys/#github-actions-trusts). It names the repository, the By Dawn
project its results go to, and optionally a branch or environment. The trust names the project, so `--project` is
optional. Pass it anyway if the repository is trusted for more than one project. By Dawn shows the step to paste when you
add the trust.

```yaml
jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   # lets the job ask GitHub for its OIDC token
      contents: read
    steps:
      - uses: actions/checkout@v4
      - run: npx vitest run --reporter=default --reporter=junit --outputFile.junit=reports/junit.xml
      - name: Send test results to SAW
        if: ${{ !cancelled() }}
        env:
          SAW_API_URL: https://workspace.dev.sawrun.com
        run: |
          curl -fsSL https://github.com/LenderCom/saw-agent-releases/releases/latest/download/saw-agent_linux_amd64.tar.gz | tar -xz
          ./saw-agent tests submit --results 'reports/*.xml'
```

- **`permissions: id-token: write`** is what lets GitHub hand the job a token. Setting `permissions` replaces the
  defaults, so list `contents: read` too, or `actions/checkout` can't read the repository. Give it to the job that
  sends results, not the whole workflow.
- **A trust with an environment** only admits a job that declares that `environment:`. GitHub puts the
  environment in the token only for a job that declares one.
- **`--auth github-oidc`** insists on the token, and `--auth api-key` on the key. By default the key wins when
  `SAW_API_KEY` is set.

### What a trust admits

A trust with no branch or environment admits any workflow in the repository, the same reach a repository secret
holding a key would have. A `pull_request` workflow started from a fork gets no token, so it can't send results.
A `pull_request_target` or `workflow_run` workflow is different. It runs in your repository, on your default branch,
with the permissions you give it, so it gets a token that matches a `main` trust. If such a job checks out and runs
the pull request's code, that code can send results as you. So:

- **Don't give `id-token: write` to a job that runs a fork's code.**
- **Do use a protected environment, with required reviewers, to narrow a trust.** A branch filter narrows a trust to
  the runs on that branch, but a `pull_request_target` run is on your default branch too.
- **Add a new trust after renaming or re-creating the repository on GitHub.** A trust matches the repository by
  its name, and from its first use by its GitHub ID too. So a renamed repository no longer matches, and one deleted
  and made again under the same name isn't trusted.

### When the token is refused

A refusal exits `4` and names the fix:

| The command says | Means |
|---|---|
| "this job has no GitHub OIDC token … give the job `permissions: id-token: write`" | The job has no `id-token: write`, or it's a `pull_request` run from a fork. With `--project` or `SAW_PROJECT` set, it says "SAW_API_KEY is not set" instead, and names the same permission. |
| "SAW refused the GitHub OIDC token … `github_oidc_no_trust`" | No trust admits this repository on this branch or environment. Check the trust's repository, branch and environment, that the job declares the environment, and that the trust isn't revoked or expired. |
| "… `github_oidc_ambiguous_trust`" | The repository is trusted for more than one project. Pass `--project`. |
| "… `github_oidc_invalid_token`" | The token isn't GitHub's, has expired, or was requested for another audience. The message says which check failed. |

## How the uploads become one run

Every job and every shard sends its own upload. By Dawn puts together the uploads that tested the same commit in the
same pipeline run, so a run split across three shards and a coverage job reads as one test run on the ticket.

- **The run ID** (`--run`) ties them together. It defaults to your CI's ID for the whole pipeline run, such as
  GitHub's run ID or GitLab's pipeline ID, so jobs of one pipeline match without you doing anything.
- **The attempt** (`--attempt`) keeps a re-run apart. Re-running a pipeline on the same commit is a new attempt,
  and By Dawn shows it as its own run rather than mixing old and new results.
- **The job and shard** (`--job`, `--shard`) tell uploads within a run apart. Sending the same repo, commit, run ID,
  attempt, job and shard again gives you back the run By Dawn already made instead of a second one. That's what makes a
  retry safe, and why each matrix leg needs its own job or shard (see [Do and don't](#do-and-dont)).

When By Dawn [launched the run](https://docs.bydawn.com/tests-launch/), it handed the workflow the launch's ID as the `saw_launch_id` input.
Put it in the step's `env`, such as `SAW_LAUNCH_ID: ${{ inputs.saw_launch_id }}`, and pass it as
`--run "$SAW_LAUNCH_ID"`: every shard then shares it, and the run fills that launch. Don't write
`${{ inputs.saw_launch_id }}` into the `run:` script itself, since GitHub pastes it in before the shell reads it and
a crafted input could run commands next to your key. On a run nobody launched the input is empty, and `--run` falls
back to your CI's ID.

By Dawn ties the run to its pull request by the pull request's number, or else by its branch, and from there to the
pull request's ticket. Pass `--ticket` only if you need to name the ticket yourself.

One upload holds one results format. If your globs find JUnit and TRX files, the command sends one upload per format
and adds the format to the job's name, such as `unit (junit)` and `unit (trx)`. Coverage files go with the first.

## What it reads from your CI

The command recognises six CI systems from their own environment variables and fills in everything else from them.
A flag you pass always wins over what it detects. These are the values that decide how uploads fit together:

| CI | Run ID | Attempt | Shards |
|---|---|---|---|
| **GitHub Actions** | `GITHUB_RUN_ID` | `GITHUB_RUN_ATTEMPT` | — (pass `--shard`) |
| **GitLab CI** | `CI_PIPELINE_ID` | — | `CI_NODE_INDEX` of `CI_NODE_TOTAL` |
| **Azure Pipelines** | `BUILD_BUILDID` | `SYSTEM_JOBATTEMPT` | `SYSTEM_JOBPOSITIONINPHASE` of `SYSTEM_TOTALJOBSINPHASE` |
| **Jenkins** | `BUILD_TAG` | — | — |
| **CircleCI** | `CIRCLE_WORKFLOW_ID` | — | `CIRCLE_NODE_INDEX` of `CIRCLE_NODE_TOTAL` |
| **Buildkite** | `BUILDKITE_BUILD_ID` | `BUILDKITE_RETRY_COUNT` + 1 | `BUILDKITE_PARALLEL_JOB` of `BUILDKITE_PARALLEL_JOB_COUNT` |

The job's name comes from `GITHUB_JOB`, `CI_JOB_NAME`, `SYSTEM_JOBDISPLAYNAME`, `STAGE_NAME` (else
`JOB_BASE_NAME`), `CIRCLE_JOB` or `BUILDKITE_LABEL`. The commit, branch and pull request number come from each CI's
usual variables, such as `GITHUB_SHA`, `CI_COMMIT_BRANCH` or `CHANGE_ID`. On a pull request or merge request build
(GitHub, GitLab and Azure Pipelines), the command sends the pull request's own head commit and branch, not the
temporary merge commit the CI checked out, so the run lands on the commit By Dawn knows.

It also takes the repository's name, the checkout's directory and a link back to the CI run from the same place, and
the link shows as **Open CI run** on the run in By Dawn. Where a CI has no attempt, it's 1. Outside a CI it recognises,
the commit comes from `git rev-parse HEAD` and the run is recorded as an upload.

## Flags

Only `--project` (or `SAW_PROJECT`) and at least one `--results` or `--coverage` are required. With GitHub OIDC,
`--project` is optional too, because the trust names the project.

| Flag | Environment | What it does |
|---|---|---|
| `--project <id>` | `SAW_PROJECT` | The By Dawn project the run belongs to. |
| `--results <glob>` | | Results files. Repeat it, or separate globs with commas. `**` matches any depth. |
| `--coverage <glob>` | | Coverage files, the same way. |
| `--repo <name>` | `SAW_REPO` | The repo's name in By Dawn. Defaults to your CI's repository name, then the `origin` remote's. |
| `--sha <sha>` | | The commit tested, all 40 (or 64) characters. Defaults to your CI's, then `git rev-parse HEAD`. |
| `--branch <name>` | | The branch. Defaults to your CI's. |
| `--pr <number>` | | The pull request's number. Defaults to your CI's. |
| `--job <name>` | | The job's name. Defaults to your CI's. Give each matrix leg its own. |
| `--run <id>` | | The ID that ties uploads into one run. Defaults to your CI's pipeline run ID. In a run By Dawn launched, pass the `saw_launch_id` input. |
| `--launch <id>` | | The By Dawn launch these results fill. Defaults to `--run` when it's a launch's ID, such as `tlaunch-6d0f…`. |
| `--attempt <n>` | | The run's attempt. Defaults to your CI's, else 1. |
| `--shard <i>/<n>` | | This upload's shard, counting from 1, such as `2/3`. At most 256 shards. |
| `--format <f>` | | Skip format detection for the results files: `junit`, `trx`, `ctrf` or `go-test-json`. |
| `--coverage-format <f>` | | The same for coverage: `lcov`, `cobertura`, `go-coverprofile` or `coverlet-json`. |
| `--runner <r>` | | The runner that wrote the results: `vitest`, `jest`, `bun`, `playwright`, `pytest`, `go`, `dotnet` or `junit`. By Dawn names tests the way that runner does. |
| `--root <dir>` | | The repo-relative directory the runner's paths start from, such as a monorepo package or Playwright's `testDir`. |
| `--checkout-dir <dir>` | | Where the repo is checked out, so By Dawn can turn absolute paths into repo paths. Defaults to your CI's workspace, then git's top level. |
| `--variant <v>` | | A variant for every result, such as `os=windows`, so a matrix leg's tests stay apart from the others'. |
| `--ticket <id>` | | The ticket the run belongs to. By default By Dawn ties it by branch or pull request. |
| `--api <url>` | `SAW_API_URL` | By Dawn's workspace API. Defaults to `https://workspace.sawrun.com`. |
| `--accounts-url <url>` | `SAW_ACCOUNTS_URL` | By Dawn's accounts API, where the key is exchanged. Defaults to the workspace host with `accounts.` in place of `workspace.`. |
| `--auth <mode>` | | `api-key`, `github-oidc` or `auto` (the default): the key when `SAW_API_KEY` is set, else the GitHub Actions job's OIDC token, as [Sign in with GitHub's OIDC token](#sign-in-with-githubs-oidc-token) describes. |
| `--allow-missing` | | A glob that matches nothing warns instead of failing. |
| `--dry-run` | | Print what would be sent, and send nothing. Needs no key. |
| `--json` | | Print each run By Dawn made as JSON. |

The key always comes from `SAW_API_KEY`. The command exchanges it for a 15-minute access token, as
[Authentication](https://docs.bydawn.com/developers/authentication/) describes, and prints neither. On GitHub Actions with no key, the
command can sign in with the job's OIDC token instead, and then `--project` is optional
([Sign in with GitHub's OIDC token](#sign-in-with-githubs-oidc-token)).

The command works out each file's format from its first few kilobytes, so `--format` is only for a file it can't
place. A file over 20 MB, or more than 32 files in one upload, is refused before anything is sent.

## Try it before CI does

`--dry-run` shows what the command would send, from your own machine, without a key:

```sh
./saw-agent tests submit --project project-3a86… --results 'reports/**/*.xml' --coverage coverage/lcov.info --dry-run
```

```text
upload 1 of 1 (dry run — nothing sent)
meta: {
  "repoName": "web",
  "sha": "4f9c0d1e…",
  "source": "upload",
  "attempt": 1,
  "reports": [ … ]
}
  r0  junit           reports/unit/junit.xml
  c0  lcov            coverage/lcov.info
```

Outside a CI it recognises, the command names the repo after your `origin` remote and records the run as an upload
rather than a CI run.

## Exit codes and errors

Every failure prints one line that starts `saw-agent tests submit:` and says what went wrong.

| Exit code | Means | For example |
|---|---|---|
| `0` | Sent. Or, with `--allow-missing`, nothing was found to send. | |
| `2` | A flag is wrong, or the project, repo or commit couldn't be found. | `no project — give --project <id> or set SAW_PROJECT (the SAW project's id)` |
| `3` | A glob matched no file. | `"coverage/lcov.info" matched no file (--allow-missing to send what was found)` |
| `4` | No key, or By Dawn refused it (or refused the GitHub OIDC token). | `SAW_API_KEY is not set — an org API key with the tests:write scope, or on GitHub Actions …` |
| `5` | By Dawn refused the upload. Its error slug and a hint follow. | `SAW refused the upload: no repo named nope in the project (HTTP 422 validation_error) — check --repo is the repo's name in SAW` |
| `6` | By Dawn couldn't be reached, or stayed busy. | `cannot send the reports after 5 tries: HTTP 503` |
| `7` | A file is over 20 MB or isn't a format By Dawn reads, or an upload would carry over 32 files. | `reports/out.xml is not a report format SAW reads — give --format if it is one` |

The command retries a dropped connection, a `429` and any `5xx` itself, up to five tries, waiting 1, 2, 4 and then 8
seconds, or as long as `Retry-After` asks. Retrying is safe, because By Dawn answers a repeated upload with the run it
already made. A refusal (exit 5) isn't retried, because sending it again would get the same answer:

- `entitlement_required`: test results aren't on for your organization.
- `forbidden`: the key doesn't have the Tests access, or the project is restricted. A key never reaches a
  [restricted project](https://docs.bydawn.com/project-access/).
- `404 not_found`: `--project` isn't a project ID in your organization.
- `422 validation_error`: the message says which field. Most often the repo's name doesn't match By Dawn's.

## Do and don't

- **Do quote your globs** (`--results 'reports/*.xml'`). Unquoted, the shell expands them first, and the command
  stops with "quote a glob so the shell does not expand it".
- **Do run the step when tests fail** (`!cancelled()`, `after_script`, `succeededOrFailed()`, `post { always }`,
  `when: always`). A failing run is the one By Dawn's worker needs.
- **Decide whether the step may fail the job.** It exits non-zero when it can't send, which you'll want to see while
  you set it up. Once it works, add `|| true` (or GitHub's `continue-on-error: true`) if a By Dawn outage should never
  turn a green job red.
- **Do use `--allow-missing` where the tests may not run at all**, such as a job that fails during install. Without
  it, a glob that matches nothing exits 3, and the job's log shows that rather than the real failure.
- **Do give each matrix leg its own job, shard or variant.** On GitHub Actions, every leg of a matrix shares one
  `GITHUB_JOB`. Two legs that send the same job and shard read as a retry of one upload, and By Dawn keeps only the
  first. Pass `--shard` for shards of one suite, or `--job "test (${{ matrix.os }})"` and
  `--variant os=${{ matrix.os }}` for legs that each run the whole suite.
- **Do name the job as your CI does** if you can. When CI is red because of tests By Dawn already knows are flaky, By Dawn
  re-runs the job quietly instead of waking the worker, but only if every red job sent results. By Dawn matches an upload
  to a red job by its name, or by the name's start for a matrix (`unit` matches `unit (ubuntu-latest, 20)`).
- **Don't send reports from a cancelled run.** They hold the tests that ran before the cancel, and nothing else.

## Next

- [Make your test runner write reports](https://docs.bydawn.com/developers/test-reports/): the flag for each runner, for results and
  coverage.
- [Test results](https://docs.bydawn.com/tests/): what By Dawn does with them: the Tests panel, failing tests for the worker, and flaky
  tests.
- [Pick up results from GitHub Actions](https://docs.bydawn.com/tests-github-artifacts/): the no-CI-change path, for repos on GitHub.
- [Set up your CI](https://docs.bydawn.com/tests-ci/): pickup and launching on GitLab CI, Azure Pipelines, Jenkins, CircleCI and Buildkite.

Source: https://docs.bydawn.com/developers/tests-submit/