Pick up results from GitHub Actions

This page shows you how to let By Dawn read the test results your GitHub Actions workflows already upload. By the end, every finished workflow run’s test reports land on its ticket and pull request, along with about a month of past runs, and you won’t have changed a line of CI.

Many workflows already keep their reports as artifacts, with actions/upload-artifact, so people can download them. By Dawn can read those same files. It needs one more permission from GitHub to do it: Actions: read, which lets By Dawn’s GitHub App list and download a repo’s workflow artifacts. It’s optional, so nothing changes until an owner or admin approves it. Your tests still run on GitHub’s runners or yours, never on By Dawn’s servers.

If your repo isn’t on GitHub, or its workflows don’t upload their reports, use saw-agent tests submit instead. Test results compares the ways in.

Approve Actions: read

Open Settings›, then Organization›, then GitHub. Once By Dawn’s GitHub App is connected, the card has a row for Actions: read (optional): “Lets SAW read the test-result and coverage artifacts your workflows already upload, including the last 30 days, with no CI change.” It says whether it’s on:

• Granted — SAW picks up your workflows’ test results.
• Not granted — artifact pickup is off until it’s approved on GitHub.
• Couldn’t check with GitHub just now. Reload the page in a moment.

To turn it on, an owner or admin chooses Review permissions on GitHub, which opens By Dawn’s app on GitHub, and accepts the new permission there. Members see the row but not the button, so ask an admin. GitHub asks again for each installation of the app, so if your organization’s repos are spread over several GitHub accounts, each one approves it.

Test results also have to be on for your organization (Turn on test results). Without them, By Dawn doesn’t pick anything up, even with the permission granted.

What happens next

As soon as the permission is accepted, By Dawn reads the last 30 days of finished workflow runs in each of your organization’s GitHub repos connected to By Dawn, oldest first, so each test’s history is in the order it happened. A repo you connect later gets the same back-fill when you add it on the Repos page. Back-fill reads up to 200 runs a repo.

From then on, By Dawn reads a workflow run’s artifacts each time the run finishes: on any branch, whatever triggered it, and whether it passed or failed. Each artifact with test reports becomes one run in By Dawn:

• named after the artifact (as the job), with Open CI run linking to the workflow run
• on the run’s commit and branch, so By Dawn ties it to the pull request open from that branch, and to its ticket
• with the run’s attempt, so re-running a workflow records a new attempt rather than mixing old and new results

A webhook GitHub sends twice, a back-fill and a live run never record the same artifact twice, so there’s nothing to clean up if one overlaps another.

Which files it picks up

By Dawn only downloads an artifact whose name says it holds test output. The name starts with saw-test-results, or contains any of test, junit, trx, ctrf, coverage, cobertura, lcov, result or report, in any case. A build output or an image isn’t downloaded at all.

Inside the artifact, By Dawn keeps the files whose names give away their format:

File name	Read as
*.xml with junit in the name, such as junit.xml or junit-api.xml	JUnit XML results
*.trx	TRX results
starting ctrf, ending .json, such as ctrf-report.json	CTRF results
lcov.info	lcov coverage
coverage*.xml or cobertura*.xml, such as coverage.cobertura.xml	Cobertura coverage, if the file is Cobertura

Names are matched in any case. A coverage.xml that’s really JaCoCo or Clover is skipped, because By Dawn reads only Cobertura coverage.

If your files are named differently, such as Maven’s TEST-*.xml or pytest’s report.xml, put them in an artifact whose name starts with saw-test-results. In that artifact By Dawn reads every file it recognises by its content, whatever it’s called:

- uses: actions/upload-artifact@v4
  if: ${{ !cancelled() }}
  with:
    name: saw-test-results-${{ matrix.shard }}
    path: |
      target/surefire-reports/TEST-*.xml
      coverage/

go test -json output and coverlet’s coverage.json aren’t picked up from artifacts. Send them with saw-agent tests submit, or write JUnit with gotestsum and Cobertura with coverlet (Make your test runner write reports).

An artifact is read in one results format: if one holds both JUnit and TRX files, By Dawn keeps the first format it finds and skips the other. Coverage files are always kept. Upload each format as its own artifact if you have both.

Limits

Limit	Value
One artifact’s zip	100 MB
Reports read from one artifact	64 MB in all, up to 32 files of 20 MB each
Files examined in one artifact	2,000
Back-fill	the last 30 days, up to 200 runs a repo

An artifact past its limit is skipped, not cut short. An artifact GitHub has expired or deleted is skipped too, so a back-fill reaches only as far back as your artifacts’ retention: 90 days on GitHub by default, less if your workflows set retention-days.

Pickup and the worker

Results picked up from artifacts are CI’s results. So when CI on a ticket’s pull request is red, the worker gets the failing tests from them, and By Dawn can re-run a red that’s only known-flaky tests instead of waking the worker (Flaky tests). For that re-run, By Dawn matches artifacts to failed jobs by name: an artifact named exactly like its job speaks for that job. When only one job is red, every artifact of the run speaks for it. In a matrix, where each leg has to upload under its own name, the re-run happens only when one leg is red, so if you want it for every leg, send those results with saw-agent tests submit.

FAQ

Do I have to change my workflows?

No, if they already upload reports with a name By Dawn looks for. If they don’t, add an actions/upload-artifact step named saw-test-results…, as above, or use saw-agent tests submit, which doesn’t need an artifact.

I approved Actions: read, but no results appear.

Check that test results are on for your organization, that the repo is connected to a By Dawn project on the Repos page, and that its artifacts’ names and files match the lists above. Back-fill only reaches runs whose artifacts haven’t expired.

Can I use pickup and saw-agent tests submit together?

Yes. They’re both CI results, so a workflow that does both records each report twice, as two runs on the same commit. Pick one per workflow.

What can By Dawn do with Actions: read?

List your workflow runs and download their artifacts. Read access on its own doesn’t let By Dawn start, cancel or change a workflow, and it doesn’t include your repository’s secrets.

Next

• Test results: what By Dawn does with the results.
• Make your test runner write reports: get your runner to write a file worth uploading.
• Repos: connect a repo, which starts its back-fill.