Results from the worker's runs

This page explains the test results By Dawn records on its own, from the tests a ticket’s worker runs. By the end you’ll know which runners it works with, and how to check that your repo’s tests are recorded.

Before a worker can call a ticket done, the mission’s Tests and build pass step runs the repo’s own tests, type checks and build on the worker’s machine (see Missions and templates). While those tests run, By Dawn asks the runner for a report as well, then records it on the ticket. It shows in the ticket’s Tests section as a Worker run, beside any run your CI sends. You don’t add anything to CI for it, and for most runners you don’t change the repo either. It’s one of the three ways results get in: Test results compares them.

Which runners it works with

By Dawn adds a report to the runners below. Each run of the step records one run per report format, whether the step passed or failed.

Runner	What By Dawn does	What your repo needs
Go	Runs each go test with -json and keeps its usual output for the log	Nothing. It works wherever go test is called, even from a Makefile.
pytest	Adds --junitxml, keeping any options you set in PYTEST_ADDOPTS	Nothing
dotnet test	Turns on the TRX logger. A .trx file your own settings write is picked up too	Nothing
Vitest	Adds its JUnit reporter, next to your usual output	The step runs a package.json script that ends in vitest (below)
bun test	Adds its JUnit reporter	The same, ending in bun test
Jest	Adds the jest-junit reporter, next to your usual output	The same, ending in jest, and jest-junit installed as a dependency

Other runners, such as Playwright or Mocha, aren’t recorded from the worker’s runs. Send their results from CI instead, with saw-agent tests submit or artifact pickup.

The reports are written outside the worker’s checkout, so they never end up in a commit, and they’re deleted when the step finishes. Anything that looks like a secret is removed from a report before it’s sent to By Dawn.

Make a JavaScript test script reportable

For Vitest, bun test and Jest, By Dawn adds its reporter flags to the end of the script the step runs. So it can only do that when the step’s command ends by running a package.json script (npm run test, pnpm run test, yarn run test or bun run test), and that script’s last command is the runner:

"test" script	Recorded?
vitest run	Yes
bun run build:deps && vitest run	Yes: only the last command needs to be the runner
NODE_ENV=test npx vitest run	Yes: settings in front and npx, bunx or pnpm exec are fine
jest --ci	Yes, if jest-junit is in dependencies or devDependencies
vitest run | tee out.txt	No: the output goes through a pipe
vitest run || true	No
vitest run && node tools/report.mjs	No: the runner isn’t last
vitest watch	No: watch, dev and bench modes aren’t test runs

If a script doesn’t fit, split it: keep a script that only runs the tests, and run your extra steps from another one.

When a failure passes on a second try

When the step fails and every failure is a named test, By Dawn runs just those tests once more on the same commit: failing files for Vitest, bun test and Jest, failing tests for Go. pytest and .NET tests aren’t re-run, and neither is a failure no test owns, such as a build error, a data race or a timeout.

If they pass the second time, the step passes, and its output says they “failed, then passed when rerun alone — recorded as flaky, not as a failure”. The run records them as flaky, so By Dawn’s flaky-test handling counts them. Fix them anyway: CI runs the suite once and re-runs nothing, so the same race can fail the pull request. If they fail again, the failure stands.

What it doesn’t record

• Coverage. The worker’s runs record test results only. Send coverage from CI.
• The failing tests a worker gets when CI is red. Those come from CI’s own results, because CI’s failure is the one the worker has to fix. See What the worker gets when CI is red.
• A report over 20 MB, or more than 32 reports of one format in one step. It’s left out, and the rest are still recorded.

Recording never changes the step’s verdict. If By Dawn can’t record a report, the step passes or fails on its own result, as it always did.

FAQ

My ticket's Tests section shows no Worker run. Why?

Usually the step’s runner isn’t one By Dawn can add a report to: a JavaScript script that doesn’t end in the runner, Jest without jest-junit, or a runner that isn’t in the list. It also shows nothing if your organization doesn’t have test results turned on, or the step hasn’t run yet.

Can I turn it off?

No. It reads only the reports the runner writes, outside your checkout, and never changes the step’s result, so there’s nothing for it to get in the way of.

Does it run my tests again?

No. It records the run the step was already doing. The one exception is the single re-run of failing tests above.

Next

• Test results: the Tests panel, failing tests for the worker, and flaky tests.
• Send test results from CI: add CI’s results and coverage.