Test cases

This page helps you work with a project’s test case library: the list of what should be tested, as cases a person can follow. By the end you’ll know how to organise it in folders, write and change a case, see who changed what, and tie cases to the tickets they verify.

The library is in Tests›, then Cases: choose Tests in the project’s sidebar, then the Cases tab. To fill it from your repos rather than by hand, see Build your test library; to bring cases from another tool, see Move from Testmo or TestRail.

Before you start

• Test management has to be on for your organization. Without it the Cases tab says Test management is not on your plan, and tickets show no cases. Contact support to turn it on.
• Members of the project can change the library. Someone with view-only access to the project can read it.

The three kinds of case

Every case in the list is one of three kinds, which By Dawn sets for you:

• Manual: steps a person follows. A case you add is manual.
• Automated: a test in one of your repos. By Dawn adds it the first time a result names it, or when the library is built, and keeps its history of results.
• Manual + automated: a manual case an automated test covers. A manual case becomes one when you link the test that checks the same thing (see Automation coverage).

Organise cases in folders

The Folders pane on the left always has All cases and Unfiled, then your folders, each with its count of cases.

• Add a folder with the folder button (New folder), or New folder inside from a folder’s menu. Names are unique within their parent folder.
• Rename, Move up and Move down are on each folder’s menu.
• Reorder by dragging a folder above, below or into another, or focus it and press ⌥↑ or ⌥↓ (Alt on Windows and Linux).
• Delete… never deletes cases. If the folder holds any, you choose where they go (Move its contents to), its parent folder unless you pick another, or Unfiled.

To move a case, drag it onto a folder, or change Folder on the case.

Find a case

Search titles and keys from the top of the list, and narrow it by kind, priority, status and tag. The list shows Active cases until you pick another status. Include sub-folders shows a folder’s cases with those of the folders inside it, and Group by By feature groups cases by their feature: tags rather than their folders. Gaps shows the cases written for what no test covers yet.

Write a case

Choose New case, type what the case checks and choose Add. It’s created as a Draft in the folder you’re viewing, and opens so you can fill it in:

• Status: Draft while you write it, Active once it’s ready to run, and Deprecated when it no longer applies. Only active cases are counted in coverage and shown by default.
• Priority: Urgent, High, Medium, Low or No priority.
• Owner: the member who looks after the case.
• Folder, Tags (press Enter or type a comma after each), Description and Preconditions.
• Steps: each has Do this, Expect this and, if it needs it, Test data. Choose Add step for another, and move or remove steps with the buttons beside them. A case holds up to 200 steps.
• Custom fields, if a project admin has added any: text, a choice from a list, or a number. A required one is marked with *.
• Attach files for anything a tester needs, such as a sample file, up to 50.

Then type What changed? if you like, and choose Save. Each save is a new revision, so the case says “Saved as revision 4”. If someone else saved while you were editing, By Dawn keeps your changes and offers Load their version or Save mine on top.

An automated case’s key, location and status come from the code, so you can’t change those. You own the rest: its title, description, steps, priority, tags, owner and links, and syncing from the code never overwrites them.

Delete a case

Delete case deletes the case and its history. A case a manual run has used can’t be deleted, and nor can an automated case. Set its status to Deprecated instead, which keeps its history and takes it out of the default list.

See its history

The case’s History tab lists every revision, newest first: who made it (a member, a By Dawn worker, an API token or By Dawn itself), when, and the note they left. Open one to see what it changed, step by step, or switch Compare with to Compared with now.

Restore this revision puts a case back as it was. It adds a new revision rather than removing any, so nothing is lost and you can restore again.

Link a case to tickets

On the case’s Links & automation tab, Verifies lists the tickets, epics and releases the case checks. Use Link to to add one, up to 50.

A linked ticket shows the case in its Cases covering this section, with the case’s latest manual result, such as Passed or Failed. A result an agent recorded is marked AI-run, because it’s weaker evidence than a person’s. Choose a case there to open it.

Automation coverage

The Automated by list on a manual case’s Links & automation tab is the automated tests that check the same thing. Link one, and the case becomes Manual + automated and counts as automated.

The coverage panel above the list sums it up for the folder you’re viewing, such as “212 of 340 cases automated · 62%”, with a bar for each folder inside it. It counts only active cases: the manual ones, and of those, the ones an automated test covers. Choose a bar to open its folder.

An automated case’s tab is called Links & coverage, and shows its recent results: how often it passed, how many runs over how many days, its median time, and whether it’s Stable, Suspected flaky, Flaky or Broken (Flaky tests).

Have an agent automate a case

Automate this case, at the top of a manual case, files a ticket for a worker to write an automated test that follows the case’s steps. Pick the repo for the test if the project has several, and choose whether to Start a worker now or leave the ticket on the board. The case links to the test by itself the first time it passes. While the ticket is open, the case shows Automating in and the ticket.

Review changes an agent proposes

Agents add to the library too: a ticket’s worker can propose the cases that check what it built, and the agent that improves your library proposes better ones. Their proposals never land on their own. They wait on the ticket, in a Case proposals section above Cases covering this, as one set marked Waiting for review.

Each item says what it would do: New case, Applies cleanly, or Merges with later edits when people have changed the case since. Changed fields show what the case Was, what’s Proposed, and what it reads Now.

• Decline an item to leave it out, or Restore it.
• Edit an item to change what it proposes. A Conflict, where a person and the agent changed the same field differently, has to be resolved this way first.
• Approve and apply saves the rest to the library in one go, linked to the ticket.
• Reject applies nothing.

Anyone who can work on the project’s board can review a proposal. Agents and API keys can’t review one.

FAQ

Why is there no Delete on this case?

It’s automated, and the code owns it: it’s marked gone when the test is removed from the repo. Delete is on manual cases only.

Where do automated cases come from?

From your test results and your repos. A test becomes a case the first time a result names it (see Test results), or when you build the library.

Does moving a case make a revision?

No. A revision records what the case says, so moving it to another folder doesn’t add one.

Next

• Run tests by hand: run the cases with your team, and record each step’s result.
• Exploratory sessions: test without a script, and keep notes as you go.
• Build your test library: fill the library from your repos.