# Run tests by hand

> Run a project's test cases by hand in By Dawn — start a manual run against a build, assign the cases, record each step from the keyboard, file a bug from a failure, retest, and close the run.

This page helps you run test cases by hand with your team: a regression pass before a release, say, or checking a
fix on staging. At the end you'll have a manual run with a result for every case and a bug ticket for every failure,
each with the steps that led to it.

A manual run is a set of cases from the [library](https://docs.bydawn.com/test-cases/) that people work through against one build, step by
step, recording what happened. It records what a person did: By Dawn runs nothing itself. Runs are in
**Tests › Runs**.

## Before you start

- **Test management and test results both have to be on** for your organization. Without one, the **Runs** tab says
  which isn't on your plan. [Contact support](https://app.dev.sawrun.com/support) to turn them on.
- **Members of the project start and work in runs.** Someone with view-only access can follow them.
- **The cases you'll run need to be in the library**, as **Active** cases, or as drafts if you include them.

## Start a run

1. On the **Runs** tab, choose **New run**.

2. Give it a **Name**, such as `Release 4.2 regression`, and a **Description** if it helps.

3. Pick the cases. Tick the folders to run (their sub-folders come too unless you clear **Include sub-folders**),
   and type tags under **Only cases tagged…** to narrow them. With nothing ticked, the run takes every active manual
   case. **Include drafts** adds draft cases, and **Include automated tests** adds the automated tests your CI reports.
   The dialog counts the cases as you go; a run holds at most 5,000. If the project has suites, **Start from** **A
   saved suite** takes a suite's cases instead.

4. Say what's being tested:
   - **Build**: **A commit** (the repo and its full commit ID), **A release**, or **No build**.
   - **Environment**: where the testers run it, such as Staging. **New environment…** adds one, with its URL.
   - **Configurations**, if the project has any, such as browsers. Tick several to start one run on each, grouped
     together, so each browser gets its own results.

5. Choose who does it under **Assign every case to**, or leave it at **Nobody yet** and assign later. Add a **Due**
   date if there's a deadline.

6. Choose **Start run**. The run opens.

A run holds up to 5,000 cases. Each case is pinned to the revision it had when the run started, so a tester reads
the steps everyone agreed, even if someone edits the case meanwhile.

## Follow a run

The **Runs** tab lists runs that aren't closed, newest first, with a bar for each one's progress, such as "12 of 40
done · 2 failed · 1 blocked · 3 to retest". A run is **Open** until its first result, then **In progress**, then
**Closed**. Filter the list by status, by who it's assigned to (**Assigned to me**) and by environment. A run past
its due date says **Overdue**.

Open a run to see its cases, grouped by folder, each with its result and who it's assigned to. Results from other
testers show up as they record them.

### Assign the cases

**Assign…** hands out the cases still to do: pick a person, which results to include (**Untested** and **Retest** to
start with) and, if you like, one folder. To assign particular cases, tick them and choose **Assign to…**. Assigning
to **Nobody (unassign)** takes them back.

Cases can also go to **The AI tester**, an agent that works through them in a browser on one of your machines. Its
results are marked **AI-run**, because they're weaker evidence than a person's.

### Add or remove cases

**Add cases** adds more with the same picker, skipping any already in the run. **Remove from run**, at the bottom of
a case, removes one that has no result yet.

## Record results

Open a case to run it. The view shows its preconditions and its steps, each with what to do and what to expect.

- **Each step**: mark it **Passed**, **Failed**, **Blocked** or **Skipped**, and **Add a note** of what happened.
  Choose the same result again to clear it.
- **The whole case** is set from its steps once every step has a result: failed if any step failed, then blocked if
  any was blocked, passed otherwise. Skipped steps don't fail it. Set **Result for the whole case** yourself to
  override it, or to record a case with no steps.
- **Actual result**: what actually happened, which a bug ticket quotes. **Note** is for anything else.
- **Screenshots**: paste one with `⌘V` (`Ctrl+V` on Windows and Linux) to put it on the
  step you're on, or **Attach** or drag files: images, logs, `.har` and `.json` files, up to 20 a step and 50 a
  case.

Everything saves as you go, and the timer at the top adds the time you spent on the case. Then **Next case**, or
**Next untested** to skip to the next case still to do.

### Keyboard shortcuts

The run is built to be worked from the keyboard. `?` shows these at any time:

| To | Press |
|---|---|
| Move to the next or previous step | `j` `k` |
| Mark the step passed, failed, blocked or skipped | `p` `f` `b` `s` |
| Mark the whole case | `⇧p` `⇧f` `⇧b` `⇧s` |
| Next case | `n` or `]` |
| Previous case | `⇧n` or `[` |
| Write the actual result | `a` |
| Write a note | `e` |
| File a bug | `x` |
| Paste a screenshot onto the step | `⌘V` |

They don't fire while you're typing in a field, so a note can't mark a step by accident.

## File a bug from a failure

On a failed or blocked case, **File bug** (`x`) files a ticket in the project with the details already
written. By Dawn fills in the case's steps with each one's result, marks the step where it **failed here**, and adds the
actual result, your notes, and the build, environment and configuration. The title starts as "Failed:" and the case's
title, so change it if you can say more, and add anything above what By Dawn fills in. Screenshots are copied onto the
ticket unless you turn that off.

If a ticket already tracks the problem, choose **Link a ticket that already tracks it** instead. Either way, the case
shows the ticket, so the next person sees it's known. Write the actual result before you file, because the ticket
quotes it.

## Retest

When the fixes are in, choose **Retest failures** to send every failed and blocked case round again, or tick some
cases and choose **Retest**. Each goes back to the same person as **Retest**, with its attempt number up by one.
Its step results, actual result and time start again, and the earlier attempt stays under **Earlier attempts**.
The case's note, files and bugs stay.

Tick **Use each case's latest revision** to retest with the steps as they read now, if the case was corrected since
the run started.

## Close the run

When every case has a result, choose **Close run**. Its results become read-only, though you can still file or link
a bug from a case. **Reopen** opens it again to record more.

Each case's **Runs** tab in the library lists the runs it was in, and a ticket the case is linked to shows its
latest manual result under **Cases covering this**.

## FAQ

**Can I change the build or the cases of a run?**

**Edit** changes its name, description, environment and due date. The build is fixed, because the results are about
that build. Add cases with **Add cases**, and start a new run to test another build.

**Why don't I see File bug?**

It's on failed and blocked cases. Mark the case first.

**Do manual runs show in a ticket's Tests section?**

No. A ticket's **Tests** section is for automated results. A manual result shows against the case, in the ticket's
**Cases covering this** section.

**Where do configurations come from?**

They're the project's list of things a run can vary on, such as browsers or devices. If the project has none, the
**Configurations** choice doesn't appear.

## Next

- [Exploratory sessions](https://docs.bydawn.com/test-sessions/): test without a script, in a timebox.
- [Plan testing for a release or cycle](https://docs.bydawn.com/test-plans/): group the runs that sign off a release or cycle.
- [Test cases](https://docs.bydawn.com/test-cases/): fix a case's steps after a run shows they're wrong.

Source: https://docs.bydawn.com/test-runs/