Guide
How to turn a PRD into a regression suite
A regression suite is only as good as its link to the spec. When a test can't say which sentence of the PRD it checks, nobody knows whether it's still right after the PRD changes, and suites rot. This guide is a method you can follow by hand; Provo automates the mechanical parts.
1. Split the PRD into numbered requirements
Go through the PRD and give every testable statement a key (R1, R2 …) and a location (section and line). Testable statements are usually list items, table rows and sentences with words like must, only, within, up to or returns. Skip background and goals; skip non-goals entirely.
Keep each requirement's exact words. Paraphrasing here is where invented behavior creeps in.
2. Write down what the spec doesn't say
Before writing any test, list the gaps as questions:
- Vague words: "quickly", "should not overwhelm", "most recent".
- Missing numbers: a retry interval, a size limit, a timeout.
- Missing error behavior: what happens at limit + 1?
- Conflicts between two requirements.
Send these to the product manager. A test that guesses the answer will pass or fail for the wrong reason.
3. Draft cases per requirement, with a quote
For each requirement write one to three cases:
| Kind | Checks | Example |
|---|---|---|
| Functional | The stated behavior works | Admin invites a teammate |
| Edge | Boundaries the spec states | Exactly 20 addresses at once |
| Negative | What the spec rules out | A non-admin can't invite |
| Regression | Behavior the spec says must keep working | Old endpoints still receive events |
| Acceptance | The criterion in Given/When/Then | Invited person joins as a member |
Every case gets the requirement key and a quote of the words it tests. If you can't find words to quote, the case is testing something the spec doesn't say: turn it into a question instead.
4. Review, then pick the regression set
Have someone who didn't write the cases review them against the quotes. Approve, edit or reject. Then mark the cases for behavior that already shipped and should keep working as your regression set (a folder or a tag works).
5. When the PRD changes, diff it
Don't edit the old PRD in place and hope. Save the new version and compare:
- Unchanged requirements: their tests stay as they are.
- Changed requirements: review every test traced to them; the old quote shows exactly what changed.
- Removed requirements: retire their tests, or keep them deliberately as regression checks.
- New requirements: draft cases for these only.
6. Run, record and close the loop
Run the regression set against a named environment and build. Record pass, fail, blocked or skipped with notes, link failures to issues, and keep a matrix of requirement → test → latest result so gaps (requirements with no test) and failures are visible.
Doing this with Provo
Provo does steps 1 and 5 in code (the requirement split and the diff), drafts steps 2 and 3 with AI under one rule (every case must quote its requirement, and code drops any case whose quote isn't in the spec), and gives you the review, runs, matrix and exports for steps 4 and 6. You can try it on a sample spec without an account.
Questions
What's the difference between a test suite and a regression suite?
A test suite checks that new behavior works as specified. A regression suite is the subset you rerun on every release to check that behavior that already worked still does. Cases traced to stable requirements are the natural regression candidates.
How many test cases should one requirement get?
Usually one to three: the main behavior, then edge and negative cases the requirement actually supports. A requirement that needs more is often several requirements written as one.
What should happen to tests when a requirement changes?
Review every test traced to it: keep it if the behavior still holds, update it if the wording changed the expected result, and retire it if the behavior is gone. A trace from each test to the requirement's exact words makes this a list, not a search.
Questions buyers ask
How do I keep a regression suite current when the PRD changes?
Save each PRD change as a new version and diff it. Tests traced to unchanged requirements stay; tests traced to changed requirements are reviewed against the old quote; tests for removed requirements are retired or kept on purpose; new requirements get new cases. Provo does the split, diff and review marking for you.