The policies it ships with

People and their agents follow these when they record in a collection made from this template. Its owners can change them.

1. A node belongs in a test collection when it documents what to verify or what happened when it was verified: a test scenario (Eval — preconditions, steps, and an expected result), a recorded run or result (Log — what happened in a given environment), the objective / charter / suite a scenario covers (Intent), or external context — a requirement, environment, evidence artifact, or defect (Reference). PASS when the candidate is one of these. FAIL when it instead belongs in another collection kind — a business-process step, a glossary term, a generic decision record, or a free-form note that is not a test, a run, an objective, or a referenced artifact.
2. Only Eval (test scenarios), Log (recorded runs and their results), Intent (the objective, charter, or suite a scenario covers), Reference (a requirement, environment, evidence artifact, or defect), and Principal (the tester, automation system, or agent that owns or runs a test) belong in a test collection. Process steps, glossary terms, and decision records live in their own collections and are cited here via Reference.
3. Only these relationship edge types belong in a test collection: `supports` (the evidence/validation spine — an Eval supports the objective or requirement it verifies, a Log supports the Eval it ran, and an evidence Reference supports a Log), `attributed_to` (the Principal who owns a scenario or ran a test), `has_parent` (nest an objective or suite under a broader test plan), `relates_to` (a see-also between scenarios, or a link from a failing run or scenario to a defect), `replaces` (a new scenario supersedes a retired one), and `derived_from` (provenance — a scenario derived from an imported spec or forked from another). Process-flow (`flows_to`) and Rule-guard (`constrained_by`) edges do not belong here.
4. Every committed (`queued` or `active`) test scenario says how to run it: its `how_to_run` carries the preconditions and the ordered steps a tester or agent follows. A scenario with no way to run it is an objective (an Intent), not a test. A `drafting` sketch may capture the idea before the steps are written.
5. Every committed scenario and run links to what it covers with a `supports` edge: a scenario `supports` the objective (Intent) or requirement (Reference) it verifies, and a run (Log) `supports` the scenario (Eval) it executed. A run with no scenario is an orphan result; a committed scenario with no objective or requirement is an untraced test.
6. Check the scenario's prose, its `how_to_run` (preconditions + steps), and its expected result (`expected` / `expected_status`). PASS when it verifies ONE behavior or condition, states the preconditions and the steps to reproduce it, and names a single observable, checkable expected result — so any tester or agent who runs it reaches the same pass/fail verdict. FAIL with a reason when it is vague (e.g. `test login`), bundles several unrelated checks into one scenario, depends on hidden state a reader cannot set up, or has no observable expected outcome to judge against.
7. Check the run's prose and fields. PASS when it records the environment it ran against — for a website or app that means the browser and version, the operating system, the device or viewport, the build / release or commit, and the URL or environment (e.g. staging vs production) — names a clear outcome (passed, failed, blocked, or skipped), and, when it failed or was blocked, says what actually happened versus what was expected and links the evidence (screenshot, recording, console or network log) or the defect. FAIL with a reason when the outcome or the environment is missing, or a failure is recorded with no actual result and no evidence.
8. A test collection has two halves. A scenario (an Eval) is the durable spec — what should be true and how to check it — and each run (a Log) is one execution of it. Don't overwrite a scenario with its latest result: record every run as a new Log so the history of passes, failures, and the environments they happened in is preserved.
9. Give each scenario a clear objective in its prose (the one behavior or condition under test), the preconditions and ordered steps in `how_to_run`, the test data it needs, and a single observable expected result in `expected` (with a coarse `expected_status` of pass or fail). Keep it atomic — one behavior per scenario — independent of other scenarios, and deterministic, so it passes or fails for exactly one reason.
10. Keep the scenario's at-a-glance verdict on the Eval itself — `last_status` (pass / fail / pending), `last_run_at`, and `last_reason` as a cache of the most recent run — and keep the full, append-only history as Log nodes. The Eval answers “is it green right now?”; the Logs answer “how has it behaved over time and across environments?”.
11. Trace every scenario to what it verifies: `supports` the objective (Intent) it serves or the requirement / acceptance criterion (Reference) it checks, and group related scenarios under an objective or suite, nesting suites with `has_parent`. Traceability is what lets you answer “which tests cover this requirement, and are they green?”.
12. For websites and apps the same scenario can pass in one environment and fail in another, so record the environment on every run, not on the scenario: the browser and version, the operating system, the device or viewport, the build / release or commit under test, and the URL or environment (local, staging, production). Drive your browser / device matrix from real user analytics — cover the combinations your users actually use first.
13. Record a clear outcome on every run: passed, failed, blocked, or skipped. Use the node's `outcome` (succeeded / failed) for the pass/fail axis and state blocked or skipped in the run's prose. Distinguish failed from blocked: failed means a step did not meet its expected result but execution could continue; blocked means an earlier failure or an environment problem stopped the run before the scenario could be judged at all.
14. On a failure, capture what actually happened and attach the evidence — screenshots, a screen recording, console and network logs, or a stack trace — as Reference nodes linked with `supports`, and link the defect you filed (a GitHub issue, a Jira ticket) as a Reference with `relates_to`. This collection is the test record, not the bug tracker: reference the external defect rather than re-litigating it here. When triaging, separate severity (how bad the failure is) from priority (how urgently it must be fixed).
15. For behavior specs, write the scenario as Given / When / Then — the preconditions as Given, the action as When, the single observable expected result as Then — across the scenario's prose and `how_to_run`. Keep it focused: one user-observable behavior per scenario and a single-digit step count. Split a vague “works on mobile” into concrete per-device scenarios that each pass or fail on their own.
16. Exploratory testing fits the same shape. Model a charter as an Intent — “Explore [area] using [approach] to discover [risks]” — run each time-boxed session as a Log that records what you did, what you found, what got in the way, and what is left (the PROOF debrief: Past, Results, Obstacles, Outlook, Feelings), and capture each bug as a Reference. Promote a recurring or important finding into an explicit Eval so it becomes a repeatable scenario.
17. Say how a scenario is run in `how_to_run` — the manual steps, or the command / suite id / spec path for an automated test — and attribute it to the Principal who owns it. A run is attributed to whoever (or whatever) executed it: a person, a CI pipeline, or an agent. Agents running a suite record each result as a Log here, so automated and manual runs share one history.
18. Walk a scenario through the lifecycle as it matures: `drafting` while you are still writing it, `queued` once it is ready to run or review, `active` when it is approved and part of the suite, and `retired` when the behavior is gone or the scenario is superseded — link the replacement with `replaces`. Completeness and quality gates apply once a scenario is committed (`queued` or `active`); a `drafting` sketch is spared. A run is a fact that already happened — record it as `active`.
19. The same Eval-plus-Log shape stretches to specialized tests without new node types — choose the Eval's `criterion` (an exact match, a response / shape match, or an llm-judge for fuzzy output) and put the specifics in the expected result and `how_to_run`: a performance test states a latency or throughput threshold as the expected result and logs the measured number; an accessibility test names the WCAG criterion and attaches the audit (e.g. an axe report) as evidence; a security test references the advisory or CVE; an API or contract test pins the expected response shape; a visual-regression test attaches the baseline and the diff. Reach for a different collection only when what you are documenting stops being “a scenario and its runs”.
20. Agents: read `GET /<handle>/api/authoring-contract.json` and write with `POST /<handle>/api/changesets.json` — create each scenario as an Eval with its `how_to_run`, `expected`, and `criterion`, wire its `supports` edge to the objective or requirement it covers, and record each run as a Log with the environment and outcome, its `supports` edge to the scenario, and any evidence References — all in one changeset, never as disconnected nodes.