On this page
A QA dashboard should show whether a specific release candidate satisfies the scenarios that matter to users. Test totals and pass rates can help locate work, but they do not establish that a required behavior was checked in the right environment. Connect each scenario to its result, candidate identity and evidence before turning the dashboard into a release decision.
This guide provides an illustrative import-application model and a copyable evidence table. Its scenario identifiers and results are synthetic. No real release is declared ready on the basis of this example.
Start with scenarios users can recognize
Define a scenario as a concrete trigger and expected outcome. For an import application, “CSV validation” is a feature area; “an operator uploads a row with a missing account ID and sees the original row number before any save” is a scenario.
Retool's QA dashboard template illustrates the dashboard category. Playwright's reporter documentation describes formats for reporting test execution. A reporter can supply valuable evidence, but your dashboard must still connect that evidence to the intended behavior and candidate.
Use a small initial scenario list that covers the representative flow and its meaningful failures. A long list of similar checks can inflate totals while leaving persistence or permission behavior unresolved.
| Scenario | Trigger | Expected outcome |
|---|---|---|
| SC-101 | Import valid synthetic CSV | Records save and receipt identifies them |
| SC-102 | Import row without account ID | Row-level error before persistence |
| SC-103 | Reload after valid import | Same saved records and receipt remain |
| SC-104 | Ordinary user requests another team's import | Access denied or unavailable |
Keep the expected result independent of the implementation. Otherwise a test can confirm what the code currently does rather than what the user needs it to do.
Record the candidate and environment
A result applies to the candidate and environment that produced it. Store a repository revision or build identifier, environment, configuration reference and execution time. These fields let the reader determine whether the evidence still describes the release being considered.
Do not combine a local persistence check with a production access check into one unqualified “passed” state. Both may be useful, but they establish different conditions. A configuration change can invalidate one without changing the source code.
For the fictional sample, use candidate DEMO-A and a local test environment. The identifier is an illustrative label, not a real commit hash. A production dashboard should store identifiers that resolve to the actual candidate and deployment.
If an automated job cannot identify its candidate, flag the evidence as incomplete. A result with unknown origin can help investigate a problem, but it should not silently satisfy a required release check.
Keep result states distinct
Use states that describe what happened. Passed means the expected outcome was observed. Failed means the check ran and the outcome differed. Blocked means a dependency prevented the check from completing. Not run means no current execution is available.
| State | Meaning | Useful next action |
|---|---|---|
| Passed | Expected outcome observed for this candidate | Retain evidence |
| Failed | Observed behavior differs | Investigate the specific discrepancy |
| Blocked | Check could not complete | Resolve dependency and rerun |
| Not run | No current check | Schedule or execute it |
Do not count Blocked or Not run as Passed. Likewise, a retry that eventually succeeds should retain the earlier failure if it helps explain instability. The latest result can be current while the history remains available.
Define skipped checks explicitly. A scenario can be inapplicable to a particular release, but record the reason and decision owner rather than removing it to improve the pass rate.
Use an evidence table that can be inspected
The following CSV is synthetic. It deliberately leaves reload behavior unresolved so the sample dashboard cannot report complete readiness.
scenario_id,candidate,environment,result,evidence_ref,remaining_action
SC-101,DEMO-A,local,Passed,RECEIPT-101,None
SC-102,DEMO-A,local,Passed,ERROR-102,None
SC-103,DEMO-A,local,Not run,,Check reload persistence
SC-104,DEMO-A,local,Blocked,,Create second-team identity
An evidence reference should lead to the relevant artifact: execution report, saved-record readback, screenshot, trace or request response. Describe what it proves. A screenshot of a success message does not establish persistence after reload.
Restrict evidence access according to the data it contains. Prefer synthetic fixtures and redact unnecessary personal information. A useful dashboard should not require a reviewer to expose private customer records in a public issue or screenshot.
Check the ingestion process too. If two reports share the same run identifier, define whether the second updates the run or is rejected. Repeated uploads should not double the scenario count. Preserve the source report so a summarized status can be investigated later.
Define readiness from required scenarios
For this example, all four scenarios are required. The release state remains incomplete because SC-103 has not run and SC-104 is blocked. The two passes should be visible, but they cannot substitute for the missing checks.
Use a rule such as: every required scenario has a current Passed result for the candidate and applicable environment, with accessible evidence. If exceptions are possible in your organization, record their exact scope, owner and rationale. The dashboard should show an exception as a decision, not rewrite a failed test as passed.
Separate technical readiness from deployment authorization. A dashboard can show that required checks are complete without automatically granting permission to deploy. Connect the actual release process only after its owners define that boundary.
When the candidate changes, identify the scenarios affected by the change. A new storage handler should reopen persistence evidence; a wording-only edit may need a narrower visual check. Do not retain outdated results as current simply because the totals are convenient.
Build the dashboard with Atoms
Atoms helps create and iterate on web applications from a written brief. A QA evidence dashboard can make scenario requirements, results and remaining actions easier to inspect. Begin with the synthetic CSV before integrating a real test reporter or deployment service.
Build a QA evidence dashboard using synthetic scenarios.
Store scenario trigger, expected outcome, required flag,
candidate, environment, result and evidence reference.
Support Passed, Failed, Blocked and Not run without merging
them. Import the sample CSV and show incomplete readiness
because SC-103 and SC-104 are unresolved. Keep result history
and reject duplicate run identifiers. Filter by candidate.
Start without automatic test-runner or deployment writes.
Start with the AI app builder guide. The context engineering guide explains how to retain the current candidate and unfinished work during handoff.
Verify the dashboard by importing the same fixture twice, changing the candidate and adding a Failed result. The required checks should remain visible, and readiness should reflect their current states. Then reload and inspect the saved evidence references.
Review the evidence behind the summary
Ask a reviewer to choose one Passed scenario and explain what the artifact establishes. Then ask them to locate the next action for a Blocked scenario. If they cannot do either, improve the relationship between the summary and its detail view.
Use counts to navigate the work, not to hide it. “Two passed, one blocked, one not run” is more actionable than a single percentage. Show the scenario and candidate beside the remaining action so the next check can be performed without reconstructing the original conversation.
Use the sample brief to start a QA evidence dashboard in Atoms.
Conclusion
A QA dashboard should preserve the relationship between required behavior, a release candidate and observable evidence. Keep incomplete states visible and read the artifacts behind the summary. Build the first dashboard with Atoms, then connect real reports only after the sample's counts, identities and readiness rules behave correctly.
Create a first version with synthetic data and review the results.
FAQs
Q1: Does a high test pass rate mean a release is ready?
It depends on which behaviors were checked and whether the results apply to the candidate and environment being released. Many passing tests can coexist with an untested permission boundary or missing persistence check. Define the required scenarios and inspect their current evidence individually. Keep blocked, failed and not-run states visible. A percentage can summarize execution, but it should not replace the release criteria. If an exception is accepted, show the decision and scope separately instead of changing the underlying result to make the summary appear complete.
Q2: What evidence should a QA dashboard store?
Store references that connect the expected behavior to an observed result: test report, saved-record readback, screenshot, trace or relevant request response. Include the candidate, environment and execution identity so the reviewer can determine whether the evidence is current. Explain the artifact's scope. A screenshot may prove a visible label, while a readback can establish what persisted. Keep the underlying report available and protect sensitive data. The dashboard should make it possible to investigate a status, rather than treating an imported pass label as sufficient proof on its own.
Q3: How should blocked tests affect release readiness?
A blocked check has not established the expected behavior. If it is required for the release, readiness remains incomplete until the dependency is resolved and the check runs, or an authorized exception is recorded. Identify the blocking condition and next action so the dashboard helps move the work forward. Do not merge blocked with passed or remove the scenario from the denominator without explanation. Retain the original state and any later execution. This makes the release decision understandable even after the dependency has been fixed and the latest result changes.
