How to Plan QA Tests With an Agent

QA test planning is the work of turning a feature, release, pull request, or product surface into a concrete list of checks before the change reaches users. A good plan starts from the user journey and the business risk, then breaks requirements into scenarios with setup, action, expected result, edge cases, and coverage gaps.

The value is not the size of the checklist. The value is knowing which failures matter, which checks can be automated now, which ones need exploratory testing, and which requirements are still too vague to release with confidence.

Why QA planning quietly costs you

QA planning fails in small teams for a simple reason: it happens after the build, when everyone wants the change shipped. The engineer remembers the happy path, the product owner remembers the acceptance criteria, and nobody has enough time left to ask how permissions, invalid input, state transitions, integrations, analytics, and recovery paths behave together.

That is how bugs slip through work that looked reviewed. The missing test was not always hard. It was often never named. A risk-based plan makes those gaps visible early enough to fix, delegate, or accept explicitly.

What the manual process looks like

Done by hand, QA planning is a short review ritual:

1. Gather the feature spec, release notes, pull request, acceptance criteria, and linked issue set.
2. Name the critical user journeys and the business risks if each journey fails.
3. Break requirements into functional cases: happy path, invalid input, boundaries, state transitions, permissions, integrations, and recovery.
4. Add non-functional checks where relevant: accessibility, performance, compatibility, security-sensitive behavior, analytics, and observability.
5. Write exploratory charters with a mission, data setup, and stop condition.
6. Compare the plan to existing test coverage and mark each scenario as automate now, automate later, manual exploratory, or not worth testing.

The process is practical, but it is easy to skip the uncomfortable parts. The questions that find real defects are the ones people postpone: what happens with half-configured data, a different role, a retry, a timeout, a stale page, or a missing permission.

What an agent can automate

An agent is useful here because the first draft of QA thinking is repetitive, structured, and better when it is exhaustive:

• Map requirements into scenarios. The agent decomposes the spec or PR into atomic cases with setup, actor, action, and observable expected result.
• Start from risk. The plan is organized around critical journeys and failure impact, not a generic quality checklist.
• Surface edge cases. Boundaries, invalid input, state transitions, permissions, integration failures, and recovery paths get named before the release review.
• Draft exploratory charters. Each charter has a mission, test data, and a stop condition so manual testing has shape.
• Call out coverage gaps. If a repository is connected, the agent inspects existing tests before calling something missing. Without repository access, it still produces a plan from the supplied spec and artifacts.

The agent should not silently ship product changes. Its job is to plan and review test coverage, then stop with gaps and questions for a human to approve.

The guardrails that make it safe

QA planning is advisory work, but it still needs boundaries. A noisy agent can create a huge plan that nobody runs. A loose agent can invent product behavior that is not in the spec.

The safe shape is a scoped assignment: feature under test, known risk areas, verification command, and an optional repository. The agent drafts the plan, marks open questions, and classifies automation targets. A human decides what to test, what to automate, and what risk to accept before release.

Set it up in Task Machine

The QA review and test planning playbook installs the QA Reviewer agent, two QA planning skills, the standing goal for visible release risks, and a starter follow-up task. Setup takes a few minutes. You need a Task Machine workspace and permission to install playbooks (workspace owners have it). A connected repository is optional. Until repository access is connected, the agent works from the spec, PR notes, and artifacts you attach.

1. Find the playbook

Open Playbooks in your workspace and search for "QA review and test planning", or browse the Engineering category. The card shows that the playbook creates one agent, one goal, and two skills.

2. Preview what it installs

Preview & install opens the full contents before anything is created: the QA Reviewer, the QA risks visible before release goal, and the qa-test-planner and breakdown-test skills. The preview also shows the requirement for a feature spec, release notes, or pull request, with connected repository access as optional.

3. Give the agent the QA planning context

Start setup asks for the repository, the feature or release under test, the risk areas, and the verification command. Use the feature field for the concrete change, not a broad product area. Use risk areas for the failures you already worry about, such as permissions, billing state, or file uploads.

4. Generate and review

Generate customized playbook turns those answers into the installed agent instructions and goal context. Review the result before creation. Confirm that the agent is scoped to planning and coverage review, that the repository context is right, and that the verification command matches the test surface.

5. Install

Install customized playbook creates the agent, goal, and skills. A follow-up lands in your inbox to start the first QA review and test planning assignment. That first task is where you attach the brief and approve the agent to produce the plan. The output waits for human review before anyone treats it as release guidance.

What good looks like

Three signs tell you the plan is doing its job:

• Every risky journey is named. The plan covers the paths that would hurt users or the business if they broke.
• Each scenario is testable. Setup, actor, action, and expected result are explicit enough for a human or implementation agent to run.
• Coverage gaps are actionable. A gap says what is missing, why it matters, and whether to automate now, automate later, test manually, or accept the risk.

Common questions

Can an agent replace a QA engineer? No. It can draft the test plan and find gaps, but release judgment stays with the human owner. The playbook is designed to stop at planning and approval.

What if there is no connected repository? The playbook still works from the supplied feature spec, release notes, pull request summary, or attached artifacts. Repository access improves coverage-gap analysis because the agent can inspect existing tests.

Should every scenario become an automated test? No. The plan should separate automate now, automate later, manual exploratory, and not worth testing. Some checks are better as short exploratory charters than permanent tests.

When should this run? Run it before implementation is considered done, not after release approval has already begun. The earlier the gaps are visible, the cheaper they are to fix.