Skip to main content

Generating tests with an AI agent

Instead of hand-writing the .ts transaction, or recording it once with codegen and hoping the selectors hold, a local coding agent — Claude Code, Codex, or similar — can drive a real browser through the flow and write the transaction for you, validating every locator against the live target before handing it over. This catches exactly the class of bug that makes a transaction flaky in production (duplicate status badges, ambiguous hasText filters, a CSS-only visual match that isn't really unique in the DOM) — things plain codegen cannot, because it only records literal actions without checking whether the resulting locator is actually unique.

1. Install a Playwright browser-automation capability

The agent needs a tool that can open a real browser and drive it (click, fill, read the DOM), not just guess from a screenshot. The standard way is the official Playwright MCP server, @playwright/mcp (by Microsoft) — it exposes browser control as MCP tools to any MCP-compatible agent.

Claude Code:

claude mcp add playwright -- npx @playwright/mcp@latest

(Anthropic's official plugin marketplace also ships a ready-made "playwright" plugin that wires this same MCP server — either path works.)

Codex CLI:

codex mcp add playwright -- npx @playwright/mcp@latest

Both commands register the server for stdio use; npx fetches @playwright/mcp on first run. This needs Node available on the machine running the agent — not the plugin's Docker image, since this step happens locally, before the test file even exists.

2. Prompt template

Validate [FLOW NAME] on [URL] and write it as a Playwright transaction for the
pandorafms.playwright.1 plugin.

What the transaction should check:
- [step 1, e.g. "open the page and confirm the title"]
- [step 2, e.g. "log in and confirm the dashboard loads"]
- [step 3, e.g. "read a value and publish it as a pandora.metric"]

Deliverable:
- Plain Playwright: wrap each meaningful step in a top-level `test.step('name', ...)`
  so it becomes a monitored phase, and use
  `test.info().annotations.push({ type: 'pandora.metric', description: 'name=value' })`
  for anything that should become a custom metric module.
- No PandoraFMS import, no DSL — this plugin harvests everything from
  Playwright's own JSON reporter.
- Validate every locator against the real target yourself (drive the browser,
  don't just infer from a snapshot) before handing me the file — fix anything
  ambiguous or strict-mode-violating first.
- If a later step depends on a hard assertion in an earlier step, tell me
  whether to keep it that way or switch to `expect.soft()` so every phase gets
  measured even when one fails.

3. After you get the file

Run it through the plugin locally before wiring it into a Discovery task — see Manual execution above — so you see the real agent/module output, not just "the test passed."

Back to top