AI Coding Agents for QA: Part 5 — Stop Writing Prompts. Start Writing Task Specs

Matviy Cherniavski

Mar 25 • AI&QA

You open Cursor, Copilot or whatever AI tool you like ...

You type: "write a login test"

The agent responds. It looks like a test. Imports are there. Structure looks familiar.

But you look closer.

Hardcoded credentials.
Wrong file location.
No page objects.
Naming convention are ignored.
And on top of all that, you run it... it fails.

────────────────────────────────────────

🧠 𝐖𝐡𝐲 𝐭𝐡𝐞 𝐀𝐠𝐞𝐧𝐭 𝐆𝐮𝐞𝐬𝐬𝐞𝐬 𝐖𝐫𝐨𝐧𝐠

Most people at this point blame the model.

"Claude is bad at tests."
"GPT doesn't understand Playwright."
"I need a better model."

But the reality is... the model did not fail you.

You gave it nothing useful to work with.

Think of the agent like a new hire. Smart. Fast. Capable.

But they have never seen your project before.

➤ They do not know where your fixtures live.

➤ They do not know how you name test files.

➤ They do not know what credential pattern you use.

➤ They do not know whether you run tests after every change.

You told them: "write a login test."

So they try to find all that information and make a lot of assumptions.

Every assumption is a guess. Every guess is a risk of being wrong.

That is an onboarding problem and a lack of proper documentation.

────────────────────────────────────────

📝 𝐖𝐡𝐚𝐭 𝐚 𝐑𝐞𝐚𝐥 𝐓𝐚𝐬𝐤 𝐒𝐩𝐞𝐜 𝐋𝐨𝐨𝐤𝐬 𝐋𝐢𝐤𝐞

In the AI coding agents world, that documentation is often called "Task Spec."

A task spec is not a longer prompt. It is a precise set of constraints that leaves the agent very little room to guess.

Here is the difference.

𝗪𝗲𝗮𝗸 𝗽𝗿𝗼𝗺𝗽𝘁:

```

write a login test

```

𝗚𝗼𝗼𝗱 𝗧𝗮𝘀𝗸 𝗦𝗽𝗲𝗰:

Write a login test.

Before making any changes, inspect the existing tests in /tests/auth/ and follow the existing suite structure, naming, and conventions.

Task:

- Add a test for successful login using the existing credentials fixture.

- Place it in the appropriate existing auth test suite.

- Do not hardcode credentials or duplicate fixture data.

- Do not create new files unless no existing test file is appropriate.

- Do not modify unrelated files.

- Do not guess. Work only from the concrete code and data in the repository. If expected login behavior is unclear from the existing tests or app code, stop and report the ambiguity.

Validation:

- Run the relevant auth test suite after making the change.

Report back with:

1. what you changed,

2. which test file and test name you added,

3. which tests you ran,

4. whether they passed or failed.

```

────────────────────────────────────────

⚡ 𝐓𝐡𝐞 𝟓 𝐓𝐡𝐢𝐧𝐠𝐬 𝐚 𝐓𝐚𝐬𝐤 𝐒𝐩𝐞𝐜 𝐀𝐥𝐰𝐚𝐲𝐬 𝐍𝐞𝐞𝐝𝐬 ∙

𝐆𝐨𝐚𝐥: What exactly should be done
𝐂𝐨𝐧𝐭𝐞𝐱𝐭: Where to look first, what already exists
𝐂𝐨𝐧𝐬𝐭𝐫𝐚𝐢𝐧𝐭𝐬: What not to do, what rules to follow
𝐒𝐮𝐜𝐜𝐞𝐬𝐬 𝐜𝐫𝐢𝐭𝐞𝐫𝐢𝐚: What done actually looks like
𝐕𝐞𝐫𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧: Run the tests, report the result

This task spec guides the agent and sets the clear expectation.

That leads to the agent writing extremely high-quality code that matches your project standards.

────────────────────────────────────────

If you want to learn how to write task specs and use AI coding agents the right way in your test automation work, read the Start Here post in this community.

It covers exactly what we build in the live workshop.

👉 Read it here: https://www.skool.com/qa-automation-career-hub/welcome-to-qa-automation-roadmap-lab-start-here

6 comments