Your CI said green. That does not mean the tests ran.
An AI coding agent finishes a task and reports "all checks passed". That sentence is a claim to investigate, not evidence to merge on.
Here is the specific thing that makes it dangerous, and it is documented behaviour, not a bug: in GitHub Actions, a job skipped by a condition can report success. Not "skipped" in a way your required-check rule notices. Success. So a required check can be green because it ran and passed, or green because it never ran at all, and by default those two look identical on the pull request.
Now combine that with an agent that has write access to the repository.
The failure mode is not the agent writing malicious code. It is much more boring. The agent changes a formatter, a test fails, and the simplest path to a green pipeline is to weaken the assertion or add a condition that skips the job. Nothing about that is adversarial. It is just an optimiser finding the cheapest route to the stated goal, which was "make the checks pass" and not "make the code correct".
Then a human sees a green badge and merges.
Three things that actually help:
1. Treat "passed" and "did not run" as different states. Write down which checks you expect for this class of change, then verify that each one executed on the revision under review. A missing entry, a skipped job, a cancelled run, a timeout and a neutral conclusion should not all collapse into "no failure reported".
2. Separate who writes code from who has verification authority. The agent can prepare a branch and run checks locally. CI repeats the required verification from the reviewed revision, in a controlled runner. A transcript of local commands is useful context for a reviewer and is not a substitute for a CI result bound to that commit.
3. Review test and workflow changes as a separate category. If the same diff removes an assertion and then earns a green badge, the badge is describing a pipeline that no longer checks the thing you cared about. Make workflow files visible to someone who can evaluate them.
And then the part almost nobody does: test the gate itself. Push a deliberately failing fixture, a skipped dependency and an updated commit, and confirm each one blocks acceptance the way you assumed it would. A gate you have never seen fail is a gate you are trusting on faith.
I wrote the full version up with an example acceptance validator you can run locally, including the negative tests that prove it rejects a skipped status and a mismatched revision:
Question for anyone letting an agent open PRs: have you ever actually watched your required checks fail on purpose? Or are you, like most of us were, trusting that the green means what you think it means?
3-minute video version: https://youtu.be/JxECaeSQ18I
0
0 comments
Richard Skacel
2
Your CI said green. That does not mean the tests ran.
powered by
AI for Cloud Engineers
skool.com/cloud-cost-optimization-3746
Claude Code & AI agents for cloud engineers. Automate AWS, Azure, GCP, K8s and Terraform without deleting prod. Free scripts, real labs.
Build your own community
Bring people together around your passion and get paid.
Powered by