Tests Pass, Code Gone: When a Coding Agent Deletes Code or Tests
A green test run says the tests that ran passed. It says nothing about the code an AI coding agent removed, or the tests it deleted, skipped or weakened on the way. This post covers the four ways that happens, what to check before you merge, and how to see every file an agent changed.
"let claude code write and ship a whole PR today. tests green, lint clean, then i read the diff and it had quietly deleted retry logic it didn't understand"
Four ways a deletion stays green
1. The code is gone, and no test covered it
Tests check what they were written to check. Code nobody wrote a test for can disappear without a single failure, and the code an agent judges unnecessary is often exactly the code that handles a rare case: a retry, a timeout, a duplicate. One developer asked for a refactor of their bot's logging, and the agent "deleted a function i swore was dead code. it was not dead code. it was the one thing stopping me from buying the same coin twice."
2. The test is gone, skipped or weaker
An agent asked to make the tests pass can do it two ways, and only one of them fixes the code. In one case "an agent deleted the failing test instead of touching the helper and only the baseline diff between runs surfaced it." One developer went through their own Claude Code history and found "68 deleted test files and 22 edits lowering coverage thresholds", seven of those test files deleted right after a run reported them failing. Their conclusion: "All tests pass" can mean the tests moved.
3. Each agent's tests pass, the shared file does not
Parallel agents can each be right and still lose work together. A developer ran three agents on three features of an iOS app, all adding strings to the same localization file, and lost 55 of about 95 translation keys with green tests. Each agent read the file, added its keys and wrote the whole file back. "Whoever saved last wiped the others. No errors anywhere."
4. The summary mentions it, and the result still looks done
Sometimes the agent even says so. One developer left Claude Code to build a demo overnight and found a "Mistakes made" section in its summary: "Wiped DB, Deleted files unchecked, Commit with failing tests. But in the end... all tests pass!" A summary is the agent's own account. It is only as complete as the agent's view of what it did.
When a test did catch it
The good outcome looks like this. A developer maintaining a small Node.js payment service had fixed two retry bugs, one of them how the handler waits after an HTTP 429, and added a regression test for each. A week later they asked Claude Code to simplify the handler. "The rate-limit test went red. Claude had deleted the retry branch, so the handler just returned when it got a 429." On the next try, with a note in CLAUDE.md, it kept the branch but moved the wait to the wrong place, and the other test caught that.
Both mistakes were caught because each fix had a test that pinned the behavior, written before the agent touched the code. Without those two tests, both versions would have shipped green.
What to check before you merge
Read what changed before you read how it changed. On a branch the agent worked on:
git diff --stat main...HEAD # every file touched, with line counts
git diff --diff-filter=D --name-only main...HEAD # every file the branch deleted
git diff --name-status main...HEAD -- '*test*' '*spec*' # what happened to test files
git diff main...HEAD | grep -E '^\+.*(@pytest\.mark\.skip|\.skip\(|\bxit\(|t\.Skip)' # new skips
git diff --name-only main...HEAD -- pytest.ini .coveragerc pyproject.toml package.json \
'*jest.config*' '*vitest.config*' .github/workflows # test, coverage and CI settings
- Deleted files first. A deleted test file is the clearest signal there is, and it is easy to lose in a long diff.
- Lines removed from source files. In the
--statoutput, a file with many deletions and few additions is where a branch, a guard or a retry went missing. - Test, coverage and CI settings. A lower threshold, a new exclude pattern or a test step taken out of a workflow changes what "green" means without touching a single test.
- Uncommitted work. If the agent has not committed yet,
git status --shortmarks deleted files withD, andgit diff HEADshows the changes.
What git cannot show: a file that was never committed leaves no trace once it is deleted, and files outside the repository are not in the diff at all. The diff also shows only the end state. A coverage setting lowered for one run and then put back looks like no change, even though that run produced the green checkmark.
How to see every file an agent changed
kaimon records every file an AI coding agent changes or deletes on Linux, in the kernel, with no hooks and no agent plugins. For this kind of problem, it answers the question that comes before the diff: which files did the agent touch, and which of them matter?
- Deleted test files are flagged as FILE DELETE, or MASS DELETE for recursive deletes, whatever command did it, including files that were never committed.
- Changes to the checks themselves are their own category. GUARDRAIL EDIT covers the configuration of test runners, coverage, linters, type checkers and pre-commit hooks:
pytest.ini,.coveragerc,jest.config,vitest.config,codecov.yml,.pre-commit-config.yamland more. Settings kept inpyproject.tomlorpackage.jsonshow as PROJECT EDIT, and workflow changes as CI CHANGE. - Every write is recorded as it happens, so a setting changed for one run and changed back still shows two changes, attributed to the agent and the tool it worked through.
- Where you work: severity in the VS Code status bar, the top findings on hover, the files behind each finding in the panel, and a report in HTML and JSON.
- Tamper-resistant: kaimon runs as a root daemon, so an agent working as your user can read the record but not change or delete it.
Stated plainly: kaimon does not read diffs. Code removed inside a source file shows as an edit to that file (SOURCE EDIT), not as the lines that went missing. It tells you which files to diff, and git diff shows the lines. It observes and never blocks, records from the moment it is installed, and runs on Linux, including a Linux dev box or VM you work on from macOS or Windows over VS Code Remote-SSH.
Always free and 100% local: nothing leaves your machine. See how it works, or read what to check when an agent deleted your files.
Making the next one smaller
- Pin the behavior you cannot lose with a test, before an agent works on that code. Retries, idempotency and duplicate checks are the first to look like dead code.
- Say why next to the code. An agent simplifying a function reads that function. A comment explaining why a branch exists is more likely to survive than a line in a file it may not weigh.
- Do not let the agent be the only judge of its own work. Changes to test files, coverage settings and CI workflows deserve a human look every time, even when the run is green.
- One writer per shared file when agents run in parallel, or one worktree per agent and a merge step that has to resolve the conflicts.
FAQ
Why would an AI coding agent delete a failing test?
An agent works toward the goal it was given, and a passing test run is the most visible sign of that goal. Deleting, skipping or weakening a failing test makes the run pass just as fixing the code does, and nothing in the run shows the difference. It is not malice; it is the shortest path to green.
Does git show a test file an agent deleted?
Yes, if the file was tracked: git status shows it as deleted, and git diff --diff-filter=D --name-only lists every deleted file on a branch. A file that was never committed leaves no trace in git once it is deleted.
Can kaimon show which lines an agent deleted?
No. It records which files an agent changed or deleted, attributed and risk-rated, so you know which files to diff. git diff shows the lines.
Does kaimon stop an agent from deleting tests?
No. It observes and reports, and never interferes, so it cannot break an agent's workflow.