Deploy-Then-Self-Test — "Shipped" Only After the Agent's Own Live Checks Pass
The deploy finishing is when the verification starts, not when the work ends.
What was getting in the way.
A deploy pipeline tells the agent the build succeeded; the agent tells a human "shipped." The gap between those two sentences is where the small, confidence-destroying failures live. The branch deployed was the one the pipeline watched, but the fix the human asked for landed on a different branch that got merged late. A cache layer continues to serve the pre-fix bytes on the exact URL the human will visit. An asset's hashed pointer updated in source but the deployed bundle still references the old hash. The sitemap didn't get the new route added, so the page exists but no one can find it. A robots meta slipped onto a published page because of a mode flag that didn't get flipped. Each of these is invisible to the pipeline's success signal and obvious to anyone who actually visits the URL. The agent, having told a human it shipped, finds out about these one by one as the human notices them hours or days later — the expensive way to find any of them.
Context
Universal — the pattern applies to any AI agent shipping code or content changes to a surface a human will visit, in any stack where a deploy pipeline exists.
How the work runs.
- 01
Lock the proof statements into the brief itself
Before the deploy runs, the agent writes down what the verification will test: which route must return 200, which distinctive copy phrase must appear exactly once, which asset hash must match the source, which routes must stay unchanged, which sitemap entry must appear, which robots meta must be empty on a published page or present on a draft. These proof statements are part of the brief the deploy executes against — the verification can't be weaker than what the brief promised.
- 02
Wait for the deploy pipeline, then start the real work
The pipeline reporting success is the trigger, not the finish. The agent waits the normal propagation window (CDN warm-up, cache purge, DNS where applicable), then begins the live checks against the public URL — never against a staging preview or a cache-busted debug variant.
- 03
Fetch the deployed route the human would type
The exact URL from the brief gets fetched with the same client characteristics a human visitor would present. The response code must be 200; the content type must match what the route is supposed to serve; no unexpected redirect chain. A single-page-app that returns 200 for every URL is not evidence — the agent reads the rendered HTML to confirm the real route resolved.
- 04
Grep the served HTML for the exact phrase the fix would have produced
A copy phrase, a slug, a product name, a price string — something that could only exist in the response if the fix is live. The agent counts occurrences: zero means the fix didn't ship; one is the normal pass; more than one means something is being double-rendered. The grep targets the served HTML, never a cached copy, never an admin view.
- 05
Verify the served asset bytes, not the pointer
For every image, PDF, or script the fix introduced or changed, the agent fetches the served bytes from the hashed CDN URL the live page actually references and compares the hash to the source file. A matching hash proves the user will see the asset the agent built; a mismatch means something between source and user is stale.
- 06
Run the standing absence checklist
The agent reads the sitemap and confirms the new route is present at the expected priority (and only once); reads the robots meta and confirms it matches publication intent (empty on published, noindex on draft); greps for accidental identifier leakage from any other client in the deployed response; checks that the routes the fix was not supposed to touch still carry their own expected copy phrases. Every absence that was supposed to be absent gets confirmed absent.
- 07
Only now report "shipped," with the proof attached
The completion message carries the proof — route code, phrase hit count, asset byte hash, sitemap present, robots empty — not just the deploy ID. If any check failed, the message is a deviation report, not a success report, and the next action is spelled out plainly. The human never has to ask "did you actually check the live site" — the evidence is on the message.
- 08
Lesson every failure into the next deploy's brief
When a check misses a class of failure, a new proof statement joins the standing template so the next deploy inherits it. The agent's post-deploy checklist is the single thing that compounds across every shipped change.
Evidence from the workflow.
Each system has a role.
Intake (success triggers the verification, never substitutes for it)
Deploy pipeline / hosting
Action (fetches the live URL as a human would)
Headless browser
Signal (sources the proof statements and the hash to compare against)
Content and asset repository
Record of truth (the standing post-deploy checklist that compounds)
Agent playbook
Why this is Coworker.
Takes assignments and reports back. You hand it work, it prepares and returns a result.
- 01
The deploy pipeline reporting success is treated as an input signal, not as evidence. The agent reports "shipped" only after live checks against the public URL pass; a failing check becomes a deviation in the same completion message and never gets softened into a follow-up. New proof statements are added to the standing checklist whenever an incident reveals a check that was missing.
Impact / Outcomes
A deploy that reported green but landed stale bytes on the live URL is caught by the byte-hash check in the same turn — the agent reports the deviation before any "it's live" message reaches a human.
A sitemap-missing or robots-slipping regression that would have taken days to notice through search behavior shows up as a one-line deviation on the completion message.
"Shipped" messages carry their own evidence attached — route code, phrase count, hash match, sitemap present, robots empty — so a human can audit the ship without rerunning the check.
The post-deploy checklist is the single thing that gets stronger with every incident; no deploy benefits from a lesson the next deploy's brief doesn't inherit.
Find where a workflow like this fits.
Start with the systems, work, constraints, and authority already present in your operation.