Autonomous coding agent: from production logs to merged fixes

From the team’s talk on the project: how the agent works, from a task to a merged pull request.

agent pull requests merged in three months: 90% of those reviewed to a decision: 57
of merged pull requests contain no human-written code: 89%
median from pull request to merge: 2.1 days
in model costs per task: ~$20

The problem

Small work that never fit into a sprint

Every sprint, the team set aside time to read production logs, find real errors and fix them or file tickets. Around that sat a long tail of other small work: security findings, bugs only a few customers hit, a limit an ops colleague needed raised. Each was an hour of code wrapped in days of planning and waiting, and it was the first thing cut when the roadmap got busy.

Engineers already used coding assistants on their laptops. But a local assistant needs a person watching it, and that person has working hours and limited attention.

Diagram: browse error logs, prioritise by frequency and severity, then fix right away or create a ticket — Before the agent: an engineer reads the error logs, ranks them, then fixes one or files a ticket.

Phase 1 · May — June

An agent with its own dev environment

01Architecture

The same setup an engineer has, in the cloud

An agent that can only read and edit code can’t check its own work. So every task gets a short-lived cloud sandbox with the application running, seeded test data, a browser and a terminal, all on infrastructure the client already had.

Sandbox. AWS CodeBuild starts an isolated container per task with Node.js, Docker, Playwright and the application’s database, cache, queue and search services.
Orchestrator. A TypeScript app on the Claude Agent SDK. Code handles provisioning, git, pull requests and tickets; investigator, implementer and verifier sub-agents do the thinking.
Verification harness. Formatting, linting, types, unit tests, diff coverage, integration tests selected for the diff and a QA pass in the browser, retried until they pass or the agent explains why it stopped.
Observability. Logs in CloudWatch, sessions and screenshots in S3, and a skill that turns any run into a readable report.

How a run flows: work arrives from tickets, logs, schedules or chat, a dispatcher starts a sandbox, and the change comes back validated.

02First production line

Production logs, triaged every day

The first job was the one the team liked least. Every day a sandbox pulls the latest errors, groups and ranks them, filters out transient and third-party noise and picks one the code can fix, skipping anything an open pull request already covers.

If there is nothing to fix, it shuts down in under three minutes for a few cents. If there is, the agent reproduces the error, fixes it and opens a pull request with the diagnosis.

TriagePull, group and rank errors; pick one fixable candidate
FixReproduce the error and change the code
VerifyRun the gates and loop until they pass
DeliverOpen a pull request, shut the sandbox down

Diagram of the daily run: provision a sandbox, pull error logs, triage, prepare the workspace, fix, verify, open a pull request; or shut down if nothing can be fixed — The daily log run: triage, fix, verify and open a pull request, or shut down if there is nothing to fix.

Phase 2 · July — August

From log errors to any task

03Any task

A button on every ticket

Tickets. A ticket automation calls the same dispatcher. Anyone can press the button; the agent plans, implements, verifies, opens a pull request and comments on the ticket with screenshots.
Pull request mentions. Reviewers mention the agent to address feedback, and it pushes a commit to the same branch. It can also just test: “walk through this checklist and report back”.
Skills and memory. Skills in each repository explain how to boot the app, which test accounts have which roles and how to reproduce common flows. After every run the agent stores up to three lessons and loads the relevant ones next time, so it stops repeating mistakes.

Diagram of the ticket run: ticket automation, provision a sandbox, pull the ticket, plan, implement, verify, run the gates, open a pull request and comment on the ticket — The ticket run: one click starts a sandbox that plans, implements, verifies and reports back on the ticket.

Diagram of agent memory: pull relevant learnings, execute the task, derive up to three learnings, push them to the memory repository — Memory: after each run the agent keeps up to three lessons and loads the relevant ones next time.

Diagram of skills: consumer-specific skills live in each app repository, global skills in the orchestrator — Skills: how to log in, seed data and reproduce a flow live next to each app’s code; shared skills live in the orchestrator.

04The new bottleneck

Code arrived faster than trust

Within weeks the agent opened pull requests about as fast as the rest of the team, and the open queue grew from 15 to 50. Testing a branch meant finding a free test server among four shared by 13 engineers and waiting up to an hour for a deploy.

So every pull request now gets its own preview environment with isolated, seeded data, torn down on merge. Once it is live, Claude picks the relevant end-to-end tests from a three-and-a-half-hour suite, runs them and posts a summary next to an automated code review. A reviewer can check a change from a phone.

Open agent PRs: 15 → 50
Test environments: 1 per PR
End-to-end tests: ≤ 25 specs

Diagram of the preview lifecycle: deploy on pull request open, redeploy on push, delete on close or merge, and a nightly job removing previews idle for five days — A preview per pull request: deployed on open, updated on every push, removed on merge or after five idle days.

Diagram of selective end-to-end tests: wait for the preview, map the diff to at most 25 specs, run them, generate and analyse the report, comment on the pull request — End-to-end tests picked for the diff: at most 25 specs run against the preview, with the report analysed in the pull request.

Diagram of the review loop: trigger from a ticket, pull request, preview, end-to-end report, code review; tag the agent for feedback, redeploy, re-run tests, merge, tear the preview down — The review loop: everything a reviewer needs arrives on the pull request, and the agent addresses feedback until it merges.

August — October 2026 · The outcome

A second engineering lane, open to the whole team.

QA engineers find a bug in end-to-end tests, hand it to the agent and ship the fix without a developer. Product managers pick suitable roadmap items and ship them themselves. Production errors that used to wait for a log review now arrive as pull requests with a fix.

Diagram: a work queue of tasks and pull requests, a pool of agent sandboxes, and engineers’ attention going to review — A pool of sandboxes works the queue; engineers spend their attention on reviewing what comes back.

Three months in production

What the numbers say

From August to October the agent ran 518 times and opened 88 pull requests. Of the 63 reviewed to a decision, 57 were merged. The merged changes were real work: a median of 160 lines across three files.

In practice that meant security fixes merged in two days without interrupting a sprint, permission changes the product owner ships in days instead of one to two weeks, bugs that never made the priority cut, and even a full role-management feature across 44 files that the team was about to turn down.

sandbox runs: 518
pull requests opened: 88
tickets picked up: 65

Examples of agent work: security fixes, ops-reported bugs, small business-logic updates, development bugs, backlog leftovers, dependency checks, and a 44-file role-management feature — The kind of work it shipped, from security fixes to a role-management feature across 44 files.

Looking back

What made it work

01
Same tools as an engineer
An agent that can run the app, the tests and a browser can check its own work. Most of the effort went into the sandbox, not the prompts.
02
Build trust, not just output
Previews, selected end-to-end tests and automated review turned a pile of pull requests into changes anyone can check in minutes.
03
Open it to everyone
Beyond the developers, QA and product teams took to it too. A button in tools they already use did more for adoption than any feature.