Case study · AI-native engineering
In short: the engineers were already using AI tools, individually and without governance, and nobody could say what it had done for delivery. We set up an AI-native engineering framework: a business plan with proper controls, guardrails on every change, skills for each role, and measurement. Within a quarter the share of delivered work executed by AI went from 46% to 81% on the flagship product, throughput roughly doubled, every change was still reviewed by a person, and no production defect was traced to AI-written code in the measured quarter.
The organisation builds and runs a large platform for enterprise and government customers: a few dozen services, two front ends, and strict security and audit obligations. Its engineers had adopted AI coding tools on their own initiative. Some used them daily; some did not trust them; nobody had agreed how they should be used, what they were allowed to see, or how to tell whether they were helping.
The situation is a familiar one in software teams of every size. Personal AI accounts, unclear data handling, no shared way of working, and results that varied with the individual rather than the process. Delivery was not noticeably faster, review load had gone up, and the organisation's auditors would have had reasonable questions about what had been sent where.
Every change had to remain reviewed and approved by a person, with no fast lane for AI-written code. Protected branches, explicit approvals and full traceability from requirement to release had to stay intact, because the client's own customers audit them. And the result had to be measurable, since a change in how a whole organisation works cannot be justified by anecdote.
First, the foundations: a business AI plan under the organisation's own control, attributable access for each person, no personal subscriptions, and clear rules on what the tools may read and write. Second, the framework around the work: the organisation's coding standards, architecture rules and security checklists were turned into checks that run on every change, so a change that does not meet them does not reach a reviewer. Third, skills for each role, so that grooming a ticket, implementing it, reviewing it and investigating an incident each follow the organisation's own practice rather than the tool's defaults.
Then we measured. Telemetry showed, per team and per quarter, how much delivered work was executed by AI, how long changes took, and what happened to quality. The numbers, not the enthusiasm, decided which teams were onboarded next and where the enablement effort went. Engineers' roles shifted toward design, review and judgment; the practices were handed over so the organisation runs and extends the framework itself.
| Measure | Before | After one quarter |
|---|---|---|
| Delivered tasks executed by AI, flagship product | 46% | 81% |
| Delivered tasks executed by AI, front end | 7% | 77% |
| Delivery throughput | baseline | about 2× |
| Changes reviewed by a person before merge | 100% | 100% |
| Production defects traced to AI-written code | not measured | none in the measured quarter |
| Overnight test-failure triage | manual, before standup | automated report, none manual |
Note that the figures are the organisation's own telemetry at two points in time; the throughput figure is a ratio of delivered tasks between consecutive quarters, and the defect figure covers AI-delivered work attributed through the organisation's incident process over the measured period.
The lesson is the same one we apply when we set up Claude or ChatGPT for a small business: the tool is the easy part. The value comes from the plan, the permissions, the rules and the skills around it, and from measuring what it does.
Services: AI-native engineering · The knowledge base
Case studies: Standards enforced by the build · The knowledge base that found the defects · All case studies
Book a free 30-minute call. We will look at how your people use AI today and what an AI-native framework would change.
