Short answer

Judge AI-assisted engineering by what reaches users safely, not by how much code is written. Track lead time, review wait, change failures and recovery, escaped defects, security findings and user acceptance, before and during a small controlled pilot on comparable work.

The decision to make is not whether to use AI tools at all. It is under what conditions they improve your delivery, and how you will know if they stop doing so. Set the baseline, the release criteria and a stop condition before the pilot starts.

Many engineering leaders are now under pressure to show a return on coding assistants. The easy numbers to report are lines of code, pull requests opened or developer-reported time saved. None of them tells you whether customers received better software sooner. A team can produce more code and deliver less value if the extra code waits in review, fails in testing or creates defects that surface after release.

Why code volume is the wrong measure

Writing code is one stage in a longer system. A change still needs to be understood by a reviewer, tested, checked for security implications, integrated with other work, released and supported. If an assistant makes the first stage faster and a downstream stage cannot absorb the extra throughput, the queue can move to that stage. Review is one candidate, but measure where work actually waits instead of assuming it.

More code also means more to maintain. Code that nobody on the team fully understands is expensive to change later, and generated code can look plausible while missing an edge case the author never considered. Our view is that the useful question is whether the whole system moves faster with the same or better quality, not whether one stage got quicker.

What the research supports, and what it does not

The most useful published research frames AI as a system-level effect rather than a fixed speed-up. The DORA 2025 report on AI-assisted software development describes AI as an amplifier: it tends to magnify an organisation’s existing strengths and weaknesses. Teams with healthy practices are better placed to benefit; teams with weak testing, unclear priorities or fragile release processes may see those problems grow.

Google Research’s page for the DORA AI Capabilities Model presents this as 2025 research and links the full model, which sets out organisational capabilities associated with getting value from AI. Read the model itself before relying on any specific capability; we do not summarise it as a checklist here.

Independent experiments show how hard the measurement is. METR’s February 2026 update on its developer productivity research explains that its later study was affected by which participants and tasks were included, and that time reporting became unreliable when people ran agents in parallel. It considers a speed-up plausible but describes the evidence for its size as weak.

Taken together, these findings are not a universal productivity rate. They do not say AI always speeds delivery up or always slows it down, and a result from one tool generation, team or task type does not transfer automatically to yours. That is why the rest of this guide is about measuring your own delivery system.

The delivery scorecard

The scorecard below is ByteIQ’s proposed set of measures for an AI-assisted delivery pilot. Several of them are familiar delivery measures; the warning signs are the patterns we would look for when assistants are introduced. Record a baseline for each before the pilot begins.

ByteIQ’s proposed delivery scorecard for AI-assisted engineering
MeasureWhat to recordWarning signUsual owner
End-to-end work lead timeTime from work starting to the change running in production. This is our chosen measure and it starts earlier than DORA’s change lead time, which begins when the change is committed to version control.Coding time falls but total lead time does not.Engineering manager
Review queueHow long changes wait for review, and how large they are.Larger changes and longer waits; reviewers skimming.Tech lead
Change failure and recoveryShare of releases needing a fix or rollback, and time to recover.More rollbacks or slower recovery after the pilot starts.Platform or operations lead
Escaped defectsDefects found by users or after release, by severity.Defects in code paths nobody on the team can explain.QA or product owner
Security reviewFindings from security review and automated scanning.New dependencies, secrets or insecure patterns arriving with generated code.Security lead
User acceptanceWhether delivered work meets the acceptance criteria first time.Features that pass tests but miss what users needed.Product owner
Team understandingWhether the author can explain the change in review.“The assistant wrote it” offered as an explanation.Tech lead

This scorecard is our own synthesis for business decision-makers. It is not a DORA metric set. Change failure and recovery will be familiar to teams who use DORA’s measures, but our work lead time deliberately starts at the beginning of the work, so do not compare it with DORA’s commit-to-production change lead time or mix the two start points in one comparison.

Where the bottleneck moves

The diagram compares a typical flow with an AI-assisted flow that has explicit quality gates. It is illustrative: the point is where checks sit, not how long any step takes.

Typical flow

  1. Ticket with a short description
  2. Developer writes the change
  3. Peer review
  4. Tests run in the pipeline
  5. Release

AI-assisted flow with gates

  1. Ticket with acceptance criteria and examples
  2. Developer and assistant draft the change in small steps
  3. Automated tests, linting and dependency checks must pass
  4. Review: the author explains the change and its risks
  5. Security review for sensitive areas
  6. Staged release with monitoring and a rollback plan

The second flow is not slower by design. Clear acceptance criteria help both the developer and the assistant, small changes are faster to review, and automated checks catch problems before a person spends time on them. The gates exist because, if assistants make changes cheaper to produce, the cost of a bad change reaching users may not fall with it. Check whether that is true for your team.

Designing a fair pilot

A simple before-and-after timing comparison can be confounded by everything else that changed. Plan the pilot so that you can tell the effect of the tools apart from everything else that changed.

  • Comparable work. Choose a stream of similar tasks, such as small features or defect fixes in one product area, and keep comparable work outside the pilot as a reference.
  • An explicit baseline. Record the scorecard measures for the same kind of work over a recent period before the pilot.
  • Unchanged release criteria. The definition of done, test requirements and review standards stay the same, so quality is not traded quietly for speed.
  • Known confounders. Task difficulty, team changes, holidays, a release freeze, the novelty effect and the learning curve can all move the numbers. Note them as they happen.
  • Selection effects. If people choose which tasks to use the assistant on, they will pick ones it suits. Agree the task stream in advance.
  • Data handling. Confirm which code, data and documents may be shared with the tool, and under what terms, before anyone starts.

When to slow down or stop

Set a stop condition before the pilot starts. A practical one is: pause new assisted work if the review queue keeps growing for an agreed number of weeks, or if escaped defects or security findings rise above the baseline, until the cause is understood. Beyond that, these are situations where we would advise caution.

  • Review capacity is already the constraint. Producing changes faster in front of an overloaded review stage can lengthen waits and push reviewers to skim, so check queue time and escaped defects.
  • Tests are thin. Without good automated tests, nobody can tell quickly whether a plausible-looking change is correct.
  • Safety-critical or heavily regulated code. Changes that could harm people or breach regulation need specialist review regardless of who or what wrote them.
  • Unclear data and licensing terms. If you do not know what the tool may retain or how generated code is licensed in your context, resolve that first.
  • Developers still learning the codebase. Generated code can hide gaps in understanding. Pair it with explanation in review rather than replacing the learning.

Next steps

  1. Pick one product area and a stream of comparable work for a time-boxed pilot. Owner: engineering manager with product owner
  2. Record a baseline for each scorecard measure from recent comparable work. Owner: engineering manager
  3. Confirm data-sharing and licensing terms for the tools and write a short usage policy. Owner: CTO with security and legal advisers
  4. Add or tighten the quality gates: automated tests, dependency checks and the rule that authors explain their changes. Owner: tech lead
  5. Agree the stop condition and who can call it, then run the pilot and log confounders as they happen. Owner: delivery lead
  6. Review the scorecard against the baseline and decide to expand, adjust or stop. Owner: CTO

Our Software Development service can help you put gates like these in place, Cloud & DevOps covers pipelines that support repeatable checks and release control, and Program Delivery helps when the pilot spans several teams. To talk about a pilot of your own, contact us about software delivery.

Sources and further reading

  1. DORA 2025 report on AI-assisted software development. Frames AI as an amplifier of existing organisational strengths and weaknesses; links the full report and the capabilities model.
  2. DORA AI Capabilities Model, Google Research publication page. Identified as 2025 research; links the full model.
  3. METR uplift study update, 24 February 2026. Explains participant and task selection effects and unreliable time reporting with parallel agents; treats the size of any speed-up as weakly evidenced.

About this guide: written by the ByteIQ Labs editorial team. The scorecard, flows and pilot design are our own analysis and contain no measured results. Research findings are summarised briefly and linked so you can read the methods yourself. More guides are on our Insights page.