Short answer

If you can write the steps down and the inputs are predictable, use conventional rules-based automation. If the steps are fixed but one of them needs judgement about messy input, such as classifying an email or pulling fields out of a scanned document, put an AI step inside a fixed workflow. Reserve a true agent, where the model chooses its own next action and tools, for work whose path genuinely cannot be known in advance and where every action it can take is either reversible or checked by a person.

We suggest starting in the middle category. The decision is less about how clever the model is and more about how much control you are prepared to hand over, and how quickly you will notice when the output is wrong.

Operations teams are being offered “agents” for invoices, customer emails, onboarding and reporting. Some of these products are rules engines with a chat window; some are fixed pipelines with one model call; a few genuinely let a model decide what to do next. They carry very different risks and running costs, yet they are often evaluated with the same impressive demonstration. This guide gives you a way to tell them apart and to choose a first project that is small enough to evaluate honestly.

Three things that get called automation

Anthropic’s engineering essay Building effective agents, first published in December 2024 and now carrying a note that its tooling examples have aged, draws a useful architectural line. In a workflow, models and tools follow code paths a developer defined in advance; in an agent, the model directs its own process and decides which tools to use. The essay also advises starting with the simplest solution that works and adding complexity only when it clearly improves results. We find it practical to split the territory into three options.

Rules-based automation

Scheduled scripts, integration platforms and “if this, then that” rules. Every input follows a path someone wrote down. Running costs are usually low and stable, and each rule can be tested on known inputs. An input that does not fit may stop the process visibly, or may pass through unnoticed unless failures are monitored, so alerts for failed or skipped items are part of the design. Its weakness is variety: a supplier who changes their invoice layout or a customer who writes in an unexpected way breaks the rule, and someone has to update it.

An AI step inside a fixed workflow

The sequence of steps is still fixed, but one or more steps use a model to interpret unstructured input: classify a request, extract named fields, summarise a document or draft a reply. The model’s output is constrained to a defined format and checked by ordinary code before anything happens. Because the path is fixed, you can test it against a set of past cases and see exactly where it fails.

An agent

The model is given a goal, a set of tools and permission to loop: look something up, decide what to do next, act, check the result and continue. That flexibility is valuable for open-ended investigation, such as gathering context across several systems before a person makes a decision. It also means the sequence of actions differs from run to run, the cost per task varies, and testing has to cover behaviour rather than a single output.

Choosing between them

Our view is that the choice should be made process by process, using the characteristics of the work rather than the ambition of the tool. The table below is ByteIQ’s proposed decision framework. Read across each row for the process you have in mind; if most of your answers fall in the first two columns, an agent adds risk without adding value.

ByteIQ’s proposed decision framework: rules, AI-assisted workflow or agent
QuestionRules-based automationAI step in a fixed workflowAgent
How varied is the input?Structured and predictable: forms, system records, fixed file layouts.Unstructured but of a known kind: emails, PDFs, free-text requests.Open-ended: the information needed is not known until the work starts.
Who decides the next step?The rule author, in advance.The workflow designer, in advance; the model only fills in a step.The model, at run time, within the tools it has been given.
What can it do?Exactly the actions coded.Actions coded after the model’s output passes validation.Any action its tools allow, so the tool list is the real permission boundary.
How do you evaluate it?Unit tests on known inputs.A representative set of past cases with known correct outputs.Scenario tests of whole runs, including attempts to push it outside its scope.
How reversible are mistakes?Each step is predictable and traceable, but a wrong rule can still make an irreversible change at scale.Validation and review can catch many bad outputs; plausible wrong results can still pass them.Depends on its permissions, audit trail and approval points for risky actions.
What does it cost to run?Tends to be lower and more stable.Tends to sit in between and is easier to estimate per item.Tends to vary more: loops, retries and tool calls differ for every task.
What usually goes wrong?Breaks silently when an input format changes, if nobody monitors failures.Plausible but wrong extraction accepted because validation was too loose.Takes an action nobody anticipated, or follows instructions hidden in the content it reads.
A sensible first useMoving data between systems on a schedule.Triage, extraction and drafting with a person approving the result.Research and preparation that ends in a recommendation, not an action.

This framework is our own proposed editorial analysis, informed by the workflow and agent distinction in the sources below. The cost and risk entries are relative tendencies, not pricing facts or guarantees. It is not an industry standard or a vendor classification.

An invoice-routing example

Notice what the design does not include. There is no agent deciding how to handle an invoice, because the path is known. The model does the one thing rules do badly, reading varied documents, and everything that moves money stays behind existing approvals. A request to change bank details is exactly the kind of instruction an attacker would hide in an email or attachment, which is why it is routed to a person rather than interpreted.

That last point is not hypothetical caution. The OWASP generative AI security project’s risk guidance treats prompt injection, where content a model reads contains instructions, and excessive agency, where a system has more permission than its task needs, as core risks to design against. The cheapest defence is not to give the system the permission in the first place.

Exception ownership and stop conditions

Automation projects rarely fail on the common case. They fail on the exceptions nobody owned. Before building anything, decide the following and write it down.

  • Who owns the exception queue. A named role, with cover for absence, and a response time the business agrees is acceptable. An exception queue without an owner simply becomes a new backlog.
  • What the person sees. The original input, what the system extracted or proposed, and why it stopped. Reviewers who see only a summary tend to approve it.
  • What stops execution automatically. Typical stop conditions are a validation failure, a required field that fails evidence checks (for example, it cannot be matched to the source document), an action outside the permitted list, an error from a connected system, or a sudden change in volume or exception rate.
  • Who can pause the whole thing. A simple switch that returns the process to manual handling, which the process owner can use without asking the delivery team.
  • What is logged. The audit events needed to investigate a complaint or an audit question: what was received, which checks ran, what action was taken and who approved it. Keep operational metadata separate from sensitive content, redact or minimise personal and confidential data, restrict who can read the logs, and agree a retention period and deletion process. Do not keep raw customer documents in logs by default.

One failure mode deserves particular attention: reviewers who stop reviewing. When a system is right most of the time, people approve its suggestions with less and less scrutiny. Sampling a share of approved items for a second check, and rotating who does it, keeps the human checkpoint meaningful.

When an agent is the wrong choice

Agents are genuinely useful for some work. They are the wrong starting point far more often than vendor demonstrations suggest. This guide recommends treating the following as warning signs.

  • The process is not stable. If two experienced staff would handle the same case differently, automating it captures the confusion rather than resolving it. Agree the process first.
  • Actions cannot be undone. Payments, deletions, contractual commitments and messages sent to customers need a person to approve them, which removes most of the reason for an agent.
  • You cannot describe what “correct” looks like. Without a set of real past cases and agreed answers, you have no way to tell whether the system works, only whether it looks convincing.
  • The volume is low. A task done a handful of times a week rarely justifies the build, monitoring and review effort. Better templates or a checklist may do more.
  • The decision materially affects people. Credit, hiring, eligibility and similar decisions bring legal and ethical obligations that this guide does not cover. They need specialist advice and a governance process before any automation.
  • Nobody will own it after launch. Model behaviour, connected systems and input formats all change. A system without an owner degrades quietly.

For broader governance, the NIST AI Risk Management Framework offers a voluntary structure for identifying and managing AI risks across an organisation. It is a useful checklist for thinking; using it is not a certification and does not by itself demonstrate legal compliance.

A reusable pilot brief

We suggest writing a one-page brief before any build starts. If a field cannot be filled in, the project is not ready. The headings below are ByteIQ’s proposed template.

  • Input. What arrives, from where, in what formats, and roughly how often.
  • Desired outcome. What should exist at the end that does not exist now, described in business terms.
  • Accountable owner. The person who answers for the process and its exceptions, not the person who builds it.
  • Access scope. The systems and actions the automation may use, and those it must never touch.
  • Representative evaluation set. Real past cases, including awkward ones, each with the answer a competent person would give.
  • Human checkpoint. Where a person reviews or approves, and what they see when they do.
  • Monitoring. What is tracked after launch: exception rate, reviewer overrides, errors from connected systems, cost per item.
  • Rollback. How the process returns to manual handling, and who can trigger it.
  • Success threshold. The level of accuracy, time saved or backlog reduced that would justify continuing, chosen by the business before the pilot starts, not after the results arrive.

Next steps

  1. List two or three candidate processes and choose the one with steady volume, a clear definition of done and a known exception path. Owner: operations lead
  2. Collect a representative evaluation set from real past cases, including the awkward ones, with agreed correct answers. Owner: process owner with a subject-matter reviewer
  3. Complete the pilot brief and agree the access scope and stop conditions before any build begins. Owner: process owner with IT and security
  4. Build the simplest version that could work, usually rules plus one AI step, and test it against the evaluation set. Owner: delivery team
  5. Run it in shadow mode, producing proposals that people compare with their own decisions, before it acts on anything. Owner: process owner
  6. Review results against the threshold you set in advance, then expand, adjust or stop. Owner: executive sponsor

If you would like a second opinion on which process to start with, our AI & Business Automation service covers pilot design and delivery, and Technology Consulting helps when the question is broader than one process. You can also tell us about the process you have in mind.

Sources and further reading

  1. Building effective agents, Anthropic engineering essay. First published December 2024. Source of the workflow and agent distinction and the advice to add complexity only when it improves results; its tool examples are dated.
  2. OWASP Top 10 for LLM and generative AI initiative. Community security guidance; used here only for the broad concepts of prompt injection and excessive agency, not for any ranking.
  3. NIST AI Risk Management Framework. Voluntary framework for managing AI risk; not a certification scheme.

About this guide: written by the ByteIQ Labs editorial team. The decision table, scenario and pilot brief are our own analysis and are labelled as such; facts attributed to other organisations link to their primary source. More practical guides are collected on our Insights page.