An AI agent receives a request to reschedule a customer’s appointment. It checks availability, cancels the original booking, creates a replacement, updates the customer record, and sends a confirmation. Then the final step fails, leaving the original booking gone, the replacement uncertain, and several systems disagreeing about what actually happened.
That failure is more than an AI quality problem. Once an agent can call APIs, modify databases, create tickets, issue refunds, deploy software, or change account state, each turn can produce real side effects. A useful way to reason about those turns is to treat them more like transactions: the agent should either complete the intended operation successfully or leave the system in a known state from which it can recover.
User Request
│
▼
Agent Turn
│
├── Read state
├── Choose action
├── Change state
├── Validate result
└── FinalizeThe goal is not to force every agent interaction into a database transaction. It is to bring transactional discipline to the parts of an agent workflow where partial completion can create inconsistent or harmful state.
The Problem Starts When the Agent Can Change the World
A read-only assistant is relatively easy to reason about. It may return a poor answer, but the underlying systems usually remain unchanged. An agent with tools is different because every action can create a new state that later actions depend on.
Consider a refund workflow:
1. Verify order
2. Issue refund
3. Mark order as refunded
4. Notify customerIf the refund succeeds but the order update fails, the payment system says one thing while the order database says another. The agent may still produce a perfectly fluent final response, but the real system is already inconsistent.
That is why an agent action should be treated as more than a tool call. The important question is not whether the model asked a tool to do something, but whether the intended state change actually occurred and whether the rest of the workflow still makes sense after it, especially once accountability in algorithmic systems becomes part of operations.
One Turn Can Hide Several Independent Failures
From the user’s perspective, “close this account” looks like one request. Internally, the agent may need to cancel subscriptions, revoke access, archive records, update billing, and send confirmation.
"Close this account"
│
▼
Cancel subscription
│
▼
Revoke access
│
▼
Archive records
│
▼
Update billing
│
▼
Send confirmationEach step can fail independently, and several of them may be irreversible or difficult to undo. A reliable workflow therefore needs to know which actions have completed, which remain pending, and which state is authoritative at every stage.
This is the core reason transactional thinking helps. Instead of viewing the turn as a single block of reasoning, you view it as a sequence of state transitions that must be validated and controlled.
Validate the Outcome, Not Just the Tool Response
Some failures are obvious. An API returns 500, a database rejects a write, or a network connection never opens. More dangerous failures are ambiguous because the agent cannot tell whether the action succeeded.
Suppose the agent sends a refund request and the connection times out before the payment service returns a response.
Agent
│
│ refund request
▼
Payment Service
│
├── Refund completed
│
X
response lostThe agent sees a timeout, but the payment provider may already have processed the refund. Retrying immediately could create a duplicate if the API treats the second call as a new operation.
The workflow should therefore validate the outcome before deciding what happens next. That might mean querying the payment by transaction ID, checking the new account state, looking for the created resource, or verifying that the expected database record exists, which fits the broader discipline in the NIST AI Risk Management Framework.
Perform Action
│
▼
Check Result
│
┌───┴─────────┐
▼ ▼
Confirmed Unknown / Invalid
│ │
▼ ▼
Continue RecoverA successful-looking response is useful evidence, but the resulting state is usually stronger evidence.
Commit Only When the Workflow Has Reached a Valid State
Traditional database transactions give us a clean model:
BEGIN
update account
insert payment
update balance
COMMITIf something fails before the transaction finishes, the database can roll back the changes. Agent workflows often span external services that cannot participate in one shared transaction, but the same idea still helps.
A workflow should have a clear point at which its result is considered final. Before that point, it is still in progress and may require recovery.
Started
│
▼
Actions in progress
│
▼
Validate final state
│
┌─┴──────────┐
▼ ▼
Valid Invalid
│ │
▼ ▼
Commit Roll backHere, commit does not necessarily mean issuing a SQL COMMIT. It means the workflow has reached a state that the system accepts as complete and consistent, even when the surrounding infrastructure is more complex than legacy modernization.
Rollback may also be conceptual. If the agent created a user account before a later step failed, rollback might mean deleting or disabling that account through another API call, rather than relying on a literal ROLLBACK TRANSACTION.
Atomic Execution Gets Hard Across Multiple Services
The ideal transactional property is atomicity: either everything happens or nothing happens. That is relatively straightforward inside one transactional database, but agent workflows usually cross boundaries.
Agent
│
├──► CRM
├──► Payment Provider
├──► Email Service
└──► Shipping APIThere is usually no global transaction manager capable of rolling all of those systems backward together. The payment service cannot undo its own operation simply because the CRM update failed, and an email cannot be unsent because the shipping API later returned an error.
This means atomic execution often has to be approximated rather than guaranteed. One practical strategy is to delay irreversible or externally visible actions until the workflow has passed earlier validation steps.
Validate request
│
▼
Reserve resource
│
▼
Process payment
│
▼
Confirm final state
│
▼
Send notificationThe workflow is not perfectly atomic, but it reduces the chance that a partially completed operation leaves the user with an impossible or misleading result.
Rollback Often Means Compensation
Some actions cannot literally be reversed. A bank transfer may require a separate reversal, an account creation may require deletion, and a shipment may need cancellation rather than an actual rewind of history.
This leads to compensating actions.
Original Action Compensation
Create booking → Cancel booking
Charge card → Refund charge
Reserve inventory → Release inventory
Enable account → Disable accountIf a later step fails, the workflow can execute compensation for the steps that already succeeded.
Step 1 success
Step 2 success
Step 3 fails
│
▼
Compensate Step 2
│
▼
Compensate Step 1This resembles the saga pattern used in distributed systems. It is not perfect rollback because compensation itself can fail, but it gives the workflow a defined recovery path instead of leaving operators to reconstruct the damage manually.
Retries Are Necessary, but They Need Guardrails
Distributed systems experience temporary failures constantly. Services throttle requests, databases become briefly unavailable, and networks drop responses. A reliable agent therefore needs retries.
Request
│
▼
Failed
│
▼
Retry
│
▼
SuccessThe danger is that the original operation may already have succeeded. If the agent retries a create or charge operation blindly, it can duplicate the side effect.
That is why retries and idempotency belong together.
Idempotency Makes Repeated Actions Safe
An idempotent operation can be repeated without creating additional unintended effects. Setting an account status to closed is naturally close to idempotent because repeating the same update still leaves the account closed.
Set account_status = "closed"
First call → closed
Second call → still closedBy contrast, adding $100 credit is not naturally idempotent. Running the same operation twice adds $200.
For side-effecting operations, systems often use an idempotency key:
POST /refund
idempotency_key = refund-order-48291If the same logical request is retried with the same key, the service can recognize that it has already processed the operation and return the existing result instead of creating another one.
First request
│
▼
Process refund
│
▼
Store key
Retry with same key
│
▼
Key already exists
│
▼
Return existing resultFor agent workflows, the important detail is that the key represents the logical action, not the individual HTTP request. If the agent generates a new key on every retry, the protection disappears.
Retry the Failed Step, Not the Entire Turn
Agents introduce another retry problem because one turn may contain several completed actions before the failure occurs.
Suppose the workflow reaches this point:
1. Create ticket ✓
2. Add customer note ✓
3. Send notification ✗Restarting the whole turn may create another ticket and duplicate the note. A more reliable system stores durable progress.
Workflow: incident-841
create_ticket completed
add_note completed
send_notification failedThe system can then resume from the failed step instead of replaying everything, which is one reason model version drift becomes more dangerous once partially completed work exists in the world.
This is an important architectural distinction. The agent’s reasoning can often be repeated cheaply, but side effects need durable checkpoints because they may not be safe to repeat.
Failure Recovery Needs More Than “Try Again”
Retries solve transient failures. They do not solve invalid input, missing permissions, violated business rules, or operations whose result is unknown.
A reliable workflow should distinguish between different kinds of failure.
Failure
│
├── Temporary
│ └── Retry
│
├── Permanent
│ └── Stop / compensate
│
└── Ambiguous
└── Verify stateA rate limit may justify waiting and retrying. A rejected payment may require stopping the workflow, while a timeout after a payment request may require checking whether the payment already exists before doing anything else.
This makes failure recovery a decision process rather than a loop that blindly repeats the same tool call.
error
│
▼
classify failure
│
├── retry safely
├── verify outcome
├── compensate
├── escalate
└── stopThe more consequential the action, the more important that distinction becomes.
The Workflow Needs an Audit Trail
When something goes wrong, operators need to reconstruct what happened. The final assistant response is not enough.
A useful audit trail records the sequence of actions and their outcomes, the same way structured logging and correlation and trace IDs make production events reconstructable.
Workflow ID: 92714
14:02:11 request received
14:02:12 account validated
14:02:14 subscription cancelled
14:02:15 refund requested
14:02:17 refund confirmed
14:02:18 customer record updated
14:02:20 confirmation sent
14:02:21 workflow committedIf recovery occurs, that should appear in the same history:
14:02:18 customer update failed
14:02:19 compensation started
14:02:21 refund reversal requested
14:02:24 reversal confirmed
14:02:25 workflow rolled backThe audit trail should make it possible to answer what the agent intended to do, which tools it called, which actions succeeded, which retries occurred, which state changes were verified, and whether compensation completed.
That record helps with debugging and incident response, but it also makes the workflow itself more reliable because the system has durable evidence of its own progress.
The Agent’s Memory Should Not Be the Source of Truth
An agent may remember that it already issued a refund. That memory is useful for reasoning, but it should not be treated as authoritative operational state.
Agent context:
"I already issued the refund."
Payment system:
refund_id = rf_7281
status = completedThe payment system is the stronger source of truth.
A reliable architecture separates three things:
Agent Context
│
└── helps reason
Workflow Store
│
└── records progress
External Systems
│
└── authoritative business stateThe model can decide what should happen next, while the workflow layer determines what has actually happened. This separation becomes especially important when the model is restarted, conversation context is truncated, or a workflow resumes hours later, and when traces need to line up with durable workflow state.
Consistent State Matters More Than Finishing Every Step
A workflow should not complete at any cost.
Suppose an agent provisions an employee across identity, payroll, email, and access-control systems. If three systems succeed and one fails, forcing the workflow forward may leave the organization with a partially provisioned account that nobody understands.
A better design defines which states are acceptable.
VALID
Provisioning complete
│
▼
All required systems active
VALID
Provisioning failed
│
▼
No active resources remain
INVALID
Partially provisioned
│
▼
Conflicting stateSometimes the safest recovery is to compensate and return to the previous state. In other cases, the workflow should stop and escalate to a human rather than attempting increasingly risky repairs.
The goal is a consistent state, not merely a completed sequence of tool calls.
Separate Agent Planning From Transactional Execution
Agents are useful because they can interpret ambiguous goals and choose actions dynamically. Those strengths do not mean the model itself needs to manage every retry, idempotency key, audit record, and rollback path.
A useful architecture separates planning from execution.
User Goal
│
▼
Agent Planner
│
▼
Proposed Actions
│
▼
Workflow Executor
│
├── Validate
├── Execute
├── Record
├── Verify
└── RecoverThe agent might decide that a customer account needs to be verified, cancelled, refunded, and notified. A deterministic workflow layer can then execute those actions while enforcing transaction IDs, permissions, checkpoints, retry policies, and recovery logic.
This division gives each part of the system the job it is better suited for. The model handles interpretation and planning, while the workflow infrastructure handles reliability.
Not Every Part of a Turn Needs a Transaction
Transactional controls should focus on consequential side effects.
Imagine an agent is asked to research several suppliers, compare them, and place an order with the best option. The research phase does not change external state, so forcing it into a transaction adds little value.
Research
│
▼
Comparison
│
▼
Decision
│
▼
──────── Transaction boundary ────────
│
▼
Create purchase
│
▼
Verify purchase
│
▼
CommitThe transaction boundary starts where state-changing behavior becomes important.
This keeps the system practical. Not every model response needs heavyweight recovery mechanics, but important side effects should have more protection than “the tool call looked successful.”
Reliable Agent Workflows Start Looking Like State Machines
Once you add validation, retries, rollback, and checkpoints, a robust agent workflow naturally starts to resemble a state machine.
┌─────────────┐
│ STARTED │
└──────┬──────┘
│
▼
┌─────────────┐
│ EXECUTING │
└──────┬──────┘
│
validate
│
┌───────┴────────┐
▼ ▼
┌─────────────┐ ┌─────────────┐
│ COMMITTED │ │ RECOVERY │
└─────────────┘ └──────┬──────┘
│
┌─────┴─────┐
▼ ▼
┌─────────┐ ┌──────────┐
│ROLLED │ │ESCALATED │
│BACK │ │ │
└─────────┘ └──────────┘The model may still choose tools and adapt to changing conditions, but the surrounding workflow has explicit states. That makes it possible to resume after crashes, detect incomplete work, prevent duplicate actions, and decide whether intervention is required.
Without that structure, a long-running agent becomes a loose sequence of tool calls whose reliability depends heavily on the model remembering everything correctly. That is too weak a foundation for important business operations.
A Transactional Agent Turn in Practice
Consider an agent that upgrades a customer from a Basic plan to a Pro plan and charges the price difference.
The workflow begins by creating a durable transaction record:
transaction_id = upgrade-58219
status = startedIt verifies the current account state and calculates the required payment. The payment request is then sent with an idempotency key tied to the workflow:
payment_key = upgrade-58219-paymentThe payment service times out.
Instead of sending another charge immediately, the workflow queries the payment provider using the same transaction identity. It discovers that the charge actually completed.
The agent then attempts to update the subscription, but that update fails. The system can safely retry the idempotent plan change or compensate by reversing the payment if the account cannot be upgraded.
Suppose the retry succeeds. The workflow validates both sides:
payment = completed
plan = ProOnly after those conditions are confirmed does it mark the workflow as committed and send the customer confirmation.
The important part is not that the agent chose the correct plan. It is that each action had a known state, retries were controlled, and the workflow had a defined recovery path if something went wrong.
Reliable Agent Workflows Depend on Controlled Side Effects
Agent systems become much more useful when they can take action, but they also inherit the same problems distributed systems have dealt with for decades. Networks fail, APIs time out, services disagree, requests are repeated, and partial operations leave behind uncertain state.
The AI component does not remove those problems. In some cases it makes them more important because the action sequence may be generated dynamically rather than encoded in a fixed workflow.
A strong pattern looks like this:
Agent chooses action
│
▼
Validate preconditions
│
▼
Execute safely
│
▼
Record result
│
▼
Validate outcome
│
┌───┴─────┐
▼ ▼
Success Failure
│ │
▼ ▼
Continue Retry / Compensate
│ │
└────┬────┘
▼
Consistent State
│
▼
CommitIdempotency makes retries safer. Durable workflow state prevents the agent from forgetting what has already happened, while audit trails make incidents reconstructable. Compensation provides a way back when real atomic rollback is impossible.
The result is not a perfectly transactional world. External systems will still fail, some actions cannot be reversed, and there will always be cases that require escalation.
The goal is more practical: every consequential agent action should leave enough structure behind that the system can determine what happened, recover safely when necessary, and end in a state the rest of the application can trust.
