Fault scenarios let AgentIdem test how an agent behaves when execution does not follow the normal happy path.
The AgentIdem Nebutex SDK currently defines three core fault scenarios:
lost_acknowledgement
before_operation_failure
duplicate_delivery
These scenario names are canonical identifiers and should be treated as stable names in documentation and reporting.
Why fault scenarios matter
State-changing agents often interact with systems that are not naturally transactional.
A write can succeed in the external system while the agent fails to observe that success.
An invocation can also be delivered more than once.
A failure can happen before a write executes or after the external state has already changed.
These cases can look similar from the agent's point of view, but they have very different side effect consequences.
AgentIdem models them separately.
Lost acknowledgement
The lost_acknowledgement scenario models this sequence:
1. the write executes
2. the side effect succeeds
3. AgentIdem records the write as successful
4. the acknowledgement is treated as lost
5. the agent may observe an injected failure
6. retry logic may execute the same logical write again
The important detail is that the side effect already happened.
Conceptually:
WRITE refund
SUCCESS
acknowledgement
LOST
agent observes failure
retry
WRITE refund
SUCCESS
This is different from throwing an exception before the write runs.
A lost acknowledgement specifically models a successful side effect whose success was not observed by the caller.
Why lost acknowledgements are dangerous
Consider a refund operation:
from agentidem import write
@write(identity=lambda payment_id: payment_id)
def refund(payment_id: str):
return payments.refund(payment_id)
The first execution succeeds:
refund("payment-123")
SUCCESS
but the acknowledgement is lost:
observation: LOST
If the agent retries:
refund("payment-123")
SUCCESS
the same logical refund may have happened twice.
AgentIdem can detect the repeated successful write using the configured logical identity.
Before-operation failure
The before_operation_failure scenario models a failure that occurs before the selected operation executes.
Conceptually:
failure injected
↓
WRITE refund
never executed
The side effect does not happen.
AgentIdem records the selected operation as failed.
This lets AgentIdem distinguish between:
operation never happened
and:
operation succeeded but acknowledgement was lost
Those two situations should not be treated as equivalent.
Lost acknowledgement vs before-operation failure
Consider the same refund operation.
Lost acknowledgement
WRITE refund
status: SUCCESS
observation: LOST
The refund happened.
The agent may still think it failed.
Before-operation failure
WRITE refund
status: FAILED
observation: FAILED
The refund did not happen.
Retrying after this failure is fundamentally different from retrying after a lost acknowledgement.
Duplicate delivery
The duplicate_delivery scenario means the entire agent invocation runs more than once.
Conceptually:
invocation 1
↓
READ get_payment
↓
WRITE refund
SUCCESS
invocation 2
↓
READ get_payment
↓
WRITE refund
SUCCESS
This models systems where execution may happen more than once even if the agent did not explicitly retry the write itself.
Examples include:
- a queue redelivering a message
- a webhook being delivered again
- an orchestrator restarting an invocation
- a background job running twice
- an at-least-once delivery system
Duplicate delivery and side effects
Suppose an agent looks like this:
from agentidem import read, write
@read
def get_payment(payment_id: str):
return payments.get(payment_id)
@write(identity=lambda payment_id: payment_id)
def refund(payment_id: str):
return payments.refund(payment_id)
def refund_agent():
payment = get_payment("payment-123")
if payment["status"] == "paid":
return refund("payment-123")
return "no_refund"
If the entire invocation is delivered twice, AgentIdem can compare or combine the resulting execution behavior.
If both invocations produce:
WRITE refund
identity: payment-123
status: SUCCESS
AgentIdem can detect the repeated successful write.
FaultCase
Fault scenarios can be represented with a generic FaultCase abstraction.
A fault case may include information such as:
scenario
operation_index
operation_name
phase
message
This allows AgentIdem to represent different fault scenarios without hardcoding every scenario into unrelated parts of the system.
Higher-level reporting should generally use the fault case and scenario model.
FaultPlan
AgentIdem may also use a lower-level FaultPlan abstraction internally.
A fault plan is useful for operation-level execution planning and fault injection.
It is an internal execution concept.
Higher-level reporting should not assume every scenario must map directly to a low-level operation plan.
Execution outcome vs safety outcome
A fault scenario can cause execution to fail without causing an unsafe side effect.
For example:
execution: failed
safety: safe
A lost acknowledgement may surface an exception while still preserving side effect safety if the retry does not repeat the successful write.
The opposite is also possible:
execution: succeeded
safety: unsafe
Duplicate delivery may complete successfully while repeating the same logical side effect.
AgentIdem therefore keeps execution outcome and safety outcome separate.
Fault results
A fault result can include information about:
- the scenario that ran
- whether execution succeeded or failed
- whether the result was safe or unsafe
- findings
- invariant results
- traced operations
A result should not automatically be considered unsafe just because an exception occurred.
Findings
AgentIdem uses structured findings to explain safety problems discovered during a fault scenario.
A finding can include:
- severity
- category or type
- message
- supporting operation information
A duplicate successful write is an ERROR-level safety finding.
Invariants
Fault scenarios can also be evaluated against user-defined invariants.
Examples include:
- at most one refund per payment
- balance never becomes negative
- exactly one order exists
- a write must not occur after cancellation
- resource state remains valid
Built-in duplicate detection and user-defined invariants remain separate checks.
Scenario summary
| Scenario | What it models | Did the selected side effect happen? |
|---|---|---|
lost_acknowledgement | A write succeeds but its acknowledgement is lost | Yes |
before_operation_failure | A failure happens before the operation executes | No |
duplicate_delivery | The complete agent invocation runs more than once | Depends on each invocation |