Managed AI in practice / FIELD NOTES
Workflow Automation Acceptance Tests: Success, Duplicates and Recovery
Write executable acceptance conditions for missing data, duplicate events, permission failures and partial recovery, using a registration workflow example.
Does one successful run mean an automated workflow is ready for everyday use?
One successful run proves one input worked. Acceptance testing must also cover actions that should not happen and who handles failure. Microsoft's troubleshooting guidance separates trigger, action and logic issues; preserve evidence at each relevant layer.
Express rules as inputs and outcomes
Give each test an ID, prerequisites, input, expected record changes, notifications and prohibited actions. Replace “works correctly” with “a missing required email creates no approved record and adds a reason to the correction queue.”
Use isolated test records and controlled notification destinations. Specify time zones and formats. Where silence is expected, verify that no notification was sent.
Include duplicates, boundaries and failures
Exercise empty fields, invalid formats, repeated events, repeated clicks, changed event order and unusual characters. Test relevant permission failures, unavailable destinations and interruptions midway through processing.
For each failure, define an observable status, owner and recovery entry point. Before retrying, identify completed steps so partial success does not become duplicated work.
Worked example: registration review
Suppose a submission creates a review record and alerts an internal reviewer. Receiving the same submission ID twice should produce one record and one alert. Missing required data should enter a correction queue without the ordinary alert.
If record creation succeeds but the alert fails, preserve the record ID and failed step. Recovery should send the outstanding alert. Count actual records and messages rather than trusting a success badge.
Acceptance and handoff checklist
Keep inputs, expected outcomes, observed outcomes and run evidence together. Retest fixes and document unresolved limits.
- Correct normal output.
- No duplicate records or messages.
- Invalid input has an owner.
- Completed steps remain identifiable.
- Recovery and pause procedures exercised.
Before you hand over the work
- Test expected actions and forbidden side effects.
- Exercise recovery before relying on it.
Demonstration projects and examples illustrate working methods. Follow the receiving party’s specifications and the scope agreed for your project.