Database maintenance can expose webhook loss windows that ordinary tests never reach. Test delivery acknowledgement, receipt persistence, worker execution, and recovery while the database is unavailable, read-only, or reconnecting.
Maintenance affects several database roles
A database may store receipts, queue state, customer records, and operation results. Those roles can fail independently. A receiver that cannot persist a receipt must not pretend that its normal durable-acceptance contract was satisfied.
For a proposed fixture plan, identify the acceptance boundary and the provider’s supported retry behaviour. Inject connection refusal, transaction interruption, lock contention, read-only mode, and a lost commit response. Keep the fake provider’s view alongside local state.
Do not assume every failed transaction definitely rolled back from the caller’s perspective. If the connection drops at commit, the application may need reconciliation before repeating a consequential write.
Rehearse a fictional maintenance transition
The database becomes read-only during a controlled change. The receiver verifies a customer-update event but cannot save its durable receipt. Under the defined contract, it should return the appropriate failure rather than acknowledge work that exists only in memory.
After recovery, the provider redelivers. The receiver accepts the event once and the worker updates the intended customer. A separate fixture crashes after the customer write commits but before local completion is recorded, testing duplicate-safe recovery.
| Maintenance condition | Required invariant |
|---|---|
| Receipt store unavailable | No false durable acceptance |
| Worker database unavailable | Accepted work remains discoverable |
| Transaction interrupted | Known or explicitly uncertain outcome |
| Commit response lost | Reconcile before replaying effects |
| Database restored | Backlog drains under capacity controls |
Inspect final rows, operation history, and acknowledgements. A test that checks only whether the application reconnects misses whether any accepted work disappeared.
Plan backlog behaviour before the window
Maintenance can cause a replay burst afterward. Control worker concurrency and preserve per-resource ordering or version rules. Define who reviews unresolved operations and how client-facing status is communicated.
More elaborate failure injection adds setup cost, but these tests target material loss and duplication risks. Use isolated fixtures and documented maintenance procedures rather than experimenting on live customer records. Re-run relevant cases after changes to persistence, queues, or database drivers.
This maintenance plan is proposed. The voice integration case study describes relevant backend and infrastructure work. Pair it with webhook outage recovery.
Maintenance webhook FAQ
Should the endpoint acknowledge before saving anything durable?
Only if its explicitly designed contract safely supports that choice; do not imply durable acceptance when none occurred.
Does reconnecting prove recovery?
No. Verify accepted work, final business state, and duplicate-effect counts.
What should happen to uncertain commits?
Reconcile through authoritative state and the operation identity before repeating the write.