One client's revoked connection should not stop every other client's calls or exhaust shared workers with retries. Failure handling needs the same explicit client boundary as data access.
Identify affected client, dependency, operation, and outcome category. Separate an explicit authorization rejection from a temporary error or an uncertain write. Each needs a different recovery path and caller-facing status.
Bound retries and shared resource use
Use client-scoped queues, limits, and failure state where appropriate to the architecture. A failing integration should not consume unbounded worker concurrency or block healthy clients behind its backlog.
OWASP's multi-tenant security guidance discusses tenant-aware resource controls. Adapt that principle to calls, background actions, and reconciliation rather than assuming a shared queue provides isolation by itself.
Work through a fictional access failure
Client A revokes calendar access while Client B remains healthy. The system suspends A's remote writes and captures approved pending requests with honest language. B continues through its normal verified path.
A's retries are bounded and visible. The system never switches to B's token or destination to make A's calls appear successful. After A reconnects, uncertain actions are reconciled before pending work resumes.
| Containment case | Required behavior |
|---|---|
| One client loses access | Affected writes suspended visibly |
| Repeated failures | Bounded retries and owned queue |
| Shared workers busy | Healthy-client capacity protected |
| Unknown prior write | Reconciliation before replay |
| Client reconnects | Identity and scope reverified |
Keep customer work distinct from service health
A recovered integration does not mean affected customers have been contacted. Track pending request age and ownership through reconciliation. Expired preferences or already-resolved requests may need staff review rather than automatic replay.
More isolation adds scheduling and monitoring complexity. A single global failure switch is simpler but can unnecessarily interrupt healthy clients. Choose controls that contain the real dependency boundary while preserving a safe global stop for wider incidents.
Test one failing client under controlled load and inspect others' outcomes. Review revoked-access recovery for the client-specific action ledger.
Questions about failure containment
Should all calls stop when one API fails?
Only if the shared dependency or risk genuinely affects them. Preserve approved intake and isolate the affected action scope where possible.
Can retries continue indefinitely?
Use bounded attempts, visible terminal or review states, and clear ownership.
What proves recovery?
Correct client access, verified controlled operations, reconciled uncertain work, and owners for remaining customer requests.