During deployment, old and new workers can overlap or a running job can be interrupted. The system should remain correct even if the same logical work is attempted again.
Do not rely on a belief that a job runs exactly once. Define the business effect that must not duplicate and make recovery safe around the actual persistence and external-action boundaries.
Separate job delivery from business completion
A queue message identifies work to attempt. An operation identity ties attempts to one intended result. A completion record explains what has actually happened. Keep these distinctions clear enough that support can decide whether another attempt is safe.
Worker coordination can reduce duplicate execution, but leases, crashes, and timeouts still require a recovery policy. Use mechanisms supported by the actual queue and database, then verify their failure semantics rather than assuming a lock solves every side effect.
Rehearse a fictional report-notification deployment
Suppose a worker builds a report and emails the owner a download link. Deploy a new worker version while a controlled job is running. Test interruption before file creation, after storage, after completion recording, and after the email provider accepts the message.
Start two workers against the same test operation where the architecture permits the exercise. Verify the intended report artefact and the agreed notification behaviour. A harmless duplicate computation may be acceptable; duplicate customer charges or repeated external writes usually require a stricter boundary.
Test an old worker processing data written by the new version. Deployment compatibility matters as well as duplication. A job payload or schema change can make overlapping versions interpret the same work differently.
Make retries and manual actions traceable
Record attempts, worker version, relevant state transitions, and external references. A support retry should identify the same logical operation unless the user is deliberately requesting new work.
Define what happens to a job marked running when its worker disappears. A timeout may make it eligible for another attempt, but the system still needs to determine whether an earlier external effect completed. Use reconciliation where direct confirmation was lost.
The tradeoff is coordination complexity versus consequence. A low-cost deterministic report can tolerate recomputation more easily than an irreversible action. Match the controls to the effect instead of imposing one generic job policy on every task.
The StackFlight product account provides related report-and-export context. This deployment exercise is fictional and includes no claim about its queue infrastructure or measured reliability.
Questions about duplicate jobs after deploys
Can a worker lock guarantee no duplicate effects?
Not by itself. Crashes and uncertain external completion can still create ambiguity. Test the effect boundary and recovery rule.
Should deployment wait for all jobs to finish?
That may help some systems, but it is not a substitute for safe retries and version compatibility. Choose the actual shutdown and rollout policy deliberately.
What should be tested first?
Interrupt execution immediately after a consequential side effect, then retry. Bring the stored state and external outcome to a job-reliability review.