Skip to content

Using a queue for slow exports and notifications

development, performance

A queue is useful when work should continue independently of a web request and can be tracked to completion. It also introduces storage, worker and retry responsibilities.

What you need before you begin

Confirm the hosting plan supports the chosen queue service and supervised workers. Define what the user sees while the requested work is pending.

01 Use a queue for slow exports and notifications

  1. Create a durable job record with a stable identifier and the minimum required inputs. Reference authorised data rather than storing unnecessary sensitive copies in the queue.

  2. Return a clear pending status after the job is accepted and provide a status or download path when the work completes.

  3. Set worker concurrency, timeout and retry limits according to resource use. Separate heavy exports from short notifications when they would otherwise block each other.

  4. Protect against duplicate processing and preserve the outcome of external actions. Use a failure state that staff can inspect and recover deliberately.

  5. Monitor queue age, completion time, failure rate and storage use. Include worker restart and code compatibility in the deployment process.

02 Example: let a customer request an export and return later

For a report export, the web request can create job export-test-42 and show Preparing your report after durable acceptance. The worker later stores the generated file and records completion.

  1. Use a test account to request a small report, then close the browser.
  2. Reopen the account's report page and confirm the same job shows its current state, rather than creating another export on every refresh.
  3. When complete, verify that the download belongs to this account and requires the intended authorization.
  4. Cause a controlled worker failure in staging and check that the status becomes failed or eligible for a bounded retry, with a useful support reference.

A queue that accepts jobs without a running consumer cannot produce the file. Monitor the age of pending work as well as the number of jobs submitted.

Confirm the expected result

Submit a harmless job, disconnect the browser and confirm it completes. Test a controlled failure and verify the user sees a meaningful status rather than an endless spinner.

Troubleshoot unexpected results

If the queue accepts work but has no consumer, jobs will remain pending. Diagnose the worker and connection settings before adding more retries or resubmitting every request.

A common question

Does adding a queue automatically make work faster?

It changes when and where work runs. Appropriate workers and resource capacity still determine completion time, and the application must make that state visible.

Sources and further reading

Was this answer helpful?

Need a little more help?

Our support team can help you find the next step.

Contact support