Move slow work out of Flask requests with Encore, RQ or Celery: a queue, a worker, retries and scheduled jobs, with working code for each and what it takes to run them in production.

With sync workers, Flask handles one request at a time per worker, so sending an email or rendering a PDF inside a view holds that worker until the work finishes. Background jobs move the slow part to a queue, and a separate worker picks it up.
This guide covers three ways to do that: Encore, which works alongside Flask, RQ and Celery. Each version sends an order receipt from a job and retries when the email provider fails.
A thread or concurrent.futures executor inside the view looks like a job queue until the first deploy. The process restarts and the work in flight disappears, a failure has no retry, and with four Gunicorn workers each one runs its own executor with no shared limit. For anything that must happen, the job needs to be written to a broker before the view returns, then handled by a process that can fail and try again.
Encore works alongside Flask as an SDK that declares infrastructure in your Python code. Encore for Python is in private beta, and you can request an invite. The queue is a Pub/Sub topic with a subscription as the worker, scheduled work is a cron job, and Flask keeps its routes and blueprints.
import os
import encore
import uvicorn
from asgiref.wsgi import WsgiToAsgi
from encore import cron, pubsub
from flask import Flask
from pydantic import BaseModel
class ReceiptRequested(BaseModel):
order_id: str
# The queue: SNS and SQS on AWS, Pub/Sub on GCP, a local broker under `encore run`.
receipts = pubsub.Topic[ReceiptRequested]("receipt-requested")
sent: set[str] = set() # stand-in for a table with a unique key
async def send_receipt(msg: pubsub.Message[ReceiptRequested]) -> None:
# Delivery is at least once, so skip orders that were already handled.
if msg.data.order_id in sent:
return
sent.add(msg.data.order_id)
print(f"receipt sent for {msg.data.order_id} (attempt {msg.delivery_attempt})")
# The worker: raising retries the message with backoff, up to max_retries.
receipt_worker = pubsub.Subscription[ReceiptRequested](
receipts, "send-receipt", handler=send_receipt, max_retries=5, ack_deadline="60s"
)
# A scheduled job: Encore sends POST /internal/cleanup every day at 03:00 UTC.
cleanup = cron.Job("cleanup", path="/internal/cleanup", schedule="0 3 * * *")
flask_app = Flask(__name__)
@flask_app.post("/orders/<order_id>/receipt")
async def queue_receipt(order_id: str):
msg_id = await receipts.publish(ReceiptRequested(order_id=order_id))
return {"queued": msg_id}, 202
@flask_app.post(cleanup.path)
def run_cleanup():
# Delete expired carts, stale sessions and so on.
return {"ok": True}
Publishing is async, so the view uses Flask's async support (pip install "flask[async]"). Subscriptions need a running event loop, so Flask is served under Uvicorn through asgiref's WsgiToAsgi, and a lifespan hook calls encore.start():
wsgi = WsgiToAsgi(flask_app)
async def app(scope, receive, send):
# The lifespan hook starts Encore's subscription handlers on Uvicorn's loop.
if scope["type"] == "lifespan":
while True:
message = await receive()
if message["type"] == "lifespan.startup":
await encore.start()
await send({"type": "lifespan.startup.complete"})
elif message["type"] == "lifespan.shutdown":
await send({"type": "lifespan.shutdown.complete"})
return
else:
await wsgi(scope, receive, send)
if __name__ == "__main__":
host, _, port = os.environ.get("ENCORE_LISTEN_ADDR", "0.0.0.0:8000").rpartition(":")
uvicorn.run(app, host=host or "0.0.0.0", port=int(port))
encore run delivers published messages to the handler in the same process, with no Redis to start. The cron job is Encore platform Managed: it does not run under encore run or in preview environments, so locally you test it by sending the POST to /internal/cleanup yourself. The service is registered with an encore.service.json file, and the SDK needs Python 3.9 or later.
On deploy, Encore creates the topic and subscription in your own account, as SNS and SQS on AWS or Pub/Sub on GCP, and provisions an IAM role for the service, or a service account on GCP. A coding agent adding an invoice PDF job writes a second topic and subscription in the same change and checks the delivery in the local trace before anything is deployed.
Encore for Python is in private beta.
RQ (Redis Queue) is the smallest job queue for Flask. Jobs are plain Python functions, imported by both the app and the worker:
# tasks.py
def send_receipt(order_id: str) -> None:
# Render the receipt and call the email provider here.
print(f"receipt sent for {order_id}")
The view enqueues the function with its arguments and returns 202:
# app.py
import os
from flask import Flask
from redis import Redis
from rq import Queue, Retry
from tasks import send_receipt
app = Flask(__name__)
redis = Redis.from_url(os.environ.get("REDIS_URL", "redis://localhost:6379/0"))
emails = Queue("emails", connection=redis)
@app.post("/orders/<order_id>/receipt")
def queue_receipt(order_id: str):
job = emails.enqueue(
send_receipt,
order_id,
retry=Retry(max=3, interval=[10, 30, 60]), # seconds between attempts
job_timeout=60,
)
return {"job_id": job.id}, 202
Start a worker with rq worker emails --url "$REDIS_URL" in deployment, or redis://localhost:6379/0 locally. Treat Redis as a trusted boundary: use an authenticated TLS connection on a private network, and do not let untrusted producers write to this queue, since RQ pickles job arguments and any queue writer can run code in the worker. Pass IDs rather than ORM objects, and a job that needs Flask extensions has to create the app and push an app context itself. Failed jobs land in the failed job registry after the retries run out, and scheduling needs the separate rq-scheduler package or an external cron.
Celery adds routing, rate limits, chords and a scheduler, and it is the default choice for larger Flask apps. The Flask docs' Celery pattern wraps each task in the app context:
from celery import Celery, Task, shared_task
from flask import Flask
def celery_init_app(app: Flask) -> Celery:
class FlaskTask(Task):
def __call__(self, *args, **kwargs):
# Tasks run inside an app context, so extensions such as
# Flask-SQLAlchemy work the same as in a view.
with app.app_context():
return self.run(*args, **kwargs)
celery_app = Celery(app.name, task_cls=FlaskTask)
celery_app.config_from_object(app.config["CELERY"])
celery_app.set_default()
app.extensions["celery"] = celery_app
return celery_app
@shared_task(autoretry_for=(ConnectionError,), retry_backoff=True, max_retries=5)
def send_receipt(order_id: str) -> None:
print(f"receipt sent for {order_id}")
def create_app() -> Flask:
app = Flask(__name__)
app.config.from_mapping(
CELERY=dict(
broker_url="redis://localhost:6379/0",
task_acks_late=True, # ack after the task finishes, not before
task_ignore_result=True,
),
)
celery_init_app(app)
@app.post("/orders/<order_id>/receipt")
def queue_receipt(order_id: str):
result = send_receipt.delay(order_id)
return {"task_id": result.id}, 202
return app
A make_celery.py module creates the app and exposes celery_app = flask_app.extensions["celery"], and the worker runs with celery -A make_celery worker --loglevel INFO. Task options such as autoretry_for and retry_backoff handle transient failures, and task_acks_late keeps a task in the queue until it finishes, at the cost of possible duplicates after a crash. Scheduled jobs need celery beat as exactly one extra process.
Choose Encore if you want the queue, the workers and the scheduled jobs declared in the Flask app and provisioned in your own AWS or GCP account, with local runs that need no Redis. Choose RQ for a small app that already has Redis and a few job types. Choose Celery when you need routing, rate limits, workflows or a mature scheduler and are ready to run the broker, workers and beat yourself.
RQ is the smallest setup: a Redis server, a Queue in the app and an rq worker process. Celery adds scheduling, routing and more retry options at the cost of more configuration. Encore for Python, currently in private beta, declares the queue and the scheduled jobs in your Flask code. The queue runs locally under encore run and in your own cloud account. The cron job is Encore platform Managed and does not run locally or in preview environments.
Yes, but only for work you can afford to lose. A thread dies with the process on a deploy or crash, has no retries, and each Gunicorn worker runs its own threads, so anything that must happen, such as sending a receipt or charging a card, belongs in a queue with a separate worker.
Subclass celery.Task so __call__ wraps self.run in app.app_context(), and pass it as task_cls when creating the Celery app. The Flask documentation's celery_init_app pattern does exactly this, so extensions such as Flask-SQLAlchemy work inside tasks.
Celery uses celery beat with a beat_schedule, RQ has the rq-scheduler add-on or cron calling a script, and some setups use a host crontab hitting an endpoint. Scheduled jobs need exactly one scheduler process, so running beat inside every web worker sends each job several times.
Deploy to your cloud on AWS and Google Cloud
without managing infrastructure
Encore lets developers and agents define application infrastructure directly in code, then automatically provisions it from local development to production in your AWS or GCP account.