Skip to main content

ERPNext how-to & fixes

ERPNext v16 Background Jobs Not Running: A Diagnostic Ladder

Emails not sending, scheduled jobs silent? The five reasons ERPNext v16 background jobs stop, and the bench and redis-cli commands to fix each.

Updated 12 Oct 202612 min readBy ManojFirst published 9 Aug 2026

When ERPNext background jobs stop in v16, the symptoms are usually the same: emails queued but never sent, scheduled reports stuck at yesterday's date, assignment notifications silent, imports and long exports that never finish. The cause is one of five things: the scheduler is disabled for the site, the RQ worker is dead, the Redis queue is unreachable, a poisoned job is blocking the long queue, or pause_scheduler was switched on and forgotten. This post walks them in that order, with the bench and redis-cli commands that prove each one.

A silent scheduler is worse than a loud one. Emails don't bounce, users don't complain until the next collections call, and by then a week of automated flows has quietly not happened. I lead ERPNext and Frappe implementations at MithTech, a team of Frappe specialists in Bengaluru. This is the runbook we hand self-hosted clients after their v15 to v16 upgrade.

Is the scheduler even running?

Answer

Run bench doctor. It prints the scheduler status of every site on the bench, the number of RQ workers attached to Redis and the pending jobs per queue. Obvious failures show here, before you start opening containers.

Run it from the bench root: inside the container on Docker, on the host on bare-metal:

For each site you want: the scheduler enabled, a non-zero worker count for the default, short and long queues, and a low or zero pending count. If any of the three is off, you have your suspect.

Two useful companions:

What's actually queued right now

Enable a per-site scheduler that got flipped off during upgrade

show-pending-jobs prints each queue and the jobs waiting in it. A long queue with dozens of jobs and an empty default queue means the workers on long are the ones to look at.

How do you tell a stuck worker from an empty queue?

The most common misdiagnosis is confusing "no jobs running" with "no jobs queued". They look the same from the desk: frappe.enqueue(...) returns a job id, the desk shows "queued", and nothing happens.

Redis has the real state. RQ (Redis Queue) keeps it there and bench wraps it. Frappe names each queue <bench_id>:<queue> (the bench ID is usually the bench folder name, frappe-bench in Docker), so list the keys first, then read their lengths. From the bench container:

List the queues Frappe created on this Redis

Show the raw queue lengths (use the names the scan printed)

Show workers currently registered

If llen is positive but rq:workers is empty, jobs are waiting and no worker is picking them up: a dead worker container, or on bare-metal a Supervisor group that failed to restart. If rq:workers has entries but llen keeps growing, workers are alive but slow, usually the long queue stuck on one expensive job that blocks everything behind it.

Why do ERPNext v16 background jobs actually stop?

SignatureFirst check
1Scheduler disabled per sitebench doctor shows the scheduler disabled, or scheduler_enabled is off in sites/SITE/site_config.jsonbench --site SITE scheduler status
2RQ worker container deadQueues have length, smembers rq:workers is emptydocker compose ps queue-long queue-short (Docker) or supervisorctl status (bare-metal)
3Redis queue unreachableWorkers restart-loop, logs show ConnectionError to redis-queue:6379redis-cli -h redis-queue ping from a worker
4long queue backlog on one poisoned joblong queue length only climbs; one job stays "started" for hoursredis-cli hgetall rq:job:JOB_ID
5pause_scheduler or maintenance modeEverything looks healthy but no ticks firegrep pause_scheduler sites/SITE/site_config.json

Walk them in order: the check at the top is cheaper than the one at the bottom.

What is the fix order on frappe_docker?

0 of 5 steps done

Tick a step’s number when it is done; the ticks stay on this device.

Confirm the scheduler service is actually up

docker compose ps scheduler should show running with a recent uptime. If it is restarting, docker compose logs --tail=200 scheduler shows why. The two common causes are MariaDB not ready when the scheduler connects (fix: a depends_on health condition) and a stale common_site_config.json from a broken earlier install (fix: re-run the configurator service; docker compose down -v only on staging, because it deletes volumes).

Confirm both worker containers are up

The standard compose.yaml runs two: queue-short (bench worker --queue short,default) and queue-long (bench worker --queue long,default,short). Both need to be running. If only one is up, jobs on the missing queue never move. Restart with docker compose up -d queue-short queue-long, then re-check with docker compose logs --tail=100 queue-long for a ConnectionError.

Check the shared Redis queue

From inside docker compose exec backend bash, run redis-cli -h redis-queue ping. Expect PONG. Anything else (a timeout, Connection refused, NOAUTH) is a wrong REDIS_QUEUE setting or a Redis container that failed its own health check. Look at docker compose logs redis-queue.

Re-enable the scheduler if it was flipped off

The upgrade to v16 does not automatically re-enable a scheduler that was paused mid-upgrade. From backend: bench --site your-site.local enable-scheduler. Confirm with bench --site your-site.local scheduler status.

Deal with a stuck `long` queue

If steps 1 to 4 pass and long is still growing, one job is holding the queue. List the jobs with redis-cli -h redis-queue --scan --pattern "rq:job:*" and inspect the started ones with redis-cli -h redis-queue hgetall rq:job:<id>. A stuck import or a scheduled method that loops is the usual culprit. bench --site <site> purge-jobs --queue long clears it, but only after you have noted the job's function so you can fix the caller.

What is the fix order on bare-metal bench?

On a Supervisor-managed bare-metal install, the parts are the same but the levers differ.

0 of 4 steps done

Tick a step’s number when it is done; the ticks stay on this device.

Check the Supervisor group

sudo supervisorctl status | grep -E "(schedule|worker)" should show the scheduler and the default, short and long workers RUNNING (bench generates them into /etc/supervisor/conf.d/<bench>.conf). Any FATAL or EXITED is your cause. Restart a group with sudo supervisorctl restart <bench-name>-workers: (a trailing colon means the whole group).

Read the tails

Workers write to <bench>/logs/worker.log (rotated). Run tail -F logs/worker.log logs/scheduler.log while you enqueue a small job (bench --site <site> execute frappe.utils.background_jobs.enqueue --kwargs "{'method': 'frappe.ping'}"): it shows whether jobs are accepted and whether they are executed.

Reconcile Redis

ps aux | grep redis-server should show two Redis processes on a standard bench: one on the cache port and one on the queue port (configured in <bench>/config/redis_cache.conf and config/redis_queue.conf). If only one is running, the queue Redis died and the scheduler and workers fail quietly. Run bench setup redis, then restart the Supervisor group.

Rule out the config-only flags

Open sites/<site>/site_config.json and check three keys: scheduler_enabled (1 or absent), pause_scheduler (0 or absent) and maintenance_mode (0). Any of these flipped stops the scheduler with no error in the logs.

When should you run bench purge-jobs — and when should you not?

bench purge-jobs clears all pending jobs across all queues for the site. It is a valid step when a queue is provably stuck and you have found the offending job, after the fix. It is a bad first step, because you lose the evidence of why the queue jammed.

The safe order: show-pending-jobs → note the function names → redis-cli hgetall rq:job:<id> on any long-started job → fix the caller → then purge-jobs.

bench purge-jobs --queue long limits the clear to one queue, which is almost always what you want. Losing scheduled emails from default because a data import jammed long is a common self-inflicted wound.

How do you monitor so this stops surprising you?

Wire these two signals into whatever monitor you already run:

  1. Scheduled Job Log freshness. Query tabScheduled Job Log for the newest modified timestamp. Frappe v16's default scheduler tick is 240 seconds (DEFAULT_SCHEDULER_TICK in frappe/utils/scheduler.py); anything older than about three ticks (around 12 minutes) means the scheduler has stopped enqueueing. It is a database signal, so it works across split containers where a process check cannot.
  2. Queue length trend. Every 60 seconds, record llen of your long queue (rq:queue:<bench_id>:long). A steady climb over an hour is the failure signature; healthy queues drain between spikes.

Both are two-line shell scripts feeding your existing metrics. Do not use a Frappe Notification to alert you that the scheduler is down: an alert sent by the scheduler cannot tell you the scheduler stopped.

What else looks like "jobs not running"?

Three nearby problems that look like stuck jobs but are not:

  • A slow database making jobs look stuck. A long SQL join inside a scheduled method pins one worker for minutes and starves everything behind it. If jobs run but very slowly, start with MariaDB tuning for ERPNext; a right-sized InnoDB buffer pool often fixes it without touching the queue.
  • Asset 404s after bench build are unrelated to jobs but often blamed on them: a desk that cannot load JavaScript cannot trigger new jobs. If workers are healthy but nothing gets enqueued from the desk, see the frappe_docker asset breakage guide.
  • The System Health Report shows red but jobs are fine. That is the file-lock false alarm linked at the top. Do not chase infrastructure for a diagnostic bug.

FAQ

Why are ERPNext background jobs not running right after my v15 to v16 upgrade?

Most often the scheduler was paused for the upgrade and bench migrate did not re-enable it. Run bench --site <site> scheduler status; if it is disabled, bench --site <site> enable-scheduler fixes it (or scheduler resume if it is paused). Next most common is a stale common_site_config.json where redis_queue still points at the old bench's Redis; restart Supervisor or the queue containers after any config edit.

How do I know if the RQ worker is running or just registered?

redis-cli -h redis-queue smembers rq:workers lists the worker IDs Redis knows about. Then redis-cli -h redis-queue hgetall rq:worker:<worker-id>: a healthy worker keeps its last_heartbeat fresh. A worker with a stale heartbeat is registered but not running, which happens when a container crashed without unregistering.

Can I have separate workers for long jobs so they don't block the queue?

Yes, and with regular imports or large reports you should. Standard frappe_docker already runs queue-long and queue-short as separate containers. On bare-metal, bench sets each worker group's process count from background_workers in common_site_config.json (it applies to default, short and long alike); change it, run bench setup supervisor and reload Supervisor, rather than editing the generated file by hand. Watch long first: it is the queue that backs up.

What does bench purge-jobs actually delete?

Every pending job in every RQ queue for the site: default, short and long. Jobs already running are not killed. Scheduled events are not lost; the scheduler enqueues them again on its next tick. If one bad job is blocking a queue, purge-jobs --queue long limited to that queue is the safer form.

Your next step

Background jobs stopped on your ERPNext instance?

ERPNext how-to & fixes

Now see what an audit finds in yours

When the same problems keep coming back, tick what you see and get an audit of the ERPNext you have, then fixes in order of what costs you most.

Comments & ratings

No comments yet. Start a new discussion.

0

Manoj · Published on 9 August 2026

Worked with us? Leave a Google review

Questions about this? Talk to the team.

Book a 30-minute Google Meet, WhatsApp us, or send a message. The engineer who works on this replies.

Send us a message

Your details go to our team's CRM and are used only to reply.