Skip to main content

ERPNext how-to & fixes

ERPNext Scheduler Shows Process Not Found in Docker — It's a False Negative

Frappe's System Health Report shows the scheduler as Process Not Found when it runs in its own container. It is a file-lock bug, not an outage.

Updated 14 Oct 202614 min readBy ManojFirst published 5 Aug 2026

You open the System Health Report in ERPNext, and Scheduler Status says Process Not Found. Your scheduler container is up. Your background jobs are running. Nothing is wrong. In any deployment where bench schedule runs in its own container, which is every standard frappe_docker Compose and Kubernetes helm setup, that field is a false negative. The health check probes a file lock on config/scheduler_process, and config/ is not a shared volume, so the web container locks its own private copy of the file and concludes nobody is running the scheduler. It was tracked as frappe/frappe#40828. The fix, PR #40866, was merged into develop on 1 September 2026, but as of 12 October 2026 it has not been backported: version-15 and version-16 (latest tag v16.51.1) still use the file lock, so there is no v15 or v16 release you can upgrade to for it.

The most confusing bugs are the ones where the software is wrong about itself. Here a diagnostic screen, added to help you find problems, reports a problem that does not exist, on the most common production topology. I am Manoj, and I lead ERPNext and Frappe programmes at MithTech, a team of Frappe specialists in Bengaluru. A liveness bug like this eats an afternoon before somebody thinks to check whether the scheduled jobs are running.

This post is part of the Frappe and ERPNext Architecture Guide, whose chapter on the scheduler and background jobs reads the code behind it.

Does this bug apply to you?

The whole thing turns on one question: does the process that renders the System Health Report share a filesystem with the process running bench schedule? In local development under bench start they are sibling processes on one disk, and the check works, which is why this never shows up until production.

Interactive

Deployment Topology Checker

Pick how you actually run Frappe or ERPNext — single container, frappe_docker Compose split, Kubernetes helm, bare-metal single host, or bare-metal multi-host — and get a verdict on whether the Process Not Found false negative applies, why, and how to confirm it yourself.

Pick the way you actually run Frappe/ERPNext. The verdict comes from one question: do the process that renders the health report and the process that runs bench schedule share the filesystem holding config/?

Affected — expect a permanent false negative

The stock compose.yaml puts gunicorn in the backend service and `bench schedule` in a separate scheduler service. x-backend-defaults mounts exactly one volume, sites. config/ is not shared, so each container writes and locks its own private copy of config/scheduler_process.

config/ shared?No
Scheduler Status fieldLies
Topologies affected4 of 6
How to confirm it yourself

Run `ls -li /home/frappe/frappe-bench/config/scheduler_process` in both containers. Different inode numbers confirm it.

What to do next

Ignore the field. Check Scheduled Job Log recency instead (tab 1 of the recipe below).

Detail: No — config/ lives in each container's writable layer. The affected/not-affected split is read from frappe_docker's compose.yaml, the production Containerfile VOLUME declaration, and frappe/helm's deployment-scheduler.yaml.

The affected list is read straight from the deployment definitions. The production Containerfile in frappe_docker declares VOLUME for sites and logs only. The x-backend-defaults anchor in compose.yaml mounts exactly one volume, sites. The helm chart's deployment-scheduler.yaml mounts sites-dir and logs. In every one of those, config/ is created by bench init inside the image layer and stays in each container's own copy-on-write layer.

Why is acquiring a lock read as "dead"?

The function at the centre of this is is_schduler_process_running() in frappe/utils/scheduler.py, at line 63 on version-16. The spelling is not a typo in this article: the real function name is missing the second "e", schduler. That is the string to grep for on v15 and v16. On develop, PR #40866 replaced it with a correctly spelled is_scheduler_process_running() that reads the Redis heartbeat.

Interactive

FileLock Probe Explainer

Walk the five steps of the health check side by side in two worlds: a shared inode where the probe works as designed, and split containers where each process locks its own private copy of config/scheduler_process. See exactly where the inverted logic turns a healthy scheduler into Process Not Found.

The liveness check is inverted: it treats a successful lock acquisition as proof the scheduler is dead. That only works when both processes contend on the same inode. Step through both worlds side by side.

1. The scheduler starts and grabs the lock

Shared inode — one filesystem

start_scheduler() opens <bench>/config/scheduler_process and calls lock.acquire(blocking=False). It succeeds, and the lock is never released — the kernel holds it for the lifetime of the process.

Private inode — split containers

Identical. Inside the scheduler container, the lock is acquired on that container's own copy of config/scheduler_process — an inode in its private writable layer.

The function is really spelled is_schduler_process_running — missing the second “e”. That is the string to grep for in frappe/utils/scheduler.py. Note the file never stores a PID; its contents are irrelevant, only the lock matters.

The mechanism, plainly: start_scheduler() acquires a non-blocking FileLock on <bench>/config/scheduler_process and never releases it; the kernel drops it when the process exits. That held lock is the entire liveness signal. is_schduler_process_running() then tries to acquire the same lock: a Timeout means somebody else holds it, so the scheduler is alive; a successful acquire means nobody holds it, so the scheduler is dead. A successful lock acquisition is the failure signal.

flock locks are held against an open file description on a specific inode on a specific filesystem. They are not visible across containers with separate writable layers, and they are not visible across hosts. So in a split deployment the backend container's probe finds its own freshly created, unlocked scheduler_process file, acquires it at once, and returns False, every time.

Two details make this worse than it needs to be. First, the file is not a PID file: its contents are never read, so there is nothing in it to cross-check. Second, in fetch_scheduler() the "Process Not Found" branch is evaluated first and short-circuits the Dormant, Active and Inactive branches, so a perfectly healthy enabled scheduler never gets a chance to report itself as anything else.

How do you tell whether your scheduler is alive?

Code recipe

Scheduler Liveness Recipe

Three tabs of runnable checks and configuration: reliable liveness checks that read the shared database and Redis instead of the file lock, workarounds clearly labelled as hacks, and the upstream Redis-heartbeat fix with the buggy code it replaces. Copy button on each tab.

Trustworthy

Everything here reads the shared database or Redis, or looks at the scheduler container directly — none of it depends on the file lock. Note that `bench doctor` and `bench --site <site> scheduler status` are config-only checks: they report whether the scheduler is enabled, paused, or in maintenance mode, never whether the process is alive. In split containers they are actively misleading, which is why they are at the bottom of this tab.

bash
# ── 1. THE RELIABLE ONE: is the Scheduled Job Log still advancing?
# The scheduler's only job is to enqueue events. Every executed
# scheduled job writes a row to the SHARED database, so this
# cannot lie about which container did the work.

# MariaDB
SELECT
  MAX(creation)                               AS last_job,
  TIMESTAMPDIFF(SECOND, MAX(creation), NOW()) AS seconds_ago,
  COUNT(*)                                    AS jobs_last_hour
FROM `tabScheduled Job Log`
WHERE creation > NOW() - INTERVAL 1 HOUR;

# PostgreSQL
SELECT
  MAX(creation)                                    AS last_job,
  EXTRACT(EPOCH FROM (NOW() - MAX(creation)))::int AS seconds_ago,
  COUNT(*)                                         AS jobs_last_hour
FROM "tabScheduled Job Log"
WHERE creation > NOW() - INTERVAL '1 hour';

# Default scheduler tick: 60 s on version-15, 240 s on version-16 (DEFAULT_SCHEDULER_TICK),
# overridable via scheduler_tick_interval in common_site_config.json.
# Newer than a few ticks => your scheduler is ALIVE, whatever the
# System Health Report says.

# Same thing from bench console:
#   frappe.get_all("Scheduled Job Type",
#       filters={"stopped": 0},
#       fields=["name", "method", "frequency", "last_execution"],
#       order_by="last_execution asc", limit=10)
# Recent, advancing last_execution values = working scheduler.


# ── 2. LOOK AT THE SCHEDULER PROCESS DIRECTLY
docker compose ps scheduler
docker compose exec scheduler ps aux | grep "bench schedule"
docker compose logs --tail=100 scheduler

# Kubernetes (verify the real labels with kubectl get deploy first)
kubectl logs deploy/<release>-erpnext-scheduler --tail=100
kubectl exec deploy/<release>-erpnext-scheduler -- ps aux | grep schedule


# ── 3. PROVE THE FALSE NEGATIVE IN TWO COMMANDS
# Different inode numbers = the two containers are locking
# different files, so the probe can never work.
docker compose exec scheduler ls -li /home/frappe/frappe-bench/config/scheduler_process
docker compose exec backend   ls -li /home/frappe/frappe-bench/config/scheduler_process

# Or run the exact probe on both sides. Expected in a split
# deployment: True inside scheduler, False inside backend.
docker compose exec scheduler python -c \
  "import frappe; from frappe.utils.scheduler import is_schduler_process_running as r; print(r())"
docker compose exec backend python -c \
  "import frappe; from frappe.utils.scheduler import is_schduler_process_running as r; print(r())"


# ── 4. RQ / QUEUE CHECKS — useful, but not about the scheduler
docker compose exec backend bench --site <site> show-pending-jobs
# Redis-backed, so cross-container accurate. A GROWING backlog with
# no Scheduled Job Log progress = your WORKERS are down. A FLAT empty
# backlog with no progress = the SCHEDULER really is down.


# ── 5. CONFIG-ONLY CHECKS — read these as "is it switched on",
#     never as "is it alive". Both are misleading in split containers.
docker compose exec backend bench doctor
#   "Workers online: N" comes from rq.Worker.all() over shared Redis,
#   so it does see other containers — but `bench schedule` is NOT an
#   RQ worker and never appears in that count. bench doctor never
#   calls is_schduler_process_running(). Its scheduler lines are
#   is_scheduler_inactive() output: config only.

docker compose exec backend bench --site <site> scheduler status
#   -> "Scheduler is enabled for site <site>"
#   This checks NO process. is_scheduler_inactive() only reads
#   maintenance_mode, pause_scheduler, disable_scheduler and
#   System Settings.enable_scheduler. It will happily say "enabled"
#   with the scheduler container stopped. Good for ruling out
#   "someone disabled it", useless for "is it running".

0 of 5 steps done

Tick a step’s number when it is done; the ticks stay on this device.

Ask the database, not the dashboard

Query Scheduled Job Log for the newest row. The scheduler's only job is to enqueue events, and every executed scheduled job writes a row to the shared database, which no container-layer quirk can distort. The default scheduler tick is 60 seconds on version-15 and 240 seconds on version-16, overridable with scheduler_tick_interval in common_site_config.json. A newest row that is a few minutes old means your scheduler is doing its job.

Look at the container

docker compose ps scheduler, then docker compose exec scheduler ps aux | grep "bench schedule", then the last hundred lines of its logs. On Kubernetes, kubectl logs on the scheduler Deployment; check the rendered labels with kubectl get deploy first, since they depend on your release name and chart values.

Prove the inode mismatch

Run ls -li /home/frappe/frappe-bench/config/scheduler_process in both the backend and scheduler containers. Different inode numbers is the proof. You can go further and run the probe itself on both sides: in a split deployment it returns True inside the scheduler container and False inside backend. That difference is the bug, reproduced in two commands.

Rule out the config-only causes

This is where bench --site <site> scheduler status earns its keep, and only here. It reads maintenance_mode, pause_scheduler, disable_scheduler and System Settings.enable_scheduler: configuration, no process check at all. It will report "enabled" with the scheduler container stopped. Use it to rule out "someone switched it off", then stop trusting it.

Replace the signal with monitoring that works

Add a Compose healthcheck or a Kubernetes liveness probe on the scheduler service; those run inside the scheduler container, where a process check means something. Then alert externally when the newest Scheduled Job Log row exceeds roughly three scheduler ticks. Do this from your monitoring stack, not from a Frappe Notification: a scheduler-driven alert cannot tell you the scheduler is down.

The scheduler checks for due jobs on a fixed tick, and webhooks vs polling plots what that check costs against a webhook for each change.

A scheduler that really has stopped matters to the books too: Auto Reconcile Payments needs it running, and if it is off nothing reconciles. Reconciliation as a job shows what to compare with the bank so a missed run is noticed.

Frappe's Email Queue is an outbox the scheduler flushes, so by default mail waits while it is off; the transactional outbox says to alert on the oldest unsent row's age.

What about bench doctor?

bench doctor is useful, but it does not answer this question. Its Workers online: count comes from rq.Worker.all() over the shared queue Redis, so it does see workers in other containers, but bench schedule is not an RQ worker and never appears in that count. A healthy Workers online: 2 tells you nothing about the scheduler. The scheduler lines it prints all come from is_scheduler_disabled() and is_scheduler_inactive(), which are config-only. bench doctor never calls is_schduler_process_running() at all. Its queue depths are Redis-backed and accurate cross-container, which makes it a good tool for the adjacent question of whether your workers are keeping up.

The official Diagnosing The Scheduler page recommends bench doctor, show-pending-jobs and purge-jobs. That page predates containerised splits and does not mention the file-lock probe. There is also no documentation page for the System Health Report itself; the source file is the only reference, which is a large part of why this is confusing.

What it actually reads
System Health Report → Scheduler Status (v15, v16)flock on config/scheduler_processNo — private inode per containerAlways wrong. Ignore it.
Scheduled Job Log recencyShared database rowsYesThe reliable answer. Use this.
Scheduled Job Type.last_executionShared database rowsYesReliable. Advancing values = working scheduler.
docker compose ps / kubectl logsThe scheduler container itselfN/A — you look directlyReliable, but manual.
bench doctorRQ workers + queue depths via Redis; scheduler configYes for workers, N/A for schedulerConfig-only for the scheduler. Misleading here.
bench --site <site> scheduler statusmaintenance_mode, pause_scheduler, disable_scheduler, System SettingsN/A — config onlyConfig-only. Says 'enabled' with the container stopped.
show-pending-jobsRedis queue depthsYesTells you about workers, not the scheduler.

Which scheduler checks survive a split-container deployment

What should you check before concluding anything is broken?

Interactive

Liveness Checklist

Eight checks to work through before treating Process Not Found as an incident — from Scheduled Job Log freshness and the accurate neighbouring fields, through the inode comparison, to the external alerting that should replace this status field entirely.

Walk all eight before you conclude anything is broken. If the first five pass, “Process Not Found” is the false negative and there is nothing to fix.

0 of 8 done

Nothing here is saved — this is a checklist for your current setup, not your ERPNext instance.

What should you know before you pick a workaround?

Sharing config/ works, on exactly one topology. A named Docker volume mounted at /home/frappe/frappe-bench/config in both backend and scheduler gives them the same inode, and the probe starts working. Use a named volume, not a host bind mount: Docker seeds a named volume from the image contents on first use, while a bind mount would empty the directory and take bench-generated config with it. This is single-host Compose only. It does not work across Kubernetes nodes or multi-host Swarm.

It is a symptom fix. You still have a filesystem lock doing distributed coordination, which is the actual design problem. And it changes behaviour you may not have been thinking about: with a genuinely shared config/, a second scheduler on the same volume will now correctly refuse to start. That is arguably an improvement, but it is a change.

Do not reach for a shared RWX PVC on Kubernetes. Advisory flock over NFS and many CSI RWX drivers is unreliable or a silent no-op. It can appear to work, then regress, or worse, let two schedulers both acquire the lock and both run. Never use file locks for cross-node coordination.

Running bench schedule inside the backend container is a dev-box move. It collapses the split so the probe's assumption holds, but you lose independent restart and scaling, and any backend replica count above 1 gives you N schedulers racing, because the flock only prevents duplicates within one filesystem.

Patching the framework is rarely worth it. frappe_docker supports custom image builds, so you can apply the merged PR #40866 diff to your v15 or v16 image yourself. If you already maintain a custom image, fine. If you don't, carrying a framework patch to fix a cosmetic status field adds work to every upgrade until a backport lands.

Where does the upstream fix stand?

The design in PR #40866 is the obvious one: publish a scheduler heartbeat key in queue Redis with a TTL, refresh it while the scheduler sleeps and again before each site, and read that key in the health report. Redis is already shared across every container, so it is the natural cross-container liveness channel, and TTL expiry gives you automatic dead-detection. The file lock stays, for the job it is good at: preventing duplicate schedulers on one filesystem. A new Redis Unavailable status separates "I cannot tell" from "it is dead".

Review found and closed one real gap: the heartbeat was only refreshed while sleeping, so a long multi-site enqueue pass could let the 120-second key expire mid-run. The accepted answer was a per-site refresh inside the loop. A background heartbeat thread was proposed and rejected: if a single site takes longer than the TTL to enqueue, something really is stuck, and a thread would hide the signal you want.

The PR was merged into develop on 1 September 2026, and issue #40828 was closed the same day. As of 12 October 2026 there is no backport to version-15 or version-16; those branches still use the file-lock check. If you need it on v16, asking for a backport on the PR is the most useful thing you can do. Re-check the branches before acting on this: a backport is the fact most likely to change.

At a glance

Answer
Who sees the false "Process Not Found"v15 and v16 deployments with the scheduler in its own container or pod
WhyThe health check probes a file lock in config/, which each container holds privately
What to trust insteadNewest Scheduled Job Log row, within a few scheduler ticks (60 s on version-15, 240 s on version-16)
What not to trustbench doctor and bench scheduler status for liveness (config only)
Upstream fixPR #40866 (Redis heartbeat), merged to develop on 1 Sep 2026; no v15/v16 backport as of 12 Oct 2026
MonitoringHealthcheck on the scheduler service plus an external alert on job-log freshness

Frequently asked questions

Why does ERPNext System Health Report say Process Not Found for the scheduler when my scheduler container is running?

Because the check is a file-lock probe on <bench>/config/scheduler_process, and config/ is not a shared volume in frappe_docker or the helm chart. The backend container locks its own private copy of that file, succeeds, and concludes nobody is holding it, which the code reads as "the scheduler is not running". Your scheduler is almost certainly fine. Confirm with Scheduled Job Log recency. Tracked as frappe/frappe#40828; the fix is merged to develop but not yet in v15 or v16.

Is Process Not Found in the Frappe System Health Report a real outage or a false alarm?

In a split Docker Compose or Kubernetes deployment it is a false alarm if Scheduled Job Log rows are still being created within a few scheduler ticks; the default tick is 60 seconds on version-15 and 240 seconds on version-16. It is a real outage if the log is stale and your scheduler container or pod is down or restart-looping. The two neighbouring fields in the same report, "Failing Scheduled Jobs" and "Oldest Unscheduled Job", are database-backed and remain reliable.

How do I check if the Frappe scheduler is actually running in Docker?

Do not use the System Health Report. Run docker compose ps scheduler and docker compose logs --tail=100 scheduler, then verify jobs are advancing by querying the newest Scheduled Job Log row. If it is within a few minutes, the scheduler is alive. bench --site <site> scheduler status reports configuration only, not liveness, so it cannot answer this.

Why is my ERPNext scheduler showing Process Not Found in Docker?

Because Frappe checks whether the scheduler is alive by trying to grab a file lock, and in Docker your web container and your scheduler container each have their own separate copy of that file. The web container grabs its own lock successfully and assumes nothing is running. The scheduler is fine; check whether scheduled jobs are still being logged.

How can I tell if the ERPNext background scheduler is really working in Kubernetes?

Look at whether scheduled jobs are still finishing, not at the health report. Run kubectl logs on the scheduler pod, and query the Scheduled Job Log table for the most recent entry. If new rows are appearing every few minutes, the scheduler is doing its job even if the dashboard says otherwise. Sharing a volume will not help you here, because the pods can be on different nodes.

Which Frappe or ERPNext version fixes the scheduler Process Not Found false negative in containers?

No v15 or v16 release yet, as of 12 October 2026. The fix, PR frappe/frappe#40866, replaces the file lock with a Redis heartbeat and was merged into develop on 1 September 2026. version-15, version-16, version-16-hotfix and the latest tag v16.51.1 still carry the old check. Watch for a backport, then read the release notes of the next v16 tag.

Does bench doctor detect that the Frappe scheduler is down?

Not directly. bench doctor reports RQ workers online via Redis (so it does see other containers), queue depths, and whether the scheduler is configured as enabled, paused or in maintenance mode. It never calls is_schduler_process_running(), and bench schedule is not an RQ worker, so it never appears in the worker count. For this purpose it is a config-only check. Use Scheduled Job Log freshness plus docker compose ps scheduler instead.

What happens if we just ignore the Process Not Found status?

Functionally, nothing: the field is cosmetic and your scheduler keeps working. The real cost is that you have no working scheduler health signal, so a real scheduler outage looks the same as the false negative and nobody notices. Replace the signal: put a healthcheck or liveness probe on the scheduler service, and alert externally when the newest Scheduled Job Log row is older than about three scheduler ticks.

None of the forum threads above name the "Process Not Found" string; they are related conversations about scheduler behaviour in Docker and Kubernetes. When this post was written I found no discuss.frappe.io topic about this bug by name, and GitHub #40828 was the main place it was written up. If you search and find little, you are not looking in the wrong place.

The short version: if your scheduler container is up and Scheduled Job Log is advancing, the report is wrong and you should leave it alone. Put a healthcheck on the scheduler service, alert on job-log freshness from outside Frappe, and watch for a v15 or v16 backport of PR #40866. Once your release has it, the status field is worth reading again; until then, treating a cosmetic red flag as an incident costs more than the bug does. We work through this on every ERPNext implementation that ships to containers.

Your next step

Running Frappe or ERPNext across containers and not sure what to trust?

ERPNext how-to & fixes

Now see what an audit finds in yours

When the same problems keep coming back, tick what you see and get an audit of the ERPNext you have, then fixes in order of what costs you most.

Comments & ratings

No comments yet. Start a new discussion.

0

Manoj · Published on 5 August 2026

Worked with us? Leave a Google review

Questions about this? Talk to the team.

Book a 30-minute Google Meet, WhatsApp us, or send a message. The engineer who works on this replies.

Send us a message

Your details go to our team's CRM and are used only to reply.