You open the System Health Report in ERPNext, and Scheduler Status says Process Not Found. Your scheduler container is up. Your background jobs are running. Nothing is wrong. In any deployment where bench schedule runs in its own container, which is every standard frappe_docker Compose and Kubernetes helm setup, that field is a false negative. The health check probes a file lock on config/scheduler_process, and config/ is not a shared volume, so the web container locks its own private copy of the file and concludes nobody is running the scheduler. It was tracked as frappe/frappe#40828. The fix, PR #40866, was merged into develop on 1 September 2026, but as of 12 October 2026 it has not been backported: version-15 and version-16 (latest tag v16.51.1) still use the file lock, so there is no v15 or v16 release you can upgrade to for it.
The most confusing bugs are the ones where the software is wrong about itself. Here a diagnostic screen, added to help you find problems, reports a problem that does not exist, on the most common production topology. I am Manoj, and I lead ERPNext and Frappe programmes at MithTech, a team of Frappe specialists in Bengaluru. A liveness bug like this eats an afternoon before somebody thinks to check whether the scheduled jobs are running.
This post is part of the Frappe and ERPNext Architecture Guide, whose chapter on the scheduler and background jobs reads the code behind it.
Does this bug apply to you?
The whole thing turns on one question: does the process that renders the System Health Report share a filesystem with the process running bench schedule? In local development under bench start they are sibling processes on one disk, and the check works, which is why this never shows up until production.
Interactive
Deployment Topology Checker
Pick how you actually run Frappe or ERPNext — single container, frappe_docker Compose split, Kubernetes helm, bare-metal single host, or bare-metal multi-host — and get a verdict on whether the Process Not Found false negative applies, why, and how to confirm it yourself.
The affected list is read straight from the deployment definitions. The production Containerfile in frappe_docker declares VOLUME for sites and logs only. The x-backend-defaults anchor in compose.yaml mounts exactly one volume, sites. The helm chart's deployment-scheduler.yaml mounts sites-dir and logs. In every one of those, config/ is created by bench init inside the image layer and stays in each container's own copy-on-write layer.
Why is acquiring a lock read as "dead"?
The function at the centre of this is is_schduler_process_running() in frappe/utils/scheduler.py, at line 63 on version-16. The spelling is not a typo in this article: the real function name is missing the second "e", schduler. That is the string to grep for on v15 and v16. On develop, PR #40866 replaced it with a correctly spelled is_scheduler_process_running() that reads the Redis heartbeat.
Interactive
FileLock Probe Explainer
Walk the five steps of the health check side by side in two worlds: a shared inode where the probe works as designed, and split containers where each process locks its own private copy of config/scheduler_process. See exactly where the inverted logic turns a healthy scheduler into Process Not Found.
The mechanism, plainly: start_scheduler() acquires a non-blocking FileLock on <bench>/config/scheduler_process and never releases it; the kernel drops it when the process exits. That held lock is the entire liveness signal. is_schduler_process_running() then tries to acquire the same lock: a Timeout means somebody else holds it, so the scheduler is alive; a successful acquire means nobody holds it, so the scheduler is dead. A successful lock acquisition is the failure signal.
flock locks are held against an open file description on a specific inode on a specific filesystem. They are not visible across containers with separate writable layers, and they are not visible across hosts. So in a split deployment the backend container's probe finds its own freshly created, unlocked scheduler_process file, acquires it at once, and returns False, every time.
Two details make this worse than it needs to be. First, the file is not a PID file: its contents are never read, so there is nothing in it to cross-check. Second, in fetch_scheduler() the "Process Not Found" branch is evaluated first and short-circuits the Dormant, Active and Inactive branches, so a perfectly healthy enabled scheduler never gets a chance to report itself as anything else.
How do you tell whether your scheduler is alive?
Code recipe
Scheduler Liveness Recipe
Three tabs of runnable checks and configuration: reliable liveness checks that read the shared database and Redis instead of the file lock, workarounds clearly labelled as hacks, and the upstream Redis-heartbeat fix with the buggy code it replaces. Copy button on each tab.
0 of 5 steps done
Tick a step’s number when it is done; the ticks stay on this device.
Ask the database, not the dashboard
Query Scheduled Job Log for the newest row. The scheduler's only job is to enqueue events, and every executed scheduled job writes a row to the shared database, which no container-layer quirk can distort. The default scheduler tick is 60 seconds on version-15 and 240 seconds on version-16, overridable with scheduler_tick_interval in common_site_config.json. A newest row that is a few minutes old means your scheduler is doing its job.
Look at the container
docker compose ps scheduler, then docker compose exec scheduler ps aux | grep "bench schedule", then the last hundred lines of its logs. On Kubernetes, kubectl logs on the scheduler Deployment; check the rendered labels with kubectl get deploy first, since they depend on your release name and chart values.
Prove the inode mismatch
Run ls -li /home/frappe/frappe-bench/config/scheduler_process in both the backend and scheduler containers. Different inode numbers is the proof. You can go further and run the probe itself on both sides: in a split deployment it returns True inside the scheduler container and False inside backend. That difference is the bug, reproduced in two commands.
Rule out the config-only causes
This is where bench --site <site> scheduler status earns its keep, and only here. It reads maintenance_mode, pause_scheduler, disable_scheduler and System Settings.enable_scheduler: configuration, no process check at all. It will report "enabled" with the scheduler container stopped. Use it to rule out "someone switched it off", then stop trusting it.
Replace the signal with monitoring that works
Add a Compose healthcheck or a Kubernetes liveness probe on the scheduler service; those run inside the scheduler container, where a process check means something. Then alert externally when the newest Scheduled Job Log row exceeds roughly three scheduler ticks. Do this from your monitoring stack, not from a Frappe Notification: a scheduler-driven alert cannot tell you the scheduler is down.
The scheduler checks for due jobs on a fixed tick, and webhooks vs polling plots what that check costs against a webhook for each change.
A scheduler that really has stopped matters to the books too: Auto Reconcile Payments needs it running, and if it is off nothing reconciles. Reconciliation as a job shows what to compare with the bank so a missed run is noticed.
Frappe's Email Queue is an outbox the scheduler flushes, so by default mail waits while it is off; the transactional outbox says to alert on the oldest unsent row's age.
What about bench doctor?
bench doctor is useful, but it does not answer this question. Its Workers online: count comes from rq.Worker.all() over the shared queue Redis, so it does see workers in other containers, but bench schedule is not an RQ worker and never appears in that count. A healthy Workers online: 2 tells you nothing about the scheduler. The scheduler lines it prints all come from is_scheduler_disabled() and is_scheduler_inactive(), which are config-only. bench doctor never calls is_schduler_process_running() at all. Its queue depths are Redis-backed and accurate cross-container, which makes it a good tool for the adjacent question of whether your workers are keeping up.
The official Diagnosing The Scheduler page recommends bench doctor, show-pending-jobs and purge-jobs. That page predates containerised splits and does not mention the file-lock probe. There is also no documentation page for the System Health Report itself; the source file is the only reference, which is a large part of why this is confusing.
| What it actually reads | |||
|---|---|---|---|
| System Health Report → Scheduler Status (v15, v16) | flock on config/scheduler_process | No — private inode per container | Always wrong. Ignore it. |
| Scheduled Job Log recency | Shared database rows | Yes | The reliable answer. Use this. |
| Scheduled Job Type.last_execution | Shared database rows | Yes | Reliable. Advancing values = working scheduler. |
| docker compose ps / kubectl logs | The scheduler container itself | N/A — you look directly | Reliable, but manual. |
| bench doctor | RQ workers + queue depths via Redis; scheduler config | Yes for workers, N/A for scheduler | Config-only for the scheduler. Misleading here. |
| bench --site <site> scheduler status | maintenance_mode, pause_scheduler, disable_scheduler, System Settings | N/A — config only | Config-only. Says 'enabled' with the container stopped. |
| show-pending-jobs | Redis queue depths | Yes | Tells you about workers, not the scheduler. |
Which scheduler checks survive a split-container deployment
What should you check before concluding anything is broken?
Interactive
Liveness Checklist
Eight checks to work through before treating Process Not Found as an incident — from Scheduled Job Log freshness and the accurate neighbouring fields, through the inode comparison, to the external alerting that should replace this status field entirely.
What should you know before you pick a workaround?
Sharing config/ works, on exactly one topology. A named Docker volume mounted at /home/frappe/frappe-bench/config in both backend and scheduler gives them the same inode, and the probe starts working. Use a named volume, not a host bind mount: Docker seeds a named volume from the image contents on first use, while a bind mount would empty the directory and take bench-generated config with it. This is single-host Compose only. It does not work across Kubernetes nodes or multi-host Swarm.
It is a symptom fix. You still have a filesystem lock doing distributed coordination, which is the actual design problem. And it changes behaviour you may not have been thinking about: with a genuinely shared config/, a second scheduler on the same volume will now correctly refuse to start. That is arguably an improvement, but it is a change.
Do not reach for a shared RWX PVC on Kubernetes. Advisory flock over NFS and many CSI RWX drivers is unreliable or a silent no-op. It can appear to work, then regress, or worse, let two schedulers both acquire the lock and both run. Never use file locks for cross-node coordination.
Running bench schedule inside the backend container is a dev-box move. It collapses the split so the probe's assumption holds, but you lose independent restart and scaling, and any backend replica count above 1 gives you N schedulers racing, because the flock only prevents duplicates within one filesystem.
Patching the framework is rarely worth it. frappe_docker supports custom image builds, so you can apply the merged PR #40866 diff to your v15 or v16 image yourself. If you already maintain a custom image, fine. If you don't, carrying a framework patch to fix a cosmetic status field adds work to every upgrade until a backport lands.
Where does the upstream fix stand?
The design in PR #40866 is the obvious one: publish a scheduler heartbeat key in queue Redis with a TTL, refresh it while the scheduler sleeps and again before each site, and read that key in the health report. Redis is already shared across every container, so it is the natural cross-container liveness channel, and TTL expiry gives you automatic dead-detection. The file lock stays, for the job it is good at: preventing duplicate schedulers on one filesystem. A new Redis Unavailable status separates "I cannot tell" from "it is dead".
Review found and closed one real gap: the heartbeat was only refreshed while sleeping, so a long multi-site enqueue pass could let the 120-second key expire mid-run. The accepted answer was a per-site refresh inside the loop. A background heartbeat thread was proposed and rejected: if a single site takes longer than the TTL to enqueue, something really is stuck, and a thread would hide the signal you want.
The PR was merged into develop on 1 September 2026, and issue #40828 was closed the same day. As of 12 October 2026 there is no backport to version-15 or version-16; those branches still use the file-lock check. If you need it on v16, asking for a backport on the PR is the most useful thing you can do. Re-check the branches before acting on this: a backport is the fact most likely to change.
At a glance
| Answer | |
|---|---|
| Who sees the false "Process Not Found" | v15 and v16 deployments with the scheduler in its own container or pod |
| Why | The health check probes a file lock in config/, which each container holds privately |
| What to trust instead | Newest Scheduled Job Log row, within a few scheduler ticks (60 s on version-15, 240 s on version-16) |
| What not to trust | bench doctor and bench scheduler status for liveness (config only) |
| Upstream fix | PR #40866 (Redis heartbeat), merged to develop on 1 Sep 2026; no v15/v16 backport as of 12 Oct 2026 |
| Monitoring | Healthcheck on the scheduler service plus an external alert on job-log freshness |
Frequently asked questions
Why does ERPNext System Health Report say Process Not Found for the scheduler when my scheduler container is running?
Because the check is a file-lock probe on <bench>/config/scheduler_process, and config/ is not a shared volume in frappe_docker or the helm chart. The backend container locks its own private copy of that file, succeeds, and concludes nobody is holding it, which the code reads as "the scheduler is not running". Your scheduler is almost certainly fine. Confirm with Scheduled Job Log recency. Tracked as frappe/frappe#40828; the fix is merged to develop but not yet in v15 or v16.
Is Process Not Found in the Frappe System Health Report a real outage or a false alarm?
In a split Docker Compose or Kubernetes deployment it is a false alarm if Scheduled Job Log rows are still being created within a few scheduler ticks; the default tick is 60 seconds on version-15 and 240 seconds on version-16. It is a real outage if the log is stale and your scheduler container or pod is down or restart-looping. The two neighbouring fields in the same report, "Failing Scheduled Jobs" and "Oldest Unscheduled Job", are database-backed and remain reliable.
How do I check if the Frappe scheduler is actually running in Docker?
Do not use the System Health Report. Run docker compose ps scheduler and docker compose logs --tail=100 scheduler, then verify jobs are advancing by querying the newest Scheduled Job Log row. If it is within a few minutes, the scheduler is alive. bench --site <site> scheduler status reports configuration only, not liveness, so it cannot answer this.
Why is my ERPNext scheduler showing Process Not Found in Docker?
Because Frappe checks whether the scheduler is alive by trying to grab a file lock, and in Docker your web container and your scheduler container each have their own separate copy of that file. The web container grabs its own lock successfully and assumes nothing is running. The scheduler is fine; check whether scheduled jobs are still being logged.
How can I tell if the ERPNext background scheduler is really working in Kubernetes?
Look at whether scheduled jobs are still finishing, not at the health report. Run kubectl logs on the scheduler pod, and query the Scheduled Job Log table for the most recent entry. If new rows are appearing every few minutes, the scheduler is doing its job even if the dashboard says otherwise. Sharing a volume will not help you here, because the pods can be on different nodes.
Which Frappe or ERPNext version fixes the scheduler Process Not Found false negative in containers?
No v15 or v16 release yet, as of 12 October 2026. The fix, PR frappe/frappe#40866, replaces the file lock with a Redis heartbeat and was merged into develop on 1 September 2026. version-15, version-16, version-16-hotfix and the latest tag v16.51.1 still carry the old check. Watch for a backport, then read the release notes of the next v16 tag.
Does bench doctor detect that the Frappe scheduler is down?
Not directly. bench doctor reports RQ workers online via Redis (so it does see other containers), queue depths, and whether the scheduler is configured as enabled, paused or in maintenance mode. It never calls is_schduler_process_running(), and bench schedule is not an RQ worker, so it never appears in the worker count. For this purpose it is a config-only check. Use Scheduled Job Log freshness plus docker compose ps scheduler instead.
What happens if we just ignore the Process Not Found status?
Functionally, nothing: the field is cosmetic and your scheduler keeps working. The real cost is that you have no working scheduler health signal, so a real scheduler outage looks the same as the false negative and nobody notices. Replace the signal: put a healthcheck or liveness probe on the scheduler service, and alert externally when the newest Scheduled Job Log row is older than about three scheduler ticks.
None of the forum threads above name the "Process Not Found" string; they are related conversations about scheduler behaviour in Docker and Kubernetes. When this post was written I found no discuss.frappe.io topic about this bug by name, and GitHub #40828 was the main place it was written up. If you search and find little, you are not looking in the wrong place.
The short version: if your scheduler container is up and Scheduled Job Log is advancing, the report is wrong and you should leave it alone. Put a healthcheck on the scheduler service, alert on job-log freshness from outside Frappe, and watch for a v15 or v16 backport of PR #40866. Once your release has it, the status field is worth reading again; until then, treating a cosmetic red flag as an incident costs more than the bug does. We work through this on every ERPNext implementation that ships to containers.
Comments & ratings
No comments yet. Start a new discussion.
Manoj · Published on 5 August 2026
Worked with us? Leave a Google review