An ERPNext bench update out of memory failure — the process is killed mid-run, exit code 137, no error stack — is almost always the Node asset build during bench update --build, not the code pull or the DB migrate. On a 2 GB VPS the fix is not automatically "buy a bigger box". Split the update into stages (pull, migrate, build), size swap correctly, cap the Node heap with NODE_OPTIONS, and if you can, build the assets on a different machine and copy them over. This ladder is how to get 2 GB and 4 GB boxes through a v16 upgrade.
The usual story behind a "the server needs more RAM" upgrade after an ERPNext update: bench update dies during the asset build, the owner assumes the whole server is undersized, and buys a machine far bigger than they need. The failure is real; the diagnosis is often wrong. I am Manoj, and I lead ERPNext and Frappe implementation and infrastructure programmes at MithTech, a team of Frappe specialists in Bengaluru. This is the ladder we walk before recommending a hardware upgrade.
How do I confirm the failure is actually OOM?
Answer
An ERPNext bench update out of memory failure has three tells: the process exits with code 137, dmesg | grep -i oom prints an oom-killer line naming a Node or Python process, and journalctl -u <bench-supervisor-unit> (or the terminal output) stops mid-sentence with no Python traceback. If your bench update ended in a clean Python exception, it was not OOM — read the exception.
Run the following in order:
1. What was the exit code?
2. Did the kernel kill something big?
3. Which processes were memory-heaviest at the time?
Signal cross-check:
| Exit code | dmesg line | What happened |
|---|---|---|
| 137 | oom-killer naming node / yarn / esbuild | Node asset build OOM'd |
| 137 | oom-killer naming python / gunicorn / uwsgi | Python side OOM'd (rare — usually pip install of a heavy dep) |
| 137 | No oom-killer line | Container/systemd sent SIGKILL for a different reason (unlikely on a bare VPS) |
| 1 or non-137 | — | Not OOM. Read the actual Python traceback and treat it as a normal failure. |
If exit was not 137, the fix ladder below does not apply — you have a code-level failure and need to read the traceback.
Add swap before you touch anything else
If your VPS has < 4 GB of RAM and no swap, add 4 GB of swap now. It is the cheapest safety net for the asset build:
Create a 4 GB swap file
Make it persistent across reboots
Verify
free -h should now show about 4 GB of swap. This is not a permanent fix: a 2 GB VPS with 4 GB of swap can complete a v16 asset build, but the build is slower because part of it runs from disk. If the build is slow but finishes, swap saved you. If it does not finish, you need one of the ladder steps below.
Don't keep adding swap. Past a few GB on a small box, thrashing costs you more time than a bigger VPS for the day would.
Which step of bench update is actually killing you?
bench update runs (in order) pull → requirements → patch → build → restart-supervisor — the flag surface is documented in bench/commands/update.py. Each stage has a distinct peak RSS profile:
| Rough peak RSS | Notes | ||
|---|---|---|---|
| Pull | bench update --pull --no-backup | ~200 MB | Git operations only; almost never OOM |
| Requirements | bench update --requirements | ~500 MB | Pip resolves + installs; can spike on a heavy dep with a C compile |
| Migrate | bench update --patch | ~400 MB | Runs schema migrations; scales with number of sites and patch complexity |
| Build | bench build | 1.5–3 GB | Node (esbuild, Vite) building JS + CSS for every app; the usual OOM victim |
| Restart | bench restart | ~100 MB | Sends signal to supervisor; negligible |
The RSS figures are rough and vary with the apps installed; measure your own with /usr/bin/time -v bench build. The build's peak is the reason a "2 GB VPS was fine yesterday and OOMs on update today" — yesterday nothing was building assets. Today bench update runs the whole pipeline as one process tree, so the peak is patch + build overlapping.
How do you split bench update into memory-safe steps?
Instead of bench update in one shot, run this on any box under 8 GB of RAM:
0 of 5 steps done
Tick a step’s number when it is done; the ticks stay on this device.
Take a backup manually
bench --site all backup --with-files. This is what bench update runs first when you do not pass --no-backup. Doing it as a separate step gets the backup safely off the critical path and lets you skip it in the update proper.
Pull code only
bench update --pull --no-backup. This runs git pull across all apps. Then bench update --requirements --no-backup installs the Python and Node dependencies the new code needs. Neither is usually a problem on a 2 GB box.
Run migrations
bench update --patch --no-backup. This runs the DB schema migrations and site-level patches. A 2 GB box usually handles this without swap.
Build assets last, with a Node heap cap
NODE_OPTIONS="--max-old-space-size=1024" bench build. This is the OOM-prone step. Capping the Node heap to 1024 MB constrains any single build worker; combined with swap, a 2 GB box will complete. For a 4 GB box, use --max-old-space-size=2048.
Restart supervisor
bench restart. Reloads gunicorn, workers and scheduler under the new code. Negligible memory.
Each step exits cleanly before the next begins, so peak memory across the whole update drops from sum(stages) to max(stages). On a 2 GB box, that is the difference between "the kernel killed everything" and an update that simply takes a while.
When should you set NODE_OPTIONS and to what?
By default, Node 20+ picks a max-old-space-size of roughly 25–50% of system RAM, which on a 4 GB VPS is ~1–2 GB — enough for a single build worker to starve everything else including MariaDB. NODE_OPTIONS="--max-old-space-size=<N>" (in MB) sets a hard ceiling; the flag is documented on the official Node.js CLI reference.
Sizing rule of thumb:
| VPS RAM | Recommended --max-old-space-size | Why |
|---|---|---|
| 2 GB | 1024 | Leaves ~1 GB for the kernel, MariaDB, and one background worker |
| 4 GB | 2048 | Leaves 2 GB for the rest of the bench + MariaDB tuning headroom |
| 8 GB | 4096 | Rarely needed to cap — but does prevent runaway workers under heavy load |
| 16 GB+ | Do not cap | Node's default is fine; capping wastes headroom |
Set it inline for one command:
Or persist it in the bench user's shell:
The persistent form is safer because supervisor-managed background jobs that shell out to Node will also see the cap.
Can you build assets on a different machine?
If your production VPS is very small (2 GB, no room to add swap, and MariaDB is already using most of it), doing the asset build on a separate host and copying the output over is a clean pattern:
-
On a build machine (a 4–8 GB VM you spin up for the upgrade; an hourly-billed cloud VM is enough):
- Clone the exact same bench directory (
git cloneof the bench repo +bench get-appfor every app on production). - Check out the same versions.
bench buildthere, where the memory peak fits.
- Clone the exact same bench directory (
-
Rsync the built assets to the production box.
sites/assets/<app>are symlinks into each app'spublicfolder, so copy what they point to (-L) and write through the symlinks on production (-K):
- On production, pull the same versions first (
bench update --pull --no-backupandbench update --requirements --no-backup), run the rsync, thenbench update --patch --no-backup && bench restart(skip--build).
Hosts that run many sites on small boxes often work this way. It also lets you build the next release on the build box while the current one keeps serving traffic.
The rest of the OOM budget: MariaDB and workers
bench build OOMs are the loudest, but two other processes commonly steal memory during an upgrade:
- MariaDB's InnoDB buffer pool sized too aggressively for the box. If you tuned
innodb_buffer_pool_sizeto 70% of RAM per the ERPNext MariaDB tuning guide, during an upgrade the build needs some of that back. Drop the buffer pool temporarily (SET GLOBAL innodb_buffer_pool_size=<smaller>) before the build, restore after. - The RQ workers. They keep running during
bench updateunless you stop them. On a 2 GB box,sudo supervisorctl stop <bench-name>:(whole group) before the upgrade andstartafter can free 200–400 MB.
Neither is required if you have adequate swap, but on a truly constrained box both are cheap wins.
Frequently asked questions
Why does bench update OOM on a 2 GB VPS when the machine was fine yesterday?
Because yesterday nothing was compiling JavaScript. The daily steady-state of ERPNext (gunicorn + workers + MariaDB + Redis) fits in 2 GB. Adding bench build's Node peak on top pushes the machine into OOM. The "the server is too small" conclusion is wrong — the steady state is fine; only the upgrade blows the ceiling. Split the update and add swap, and the same box can usually complete it.
Is --no-backup safe in production?
It's safe if you take the backup yourself in a separate step before starting. --no-backup skips the bench update-managed backup; it does not delete data. Manually running bench --site all backup --with-files before, then running bench update --no-backup, is a common way to run production upgrades: you get a backup at a known time, and the update runs without the backup step pinning extra memory.
Does more swap always help?
Up to a point. Past that, swap thrashing costs you more time than a bigger VPS would. A small VPS with a lot of swap will complete bench build, but slowly. If your upgrade window is short, pay for the bigger box for the day and downsize after.
Does exit code 137 always mean OOM on Linux?
Usually. Exit 128 + N on Linux means the process was killed by signal N; 137 = 128 + 9 where 9 is SIGKILL. The kernel's OOM killer is by far the most common source of an unsolicited SIGKILL. The other sources — a manual kill -9, a Docker container hitting --memory limits, a systemd MemoryMax — leave different fingerprints in dmesg and journalctl. dmesg | grep -i oom gives you the definitive answer in one line.
Comments & ratings
No comments yet. Start a new discussion.
Manoj · Published on 9 August 2026
Worked with us? Leave a Google review