All posts

30 apps on one server: systemd, Docker and Firecracker, measured

I wanted all my small apps on one server, each walled off from the others. So I deployed the same 30 apps as systemd services, Docker containers and Firecracker micro-VMs, and measured the memory: 100, 130 and 583 MiB per app.

Like everyone else, I've built dozens of small apps with coding agents. Some are just for me, others a few people use too. I have a small VPS, and I wanted one clean way to deploy all of them on it, securely, with the data staying on my own machine.

You could say "just deploy on Vercel". But my apps don't share a stack: a Next.js wiki on SQLite, a Django CRM on PostgreSQL, a static docs site. Vercel isn't flexible enough for that mix, and the data would live on someone else's servers. I have a VPS, I want everything on it: HTTPS everywhere, a password in front of the personal sites, monitoring, and each app walled off from the others.

That last part has a price, paid once per app. On one Linux machine, there are three common ways to keep apps apart: a systemd service that systemd sandboxes itself, a Docker container, or a micro-VM with a kernel of its own. On a small VPS, memory decides how many apps fit. So I measured it: I deployed the same 30 apps all three ways, ran the same realistic traffic against all of them at once, and read the memory.

Memory per app, with 30 apps deployed
  1. systemd services100 MiB
  2. Docker containers130 MiB
  3. Firecracker micro-VMs583 MiB

The same apps, the same traffic, the same latency in all three, a p95 of 69 ms on the wiki.

The setup

  • The server: a Scaleway Elastic Metal EM-I120E-NVMe, AMD EPYC 8024P, 64 GB of RAM, Debian 13. Not my VPS: Firecracker needs KVM, which a VPS rarely exposes, and 30 micro-VMs need 18 GiB.
  • systemd: each app is a service under its own account, sandboxed as shown below. sitesolide 0.13.0, the deploy tool I build, wrote the units; its dashboard, portal, analytics and monitor kept running, and count in the machine's fixed cost.
  • Docker: Docker 29.9.2 with Compose and its defaults, each app's database in a container of its own.
  • Micro-VMs: Kata Containers 4.3.1 with Firecracker 1.17.0, running the same Compose files: one micro-VM per container.
  • Everywhere: Caddy in front of every app, the same memory caps, worker counts and runtime versions, backups off. One setup at a time, with a reboot in between.
  • The apps: the kind I build. A docs site (Astro, static, about 300 pages), a wiki (Next.js and SQLite) and a CRM (Django, gunicorn and PostgreSQL), each written by a coding agent. Ten copies of each, each seeded with its own data: 10,000 pages per wiki; 20,000 contacts, 2,000 companies and 100,000 activities per CRM.
  • The traffic: user sessions, not one URL hit in a loop. On the CRM, a user signs in, filters the contacts, opens three to eight of them, logs an activity one time in four and exports the CSV one time in fifty. On the wiki: sign in, browse, one or two searches, an edit one time in five. On the docs site: three to ten pages, each with its stylesheet and pictures, as a browser fetches them. Four seconds of pause on average between requests, a new session on each app every 15 seconds. The agent that wrote each app also wrote its scenario, and k6, Grafana's load-testing tool, replays it from the same server.
  • A run: deploy the 10 docs sites, then the 10 wikis, then the 10 CRMs. After each batch, 3 minutes of that traffic on every app deployed, then 3 minutes of rest.
  • The measure: MemTotal - MemAvailable, the median over the rest. The cost per app is the memory with 30 apps, minus the memory with none, divided by 30.

The traffic is the point. An app that has just started holds almost nothing. One that has been used for a while holds its caches, its connections and its database buffers, and that's the memory you actually pay for.

Results

With all 30 apps deployed, the machine used 4.2 GiB with systemd, 4.8 GiB with Docker and 18.0 GiB with micro-VMs, each setup's own services included. Every request succeeded in all three setups, at the same latency: a p95 of 69 ms on the wiki and about 300 ms on the CRM.

Memory per copy, by app

docs, a static site

  1. systemd15 MiB
  2. Docker23 MiB
  3. Firecracker212 MiB

wiki, Next.js and SQLite

  1. systemd101 MiB
  2. Docker149 MiB
  3. Firecracker708 MiB

crm, Django and PostgreSQL

  1. systemd184 MiB
  2. Docker218 MiB
  3. Firecracker828 MiB

The docs site is the odd one. With systemd it has no process at all: Caddy serves its files. The container setups run it in an nginx container. Leave it out, and an app costs 143 MiB as a systemd service, 183 in Docker and 768 in a micro-VM.

Where the memory goes

I also summed the memory of every process (its PSS) with the 30 apps deployed:

systemdDockerMicro-VMs
The apps3,121 MiB3,148 MiB16,006 MiB
Helpers beside each containernone453 MiB995 MiB
Shared daemons137 MiB255 MiB135 MiB
  • systemd runs nothing beside an app: it sets the sandbox up when it starts the service, then the app runs alone. The daemons are Caddy and sitesolide's own services.
  • Docker keeps a containerd shim per container (40 of them, 294 MiB) and a docker-proxy per published port (30, 159 MiB), beside dockerd, containerd and Caddy.
  • In a micro-VM, the host sees a single Firecracker process, which holds the guest's kernel, its page cache and the app. Kata gives each VM 256 MiB plus the container's memory cap, and memory the guest has touched stays with it. Each container also keeps a Kata shim and a nerdctl process that collects its logs.

How systemd sandboxes a service

systemd is usually just the thing that starts services, but it can also lock them down. A unit file asks for the isolation; systemd builds it with the same kernel features containers use as it launches the process, then gets out of the way. Here is the unit of one CRM copy, trimmed:

[Service]
User=site-crm-01
ExecStart=/bin/sh /srv/sites/crm-01/app/run.sh

# Files: the system read-only, /srv emptied, the app's own folders put back
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
PrivateDevices=true
TemporaryFileSystem=/srv:ro
BindReadOnlyPaths=/srv/sites/crm-01/app
BindPaths=/srv/sites/crm-01/data
ProtectProc=invisible

# Privileges and system calls
NoNewPrivileges=true
RestrictNamespaces=true
SystemCallFilter=@system-service

# Network: the loopback only
IPAddressDeny=any
IPAddressAllow=localhost

# Memory
MemoryMax=512M
  • Files go through a mount namespace: the app sees a read-only system, an empty /srv holding only its own folders, its own /tmp, and no other account's processes.
  • System calls go through a seccomp filter: outside the @system-service set, a call fails with EPERM.
  • Network goes through an eBPF filter on the service's cgroup: the loopback only, where Caddy reaches the app.
  • Memory is capped by the cgroup, as Docker caps a container.

systemd-analyze security <unit> scores what a unit leaves open, directive by directive.

The trade-offs

systemdDockerMicro-VMs
Memory per app100 MiB130 MiB583 MiB
Disk for 30 apps6.6 GiB3.9 GiB6.1 GiB
An image per appnoyesyes
A kernel per appnonoyes
  • Disk. Every systemd copy carries its own dependencies, where Docker's ten copies share one image's layers.
  • Images. A container runs the same image anywhere. A systemd service runs on the host's own system and the runtimes installed on it.
  • The floor. With nothing deployed, the machine used 1.3 GiB in the systemd setup and 1.0 GiB under Docker, though its processes weighed less than Docker's daemons. I haven't explained that gap yet; it's in the raw data.
  • The kernel. systemd services and containers share the host's kernel, so a kernel flaw is a way out of either. A micro-VM puts a kernel of its own between the app and the host, and that's what its memory pays for.
  • Deploy times compare the tools rather than the isolation, so they stay in the raw data: Docker started images built beforehand, while sitesolide built each app on the server.

How I read it: for my own apps, a sandboxed systemd service isolates each one for the cost of the app alone. Docker adds about 30 MiB per app and gives you images. For code I didn't write, a micro-VM costs about six times the memory, and puts a kernel of its own between that code and the host.

Deploying them

Measuring the memory was the easy part. The rest of my list is what I built sitesolide for, and it deployed the systemd side of this benchmark. It's what every app needs around it, whatever isolates it: one command to deploy, and on the server:

  • HTTPS for every app, certificates included;
  • a code or a company sign-in in front of private apps: Google Workspace, Microsoft Entra, Okta or any OpenID Connect provider, with roles per project;
  • secrets on the server, never in git, set by developers who never read them back;
  • hourly backups with restic, and a monitor that checks every site each minute;
  • personal tokens for colleagues and agents instead of root SSH, --json everywhere, and an MCP server for Claude Code, Codex or Cursor.

Today it runs each app as a sandboxed systemd service. Next, each app will choose its isolation: a container, or a micro-VM for the code that needs one. It's one machine: no failover, and a deploy restarts the app.

Try it, or check the numbers

sitesolide is MIT licensed, on GitHub. Everything behind these numbers, the protocol, the apps, the scripts and the raw measurements, is in sitesolide-bench. One run per setup isn't a study: run it on your own hardware, tune Docker your way, and send me what you get.