How it works

The parts, what each one is allowed to do, and why the boundaries are drawn where they are.

One machine, on purpose. There is no staging environment and no automatic failover. What breaks production breaks every site at once, and nothing repairs itself. That constraint is what keeps the system small enough to understand completely, and it is why the deployment path verifies what it changed rather than assuming, and puts Caddy's configuration back when the new one fails. It does not keep the previous version of a project's code: see what a deploy costs.

The server is the source of truth. For a deployed project, its general access and its secrets are what the machine carries, not what the repository says. The CLI reads the machine, and refuses to contradict it.

Nothing in the repository names a machine. The server, the zone and the contact address have no defaults at all: the CLI stops and points at sitesolide init rather than guessing. That is what makes this repository publishable, and what makes your copy yours.

There is no registry, no database of projects, no control plane. A project exists because /srv/sites/<slug> exists on the machine, and it is served because Caddy's wildcard block resolves <slug>.<zone> to /srv/sites/<slug>/public by naming convention. Adding a project changes no shared configuration file.

A static project stops there: Caddy serves the files and nothing runs. An app gets two more files, both generated from its manifest, a systemd unit and a Caddy block, and costs 8 to 40 MB of memory.

A project that runs several processes, a front, the API it calls, a worker behind them, a job every night, declares them under services and gets one unit per service. The first that runs for good keeps the project's name, <slug>.service, and the others are <slug>.<name>.service, hanging on it: starting, stopping or restarting the main unit does the same to them, which is why the dashboard, the steward and the collector, which know a project by its main unit, need nothing more. The block sends each service its own paths.

Those two generated files live nowhere but on the machine. deploy generates them from the manifest each time, installs the unit, and hands the block to bin/deploy-caddy.sh, which validates it on the machine with the blocks already in service: a block is only correct in the company of the others, and the machine is where they all are. A deployment never touches another project's block.

A service is one of three kinds. A server listens on its port, for Caddy or for its siblings. A worker listens on nothing and runs for good: systemd restarts it when it falls, waiting longer after each failure, up to five minutes. A scheduled service runs at the times its schedule names, started by a timer of its own, <slug>.<name>.timer, and between two runs nothing runs. The main unit wants the timer, never the service, which would otherwise run at every start of the project. Timer and service both hang on the main unit, PartOf= it like every other, and systemd passes a restart on to them as a try-restart: restarting the project restarts the timers and a run in progress, and never starts a run that was not going. A project whose services are all scheduled has nothing that runs for good to name after itself: its main unit is then an anchor, which runs nothing, stays active once started, and wants the timers, so that a reboot, a deployment and the dashboard's restart reach them as they reach any project's services.

A project whose services are all internal, and that has no publicDir, serves no page. Its block answers its address with a line saying so, rather than nothing: every probe of the machine, the deployment's, the Caddy step's and the gatekeeper's, asks each project's address, and the portal and a preview code still close it. See docs/manifest.md.

Each project gets a system account, a directory under /srv/sites/<slug> and a systemd unit that confines it. /srv is replaced by an empty mount and only that project's own directories are bound back in, so a compromised project cannot read another one's files even if Unix permissions would have allowed it. An app that declares no secret has /etc/sitesolide made inaccessible, which hides even the names of the files.

Its system calls are those of systemd's @system-service set, what a Bun, Node, Python or Go server needs: none that mounts, loads a kernel module, reboots, swaps, sets the clock or traces another process. Any other fails with EPERM, as a call the account may not make, rather than killing the process. A program that needs one more call gets it from a drop-in on the machine, /etc/systemd/system/<slug>.service.d/syscalls.conf holding [Service] and SystemCallFilter=ptrace, say: systemd adds it to the set, and a deployment, which reads the unit's own file alone, leaves the drop-in where it is.

Services cannot reach each other over the loopback either, except those of one project, see the loopback rule.

The walls keep a project from reading its neighbours, not from slowing them down, so work in the background yields to the sites: a worker or a scheduled service gets a fifth of a site's share of the processor when they compete (CPUWeight=20 against 100), and 512 processes and threads at most. cpu caps any service outright, a share of one core.

This is the part worth reading twice. Each component has exactly one job and no more.

The dashboard reads nothing itself. Its unit replaces /srv with an empty mount, so it cannot open another project's files, and the loopback rule stops it reaching other services. What it displays comes from a snapshot file a root timer wrote for it. A compromised dashboard therefore leaks exactly what the page already showed.

Only the gatekeeper touches Caddy from the machine, and only for the one project named in its unit. Four unit templates, one per action, on (Restricted), off (Public), code (Anyone with the code) and renew (a new code), rather than one instance, so that each can only write into /srv/sites/%i, the blocks and the preview locks: a compromised gatekeeper holds the one site you named it for. The dashboard reaches it through the steward, and so does sitesolide lock, over the owner's SSH: one path changes a site's general access, whoever asks.

People with access are judged as root. Per project, each person or domain with access holds a role, each including the ones below it: Can open opens the site when its general access is restricted; Viewer also sees the project in the dashboard; Developer also deploys it, restarts it and writes its secrets without ever reading one back; Admin also reads its secrets, chooses its general access, gives and takes away access to it, a role at most their own, and restores its backups. The steward keeps them in one registry, /var/lib/sitesolide-steward/access.json, root's alone, judges every change by its rules, and writes after each one the projection the portal decides from, /etc/sitesolide-portal/access.json, which root writes and only the portal's account reads: a removal holds at the next request. Someone inside the company's domains signs in with their company account and may hold any role; someone outside them gets password access, Can open alone, a password the steward draws and shows once; a domain is Can open alone. The owner, whoever holds the dashboard's password, may do everything. See dashboard/README.md.

The dashboard believes no name it is told. Someone with a role above Can open signs in to it with their company account: the portal, the one authority on who someone is, signs an assertion of the email the provider verified, for the dashboard, for five minutes, once; the steward checks it with a public key it keeps itself before it opens the session, and judges every write the person asks for against its own registry, at that moment. The key pair is the steward's: it lays the private half where only the portal's account reads it, /etc/sitesolide-portal/assertion.key, beside the projection. A compromised dashboard acts only for the people whose sessions pass through it, within their roles, and raises nobody above Can open without an unlock. What a person sees, the dashboard filters from data it already holds.

A secret read, a site made public, opened with a code or given a new code, a restore, a role above Can open given, password access given, a token minted, each waits for the person's own unlock: a forced sign-in at the provider, which the portal asks for (prompt=login, max_age=0) and reads back in the provider's own ID token, and the steward checks again, for ten minutes and that person's session alone. The owner's unlock is the dashboard's password, retyped. Password access waits because it lets in someone from outside the company, whom no company account vouches for. Giving Can open to a company account or to one of the company's listed domains, restricting a site, and taking access away, never wait for either: closing someone out must not wait for a password. A whole domain waits while the company's domains are not listed, since anyone the provider vouches for would then come in. The steward never hands a Developer a value, whatever the dashboard relays. A token minted with Manage secrets writes without an unlock, and that is the opt-in: minting it took its maker's unlock, and ticking the permission is the decision to let whoever holds the token set the variables of <slug>.env until it expires or is revoked. It never reads a value back, whoever made it, and the steward bounds it per hour, 120 changes, 30 restarts and 60 refused requests, its refused changes of access among them, every change and every refusal within the bound journaled under the token. What a token lets in stops once the access log, which keeps every change of access 180 days, is half full: the other half stays the people's.

A person's own tokens are never stronger than the person. A Developer mints a token that deploys their project, an Admin one that may also deploy public sites, declare a domain or reach outside hosts, and someone the owner granted the right to create projects one that creates them, which makes them Admin of what it creates. The steward reads the registry at every use of such a token, and again when the installer starts: a role lowered to Viewer stops that project's deployments at the next request, and someone who no longer signs in to the dashboard takes their tokens with them. A token gives Can open alone, to the company's people and domains, never password access. A Developer's token may also manage secrets, Manage secrets in the dialog: it lists variable names and sets and removes the variables of <slug>.env, on each project while they are Developer or Admin there, and any token that deploys a project restarts it. See dashboard/README.md and access.md.

Everything that reloads Caddy shares one lock, /run/sitesolide-gatekeeper/caddy.lock. The CLI, the deploy scripts and the gatekeeper all take it. Without it, a site made restricted from the dashboard between a read and a write was silently overwritten, and served in the clear.

An nftables rule reserves ports 3000 to 3099, and Caddy's admin API on 2019, to Caddy and root. That API authenticates nobody: a service that could reach it would open every restricted site at once, or stop Caddy.

This is what makes a restricted project able to trust Caddy. It has no sign-in of its own; it trusts the header Caddy puts on the request, and that trust is only founded because nothing else on the machine can forge it.

The rule judges who connects, never who listens. A project's account may bind any port left free, and Caddy, which reaches the whole range, connects to whoever listens there: an app waiting on the portal's port while the portal restarts would answer every forward_auth as it liked, and receive the owner's password typed on a sign-in page. So each generated unit says where its service may listen (SocketBindAllow=, SocketBindDeny=): a server on its own port alone among 2019, 3000 to 3099 and the egress proxy's 3128 and 3129, a worker and a scheduled service on none of them, a service's backup command neither; a bind refused fails with EPERM. Ports outside those stay free: nobody is sent there. systemd enforces the lines through BPF, and a systemd built without +BPF_FRAMEWORK ignores them silently.

One exception, deliberate: the dashboard may reach the portal, where it has the sign-ins of people with a company account sealed and redeemed, reads how people sign in, and reads the portal's audit. Those routes have no other guard than this rule, and a test refuses any fragment that would expose them. Root reaches them too, with one question about access and nothing else: the steward asks GET /admin/access, whether the portal decides from its projection, through sitesolide-portal-relay, a systemd proxy that forwards a socket only root opens to the portal's port and to nothing else, so that the steward's own unit keeps no network at all. Nobody changes who may open a site through the portal any more: the steward writes the projection the portal reads, and the routes that did it, PUT /admin/sharing/:host, /admin/guests and DELETE /admin/invites/:id, answer 410 moved to whoever asks.

And one set: a project that declares several services reaches its own ports, and nobody else's. Its front calls its API, its worker calls both, and a neighbour still reaches none of them. The set holds port . uid pairs, rebuilt by deploy from the manifests on the machine into /etc/sitesolide-loopback-projects.nft, and replayed after the table at boot. A worker and a scheduled service declare no port, and need no pair of their own: they run as the project's account, which its servers' pairs already let through. A project with a single service does not appear in the set, nor does one whose services all listen on nothing: it has nothing of its own to call.

A service reaches the loopback and nothing else, DNS included, unless its manifest says otherwise. "network": "outbound" lifts that for everything; egress lists the hosts it may reach instead, and keeps the rest closed.

The unit of a project with egress still refuses every address but the loopback. Its HTTP clients are pointed at the egress proxy, on 127.0.0.1:3128, outside the range the loopback rule closes, which lets the listed hosts through. The proxy runs as its own account and decides on three readings:

• who calls, from the kernel: the uid of the caller's socket in /proc/net/tcp, turned into site-<slug>. Nothing the caller sends can say otherwise;
• what that project may reach, from its manifest on the machine, which the project's own service cannot rewrite;
• where the host really points: every address it resolves to is checked, and one inside the machine, a private network or the cloud's metadata service refuses it.

Connectors go one step further: credentials an administrator defines in the dashboard, slack, github, and grants to a project. The app calls the proxy in plain HTTP on the loopback, and the proxy forwards over HTTPS with the credential added. A project gets one only when its manifest asks for it and the dashboard granted it: the manifest is written by whoever wrote the app, and must not be able to grant itself a company's credential.

What it does not stop matters as much: a project can still send whatever it likes to a host on its own list. The list is the boundary. See egress/README.md for the threat model.

Two paths, and they must not be confused.

The zone gets one wildcard certificate over DNS-01. That is the only way to get a wildcard, and it needs a token that can edit the zone. Every project under the zone shares that certificate. A block that diverged on TLS policy would silently get its own certificate instead, so they all import the same snippet.

A customer domain gets its own certificate over HTTP-01, issued on first request. The token on the machine cannot edit someone else's zone, and the customer usually keeps their DNS elsewhere. An app's block claims every name the table carries for its domain, the www of a second level domain and the aliases, and sends them to the domain itself: a name no block claimed fell into the nameless block of static files, which serves public/ and never wakes the service. That block now refuses an app's folder outright, with a 404: a name the table still carries and the app's block does not claim would otherwise serve its files past its portal.

On-demand issuance is guarded by the ask endpoint: Caddy asks the shared service, during the TLS handshake, whether a domain is allowed. It answers yes only for domains in the table. Without that endpoint, anyone pointing DNS at the machine would trigger issuances in your name, and those are counted.

That endpoint is why api/ exists and why it stays tiny. A mistake in it takes down certificate issuance for everyone at once.

The machine is the source of truth. /etc/sitesolide holds the environment files, and they are read, placed and replaced from the dashboard's Secrets section, which also restarts the service.

A service never reads its own secret file. systemd reads it as root, before dropping privileges, and hands the variables to the process. The file stays 0600, owned by an account the service does not have.

deploy never pushes a secret. It checks that each file the manifest declares is the project's, <slug>.env or <slug>-<name>, never the machine's own nor another site's, and, when it is on the machine, that it belongs to site-<slug>: the unit reads every declared file as root, so a file declared is a file handed to whoever wrote the manifest. When one is missing, it stops and says where to create it in the dashboard. Where a file lands, whom it belongs to and under which mode follow from its name and its site. See secrets.md.

Two guard rails do not bend. Any PASSWORD_HASH, in any file, changes only through Change password: the hash is never read back and never restored. And dashboard.env is owned by root, so the dashboard's own service cannot rewrite the hash that unlocks the secrets.

Once installed, the machine snapshots every app's data folder every hour into a restic repository, keeps a day of hourly snapshots, a week of daily ones and a month of weekly ones, and can copy them, encrypted on the machine, to a second repository in a bucket elsewhere, which stock restic reads back on any machine. A snapshot is restored one project at a time from the dashboard's Backups section, which saves the current data first so that a restore can itself be undone, and puts it back if the service does not come back on the restored data. Two sites are restored by hand only: the dashboard, which would cut off the page doing it, and the portal, whose old copy would bring back its old audit and the tables it kept before the access registry.

The copy to a bucket is off until you configure one. Until then the snapshots stay on the machine's own disk, and a machine lost takes them with it. Turning it on takes a bucket, an access key for it, and a passphrase kept somewhere else, the only way to read the copy back: see dashboard/src/backup/README.md.

The boundary is the same as everywhere: root never opens a project's file, nor walks its folder. The copy, which measures the data too, and the extraction run as the project's own account, in a transient unit with its service's walls and no network, each within its own time; restic, run by root, stores what the copy streams, in a repository no project can read. A manifest opts out with "backup": false. A service that keeps a server database in the data, PostgreSQL for one, declares a backup command that leaves a consistent copy of it before the copy, which archives that copy instead of the live files; a running server no service declares fails the snapshot rather than being copied file by file. See docs/manifest.md and dashboard/src/backup/README.md.

setup turns on Debian's unattended-upgrades, which installs the security updates every day. Bun and Caddy, which come from outside Debian's security archive, are not among them.

Nothing reboots the machine. A new kernel is installed and waits: the machine keeps running the old one, with the hole the update closed, until someone reboots it, and so does every process started before an update that replaced one of its libraries. A reboot stops every site at once, for as long as the machine takes to boot, and when that happens is the owner's choice, not setup's.

What the owner does. unattended-upgrades writes /run/reboot-required after a kernel update, and /run/reboot-required.pkgs says which packages asked for it. When the file is there, reboot at a quiet hour:

ssh you@your-machine 'cat /run/reboot-required.pkgs 2>/dev/null; uname -r'
ssh you@your-machine 'sudo systemctl reboot'

Everything setup and the deployments installed is enabled and starts again on its own: Caddy, the platform's services, each project's units and timers, the loopback rule. The monitor's next pass says whether every site answers.

To have it done unattended instead, at a fixed hour and only when an update asks for it, tell unattended-upgrades so in a file of your own, which setup never rewrites:

ssh you@your-machine "printf '%s\n' 'Unattended-Upgrade::Automatic-Reboot \"true\";' 'Unattended-Upgrade::Automatic-Reboot-Time \"04:00\";' | sudo tee /etc/apt/apt.conf.d/60reboot"

Two different things, often confused.

A preview lock closes one site behind a six-character code, for showing work to a client before launch: its general access is then Anyone with the code. The code is drawn on the machine by the gatekeeper and lives there, never in the repository, in a codes file and in the fragment Caddy compares it from, /etc/caddy/locks/verrous.caddy, which only root and Caddy's group read: a project's service sees /etc/caddy. The owner and the site's Admins read it in its Access section, with the link that carries it, and draw a new one there. Every URL of the locked host is rewritten to a code page that stands on its own, inline CSS, inline icon, no external request, because anything it asked for would come back as HTML.

A site closes on every address it answers on. Its own domain answers as its preview does, the portal's guard or the code alike, and the domain's other names, its aliases and its www, send there. Every block that serves a site's content off its preview says whose site it is: the domain's block in its fragment, and, for a static site, Caddy's nameless block from the domain table. The preview locks and the portal read that, never the domain's name, so a domain switched to later closes with the site the moment it is served.

The portal stands in front of every restricted site, and Caddy consults it with forward_auth before every request. A project's general access is one of three: Public, anyone opens it; Restricted, only its people with access, the portal in front; Anyone with the code, the preview lock above. Who opens a restricted site is its people with access: people signing in with their company account, Google Workspace, Microsoft Entra or any OpenID Connect provider, named by email or as a whole domain, each with a role; people outside the company with password access, one password per person and per site, drawn by the steward, shown once, until an expiry chosen when it is given; and everywhere, the owner with the owner's password, and the admin emails. They are given from a site's Access in the dashboard, or with sitesolide share from a project's folder, over the owner's SSH, or with a token through the dashboard and the steward, which lets a token give Can open alone, to the company's people and domains. Every change goes to the steward's registry and its projection, never to Caddy, and holds from the next request; changing the general access, among the three, is what goes through the gatekeeper, a switch between Restricted and the code in one transaction.

A restricted project writes no sign-in of its own. It trusts Caddy, which is sound only because of the loopback rule. It also learns who came in, from three request headers, X-Sitesolide-User, X-Sitesolide-User-Name and X-Sitesolide-Role: the role on the project, from visitor (Can open) to admin, and admin for the owner's password and the admin emails; password access comes as visitor with no X-Sitesolide-User, the email having been typed by whoever gave it, not verified by a provider. Its block takes the visitor's own headers off every request before copying the portal's on, so a site can trust them once its block has been deployed with them. A public site has them taken off too, and learns nobody. See portal/README.md.

Every component that acts records what it did in its own database, in one shape: an ISO date, an actor, a dotted action, a target, and a detail in JSON that never carries a secret value. An actor is an email, of someone with access to a project or of someone given password access, owner for whoever holds the dashboard's password, token:<id> for a token, password:<id> for a password access given under a name rather than an email (guest:<id> on the rows written before the access registry), anonymous before anyone is known, or system. The steward writes a person's email as it verified it, never as a request names it. A deployment by a person's own token is recorded under token:<id>, the person's email in its detail's member field, a name the rows keep; so is a variable a token sets or removes, or a restart it makes, in the steward's journal, the member field empty for an owner's token. Nothing gathers these tables on the machine: the dashboard's Activity page reads each through the road it already takes to that component, and merges them, newest first. How long each keeps its rows is in dashboard/README.md.

Changes of access are the steward's to record: the portal, which no longer keeps a list of who may open a site, records sign-ins and sign-outs alone. The rows it wrote before the access registry, sharing.update, guest.create and guest.revoke, still read, as do the steward's member.* rows of before, which its access.*, people.create and dashboard.* events replace; their action names stay as they were written, and the Activity page says them in today's words. The steward keeps every accepted change of access 180 days, in a file of its own beside its journal, which rotates by line count: a refusal stays in the journal.

The steward's rows say how each operation ended, ok, rejects or failure, in their detail's result. Their source is the steward's journal, which has no ids: the dashboard gives them the shared shape as it reads them. A line of the journal written before it named its actor reads as owner, the only one it acted for then.

The Activity page shows the owner the whole machine; a person, the rows of their projects and their own. Each row reads as a sentence, "Gave dana@example.com Developer on cms", and Access changes narrows it to the changes of access.

In order, and the order is the point:

1. Read the manifest, refuse it if anything is wrong. Nothing has been sent yet.
2. Read the machine: does the dashboard say this project is restricted? Is the block in service one the generator would write, from this manifest or from the one the machine carries, or one edited by hand there, which stops everything without --force? Does systemd already run a service of that name that deploy did not write? Does another project already declare one of its ports? For a project with several services, does the loopback rule have room for its own ports? For a project with egress or connectors, does the egress proxy run? For a project with scheduled services, does systemd read each schedule as a time?
3. Build locally.
4. Create the system account and the directories.
5. Install the systemd units that are missing, timers included, remove those of services and schedules the manifest no longer declares, timers first.
6. Upload the code, then the public files.
7. Take the Caddy lock, read the general access and every port again under it, then put down the manifest, and rebuild the loopback's project set when the manifests say something other than what it carries.
8. Run install as the project's account, in its service's walls.
9. Restart the project's units, its timers among them but never a scheduled service, which its timer starts; check each one is active, and write the block.
10. Verify over HTTPS that the site answers, and, ten seconds at least after the restart, that every service that runs for good is still active and was not restarted by systemd meanwhile: a process that falls at once passes the first check.

For a restricted project, the portal goes in front before its files, and comes off after them. Placed the other way around, its files would be served in the clear by the wildcard block for the length of the deployment, and indefinitely if the command was interrupted.

An app is replaced in place, with a gap. There is no second copy started beside the first and no switch of traffic: step 6 sends the new code over the folder the running process was started from, step 8 runs install in that same folder, and step 9 restarts the service. From the restart until the new process listens on its port, Caddy finds nobody there and answers a 502: a visitor who arrives in that gap gets an error page, for as long as the app takes to start. Between steps 6 and 9, the old process still serves, with the new files under it: one that loads code lazily may load a file of the new version. If no visitor may see that, deploy at a quiet hour.

There is no automatic rollback of the code. The rsync deletes what the new version no longer has, and nothing keeps the previous one on the machine. A service that does not start fails the deployment at step 9, and one that falls within ten seconds fails it at step 10, each saying so; the new code stays in place either way, and the site answers 502 until a deployment that works. The way back is the previous commit, deployed from your checkout. The one thing a deployment puts back by itself is Caddy's configuration: a block Caddy refuses, or a reload that fails its check, brings back the previous one. A deployment never touches the data folder, but what the new version writes there as it starts, a schema migrated, stays after a redeploy of the old one.

A static site has neither cost: its files are sent in place and nothing restarts. A visitor during the transfer may get a page of the old version and a file of the new.

--dry-run prints the generated unit and block, and every command it would run, without touching anything, nor running the build, which is the folder's own code; --dry-run --build runs it too.

Someone with a token deploys over HTTPS instead of SSH, and the same order runs on the machine, in the installer, with the same decisions: see dashboard/README.md, "The control API", and access.md.

• docs/install.md, from nothing to a first deployment
• docs/manifest.md, every key, and what it changes
• docs/commands.md, what the CLI does
• infra/README.md, the machine itself
• dashboard/README.md, the dashboard, the steward and the gatekeeper
• dashboard/src/backup/README.md, the data snapshots and their restore
• portal/README.md, the owner's password, password access, signing in with a company account, the projection it decides from and the identity headers
• analytics/README.md, how a visit is counted, and why it is anonymous
• monitor/README.md, what is checked every minute, and who hears of it
• egress/README.md, the hosts a project may reach, and the credentials it is lent