On-premise deployment

Applies to: SimpleTwo 0.9.x · Checked: 20.09.2026

By the end of this article you have an installation where an employee signs in with their corporate account, exchanges messages, places calls, holds a meeting with an external participant by link, and reaches the corporate network through a secure tunnel. The order of the steps is not arbitrary: each service is configured against the previous one, and some steps refuse to run when the previous one is not ready.

Not covered here: cloud delivery, an air-gapped installation, telephony and conference recording — they are optional and have articles of their own.

What is installed by hand, and what installs itself

Exactly one service is installed by hand: coordinator. Every other role (the database, chat, the directory, SFU, TURN, VPN nodes and the rest) it installs through its agent on each host. The agent goes onto a host one of two ways:

• yourself (pull) — run the console's join command on the host. The host needs no inbound SSH: the agent calls the coordinator;
• through the coordinator (push over SSH) — give the host's address, an SSH user and a key: the coordinator logs in once and runs the same command for you; the key is not stored.

From there it is the same: the agent fetches its role's desired state, binary, configuration and certificate from the coordinator, installs and updates the service and reports what it did. The binaries of every role, and Xray, are already inside the coordinator's image — nothing else has to be downloaded.

1. What you are deploying

One private network, with every access rule inside it written down explicitly. The hosts share a private network, and between them there are only the rules the control plane writes itself:

• the database accepts two hosts only — messaging and directory, each as a /32 in pg_hba;
• the bus accepts only those that use it — the sfu, mcu and recorder nodes — by a firewall rule on its own host, plus a password.

Both rules are re-applied at every state poll, so neither has to be remembered or maintained by hand.

                              internet
                                 │
     ┌──────────────┬─────────────┼──────────────┬───────────────┐
     │              │             │              │               │
 coordinator    messaging     sfu ×N         turn           VPN nodes
  (80/443)    chat.s2.<domain> calls.s2.<domain>  3478 +       (gateway)
     │              │             │          49152–65535    by IP, 443/tcp
     └──────────────┴─────────────┴──────────────┴───────────────┘
                 one private network 10.x.x.x
                              │
              ┌───────────────┼────────────────┐
         postgres          directory         redis
      only the /32 of     its own database   only the addresses
      messaging and directory  on that postgres  that use the bus

Role	Public	On the private network	What for
coordinator	yes, 80 and 443	yes	the control plane: sign-in through your identity provider, policy, the node registry, the console, the product options, certificate issuance, installing every other role
postgres	no	yes	the chat, directory and calendar database; unreachable from outside, and inside it accepts only the messaging and directory hosts, each as a /32 in pg_hba
messaging	yes	yes	chat, threads, files, call signalling; the product server
directory	yes	yes	the directory and sign-in (identity) and the calendars, ICS subscriptions, CalDAV — one service since 08.10.2026. Public because its readers are not our client: a subscribed Outlook refreshes on its own, a CalDAV client polls. Keeps its own database on the same postgres
s3	no	yes	object storage: attachments, avatars and recordings. An object here is an ordinary file, so a backup is filesystem work. It accepts only the messaging and recorder hosts, each as a /32 on port 9000. The role is optional: without it attachments live on the messaging disk — one host, one backup and no second replica. Instead of the role you can connect the organisation's own storage
redis	no	yes	the coordination bus. Needed from the second sfu node and from the second mcu node. Its only boundary is a firewall rule, so ufw or firewalld is mandatory on the bus host
sfu	yes	yes	the media server: conference media and call signalling. It forwards streams, it does not mix
turn	yes	—	a relay for clients behind strict NAT
gateway	yes, 443/tcp	a DMZ for corporate egress	VPN nodes (VLESS+Reality), one per location
recorder	no	yes	conference recording (optional). The only role that needs Docker: it runs the ready-made livekit/egress image under systemd rather than a binary we build. Installed after redis and at least one sfu — it takes jobs from the bus and joins the room as an ordinary participant. Storage credentials never reach its disk: the bucket and the keys arrive with each recording request
mcu	yes	yes	meeting-room terminals over SIP (your Poly, Cisco devices). Not the PSTN gateway: it has no carrier trunks — that is the pstn role
pstn	yes, SIP 5060 and RTP towards the carriers' trunks	yes	the PSTN gateway: a carrier's SIP trunks, dialling into a meeting and calling telephone numbers. A host of its own, never shared with mcu; needs redis, messaging and sfu
support	no	a DMZ beside the coordinator	the support bot daemon (optional)

2. What you need beforehand

Hosts and sizes to start with

One table for the whole installation: what is installed, how much of everything, on which network and what limits it. The numbers are a starting point for an organisation of up to ~150 people (size S in the product's load profile); for larger organisations see "How many nodes for calls" — only media scales there.

Every host is x86-64 (amd64). The server side is built for this architecture only: the join script will not take an ARM (aarch64) host and says "only x86-64 is supported" before it installs the agent. That includes the FreeBSD storage host.

Role	vCPU / RAM / disk	Network	Where	Limited by
coordinator	2 / 4 / 20 GB	100 Mbit/s, public IPv4	public	nothing; it holds SQLite and certificates in a volume
messaging	2 / 4 / 100 GB SSD	100 Mbit/s, public IPv4	public	disk, while attachments live on it: with the s3 role or external storage, 20 GB is enough
s3	2 / 4 / disk by media	1 Gbit/s inside the installation	private	disk — see below; optional, unnecessary with external storage
sfu, each	4 / 8 / 20 GB	1 Gbit/s, public IPv4	public plus a leg into the private network (redis)	bandwidth — see below
turn	2 / 2 / 20 GB	1 Gbit/s, public IPv4	public	bandwidth: around 15% of participants go through it, in both directions
gateway, each	1 / 1 / 20 GB	1 Gbit/s, public IPv4	public	bandwidth
directory	2 / 4 / 20 GB	100 Mbit/s, public IPv4	public	polling rather than people: a subscribed Outlook and a CalDAV client arrive on their own schedule
postgres	2 / 4 / 40 GB SSD	1 Gbit/s inside	private	disk; unnecessary with an external cluster
redis	1 / 1 / 10 GB	1 Gbit/s inside	private	nothing; it holds room routing, not data. Unnecessary with an external one
support	2 / 4 / 20 GB	100 Mbit/s inside	private (DMZ)	memory — see below; optional

How much disk for object storage

Counted by media, not by people. Per person per year, by the load profile and the current retention defaults:

	Assumption	Per person per year
attachments	8 files per working day × 250 days, 400 KB on average	≈ 0.8 GB
thumbnails	one per image, ~40 KB, half the files are images	≈ 0.04 GB
avatars	one per person, changed a few times a year	negligible
meeting recordings	not included: an hour of a room at 720p is about 0.9 GB, and that is the organisation's policy

So ≈ 1 TB per 1000 people per year of messaging files, twice that with regular meeting recording. The role is disk-bound: neither CPU nor network is the limit at this size. Storage is the second stateful host after the database, and it has to be in the backup.

A live example to check against: a 200 GB disk yields 193 GiB for objects (ZFS takes the rest for metadata and reserve) — about 230 person-years of messaging, so a 200-person organisation fills it in its second year. Size it for the retention period rather than for today: a disk in the pool is grown by replacing it with a bigger one (zpool replace), which is a data-copying operation rather than a settings change.

Storage on FreeBSD with ZFS

The recommended shape for production, and the reason the engine was chosen: versitygw is an S3 gateway in front of a filesystem, an object is a file and its metadata lives in extended attributes. Every ZFS property works at the object level: a snapshot is a consistent copy of every attachment, zfs send is a copy at a second site, a scrub catches bit rot, and a single attachment can be retrieved from .zfs/snapshot without our software at all.

A FreeBSD host joins the platform like any other: the agent is built for FreeBSD (x86-64) too, the coordinator hands it the right binary, and the service is installed through rc.d and daemon(8). The platform now partitions the object disk itself — nothing has to be prepared by hand before joining.

1. The machine. Two disks: a boot disk and a second, empty one for objects. Size the second from the table above. Empty is not a formality: the agent will only take a disk that is not mounted, not partitioned and not part of a pool (see below).

2. Joining. In the console: Servers and deployment → Add host, role s3, the host's private address and the data disk — the device name as the host itself calls it (vtbd1, da1, nvd0). The private address is mandatory: storage listens on exactly that, and it cannot listen on a hostname — the service simply will not come up. Run the command from the card on the host as root. It detects the system itself (uname -s), takes the FreeBSD agent and installs the simpletwo_agent service. If the coordinator is running with a self-signed certificate, the host needs curl (pkg install -y curl): the base fetch cannot pin a key, and instead of a silent unprotected download the command says so honestly.

The disk can be named later too — Servers → Configure on the host. The field shows what the host reports about its disks: name, size and whether it is free, so the device name does not have to be guessed.

3. What the platform does next. It creates the simpletwo pool on that disk and the simpletwo/s3 dataset with xattr=sa, recordsize=1M, atime=off, compression=lz4, mounts it at /var/db/simpletwo-s3 and puts data, iam and versions there. xattr=sa is mandatory: versitygw keeps an object's metadata in extended attributes, and without sa each attribute becomes a separate hidden file. Then it installs versitygw, writes /usr/local/etc/simpletwo-s3.env (the keys live only there, never in the command line), an rc.d script using daemon(8), and a pf rule: port 9000 is open only to the messaging and recorder hosts, each as a /32. On a host without pfctl the deployment fails rather than leaving the store open to the whole private network.

If the disk field is left empty, objects land on the boot disk. The platform will do that and log it — but such a store has no snapshot-as-backup, no zfs send, and no way to recover one attachment from .zfs/snapshot.

4. Moving what has already accumulated. If attachments used to live on the messaging host's disk, they stay there: enabling storage moves nothing. The move is one command on the messaging host, with the service's own environment:

set -a; . /etc/simpletwo/messaging.env; set +a
/usr/local/bin/simpletwo-messaging import-media -n   # what would be copied
/usr/local/bin/simpletwo-messaging import-media      # the move itself

It can be run as many times as you like and at any moment, including after storage is already enabled. An object already in the store and of the same length is skipped, so a second run that copied nothing is the sign that the move is finished (there is deliberately no "migrated" flag: a flag can lie, a second run cannot). Nothing is deleted: the disk stays the source until you clear it yourself. If storage was enabled before the move, old attachments answer 404 — they have not gone anywhere, they are on messaging, and the import puts them where the service is already looking.

When the platform refuses a disk

zpool create is the one platform operation that destroys data, so the agent refuses at any doubt and never passes -f. It will not touch a disk that is mounted, partitioned (gpart) or already in a pool, and it will not accept a name that does not look like a device. It will not mount the dataset over a directory that already holds objects: they would not be lost but hidden, which is worse, because nobody would be told. And it refuses if a simpletwo pool already exists on the host but is built from a different disk: that means the field was changed, and moving data is zfs send, not a settings edit.

Every refusal is visible in the console on the host's row, with the reason and what to do.

Roles and operating systems

On FreeBSD the platform holds only the s3 role. The other roles are systemd services, apt packages and Linux binaries: an invitation to such a role is refused for a FreeBSD host at join time, with the reason. Storage you already have is still connected as external — Settings → External stores.

3. Step 1. The coordinator

From the registry — the whole installation, copyable line by line:

mkdir -p /opt/simpletwo && cd /opt/simpletwo
curl -fsSL https://docs.simpletwo.ru/deploy/docker-compose.coordinator.yml -o docker-compose.yml

REGISTRY=gitlab.itglobal.com:5050
IMAGE=$REGISTRY/simpletwo/platform/coordinator
echo "COORDINATOR_IMAGE=$IMAGE:v0.9.216" > .env    # for the current version see below

docker login $REGISTRY                             # GitLab login, password is a token with read_registry
docker compose pull && docker compose up -d

Which version is current is something to ask rather than guess — and to ask the repository tags, not the registry: the registry holds branch images tagged with commit hashes beside the versions, there are hundreds of them and they come first, so the first page of a registry query returns nothing but hashes and not a single version. Repository tags are short and sorted:

curl -fsS -H "PRIVATE-TOKEN: <token>" \
  "https://gitlab.itglobal.com/api/v4/projects/2081/repository/tags?search=^v0.9&per_page=3" |
  grep -o '"name":"[^"]*"'

Every such tag has an image with the same name: the image build runs on the v* tag.

There is a latest tag too, but write a version into .env: then it is visible what is deployed, and docker compose pull will not pull a new core at an inconvenient moment.

From an archive — when there is no registry access:

mkdir -p /opt/simpletwo && cd /opt/simpletwo
docker load < coordinator-v0.9.214.tar        # prints the loaded image's name
curl -fsSL https://docs.simpletwo.ru/deploy/docker-compose.coordinator.yml -o docker-compose.yml
echo "COORDINATOR_IMAGE=<the name from docker load>" > .env
docker compose up -d                          # no pull: the image is already local

There is not a single secret in the compose file, and that is deliberate: the coordinator generates its own secrets (the token signing secret, the node token, the administrator account) at first start and keeps them in the coordinator-data volume. Ports are 80:8080 and 443:8443 — TLS is terminated inside the coordinator and no external reverse proxy is needed.

Check:

docker compose ps                       # healthy
curl -fsS http://127.0.0.1/healthz      # 200

Take the one-time bootstrap token — it is printed to the log at first start and kept in the volume:

docker compose logs coordinator | grep -i bootstrap
docker compose exec coordinator cat /data/bootstrap-token

Open http://<address>/admin, paste the token, set the administrator password and the public URL (https://s2.<domain>). Right after that the console asks you to set up a second factor — an authenticator app (Google Authenticator, Microsoft Authenticator and the like): add the secret and enter the code. The console does not open without it — the server enforces this, not just the page. From then on sign-in is the password plus the app's code; the token stops working.

The volume is your whole installation

coordinator-data holds the secrets, the control-plane database and the certificates. Losing it means reinstalling the installation and re-enrolling every node. It is the first item in the backup.

4. Step 2. The coordinator's name and certificate

The coordinator issues its own Let's Encrypt certificate, but does not start doing so until it has been given a name. Until then it answers with a self-signed certificate — the browser complains, and the console shows self-signed in the Source row.

1. Create an A record s2.<domain> pointing at the coordinator host's public address and wait for it to propagate: dig +short s2.<domain> must return that address.

2. Give it the name — either of two equivalent ways.

   By environment variable (simpler for an installation assembled from the compose file): set ST_TLS_DOMAIN=s2.<domain> in deploy/docker-compose.coordinator.yml and, if you want expiry warnings, ST_TLS_EMAIL=<address>. Restart the coordinator — issuance starts at boot.

   In the console: Settings → TLS / Certificate → the Let's Encrypt (auto) card → Domain = s2.<domain>, Contact email if you want one → the Enable Let's Encrypt button.

3. If the installation has several names (white-label), add them under Tenant domains in the same card. A certificate for each is issued on first request — but only after step 2: the list by itself issues nothing, it only permits names.

4. Check from outside, not from the host itself:

curl -fsS https://s2.<domain>/healthz
curl -fsS https://s2.<domain>/.well-known/simpletwo.json

About the email

The address is optional. Let's Encrypt issues a certificate without a contact; the difference is that when renewal fails there is nobody to warn — the "your certificate expires in 20 days" message goes exactly there. Use the operations team's shared mailbox, not a personal one.

When issuance fails

The host must be reachable from the internet on 80 and 443 at the moment of issuance — that is how both validation methods work. If the DNS record has not propagated yet, issuance fails; that is not a misconfiguration, retry later with Renew (Let's Encrypt).

5. Step 3. The roles, strictly in order

Every role is installed from the console: Servers & deploys → Add a host. Choose the role, the GEO and the role's parameters, including the private address — the one the other hosts see it by — and press Generate join command. Run the command on the new host as root; or fill in Run it for me on (the host's address), SSH user and the key, and the coordinator runs it itself, once. The command is single-use and valid for an hour. Progress is visible in the Provisioning log, and the status goes pending → deploying → waiting → active.

The bus host needs a firewall

redis is protected by a password and by a rule allowing 6379 only to those that use the bus — the sfu, mcu and recorder nodes. On one private network that rule is the bus's entire boundary, so on a host with neither ufw nor firewalld the installation refuses rather than bringing up a database reachable by the whole network. The control plane maintains the address list itself: add an SFU node, telephony or recording and the rule grows at the next state poll.

The order:

1) postgres → 2) messaging → 3) directory → reinstall postgres once → 4) redis → 5) sfu node A → 6) sfu node B → 7) turn

Why postgres is reinstalled: the database's access rules narrow to the /32 addresses of the messaging and directory hosts, and they cannot be written while those hosts do not exist. One reinstall closes both.

The one rule that silently breaks calls

A private address must not be handed to clients. The host's private address goes into the private address field; the address clients reach the SFU by goes into node IP. Swap them and the call will establish, media will not flow, the connection will time out — and nothing will appear in any log.

Two more things the product will not let you get wrong:

• A second sfu node cannot be installed without redis — the installation refuses. Two nodes without the bus keep separate room tables: two people "in one meeting" on different nodes cannot see each other, and nobody is told.
• The calendar comes with the directory role. While there is no such host, clients hide the calendar screens rather than showing buttons that do not work.

Chat: name and certificate

After installing messaging, open the Messaging TLS card: set the hostname chat.s2.<domain> → Save config → Redeploy → Issue. Here the certificate is obtained by the service itself — over TLS-ALPN-01 on :443, without port 80 and without the coordinator — and the address https://chat.s2.<domain> reaches discovery automatically. The calendar works the same way. To rehearse the whole thing there is a staging flag: a Let's Encrypt test certificate that spends no rate limit.

Calls: the address for clients

The address for clients is the sfu node's public name (calls.s2.<domain>), given when it is deployed or in its settings on the Servers & deploys tab; there is no separate "Client URL" field — each SFU node hands clients its own address. The certificate's name is taken from it. The certificate for sfu is ordered by the coordinator, which sends it to the host together with the desired state; the same goes for turn and for an mcu node with a public name — its certificate serves SIP over TLS on 5061.

What the role's host needs for that:

• the name from Client URL points at this host (an A record);
• port 80 is free and reachable from the internet: the agent serves the domain-validation challenge on it. Occupied by another server, and installing the role says so plainly;
• nothing else: the certificate authority's contact address is shared across the installation (the one given to the coordinator) and is optional.

The state is visible in the Installed servers table: each host's certificate row says issuing (an order is in flight), a validation error, or the expiry date. A ready certificate can also be brought in — Import on the same row; an imported one is never reissued by the coordinator.

If the role already has a TLS front

Your own nginx or load balancer in front of sfu is still allowed. Then the name in Client URL belongs to the front rather than to the role's host, and the front holds the certificate — the coordinator does not know the role host's name in that case and orders nothing.

The check after each role

curl -fsS https://chat.s2.<domain>/healthz     # messaging
curl -fsS https://s2.<domain>/healthz          # the coordinator sees the roles' state

Every installed role has a status row in the console: the coordinator polls the roles' /healthz itself. A role in active with a red state is an installed service that does not work; look at Host logs.

Another role on the same host

One machine can carry several roles — of different platforms too, say SimpleTwo's directory and SimpleZero's id. Each role is a node of its own under the same agent: its own credential, its own row in Servers; the roles already running are not touched.

1. In Servers open the host row's menu (⋯) → Add a role to this host. Only a host under the agent has it; a host connected over SSH is moved To agent first.
2. Platform — when several are enabled and you may deploy in more than one. An administrator limited to one platform sees that platform's roles only.
3. Role. Each role shows what it would take beside the ones running: port conflicts, roles that need a host of their own (gateway, s3), and the cores, memory and disk the host would carry together — against what the host reported about itself. Only a role that fits can be chosen.
4. The role's fields — the same as in Add a host. Check and create the command checks the host again with them (a public name, for one, adds port 443) and gives the command.
5. Run the command on this host as root. The invitation is bound to it: on any other machine it is refused and stays unused. The new role's credential goes to /etc/simpletwo/agent.d/<role>.env beside the first one.
6. The wizard waits for the node: "waiting for the command" → "the host joined the role" → "up and reporting". A new row appears in Servers, and the machine's rows list its other roles.

If the roles conflict, the invitation's exchange refuses with the reason — the same check as in the wizard.

6. Step 4. Directory and sign-in

1. Settings → Single sign-on (OIDC) — connect the corporate provider (OIDC/SAML/ADFS).
2. Groups / roles → profile — map directory groups to roles and profile attributes.
3. Roles — check who is an administrator, who may host meetings, who sees telephony.

The ADFS article is needed in two cases: when people sign in with an address other than their UPN, and when the manager in AD is a DN rather than a mail address.

7. Step 5. Clients and accepting the installation

The client is called SimpleTwo Connect: iOS, Android, macOS and the web client (on Windows people use the web client — there is no separate build in 0.9.x). A user types their work email and the app finds the installation by its domain. Where to get the builds and how to distribute them is in Distributing clients.

Acceptance is not "the services are running" but a whole scenario:

1. An employee signs in with their corporate account.
2. A chat message reaches a second device and a second employee.
3. A one-to-one call: audible both ways.
4. A meeting: an external participant enters from a browser by link, waits in the lobby, is let in.
5. The tunnel: a client connects to a VPN node and opens an internal resource.
6. Every /healthz answers.

warning

A green deploy job and a running container are not a sign of a working service. A deploy is not finished until every service has answered /healthz and the scenario above has passed in full.

8. Optional roles

Role	When it is needed	What matters
gateway	access to the corporate network, or egress in a particular location	create the location first, then install the node. Reality keys are generated on the node and only the public one leaves it. No DNS required
recorder	conference recording	the only role that runs as a container: it records the room by driving headless Chrome. CPU-bound: 4 vCPU / 4 GB per simultaneous recording, plus disk. It listens to nothing outward — it takes jobs from the bus and writes into object storage. Requires redis and configured storage
mcu	meeting-room terminals over SIP (see Meeting rooms)	A deployed messaging is required: the MCU asks it which meeting a number opens. There may be several nodes: devices find them through DNS SRV records, and the nodes see each other's load through redis — the same one the SFU pool uses; it is mandatory from the second node, and a single node without it runs standalone. Media is forwarded without transcoding today; the load with transcoding will be sized at the next stage. More in Meeting rooms
pstn	the telephone network	a host of its own, never shared with mcu; redis, messaging and sfu must exist before the invitation. 4 vCPU / 4 GB is roughly ~80 concurrent calls; a routable IPv4 or NAT 1:1 with 5060 and the RTP range forwarded. More in Telephony
s3	attachments beyond one disk, several messaging replicas, or meeting recordings	object storage (versitygw, S3-compatible). It listens to nothing outward: 9000 is open only to the messaging and recorder hosts. Keys and buckets are created by the coordinator, nothing is typed by hand. A firewall on the host is mandatory — ufw or firewalld on Linux, pf on FreeBSD (the agent writes the anchor and enables pf itself); without one the deployment fails rather than leaving the store open to the whole private network. The alternative is external storage under Settings → External stores
mail and mx	you need mail	mail holds the mailboxes (database on postgres, message bodies in object storage — it does not start without storage) and answers clients over JMAP on 443 of its public name; mx is the edge in the DMZ on 25, 587 and 465, two nodes. Order, DNS records and mail_url — in mail
drive	you need personal and shared files	database on postgres, files in object storage (the simpletwo-drive bucket). The public name needs an A record to the host and 443 from the internet: the service obtains its certificate itself, and the coordinator announces drive_url to clients once https://<name>/healthz answers. See files
support	a support bot	installed in the DMZ beside the coordinator. Inbound is signed webhooks from the messaging host on :8087 only; it is reachable neither from the internet nor from clients. It holds its own model-provider key and its own knowledge base; the GitLab token needs the api scope and at least the Reporter role on the product project: the bot files issues and follows them
monitoring	metrics, logs and traces of every host	an option for any installation, one per installation; its interface opens from the console. It holds no product data: without it only the history is lost

9. Upgrade and backup

Upgrading the coordinator is a new image tag:

cd /opt/simpletwo
sed -i 's|:v[0-9.]*$|:v0.9.215|' .env      # only the tag; the image path stays
docker compose pull && docker compose up -d
curl -fsS http://127.0.0.1/healthz

Rolling back is the same file with the previous tag: earlier images stay in the registry. If a GitLab Runner is installed on the coordinator host, the deploy job on a v* tag performs the same steps.

Upgrading the roles is Redeploy on each host in the console. The new role binary is already inside the new coordinator image, hence the order: upgrade the coordinator first, then reinstall the roles.

The backup is three things, and the first matters most:

1. the coordinator-data volume (secrets, the control-plane database, certificates);
2. the postgres databases (chat and calendar) — ordinary scheduled pg_dump;
3. attachments on the messaging host, while they live on its disk.

The full procedure, including restoring and the master key file, is in Upgrade and backup.

10. When something does not work

Where to look: in the console — Coordinator logs, Host logs, Provisioning log, Audit log; on the coordinator host — docker compose logs coordinator; on role hosts — journalctl -u <the role's unit> -f.

Symptom	Likely cause
Installing a role stops immediately	when the coordinator installs it — SSH: wrong user, a key without access, no curl on the host. Run the join command on the host yourself to see where it stops
The role is active with a red state	the service is installed and did not come up: see Host logs and journalctl on the host
The browser shows a self-signed certificate although the name was added	the name is only in Tenant domains and Let's Encrypt is not enabled: the TLS card shows Source = self-signed. Step 2 above — name it in ST_TLS_DOMAIN or press Enable Let's Encrypt
sfu, turn or mcu has a certificate stuck in issuing	port 80 on the role's host is occupied or closed from outside — the agent cannot answer the domain-validation challenge
The certificate row shows an error and "next automatic attempt at …"	after a failure the next automatic order waits (5 → 15 → 30 → 60 minutes), so the certificate authority does not block the name after five failed validations in an hour. Once the cause is fixed press Issue — a manual order does not wait
The certificate is not issued and the log says invalidContact	the address is in a domain the authority does not accept (ops@example.test, say). The email is optional: clear it and retry, or use an address in a real domain
The certificate is not issued	the host is unreachable from the internet on 80/443, or the DNS record has not propagated
A client cannot find the installation by email	there is no s2.<domain>, or no /.well-known/simpletwo.json
Chat works, calls do not	the sfu node has no public name, or no API key in Telephony → Calls & conferences (SFU), or the SFU has no TLS front of its own
The call connects, no sound	a private address ended up in node IP (see §5), or the turn ports are blocked
Two people in one meeting cannot see each other	two sfu nodes without redis
A client cannot connect to a VPN node	the shortId, public key or SNI do not match; or the node sends no heartbeat and is considered stale
The calendar screens are missing in the client	the directory role is not installed or has not announced its name — expected behaviour, not a fault
"the database at … is unreachable from this host"	the directory or messaging role refused to deploy while it cannot see Postgres: deploy the postgres role and give the host a route to it
"wrong SSH host key", the host was rebuilt	you recreated a host under the same name: Servers and deployment → the ⋯ menu on the host → Accept the new host key

A role refuses to deploy because of the database

directory and messaging dial the database address from their DSN before configuring themselves. With no answer the role refuses, with a message naming the address and the reason — and that is deliberate.

Both services exit when they cannot open their store, so such a host used to end up with an installed unit, an endless systemd restart loop — and a green success in the console. The deployment reported success exactly where nothing worked.

What to do: deploy the postgres role before the roles that depend on it (the order in §5 is not a recommendation), check that the role's host has a route to it, and retry. If the database lives outside the installation, the address in the role's parameters must be reachable from the role's host, not from the coordinator.

A host was rebuilt and the coordinator will not connect

The coordinator remembers each host's SSH key at first connection and checks it afterwards — that is what tells your host apart from somebody else's answering at the same address. If you recreated the host, its key is new, and the connection is refused with a message naming both fingerprints: the trusted one and the presented one.

Compare the presented one with what the host itself says:

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub   # on the rebuilt host

If they match, press Accept the new host key in that host's ⋯ menu and the next connection will remember it. If they do not, do not accept: something else is answering at that address.