docs: correct the self-hosted server path and document server.toml

Audit of the public repo against the actual server and konduit-ctl source.
The happy path was documented; the failure modes were not.

- server.toml had no reference at all. Add docs/server-config.md covering
  every key, [tls], the control API, and key handling.
- The mantra could not recover the identity key on its own:
  derive_from_passphrase needs the [kdf] salt, which is 32 random bytes
  generated once at bootstrap. bootstrap says so in the file it generates
  (core/src/config.rs:403); the quickstart claimed the opposite. Fixed, with
  a backup step added before it is needed rather than after.
- konduit-server never appeared in a download step, yet
  konduit-server.service expects /opt/konduit/konduit-server - a path no
  instruction created. Added the install section and moved setcap after the
  copy, since capabilities do not survive every cross-filesystem copy and a
  binary that silently lost cap_net_bind_service fails at bind time.
- Hot config reload was a headline feature with no documented mechanism. It
  is a control API on 127.0.0.1:3001, and add-client only edits the file on
  disk, so a new peer is not live until something reloads it. Documented,
  including that /reload does not disconnect revoked peers but /sync does.
- [tls] native termination was entirely undocumented; the key must be PKCS#8.
- Dropped a README claim about a konduit-admin-ui unit that does not exist,
  and fixed a systemd copy path that only resolved if this repo had been
  cloned to /opt/konduit.

#k-ops/#1/#211

README.md

@@ -60,16 +60,22 @@ rather than assuming a row is present. Android is on Google Play either way.
 ### Linux (CLI)
 
 ```bash
-# Download konduit and konduit-ctl from the Releases page, then:
+# Download konduit-server, konduit-ctl and konduit from the Releases page, then:
-chmod +x konduit konduit-ctl
+chmod +x konduit-server konduit-ctl konduit
 
-# Bootstrap a server
+# Bootstrap a server (run from the directory that will hold server.toml)
 echo "your secret mantra" | ./konduit-ctl bootstrap -l vpn.example.com:443 -p -
 
+# Add a client, and reload the server so it picks the peer up
+./konduit-ctl add-client my-laptop --reload
+
 # Connect a client
-./konduit --config client.toml
+./konduit --config my-laptop.client.toml
 ```
 
+Running a server end to end — install, capabilities, NAT, systemd — is covered
+in the [Server Quickstart](docs/server-quickstart.md).
+
 `--config`/`-c` is a top-level flag, not a `connect` flag. The `connect`
 subcommand takes the peer details individually instead:
 
@@ -108,9 +114,10 @@ Coming soon.
 **CLI & server**
 - [Client Quickstart](docs/client-quickstart.md) — download, configure, connect, run as a systemd service
 - [Server Quickstart](docs/server-quickstart.md) — install, provision, NAT setup for iptables and firewalld
+- [Server Configuration](docs/server-config.md) — every `server.toml` key, the control API, and how the server identity key is protected
 - [SOCKS5 Modes](docs/socks-modes.md) — reach the server through a proxy, or expose a local SOCKS5 listener
 - [Connection Tuning](docs/connection-tuning.md) — opt-in connection pooling for unstable links
-- [systemd units](docs/systemd/) — service files for konduit-server, konduit (client), and konduit-admin-ui
+- [systemd units](docs/systemd/) — service files for konduit-server and konduit (client)
 
 ## Architecture
 

docs/client-quickstart.md

@@ -39,11 +39,15 @@ connected · no udp, no root, port 443
 To connect automatically on boot, use the provided systemd unit:
 
 ```bash
-sudo cp /opt/konduit/docs/systemd/konduit.service /etc/systemd/system/
+# konduit.service ships in this repo at docs/systemd/ — copy it from the repo root
+sudo cp docs/systemd/konduit.service /etc/systemd/system/
 sudo systemctl daemon-reload
 sudo systemctl enable --now konduit
 ```
 
+The unit expects the binary at `/opt/konduit/konduit` and the config at
+`/opt/konduit/client.toml` — edit both paths if you installed elsewhere.
+
 The unit runs as root (required for TUN device creation) and restarts automatically on failure.
 
 ## 5. Capabilities (alternative to root)

docs/server-config.md

@@ -0,0 +1,212 @@
+# Server Configuration Reference
+
+`server.toml` is written by `konduit-ctl bootstrap` and appended to by
+`konduit-ctl add-client`. You rarely need to hand-edit it, but when you do,
+this is every key the server reads.
+
+## Layout
+
+```toml
+[server]
+listen_addr = "0.0.0.0"
+listen_port = 443
+public_addr = "vpn.example.com"
+keepalive_interval = 30
+identity_key = "<64 hex chars>"
+private_key = "<64 hex chars>"
+
+[kdf]
+version = 2
+algorithm = "argon2id"
+m_cost = 262144
+t_cost = 3
+p_cost = 4
+salt_hex = "<64 hex chars>"
+
+[network]
+tun_name = "tun0"
+server_ip = "10.10.0.1"
+netmask = "255.255.255.0"
+mtu = 1300
+dns = ["1.1.1.1"]
+
+[[peers]]
+peer_id = "my-laptop"
+public_key = "<64 hex chars>"
+ip_address = "10.10.0.10"
+allowed_routes = ["0.0.0.0/0"]
+enabled = true
+```
+
+## `[server]`
+
+| Key | Type | Default | Notes |
+|---|---|---|---|
+| `listen_addr` | string | `0.0.0.0` | Address the VPN listener binds. |
+| `listen_port` | u16 | `443` | Port the VPN listener binds. Set by the `-l` flag on `bootstrap`. |
+| `public_addr` | string | — | Hostname or IP written into every client config. Set by the `-l` flag on `bootstrap`. |
+| `public_port` | u16 | = `listen_port` | Port written into client configs when it differs from `listen_port`. Set by `--public-port`. |
+| `keepalive_interval` | u64 | `30` | Seconds between keepalives on an idle tunnel. |
+| `identity_key` | hex or `file://` | — | **The secret.** Ed25519 private identity, 32 bytes hex. See [Key handling](#key-handling). |
+| `private_key` | hex or `file://` | — | Legacy X25519 key, unused by the current handshake. Left in place for compatibility. |
+| `fallbacks` | list of IPs | `[]` | Extra server addresses a client may fall back to. |
+
+`public_addr` is the value clients dial. Bootstrap with the **hostname** you will
+actually serve, not a throwaway IP — the string is baked into every client
+config, and changing it later means re-issuing every client file.
+
+Both key fields accept either inline hex or a `file://` URI. A relative
+`file://./server.key` resolves against the config file's directory; an absolute
+path is used as-is. Anything that isn't `file://` is treated as literal hex.
+
+## `[kdf]`
+
+| Key | Type | Notes |
+|---|---|---|
+| `version` | int | KDF format version. `2` is current. |
+| `algorithm` | string | `argon2id`. |
+| `m_cost` | int | Memory in KiB. `262144` = 256 MiB. |
+| `t_cost` | int | Iterations. |
+| `p_cost` | int | Parallelism. |
+| `salt_hex` | string | 32 random bytes, hex. **Unique per bootstrap.** |
+
+**The server does not read this section at runtime** — it parses and ignores it.
+It exists so the identity key can be re-derived from your mantra later. Treat it
+as part of the backup, not as tuning you can adjust: see
+[Key handling](#key-handling).
+
+## `[network]`
+
+| Key | Type | Default | Notes |
+|---|---|---|---|
+| `tun_name` | string | `tun0` | Interface name. **Set this explicitly** — see below. |
+| `server_ip` | IP | `10.10.0.1` | Server's address on the VPN. Its `/24` defines the client pool. |
+| `netmask` | IP | `255.255.255.0` | VPN subnet mask. |
+| `mtu` | int | `1300` | TUN MTU. |
+| `dns` | list of IPs | `[]` | DNS servers pushed to clients. |
+
+`tun_name` defaults to `tun0`, and the TUN layer falls back to the first free
+`tunN` if that name is taken. On a host that also runs WireGuard or OpenVPN the
+actual name then shifts underneath you, and any firewall rule keyed to it stops
+matching. Pin it:
+
+```toml
+[network]
+tun_name = "konduit0"
+```
+
+Your masquerade rule must match whatever subnet `server_ip` implies. The default
+is `10.10.0.1` with a `/24` mask, so clients land in `10.10.0.0/24`:
+
+```bash
+sudo iptables -t nat -I POSTROUTING 1 -s 10.10.0.0/24 -o $IFACE -j MASQUERADE
+```
+
+If you change `server_ip`, change that rule to match. `add-client` allocates
+addresses by scanning existing peers in the same `/24`, starting at `.10` and
+failing once `.254` is passed.
+
+## `[[peers]]`
+
+| Key | Type | Default | Notes |
+|---|---|---|---|
+| `peer_id` | string | — | Unique label. Becomes `<peer_id>.client.toml`. |
+| `public_key` | hex | — | Client's Ed25519 public key. |
+| `ip_address` | IP | — | Assigned inside the VPN subnet. |
+| `enabled` | bool | `true` | `false` stops the peer being accepted. Takes effect on new connections; see [Reloading](#reloading). |
+| `allowed_routes` | list of CIDR | `["0.0.0.0/0"]` | What this client is permitted to route. |
+| `tenant` | string | — | Optional grouping label. Not written by `add-client`. |
+
+Routing is server-side policy. A client asking for a route it isn't listed for
+does not get it, so `allowed_routes` is the knob for splitting access — set
+`["10.0.0.0/8"]` for a LAN-only peer and `["0.0.0.0/0"]` for full tunnel.
+
+## `[tls]`
+
+Optional. When present the server terminates TLS itself instead of expecting a
+TCP-passthrough proxy in front:
+
+```toml
+[tls]
+cert_file = "/etc/konduit/fullchain.pem"
+key_file  = "/etc/konduit/privkey-pkcs8.pem"
+```
+
+| Key | Type | Notes |
+|---|---|---|
+| `cert_file` | path | PEM certificate, full chain. |
+| `key_file` | path | Private key — **must be PKCS#8**. |
+
+A PKCS#1 key (`BEGIN RSA PRIVATE KEY`) is rejected at startup. Convert it:
+
+```bash
+sudo openssl pkcs8 -topk8 -inform PEM -outform PEM \
+  -in privkey.pem -out privkey-pkcs8.pem
+```
+
+Omit `[tls]` entirely if a proxy terminates the connection for you.
+
+## Reloading
+
+The server exposes an unauthenticated control API on `127.0.0.1:3001`. It is
+bound to loopback and assumes it stays there.
+
+| Method | Path | Effect |
+|---|---|---|
+| `GET` | `/status` | Uptime and basic server state. |
+| `GET` | `/peers` | Currently connected peers. |
+| `GET` | `/config` | Current configuration. |
+| `POST` | `/reload` | Re-read `server.toml` from disk. |
+| `POST` | `/sync` | Replace `server.toml` with the request body, then reload. |
+
+`reload` swaps the parsed configuration. It does **not** disconnect anyone:
+tunnels already up keep running on the old config until they end, so revoking a
+peer with `enabled = false` stops new connections but leaves an existing session
+up. Restarting the service is the blunt way to drop everything.
+
+`sync` is the one that tears down peers it no longer wants — it disconnects any
+peer that is disabled or absent from the uploaded config.
+
+From the CLI:
+
+```bash
+./konduit-ctl reload                 # POST /reload
+./konduit-ctl status                 # GET  /status
+./konduit-ctl peers                  # GET  /peers
+./konduit-ctl push                   # upload local server.toml via POST /sync
+
+./konduit-ctl add-client my-laptop --reload   # add, then reload in one step
+```
+
+`add-client` without `--reload` only edits `server.toml` on disk. The running
+server keeps its old peer list until something reloads it.
+
+> **Never bind the control API to a public address.** `--control-addr
+> 0.0.0.0:3001` exposes `POST /sync`, which writes an attacker-supplied file
+> straight to `server.toml` — that file holds the server's private identity key
+> and every client's public key. Reach it over an SSH tunnel if you need remote
+> administration.
+
+## Key handling
+
+`server.toml` contains the server's **private identity key** in plaintext. Anyone
+who reads the file can impersonate the server to every client.
+
+`salt_hex` in `[kdf]` is 32 random bytes generated at bootstrap time and never
+derived from anything you chose. Losing `server.toml` means losing the salt, and
+the mantra alone **cannot** regenerate the key — there is nothing to seed it
+with. The mantra is one half of the key material; `server.toml` is the other.
+
+```bash
+# Back up both, or accept that recovery is impossible.
+sudo install -m 600 server.toml /root/konduit-server.toml.bak
+```
+
+Confirm your backup parses before you need it:
+
+```bash
+./konduit-server --config /root/konduit-server.toml.bak --validate
+```
+
+`konduit-ctl gen-root` prints the public identity and the full KDF parameter set
+in one block, which is the other thing worth storing off-box.

docs/server-quickstart.md

@@ -1,50 +1,129 @@
 # Konduit Server Quickstart
 
-## 1. Prerequisites
+Running your own Konduit server takes three binaries: `konduit-server` (the VPN
+server), `konduit-ctl` (provisioning), and `konduit` (a client, for testing).
 
-### IP Forwarding
+## 1. Install
+
+Download `konduit-server-<version>-linux-x86_64` and `konduit-ctl` from the
+[Releases](../../releases) page, plus `konduit-<version>-linux-x86_64` if you
+want to test the tunnel from the same host.
+
+```bash
+sudo mkdir -p /opt/konduit
+sudo cp konduit-server konduit-ctl konduit /opt/konduit/
+cd /opt/konduit
+sudo chmod +x konduit-server konduit-ctl konduit
+```
+
+Run everything from `/opt/konduit` — `konduit-ctl` writes `server.toml` into the
+current directory, and the systemd unit expects the config at
+`/opt/konduit/server.toml`.
+
+### Capabilities
+
+Grant the binary permission to create TUN devices and bind port 443 without
+running as root:
+
+```bash
+sudo setcap cap_net_admin,cap_net_bind_service=+ep /opt/konduit/konduit-server
+```
+
+Do this **after** copying the binary. Capabilities are stored as an extended
+attribute on the file, and a copy made across filesystems can drop them — a
+`konduit-server` that quietly lost `cap_net_bind_service` fails with a permission
+error the moment it tries to bind 443. The unit in step 6 runs as `User=root`,
+where capabilities are irrelevant; this matters for running it by hand.
+
+To confirm what the binary actually ended up with:
+
+```bash
+getcap /opt/konduit/konduit-server
+```
+
+## 2. Prerequisites
+
+### IP forwarding
 
 ```bash
 sudo sysctl -w net.ipv4.ip_forward=1
 # Persist: add 'net.ipv4.ip_forward=1' to /etc/sysctl.conf
 ```
 
-### Capabilities
+### TUN device
-
-Konduit needs permission to create TUN devices and bind privileged ports without running as root.
 
 ```bash
-sudo setcap cap_net_admin,cap_net_bind_service=+ep ./konduit-server
+ls /dev/net/tun || sudo modprobe tun
-sudo setcap cap_net_admin=+ep ./konduit
 ```
 
-## 2. Bootstrap
+Without it the server cannot create its interface and exits at startup.
 
-Konduit derives its identity key from a **mantra** (secret phrase) using Argon2id. Keep the mantra safe — it lets you recover the same identity if the config file is lost.
+## 3. Bootstrap
 
-**Standard (konduit binds on 443):**
+Konduit derives its identity key from a **mantra** (secret phrase) using
+Argon2id.
 
 ```bash
+cd /opt/konduit
 echo "your secret mantra phrase here" | ./konduit-ctl bootstrap -l vpn.example.com:443 -p -
 ```
 
-**Behind a reverse proxy (HAProxy on 443, konduit on 8443):**
+`-l` sets both the port the server binds and the hostname written into every
+client config, so use the name you will actually serve. Bootstrap refuses to
+overwrite an existing `server.toml` unless you pass `--force`.
+
+It prints the server's public identity key — clients need this, and it is
+already inside the generated `.client.toml` files.
+
+**Behind a reverse proxy** (HAProxy on 443, konduit on 8443):
 
 ```bash
-echo "your secret mantra phrase here" | ./konduit-ctl bootstrap -l vpn.example.com:8443 --public-port 443 -p -
+echo "your secret mantra phrase here" | \
+  ./konduit-ctl bootstrap -l vpn.example.com:8443 --public-port 443 -p -
 ```
 
-`--public-port` sets the port written into client configs, so they connect to 443 even though konduit listens on 8443 — useful when running behind any TCP-passthrough reverse proxy (e.g. HAProxy, nginx `stream` module).
+`--public-port` sets the port written into client configs, so they connect to
+443 even though konduit listens on 8443 — useful when running behind any
+TCP-passthrough reverse proxy (e.g. HAProxy, nginx `stream` module). Konduit
+can also terminate TLS itself; see `[tls]` in the
+[server configuration reference](server-config.md).
 
-## 3. Add a Client
+> **Back up `server.toml` now, before you need it.** It holds the server's
+> private identity key in plaintext, and its `[kdf] salt_hex` is 32 random bytes
+> generated once at bootstrap. The mantra is only half the key material — if you
+> lose the file you lose the salt, and the mantra alone cannot regenerate the
+> key. See [Key handling](server-config.md#key-handling).
+
+```bash
+sudo install -m 600 server.toml /root/konduit-server.toml.bak
+```
+
+## 4. Add a Client
 
 ```bash
 ./konduit-ctl add-client my-laptop
 ```
 
-Creates `my-laptop.client.toml` and appends the peer to `server.toml`. Transfer the `.client.toml` to the client device, or scan the QR payload with the mobile app.
+Creates `my-laptop.client.toml` and appends the peer to `server.toml`. Transfer
+the `.client.toml` to the client device, or scan the QR payload with the mobile
+app.
 
-## 4. TUN Interface Name
+**The running server does not see this yet.** `add-client` only edits the file on
+disk; the live peer list updates on the next reload. Either add `--reload` to the
+command, or reload separately:
+
+```bash
+./konduit-ctl add-client my-laptop --reload
+# or, on an already-running server:
+./konduit-ctl reload
+```
+
+`--reload` talks to the control API on `127.0.0.1:3001`. Running `konduit-ctl`
+on a different machine than the server needs `--control-url`, and the API is
+bound to loopback — see [Reloading](server-config.md#reloading) before changing
+that.
+
+## 5. TUN Interface Name
 
 Set a fixed TUN interface name so firewall rules are stable:
 
@@ -54,24 +133,37 @@ Set a fixed TUN interface name so firewall rules are stable:
 tun_name = "konduit0"
 ```
 
-Without this, konduit picks the first available `tunN`, which shifts if other VPN interfaces exist.
+The default is `tun0`, but konduit falls back to the first free `tunN` if that
+name is taken. On a host that also runs WireGuard or OpenVPN the name then
+shifts underneath you and any firewall rule keyed to it stops matching. The
+masquerade rule in step 7 uses this name.
 
-## 5. Run
+## 6. Run
 
 ```bash
 ./konduit-server --config server.toml
 ```
 
+Check the file before starting for real:
+
+```bash
+./konduit-server --config server.toml --validate
+```
+
 For persistent operation, use the provided systemd units in [docs/systemd/](systemd/). Copy `konduit-server.service` to `/etc/systemd/system/`, then:
 
 ```bash
 sudo systemctl daemon-reload
 sudo systemctl enable --now konduit-server
+sudo journalctl -u konduit-server -f
 ```
 
-## 6. NAT / Masquerade
+## 7. NAT / Masquerade
 
-VPN clients send packets with source IP `10.10.0.x`. Masquerade rewrites this to the server's public IP so internet hosts can reply.
+VPN clients send packets with source IP `10.10.0.x` — the `/24` implied by the
+default `server_ip = "10.10.0.1"`. Masquerade rewrites this to the server's
+public IP so internet hosts can reply. If you changed `server_ip`, match the rule
+to the new subnet.
 
 ### firewalld (RHEL, Fedora, recent Ubuntu)
 
@@ -99,12 +191,39 @@ sudo iptables -I FORWARD 2 -i $IFACE -o konduit0 -m state --state RELATED,ESTABL
 sudo apt install iptables-persistent && sudo netfilter-persistent save
 ```
 
-## 7. Verify
+## 8. Verify
 
 ```bash
 # Connect a client and confirm the tunnel comes up
-./konduit --config client.toml
+./konduit --config my-laptop.client.toml
+
+# Is the peer actually connected?
+./konduit-ctl peers
 
 # Check konduit logs
 journalctl -u konduit-server -f
 ```
+
+`./konduit-ctl peers` returning your peer's address is the check worth having —
+it distinguishes "the client authenticated" from "the client is routing traffic",
+and only the second one proves NAT is set up.
+
+## Troubleshooting
+
+| Symptom | Cause |
+|---|---|
+| `Permission denied` binding 443 | `cap_net_bind_service` was lost in a copy. Re-run `setcap` against the final path and confirm with `getcap`. |
+| `No such file or directory` on startup | `/dev/net/tun` missing — `sudo modprobe tun`. |
+| Tunnel connects, but no internet | Masquerade missing or scoped to the wrong subnet. Check step 7 matches your `server_ip`. |
+| iptables rules look right but do nothing | firewalld is running and ignoring them. Use `firewall-cmd`. |
+| New client cannot connect | Peer was added to `server.toml` but never reloaded. Run `konduit-ctl reload`. |
+| Revoked client still has access | `/reload` does not disconnect live sessions. Restart the service. |
+| `IP pool exhausted (10.10.0.10-254)` | The `/24` is full. Widen `netmask` or move `server_ip` to another subnet. |
+
+For more detail than the table, set `RUST_LOG=debug`:
+
+```bash
+sudo systemctl edit konduit-server
+# [Service]
+# Environment=RUST_LOG=debug
+```