Docs: server agent and Hub 2.1.0 plan (phase 1d, decisions 27-37)
Agent on every server runs installs, scheduled/emergency updates and uninstalls on signed jobs from the Hub, polls only (no listening port), appends to frpc.ini in marked blocks, builds .env files from per-server config in the Hub. FastPanel and DNS stay manual (emailed instructions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,477 @@
|
||||
# Server agent 1.0.0 + Hub 2.1.0 — installs, updates and uninstalls from the Hub
|
||||
|
||||
Planned 2026-11-12 (ROADMAP-v3 phase **1d**, decisions 27–37). Goal: when a new church signs up,
|
||||
Joshua picks a server in the Hub and the rest happens by itself, with a report by email. Updates
|
||||
are scheduled from the Hub with notice to the church, and every server's health is visible there.
|
||||
|
||||
This replaces the old rule "the Hub never deploys" (decisions 10 and 23). The part that matters
|
||||
stays: **no site depends on the Hub to keep running**, and everything still goes through the
|
||||
`crosscode` script (EVENTS-2.2-PLAN §4), which keeps working by hand over SSH.
|
||||
|
||||
## Contents
|
||||
|
||||
1. [The shape](#1-the-shape)
|
||||
2. [Rules](#2-rules)
|
||||
3. [The agent](#3-the-agent)
|
||||
4. [Jobs and their security](#4-jobs-and-their-security)
|
||||
5. [Configuration: server, defaults, per install](#5-configuration-server-defaults-per-install)
|
||||
6. [frp and the central server](#6-frp-and-the-central-server)
|
||||
7. [New install](#7-new-install)
|
||||
8. [Updates: scheduled, emergency, batch](#8-updates-scheduled-emergency-batch)
|
||||
9. [Uninstall](#9-uninstall)
|
||||
10. [Servers page and alerts](#10-servers-page-and-alerts)
|
||||
11. [Reports and emails](#11-reports-and-emails)
|
||||
12. [What changes in each repo](#12-what-changes-in-each-repo)
|
||||
13. [Testing](#13-testing)
|
||||
14. [Rolling it out](#14-rolling-it-out)
|
||||
15. [Decisions](#decisions)
|
||||
|
||||
---
|
||||
|
||||
## 1. The shape
|
||||
|
||||
```
|
||||
Cross Code Hub (hub.crosscode.co.za)
|
||||
servers · jobs · schedules · port list · server config · reports
|
||||
▲ agents poll: "any jobs?" + stats + job results (outbound HTTPS only)
|
||||
│
|
||||
┌────────┴─────────────┐ ┌──────────────────────┐ ┌─────────────────────────────┐
|
||||
│ Church server A │ │ Church server B │ │ Central server (FastPanel) │
|
||||
│ crosscode-agent │ │ crosscode-agent │ │ crosscode-agent --central │
|
||||
│ └ runs `crosscode` │ │ └ runs `crosscode` │ │ read-only: frps ports, │
|
||||
│ PM2 sites, Postgres │ │ … │ │ allow_ports, bind address │
|
||||
│ frpc ═══tunnel═══════╪═══╪══════════════════════╪══▶│ frps ◀─ nginx (FastPanel, │
|
||||
└──────────────────────┘ └──────────────────────┘ │ set up by Joshua) │
|
||||
└─────────────────────────────┘
|
||||
```
|
||||
|
||||
## 2. Rules
|
||||
|
||||
1. **The agent listens on nothing.** No HTTP server, no open port, no URL, not even on
|
||||
`127.0.0.1`. It only makes outgoing HTTPS requests to the Hub (poll, stats, results). A test
|
||||
checks that the running agent has no listening socket.
|
||||
2. **The agent only runs the `crosscode` script**, with arguments built from a fixed list of job
|
||||
types (§4). There is no "run this command" job.
|
||||
3. **Jobs are signed by the Hub** and checked by the agent against the Hub key pinned when the
|
||||
server joined. Unsigned, expired or replayed jobs are refused and reported.
|
||||
4. **No site depends on the Hub or the agent.** If either is down, sites keep running and
|
||||
`crosscode` still works by hand. Scheduled jobs that can't be delivered are reported as missed,
|
||||
never run late without a fresh notice (§8).
|
||||
5. **The agent never touches FastPanel or DNS.** FastPanel setup is an email to Joshua (§7, §11).
|
||||
6. **Files the agent edits are only ever added to, never replaced**: `/etc/frp/frpc.ini` and each
|
||||
install's `.env` (§5, §6). Anything written by hand is never changed.
|
||||
7. **Secrets generated for an install stay on its server** (`JWT_SECRET`, the database password).
|
||||
They're never sent to the Hub, never put in a report or email.
|
||||
|
||||
## 3. The agent
|
||||
|
||||
- **Where:** `apps/agent` in this monorepo, Node 24 + TypeScript, tagged `agent@1.0.0`. Installed
|
||||
in `/opt/crosscode/agent` with its config in `/etc/crosscode/agent.json` (mode 600, root).
|
||||
- **Run by systemd** (`crosscode-agent.service`, restarts on failure), not PM2, because it manages
|
||||
PM2. Runs as root, as `crosscode install` needs (PostgreSQL, PM2 for the app user, frpc).
|
||||
- **Joining:** Hub → Servers → **Add server** gives a one-time token (valid 1 hour). On the server:
|
||||
`curl … | bash -s -- --hub https://hub.crosscode.co.za --token …` installs Node if needed, the
|
||||
agent and `/opt/crosscode/tools`, then the agent:
|
||||
1. makes an **Ed25519 key pair** (signs its requests to the Hub) and an **X25519 key pair**
|
||||
(receives encrypted config, §5); the private keys never leave the server,
|
||||
2. sends the token and both public keys; the Hub replies with the server id and **the Hub's
|
||||
job-signing public key**, which the agent pins.
|
||||
- **Polling:** every 30 s `POST /api/agent/v1/poll` (signed) with a short status; the reply holds
|
||||
any jobs due. Every 60 s the stats (§10). Requests back off when the Hub doesn't answer.
|
||||
- **Modes:** `church` (default) and `central` (§6). The central agent can't run any job but
|
||||
`check-ports` and `status`.
|
||||
- **Updating itself:** an `update-agent` job checks out the newest `agent@…` tag, builds it next
|
||||
to the running copy, switches the systemd unit over and reports its new version. If the new one
|
||||
doesn't poll within 2 minutes, a systemd timer starts the previous one again.
|
||||
- **One job at a time per server**, in the order the Hub gives them. A job has a time limit
|
||||
(install 30 min, deploy 20 min); a job past it is stopped and reported as failed.
|
||||
|
||||
## 4. Jobs and their security
|
||||
|
||||
**Job types** (and the `crosscode` command each runs):
|
||||
|
||||
| Job | Runs | Needs 2FA again in the Hub |
|
||||
|-----|------|:-:|
|
||||
| `install` | `crosscode install <church> --products … --json` | no |
|
||||
| `add-product` | `crosscode add <church> <product> --json` | no |
|
||||
| `deploy` | `crosscode deploy <church> [products] [--version …] --json` | emergency only |
|
||||
| `adopt` / `adopt-undo` | `crosscode adopt …` | no |
|
||||
| `restart` | `crosscode restart <church> [product]` (PM2 reload) | no |
|
||||
| `export` | `crosscode export <church> --json` (§9) | no |
|
||||
| `uninstall` | `crosscode uninstall <church> --json` | **yes**, and typing the short name |
|
||||
| `purge` | `crosscode purge <church>` (deletes the 30-day keep, §9) | no (scheduled at uninstall) |
|
||||
| `update-agent` | §3 | no |
|
||||
| `check-ports` | central only (§6) | no |
|
||||
| `status` | `crosscode status --json` | no |
|
||||
|
||||
**A signed job** (`crosscode.agent-job/1`, in `packages/contracts`):
|
||||
|
||||
```json
|
||||
{
|
||||
"schema": "crosscode.agent-job/1",
|
||||
"id": "job_…",
|
||||
"serverId": "srv_…",
|
||||
"type": "deploy",
|
||||
"args": { "church": "hope", "products": ["events"], "version": "events@2.4.1" },
|
||||
"config": { "alg": "x25519-xsalsa20-poly1305", "ciphertext": "…" },
|
||||
"notBefore": "2026-11-20T12:00:00Z",
|
||||
"expiresAt": "2026-11-20T12:15:00Z",
|
||||
"signature": "…"
|
||||
}
|
||||
```
|
||||
|
||||
- The agent checks: the signature (pinned Hub key), `serverId` is its own, it's within
|
||||
`notBefore`–`expiresAt`, the id hasn't been seen (it keeps the ids of the last 90 days), the
|
||||
type is known and every `args` value passes that type's schema (short names
|
||||
`^[a-z][a-z0-9-]{1,30}$`, products from the enum, versions as `<product>@x.y.z`). Arguments are
|
||||
passed to `crosscode` as an argument array, never through a shell.
|
||||
- `config` holds whatever the job needs from §5, encrypted to **this server's** X25519 key, so
|
||||
job rows and logs in the Hub never hold it in plain text.
|
||||
- **Results** (`crosscode.agent-report/1`): the job id, status (`succeeded`, `failed`,
|
||||
`rolled-back`, `refused`, `expired`), each step with its start, end and outcome, and the last
|
||||
200 log lines. The script's `--json` output is that report; the agent adds the log lines and
|
||||
sends it, signed with its own key. Logs are filtered for anything that looks like a secret
|
||||
before they leave the server.
|
||||
- **The Hub side:** job-signing private key in the Hub's `.env` (not in the database), its own
|
||||
key separate from the licence-signing key. 2FA re-confirmation for uninstall and emergency
|
||||
deploys. Every job is in the audit log with who made it.
|
||||
|
||||
## 5. Configuration: server, defaults, per install
|
||||
|
||||
An install's `.env` files are built from four layers. Each variable comes from exactly one layer.
|
||||
|
||||
| Layer | Set where | Examples | Sent how |
|
||||
|-------|-----------|----------|----------|
|
||||
| **Server** | Hub → Servers → *server* → Configuration, **once per server** (editable) | Database host, port and an admin role that can create databases and roles (or "local PostgreSQL, through `sudo -u postgres`"), `TRUST_PROXY`, app user, backup folder | Stored AES-GCM encrypted in the Hub (as install secrets are today); sent to the agent encrypted to its key with each job that needs it. The agent doesn't keep it on disk |
|
||||
| **Defaults** | Hub → Settings → Install defaults, once | `HUB_URL`, `APP_TIMEZONE` (Africa/Johannesburg), job switches (`DAILY_SUMMARY_ENABLED=true`, `BACKUP_ENABLED=true`, …), `HUB_HEARTBEAT_ENABLED=true`, optional platform `SENTRY_DSN` | With the job, encrypted the same way |
|
||||
| **Install** | The New install form (§7) | Domain → `FRONTEND_URL`, `APP_BASE_URL`, `BACKEND_URL` and the frontend's `NEXT_PUBLIC_API_URL` (all `https://<domain>`, since FastPanel sends `/api` to the backend on the same domain); timezone if not the default; the licence key; ports from the port list (§6) → `PORT` | With the job |
|
||||
| **Generated** | On the server by `crosscode install` | `JWT_SECRET`, the database password (so `DATABASE_URL`), `NODE_ENV=production` | Never leaves the server (rule 7) |
|
||||
|
||||
**Per-product env template.** Each product has `apps/<product>/env.template.json` in the
|
||||
monorepo, listing every variable with its layer, whether it's required, and a default:
|
||||
|
||||
```json
|
||||
{ "schema": "crosscode.env-template/1", "product": "events", "files": {
|
||||
"backend/.env": [
|
||||
{ "name": "DATABASE_URL", "layer": "generated", "required": true },
|
||||
{ "name": "FRONTEND_URL", "layer": "install", "from": "https://{domain}", "required": true },
|
||||
{ "name": "TRUST_PROXY", "layer": "server", "required": true },
|
||||
{ "name": "BACKUP_ENABLED", "layer": "defaults", "default": "true" }
|
||||
],
|
||||
"frontend/.env.production": [
|
||||
{ "name": "NEXT_PUBLIC_API_URL", "layer": "install", "from": "https://{domain}", "required": true }
|
||||
] } }
|
||||
```
|
||||
|
||||
- The Hub reads the template of the release being installed (synced with the release, §5 of the
|
||||
roadmap) and **refuses to create the job if a required server or defaults value is missing**,
|
||||
saying which, before anything runs on the server.
|
||||
- **Install:** `crosscode install` writes each `.env` from the template (mode 600, app user).
|
||||
- **Updates:** before building, `crosscode deploy` compares the new release's template with the
|
||||
`.env`. A new variable is **appended** with its value (and a comment with the date and
|
||||
release); existing lines are never changed or removed. A new *required* variable with no value
|
||||
in any layer stops the deploy at the checks, before anything is touched.
|
||||
- **Changing a server value later** (e.g. a new database host) is a `reconfigure` step of a
|
||||
deploy: the agent shows which lines would change in each install, and only changes them when
|
||||
Joshua confirms in the Hub. This is the one exception to "only append", and it never runs by
|
||||
itself.
|
||||
- **URLs are checked before and after.** Before: the form checks that the domain's DNS points at
|
||||
the central server and warns if not (the install still runs). After Joshua marks the proxy done:
|
||||
the Hub checks `https://<domain>/` (frontend answers), `https://<domain>/health` (backend
|
||||
answers with the right version) and that `/health` reports the `FRONTEND_URL` it was given.
|
||||
|
||||
**For Joshua to set per server before the first install:** the database settings and
|
||||
`TRUST_PROXY`. Behind frp the app sees every connection from frpc on `127.0.0.1`, and the
|
||||
visitor's address arrives in `X-Forwarded-For` from FastPanel's nginx, so the expected value is
|
||||
`loopback` (the Hub suggests it).
|
||||
|
||||
## 6. frp and the central server
|
||||
|
||||
**Today:** `/etc/frp/frpc.ini` on each church server, `tcp` tunnels, frontend and backend each
|
||||
with their own `remote_port`. FastPanel on the central server proxies a domain to
|
||||
`127.0.0.1:<remote_port>` with Let's Encrypt.
|
||||
|
||||
**Port list in the Hub.** frps opens every `remote_port` on the one central server, so they must
|
||||
be unique across every church server. The Hub hands them out from a range (default
|
||||
**20000–29999**, set in Hub settings to match `allow_ports`), two per product install (web, api),
|
||||
and **uses the same number for `local_port`**, so it's also free on the church server and there's
|
||||
one number per tunnel to think about. Ports already in use (today's `3000`/`5000` and anything
|
||||
the central agent sees listening) are recorded as taken. Ports are released 30 days after an
|
||||
uninstall.
|
||||
|
||||
**Tunnel names** must also be unique on frps: `<church>-<product>-web` and
|
||||
`<church>-<product>-api` (e.g. `hope-events-web`). Existing entries such as `[events-frontend]`
|
||||
stay as they are.
|
||||
|
||||
**What the agent adds to frpc.ini** (one block per product install, appended at the end):
|
||||
|
||||
```ini
|
||||
# >>> crosscode:hope:events (added by crosscode-agent 2026-11-20, do not edit by hand)
|
||||
[hope-events-web]
|
||||
type = tcp
|
||||
local_ip = 127.0.0.1
|
||||
local_port = 20010
|
||||
remote_port = 20010
|
||||
|
||||
[hope-events-api]
|
||||
type = tcp
|
||||
local_ip = 127.0.0.1
|
||||
local_port = 20011
|
||||
remote_port = 20011
|
||||
# <<< crosscode:hope:events
|
||||
```
|
||||
|
||||
- **Only appended to, never replaced.** Before a change: a copy to
|
||||
`/etc/frp/backups/frpc.ini.<timestamp>`; write to a temporary file; `frpc verify -c`; then move
|
||||
it into place. If the check fails, nothing is changed. Uninstall removes only the lines between
|
||||
that install's own markers; nothing outside a marked block is ever changed.
|
||||
- **Reload without dropping other sites.** Once per server (agreed 2026-11-12) the agent adds to
|
||||
`[common]`, if they're not there:
|
||||
```ini
|
||||
admin_addr = 127.0.0.1
|
||||
admin_port = 7400
|
||||
```
|
||||
so `frpc reload -c /etc/frp/frpc.ini` picks up new tunnels without restarting frpc. This is the
|
||||
only change ever made in `[common]`, and it's reported. (`admin_addr` is loopback only, so it
|
||||
adds no way in from outside.)
|
||||
- The `.ini` format is deprecated in newer frp versions in favour of TOML. The agent reads the
|
||||
format the server has; moving to TOML is a separate job for later.
|
||||
|
||||
**The central agent** (`--central`, on the FastPanel server, read-only) answers `check-ports`:
|
||||
|
||||
- the port is inside `allow_ports` in `frps.ini`,
|
||||
- frps has the tunnel open (the port is listening; frps's dashboard API is used if it's enabled
|
||||
on `127.0.0.1`),
|
||||
- **`proxy_bind_addr = 127.0.0.1` is set**, so tunnels are only reachable through FastPanel's
|
||||
nginx, not straight from the internet on `<central IP>:<port>` (which would bypass HTTPS and
|
||||
expose the backend). If it's missing, every report and the Servers page show a warning.
|
||||
|
||||
It changes nothing on the central server: no frps edits, no firewall changes, no FastPanel.
|
||||
|
||||
## 7. New install
|
||||
|
||||
**Hub → Customers → *customer* → New install** (or a new customer first):
|
||||
|
||||
1. **Form:** products (Events now; Accounts / Website / App as they ship), **short name**
|
||||
(`hope`, `cross`, `bridge`; checked unique across all servers and used for folders, PM2 names,
|
||||
database names and tunnel names), address (own domain or Cross Code address; per product the
|
||||
domain is shown, e.g. `events.hope.co.za`), timezone, **site admin name and email**, and the
|
||||
**server**: a list showing each server's free RAM, free disk, CPU load over the last day, sites
|
||||
on it and agent status, with servers below the free-RAM threshold (default 1.5 GB per product
|
||||
install) marked as not recommended.
|
||||
2. **Checks in the Hub** before the job exists: the server agent is online, server config and
|
||||
defaults are complete for the release's env template (§5), ports are free, the short name is
|
||||
free, DNS (warning only).
|
||||
3. The Hub **creates the licence(s)**, assigns ports, and queues the `install` job.
|
||||
4. **On the server** (`crosscode install`, each step in the report):
|
||||
checks (disk, RAM, Node, PostgreSQL reachable) → folder `/var/www/sites/<church>/<product>`
|
||||
→ checkout of the release tag → `.env` files (§5) → database and role → install dependencies
|
||||
→ migrations → build → PM2 processes on the given ports, `pm2 save` → `church.env` → frpc
|
||||
block and reload (§6) → local `/health` answers with the right version → the site registers
|
||||
with the Hub using its licence.
|
||||
5. **On failure** anything this job created is removed again (PM2 processes, frpc block, database
|
||||
and role, folder), in reverse order, and the report says what failed and what was cleaned up.
|
||||
6. **Central check:** the Hub asks the central agent to `check-ports` for both ports.
|
||||
7. **Emails to Joshua:** the install report and, if everything passed, the **FastPanel email**
|
||||
(§11). The install shows **Waiting for proxy** in the Hub.
|
||||
8. Joshua sets up FastPanel by hand and clicks **Proxy done**. The Hub checks the public URLs
|
||||
(§5) and marks it **Live**, or says what didn't answer. When Live, the site admin gets the
|
||||
welcome email with the setup-wizard link.
|
||||
|
||||
## 8. Updates: scheduled, emergency, batch
|
||||
|
||||
**Scheduled** (Hub → Install → Schedule update, or Updates → Schedule):
|
||||
|
||||
- Joshua picks the release and the time. The time must be **at least 4 hours away**; the Hub
|
||||
refuses anything sooner. A warning (not a block) for Sunday 06:00–13:00 and for days with an
|
||||
event of that church running (from the stats).
|
||||
- Straight away: an email to the site admin(s) ("Cross Events will be updated on Thursday 20
|
||||
November at 14:00; the site will be unavailable for a few minutes", plus the what's-new
|
||||
summary), and a **banner** through the existing Notices, starting now and ending after the
|
||||
update. A reminder email 1 hour before.
|
||||
- At the time the Hub releases the `deploy` job (`notBefore` = the time, `expiresAt` = 15 min
|
||||
later). If the agent doesn't pick it up in that window the update is **missed**: nothing runs,
|
||||
Joshua is emailed, and a new time needs a new notice.
|
||||
- Afterwards: the report to Joshua; to the site admin "Updated to 2.4.1" with what's new, or
|
||||
"The update didn't go through; the site is running the previous version" if it rolled back.
|
||||
- Cancelling or moving it updates the banner and emails the site admin.
|
||||
|
||||
**Emergency:** the same, with at least **15 minutes'** notice, a required reason (shown in the
|
||||
email and banner), and 2FA again. Meant for security fixes and broken releases.
|
||||
|
||||
**Batch:** "every Events install to 2.4.1" (or a selection), at one time with one notice. The
|
||||
Hub runs them **one at a time** (on each server, and across servers) and **stops the batch at the
|
||||
first failure**, emailing Joshua; the remaining installs get a "postponed" email and their banner
|
||||
is removed.
|
||||
|
||||
**Rules carried over:** the release's requirements (`— requires accounts 1.2.0`, decision 24)
|
||||
are checked in the Hub before scheduling and again by the script; each product rolls back on its
|
||||
own (§8 of the roadmap). A church's **Request update** now shows as "Requested" next to
|
||||
Schedule update. Moving a 2.1.x install to 2.2.0 is an `adopt` job, scheduled the same way.
|
||||
|
||||
## 9. Uninstall
|
||||
|
||||
Hub → Install → **Uninstall** (2FA again, type the short name):
|
||||
|
||||
1. Joshua picks the date (at least 4 hours away, default 7 days) and the site admin is emailed
|
||||
and gets a banner, as for an update.
|
||||
2. **Export first** (`crosscode export`): the church's data as **JSON in a format of our own**,
|
||||
`crosscode.church-export/1` (defined in `packages/contracts`): events, sessions, ticket types,
|
||||
registrations and attendees, people (name, email, phone, consent), payments (amounts, dates,
|
||||
references), check-ins, rosters, settings, and the uploads as files next to it. Written in
|
||||
plain terms with our own field names; **no database tables, Prisma models or internal ids**.
|
||||
One ZIP: `export.json`, `uploads/`, `README.txt` explaining the fields.
|
||||
3. The agent **encrypts the ZIP with a random key** and uploads it to the Hub. The key is only in
|
||||
the link emailed to the site admin (`…/download/<id>#<key>`; the part after `#` is never sent
|
||||
to the Hub), so the Hub stores the file but can't read it, and the roadmap's "the Hub only sees
|
||||
aggregate stats" still holds. The download page decrypts it in the browser. Available for
|
||||
**30 days**.
|
||||
4. **Uninstall:** final database backup and an archive of the uploads to
|
||||
`/var/www/sites/.removed/<church>-<date>/` (root only) → PM2 processes removed, `pm2 save` →
|
||||
the frpc block removed and reloaded → database and role dropped → the site folder removed →
|
||||
the licence released in the Hub.
|
||||
5. **Emails:** to the site admin, the download link and the date it expires; to Joshua, the report
|
||||
and "remove the FastPanel site `events.hope.co.za` (ports 20010, 20011)".
|
||||
6. **After 30 days:** a `purge` job deletes the kept backup and archive, and the Hub deletes the
|
||||
encrypted export. Joshua is emailed **3 days before**, and can extend it once by 30 days. The
|
||||
Hub keeps a record that it was deleted, and when (for POPIA).
|
||||
|
||||
## 10. Servers page and alerts
|
||||
|
||||
**Hub → Servers:** one card per server (church servers and the central one).
|
||||
|
||||
- **Every 60 s** the agent sends: RAM (total, used, available, swap), CPU (% and 1/5/15-minute
|
||||
load), disk per mount (used, free), uptime, versions (OS, Node, PostgreSQL, frpc, agent), frpc
|
||||
running, and **per PM2 process** (church, product, memory, CPU, restarts, uptime).
|
||||
- **Kept:** per minute for 7 days, per hour (average and peak) for 90 days.
|
||||
- **Server page:** RAM and CPU charts for 24 h / 7 days / 90 days with **spikes marked** (peaks
|
||||
above 85%), disk bars, the sites on it with each process's memory, recent jobs, the frpc blocks
|
||||
it manages and the ports they use.
|
||||
- **Alerts** (email to Joshua, through the existing alerts, once per incident and again when it
|
||||
clears): agent not heard from for 5 minutes; disk over 85%; RAM over 90% for 5 minutes; a PM2
|
||||
process restarting more than 5 times in 10 minutes; frpc not running; on the central server,
|
||||
`proxy_bind_addr` missing or a managed tunnel not listening.
|
||||
|
||||
## 11. Reports and emails
|
||||
|
||||
All in the Hub (Jobs page, with the full report) and by email:
|
||||
|
||||
| Email | To | When |
|
||||
|-------|----|------|
|
||||
| Install report | Joshua | Every install, success or failure: each step with its time, ports, the central check, the log tail on failure, what was cleaned up |
|
||||
| **FastPanel setup** | Joshua | After a successful install. What *he* does in FastPanel; the agent does none of it (below) |
|
||||
| Welcome | Site admin | When the install is Live |
|
||||
| Update planned / reminder / done / didn't go through | Site admin | §8 |
|
||||
| Update report | Joshua | Every update (and batch summary) |
|
||||
| Missed job | Joshua | A scheduled job wasn't picked up in time |
|
||||
| Uninstall planned / export ready | Site admin | §9 |
|
||||
| Uninstall report, purge in 3 days | Joshua | §9 |
|
||||
| Alerts | Joshua | §10 |
|
||||
|
||||
**The FastPanel email** (an instruction to Joshua, nothing is set up automatically):
|
||||
|
||||
```
|
||||
Set up in FastPanel: events.hope.co.za (Hope Family Church, Cross Events)
|
||||
|
||||
1. Add the site events.hope.co.za.
|
||||
2. Proxy:
|
||||
/api, /uploads, /health, /docs → http://127.0.0.1:20011 (backend)
|
||||
everything else → http://127.0.0.1:20010 (frontend)
|
||||
Pass the Host header and X-Forwarded-For / X-Forwarded-Proto: https.
|
||||
Allow uploads up to 25 MB.
|
||||
3. Turn on Let's Encrypt for the site.
|
||||
4. DNS: events.hope.co.za → <central server IP> (checked: ✅ points there / ⚠ not yet)
|
||||
5. Then click "Proxy done" in the Hub: <link>
|
||||
|
||||
Central check: both ports are open in frps and only on 127.0.0.1.
|
||||
```
|
||||
|
||||
## 12. What changes in each repo
|
||||
|
||||
**`crosscode-platform`**
|
||||
- `apps/agent` (new, `agent@1.0.0`) with its install script and systemd unit.
|
||||
- `scripts/` (the `crosscode` script): `--json` reports on every command, `uninstall`, `export`,
|
||||
`purge`, `restart`, the frpc block handling (lib/frpc.sh), the env template handling
|
||||
(lib/env.sh), ports given on the command line instead of picked.
|
||||
- `apps/events`: `env.template.json`, and the export (`crosscode export` calls a backend script
|
||||
that writes `crosscode.church-export/1`).
|
||||
- `packages/contracts` **1.2.0**: `crosscode.agent-job/1`, `crosscode.agent-report/1`,
|
||||
`crosscode.agent-stats/1`, `crosscode.env-template/1`, `crosscode.church-export/1`.
|
||||
|
||||
**`events-hub` (Hub 2.1.0)**
|
||||
- Models: `Server` (name, mode, public keys, encrypted config, last seen, versions), `ServerStat`
|
||||
(minute and hourly), `Job` (type, args, status, schedule, who, report), `ScheduledUpdate` /
|
||||
batch, `Port` (number, server, install, released at), `Export` (encrypted blob, expires).
|
||||
`Install` gets `serverId`, `shortName`, `status` (installing, waiting-for-proxy, live,
|
||||
uninstalling, removed).
|
||||
- Agent API `/api/agent/v1/*` (join, poll, stats, report, export upload), signed by the agent.
|
||||
- Pages: Servers, Server, New install, Jobs, Updates (schedule, batch), Install defaults, the
|
||||
public export download page.
|
||||
- Emails in §11; alerts in §10; the job-signing key in `.env`.
|
||||
|
||||
**`crosscode-site`:** nothing (this changes how Joshua works, not what churches buy).
|
||||
|
||||
## 13. Testing
|
||||
|
||||
- **Agent:** unit tests for job checking (bad signature, wrong server, expired, replayed,
|
||||
unknown type, an argument with shell characters), and **a test that the running agent has no
|
||||
listening socket** (`ss -ltnp` shows nothing for its process).
|
||||
- **Script:** bats tests for the frpc blocks (append to a file with hand-written entries and
|
||||
comments, remove one block, the other blocks and the hand-written lines unchanged byte for byte;
|
||||
a failed `frpc verify` leaves the file untouched; `[common]` admin lines added only once), the
|
||||
env template (new variable appended, existing lines untouched, required one missing → refused),
|
||||
and the export (a demo church's export validates against `crosscode.church-export/1` and has no
|
||||
internal ids).
|
||||
- **On VMs** (multipass): a "central" VM with frps (with `proxy_bind_addr = 127.0.0.1`) and nginx
|
||||
in place of FastPanel, two church VMs with frpc and a hand-written tunnel, the Hub running
|
||||
locally:
|
||||
1. Join all three. Stats show up.
|
||||
2. New install on church VM 1 → report, FastPanel email, ports open on the central VM, the
|
||||
hand-written tunnel still works. Add the nginx site by hand, Proxy done → Live.
|
||||
3. A second church on VM 2 → different ports, unique tunnel names.
|
||||
4. Schedule an update 4 h ahead (time moved forward in the test) → emails, banner, update,
|
||||
report. Try 3 h → refused. Emergency with 15 min → needs 2FA.
|
||||
5. A broken release → rolled back, "didn't go through" email, batch stopped.
|
||||
6. Hub stopped during a scheduled update → missed, nothing runs, sites keep running.
|
||||
7. Uninstall → export downloads and decrypts in the browser, everything removed, the other
|
||||
church's tunnel and the hand-written one untouched. Purge after 30 days (clock moved).
|
||||
8. A forged job (wrong key) and a replayed job → refused and reported.
|
||||
|
||||
## 14. Rolling it out
|
||||
|
||||
1. Contracts 1.2.0 → the `crosscode` script changes (with Events 2.2.0 out first) → agent 1.0.0
|
||||
→ Hub 2.1.0, each released as usual (branch → merge → tag → push).
|
||||
2. On the central server: add `proxy_bind_addr = 127.0.0.1` to `frps.ini` if it's missing (check
|
||||
that each FastPanel site still answers), then join the central agent.
|
||||
3. Join each church server; set its server config in the Hub. Existing installs are linked to
|
||||
their server and their current ports recorded in the port list (read from `church.env` and
|
||||
`frpc.ini`; nothing is changed).
|
||||
4. First real use: the next new church. Existing churches get their next update through a
|
||||
scheduled update.
|
||||
5. Docs: CHANGELOGs, HANDOVER.md (1d done), `events-hub/deployment.md` (the job-signing key),
|
||||
a server how-to in `apps/agent/README.md`.
|
||||
|
||||
## Decisions
|
||||
|
||||
Agreed with Joshua on 2026-11-12 (ROADMAP-v3 decisions 27–37):
|
||||
|
||||
1. The Hub can install, update and uninstall through an agent on each server, with **signed jobs
|
||||
of fixed types**; the agent only runs the `crosscode` script; uninstall and emergency updates
|
||||
need 2FA again.
|
||||
2. **The agent has no URL or listening port**; it only polls the Hub.
|
||||
3. **Scheduled updates at least 4 hours ahead, emergency updates at least 15 minutes**, with an
|
||||
email and banner to the site admin; Joshua picks the time.
|
||||
4. **Uninstall:** the church gets a **JSON export in our own format** (no internal data
|
||||
structures), encrypted so the Hub can't read it, downloadable for 30 days; backups kept 30
|
||||
days, then deleted.
|
||||
5. All church servers are behind frp to **one central FastPanel server**. The Hub keeps the port
|
||||
list; **frpc.ini is only appended to**; the `[common]` admin lines are added once.
|
||||
6. **FastPanel and DNS are never automated**; Joshua gets an email with what to set up.
|
||||
7. **Database and other server settings are set once per server in the Hub** and sent encrypted
|
||||
with each job; `.env` files are built from a per-product template and only appended to on
|
||||
updates; generated secrets stay on the server.
|
||||
8. A read-only agent on the central server checks ports and `proxy_bind_addr`.
|
||||
|
||||
## Still open
|
||||
|
||||
Nothing.
|
||||
+8
-3
@@ -1,6 +1,6 @@
|
||||
# Handover — Cross Code, all products
|
||||
|
||||
Last updated: 2026-11-05. **The plan is [ROADMAP-v3.md](./ROADMAP-v3.md)** (decided 2026-11-03,
|
||||
Last updated: 2026-11-12. **The plan is [ROADMAP-v3.md](./ROADMAP-v3.md)** (decided 2026-11-03,
|
||||
decisions table near the end). This file says where each phase stands and what comes next.
|
||||
Product-specific notes (where the code lives, known gaps) stay with each product: Cross Events'
|
||||
are in `hope-events/docs/HANDOVER.md` (`apps/events/docs/HANDOVER.md` from Events 2.2.0), and
|
||||
@@ -14,7 +14,7 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05.
|
||||
|
||||
| Repo | What |
|
||||
|------|------|
|
||||
| `joshua/crosscode-platform` (this one) | Contracts, the apps (Accounts, Website, App, and Events from 2.2.0), shared packages, the `crosscode` script (from Events 2.2.0), and these docs |
|
||||
| `joshua/crosscode-platform` (this one) | Contracts, the apps (Accounts, Website, App, and Events from 2.2.0), shared packages, the `crosscode` script (from Events 2.2.0), the server agent (`apps/agent`, phase 1d), and these docs |
|
||||
| `joshua/hope-events` | Cross Events up to 2.1.x. Archived after the 2.2.0 move |
|
||||
| `joshua/events-hub` | Cross Code Hub. Its own server, its own `scripts/deploy.sh` |
|
||||
| `joshua/crosscode-site` | crosscode.co.za. Updated at the end of every phase that changes what it says (ROADMAP §9) |
|
||||
@@ -27,6 +27,7 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05.
|
||||
| 1 | Hub 2.0.0 | **Released** 2026-11-05 (`events-hub` `v2.0.0`; `docs/HUB-2.0-PLAN.md` there). **Not deployed yet:** `events-hub/deployment.md` → "Upgrading to 2.0" (run `check-hub2` first). crosscode-site's address prices are merged, **not deployed yet** |
|
||||
| 1b | Cross Events 2.2.0: moves here as `apps/events`, `crosscode` script | **Next.** Planned and decided in [EVENTS-2.2-PLAN.md](./EVENTS-2.2-PLAN.md) |
|
||||
| 1c | Cross Events 2.3.0: authenticator app 2FA (TOTP + recovery codes) | Not started (`TwoFactorMethod` enum in Events' `prisma/schema.prisma`) |
|
||||
| 1d | Agent 1.0.0 + Hub 2.1.0 (+ contracts 1.2.0): server agent, installs / scheduled updates / uninstalls from the Hub, server stats | **Planned** 2026-11-12 in [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md) (ROADMAP decisions 27–37). Starts after 1b; can run alongside 1c |
|
||||
| 2 | Cross Events 2.4.0: protocol v2, Linked products, event feed, `packages/seo` | Not started. Events keeps speaking protocol 1 until then |
|
||||
| 3–7 | Accounts 1.0.0 → Website 1.0.0 → Events 3.0.0 → App 1.0.0 → App 1.x | Not started |
|
||||
|
||||
@@ -37,7 +38,11 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05.
|
||||
2. **Cross Events 2.2.0**: [EVENTS-2.2-PLAN.md](./EVENTS-2.2-PLAN.md). Decided 2026-11-05:
|
||||
`adopt` builds next to the running install and switches over, and `adopt --undo` is built
|
||||
(ROADMAP decisions 25–26).
|
||||
3. **Cross Events 2.3.0** (authenticator app 2FA), then 2.4.0, as in ROADMAP §9.
|
||||
3. **Agent 1.0.0 + Hub 2.1.0** (phase 1d): [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md). Before the
|
||||
first install through it: check `proxy_bind_addr = 127.0.0.1` in `frps.ini` on the central
|
||||
server (worth doing now: without it, tunnel ports such as `:5000` may be reachable straight from
|
||||
the internet), and set each server's database settings and `TRUST_PROXY` in the Hub.
|
||||
4. **Cross Events 2.3.0** (authenticator app 2FA), then 2.4.0, as in ROADMAP §9.
|
||||
|
||||
## Things to remember across phases
|
||||
|
||||
|
||||
+69
-8
@@ -20,6 +20,12 @@ updates"), the Hub tracks releases per product (§5), and the contracts reach th
|
||||
committed to its repo instead of through a package registry (§2). Cross Events versions in §9
|
||||
moved up by one (the move is 2.2.0). See decisions 21–24.
|
||||
|
||||
**Updated 2026-11-12:** a **server agent** on each server installs, updates and uninstalls
|
||||
church sites on signed jobs from the Hub (phase 1d, [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md)).
|
||||
The Hub can now trigger changes, but no site depends on it to keep running and the agent only
|
||||
polls (it has no URL or open port). Rule 2 (§1), §5, §8 and §9 are updated; decisions 27–37
|
||||
change decisions 10 and 23.
|
||||
|
||||
---
|
||||
|
||||
## Contents
|
||||
@@ -61,8 +67,10 @@ own process and its own database. The Hub only sees aggregate stats and licences
|
||||
**Rules**
|
||||
|
||||
1. **No shared databases, ever.** Products only talk over HTTPS using documented contracts (§2).
|
||||
2. **The Hub is never on the runtime path.** It stays read-only toward installs: installs push to
|
||||
it, it never reaches into them, and a Hub outage switches nobody off.
|
||||
2. **The Hub is never on the runtime path.** Installs push to it, it never connects to them, and
|
||||
a Hub outage switches nobody off. **Changes to a site are made by the server agent** on its
|
||||
server (from phase 1d): it polls the Hub for **signed jobs of fixed types**, only ever runs the
|
||||
`crosscode` script, and has no URL or listening port of its own (§8, decisions 27–37).
|
||||
3. **One maintained codebase per product.** Each church runs a tagged release of it. Nothing is
|
||||
built specially for one church.
|
||||
4. **The origin of a record owns it.** Copies elsewhere are read-only and link back to the origin.
|
||||
@@ -317,7 +325,14 @@ product's customers, licences, stats, AI usage and billing.
|
||||
before it deploys (§8). By default products never need to update together: the contract rules
|
||||
(§2) and protocol v1 + v2 make that possible, so requirements are only for real breaking moves
|
||||
such as Events 3.0.
|
||||
- **Request update** works for every product. The Hub still never runs anything on an install.
|
||||
- **Request update** works for every product. From Hub 2.1.0 Joshua answers it with a
|
||||
**scheduled update** (at least 4 hours' notice to the site admin, emergency at least 15
|
||||
minutes), carried out by the server agent (§8).
|
||||
- **Servers (Hub 2.1.0):** every server's agent, its stats (RAM, CPU, disk, spikes, PM2
|
||||
processes per church), alerts, the frp port list, server configuration (database and other
|
||||
per-server `.env` values, stored encrypted), new installs with a choice of server, jobs with
|
||||
emailed reports, and uninstalls with a 30-day encrypted export for the church
|
||||
([AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md)).
|
||||
- **Domain:** `hub.crosscode.co.za`. `events-hub.crosscode.co.za` stays as an alias so existing
|
||||
installs keep working.
|
||||
- Dashboard: installs per product, versions behind latest, health, licence status, monthly
|
||||
@@ -463,7 +478,7 @@ browsed without an account. Personal actions need sign-in through Accounts.
|
||||
|
||||
| Repo | What |
|
||||
|------|------|
|
||||
| `joshua/crosscode-platform` | **Every product a church installs**, and what they share: `apps/events` (Cross Events, moved here in 2.2.0), `apps/accounts`, `apps/website`, `apps/app`, `packages/contracts`, `packages/ui`, `packages/seo` (§11), and the `crosscode` script (`scripts/`) |
|
||||
| `joshua/crosscode-platform` | **Every product a church installs**, and what they share: `apps/events` (Cross Events, moved here in 2.2.0), `apps/accounts`, `apps/website`, `apps/app`, `packages/contracts`, `packages/ui`, `packages/seo` (§11), the `crosscode` script (`scripts/`) and the server agent (`apps/agent`, phase 1d) |
|
||||
| `joshua/hope-events` | Cross Events up to 2.1.x. **Archived** after the move (read-only, README points to the monorepo), and kept so installs that haven't moved yet can still deploy or roll back a `v2.1.x` tag |
|
||||
| `joshua/events-hub` | Cross Code Hub (renamed in the UI; repo name can stay). Stays on its own: it runs on its own server, is never installed for a church, and is deployed with its own `scripts/deploy.sh`. Gets the contracts as a committed file (§2) |
|
||||
| `joshua/crosscode-site` | The marketing site (crosscode.co.za). Updated at the end of every phase that changes what's on sale or what a product does (see §9, "Updating crosscode-site") |
|
||||
@@ -473,8 +488,10 @@ Playwright, Jest), so code and habits carry over.
|
||||
|
||||
### Releases and updates
|
||||
|
||||
**What doesn't change:** git is how code reaches a server, the Hub only records releases and
|
||||
passes on update requests (it never deploys, §1 rule 2), and Joshua runs the deploys.
|
||||
**What doesn't change:** git is how code reaches a server, and every install, update and
|
||||
uninstall goes through the `crosscode` script. Until phase 1d Joshua runs it by hand; from
|
||||
phase 1d the **server agent** runs it on signed jobs from the Hub (below), and it still works by
|
||||
hand over SSH.
|
||||
|
||||
**Releasing.** Each product is tagged on its own in the monorepo: `events@2.2.0`,
|
||||
`accounts@1.0.0`, `website@1.0.0`, `app@1.0.0` (and `contracts@1.0.0`, which nobody installs).
|
||||
@@ -564,6 +581,30 @@ would be down for the whole build. The steps are in
|
||||
The PM2 names, nginx config (it only proxies to ports), database and its name stay the same.
|
||||
Installs that haven't moved yet keep working with `deploy.sh` on the archived repo.
|
||||
|
||||
### Server agent (phase 1d)
|
||||
|
||||
Planned in [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md). In short:
|
||||
|
||||
- **`crosscode-agent`** (`apps/agent`, systemd) on every church server. It **only polls** the Hub
|
||||
(outgoing HTTPS; no URL, no listening port), sends server stats every minute, and runs
|
||||
**signed jobs of fixed types** (install, add product, deploy, adopt, restart, export,
|
||||
uninstall), each by calling the `crosscode` script. Uninstall and emergency updates need 2FA
|
||||
again in the Hub.
|
||||
- **Servers are behind frp** to one central FastPanel server. The Hub hands out the tunnel ports
|
||||
(unique across all servers). The agent **only appends** marked blocks to `/etc/frp/frpc.ini` and
|
||||
removes only its own blocks. A read-only agent on the central server checks the ports are open
|
||||
and only bound to `127.0.0.1`. **FastPanel and DNS are never automated:** Joshua gets an email
|
||||
with what to set up.
|
||||
- **`.env` files** are built from a per-product template: database and other server values set
|
||||
**once per server in the Hub** (sent encrypted), platform defaults, the install's domain and
|
||||
ports, and secrets generated on the server (never sent anywhere). Updates only append new
|
||||
variables.
|
||||
- **Updates** are scheduled at least 4 hours ahead (emergency 15 minutes) with an email and banner
|
||||
to the site admin, and run one install at a time. **Uninstall** gives the church a JSON export
|
||||
in our own format (encrypted, 30 days) and keeps backups for 30 days.
|
||||
- New `crosscode` commands for it: `uninstall`, `export`, `purge`, `restart`, and `--json`
|
||||
reports on every command.
|
||||
|
||||
---
|
||||
|
||||
## 9. Build order
|
||||
@@ -574,6 +615,7 @@ Installs that haven't moved yet keep working with `deploy.sh` on the archived re
|
||||
| 1 | **Hub 2.0.0** | Cross Code Hub: products, licence per product, per-customer billing with bundles, protocol v2 (v1 still accepted), sibling discovery, AI usage per product, **releases per product with requirements** (§5), contracts as a committed file (§2). **Released 2026-11-05** (crosscode-site address prices merged) | Must exist before a second product is sold, and before Events releases come from the monorepo |
|
||||
| 1b | **Cross Events 2.2.0** | **Moves into the monorepo** as `apps/events`, with its git history (merged in, not copied); its packages become workspaces; the **`crosscode` script** (`deploy`, `status`, `install`, `adopt`; §8) replaces `scripts/deploy.sh` and `scripts/install.sh`; every install moved with `crosscode adopt`; `hope-events` archived. No feature changes, so a problem is clearly the move | Before more products, so every product is released and deployed the same way from the start |
|
||||
| 1c | **Cross Events 2.3.0** | **Authenticator app 2FA** (TOTP) next to email and WhatsApp, with recovery codes; admins can see which method each staff account uses | Independent and useful now; the Accounts migration later carries it over |
|
||||
| 1d | **Agent 1.0.0 + Hub 2.1.0** (contracts 1.2.0) | **Server agent** on every server (polls only, signed jobs, runs the `crosscode` script); Hub: Servers page with stats and alerts, server configuration, frp port list, New install with a choice of server, scheduled / emergency / batch updates with notice to the church, uninstall with an encrypted JSON export, emailed reports. [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md) | Right after 1b (it runs the `crosscode` script), so every new church from then on is installed from the Hub. Can run alongside 1c |
|
||||
| 2 | **Cross Events 2.4.0** | Reports `product: "events"` (protocol v2), **Linked products**, the event feed (publish + show events from siblings), `packages/seo` (§11) with canonical links for imported events | Small; ready for when the Website exists |
|
||||
| 3 | **Accounts 1.0.0** | OIDC provider, sign-in, 2FA (email, WhatsApp, authenticator app), password reset, consent and notice, export / delete fan-out, admin, install script | The Website is built on it from day one |
|
||||
| 4 | **Church Website 1.0.0** | Public site with the **four templates** (§6), admin, simple events + feed, SEO (§11), Hope Family Church as the first church, then a second test church on a different template to prove it repeats | More churches need a website than an app; it also covers the "website only" case |
|
||||
@@ -596,6 +638,7 @@ At the end of each phase, update it in the same sitting and deploy it (its `DEPL
|
||||
| 1 Hub 2.0.0 | Bundle discounts (15% / 25%) and the address add-on (own domain R0 · Cross Code address R10 · domain included R25) in `src/data/pricing.mjs`, worded so it's clear the Website and App are still coming. Legal pages if billing terms change |
|
||||
| 1b Events 2.2.0 | Nothing (the move doesn't change what Cross Events does) |
|
||||
| 1c Events 2.3.0 | Authenticator app 2FA in the Cross Events feature list (`src/data/cross-events.mjs`) and the security / POPIA wording |
|
||||
| 1d Agent 1.0.0 + Hub 2.1.0 | Nothing (it changes how installs are run, not what churches buy) |
|
||||
| 2 Events 2.4.0 | "Works with the Church Website and App" (event feed, Linked products) on the Cross Events page, as "coming" until those ship |
|
||||
| 3 Accounts 1.0.0 | "One sign-in for all your church's products" on the Products page. Accounts isn't sold, so no page or price of its own |
|
||||
| 4 Website 1.0.0 | **Church Website goes on sale:** `status: "available"` and its own page in `src/data/products.mjs` / `src/pages/products/`, R99 + storage add-on in `pricing.mjs`, **a showcase of the four templates** (a page per template with screenshots of the same demo church in each), screenshots from a demo church (fictional people only), FAQ, home page, contact form product list, `SoftwareApplication` JSON-LD (§11) |
|
||||
@@ -620,6 +663,8 @@ Hub never disagree. Run the site's tests and build before deploying it.
|
||||
- crosscode-site shows the product as available, with its page, prices and screenshots.
|
||||
- The §11 SEO checks pass (Playwright and Lighthouse), for the Website in all four templates.
|
||||
- A database backup is restored successfully.
|
||||
- (From phase 1d) The product installs, updates and uninstalls through the server agent from the
|
||||
Hub, with its `.env` template complete and its uninstall export in `crosscode.church-export/1`.
|
||||
|
||||
---
|
||||
|
||||
@@ -757,7 +802,7 @@ ranking.
|
||||
| 7 | Hub | Becomes **Cross Code Hub**: every product's licences, stats, AI and billing in one place, still read-only toward installs |
|
||||
| 8 | Stack | Next.js + Express + Prisma + PostgreSQL + pg-boss, as Cross Events (not the specs' React + Vite, Redis + BullMQ, Socket.IO) |
|
||||
| 9 | Pricing | Income bands per product, bundles 15% (two) / 25% (three); every App module for every church (not the website spec's Starter / Growth / Pro tiers) |
|
||||
| 10 | Hub provisioning | Scripts Joshua runs; the Hub records installs but never deploys or rolls back (not the specs' automated provisioning) |
|
||||
| 10 | Hub provisioning | Scripts Joshua runs; the Hub records installs but never deploys or rolls back (not the specs' automated provisioning). **Changed 2026-11-12** by decisions 27–37: the server agent runs the script on signed jobs from the Hub |
|
||||
| 11 | Installs per product | **One install of each product per customer**; the Hub refuses a second licence for the same product |
|
||||
| 12 | House of Prayer defaults | 1-hour slots, book up to 12 months ahead, recurring bookings for up to 12 months from the day of booking, no cancelling within 8 hours of the slot |
|
||||
| 13 | App notifications | **Email and WhatsApp only.** No push notifications |
|
||||
@@ -775,7 +820,7 @@ ranking.
|
||||
|---|----------|----------|
|
||||
| 21 | Repos | **Cross Events moves into `crosscode-platform`** (`apps/events`, Events 2.2.0, history kept); `hope-events` is archived. The Hub stays in its own repo (own server, never installed for a church) |
|
||||
| 22 | Contracts for the Hub | A packed release **committed to the Hub's repo** (`file:vendor/…`), not Gitea's package registry, so no deploy depends on a registry or a token (§2) |
|
||||
| 23 | Deploying | **One `crosscode` script** for every product, with every church under `/var/www/sites/<church>/<product>`: `crosscode deploy <church> [events] [website] [app]` always updates Accounts and then the products named, in a fixed order, each rolled back on its own if it fails. Still git checkouts of tags, still run by Joshua; the Hub never deploys (§8) |
|
||||
| 23 | Deploying | **One `crosscode` script** for every product, with every church under `/var/www/sites/<church>/<product>`: `crosscode deploy <church> [events] [website] [app]` always updates Accounts and then the products named, in a fixed order, each rolled back on its own if it fails. Still git checkouts of tags, still run by Joshua; the Hub never deploys (§8). **Changed 2026-11-12** by decision 27: from phase 1d the server agent runs it on jobs from the Hub |
|
||||
| 24 | Product requirements | A release can require a minimum version of a sibling (`— requires accounts 1.2.0` in `WHATS-NEW.md`). The Hub **disables Request update** until it's met, and the script refuses to deploy without it (§5, §8) |
|
||||
|
||||
**Added 2026-11-05**
|
||||
@@ -785,6 +830,22 @@ ranking.
|
||||
| 25 | Moving existing Events installs | `crosscode adopt` **builds the new checkout next to the running install and switches PM2 over** (seconds of downtime) instead of moving the running folder; the old folder is untouched, so a failure just starts it again (§8) |
|
||||
| 26 | Undoing a move | **`crosscode adopt <church> --undo`** goes back to the old folder and its 2.1.x while it's kept (14 days) (§8) |
|
||||
|
||||
**Added 2026-11-12** (phase 1d, [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md))
|
||||
|
||||
| # | Question | Decision |
|
||||
|---|----------|----------|
|
||||
| 27 | Deploying from the Hub | A **server agent** on every server runs installs, updates and uninstalls on **signed jobs of fixed types** from the Hub, always through the `crosscode` script (which still works by hand). Changes decisions 10 and 23; rule 2 (§1) reworded |
|
||||
| 28 | The agent's exposure | **It has no URL or listening port.** It only polls the Hub over outgoing HTTPS |
|
||||
| 29 | Confirming risky jobs | Uninstall and emergency updates need **2FA again** in the Hub; uninstall also needs the short name typed |
|
||||
| 30 | Notice before updates | **Scheduled: at least 4 hours**; **emergency: at least 15 minutes**, with a reason. The site admin gets an email and a banner straight away. Joshua picks the time. A job the agent misses doesn't run late |
|
||||
| 31 | Uninstall | The church gets a **JSON export in our own format** (`crosscode.church-export/1`, no internal tables or ids), encrypted so the Hub can't read it, downloadable for **30 days**. The final backup is kept 30 days, then deleted (Joshua emailed 3 days before) |
|
||||
| 32 | frp | Every church server is behind frp to **one central FastPanel server**. The Hub keeps the **port list** (unique across servers, same number local and remote), tunnels are named `<church>-<product>-web/-api` |
|
||||
| 33 | frpc.ini | **Only appended to**, in marked blocks, checked with `frpc verify`, with a backup first; uninstall removes only its own block. `admin_addr = 127.0.0.1` / `admin_port = 7400` added once to `[common]` so frpc reloads without dropping other sites |
|
||||
| 34 | FastPanel and DNS | **Never automated.** Joshua gets an email with the proxy rules, Let's Encrypt and DNS to set up, then clicks *Proxy done*; the Hub checks the public URLs |
|
||||
| 35 | Central server | A **read-only** agent on the central server checks the ports are allowed and open and that frps binds tunnels to `127.0.0.1` (`proxy_bind_addr`). It changes nothing there |
|
||||
| 36 | `.env` configuration | **Database and other server values are set once per server in the Hub**, stored encrypted and sent encrypted to that server's agent with each job. `.env` files come from a per-product template; updates only append new variables; secrets are generated on the server and never leave it |
|
||||
| 37 | Server stats | Every agent reports RAM, CPU, disk and PM2 processes every minute; the Hub keeps minutes for 7 days and hours for 90 days, shows spikes, and emails alerts |
|
||||
|
||||
## Still open
|
||||
|
||||
Nothing. crosscode-site is updated at the end of each phase (§9, "Updating crosscode-site"); the
|
||||
|
||||
Reference in New Issue
Block a user