Compare commits

..
Author SHA1 Message Date
joshuaandClaude Opus 5.5 eb4e44a735 Release events@2.3.0, contracts@1.3.0 and agent@1.0.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ls1vMdni7EfytGYqQTprME
2026-11-17 22:23:11 +02:00
joshua 3bcde954fb Merge branch 'feat/totp' 2026-11-17 22:23:11 +02:00
joshua 328b2ce11a Merge branch 'feat/agent-1.0' 2026-11-17 22:15:31 +02:00
joshuaandClaude Opus 5.5 f4955bee9d Fetch tags with --force: the release tag on the server wins
Found in the VM test: when a tag had moved (a re-tagged release), a church's
fetch refused it and every later deploy failed its checks. The deploy checks
out the tag the Hub asked for, so the remote's tag is the one to trust.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ls1vMdni7EfytGYqQTprME
2026-11-17 19:08:53 +02:00
joshuaandClaude Opus 5.5 570f2dda5a PM2 runs under its own systemd unit, never as a child of the agent
Found in the VM test: a site's PM2 daemon started from the agent's install
job belonged to the agent's service, so stopping or updating the agent took
down every site it had installed. Before any PM2 command as a site user,
the scripts now make sure that user's daemon runs under pm2-<user> (moving
one that runs elsewhere, keeping its process list). The agent's stats no
longer start an empty daemon by asking one that isn't running. Uninstall
stops the unit before removing it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 16:40:12 +02:00
joshuaandClaude Opus 5.5 19709c2c75 frpc: restart it the first time the admin lines are added (a reload can't work yet)
Found in the VM test: the new install's tunnels never opened, because
frpc only reads admin_addr when it starts, so the first reload failed.
That first time (and whenever a reload fails), frpc is restarted under
systemd and the report says so; later changes reload as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:48:28 +02:00
joshuaandClaude Opus 5.5 f3f6dc8f7a Events env template: HUB_PUBLIC_KEY in the defaults, for a Hub other than production
Without it a site installed from a staging Hub (or the VM test's Hub)
can't check its licence. Blank in production, so it isn't written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:35:50 +02:00
joshuaandClaude Opus 5.5 0d7e6b52d2 agent install / self-update: install the root's build tools too
Found in the VM test: npm ci -w packages/contracts -w apps/agent skipped
the monorepo root's devDependencies (tsup, typescript), so the agent
couldn't be built. --include-workspace-root brings them in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:35:13 +02:00
joshuaandClaude Opus 5.5 2479cbbd0a deploy: after a failed migration, mark it rolled back so later deploys aren't blocked
Found in the VM test: Prisma records a failed migration and then refuses
every later migrate deploy (P3009), so one broken release left a church
unable to update. On rollback, failed migrations the previous release
doesn't have are marked rolled back (prisma migrate resolve); the
backup still has the database as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:30:18 +02:00
joshuaandClaude Opus 5.5 c8d46ef2d9 adopt: apply the release's new migrations in the switch-over (events@2.2.0 has one)
Found in the VM test: adopt refused events@2.2.0 because the AI tokens
migration made it into the release, and adopt only moved installs with
no migrations. Now the migrations the release has and the old install
doesn't run after the backup, while neither version runs; a failure
starts the old install again. They're recorded (EVENTS_ADOPT_MIGRATIONS)
so adopt --undo can still go back past them (they only add columns).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 15:15:09 +02:00
joshuaandClaude Opus 5.5 ee65ff358c Events 2.3.0 (phase 1c): two-factor with an authenticator app, recovery codes
- TOTP (RFC 6238) next to email and WhatsApp: QR code and key to set up,
  the app's first code confirms it, a code can't be used twice, nothing
  is sent. The secret is stored encrypted.
- 10 single-use recovery codes whenever two-factor is turned on, shown
  once; accepted at sign-in in place of any code; a new set on request.
- Admins see each account's method; Reset 2FA clears the app and codes.
- Profile & Security, sign-in and Users pages; API types; 10 new tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 14:55:22 +02:00
joshuaandClaude Opus 5.5 b207d17db8 WIP Events 2.3.0: TOTP util, schema and migration for authenticator-app 2FA
Not wired up yet: twoFactor.js, login, enable/confirm/disable, recovery
codes endpoints and the frontend still to do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:37:52 +02:00
joshuaandClaude Opus 5.5 92869d2683 crosscode: envtpl as .cjs (Node CommonJS for the root lint)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:26:00 +02:00
joshuaandClaude Opus 5.5 d61b3e5978 Docs: agent 1.0 built (changes while building), handover, changelogs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 13:19:41 +02:00
joshuaandClaude Opus 5.5 1eeaf4d94f Agent 1.0.0: polls the Hub for signed jobs, runs crosscode, sends reports and stats
apps/agent (docs/AGENT-1.0-PLAN.md §3-4, §10): listens on nothing; checks
every job (pinned Hub key, server, time window, replays for 90 days,
type per mode, strict arguments) before running crosscode with an
argument array; opens the sealed config into a root-only file only for
the job; redacts log lines; keeps unsent reports in an outbox; encrypts
exports before upload; check-ports on the central server (read-only);
self-update with automatic revert. One bundled dist/cli.js, a systemd
unit and the install script that joins a server with a one-time token.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:58:08 +02:00
joshuaandClaude Opus 5.5 e9e9b5a97f crosscode: --json reports, --config, env templates, frpc tunnels, restart/export/uninstall/purge
- Every command takes --json (crosscode.script-report/1 on stdout, the
  log on stderr) and --config (the .env values by layer, from the agent).
- .env files come from the release's env.template.json (lib/envtpl.js):
  written whole on install; on deploy only new variables are appended,
  and a new required one with no value stops the deploy at the checks.
- install --frp: Hub-given ports, a marked frpc.ini block (backup,
  frpc verify, reload; [common] admin lines added once), no nginx.
  A database on another server via DB_ADMIN_URL. Refuses an existing
  database or role. Everything the run made is removed if it fails.
- restart, export (crosscode.church-export/1 ZIP), uninstall (final
  backup, uploads and .env kept 30 days) and purge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:50:19 +02:00
joshuaandClaude Opus 5.5 350ab9f4e8 Events: env template, church export, register with HUB_LICENCE_KEY
- apps/events/env.template.json (crosscode.env-template/1): every .env
  variable with its layer (server, defaults, install, generated).
- scripts/export-church.ts writes crosscode.church-export/1 (export-only
  references, no internal ids or secrets) for crosscode export.
- Installs made from the Hub register by themselves on first start when
  HUB_LICENCE_KEY is set (retries while the Hub is unreachable).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:06:01 +02:00
joshuaandClaude Opus 5.5 90a98a4306 contracts 1.3.0: agent jobs, reports, stats, join/poll, env templates, church export
Jobs of fixed types with strict argument schemas, signed by the Hub with
Ed25519 over canonical JSON. Agent requests signed with the server's
Ed25519 key. Config sealed to the server's X25519 key (X25519 + HKDF +
AES-256-GCM; Node has no XSalsa20). Exports encrypted with a key that
only travels in the download link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-17 12:02:24 +02:00
98 changed files with 8437 additions and 172 deletions
+35 -1
View File
@@ -2,7 +2,41 @@
Each package and app has its own section and its own tags (`contracts@1.0.0`, `accounts@1.0.0`, …).
## Unreleased
## events@2.3.0, contracts@1.3.0, agent@1.0.0 - 2026-11-17
Phase 1c (authenticator-app sign-in) and phase 1d (docs/AGENT-1.0-PLAN.md): installs, updates and
uninstalls from the Hub (Hub 2.1.0) through the agent. Tested end to end on a VM (church server +
central server + Hub, AGENT-1.0-PLAN §13 and EVENTS-2.2-PLAN §6).
### Events (2.3.0)
- Two-factor with an authenticator app, and recovery codes. The env template, the church export
and registering by itself from `HUB_LICENCE_KEY` for installs from the Hub. Details in
`apps/events/CHANGELOG.md`.
### Contracts (1.3.0)
- `crosscode.agent-job/1` (fixed job types, strict arguments), `crosscode.agent-report/1`,
`crosscode.script-report/1`, `crosscode.agent-stats/1`, the agent join / poll requests,
`crosscode.env-template/1` and `crosscode.church-export/1`.
- `@crosscode/contracts/signing`: Ed25519 jobs (over canonical JSON) and agent requests, config
sealed to a server (X25519 + HKDF + AES-256-GCM), export encryption.
### Agent (1.0.0)
- `apps/agent`: polls the Hub, checks every job (pinned key, server, time window, replays, type,
arguments) and runs the crosscode script for it; stats every minute; listens on nothing.
Install script, systemd unit, self-update with automatic revert.
### crosscode script
- `--json` reports and `--config` on every command; `.env` files from each release's env template
(appended to, never rewritten, on deploy); `install --frp` with Hub-given ports, a database on
another server, and clean-up on failure; frpc.ini blocks; `restart`, `export`, `uninstall`, `purge`.
- Fixed in the VM test: `adopt` applies a release's new migrations in the switch-over; a failed
migration is marked rolled back so later deploys aren't blocked; frpc is restarted once when its
admin lines are first added; a site's PM2 always runs under its own systemd unit (`pm2-<user>`),
so restarting or updating the agent never stops a site; tags are fetched with `--force`.
## events@2.2.0, contracts@1.2.0 - 2026-11-17
+55
View File
@@ -0,0 +1,55 @@
# Cross Code server agent
Runs on every server (docs/AGENT-1.0-PLAN.md). It polls Cross Code Hub every 30 seconds for
signed jobs, runs the `crosscode` script for them, sends the results, and sends the server's
stats every 60 seconds. **It listens on nothing**: no port, no URL; every request goes out to the Hub.
## Joining a server
Hub → Servers → **Add server** shows a command with a one-time token (valid an hour):
```bash
curl -fsSL https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/apps/agent/deploy/install.sh \
| sudo bash -s -- --hub https://hub.crosscode.co.za --token <token>
```
Add `--central` on the central (FastPanel) server: there the agent is read-only and only runs
`check-ports` and `status`. The script installs Node 24 if needed, the agent in
`/opt/crosscode/agent/releases/<version>` (with `current` pointing at it), the `crosscode` script in
`/opt/crosscode/tools` (church servers), makes the server's two key pairs, joins, and starts
`crosscode-agent.service`.
## On the server
| What | Where |
|------|-------|
| Config (Hub address, server id, the Hub's pinned job key) | `/etc/crosscode/agent.json` |
| Private keys (never leave the server) | `/etc/crosscode/agent-ed25519.pem`, `agent-x25519.pem` |
| Seen job ids (90 days), unsent reports, exports in progress | `/var/lib/crosscode-agent/` |
| A job's opened config, only while it runs | `/run/crosscode-agent/` |
| Logs | `journalctl -u crosscode-agent` |
## Security
- **Jobs are signed** by the Hub's job key (Ed25519, pinned when joining). The agent refuses a
job that's unsigned, for another server, outside its time window, already seen, of an unknown
type, or with an argument outside its schema. Refusals are reported to the Hub.
- **Only the `crosscode` script runs**, with arguments built from the checked job (an array,
never a shell). There's no "run this command" job.
- **Config** (database admin, server settings) arrives sealed to this server's X25519 key. It's
opened into a root-only file for the job and removed straight after.
- **Log lines** sent with a report are filtered for passwords, tokens, keys and licence keys.
- **Exports** are encrypted on the server with a new key; the key travels in the job's report
so the Hub can email the download link, and the Hub doesn't store it.
## Updates
An `update-agent` job builds the new release next to the running one, switches `current`, and
restarts. If the new agent hasn't polled the Hub within 150 seconds, `revert.sh` switches back.
## Development
```bash
npm run build -w apps/agent # dist/cli.js: one file, everything bundled
npm test -w apps/agent # the no-listening-socket test runs on Linux, after a build
```
+23
View File
@@ -0,0 +1,23 @@
# The Cross Code server agent (docs/AGENT-1.0-PLAN.md §3). Root, because the crosscode script it
# runs needs root (PostgreSQL, PM2 for each app user, frpc). It listens on nothing.
[Unit]
Description=Cross Code server agent
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/node /opt/crosscode/agent/current/apps/agent/dist/cli.js run
Restart=always
RestartSec=10
RuntimeDirectory=crosscode-agent
RuntimeDirectoryMode=0700
StateDirectory=crosscode-agent
StateDirectoryMode=0700
Environment=NODE_ENV=production
# Logs: journalctl -u crosscode-agent
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=multi-user.target
+91
View File
@@ -0,0 +1,91 @@
#!/usr/bin/env bash
# Installs the Cross Code server agent and joins this server to the Hub
# (docs/AGENT-1.0-PLAN.md §3). Hub → Servers → Add server shows the command, with a one-time token:
#
# curl -fsSL https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/apps/agent/deploy/install.sh \
# | sudo bash -s -- --hub https://hub.crosscode.co.za --token <token> [--central]
#
# Church servers also get the crosscode script (/opt/crosscode/tools, linked as
# /usr/local/bin/crosscode). The central server (--central) only gets the agent, which is
# read-only there. Running it again updates nothing that's there already; use --force to join again.
set -Eeuo pipefail
REPO="${CROSSCODE_REPO:-https://c-git.crosscode.co.za/joshua/crosscode-platform.git}"
ROOT=/opt/crosscode/agent
TOOLS=/opt/crosscode/tools
HUB="" TOKEN="" MODE=church VERSION="" FORCE=""
die() { printf '!!> %s\n' "$*" >&2; exit 1; }
step() { printf '==> %s\n' "$*"; }
while [ $# -gt 0 ]; do
case "$1" in
--hub) HUB="$2"; shift 2 ;;
--token) TOKEN="$2"; shift 2 ;;
--central) MODE=central; shift ;;
--version) VERSION="$2"; shift 2 ;;
--force) FORCE=--force; shift ;;
*) die "Unknown option $1" ;;
esac
done
[ "$(id -u)" = 0 ] || die "Run as root (sudo bash -s -- …)."
[[ "$HUB" =~ ^https?:// ]] || die "--hub <url> is required."
[[ "$TOKEN" =~ ^[A-Za-z0-9_-]{32,128}$ ]] || die "--token is required (Hub → Servers → Add server)."
[ -z "$VERSION" ] || [[ "$VERSION" =~ ^agent@[0-9]+\.[0-9]+\.[0-9]+$ ]] || die "--version: expected agent@x.y.z."
step "System packages"
export DEBIAN_FRONTEND=noninteractive
apt-get update -q
apt-get install -y -q git curl ca-certificates
if ! command -v node >/dev/null || [ "$(node -p 'process.versions.node.split(".")[0]')" -lt 24 ]; then
step "Node.js 24"
curl -fsSL https://deb.nodesource.com/setup_24.x | bash -
apt-get install -y -q nodejs
fi
if [ -z "$VERSION" ]; then
VERSION="$(git ls-remote --tags --refs "$REPO" 'agent@*' | sed 's#.*refs/tags/##' | sort -t@ -k2 -V | tail -n 1)"
[ -n "$VERSION" ] || die "No agent@x.y.z release yet."
fi
DIR="$ROOT/releases/${VERSION#agent@}"
step "Agent $VERSION in $DIR"
mkdir -p "$ROOT/releases"
chmod 755 /opt/crosscode "$ROOT" 2>/dev/null || true
if [ ! -d "$DIR/.git" ]; then
git clone --quiet --depth 1 --branch "$VERSION" "$REPO" "$DIR"
fi
(
cd "$DIR"
NODE_ENV=development npm ci --no-audit --no-fund --include=dev --include-workspace-root -w packages/contracts -w apps/agent
npm run build -w packages/contracts
npm run build -w apps/agent
)
[ "$(node "$DIR/apps/agent/dist/cli.js" version)" = "${VERSION#agent@}" ] || die "The build doesn't report $VERSION."
ln -sfn "$DIR" "$ROOT/current"
install -m 755 "$DIR/apps/agent/deploy/revert.sh" "$ROOT/revert.sh"
if [ "$MODE" = church ]; then
step "The crosscode script ($TOOLS)"
[ -d "$TOOLS/.git" ] || git clone --quiet "$REPO" "$TOOLS"
ln -sfn "$TOOLS/scripts/crosscode" /usr/local/bin/crosscode
fi
step "Joining the Hub ($HUB)"
mkdir -p /etc/crosscode /var/lib/crosscode-agent
chmod 700 /etc/crosscode /var/lib/crosscode-agent
args=(join --hub "$HUB" --token "$TOKEN")
[ "$MODE" = central ] && args+=(--central)
[ -n "$FORCE" ] && args+=("$FORCE")
if [ -f /etc/crosscode/agent.json ] && [ -z "$FORCE" ]; then
echo " Already joined (/etc/crosscode/agent.json); keeping it."
else
node "$ROOT/current/apps/agent/dist/cli.js" "${args[@]}"
fi
step "systemd"
install -m 644 "$DIR/apps/agent/deploy/crosscode-agent.service" /etc/systemd/system/crosscode-agent.service
systemctl daemon-reload
systemctl enable --now crosscode-agent
systemctl restart crosscode-agent
step "Done: the server shows on Hub → Servers within a minute (journalctl -u crosscode-agent)."
+14
View File
@@ -0,0 +1,14 @@
#!/usr/bin/env bash
# Started by a one-shot systemd timer after an agent update (src/update.ts). If the new agent
# hasn't polled the Hub by now, update-pending.json is still there: switch back to the previous
# release and restart. Copied to /opt/crosscode/agent/revert.sh by install.sh.
set -euo pipefail
ROOT=/opt/crosscode/agent
PENDING=/var/lib/crosscode-agent/update-pending.json
[ -f "$PENDING" ] || exit 0
[ -L "$ROOT/previous" ] || { echo "crosscode-agent revert: no previous release" >&2; exit 1; }
ln -sfn "$(readlink "$ROOT/previous")" "$ROOT/current.tmp"
mv -T "$ROOT/current.tmp" "$ROOT/current"
rm -f "$PENDING"
systemctl restart crosscode-agent
echo "crosscode-agent revert: the new agent didn't poll in time; back on $(readlink "$ROOT/current")" >&2
+7
View File
@@ -0,0 +1,7 @@
module.exports = {
testEnvironment: "node",
testMatch: ["<rootDir>/tests/**/*.test.ts"],
transform: { "^.+\\.ts$": ["ts-jest", { tsconfig: { module: "commonjs", moduleResolution: "node", isolatedModules: true } }] },
moduleFileExtensions: ["ts", "js", "json"],
modulePathIgnorePatterns: ["<rootDir>/dist/"],
};
+18
View File
@@ -0,0 +1,18 @@
{
"name": "@crosscode/agent",
"version": "1.0.0",
"private": true,
"description": "The server agent: runs the crosscode script for signed jobs from Cross Code Hub, reports stats (docs/AGENT-1.0-PLAN.md)",
"license": "UNLICENSED",
"bin": { "crosscode-agent": "./dist/cli.js" },
"scripts": {
"build": "tsup",
"typecheck": "tsc -p tsconfig.json --noEmit",
"test": "jest"
},
"dependencies": {
"@crosscode/contracts": "*",
"zod": "^4.6.7"
},
"engines": { "node": ">=24" }
}
+98
View File
@@ -0,0 +1,98 @@
import { execFile } from "node:child_process";
import fs from "node:fs";
import { paths } from "./paths";
/**
* The central server (AGENT-1.0-PLAN §6): read-only checks of frps for `check-ports`. Nothing on
* the central server is ever changed: no frps edits, no firewall, no FastPanel.
*/
export interface FrpsSettings {
proxyBindAddr: string | null;
allowPorts: string | null;
}
/** frps.ini's [common] proxy_bind_addr and allow_ports (null when not set or no file). */
export function parseFrpsIni(text: string): FrpsSettings {
let inCommon = false;
const out: FrpsSettings = { proxyBindAddr: null, allowPorts: null };
for (const raw of text.split(/\r?\n/)) {
const line = raw.trim();
if (/^\[.*\]$/.test(line)) { inCommon = line === "[common]"; continue; }
if (!inCommon || line.startsWith("#") || line.startsWith(";")) continue;
const m = /^([a-z_]+)\s*=\s*(.*)$/.exec(line);
if (!m) continue;
if (m[1] === "proxy_bind_addr") out.proxyBindAddr = m[2]!.trim() || null;
if (m[1] === "allow_ports") out.allowPorts = m[2]!.trim() || null;
}
return out;
}
export function readFrps(file = paths.frpsIni): FrpsSettings | null {
try {
return parseFrpsIni(fs.readFileSync(file, "utf8"));
} catch {
return null;
}
}
/** Whether a port is inside allow_ports ("2000-3000,3001,4000-5000"); no allow_ports allows all. */
export function portAllowed(port: number, allowPorts: string | null): boolean {
if (!allowPorts) return true;
return allowPorts.split(",").map((s) => s.trim()).filter(Boolean).some((range) => {
const [a, b] = range.split("-").map((n) => Number(n.trim()));
if (!Number.isFinite(a)) return false;
return b === undefined ? port === a : Number.isFinite(b) && port >= a! && port <= b;
});
}
/** Listening TCP sockets from `ss -ltnH`: port → local addresses. */
export function parseSs(text: string): Map<number, string[]> {
const out = new Map<number, string[]>();
for (const line of text.split("\n")) {
const cols = line.trim().split(/\s+/);
const local = cols[3];
if (!local) continue;
const i = local.lastIndexOf(":");
const port = Number(local.slice(i + 1));
const addr = local.slice(0, i).replace(/^\[|\]$/g, "").replace(/%.*$/, "");
if (!Number.isFinite(port)) continue;
out.set(port, [...(out.get(port) ?? []), addr]);
}
return out;
}
const LOOPBACK = new Set(["127.0.0.1", "::1", "::ffff:127.0.0.1"]);
export interface PortCheck {
port: number;
allowed: boolean;
listening: boolean;
loopbackOnly: boolean;
ok: boolean;
}
export function checkPorts(ports: number[], frps: FrpsSettings | null, listening: Map<number, string[]>): { ports: PortCheck[]; proxyBindAddr: string | null; warnings: string[] } {
const warnings: string[] = [];
if (!frps) warnings.push(`No frps.ini at ${paths.frpsIni}.`);
const bind = frps?.proxyBindAddr ?? null;
if (frps && (!bind || !LOOPBACK.has(bind))) warnings.push(`proxy_bind_addr is ${bind ? `"${bind}"` : "not set"}: tunnels can be reached straight from the internet. Set proxy_bind_addr = 127.0.0.1 in frps.ini.`);
const results = ports.map((port) => {
const addrs = listening.get(port) ?? [];
const allowed = portAllowed(port, frps?.allowPorts ?? null);
const loopbackOnly = addrs.length > 0 && addrs.every((a) => LOOPBACK.has(a));
return { port, allowed, listening: addrs.length > 0, loopbackOnly, ok: allowed && addrs.length > 0 && loopbackOnly };
});
for (const r of results) {
if (!r.allowed) warnings.push(`Port ${r.port} isn't in allow_ports.`);
else if (!r.listening) warnings.push(`Port ${r.port} isn't open in frps (is the church server's frpc connected?).`);
else if (!r.loopbackOnly) warnings.push(`Port ${r.port} is open on more than 127.0.0.1.`);
}
return { ports: results, proxyBindAddr: bind, warnings };
}
export function listeningNow(): Promise<Map<number, string[]>> {
return new Promise((resolve) => {
execFile("ss", ["-ltnH"], { timeout: 10_000 }, (err, stdout) => resolve(err ? new Map() : parseSs(String(stdout))));
});
}
+87
View File
@@ -0,0 +1,87 @@
import fs from "node:fs";
import os from "node:os";
import { agent } from "@crosscode/contracts";
import { generateEd25519, generateX25519 } from "@crosscode/contracts/signing";
import { configFile, ed25519File, loadConfig, saveConfig, writePrivate, x25519File } from "./config";
import { hubClient, postUnsigned } from "./hub";
import { Agent } from "./loop";
import { AGENT_VERSION } from "./stats";
/**
* crosscode-agent (AGENT-1.0-PLAN §3)
* crosscode-agent join --hub https://hub.crosscode.co.za --token <one-time token> [--central]
* crosscode-agent run what systemd runs (crosscode-agent.service)
* crosscode-agent version
*/
const USAGE = `crosscode-agent ${AGENT_VERSION}
crosscode-agent join --hub <url> --token <token> [--central] [--force]
Join this server to the Hub with a one-time token (Hub → Servers → Add server).
Makes the server's key pairs; the private keys never leave it.
crosscode-agent run poll the Hub for jobs and send stats (run by systemd)
crosscode-agent version
`;
function flag(args: string[], name: string): string | undefined {
const i = args.indexOf(name);
return i >= 0 ? args[i + 1] : undefined;
}
export async function join(args: string[]): Promise<void> {
const hub = flag(args, "--hub");
const token = flag(args, "--token");
const mode: agent.AgentMode = args.includes("--central") ? "central" : "church";
if (!hub || !/^https?:\/\//.test(hub)) throw new Error("--hub <url> is required (https://hub.crosscode.co.za).");
if (!token) throw new Error("--token <token> is required (Hub → Servers → Add server).");
if (fs.existsSync(configFile()) && !args.includes("--force")) {
throw new Error(`This server has already joined (${configFile()}). Use --force to join again (the Hub then sees a new server).`);
}
const ed = generateEd25519();
const x = generateX25519();
const reply = agent.joinResponseSchema.parse(await postUnsigned(hub, "/api/agent/v1/join", {
agentApiVersion: agent.AGENT_API_VERSION,
token,
mode,
hostname: os.hostname().slice(0, 253),
agentVersion: AGENT_VERSION,
publicKeys: { ed25519: ed.publicKey, x25519: x.publicKey },
}));
writePrivate(ed25519File(), ed.privateKeyPem);
writePrivate(x25519File(), x.privateKeyPem);
saveConfig({ hubUrl: hub, serverId: reply.serverId, mode, hubJobKey: reply.hubJobKey, pollSeconds: reply.pollSeconds, statsSeconds: reply.statsSeconds, joinedAt: new Date().toISOString() });
console.log(`Joined the Hub as ${reply.serverId} (${mode}). Start the agent: systemctl enable --now crosscode-agent`);
}
export async function main(argv = process.argv.slice(2)): Promise<void> {
const [command, ...args] = argv;
switch (command) {
case "version":
case "--version":
console.log(AGENT_VERSION);
return;
case "join":
await join(args);
return;
case "run": {
if (process.getuid && process.getuid() !== 0 && !process.env.CROSSCODE_AGENT_ALLOW_NON_ROOT) {
throw new Error("Run the agent as root (it runs crosscode, which needs root).");
}
const { config, keys } = loadConfig();
const a = new Agent({ config, keys, hub: hubClient(config, keys) });
for (const sig of ["SIGTERM", "SIGINT"] as const) process.on(sig, () => { a.stop(); setTimeout(() => process.exit(0), 1000).unref(); });
await a.start();
return;
}
default:
process.stdout.write(USAGE);
if (command && command !== "help" && command !== "--help") process.exitCode = 1;
}
}
if (require.main === module) {
main().catch((e) => {
console.error(`crosscode-agent: ${(e as Error).message}`);
process.exit(1);
});
}
+58
View File
@@ -0,0 +1,58 @@
import type { agent } from "@crosscode/contracts";
/**
* The `crosscode` arguments for a job (AGENT-1.0-PLAN §4): built from the checked arguments only,
* as an array passed straight to the process (never through a shell). Jobs the agent handles
* itself (update-agent, check-ports) have none.
*
* `--config <file>` is added by the runner when the job brings config; `--json` always.
*/
export function crosscodeArgs(job: agent.AgentJob, opts: { exportFile?: string } = {}): string[] | null {
switch (job.type) {
case "install": {
const a = job.args;
if (a.products.length !== 1 || a.products[0] !== "events") throw new Error("This agent can only install Events so far.");
const domain = a.domains.events;
const ports = a.ports.events;
if (!domain || !ports) throw new Error("An Events install needs its domain and ports.");
// The church's address: the Events domain without "events." (crosscode uses it for the
// church.env ADDRESS; the domain itself goes in --events-domain).
const address = domain.startsWith("events.") ? domain.slice("events.".length) : domain;
const out = ["install", a.church, "--products", "events", "--domain", address, "--events-domain", domain,
"--api-port", String(ports.api), "--web-port", String(ports.web), "--frp"];
const v = a.versions?.events;
if (v) out.push("--version", v);
return out;
}
case "add-product":
return ["add", job.args.church, job.args.product];
case "deploy": {
const out = ["deploy", job.args.church, ...job.args.products];
for (const v of job.args.versions ?? []) out.push("--version", v);
if (job.args.check) out.push("--check");
return out;
}
case "adopt": {
const out = ["adopt", job.args.church, "--events-dir", job.args.eventsDir];
if (job.args.version) out.push("--version", job.args.version);
if (job.args.address) out.push("--address", job.args.address);
return out;
}
case "adopt-undo":
return ["adopt", job.args.church, "--undo"];
case "restart":
return ["restart", job.args.church, ...(job.args.product ? [job.args.product] : [])];
case "export":
if (!opts.exportFile) throw new Error("export needs a file to write");
return ["export", job.args.church, "--out", opts.exportFile];
case "uninstall":
return ["uninstall", job.args.church, "--confirm", job.args.church];
case "purge":
return ["purge", job.args.church, "--confirm", job.args.church];
case "status":
return ["status", ...(job.args.church ? [job.args.church] : []), "--no-fetch"];
case "update-agent":
case "check-ports":
return null;
}
}
+49
View File
@@ -0,0 +1,49 @@
import fs from "node:fs";
import path from "node:path";
import { z } from "zod";
import { agent } from "@crosscode/contracts";
import { publicKeyOf } from "@crosscode/contracts/signing";
import { paths } from "./paths";
/** /etc/crosscode/agent.json, written by `crosscode-agent join`. No secrets: the keys are files next to it. */
export const agentConfigSchema = z.object({
hubUrl: z.url({ protocol: /^https?$/ }),
serverId: z.string().regex(/^srv_[A-Za-z0-9_-]{8,64}$/),
mode: agent.agentModeSchema,
/** The Hub's job-signing key, pinned when joining. */
hubJobKey: z.string().regex(/^[A-Za-z0-9_-]{43}$/),
pollSeconds: z.int().min(5).max(600).default(30),
statsSeconds: z.int().min(15).max(3600).default(60),
joinedAt: z.string(),
});
export type AgentConfig = z.infer<typeof agentConfigSchema>;
export interface Keys {
ed25519Pem: string;
x25519Pem: string;
}
export const configFile = () => path.join(paths.configDir, "agent.json");
export const ed25519File = () => path.join(paths.configDir, "agent-ed25519.pem");
export const x25519File = () => path.join(paths.configDir, "agent-x25519.pem");
/** Writes a file only root can read, atomically. */
export function writePrivate(file: string, text: string): void {
fs.mkdirSync(path.dirname(file), { recursive: true, mode: 0o700 });
const tmp = `${file}.tmp-${process.pid}`;
fs.writeFileSync(tmp, text, { mode: 0o600 });
fs.renameSync(tmp, file);
}
export function loadConfig(): { config: AgentConfig; keys: Keys } {
const config = agentConfigSchema.parse(JSON.parse(fs.readFileSync(configFile(), "utf8")));
const keys = { ed25519Pem: fs.readFileSync(ed25519File(), "utf8"), x25519Pem: fs.readFileSync(x25519File(), "utf8") };
// A broken pair would sign requests the Hub can't check; better to stop at start.
publicKeyOf(keys.ed25519Pem);
publicKeyOf(keys.x25519Pem);
return { config, keys };
}
export function saveConfig(config: AgentConfig): void {
writePrivate(configFile(), `${JSON.stringify(config, null, 2)}\n`);
}
+75
View File
@@ -0,0 +1,75 @@
import { signAgentRequest } from "@crosscode/contracts/signing";
import type { AgentConfig, Keys } from "./config";
/**
* Requests to the Hub. The agent only ever makes outgoing requests (it listens on nothing,
* AGENT-1.0-PLAN §2 rule 1); every one after joining is signed with the server's Ed25519 key.
*/
export class HubRequestError extends Error {
constructor(message: string, readonly status: number, readonly code: string | null) {
super(message);
}
}
export type Fetch = typeof fetch;
export interface HubClient {
post(path: string, body: unknown): Promise<unknown>;
put(path: string, body: Uint8Array, contentType: string): Promise<unknown>;
}
const TIMEOUT_MS = 30_000;
const UPLOAD_TIMEOUT_MS = 10 * 60_000;
async function send(fetcher: Fetch, url: string, init: RequestInit, timeoutMs: number): Promise<unknown> {
let res: Response;
try {
res = await fetcher(url, { ...init, signal: AbortSignal.timeout(timeoutMs) });
} catch (e) {
throw new HubRequestError(`The Hub can't be reached: ${(e as Error).message}`, 0, "HUB_UNREACHABLE");
}
if (res.status === 204) return null;
let body: unknown;
try {
body = await res.json();
} catch {
body = null;
}
if (!res.ok) {
const b = (body ?? {}) as { message?: string; code?: string };
throw new HubRequestError(b.message || `The Hub answered ${res.status}`, res.status, b.code ?? null);
}
return body;
}
/** An unsigned request (only `join`, where the one-time token is the proof). */
export function postUnsigned(hubUrl: string, path: string, body: unknown, fetcher: Fetch = fetch): Promise<unknown> {
return send(fetcher, `${hubUrl.replace(/\/$/, "")}${path}`, {
method: "POST",
headers: { "content-type": "application/json", "user-agent": "crosscode-agent" },
body: JSON.stringify(body),
}, TIMEOUT_MS);
}
export function hubClient(config: AgentConfig, keys: Keys, fetcher: Fetch = fetch): HubClient {
const base = config.hubUrl.replace(/\/$/, "");
const signed = (raw: string | Uint8Array) => signAgentRequest({ serverId: config.serverId, privateKeyPem: keys.ed25519Pem, rawBody: raw });
return {
post(path, body) {
const raw = JSON.stringify(body);
return send(fetcher, `${base}${path}`, {
method: "POST",
headers: { "content-type": "application/json", "user-agent": "crosscode-agent", ...signed(raw) },
body: raw,
}, TIMEOUT_MS);
},
put(path, body, contentType) {
return send(fetcher, `${base}${path}`, {
method: "PUT",
headers: { "content-type": contentType, "user-agent": "crosscode-agent", ...signed(body) },
body,
}, UPLOAD_TIMEOUT_MS);
},
};
}
+226
View File
@@ -0,0 +1,226 @@
import fs from "node:fs";
import path from "node:path";
import { agent } from "@crosscode/contracts";
import { encryptExport } from "@crosscode/contracts/signing";
import { checkPorts, listeningNow, readFrps } from "./central";
import { crosscodeArgs } from "./command";
import type { AgentConfig, Keys } from "./config";
import { type HubClient, HubRequestError } from "./hub";
import { paths } from "./paths";
import { runCrosscode, type RunResult } from "./runner";
import { AGENT_VERSION, collectStats } from "./stats";
import { confirmUpdate, updateAgent } from "./update";
import { checkJob, SeenJobs } from "./verify";
/**
* The agent's life: poll the Hub every 30 s for jobs due, send stats every 60 s, run one job at a
* time and send each result. Reports that can't be sent are kept in an outbox and sent later, so
* a Hub outage never loses a job's result. Nothing here listens: every request goes out.
*/
const log = (...a: unknown[]) => console.log(new Date().toISOString(), ...a);
export interface AgentDeps {
config: AgentConfig;
keys: Keys;
hub: HubClient;
seen?: SeenJobs;
/** For tests: replaces running crosscode and the stats. */
run?: (args: string[], job: agent.AgentJob) => Promise<RunResult>;
stats?: () => Promise<agent.AgentStats>;
now?: () => Date;
}
const outboxDir = () => path.join(paths.stateDir, "outbox");
function report(job: { id: string; type: agent.JobType }, serverId: string, r: Omit<RunResult, "log"> & { log?: string[] }): agent.AgentReport {
return {
schema: "crosscode.agent-report/1",
jobId: job.id,
serverId,
type: job.type,
status: r.status,
startedAt: r.startedAt,
finishedAt: r.finishedAt,
steps: r.steps.slice(0, 200),
result: r.result,
...(r.message ? { message: r.message.slice(0, 2000) } : {}),
log: (r.log ?? []).slice(-200),
};
}
export class Agent {
private seen: SeenJobs;
private stopped = false;
private confirmed = false;
private failures = 0;
constructor(private deps: AgentDeps) {
this.seen = deps.seen ?? new SeenJobs();
}
private now(): Date {
return this.deps.now?.() ?? new Date();
}
/** Sends a report, or keeps it in the outbox to send later. */
async send(r: agent.AgentReport): Promise<void> {
try {
await this.deps.hub.post("/api/agent/v1/report", r);
} catch (e) {
log(`[report] ${r.jobId}: couldn't send (${(e as Error).message}); kept to send later.`);
fs.mkdirSync(outboxDir(), { recursive: true, mode: 0o700 });
fs.writeFileSync(path.join(outboxDir(), `${r.jobId}.json`), JSON.stringify(r), { mode: 0o600 });
}
}
async flushOutbox(): Promise<void> {
let files: string[];
try { files = fs.readdirSync(outboxDir()).filter((f) => f.endsWith(".json")); } catch { return; }
for (const f of files) {
const file = path.join(outboxDir(), f);
try {
await this.deps.hub.post("/api/agent/v1/report", JSON.parse(fs.readFileSync(file, "utf8")));
fs.rmSync(file, { force: true });
} catch (e) {
// A report the Hub refuses outright (4xx) would never go: drop it rather than retry forever.
if (e instanceof HubRequestError && e.status >= 400 && e.status < 500) fs.rmSync(file, { force: true });
return;
}
}
}
/** One poll: fetch the jobs due and run them, one after the other. Returns the next delay (s). */
async pollOnce(): Promise<number> {
const { config } = this.deps;
let reply: { jobs: unknown[]; pollSeconds: number };
try {
reply = agent.pollResponseSchema.parse(await this.deps.hub.post("/api/agent/v1/poll", {
agentApiVersion: agent.AGENT_API_VERSION, agentVersion: AGENT_VERSION, mode: config.mode, runningJobId: null,
}));
} catch (e) {
this.failures++;
const delay = Math.min(300, config.pollSeconds * 2 ** Math.min(this.failures, 4));
log(`[poll] ${(e as Error).message}; next try in ${delay}s.`);
return delay;
}
this.failures = 0;
if (!this.confirmed) {
confirmUpdate();
this.confirmed = true;
}
await this.flushOutbox();
for (const raw of reply.jobs) {
if (this.stopped) break;
await this.handle(raw);
}
return reply.pollSeconds || config.pollSeconds;
}
async handle(raw: unknown): Promise<void> {
const { config } = this.deps;
const verdict = checkJob(raw, config, this.seen, this.now());
const at = this.now().toISOString();
if (!verdict.ok) {
log(`[job] ${verdict.id ?? "(no id)"} ${verdict.status}: ${verdict.message}`);
if (verdict.id && verdict.type && !this.seen.has(verdict.id)) {
this.seen.add(verdict.id);
await this.send(report({ id: verdict.id, type: verdict.type }, config.serverId, { status: verdict.status, steps: [], result: {}, message: verdict.message, startedAt: at, finishedAt: at }));
}
return;
}
const job = verdict.job;
// Seen before it runs, so a crash part-way can never run it twice.
this.seen.add(job.id);
log(`[job] ${job.id} ${job.type} ${JSON.stringify(job.args)}`);
const result = await this.execute(job);
log(`[job] ${job.id} ${result.status}${result.message ? `: ${result.message}` : ""}`);
await this.send(report(job, config.serverId, result));
if ((result as RunResult & { restart?: boolean }).restart) {
log("[agent] Exiting so systemd starts the new version.");
this.stopped = true;
setTimeout(() => process.exit(0), 500);
}
}
private async execute(job: agent.AgentJob): Promise<RunResult & { restart?: boolean }> {
const startedAt = this.now().toISOString();
const limit = agent.JOB_TIME_LIMIT_MINUTES[job.type];
try {
switch (job.type) {
case "update-agent": {
const u = await updateAgent(job.args.version, AGENT_VERSION);
return { status: u.status, steps: u.steps, result: u.result, message: u.message, log: [], startedAt, finishedAt: new Date().toISOString(), restart: u.restart };
}
case "check-ports": {
const res = checkPorts(job.args.ports, readFrps(), await listeningNow());
const ok = res.ports.every((p) => p.ok) && res.warnings.length === 0;
return { status: ok ? "succeeded" : "failed", steps: [], result: res as unknown as Record<string, unknown>, message: res.warnings.join(" ") || undefined, log: [], startedAt, finishedAt: new Date().toISOString() };
}
case "export":
return await this.runExport(job, limit);
default: {
const args = crosscodeArgs(job);
if (!args) throw new Error(`No command for ${job.type}`);
return await this.run(args, job, limit);
}
}
} catch (e) {
return { status: "failed", steps: [], result: {}, message: (e as Error).message, log: [], startedAt, finishedAt: new Date().toISOString() };
}
}
private run(args: string[], job: agent.AgentJob, limit: number): Promise<RunResult> {
if (this.deps.run) return this.deps.run(args, job);
return runCrosscode({ args, sealedConfig: job.config ?? null, x25519Pem: this.deps.keys.x25519Pem, timeLimitMinutes: limit });
}
/**
* export: crosscode writes the ZIP, the agent encrypts it with a fresh key and uploads it. The
* key goes to the Hub only in this job's report, so it can email the download link; the Hub
* doesn't keep it (AGENT-1.0-PLAN §9).
*/
private async runExport(job: Extract<agent.AgentJob, { type: "export" }>, limit: number): Promise<RunResult> {
const dir = path.join(paths.stateDir, "exports");
fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
const zip = path.join(dir, `${job.args.exportId}.zip`);
fs.rmSync(zip, { force: true });
try {
const r = await this.run(crosscodeArgs(job, { exportFile: zip })!, job, limit);
if (r.status !== "succeeded") return r;
const { key, file } = encryptExport(fs.readFileSync(zip));
await this.deps.hub.put(`/api/agent/v1/exports/${job.args.exportId}`, file, "application/octet-stream");
return { ...r, result: { ...r.result, file: undefined, encryptedBytes: file.length, key } };
} finally {
fs.rmSync(zip, { force: true });
}
}
async statsOnce(): Promise<void> {
try {
const stats = await (this.deps.stats ?? (() => collectStats(this.deps.config.mode)))();
await this.deps.hub.post("/api/agent/v1/stats", stats);
} catch (e) {
log(`[stats] ${(e as Error).message}`);
}
}
/** Runs until stopped: polls and stats on their own timers, never at the same time as a job. */
async start(): Promise<void> {
log(`[agent] ${AGENT_VERSION} (${this.deps.config.mode}) for ${this.deps.config.serverId}, Hub ${this.deps.config.hubUrl}`);
let nextStats = 0;
while (!this.stopped) {
if (Date.now() >= nextStats) {
await this.statsOnce();
nextStats = Date.now() + this.deps.config.statsSeconds * 1000;
}
const delay = await this.pollOnce();
await new Promise((r) => setTimeout(r, Math.max(5, Math.min(delay, this.deps.config.statsSeconds)) * 1000));
}
}
stop(): void {
this.stopped = true;
}
}
+22
View File
@@ -0,0 +1,22 @@
/**
* Where the agent keeps things on a server (docs/AGENT-1.0-PLAN.md §3). Each can be moved with an
* environment variable, which the tests use.
*/
const env = (name: string, fallback: string): string => process.env[name] || fallback;
export const paths = {
/** agent.json and the two private keys, root only (mode 600). */
get configDir() { return env("CROSSCODE_AGENT_CONFIG_DIR", "/etc/crosscode"); },
/** Seen job ids (replay protection) and exports waiting to upload. */
get stateDir() { return env("CROSSCODE_AGENT_STATE_DIR", "/var/lib/crosscode-agent"); },
/** A job's opened config, only while the job runs (tmpfs). */
get runDir() { return env("CROSSCODE_AGENT_RUN_DIR", "/run/crosscode-agent"); },
/** The crosscode script (a checkout of the monorepo in /opt/crosscode/tools). */
get crosscode() { return env("CROSSCODE_BIN", "/usr/local/bin/crosscode"); },
/** Agent releases: releases/<version>/ (checkouts), current → the running one, previous. */
get agentRoot() { return env("CROSSCODE_AGENT_ROOT", "/opt/crosscode/agent"); },
/** frps.ini on the central server. */
get frpsIni() { return env("CROSSCODE_FRPS_INI", "/etc/frp/frps.ini"); },
/** The churches on a church server. */
get sites() { return env("CROSSCODE_SITES", "/var/www/sites"); },
};
+51
View File
@@ -0,0 +1,51 @@
/**
* Log lines are filtered for anything that looks like a secret before they leave the server
* (AGENT-1.0-PLAN §4, rule 7). The crosscode script never prints .env values, but tools it runs
* (npm, Prisma, git) might print a URL with a password or a token in an error.
*/
const RULES: [RegExp, string][] = [
// user:password@ in any URL (postgresql://, https://).
[/([a-z][a-z0-9+.-]*:\/\/[^\s:/@]+:)[^\s@/]+@/gi, "$1***@"],
// KEY=value and KEY: value for keys that hold secrets.
[/\b([A-Z0-9_]*(SECRET|PASSWORD|PASSWD|TOKEN|API_KEY|PRIVATE_KEY|DSN|LICENCE_KEY|LICENSE_KEY)[A-Z0-9_]*)(\s*[=:]\s*)("[^"]*"|'[^']*'|\S+)/g, "$1$3***"],
// Authorization headers and bearer tokens.
[/\b(authorization\s*:\s*)(bearer|basic)\s+\S+/gi, "$1$2 ***"],
[/\bbearer\s+[A-Za-z0-9._~+/=-]{16,}/gi, "Bearer ***"],
// PEM blocks.
[/-----BEGIN [A-Z ]*PRIVATE KEY-----[\s\S]*?-----END [A-Z ]*PRIVATE KEY-----/g, "[private key]"],
// Cross Code licence keys (CEH-7K2M-…).
[/\b(CEH|CCA|CCW|CCP)-[A-Z0-9]{4}(-[A-Z0-9]{4}){3}\b/g, "$1-****"],
// Long random-looking strings (hex or base64 of 32+ characters): likely keys or tokens.
[/\b[A-Fa-f0-9]{40,}\b/g, "***"],
[/\b[A-Za-z0-9+/_-]{48,}={0,2}/g, "***"],
];
export function redact(line: string): string {
let out = line;
for (const [re, to] of RULES) out = out.replace(re, to);
return out.length > 1000 ? `${out.slice(0, 997)}...` : out;
}
/** Keeps the last `max` lines of output, redacted, without holding all of it in memory. */
export class LogTail {
private lines: string[] = [];
private partial = "";
constructor(private max = 200) {}
push(chunk: string): void {
const parts = (this.partial + chunk).split(/\r?\n/);
this.partial = parts.pop() ?? "";
for (const p of parts) this.add(p);
}
private add(line: string): void {
this.lines.push(redact(line));
if (this.lines.length > this.max) this.lines.shift();
}
end(): string[] {
if (this.partial) {
this.add(this.partial);
this.partial = "";
}
return [...this.lines];
}
}
+116
View File
@@ -0,0 +1,116 @@
import { spawn } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
import { agent } from "@crosscode/contracts";
import { openConfig } from "@crosscode/contracts/signing";
import { paths } from "./paths";
import { LogTail, redact } from "./redact";
/**
* Runs `crosscode … --json` for a job: argument array (no shell), the job's config opened to a
* file only root can read and removed as soon as the job ends, a time limit per type (the whole
* process group is stopped past it), and the last 200 log lines, redacted.
*/
export interface RunResult {
status: agent.JobStatus;
steps: agent.ReportStep[];
result: Record<string, unknown>;
message?: string;
log: string[];
startedAt: string;
finishedAt: string;
}
export interface RunOptions {
args: string[];
sealedConfig?: agent.Sealed | null;
x25519Pem: string;
timeLimitMinutes: number;
/** For tests. */
bin?: string;
}
const iso = (d = new Date()) => d.toISOString();
function writeConfig(sealed: agent.Sealed, x25519Pem: string): string {
const opened = agent.jobConfigSchema.parse(openConfig(sealed, x25519Pem));
fs.mkdirSync(paths.runDir, { recursive: true, mode: 0o700 });
const file = path.join(paths.runDir, `config-${process.pid}-${Date.now()}.json`);
fs.writeFileSync(file, JSON.stringify(opened), { mode: 0o600 });
return file;
}
export async function runCrosscode(opts: RunOptions): Promise<RunResult> {
const startedAt = iso();
let configFile: string | null = null;
try {
if (opts.sealedConfig) {
try {
configFile = writeConfig(opts.sealedConfig, opts.x25519Pem);
} catch (e) {
return { status: "refused", steps: [], result: {}, message: `The job's config couldn't be opened with this server's key: ${(e as Error).message}`, log: [], startedAt, finishedAt: iso() };
}
}
const args = [...opts.args, "--json", ...(configFile ? ["--config", configFile] : [])];
return await new Promise<RunResult>((resolve) => {
const tail = new LogTail(200);
let stdout = "";
const child = spawn(opts.bin ?? paths.crosscode, args, {
detached: true,
stdio: ["ignore", "pipe", "pipe"],
env: { ...process.env, CROSSCODE_NO_UPDATE: process.env.CROSSCODE_NO_UPDATE ?? "", LC_ALL: "C.UTF-8" },
});
let timedOut = false;
const timer = setTimeout(() => {
timedOut = true;
try { process.kill(-child.pid!, "SIGTERM"); } catch { /* already gone */ }
setTimeout(() => { try { process.kill(-child.pid!, "SIGKILL"); } catch { /* gone */ } }, 10_000).unref();
}, opts.timeLimitMinutes * 60_000);
child.stdout.setEncoding("utf8").on("data", (d: string) => { stdout += d; if (stdout.length > 5_000_000) stdout = stdout.slice(-1_000_000); });
child.stderr.setEncoding("utf8").on("data", (d: string) => tail.push(d));
child.on("error", (e) => {
clearTimeout(timer);
resolve({ status: "failed", steps: [], result: {}, message: `crosscode couldn't be started: ${e.message}`, log: tail.end(), startedAt, finishedAt: iso() });
});
child.on("close", (code) => {
clearTimeout(timer);
const log = tail.end();
const report = parseReport(stdout);
if (timedOut) {
resolve({ status: "failed", steps: report?.steps ?? [], result: report?.result ?? {}, message: `Stopped after the time limit (${opts.timeLimitMinutes} minutes).`, log, startedAt, finishedAt: iso() });
return;
}
if (!report) {
resolve({ status: code === 0 ? "succeeded" : "failed", steps: [], result: {}, message: code === 0 ? undefined : `crosscode ended with code ${code} without a report.`, log, startedAt, finishedAt: iso() });
return;
}
resolve({
status: report.status,
steps: report.steps,
result: report.result,
message: report.message ? redact(report.message) : undefined,
log,
startedAt: report.startedAt,
finishedAt: report.finishedAt,
});
});
});
} finally {
if (configFile) fs.rmSync(configFile, { force: true });
}
}
/** The last line of stdout that's a valid script report. */
export function parseReport(stdout: string): agent.ScriptReport | null {
const lines = stdout.trim().split("\n").reverse();
for (const line of lines) {
try {
const parsed = agent.scriptReportSchema.safeParse(JSON.parse(line));
if (parsed.success) return parsed.data;
} catch {
// not JSON
}
}
return null;
}
+167
View File
@@ -0,0 +1,167 @@
import { execFile, execFileSync } from "node:child_process";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import type { agent } from "@crosscode/contracts";
import { readFrps } from "./central";
import { paths } from "./paths";
/**
* The server's stats every 60 s (AGENT-1.0-PLAN §10, crosscode.agent-stats/1): RAM, CPU, disks,
* uptime, versions, frpc, and each PM2 process with its church and product. Read from /proc and
* the tools that are there; anything missing is null rather than an error.
*/
export const AGENT_VERSION = "1.0.0";
function run(cmd: string, args: string[], timeoutMs = 10_000): Promise<string | null> {
return new Promise((resolve) => {
execFile(cmd, args, { timeout: timeoutMs, maxBuffer: 10_000_000 }, (err, stdout) => resolve(err ? null : String(stdout)));
});
}
export function parseMeminfo(text: string): agent.AgentStats["ram"] {
const kb = (key: string) => Number(new RegExp(`^${key}:\\s+(\\d+)`, "m").exec(text)?.[1] ?? 0) * 1024;
const total = kb("MemTotal");
const available = kb("MemAvailable");
return { totalBytes: total, usedBytes: Math.max(0, total - available), availableBytes: available, swapTotalBytes: kb("SwapTotal"), swapUsedBytes: Math.max(0, kb("SwapTotal") - kb("SwapFree")) };
}
/** CPU busy % between two /proc/stat readings. */
export function cpuPercent(a: string, b: string): number {
const parse = (t: string) => {
const f = (/^cpu\s+(.+)$/m.exec(t)?.[1] ?? "").trim().split(/\s+/).map(Number);
const idle = (f[3] ?? 0) + (f[4] ?? 0);
return { idle, total: f.reduce((s, n) => s + (Number.isFinite(n) ? n : 0), 0) };
};
const x = parse(a), y = parse(b);
const total = y.total - x.total;
if (total <= 0) return 0;
return Math.min(100, Math.max(0, Math.round(((total - (y.idle - x.idle)) / total) * 1000) / 10));
}
async function disks(): Promise<agent.AgentStats["disks"]> {
const out = await run("df", ["-P", "-B1", "-x", "tmpfs", "-x", "devtmpfs", "-x", "squashfs", "-x", "overlay"]);
if (!out) return [];
return out.trim().split("\n").slice(1).map((l) => l.trim().split(/\s+/)).filter((c) => c.length >= 6 && c[5]!.startsWith("/"))
.slice(0, 30).map((c) => ({ mount: c[5]!, totalBytes: Number(c[1]), usedBytes: Number(c[2]), freeBytes: Number(c[3]) }));
}
function osName(): string {
try {
const t = fs.readFileSync("/etc/os-release", "utf8");
return /^PRETTY_NAME="?([^"\n]+)"?/m.exec(t)?.[1] ?? `${os.type()} ${os.release()}`;
} catch {
return `${os.type()} ${os.release()}`;
}
}
const versionIn = (text: string | null) => (text ? /(\d+\.\d+(\.\d+)?)/.exec(text)?.[1] ?? null : null);
/** church.env → { app user → [{ church, product, names }] }, read as data like the script does. */
export function churchProcesses(sitesDir = paths.sites): Map<string, { church: string; product: agent.AgentStats["processes"][number]["product"]; names: string[] }[]> {
const byUser = new Map<string, { church: string; product: "events" | "accounts" | "website" | "app"; names: string[] }[]>();
let entries: string[];
try { entries = fs.readdirSync(sitesDir); } catch { return byUser; }
for (const church of entries) {
if (!/^[a-z][a-z0-9-]{0,39}$/.test(church)) continue;
let text: string;
try { text = fs.readFileSync(path.join(sitesDir, church, "church.env"), "utf8"); } catch { continue; }
const vars = new Map<string, string>();
for (const line of text.split(/\r?\n/)) {
const m = /^([A-Z][A-Z0-9_]*)=("([^"]*)"|'([^']*)'|([^#\s]*))/.exec(line);
if (m) vars.set(m[1]!, m[3] ?? m[4] ?? m[5] ?? "");
}
for (const product of ["accounts", "events", "website", "app"] as const) {
const p = product.toUpperCase();
const user = vars.get(`${p}_USER`);
const names = (vars.get(`${p}_PM2`) ?? "").split(/\s+/).filter(Boolean);
if (!user || !names.length || !/^[a-z_][a-z0-9_-]{0,31}$/.test(user)) continue;
const list = byUser.get(user) ?? [];
list.push({ church, product, names });
byUser.set(user, list);
}
}
return byUser;
}
interface Pm2Proc { name?: string; monit?: { memory?: number; cpu?: number }; pm2_env?: { status?: string; restart_time?: number; pm_uptime?: number } }
function pm2Running(user: string): boolean {
try {
const home = execFileSync("getent", ["passwd", user], { encoding: "utf8" }).split(":")[5];
const pid = Number(fs.readFileSync(`${home}/.pm2/pm2.pid`, "utf8").trim());
if (!pid) return false;
process.kill(pid, 0);
return true;
} catch {
return false;
}
}
async function pm2Processes(): Promise<agent.AgentStats["processes"]> {
const out: agent.AgentStats["processes"] = [];
for (const [user, installs] of churchProcesses()) {
// Only ask a daemon that's running: `pm2 jlist` would otherwise start an empty one inside this
// agent's service, which dies with the agent (found in the VM test).
if (!pm2Running(user)) continue;
const json = await run("sudo", ["-n", "-u", user, "-H", "--", "pm2", "jlist"]);
let list: Pm2Proc[];
try { list = JSON.parse(json ?? "[]") as Pm2Proc[]; } catch { list = []; }
for (const p of list) {
const owner = installs.find((i) => i.names.includes(p.name ?? ""));
out.push({
name: String(p.name ?? "?").slice(0, 100),
church: owner?.church ?? null,
product: owner?.product ?? null,
status: String(p.pm2_env?.status ?? "unknown").slice(0, 40),
memoryBytes: Number(p.monit?.memory ?? 0),
cpuPercent: Number(p.monit?.cpu ?? 0),
restarts: Math.max(0, Math.trunc(Number(p.pm2_env?.restart_time ?? 0))),
uptimeSeconds: p.pm2_env?.pm_uptime ? Math.max(0, (Date.now() - p.pm2_env.pm_uptime) / 1000) : 0,
});
}
}
return out.slice(0, 200);
}
/** A systemd unit is active; null when there's no systemctl. (is-active exits non-zero when inactive.) */
function active(unit: string): Promise<boolean | null> {
return new Promise((resolve) => {
execFile("systemctl", ["is-active", unit], { timeout: 10_000 }, (err, stdout) => {
if (err && (err as NodeJS.ErrnoException).code === "ENOENT") resolve(null);
else resolve(String(stdout).trim() === "active");
});
});
}
export async function collectStats(mode: agent.AgentMode): Promise<agent.AgentStats> {
const read = (f: string) => { try { return fs.readFileSync(f, "utf8"); } catch { return ""; } };
const stat1 = read("/proc/stat");
await new Promise((r) => setTimeout(r, 1000));
const stat2 = read("/proc/stat");
const [load1, load5, load15] = os.loadavg();
const [diskList, postgres, frpc, processes, frpcRunning] = await Promise.all([
disks(),
run("psql", ["--version"]),
run("frpc", ["-v"]),
mode === "church" ? pm2Processes() : Promise.resolve([]),
mode === "church" ? active("frpc") : Promise.resolve(null),
]);
const stats: agent.AgentStats = {
schema: "crosscode.agent-stats/1",
at: new Date().toISOString(),
ram: parseMeminfo(read("/proc/meminfo")),
cpu: { percent: cpuPercent(stat1, stat2), load1: load1 ?? 0, load5: load5 ?? 0, load15: load15 ?? 0, cores: Math.max(1, os.cpus().length) },
disks: diskList,
uptimeSeconds: os.uptime(),
versions: { os: osName().slice(0, 200), node: process.versions.node, postgres: versionIn(postgres), frpc: versionIn(frpc), agent: AGENT_VERSION },
frpcRunning,
processes,
};
if (mode === "central") {
const frps = readFrps();
stats.central = { proxyBindAddr: frps?.proxyBindAddr ?? null, allowPorts: frps?.allowPorts ?? null, frpsRunning: await active("frps") };
}
return stats;
}
+103
View File
@@ -0,0 +1,103 @@
import { execFile } from "node:child_process";
import fs from "node:fs";
import path from "node:path";
import type { agent } from "@crosscode/contracts";
import { paths } from "./paths";
/**
* update-agent (AGENT-1.0-PLAN §3): the newest agent@x.y.z is checked out and built next to the
* running copy, `current` is switched to it and the agent exits, so systemd starts the new one.
* A one-shot systemd timer switches back to `previous` unless the new agent clears
* update-pending.json by polling the Hub within 150 seconds.
*
* /opt/crosscode/agent/releases/<version>/ checkouts of the monorepo at agent@<version>
* /opt/crosscode/agent/current, previous symlinks to releases
* /opt/crosscode/agent/revert.sh written by the install script
*/
const REPO = process.env.CROSSCODE_REPO || "https://c-git.crosscode.co.za/joshua/crosscode-platform.git";
const CONFIRM_SECONDS = 150;
const pendingFile = () => path.join(paths.stateDir, "update-pending.json");
function exec(cmd: string, args: string[], cwd?: string, timeoutMs = 15 * 60_000): Promise<string> {
return new Promise((resolve, reject) => {
execFile(cmd, args, { cwd, timeout: timeoutMs, maxBuffer: 20_000_000, env: { ...process.env, NODE_ENV: "development" } }, (err, stdout, stderr) => {
if (err) reject(new Error(`${cmd} ${args.join(" ")}: ${String(stderr).trim().split("\n").slice(-3).join(" ") || err.message}`));
else resolve(String(stdout));
});
});
}
export interface UpdateOutcome {
status: agent.JobStatus;
message: string;
steps: agent.ReportStep[];
result: Record<string, unknown>;
/** Exit after reporting, so systemd starts the new version. */
restart: boolean;
}
export async function updateAgent(tag: string, currentVersion: string): Promise<UpdateOutcome> {
const version = tag.replace(/^agent@/, "");
const steps: agent.ReportStep[] = [];
const step = async (name: string, fn: () => Promise<void>) => {
const s: agent.ReportStep = { name, startedAt: new Date().toISOString(), finishedAt: null, outcome: "ok" };
steps.push(s);
try {
await fn();
} catch (e) {
s.outcome = "failed";
s.detail = (e as Error).message.slice(0, 2000);
throw e;
} finally {
s.finishedAt = new Date().toISOString();
}
};
if (version === currentVersion) return { status: "succeeded", message: `Already on ${tag}.`, steps, result: { version }, restart: false };
const root = paths.agentRoot;
const dir = path.join(root, "releases", version);
try {
await step(`Checking out ${tag}`, async () => {
if (!fs.existsSync(path.join(dir, ".git"))) {
fs.mkdirSync(path.dirname(dir), { recursive: true });
await exec("git", ["clone", "--quiet", "--depth", "1", "--branch", tag, REPO, dir]);
}
});
await step("Installing and building", async () => {
await exec("npm", ["ci", "--no-audit", "--no-fund", "--include=dev", "--include-workspace-root", "-w", "packages/contracts", "-w", "apps/agent"], dir);
await exec("npm", ["run", "build", "-w", "packages/contracts"], dir);
await exec("npm", ["run", "build", "-w", "apps/agent"], dir);
});
await step("Checking the new build", async () => {
const out = await exec(process.execPath, [path.join(dir, "apps/agent/dist/cli.js"), "version"], dir, 30_000);
if (out.trim() !== version) throw new Error(`The new build says it's ${out.trim()}, not ${version}.`);
});
await step("Switching over", async () => {
const current = path.join(root, "current");
const previous = path.join(root, "previous");
const old = fs.readlinkSync(current);
fs.writeFileSync(pendingFile(), JSON.stringify({ from: currentVersion, to: version, at: new Date().toISOString() }), { mode: 0o600 });
replaceLink(previous, old);
replaceLink(current, dir);
// If the new agent doesn't poll within CONFIRM_SECONDS, revert.sh puts `previous` back.
await exec("systemd-run", [`--on-active=${CONFIRM_SECONDS}`, `--unit=crosscode-agent-revert-${Date.now()}`, path.join(root, "revert.sh")]);
});
} catch (e) {
return { status: "failed", message: `The agent wasn't updated: ${(e as Error).message}`, steps, result: { from: currentVersion, to: version }, restart: false };
}
return { status: "succeeded", message: `Switching to ${tag}; it confirms by polling within ${CONFIRM_SECONDS} seconds.`, steps, result: { from: currentVersion, to: version }, restart: true };
}
function replaceLink(link: string, target: string): void {
const tmp = `${link}.tmp-${process.pid}`;
fs.rmSync(tmp, { force: true });
fs.symlinkSync(target, tmp);
fs.renameSync(tmp, link);
}
/** Called after the first successful poll: this version works, so it stays. */
export function confirmUpdate(): void {
fs.rmSync(pendingFile(), { force: true });
}
+78
View File
@@ -0,0 +1,78 @@
import fs from "node:fs";
import path from "node:path";
import { agent } from "@crosscode/contracts";
import { verifyJob } from "@crosscode/contracts/signing";
import type { AgentConfig } from "./config";
import { paths } from "./paths";
/**
* Whether to run a job (AGENT-1.0-PLAN §4). Checked in this order, and a job is refused at the
* first check it fails:
* 1. it's an object with an id and type the agent can report on,
* 2. the signature verifies with the Hub key pinned when the server joined,
* 3. it's for this server,
* 4. it's within notBefore … expiresAt (an expired job is reported as `expired`, never run late),
* 5. its id hasn't been seen in the last 90 days (replays),
* 6. its type is one this agent runs (a central agent only runs check-ports, status, update-agent),
* 7. every argument passes that type's schema (unknown arguments refused).
*/
export type Verdict =
| { ok: true; job: agent.AgentJob }
| { ok: false; status: "refused" | "expired"; message: string; id: string | null; type: agent.JobType | null };
const SEEN_DAYS = 90;
/** Job ids already run or refused, with when (replay protection). */
export class SeenJobs {
private seen = new Map<string, number>();
constructor(private file = path.join(paths.stateDir, "seen-jobs.json")) {
try {
const data = JSON.parse(fs.readFileSync(this.file, "utf8")) as Record<string, number>;
for (const [id, at] of Object.entries(data)) this.seen.set(id, at);
} catch {
// First run, or an unreadable file: start empty (signatures and expiry still apply).
}
}
has(id: string): boolean {
return this.seen.has(id);
}
add(id: string, now = Date.now()): void {
this.seen.set(id, now);
const cutoff = now - SEEN_DAYS * 86_400_000;
for (const [k, at] of this.seen) if (at < cutoff) this.seen.delete(k);
fs.mkdirSync(path.dirname(this.file), { recursive: true, mode: 0o700 });
const tmp = `${this.file}.tmp-${process.pid}`;
fs.writeFileSync(tmp, JSON.stringify(Object.fromEntries(this.seen)), { mode: 0o600 });
fs.renameSync(tmp, this.file);
}
}
const typeOf = (v: unknown): agent.JobType | null => (agent.jobTypeSchema.safeParse(v).success ? (v as agent.JobType) : null);
export function checkJob(raw: unknown, config: AgentConfig, seen: SeenJobs, now = new Date()): Verdict {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) return { ok: false, status: "refused", message: "Not a job.", id: null, type: null };
const j = raw as Record<string, unknown>;
const id = typeof j.id === "string" && /^job_[A-Za-z0-9_-]{8,64}$/.test(j.id) ? j.id : null;
const type = typeOf(j.type);
const refuse = (message: string): Verdict => ({ ok: false, status: "refused", message, id, type });
if (!id) return refuse("The job has no valid id.");
if (!verifyJob(j, config.hubJobKey)) return refuse("The signature doesn't match the Hub's key.");
if (j.serverId !== config.serverId) return refuse(`The job is for ${String(j.serverId)}, not this server.`);
const notBefore = Date.parse(String(j.notBefore));
const expiresAt = Date.parse(String(j.expiresAt));
if (!Number.isFinite(notBefore) || !Number.isFinite(expiresAt) || expiresAt <= notBefore) return refuse("The job's times aren't valid.");
if (now.getTime() > expiresAt) return { ok: false, status: "expired", message: `The job expired at ${new Date(expiresAt).toISOString()}; it wasn't run.`, id, type };
// A minute's leeway for clocks that differ a little.
if (now.getTime() < notBefore - 60_000) return refuse(`The job isn't due until ${new Date(notBefore).toISOString()}.`);
if (seen.has(id)) return refuse("This job was already received (a replay).");
if (!type) return refuse(`Unknown job type ${JSON.stringify(j.type)}.`);
if (config.mode === "central" && !agent.CENTRAL_JOB_TYPES.includes(type)) return refuse(`A central agent doesn't run ${type} jobs.`);
const parsed = agent.agentJobSchema.safeParse(raw);
if (!parsed.success) {
const issue = parsed.error.issues[0];
return refuse(`The job's ${issue?.path.join(".") || "content"} isn't valid: ${issue?.message ?? "unknown problem"}.`);
}
return { ok: true, job: parsed.data };
}
+36
View File
@@ -0,0 +1,36 @@
import { agent } from "@crosscode/contracts";
import { crosscodeArgs } from "../src/command";
import { job } from "./helpers";
const parsed = (over: Record<string, unknown>) => agent.agentJobSchema.parse(job(over));
describe("crosscode arguments for each job", () => {
it("install: behind frp with the Hub's ports and the release", () => {
expect(crosscodeArgs(parsed({
type: "install",
args: { church: "hope", products: ["events"], versions: { events: "events@2.4.1" }, domains: { events: "events.hope.co.za" }, ports: { events: { web: 20010, api: 20011 } } },
}))).toEqual(["install", "hope", "--products", "events", "--domain", "hope.co.za", "--events-domain", "events.hope.co.za", "--api-port", "20011", "--web-port", "20010", "--frp", "--version", "events@2.4.1"]);
});
it("deploy, adopt, restart, uninstall, purge, status", () => {
expect(crosscodeArgs(parsed({}))).toEqual(["deploy", "hope", "events", "--version", "events@2.4.1"]);
expect(crosscodeArgs(parsed({ args: { church: "hope", products: ["events"], check: true } }))).toEqual(["deploy", "hope", "events", "--check"]);
expect(crosscodeArgs(parsed({ type: "adopt", args: { church: "hope", eventsDir: "/var/www/hope-events", version: "events@2.2.0" } })))
.toEqual(["adopt", "hope", "--events-dir", "/var/www/hope-events", "--version", "events@2.2.0"]);
expect(crosscodeArgs(parsed({ type: "adopt-undo", args: { church: "hope" } }))).toEqual(["adopt", "hope", "--undo"]);
expect(crosscodeArgs(parsed({ type: "restart", args: { church: "hope" } }))).toEqual(["restart", "hope"]);
expect(crosscodeArgs(parsed({ type: "uninstall", args: { church: "hope" } }))).toEqual(["uninstall", "hope", "--confirm", "hope"]);
expect(crosscodeArgs(parsed({ type: "purge", args: { church: "hope" } }))).toEqual(["purge", "hope", "--confirm", "hope"]);
expect(crosscodeArgs(parsed({ type: "status", args: {} }))).toEqual(["status", "--no-fetch"]);
});
it("export writes to the file the agent gives", () => {
const j = parsed({ type: "export", args: { church: "hope", exportId: "exp_abcdefgh" } });
expect(crosscodeArgs(j, { exportFile: "/var/lib/crosscode-agent/exports/exp_abcdefgh.zip" })).toEqual(["export", "hope", "--out", "/var/lib/crosscode-agent/exports/exp_abcdefgh.zip"]);
});
it("the agent's own jobs run no crosscode command", () => {
expect(crosscodeArgs(parsed({ type: "check-ports", args: { ports: [20010] } }))).toBeNull();
expect(crosscodeArgs(parsed({ type: "update-agent", args: { version: "agent@1.0.1" } }))).toBeNull();
});
});
+44
View File
@@ -0,0 +1,44 @@
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
import { generateEd25519, generateX25519, signJob } from "@crosscode/contracts/signing";
import type { AgentConfig } from "../src/config";
export const hubKeys = generateEd25519();
export const serverKeys = { ed25519: generateEd25519(), x25519: generateX25519() };
export const config: AgentConfig = {
hubUrl: "https://hub.test",
serverId: "srv_test00001",
mode: "church",
hubJobKey: hubKeys.publicKey,
pollSeconds: 30,
statsSeconds: 60,
joinedAt: "2026-11-20T00:00:00.000Z",
};
export const NOW = new Date("2026-11-20T12:05:00.000Z");
let n = 0;
export function job(over: Record<string, unknown> = {}, key = hubKeys.privateKeyPem) {
n++;
return signJob({
schema: "crosscode.agent-job/1",
id: `job_test${String(n).padStart(6, "0")}`,
serverId: config.serverId,
type: "deploy",
args: { church: "hope", products: ["events"], versions: ["events@2.4.1"] },
config: null,
notBefore: "2026-11-20T12:00:00.000Z",
expiresAt: "2026-11-20T12:15:00.000Z",
...over,
}, key);
}
/** A temporary state folder per test file. */
export function tempState(): string {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "agent-test-"));
process.env.CROSSCODE_AGENT_STATE_DIR = dir;
process.env.CROSSCODE_AGENT_RUN_DIR = path.join(dir, "run");
return dir;
}
+112
View File
@@ -0,0 +1,112 @@
import fs from "node:fs";
import path from "node:path";
import { agent } from "@crosscode/contracts";
import { decryptExport } from "@crosscode/contracts/signing";
import { HubRequestError, type HubClient } from "../src/hub";
import { Agent } from "../src/loop";
import type { RunResult } from "../src/runner";
import { SeenJobs } from "../src/verify";
import { config, job, NOW, serverKeys, tempState } from "./helpers";
const state = tempState();
function fakeHub(jobs: unknown[]) {
const posts: { path: string; body: unknown }[] = [];
const puts: { path: string; body: Uint8Array }[] = [];
let down = false;
const hub: HubClient = {
async post(p, body) {
if (down) throw new HubRequestError("down", 0, "HUB_UNREACHABLE");
posts.push({ path: p, body });
if (p === "/api/agent/v1/poll") return { jobs: jobs.splice(0), pollSeconds: 30 };
return null;
},
async put(p, body) {
puts.push({ path: p, body });
return null;
},
};
return { hub, posts, puts, setDown: (v: boolean) => { down = v; } };
}
const ok = (over: Partial<RunResult> = {}): RunResult => ({ status: "succeeded", steps: [{ name: "x", startedAt: NOW.toISOString(), finishedAt: NOW.toISOString(), outcome: "ok" }], result: {}, log: ["==> done"], startedAt: NOW.toISOString(), finishedAt: NOW.toISOString(), ...over });
type Run = (args: string[], job: agent.AgentJob) => Promise<RunResult>;
function make(jobs: unknown[], run: jest.Mock<ReturnType<Run>, Parameters<Run>> = jest.fn<ReturnType<Run>, Parameters<Run>>(async () => ok())) {
const h = fakeHub(jobs);
const a = new Agent({ config, keys: { ed25519Pem: serverKeys.ed25519.privateKeyPem, x25519Pem: serverKeys.x25519.privateKeyPem }, hub: h.hub, seen: new SeenJobs(path.join(state, `seen-${Math.random()}.json`)), run, now: () => NOW });
return { a, run, ...h };
}
const reports = (posts: { path: string; body: unknown }[]) => posts.filter((p) => p.path === "/api/agent/v1/report").map((p) => p.body as agent.AgentReport);
describe("the agent loop", () => {
it("runs a valid job once and reports it", async () => {
const j = job();
const { a, run, posts } = make([j, j]);
await a.pollOnce();
expect(run).toHaveBeenCalledTimes(1);
expect(run.mock.calls[0]).toEqual([["deploy", "hope", "events", "--version", "events@2.4.1"], expect.objectContaining({ id: j.id })]);
// The second copy is a replay: refused quietly (the first one's report already went).
const rs = reports(posts);
expect(rs.map((r) => [r.jobId, r.status])).toEqual([[j.id, "succeeded"]]);
expect(agent.agentReportSchema.safeParse(rs[0]).success).toBe(true);
});
it("reports refused and expired jobs without running them", async () => {
const forged = { ...job(), signature: "A".repeat(86) };
const late = job({ expiresAt: "2026-11-20T12:01:00.000Z", notBefore: "2026-11-20T11:00:00.000Z" });
const { a, run, posts } = make([forged, late]);
await a.pollOnce();
expect(run).not.toHaveBeenCalled();
expect(reports(posts).map((r) => r.status)).toEqual(["refused", "expired"]);
});
it("keeps a report it can't send and sends it on the next poll", async () => {
const j = job();
let setDownLater: (v: boolean) => void = () => {};
const { a, posts, setDown } = make([j], jest.fn<ReturnType<Run>, Parameters<Run>>(async () => { setDownLater(true); return ok(); }));
setDownLater = setDown;
await a.pollOnce();
expect(reports(posts)).toEqual([]);
expect(fs.readdirSync(path.join(state, "outbox"))).toContain(`${j.id}.json`);
setDown(false);
await a.pollOnce();
expect(reports(posts).map((r) => r.jobId)).toEqual([j.id]);
expect(fs.readdirSync(path.join(state, "outbox"))).toEqual([]);
});
it("backs off while the Hub is unreachable", async () => {
const { a, setDown } = make([]);
setDown(true);
expect(await a.pollOnce()).toBe(60);
expect(await a.pollOnce()).toBe(120);
setDown(false);
expect(await a.pollOnce()).toBe(30);
});
it("export: encrypts the ZIP, uploads it, and only the report carries the key", async () => {
const j = job({ type: "export", args: { church: "hope", exportId: "exp_abcdefgh" } });
const run = jest.fn<ReturnType<Run>, Parameters<Run>>(async (args: string[]) => {
fs.writeFileSync(args[3]!, "PK zip bytes");
return ok({ result: { file: args[3], bytes: 12 } });
});
const { a, puts, posts } = make([j], run);
await a.pollOnce();
expect(puts).toHaveLength(1);
expect(puts[0]!.path).toBe("/api/agent/v1/exports/exp_abcdefgh");
const r = reports(posts)[0]!;
expect(r.status).toBe("succeeded");
const key = r.result.key as string;
expect(decryptExport(puts[0]!.body, key).toString()).toBe("PK zip bytes");
expect(Buffer.from(puts[0]!.body).includes(Buffer.from("PK zip"))).toBe(false);
expect(fs.readdirSync(path.join(state, "exports"))).toEqual([]);
});
it("a job that throws is reported as failed", async () => {
const { a, posts } = make([job()], jest.fn<ReturnType<Run>, Parameters<Run>>(async () => { throw new Error("boom"); }));
await a.pollOnce();
expect(reports(posts)[0]).toMatchObject({ status: "failed", message: "boom" });
});
});
@@ -0,0 +1,72 @@
import { spawn } from "node:child_process";
import fs from "node:fs";
import http from "node:http";
import os from "node:os";
import path from "node:path";
import { config, serverKeys } from "./helpers";
/**
* AGENT-1.0-PLAN §2 rule 1: the agent listens on nothing. Starts the built agent (dist/cli.js)
* against a fake Hub served by this test, lets it poll, then compares the agent's open sockets
* with every listening TCP socket the kernel knows (/proc/net/tcp, tcp6). Linux only (the
* servers); run `npm run build -w apps/agent` first.
*/
const cli = path.join(__dirname, "..", "dist", "cli.js");
const linux = process.platform === "linux" && fs.existsSync(cli);
const maybe = linux ? it : it.skip;
function listeningInodes(): Set<string> {
const out = new Set<string>();
for (const f of ["/proc/net/tcp", "/proc/net/tcp6"]) {
let text: string;
try { text = fs.readFileSync(f, "utf8"); } catch { continue; }
for (const line of text.split("\n").slice(1)) {
const cols = line.trim().split(/\s+/);
if (cols[3] === "0A") out.add(cols[9]!); // 0A = LISTEN
}
}
return out;
}
function socketInodes(pid: number): Set<string> {
const out = new Set<string>();
for (const fd of fs.readdirSync(`/proc/${pid}/fd`)) {
try {
const m = /^socket:\[(\d+)\]$/.exec(fs.readlinkSync(`/proc/${pid}/fd/${fd}`));
if (m) out.add(m[1]!);
} catch { /* closed meanwhile */ }
}
return out;
}
describe("the running agent", () => {
maybe("has no listening socket", async () => {
let polls = 0;
const hub = http.createServer((req, res) => {
if (req.url === "/api/agent/v1/poll") polls++;
res.setHeader("content-type", "application/json");
res.end(req.url === "/api/agent/v1/poll" ? JSON.stringify({ jobs: [], pollSeconds: 5 }) : "{}");
});
await new Promise<void>((r) => hub.listen(0, "127.0.0.1", r));
const port = (hub.address() as { port: number }).port;
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "agent-nolisten-"));
fs.writeFileSync(path.join(dir, "agent.json"), JSON.stringify({ ...config, hubUrl: `http://127.0.0.1:${port}`, pollSeconds: 5, statsSeconds: 15 }));
fs.writeFileSync(path.join(dir, "agent-ed25519.pem"), serverKeys.ed25519.privateKeyPem);
fs.writeFileSync(path.join(dir, "agent-x25519.pem"), serverKeys.x25519.privateKeyPem);
const child = spawn(process.execPath, [cli, "run"], {
env: { ...process.env, CROSSCODE_AGENT_CONFIG_DIR: dir, CROSSCODE_AGENT_STATE_DIR: path.join(dir, "state"), CROSSCODE_AGENT_ALLOW_NON_ROOT: "1", CROSSCODE_SITES: path.join(dir, "sites") },
stdio: "ignore",
});
try {
for (let i = 0; i < 40 && polls === 0; i++) await new Promise((r) => setTimeout(r, 250));
expect(polls).toBeGreaterThan(0);
const listening = listeningInodes();
const mine = [...socketInodes(child.pid!)].filter((inode) => listening.has(inode));
expect(mine).toEqual([]);
} finally {
child.kill("SIGTERM");
hub.close();
}
}, 30_000);
});
+79
View File
@@ -0,0 +1,79 @@
import { checkPorts, parseFrpsIni, parseSs, portAllowed } from "../src/central";
import { LogTail, redact } from "../src/redact";
import { churchProcesses, cpuPercent, parseMeminfo } from "../src/stats";
import fs from "node:fs";
import os from "node:os";
import path from "node:path";
describe("log lines are redacted before they leave the server", () => {
it.each([
["Error: connect postgresql://hope_events:Zx81kLmQ@localhost:5432/hope_events", "postgresql://hope_events:***@localhost"],
["JWT_SECRET=abcdefghijklmnop", "JWT_SECRET=***"],
['DB_PASSWORD: "hunter2 x"', "DB_PASSWORD: ***"],
["Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.payload.sig", "Authorization: Bearer ***"],
["licence CEH-7K2M-9QXA-P4RT-WD8C refused", "CEH-****"],
["token a3f9c1d27b8e4f60a3f9c1d27b8e4f60a3f9c1d2", "token ***"],
])("%s", (line, expected) => {
const out = redact(line);
expect(out).toContain(expected);
});
it("keeps ordinary lines", () => {
expect(redact("==> events: migrations")).toBe("==> events: migrations");
});
it("keeps only the last lines", () => {
const t = new LogTail(3);
t.push("1\n2\n3\n4");
t.push("\n5\n");
expect(t.end()).toEqual(["3", "4", "5"]);
});
});
describe("central server checks", () => {
const ini = "[common]\nbind_port = 7000\nproxy_bind_addr = 127.0.0.1\nallow_ports = 20000-29999,3000\n\n[web]\nproxy_bind_addr = 0.0.0.0\n";
it("reads [common] only", () => {
expect(parseFrpsIni(ini)).toEqual({ proxyBindAddr: "127.0.0.1", allowPorts: "20000-29999,3000" });
});
it("allow_ports ranges", () => {
expect(portAllowed(20010, "20000-29999,3000")).toBe(true);
expect(portAllowed(3000, "20000-29999,3000")).toBe(true);
expect(portAllowed(5000, "20000-29999,3000")).toBe(false);
expect(portAllowed(5000, null)).toBe(true);
});
it("ports must be allowed, open and bound to 127.0.0.1 only", () => {
const ss = "LISTEN 0 4096 127.0.0.1:20010 0.0.0.0:*\nLISTEN 0 4096 0.0.0.0:20011 0.0.0.0:*\nLISTEN 0 4096 [::1]:20012 [::]:*\n";
const res = checkPorts([20010, 20011, 20012, 20013], parseFrpsIni(ini), parseSs(ss));
expect(res.ports.map((p) => [p.port, p.ok])).toEqual([[20010, true], [20011, false], [20012, true], [20013, false]]);
expect(res.warnings.join(" ")).toMatch(/20011 is open on more than 127.0.0.1/);
expect(res.warnings.join(" ")).toMatch(/20013 isn't open/);
});
it("warns when proxy_bind_addr is missing", () => {
const res = checkPorts([20010], parseFrpsIni("[common]\nbind_port = 7000\n"), parseSs("LISTEN 0 1 127.0.0.1:20010 0.0.0.0:*"));
expect(res.warnings.join(" ")).toMatch(/proxy_bind_addr is not set/);
});
});
describe("stats", () => {
it("RAM from /proc/meminfo", () => {
const r = parseMeminfo("MemTotal: 4000000 kB\nMemFree: 100 kB\nMemAvailable: 1000000 kB\nSwapTotal: 2000 kB\nSwapFree: 500 kB\n");
expect(r).toEqual({ totalBytes: 4096000000, usedBytes: 3072000000, availableBytes: 1024000000, swapTotalBytes: 2048000, swapUsedBytes: 1536000 });
});
it("CPU % between two readings", () => {
expect(cpuPercent("cpu 100 0 100 700 100 0 0 0", "cpu 150 0 150 750 150 0 0 0")).toBe(50);
});
it("PM2 processes are matched to churches through church.env", () => {
const sites = fs.mkdtempSync(path.join(os.tmpdir(), "sites-"));
fs.mkdirSync(path.join(sites, "hope"));
fs.writeFileSync(path.join(sites, "hope", "church.env"), 'CHURCH=hope\nEVENTS_USER=cc-hope\nEVENTS_PM2="hope-events-backend hope-events-frontend"\n');
fs.mkdirSync(path.join(sites, "Bad Name"));
const map = churchProcesses(sites);
expect([...map.entries()]).toEqual([["cc-hope", [{ church: "hope", product: "events", names: ["hope-events-backend", "hope-events-frontend"] }]]]);
});
});
+61
View File
@@ -0,0 +1,61 @@
import path from "node:path";
import { generateEd25519 } from "@crosscode/contracts/signing";
import { checkJob, SeenJobs } from "../src/verify";
import { config, job, NOW, tempState } from "./helpers";
const dir = tempState();
const fresh = () => new SeenJobs(path.join(dir, `seen-${Math.random()}.json`));
describe("checking a job (AGENT-1.0-PLAN §4)", () => {
it("accepts a signed job for this server, in its window", () => {
expect(checkJob(job(), config, fresh(), NOW)).toMatchObject({ ok: true });
});
it("refuses a job signed with another key", () => {
expect(checkJob(job({}, generateEd25519().privateKeyPem), config, fresh(), NOW)).toMatchObject({ ok: false, status: "refused", message: expect.stringContaining("signature") });
});
it("refuses a job whose content was changed after signing", () => {
const j = job();
expect(checkJob({ ...j, args: { ...j.args, church: "other" } }, config, fresh(), NOW)).toMatchObject({ ok: false, status: "refused" });
});
it("refuses a job for another server", () => {
expect(checkJob(job({ serverId: "srv_other0001" }), config, fresh(), NOW)).toMatchObject({ ok: false, message: expect.stringContaining("not this server") });
});
it("reports an expired job as expired, and never runs it", () => {
expect(checkJob(job(), config, fresh(), new Date("2026-11-20T12:16:00.000Z"))).toMatchObject({ ok: false, status: "expired" });
});
it("refuses a job that isn't due yet (beyond a minute of clock difference)", () => {
expect(checkJob(job(), config, fresh(), new Date("2026-11-20T11:58:00.000Z"))).toMatchObject({ ok: false, status: "refused" });
expect(checkJob(job(), config, fresh(), new Date("2026-11-20T11:59:30.000Z"))).toMatchObject({ ok: true });
});
it("refuses a replayed job, also after a restart", () => {
const file = path.join(dir, "seen-replay.json");
const seen = new SeenJobs(file);
const j = job();
expect(checkJob(j, config, seen, NOW)).toMatchObject({ ok: true });
seen.add(j.id);
expect(checkJob(j, config, seen, NOW)).toMatchObject({ ok: false, message: expect.stringContaining("replay") });
expect(checkJob(j, config, new SeenJobs(file), NOW)).toMatchObject({ ok: false, message: expect.stringContaining("replay") });
});
it("refuses an unknown type, and arguments with shell characters", () => {
expect(checkJob(job({ type: "shell", args: { command: "id" } }), config, fresh(), NOW)).toMatchObject({ ok: false, message: expect.stringContaining("Unknown job type") });
expect(checkJob(job({ args: { church: "hope;reboot", products: ["events"] } }), config, fresh(), NOW)).toMatchObject({ ok: false, message: expect.stringContaining("isn't valid") });
expect(checkJob(job({ type: "adopt", args: { church: "hope", eventsDir: "/var/www/$(id)" } }), config, fresh(), NOW)).toMatchObject({ ok: false });
});
it("a central agent only runs check-ports, status and update-agent", () => {
const central = { ...config, mode: "central" as const };
expect(checkJob(job(), central, fresh(), NOW)).toMatchObject({ ok: false, message: expect.stringContaining("central") });
expect(checkJob(job({ type: "check-ports", args: { ports: [20010, 20011] } }), central, fresh(), NOW)).toMatchObject({ ok: true });
});
it("refuses things that aren't jobs", () => {
for (const raw of [null, "job", [], { id: "nope" }]) expect(checkJob(raw, config, fresh(), NOW)).toMatchObject({ ok: false, status: "refused" });
});
});
+9
View File
@@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": ".",
"types": ["node", "jest"],
"declaration": false
},
"include": ["src", "tests", "tsup.config.ts"]
}
+14
View File
@@ -0,0 +1,14 @@
import { defineConfig } from "tsup";
// One self-contained file (contracts and zod bundled in), so a server needs no node_modules to run
// the agent: /opt/crosscode/agent/current/apps/agent/dist/cli.js.
export default defineConfig({
entry: ["src/cli.ts"],
format: ["cjs"],
platform: "node",
target: "node24",
noExternal: [/.*/],
clean: true,
sourcemap: true,
banner: { js: "#!/usr/bin/env node" },
});
+26 -1
View File
@@ -5,7 +5,32 @@ All notable changes to this project are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
## [2.3.0] - 2026-11-17
Authenticator-app sign-in and recovery codes (ROADMAP-v3 phase 1c), and what installs and
updates from the Hub need (crosscode agent 1.0, Hub 2.1.0).
### Added
- **Two-factor with an authenticator app** (TOTP, RFC 6238) next to email and WhatsApp: Profile &
Security → Two-factor → "An authenticator app" shows a QR code (drawn in the browser) and the key;
the app's first code confirms it. Codes can't be used twice; nothing is sent, so there's no resend.
The secret is stored encrypted (`User.totpSecret`).
- **Recovery codes**: 10 single-use codes each time two-factor is turned on (any method), shown
once with Download and Copy; one works at sign-in in place of any code ("Use a recovery code").
A new set from Profile & Security (password needed). Using one, and making a new set, show in
the account's activity.
- **Admins see each account's method** in Users ("2FA: app / email / WhatsApp"); Reset 2FA also
removes the app's secret and the recovery codes.
- API (v2): `authenticator` in `/me/two-factor/enable` (answers with `secret` and `otpauthUrl`),
`recoveryCodes` from `/me/two-factor/confirm`, `POST /me/two-factor/recovery-codes`;
`twoFactorMethod` and `recoveryCodesLeft` on `/me`, `twoFactorMethod` on users.
- Migration `20261118150000_authenticator_app`.
- `env.template.json`: every `.env` variable with its layer, for installs and updates from the Hub.
- `scripts/export-church.ts`: the church's data as `crosscode.church-export/1` (for `crosscode export`).
- A site installed from the Hub registers by itself on first start when `HUB_LICENCE_KEY` is set
(with `HUB_ADMIN_NAME` / `HUB_ADMIN_EMAIL` as its admin contact), retrying while the Hub is
unreachable.
## [2.2.0] - 2026-11-17
+11
View File
@@ -12,6 +12,17 @@ Write for church admins, not developers: no environment variables, endpoints or
developer log is CHANGELOG.md. A tagged version without a section here shows "No details given".
-->
## 2.3.0
- **Sign in with an authenticator app.** Two-factor sign-in can now use an app on your phone
(Google Authenticator, Microsoft Authenticator, 1Password and others) instead of a code by email
or WhatsApp. It's the most secure option and works without signal. Set it up under Profile &
Security.
- **Recovery codes.** When you turn on two-factor you get 10 single-use recovery codes to keep
somewhere safe: if you lose your phone, one of them lets you sign in.
- **Admins can see how each person signs in**: the Users page shows whether each account uses an
app, email or WhatsApp for its codes.
## 2.2.0
- **AI use is now shown in tokens.** Site settings → AI shows how much of this month's AI allowance
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@crosscode/events-backend",
"version": "2.2.0",
"version": "2.3.0",
"description": "Event Management System Backend",
"main": "src/index.js",
"scripts": {
@@ -0,0 +1,18 @@
-- 2.3.0: two-factor with an authenticator app (TOTP), and recovery codes.
ALTER TYPE "TwoFactorMethod" ADD VALUE 'authenticator';
ALTER TABLE "User" ADD COLUMN "totpSecret" TEXT;
ALTER TABLE "User" ADD COLUMN "totpLastStep" INTEGER;
ALTER TABLE "TwoFactorChallenge" ADD COLUMN "pendingSecret" TEXT;
CREATE TABLE "RecoveryCode" (
"id" TEXT NOT NULL,
"userId" TEXT NOT NULL,
"codeHash" TEXT NOT NULL,
"usedAt" TIMESTAMP(3),
"createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT "RecoveryCode_pkey" PRIMARY KEY ("id")
);
CREATE UNIQUE INDEX "RecoveryCode_codeHash_key" ON "RecoveryCode"("codeHash");
CREATE INDEX "RecoveryCode_userId_idx" ON "RecoveryCode"("userId");
ALTER TABLE "RecoveryCode" ADD CONSTRAINT "RecoveryCode_userId_fkey" FOREIGN KEY ("userId") REFERENCES "User"("id") ON DELETE CASCADE ON UPDATE CASCADE;
ALTER TYPE "SecurityEventType" ADD VALUE 'recovery_code_used';
ALTER TYPE "SecurityEventType" ADD VALUE 'recovery_codes_replaced';
+24
View File
@@ -52,11 +52,16 @@ enum SecurityEventType {
password_reset
two_factor_enabled
two_factor_disabled
// 2.3.0: a recovery code signed in (the app or phone may be lost); a new set was made.
recovery_code_used
recovery_codes_replaced
}
enum TwoFactorMethod {
email
whatsapp
// 2.3.0: codes from an authenticator app (TOTP); nothing is sent.
authenticator
}
model User {
@@ -75,6 +80,11 @@ model User {
notificationPreference NotificationPreference @default(email)
twoFactorEnabled Boolean @default(false)
twoFactorMethod TwoFactorMethod?
// 2.3.0: the authenticator app's secret (encrypted, utils/encryption.js) and the last 30-second
// step a code was used for, so a code can't be used twice.
totpSecret String?
totpLastStep Int?
recoveryCodes RecoveryCode[]
// POPIA: opted in to "News & announcements" (marketing) bulk messages, per channel.
// Event updates to registrants don't depend on these. History in CommunicationConsent.
newsEmail Boolean @default(false)
@@ -713,6 +723,8 @@ model TwoFactorChallenge {
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
purpose String // 'login' | 'enable'
method TwoFactorMethod
// 2.3.0: an authenticator app being set up: its secret (encrypted) until the first code confirms it.
pendingSecret String?
codeHash String
attempts Int @default(0)
lastSentAt DateTime @default(now())
@@ -2353,3 +2365,15 @@ model AiCall {
@@index([createdAt])
@@index([userId, createdAt])
}
// 2.3.0: single-use codes for when the authenticator app (or phone) is lost. Only an HMAC is kept.
model RecoveryCode {
id String @id @default(uuid())
userId String
user User @relation(fields: [userId], references: [id], onDelete: Cascade)
codeHash String @unique
usedAt DateTime?
createdAt DateTime @default(now())
@@index([userId])
}
@@ -0,0 +1,57 @@
/**
* Writes the church's data as crosscode.church-export/1 (src/services/churchExport.ts) into a
* folder: export.json and README.txt. `crosscode export` runs it, copies public/uploads into the
* folder's uploads/ and zips it. Run from backend/: npx tsx scripts/export-church.ts --out-dir <dir>
*/
import 'dotenv/config';
import fs from 'node:fs';
import path from 'node:path';
import { buildExport, readExportInput } from '../src/services/churchExport';
import { prisma } from '../src/v2/legacy';
const README = `Your church's data from Cross Events
=====================================
export.json holds your events, people, registrations, payments, check-ins and volunteer
rosters, in plain terms (crosscode.church-export/1). Records refer to each other with short
references made for this file: "p12" is a person, "e3" an event, "t5" a ticket type and "r40"
a registration. They mean nothing outside this file.
people everyone with an account: name, email, phone, role, what they agreed to
events each event with its sessions and ticket types (prices in cents)
registrations who registered for what, the people attending, their answers to the form
payments amounts in cents, how and when they were paid, the provider's reference
checkIns who was checked in to which event, and when
rosters volunteer roles and who served
settings the site's name, contact details and branding
files uploaded files, in the uploads/ folder next to this file
Not included: passwords, security codes and logs about how the site ran.
`;
function listFiles(dir: string, base = dir): { path: string; bytes: number }[] {
if (!fs.existsSync(dir)) return [];
return fs.readdirSync(dir, { withFileTypes: true }).flatMap(d => {
const full = path.join(dir, d.name);
if (d.isDirectory()) return d.name === 'tickets-temp' ? [] : listFiles(full, base);
if (!d.isFile() || !/^[A-Za-z0-9._-]+$/.test(d.name)) return [];
return [{ path: `uploads/${path.relative(base, full).split(path.sep).join('/')}`, bytes: fs.statSync(full).size }];
});
}
async function main() {
const i = process.argv.indexOf('--out-dir');
const out = i > 0 ? process.argv[i + 1] : '';
if (!out) throw new Error('Usage: export-church.ts --out-dir <dir>');
fs.mkdirSync(out, { recursive: true });
const { version } = require('../package.json') as { version: string };
const data = buildExport(await readExportInput(version));
data.files = listFiles(path.join(__dirname, '..', 'public', 'uploads'));
fs.writeFileSync(path.join(out, 'export.json'), `${JSON.stringify(data, null, 2)}\n`);
fs.writeFileSync(path.join(out, 'README.txt'), README);
console.log(`export.json: ${data.people.length} people, ${data.events.length} events, ${data.registrations.length} registrations, ${data.files.length} files`);
}
main()
.catch(e => { console.error(`Export failed: ${(e as Error).message}`); process.exitCode = 1; })
.finally(() => prisma.$disconnect().catch(() => {}));
+3 -1
View File
@@ -396,7 +396,9 @@ app.listen(PORT, () => {
// backup, temp cleanup) run on the pg-boss job queue — see src/jobs/index.js.
require('./jobs').startJobs().catch(e => console.warn('[jobs] Not started:', e?.message || e));
// Check in with the Hub 30 s after starting, so the licence is fresh without waiting 15 minutes.
// Installs made from the Hub register by themselves (HUB_LICENCE_KEY), then check in with the
// Hub 30 s after starting, so the licence is fresh without waiting 15 minutes.
require('./hub/client').connectFromEnv().catch(() => {});
require('./hub/client').heartbeatSoon();
// Scheduled emails worker (polling)
@@ -11,6 +11,9 @@ const {
resendChallenge,
clearTrustedDevices,
requiresTwoFactor,
replaceRecoveryCodes,
recoveryCodesLeft,
clearTwoFactor,
} = require('../utils/twoFactor');
// ─── Helpers ────────────────────────────────────────────────────────────────
@@ -99,7 +102,7 @@ const startEnableTwoFactor = async (req, res) => {
}
if (!TWO_FACTOR_METHODS.includes(method)) {
res.status(400);
throw new Error('Choose email or WhatsApp for your verification codes');
throw new Error('Choose email, WhatsApp or an authenticator app for your verification codes');
}
if (!(await availableMethods(user)).includes(method)) {
res.status(400);
@@ -109,7 +112,8 @@ const startEnableTwoFactor = async (req, res) => {
}
// Unlike login, don't fall back to email here — the point of this step is to
// prove the channel the user picked actually reaches them.
// prove the channel the user picked actually reaches them. For an authenticator app the
// answer carries the secret and its otpauth:// link (shown as a QR code) until confirmed.
res.json(await createChallenge(user, 'enable', method, { allowFallback: false }));
} catch (error) {
res.status(errorStatus(error, res)).json({ message: safeErrorMessage(error) });
@@ -124,13 +128,20 @@ const confirmEnableTwoFactor = async (req, res) => {
const { challengeId, code } = req.body || {};
const challenge = await verifyChallenge(challengeId, code, 'enable', req.user.id);
// An authenticator app: the confirmed secret is kept (encrypted). Another method: any app is
// forgotten. Either way a fresh set of recovery codes, shown once (2.3.0).
await prisma.user.update({
where: { id: req.user.id },
data: { twoFactorEnabled: true, twoFactorMethod: challenge.method, updatedAt: new Date() },
data: {
twoFactorEnabled: true, twoFactorMethod: challenge.method, updatedAt: new Date(),
...(challenge.method === 'authenticator' ? { totpSecret: challenge.pendingSecret } : { totpSecret: null, totpLastStep: null }),
},
});
await prisma.twoFactorChallenge.update({ where: { id: challenge.id }, data: { pendingSecret: null } }).catch(() => {});
const recoveryCodes = await replaceRecoveryCodes(req.user.id);
logSecurityEvent({ userId: req.user.id, type: 'two_factor_enabled', ip: getClientIp(req), userAgent: req.headers['user-agent'] }).catch(() => {});
res.json({ twoFactorEnabled: true, twoFactorMethod: challenge.method });
res.json({ twoFactorEnabled: true, twoFactorMethod: challenge.method, recoveryCodes });
} catch (error) {
res.status(errorStatus(error, res)).json({ message: safeErrorMessage(error) });
}
@@ -157,7 +168,7 @@ const disableTwoFactor = async (req, res) => {
await prisma.user.update({
where: { id: user.id },
data: { twoFactorEnabled: false, twoFactorMethod: null, updatedAt: new Date() },
data: { ...(await clearTwoFactor(user.id)), updatedAt: new Date() },
});
await clearTrustedDevices(user.id);
if (user.twoFactorEnabled) {
@@ -170,6 +181,27 @@ const disableTwoFactor = async (req, res) => {
}
};
// @desc A new set of recovery codes (the old ones stop working), shown once (2.3.0)
// @route POST /api/v2/me/two-factor/recovery-codes
// @access Private
const replaceMyRecoveryCodes = async (req, res) => {
try {
const { password } = req.body || {};
const user = await prisma.user.findUnique({ where: { id: req.user.id } });
if (!user) { res.status(404); throw new Error('User not found'); }
if (!user.twoFactorEnabled) { res.status(400); throw new Error('Turn on two-factor authentication first.'); }
if (typeof password !== 'string' || !(await comparePassword(password, user.password))) {
res.status(400);
throw new Error('Incorrect password');
}
const recoveryCodes = await replaceRecoveryCodes(user.id);
logSecurityEvent({ userId: user.id, type: 'recovery_codes_replaced', ip: getClientIp(req), userAgent: req.headers['user-agent'] }).catch(() => {});
res.json({ recoveryCodes });
} catch (error) {
res.status(errorStatus(error, res)).json({ message: safeErrorMessage(error) });
}
};
// @desc Re-confirm the logged-in user's password without starting a new session
// (the self-service kiosk uses this before letting the event be changed)
// @route POST /api/users/verify-password
@@ -220,7 +252,7 @@ const getUserProfile = async (req, res) => {
const noticePromptPending = !req.impersonation && await require('../utils/privacyNotice').noticePromptPending({ noticeVersionAccepted });
// Campus-scoped staff: the campuses they're limited to (null = every campus).
const campusScope = (await require('../utils/campusScope').scopeFor(req))?.campusIds || null;
res.json({ ...rest, twoFactorRequired: requiresTwoFactor(user), newsPromptPending: !newsAskedAt && !req.impersonation, noticePromptPending, campusScope });
res.json({ ...rest, twoFactorRequired: requiresTwoFactor(user), recoveryCodesLeft: user.twoFactorEnabled ? await recoveryCodesLeft(user.id) : 0, newsPromptPending: !newsAskedAt && !req.impersonation, noticePromptPending, campusScope });
} else {
res.status(404);
throw new Error('User not found');
@@ -253,7 +285,7 @@ const getUsers = async (req, res) => {
const select = {
id: true, name: true, email: true, role: true,
phoneNumber: true, notificationPreference: true, twoFactorEnabled: true, createdAt: true, updatedAt: true, isActive: true
phoneNumber: true, notificationPreference: true, twoFactorEnabled: true, twoFactorMethod: true, createdAt: true, updatedAt: true, isActive: true
};
// Build filters
@@ -406,8 +438,8 @@ const updateUser = async (req, res) => {
// but never on — enabling requires the user to prove the channel themselves.
const resetTwoFactor = twoFactorEnabled === false && user.twoFactorEnabled;
if (resetTwoFactor) {
data.twoFactorEnabled = false;
data.twoFactorMethod = null;
// The app's secret and the recovery codes go too (2.3.0).
Object.assign(data, await clearTwoFactor(user.id));
}
// Promoted to a role that requires 2FA without having it: end their sessions
@@ -443,6 +475,7 @@ const updateUser = async (req, res) => {
phoneNumber: true,
notificationPreference: true,
twoFactorEnabled: true,
twoFactorMethod: true,
createdAt: true,
updatedAt: true,
isActive: true,
@@ -820,6 +853,7 @@ module.exports = {
startEnableTwoFactor,
confirmEnableTwoFactor,
disableTwoFactor,
replaceMyRecoveryCodes,
verifyPassword,
getUserProfile,
updateUserProfile,
+30 -1
View File
@@ -216,4 +216,33 @@ function heartbeatSoon(delayMs = 30 * 1000) {
soonTimer.unref?.();
}
module.exports = { PROTOCOL_VERSION, HUB_KEYS, hubUrl, HubError, signedPost, connect, heartbeat, heartbeatSoon, requestUpdate, cancelUpdateRequest, snoozeUpdate };
/**
* Installs made from the Hub (crosscode install, docs/AGENT-1.0-PLAN.md §7) have the licence key in
* HUB_LICENCE_KEY: the site registers by itself on its first start, instead of in /setup. Does
* nothing once registered, or without the key. Retries a few times (the Hub may be briefly
* unreachable); a refused key is logged and left for /setup. Never throws.
*/
async function connectFromEnv({ attempts = 5, delayMs = 60 * 1000, sleep = ms => new Promise(r => setTimeout(r, ms).unref?.()) } = {}) {
const key = String(process.env.HUB_LICENCE_KEY || '').trim();
if (!key) return { connected: false, reason: 'no_key' };
for (let i = 1; i <= attempts; i++) {
try {
const stored = await read(['hub_install_id']);
if (stored.hub_install_id) return { connected: false, reason: 'already' };
const name = String(process.env.HUB_ADMIN_NAME || '').trim() || null;
const email = String(process.env.HUB_ADMIN_EMAIL || '').trim() || null;
await connect(key, name || email ? { name, email } : null);
console.log('[hub] Registered with the Hub using HUB_LICENCE_KEY.');
return { connected: true };
} catch (e) {
const status = e?.statusCode || 0;
console.warn(`[hub] Registering with HUB_LICENCE_KEY failed (attempt ${i} of ${attempts}):`, e?.message || e);
// The Hub said no (unknown, revoked or in-use key): retrying won't help.
if (status >= 400 && status < 500) return { connected: false, reason: e?.code || 'refused' };
if (i < attempts) await sleep(delayMs);
}
}
return { connected: false, reason: 'unreachable' };
}
module.exports = { PROTOCOL_VERSION, HUB_KEYS, hubUrl, HubError, signedPost, connect, connectFromEnv, heartbeat, heartbeatSoon, requestUpdate, cancelUpdateRequest, snoozeUpdate };
+6 -1
View File
@@ -16,7 +16,8 @@ const auth = require('../config/auth') as {
};
const twoFactor = require('../utils/twoFactor') as {
createChallenge(user: any, purpose: string, method: string): Promise<{ challengeId: string; method: string; destination: string | null }>;
verifyChallenge(challengeId: string, code: string, purpose: string): Promise<{ userId: string; method: string }>;
verifyChallenge(challengeId: string, code: string, purpose: string): Promise<{ userId: string; method: string; usedRecoveryCode?: boolean }>;
recoveryCodesLeft(userId: string): Promise<number>;
issueTrustedDevice(userId: string, userAgent?: string): Promise<string>;
isTrustedDevice(userId: string, deviceToken?: string): Promise<boolean>;
clearTrustedDevices(userId: string): Promise<unknown>;
@@ -279,6 +280,10 @@ export async function verifyLoginCode(input: { challengeId: string; code: string
user = await prisma.user.update({ where: { id: user.id }, data: { twoFactorEnabled: true, twoFactorMethod: challenge.method as 'email' | 'whatsapp', updatedAt: new Date() }, select: LOGIN_USER_SELECT });
logSecurityEvent({ userId: user.id, type: 'two_factor_enabled', ip: info.ip, userAgent: info.userAgent }).catch(() => {});
}
// 2.3.0: a recovery code was used instead (the app or phone may be lost): it's in the activity log.
if (challenge.usedRecoveryCode) {
logSecurityEvent({ userId: user.id, type: 'recovery_code_used', ip: info.ip, userAgent: info.userAgent }).catch(() => {});
}
const extra = input.rememberDevice === true ? { deviceToken: await twoFactor.issueTrustedDevice(user.id, info.userAgent) } : {};
return completeLogin(user, info, extra);
}
@@ -0,0 +1,142 @@
/**
* The church's data when it leaves: crosscode.church-export/1 (packages/contracts,
* docs/AGENT-1.0-PLAN.md §9). Plain terms and our own field names: no tables, Prisma models or
* internal ids. Records point at each other with references made for the export (p1, e1, t1, r1)
* that mean nothing outside the file. `crosscode export` runs scripts/export-church.ts, which
* writes this as export.json next to the uploads.
*
* Not included: passwords, 2FA secrets, tokens, provider keys, audit and access logs (they're
* about the site's running, not the church's data), and analytics.
*/
import { prisma } from '../v2/legacy';
type Row = Record<string, any>; // eslint-disable-line @typescript-eslint/no-explicit-any -- rows from Prisma
const iso = (d: unknown): string | null => (d instanceof Date ? d.toISOString() : d ? new Date(String(d)).toISOString() : null);
const cents = (rand: unknown): number => Math.round(Number(rand || 0) * 100);
/** Hands out p1, p2 … for ids, in the order they're first seen. */
class Refs {
private map = new Map<string, string>();
constructor(private prefix: string) {}
of(id: string | null | undefined): string | null {
if (!id) return null;
let r = this.map.get(id);
if (!r) {
r = `${this.prefix}${this.map.size + 1}`;
this.map.set(id, r);
}
return r;
}
has(id: string | null | undefined): boolean {
return !!id && this.map.has(id);
}
}
/** Settings worth giving back (never secrets: those are stored encrypted under other keys). */
export const EXPORT_SETTING_KEYS = [
'org_name', 'org_tagline', 'org_email', 'org_phone', 'org_address', 'org_website', 'brand_primary_color', 'logo_url', 'favicon_url',
'app_name', 'timezone', 'currency', 'privacy_contact_email',
];
export type ExportInput = {
users: Row[]; events: Row[]; registrations: Row[]; payments: Row[]; usages: Row[]; slots: Row[]; roles: Row[];
settings: Row[]; fields: Row[]; version: string;
};
/** Builds the export from rows (kept separate from the database reads, so it can be tested). */
export function buildExport(input: ExportInput, now = new Date()) {
const people = new Refs('p');
const events = new Refs('e');
const tickets = new Refs('t');
const regs = new Refs('r');
const labels = new Map(input.fields.map(f => [f.id, String(f.label)]));
const settings = Object.fromEntries(input.settings.filter(s => EXPORT_SETTING_KEYS.includes(s.key)).map(s => [s.key, String(s.value)]));
const out = {
schema: 'crosscode.church-export/1' as const,
exportedAt: now.toISOString(),
product: 'events' as const,
productVersion: input.version,
church: { name: settings.org_name || 'Church', timezone: process.env.APP_TIMEZONE || 'Africa/Johannesburg' },
people: input.users.map(u => ({
ref: people.of(u.id)!,
name: String(u.name || ''),
email: u.email || null,
phone: u.phoneNumber || null,
role: String(u.role || 'user'),
createdAt: iso(u.createdAt),
consent: { email: typeof u.newsEmail === 'boolean' ? u.newsEmail : null, whatsapp: typeof u.newsWhatsApp === 'boolean' ? u.newsWhatsApp : null, privacyNoticeAcceptedAt: null },
})),
events: input.events.map(e => ({
ref: events.of(e.id)!,
title: String(e.title || ''),
description: e.description || null,
startsAt: iso(e.startDate),
endsAt: iso(e.endDate),
location: e.location || null,
sessions: (e.sessions || []).map((s: Row) => ({ label: s.label || null, startsAt: iso(s.startsAt), endsAt: iso(s.endsAt) })),
ticketTypes: (e.eventOptions || []).map((o: Row) => ({ ref: tickets.of(o.id)!, name: String(o.name || ''), priceCents: cents(o.price), currency: 'ZAR' })),
capacity: null as number | null,
published: !!e.isActive && !e.isHidden,
})),
registrations: input.registrations.filter(r => events.has(r.eventId)).map(r => ({
ref: regs.of(r.id)!,
eventRef: events.of(r.eventId)!,
personRef: people.has(r.userId) ? people.of(r.userId) : null,
status: String(r.status),
createdAt: iso(r.createdAt),
attendees: (r.attendees || []).map((a: Row) => ({
name: String(a.name || ''),
email: a.email || null,
ticketTypeRef: tickets.has(a.eventOptionId) ? tickets.of(a.eventOptionId) : null,
checkedInAt: iso(a.ticket?.usages?.[0]?.scannedAt),
})),
answers: (r.formResponses || []).flatMap((fr: Row) => (fr.answers || []).map((x: Row) => ({ question: labels.get(x.fieldId) || 'Question', answer: String(x.value ?? '') }))),
})),
payments: input.payments.map(p => ({
registrationRef: regs.has(p.registrationId) ? regs.of(p.registrationId) : null,
amountCents: cents(p.amount),
currency: 'ZAR',
method: String(p.method || 'unknown'),
status: String(p.status || 'completed'),
paidAt: iso(p.createdAt),
reference: p.externalId || null,
})),
checkIns: input.usages.filter(u => events.has(u.ticket?.eventId)).map(u => ({
eventRef: events.of(u.ticket.eventId)!,
personRef: people.has(u.ticket?.holderUserId || u.ticket?.userId) ? people.of(u.ticket.holderUserId || u.ticket.userId) : null,
name: u.ticket?.attendee?.name || null,
at: iso(u.scannedAt),
})),
rosters: input.slots.map(s => {
const role = input.roles.find(r => r.id === s.roleId);
return {
eventRef: role && events.has(role.eventId) ? events.of(role.eventId) : null,
role: String(role?.name || 'Volunteer'),
personRef: people.has(s.userId) ? people.of(s.userId) : null,
startsAt: iso(role?.startsAt),
endsAt: iso(role?.endsAt),
};
}),
settings,
files: [] as { path: string; bytes: number }[],
};
return out;
}
/** Reads everything the export needs. */
export async function readExportInput(version: string): Promise<ExportInput> {
const [users, events, registrations, payments, usages, slots, roles, settings, fields] = await Promise.all([
prisma.user.findMany({ orderBy: { createdAt: 'asc' } }),
prisma.event.findMany({ orderBy: { startDate: 'asc' }, include: { sessions: { orderBy: { order: 'asc' } }, eventOptions: true } }),
prisma.registration.findMany({ orderBy: { createdAt: 'asc' }, include: { attendees: { orderBy: { order: 'asc' }, include: { ticket: { include: { usages: { orderBy: { scannedAt: 'asc' }, take: 1 } } } } }, formResponses: { include: { answers: true } } } }),
prisma.payment.findMany({ orderBy: { createdAt: 'asc' } }),
prisma.ticketUsage.findMany({ orderBy: { scannedAt: 'asc' }, include: { ticket: { include: { attendee: true } } } }),
prisma.volunteerSlot.findMany(),
prisma.servingRole.findMany(),
prisma.appSetting.findMany(),
prisma.eventFormField.findMany(),
]);
return { users, events, registrations, payments, usages, slots, roles, settings, fields, version } as ExportInput;
}
+80
View File
@@ -0,0 +1,80 @@
/**
* Codes from an authenticator app (2.3.0; RFC 6238 TOTP: SHA-1, 6 digits, 30 seconds), the same
* as Cross Code Hub's. `matchStep` says which 30-second step a code belongs to, so a code that was
* used once is refused the second time (utils/twoFactor.js keeps the last step used).
*/
const crypto = require('crypto');
const BASE32 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567';
const STEP = 30;
function base32Encode(buf) {
let bits = 0;
let value = 0;
let out = '';
for (const byte of buf) {
value = (value << 8) | byte;
bits += 8;
while (bits >= 5) {
out += BASE32[(value >>> (bits - 5)) & 31];
bits -= 5;
}
}
if (bits > 0) out += BASE32[(value << (5 - bits)) & 31];
return out;
}
function base32Decode(str) {
let bits = 0;
let value = 0;
const out = [];
for (const ch of String(str).toUpperCase().replace(/=+$/, '')) {
const idx = BASE32.indexOf(ch);
if (idx === -1) continue;
value = (value << 5) | idx;
bits += 5;
if (bits >= 8) {
out.push((value >>> (bits - 8)) & 255);
bits -= 8;
}
}
return Buffer.from(out);
}
/** A new secret: 20 random bytes, base32 (what the app stores). */
function generateSecret() {
return base32Encode(crypto.randomBytes(20));
}
function codeAt(secret, counter) {
const buf = Buffer.alloc(8);
buf.writeBigUInt64BE(BigInt(counter));
const hmac = crypto.createHmac('sha1', base32Decode(secret)).update(buf).digest();
const offset = hmac[hmac.length - 1] & 0xf;
const num = ((hmac[offset] & 0x7f) << 24) | (hmac[offset + 1] << 16) | (hmac[offset + 2] << 8) | hmac[offset + 3];
return String(num % 1_000_000).padStart(6, '0');
}
/**
* The step `code` is right for (now, or one step either side for clock drift), or null. Steps at
* or before `afterStep` don't count: that code was already used.
*/
function matchStep(secret, code, { now = Date.now(), afterStep = -1 } = {}) {
const clean = String(code || '').replace(/\D/g, '');
if (clean.length !== 6) return null;
const counter = Math.floor(now / 1000 / STEP);
for (const c of [counter - 1, counter, counter + 1]) {
if (c <= afterStep) continue;
if (crypto.timingSafeEqual(Buffer.from(codeAt(secret, c)), Buffer.from(clean))) return c;
}
return null;
}
/** The otpauth:// link an authenticator app scans (as a QR code). */
function otpauthUrl(secret, account, issuer) {
const label = `${issuer}:${account}`;
return `otpauth://totp/${encodeURIComponent(label)}?secret=${secret}&issuer=${encodeURIComponent(issuer)}&algorithm=SHA1&digits=6&period=${STEP}`;
}
module.exports = { STEP, generateSecret, codeAt, matchStep, otpauthUrl, base32Encode, base32Decode };
+108 -4
View File
@@ -16,7 +16,9 @@ const RESEND_COOLDOWN_MS = 60 * 1000; // minimum gap between sends
const CHALLENGE_MAX_AGE_MS = 30 * 60 * 1000; // after this, log in again rather than resend
const TRUSTED_DEVICE_TTL_MS = 30 * 24 * 60 * 60 * 1000;
const METHODS = ['email', 'whatsapp'];
// 2.3.0: 'authenticator' — codes from an app (TOTP, utils/totp.js); nothing is sent.
const METHODS = ['email', 'whatsapp', 'authenticator'];
const RECOVERY_CODE_COUNT = 10;
// Roles that must use two-factor authentication.
const REQUIRED_ROLES = ['admin', 'supervisor'];
@@ -67,9 +69,75 @@ async function availableMethods(user) {
const methods = [];
if (hasRealEmail(user)) methods.push('email');
if (isValidZAPhone(user?.phoneNumber) && (await isConfigured())) methods.push('whatsapp');
// An authenticator app needs nothing from the site, so it's always possible.
methods.push('authenticator');
return methods;
}
// ─── Authenticator app (2.3.0) ───────────────────────────────────────────────
const totp = require('./totp');
const { encrypt, decrypt } = require('./encryption');
/** A new secret for setting up an app, with the link and text to show (not saved yet). */
function newAuthenticatorSecret(user) {
const { getSettingSync } = require('./settingsCache');
const issuer = getSettingSync('org_name', process.env.ORG_NAME || 'Cross Events');
const secret = totp.generateSecret();
return { secret, otpauthUrl: totp.otpauthUrl(secret, user.email, issuer) };
}
// Recovery codes: xxxxx-xxxxx from an alphabet without look-alikes; stored as an HMAC.
const RECOVERY_ALPHABET = 'abcdefghjkmnpqrstuvwxyz23456789';
const normaliseRecovery = (code) => String(code || '').toLowerCase().replace(/[^a-z0-9]/g, '');
const hashRecovery = (code) => hmac(`2fa-recovery:${normaliseRecovery(code)}`);
const looksLikeRecoveryCode = (code) => normaliseRecovery(code).length === 10;
function randomRecoveryCode() {
let s = '';
for (let i = 0; i < 10; i++) s += RECOVERY_ALPHABET[crypto.randomInt(0, RECOVERY_ALPHABET.length)];
return `${s.slice(0, 5)}-${s.slice(5)}`;
}
/** A new set of recovery codes (the old ones stop working). Returns the codes to show once. */
async function replaceRecoveryCodes(userId, db = prisma) {
const codes = Array.from({ length: RECOVERY_CODE_COUNT }, randomRecoveryCode);
await db.recoveryCode.deleteMany({ where: { userId } });
await db.recoveryCode.createMany({ data: codes.map(c => ({ userId, codeHash: hashRecovery(c) })) });
return codes;
}
async function recoveryCodesLeft(userId) {
return prisma.recoveryCode.count({ where: { userId, usedAt: null } });
}
/** Uses a recovery code if it's one of this user's unused ones (single use, race-safe). */
async function useRecoveryCode(userId, code) {
if (!looksLikeRecoveryCode(code)) return false;
const used = await prisma.recoveryCode.updateMany({ where: { userId, codeHash: hashRecovery(code), usedAt: null }, data: { usedAt: new Date() } });
return used.count === 1;
}
/**
* Checks an app code against `secret` and records its step, so the same code can't be used again
* (the update only succeeds if no later step was recorded meanwhile).
*/
async function useAuthenticatorCode(user, secret, code) {
const step = totp.matchStep(secret, code, { afterStep: user.totpLastStep ?? -1 });
if (step === null) return false;
const saved = await prisma.user.updateMany({
where: { id: user.id, OR: [{ totpLastStep: null }, { totpLastStep: { lt: step } }] },
data: { totpLastStep: step },
});
return saved.count === 1;
}
/** Turns 2FA off completely: the app's secret and the recovery codes go too. */
async function clearTwoFactor(userId, db = prisma) {
await db.recoveryCode.deleteMany({ where: { userId } });
return { twoFactorEnabled: false, twoFactorMethod: null, totpSecret: null, totpLastStep: null };
}
/**
* True when this user's role must use two-factor authentication.
* TWO_FACTOR_REQUIRED_ROLES=false turns the requirement off (break-glass).
@@ -143,6 +211,20 @@ async function deliverCode(user, method, code, { allowFallback = true } = {}) {
* purpose for this user is discarded, so only the newest code works.
*/
async function createChallenge(user, purpose, method, options) {
if (method === 'authenticator') {
// Nothing is sent: the code comes from the app. Setting it up carries the new secret,
// encrypted, until the first code proves the app has it.
const setup = purpose === 'enable' ? newAuthenticatorSecret(user) : null;
// (Looked up here: the user passed in may be a selection that leaves the secret out.)
if (purpose !== 'enable' && !(await prisma.user.findUnique({ where: { id: user.id }, select: { totpSecret: true } }))?.totpSecret) {
throw fail('No authenticator app is set up for this account.', 400);
}
await prisma.twoFactorChallenge.deleteMany({ where: { userId: user.id, purpose } });
const challenge = await prisma.twoFactorChallenge.create({
data: { userId: user.id, purpose, method: 'authenticator', codeHash: '', pendingSecret: setup ? encrypt(setup.secret) : null, expiresAt: new Date(Date.now() + CODE_TTL_MS) },
});
return { challengeId: challenge.id, method: 'authenticator', destination: 'your authenticator app', ...(setup ? { secret: setup.secret, otpauthUrl: setup.otpauthUrl } : {}) };
}
const code = generateCode();
const sent = await deliverCode(user, method, code, options);
@@ -185,9 +267,23 @@ async function verifyChallenge(challengeId, code, purpose, userId) {
if (counted.count !== 1) throw invalid();
const challenge = await prisma.twoFactorChallenge.findUnique({ where: { id: challengeId }, include: { user: true } });
if (!challenge || !safeEqual(challenge.codeHash, hashCode(submitted))) {
let ok = false;
let usedRecoveryCode = false;
if (challenge?.method === 'authenticator') {
const stored = purpose === 'enable' ? challenge.pendingSecret : challenge.user?.totpSecret;
ok = !!stored && (await useAuthenticatorCode(challenge.user, decrypt(stored), submitted));
} else if (challenge) {
ok = safeEqual(challenge.codeHash, hashCode(submitted));
}
// At sign-in, a recovery code works in place of any code (2.3.0).
if (!ok && challenge && purpose === 'login' && challenge.user?.twoFactorEnabled && (await useRecoveryCode(challenge.userId, code))) {
ok = true;
usedRecoveryCode = true;
}
if (!ok) {
const left = challenge ? MAX_ATTEMPTS - challenge.attempts : 0;
throw fail(left > 0 ? `Incorrect code. ${left} attempt${left === 1 ? '' : 's'} left.` : 'Too many incorrect attempts. Please request a new code.');
const again = challenge?.method === 'authenticator' ? 'Please log in again.' : 'Please request a new code.';
throw fail(left > 0 ? `Incorrect code. ${left} attempt${left === 1 ? '' : 's'} left.` : `Too many incorrect attempts. ${again}`);
}
// Single use — if two requests race with the right code, only one wins.
@@ -197,7 +293,7 @@ async function verifyChallenge(challengeId, code, purpose, userId) {
});
if (used.count !== 1) throw invalid();
return challenge;
return { ...challenge, usedRecoveryCode };
}
/** Replaces the code on an existing challenge and sends it again. */
@@ -208,6 +304,7 @@ async function resendChallenge(challengeId, purpose, userId) {
const challenge = await prisma.twoFactorChallenge.findUnique({ where: { id: challengeId }, include: { user: true } });
if (!challenge || challenge.purpose !== purpose || challenge.usedAt || (userId && challenge.userId !== userId)) throw expired();
if (Date.now() - challenge.createdAt.getTime() > CHALLENGE_MAX_AGE_MS) throw expired();
if (challenge.method === 'authenticator') throw fail('Nothing is sent for an authenticator app: open the app for the current code, or use one of your recovery codes.');
const wait = RESEND_COOLDOWN_MS - (Date.now() - challenge.lastSentAt.getTime());
if (wait > 0) throw fail(`Please wait ${Math.ceil(wait / 1000)} seconds before requesting another code.`, 429);
@@ -264,6 +361,13 @@ async function clearTrustedDevices(userId) {
module.exports = {
METHODS,
RECOVERY_CODE_COUNT,
replaceRecoveryCodes,
recoveryCodesLeft,
useRecoveryCode,
useAuthenticatorCode,
clearTwoFactor,
newAuthenticatorSecret,
REQUIRED_ROLES,
requiresTwoFactor,
enforcementMethod,
+2
View File
@@ -51,6 +51,8 @@ export const user = obj({
role: z.enum(['admin', 'supervisor', 'staff', 'user']),
isActive: optBool,
notificationPreference: z.enum(['email', 'whatsapp', 'both']).nullish(),
twoFactorEnabled: optBool,
twoFactorMethod: z.enum(['email', 'whatsapp', 'authenticator']).nullish().describe('How their sign-in codes come (2.3.0: authenticator = an app)'),
homeCampusId: optStr,
createdAt: optDate,
});
+5 -1
View File
@@ -26,7 +26,7 @@ const toMe = (u: MeRow): z.input<typeof me> => ({
});
/* eslint-disable @typescript-eslint/no-explicit-any -- the JS helpers below aren't typed yet */
const { requiresTwoFactor } = require('../../utils/twoFactor') as { requiresTwoFactor(user: object): boolean };
const { requiresTwoFactor, recoveryCodesLeft } = require('../../utils/twoFactor') as { requiresTwoFactor(user: object): boolean; recoveryCodesLeft(userId: string): Promise<number> };
const privacyNotice = require('../../utils/privacyNotice') as { noticePromptPending(u: { noticeVersionAccepted: unknown }): Promise<false | 'first' | 'updated'> };
const campusScope = require('../../utils/campusScope') as { scopeFor(req: unknown): Promise<{ campusIds: string[] | null } | null> };
@@ -39,6 +39,8 @@ export const getMe = defineRoute({
access: 'user',
response: single(me.extend({
twoFactorRequired: z.boolean().describe('Their role must use two-step sign-in'),
twoFactorMethod: z.enum(['email', 'whatsapp', 'authenticator']).nullable().describe('How their codes come (2.3.0: authenticator = an app)'),
recoveryCodesLeft: z.number().int().describe('Unused recovery codes (0 with two-step sign-in off)'),
newsPromptPending: z.boolean(),
noticePromptPending: z.union([z.literal(false), z.enum(['first', 'updated'])]),
campusScope: z.array(id).nullable(),
@@ -54,6 +56,8 @@ export const getMe = defineRoute({
data: {
...toMe(u),
twoFactorRequired: requiresTwoFactor(u),
twoFactorMethod: u.twoFactorEnabled ? (u.twoFactorMethod ?? null) : null,
recoveryCodesLeft: u.twoFactorEnabled ? await recoveryCodesLeft(u.id) : 0,
newsPromptPending: !u.newsAskedAt && !impersonating,
noticePromptPending: impersonating ? false as const : await privacyNotice.noticePromptPending({ noticeVersionAccepted: u.noticeVersionAccepted }),
campusScope: (await campusScope.scopeFor(req))?.campusIds ?? null,
@@ -42,7 +42,7 @@ const SCOPE_LIST = 'scopeList (the handler filters by listWhere)';
// ─── Documented shapes (docs.ts) ─────────────────────────────────────────────
const challenge = D.obj({ challengeId: D.str, method: z.enum(['email', 'whatsapp']), destination: D.str.describe('Where the code went, partly hidden') });
const challenge = D.obj({ challengeId: D.str, method: z.enum(['email', 'whatsapp', 'authenticator']), destination: D.str.describe('Where the code went, partly hidden') });
const signedIn = D.obj({ message: D.optStr, id: D.str, name: D.str, email: D.str, role: D.str, token: D.str.describe('A new sign-in token') });
const kind = z.enum(['update', 'news']).describe('update: about something they booked (always sent). news: needs their news consent');
const attendeeFilter = D.obj({
@@ -90,8 +90,9 @@ v1Route({ method: 'post', path: '/me/sessions/revoke', tag: 'Me', summary: 'Sign
v1Route({ method: 'post', path: '/me/close', tag: 'Me', summary: 'Close my account', description: 'Needs the password. Personal details are removed; payment records stay, anonymised.', access: 'user', body: loose, documented: { body: D.obj({ password: D.str, deleteData: D.optBool.describe('Also ask for my personal information to be deleted') }), response: D.message }, v1: users.closeAccount });
v1Route({ method: 'get', path: '/me/activity', tag: 'Me', summary: 'Recent sign-ins and changes to my account', access: 'user', documented: { response: z.array(D.obj({ id: D.str, type: D.str.describe('login, password_changed, two_factor_enabled…'), device: D.optStr, createdAt: D.date })) }, v1: users.getMyActivity });
v1Route({ method: 'post', path: '/me/verify-password', tag: 'Me', summary: 'Check my password (before a sensitive change)', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ password: D.str }), response: D.ok }, v1: users.verifyPassword });
v1Route({ method: 'post', path: '/me/two-factor/enable', tag: 'Me', summary: 'Start turning on two-step sign-in', description: 'Sends a code to confirm with.', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ method: z.enum(['email', 'whatsapp']), password: D.str }), response: challenge }, v1: users.startEnableTwoFactor });
v1Route({ method: 'post', path: '/me/two-factor/confirm', tag: 'Me', summary: 'Confirm two-step sign-in with the code', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ challengeId: D.str, code: D.str }), response: D.obj({ twoFactorEnabled: z.literal(true), twoFactorMethod: D.str }) }, v1: users.confirmEnableTwoFactor });
v1Route({ method: 'post', path: '/me/two-factor/enable', tag: 'Me', summary: 'Start turning on two-step sign-in', description: 'Sends a code to confirm with, or for `authenticator` (2.3.0) answers with `secret` and `otpauthUrl` to show as a QR code; the app\x27s first code confirms it.', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ method: z.enum(['email', 'whatsapp', 'authenticator']), password: D.str }), response: challenge.extend({ secret: D.optStr.describe('authenticator: the key to type in by hand'), otpauthUrl: D.optStr.describe('authenticator: show as a QR code') }) }, v1: users.startEnableTwoFactor });
v1Route({ method: 'post', path: '/me/two-factor/confirm', tag: 'Me', summary: 'Confirm two-step sign-in with the code', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ challengeId: D.str, code: D.str }), response: D.obj({ twoFactorEnabled: z.literal(true), twoFactorMethod: D.str, recoveryCodes: z.array(D.str).describe('Shown once: single-use codes for when the app or phone is lost') }) }, v1: users.confirmEnableTwoFactor });
v1Route({ method: 'post', path: '/me/two-factor/recovery-codes', tag: 'Me', summary: 'A new set of recovery codes', description: 'Needs the password. The old codes stop working; the new ones are shown once.', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ password: D.str }), response: D.obj({ recoveryCodes: z.array(D.str) }) }, v1: users.replaceMyRecoveryCodes });
v1Route({ method: 'post', path: '/me/two-factor/disable', tag: 'Me', summary: 'Turn off two-step sign-in', access: 'user', middleware: [loginLimiter], body: loose, documented: { body: D.obj({ password: D.str }), response: D.obj({ twoFactorEnabled: z.literal(false), twoFactorMethod: z.null() }) }, v1: users.disableTwoFactor });
v1Route({
method: 'get', path: '/users/exists', tag: 'Accounts', summary: 'Is there an account with this email or phone?',
@@ -0,0 +1,56 @@
// The church's data when it leaves (crosscode.church-export/1): plain terms, export-only
// references, no internal ids and no secrets.
jest.mock('../src/config/db', () => ({}));
import { buildExport } from '../src/services/churchExport';
const d = (s: string) => new Date(s);
const input = {
version: '2.3.0',
users: [
{ id: 'u-uuid-1', name: 'Thandi', email: 't@x.org', phoneNumber: '+27821234567', role: 'admin', createdAt: d('2026-01-01T08:00:00Z'), newsEmail: true, newsWhatsApp: false, password: '$2b$hash', tokenVersion: 3 },
{ id: 'u-uuid-2', name: 'Sipho', email: 's@x.org', phoneNumber: null, role: 'user', createdAt: d('2026-02-01T08:00:00Z'), newsEmail: false, newsWhatsApp: true },
],
events: [{
id: 'ev-uuid-1', title: 'Camp', description: null, startDate: d('2026-12-04T16:00:00Z'), endDate: d('2026-12-06T12:00:00Z'), location: 'Camp X', isActive: true, isHidden: false,
sessions: [{ id: 's-1', label: 'Friday', startsAt: d('2026-12-04T16:00:00Z'), endsAt: d('2026-12-04T21:00:00Z') }],
eventOptions: [{ id: 'opt-uuid-1', name: 'Camper', price: 600 }],
}],
registrations: [{
id: 'reg-uuid-1', eventId: 'ev-uuid-1', userId: 'u-uuid-2', status: 'paid', createdAt: d('2026-11-01T09:00:00Z'),
attendees: [{ id: 'att-1', name: 'Sipho', email: 's@x.org', eventOptionId: 'opt-uuid-1', ticket: { usages: [{ scannedAt: d('2026-12-04T16:30:00Z') }] } }],
formResponses: [{ answers: [{ fieldId: 'f-1', value: 'None' }] }],
}],
payments: [{ id: 'pay-1', registrationId: 'reg-uuid-1', amount: 600, method: 'card', status: 'completed', createdAt: d('2026-11-01T09:05:00Z'), externalId: 'YOCO-1' }],
usages: [{ id: 'use-1', scannedAt: d('2026-12-04T16:30:00Z'), ticket: { eventId: 'ev-uuid-1', userId: 'u-uuid-2', holderUserId: null, attendee: { name: 'Sipho' } } }],
slots: [{ id: 'slot-1', roleId: 'role-1', userId: 'u-uuid-1' }],
roles: [{ id: 'role-1', eventId: 'ev-uuid-1', name: 'Kitchen', startsAt: null, endsAt: null }],
settings: [{ key: 'org_name', value: 'Hope Church' }, { key: 'smtp_password', value: 'enc:secret' }, { key: 'hub_install_secret', value: 'x' }],
fields: [{ id: 'f-1', label: 'Dietary needs' }],
};
describe('church export', () => {
const out = buildExport(input, new Date('2026-11-20T12:00:00Z'));
test('uses export-only references that link up', () => {
expect(out.people.map(p => p.ref)).toEqual(['p1', 'p2']);
expect(out.events[0]).toMatchObject({ ref: 'e1', ticketTypes: [{ ref: 't1', priceCents: 60000 }], published: true });
expect(out.registrations[0]).toMatchObject({ ref: 'r1', eventRef: 'e1', personRef: 'p2', attendees: [{ ticketTypeRef: 't1', checkedInAt: '2026-12-04T16:30:00.000Z' }], answers: [{ question: 'Dietary needs', answer: 'None' }] });
expect(out.payments[0]).toMatchObject({ registrationRef: 'r1', amountCents: 60000, reference: 'YOCO-1' });
expect(out.checkIns[0]).toMatchObject({ eventRef: 'e1', personRef: 'p2', name: 'Sipho' });
expect(out.rosters[0]).toMatchObject({ eventRef: 'e1', role: 'Kitchen', personRef: 'p1' });
expect(out.church.name).toBe('Hope Church');
});
test('has no internal ids and no secrets', () => {
const text = JSON.stringify(out);
for (const secret of ['uuid', '$2b$hash', 'enc:secret', 'hub_install_secret', 'smtp_password', 'tokenVersion', 'password']) expect(text).not.toContain(secret);
const keys: string[] = [];
const walk = (v: unknown): void => {
if (Array.isArray(v)) v.forEach(walk);
else if (v && typeof v === 'object') for (const [k, x] of Object.entries(v)) { keys.push(k); walk(x); }
};
walk(out);
expect(keys.filter(k => /^(id|.*Id|uuid|_id)$/.test(k))).toEqual([]);
});
});
@@ -0,0 +1,62 @@
// Installs made from the Hub register by themselves with HUB_LICENCE_KEY (AGENT-1.0-PLAN §7).
const mockSettings = new Map();
jest.mock('../src/config/db', () => ({
appSetting: {
findMany: jest.fn(async ({ where }) => [...mockSettings].filter(([key]) => where.key.in.includes(key)).map(([key, value]) => ({ key, value }))),
upsert: jest.fn(async ({ where, create }) => { mockSettings.set(where.key, create.value); }),
deleteMany: jest.fn(async ({ where }) => { mockSettings.delete(where.key); }),
},
}));
jest.mock('../src/hub/report', () => ({ APP_VERSION: '2.3.0', installInfo: async () => ({ name: 'Hope', url: 'https://events.hope.co.za', version: '2.3.0' }) }));
jest.mock('../src/hub/publicKey', () => ({ hubPublicKey: () => 'pem' }));
jest.mock('../src/hub/licence', () => ({ verifyToken: () => ({ installId: 'inst_1' }), clearLicenceCache: () => {} }));
const hub = require('../src/hub/client');
const reply = (status, body) => ({ ok: status < 400, status, json: async () => body });
const noSleep = async () => {};
describe('registering with HUB_LICENCE_KEY', () => {
let bodies;
beforeEach(() => {
mockSettings.clear();
bodies = [];
process.env.HUB_LICENCE_KEY = 'CEH-7K2M-9QXA-P4RT-WD8C';
process.env.HUB_ADMIN_NAME = 'Jane Admin';
process.env.HUB_ADMIN_EMAIL = 'jane@hope.co.za';
global.fetch = jest.fn(async (_url, opts) => {
bodies.push(JSON.parse(opts.body));
return reply(201, { installId: 'inst_1', installSecret: 's3cret', licence: 'hel1.x.y', heartbeatIntervalMinutes: 15 });
});
});
afterEach(() => {
delete global.fetch;
delete process.env.HUB_LICENCE_KEY;
delete process.env.HUB_ADMIN_NAME;
delete process.env.HUB_ADMIN_EMAIL;
});
test('registers once, with the admin contact, and stores the install', async () => {
expect(await hub.connectFromEnv({ sleep: noSleep })).toEqual({ connected: true });
expect(bodies[0]).toMatchObject({ licenceKey: 'CEH-7K2M-9QXA-P4RT-WD8C', install: { adminContact: { name: 'Jane Admin', email: 'jane@hope.co.za' } } });
expect(mockSettings.get('hub_install_id')).toBe('inst_1');
expect(await hub.connectFromEnv({ sleep: noSleep })).toEqual({ connected: false, reason: 'already' });
expect(global.fetch).toHaveBeenCalledTimes(1);
});
test('does nothing without the key', async () => {
delete process.env.HUB_LICENCE_KEY;
expect(await hub.connectFromEnv({ sleep: noSleep })).toEqual({ connected: false, reason: 'no_key' });
expect(global.fetch).not.toHaveBeenCalled();
});
test('a refused key isn\'t tried again; an unreachable Hub is', async () => {
global.fetch = jest.fn(async () => reply(404, { code: 'LICENCE_NOT_FOUND', message: 'Unknown key' }));
expect(await hub.connectFromEnv({ sleep: noSleep })).toEqual({ connected: false, reason: 'LICENCE_NOT_FOUND' });
expect(global.fetch).toHaveBeenCalledTimes(1);
global.fetch = jest.fn(async () => { throw new Error('ECONNREFUSED'); });
expect(await hub.connectFromEnv({ attempts: 3, sleep: noSleep })).toEqual({ connected: false, reason: 'unreachable' });
expect(global.fetch).toHaveBeenCalledTimes(3);
});
});
@@ -0,0 +1,163 @@
// Two-factor with an authenticator app and recovery codes (2.3.0): setting it up, signing in with
// the app's code (once only), recovery codes (single use), and turning it off.
process.env.JWT_SECRET = 'test-secret';
process.env.NODE_ENV = 'test';
function matches(row, where = {}) {
return Object.entries(where).every(([key, cond]) => {
if (key === 'OR') return cond.some(c => matches(row, c));
const value = row[key];
if (cond === null) return value == null;
if (cond && typeof cond === 'object' && !(cond instanceof Date)) {
if ('gt' in cond && !(value > cond.gt)) return false;
if ('lt' in cond && !(value != null && value < cond.lt)) return false;
return true;
}
return value === cond;
});
}
function model(defaults = () => ({})) {
let rows = [];
let seq = 0;
const apply = (row, data) => { for (const [k, v] of Object.entries(data)) row[k] = v && typeof v === 'object' && 'increment' in v ? row[k] + v.increment : v; };
return {
rows: () => rows,
reset: () => { rows = []; },
create: async ({ data }) => { const r = { id: `id-${++seq}`, ...defaults(), ...data }; rows.push(r); return { ...r }; },
createMany: async ({ data }) => { for (const d of data) rows.push({ id: `id-${++seq}`, ...defaults(), ...d }); return { count: data.length }; },
findUnique: async ({ where, include, select }) => {
const r = rows.find(x => matches(x, where));
if (!r) return null;
const out = { ...r };
if (include?.user) out.user = { ...mockDb.user.rows().find(u => u.id === r.userId) };
if (select) return Object.fromEntries(Object.keys(select).map(k => [k, r[k]]));
return out;
},
update: async ({ where, data }) => { const r = rows.find(x => matches(x, where)); apply(r, data); return { ...r }; },
updateMany: async ({ where, data }) => { const hit = rows.filter(x => matches(x, where)); hit.forEach(r => apply(r, data)); return { count: hit.length }; },
deleteMany: async ({ where }) => { const before = rows.length; rows = rows.filter(x => !matches(x, where)); return { count: before - rows.length }; },
count: async ({ where }) => rows.filter(x => matches(x, where)).length,
};
}
const mockDb = {
user: model(),
twoFactorChallenge: model(() => ({ attempts: 0, usedAt: null, lastSentAt: new Date(), createdAt: new Date(), pendingSecret: null })),
recoveryCode: model(() => ({ usedAt: null, createdAt: new Date() })),
trustedDevice: model(),
};
jest.mock('../src/config/db', () => mockDb);
jest.mock('../src/utils/settingsCache', () => ({ getSettingSync: (_key, fallback) => fallback }));
const twoFactor = require('../src/utils/twoFactor');
const totp = require('../src/utils/totp');
const { decrypt } = require('../src/utils/encryption');
const STEP_MS = totp.STEP * 1000;
const codeNow = (secret, offsetSteps = 0) => totp.codeAt(secret, Math.floor(Date.now() / STEP_MS) + offsetSteps);
beforeEach(() => {
for (const m of Object.values(mockDb)) m.reset();
mockDb.user.rows().push({ id: 'u1', name: 'Jane', email: 'jane@example.com', role: 'admin', twoFactorEnabled: false, totpSecret: null, totpLastStep: null });
});
async function setUpApp() {
const user = mockDb.user.rows()[0];
const ch = await twoFactor.createChallenge(user, 'enable', 'authenticator');
return { ch, user };
}
describe('setting up an authenticator app', () => {
test('gives a secret and an otpauth link, keeps it only encrypted, and sends nothing', async () => {
const { ch } = await setUpApp();
expect(ch).toMatchObject({ method: 'authenticator', destination: 'your authenticator app' });
expect(ch.secret).toMatch(/^[A-Z2-7]{32}$/);
expect(ch.otpauthUrl).toMatch(/^otpauth:\/\/totp\/.+secret=[A-Z2-7]{32}&issuer=/);
const row = mockDb.twoFactorChallenge.rows()[0];
expect(row.pendingSecret).not.toContain(ch.secret);
expect(decrypt(row.pendingSecret)).toBe(ch.secret);
});
test('the app\'s current code confirms it; a wrong code doesn\'t', async () => {
const { ch } = await setUpApp();
await expect(twoFactor.verifyChallenge(ch.challengeId, '000000', 'enable', 'u1')).rejects.toThrow(/Incorrect code/);
const ok = await twoFactor.verifyChallenge(ch.challengeId, codeNow(ch.secret), 'enable', 'u1');
expect(ok.method).toBe('authenticator');
expect(decrypt(ok.pendingSecret)).toBe(ch.secret);
});
test('resend is refused: nothing is sent for an app', async () => {
const { ch } = await setUpApp();
await expect(twoFactor.resendChallenge(ch.challengeId, 'enable', 'u1')).rejects.toThrow(/authenticator app/);
});
test('it\'s always among the available methods', async () => {
expect(await twoFactor.availableMethods({ email: 'x@guest.local', phoneNumber: null })).toEqual(['authenticator']);
});
});
describe('signing in with the app', () => {
let secret;
beforeEach(async () => {
const { ch } = await setUpApp();
secret = ch.secret;
Object.assign(mockDb.user.rows()[0], { twoFactorEnabled: true, twoFactorMethod: 'authenticator', totpSecret: mockDb.twoFactorChallenge.rows()[0].pendingSecret });
});
test('the current code works once; the same code again is refused', async () => {
const user = mockDb.user.rows()[0];
const a = await twoFactor.createChallenge(user, 'login', 'authenticator');
expect(a.secret).toBeUndefined();
const code = codeNow(secret);
await expect(twoFactor.verifyChallenge(a.challengeId, code, 'login')).resolves.toMatchObject({ usedRecoveryCode: false });
const b = await twoFactor.createChallenge(user, 'login', 'authenticator');
await expect(twoFactor.verifyChallenge(b.challengeId, code, 'login')).rejects.toThrow(/Incorrect code/);
});
test('a code from the step before still works (clock drift); one from two steps back doesn\'t', async () => {
const user = mockDb.user.rows()[0];
const a = await twoFactor.createChallenge(user, 'login', 'authenticator');
await expect(twoFactor.verifyChallenge(a.challengeId, codeNow(secret, -2), 'login')).rejects.toThrow(/Incorrect/);
await expect(twoFactor.verifyChallenge(a.challengeId, codeNow(secret, -1), 'login')).resolves.toBeTruthy();
});
test('no app set up: signing in with it is refused', async () => {
mockDb.user.rows()[0].totpSecret = null;
await expect(twoFactor.createChallenge(mockDb.user.rows()[0], 'login', 'authenticator')).rejects.toThrow(/No authenticator app/);
});
});
describe('recovery codes', () => {
test('ten codes, single use, accepted at sign-in in place of the app\'s code', async () => {
const { ch } = await setUpApp();
Object.assign(mockDb.user.rows()[0], { twoFactorEnabled: true, twoFactorMethod: 'authenticator', totpSecret: mockDb.twoFactorChallenge.rows()[0].pendingSecret });
const codes = await twoFactor.replaceRecoveryCodes('u1', mockDb);
expect(codes).toHaveLength(10);
expect(new Set(codes).size).toBe(10);
codes.forEach(c => expect(c).toMatch(/^[a-z2-9]{5}-[a-z2-9]{5}$/));
expect(JSON.stringify(mockDb.recoveryCode.rows())).not.toContain(codes[0]);
const user = mockDb.user.rows()[0];
const a = await twoFactor.createChallenge(user, 'login', 'authenticator');
await expect(twoFactor.verifyChallenge(a.challengeId, codes[3].toUpperCase().replace('-', ' '), 'login')).resolves.toMatchObject({ usedRecoveryCode: true });
expect(await twoFactor.recoveryCodesLeft('u1')).toBe(9);
const b = await twoFactor.createChallenge(user, 'login', 'authenticator');
await expect(twoFactor.verifyChallenge(b.challengeId, codes[3], 'login')).rejects.toThrow(/Incorrect/);
expect(ch).toBeTruthy();
});
test('a new set replaces the old one', async () => {
const first = await twoFactor.replaceRecoveryCodes('u1', mockDb);
await twoFactor.replaceRecoveryCodes('u1', mockDb);
expect(await twoFactor.useRecoveryCode('u1', first[0])).toBe(false);
expect(await twoFactor.recoveryCodesLeft('u1')).toBe(10);
});
test('turning 2FA off removes them and the app\'s secret', async () => {
await twoFactor.replaceRecoveryCodes('u1', mockDb);
const data = await twoFactor.clearTwoFactor('u1', mockDb);
expect(data).toEqual({ twoFactorEnabled: false, twoFactorMethod: null, totpSecret: null, totpLastStep: null });
expect(await twoFactor.recoveryCodesLeft('u1')).toBe(0);
});
});
@@ -117,15 +117,15 @@ afterEach(() => jest.restoreAllMocks());
describe('availableMethods', () => {
test('offers WhatsApp only with a valid phone and a configured gateway', async () => {
expect(await availableMethods(USER)).toEqual(['email', 'whatsapp']);
expect(await availableMethods(USER)).toEqual(['email', 'whatsapp', 'authenticator']);
mockWhatsApp.configured = false;
expect(await availableMethods(USER)).toEqual(['email']);
expect(await availableMethods(USER)).toEqual(['email', 'authenticator']);
mockWhatsApp.configured = true;
expect(await availableMethods({ ...USER, phoneNumber: null })).toEqual(['email']);
expect(await availableMethods({ ...USER, phoneNumber: null })).toEqual(['email', 'authenticator']);
});
test('offers nothing by email for a guest placeholder address', async () => {
expect(await availableMethods({ ...USER, email: 'x@guest.local' })).toEqual(['whatsapp']);
expect(await availableMethods({ ...USER, email: 'x@guest.local' })).toEqual(['whatsapp', 'authenticator']);
});
});
+114
View File
@@ -0,0 +1,114 @@
{
"schema": "crosscode.env-template/1",
"product": "events",
"files": {
"backend/.env": [
{
"name": "NODE_ENV",
"layer": "generated",
"required": true,
"description": "production"
},
{
"name": "PORT",
"layer": "install",
"from": "{apiPort}",
"required": true,
"description": "The backend's port (from the Hub's port list)"
},
{
"name": "DATABASE_URL",
"layer": "generated",
"required": true,
"description": "Made on the server with a new password; never leaves it"
},
{
"name": "JWT_SECRET",
"layer": "generated",
"required": true,
"description": "Also encrypts stored secrets: never change it"
},
{
"name": "FRONTEND_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
},
{
"name": "APP_BASE_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
},
{
"name": "BACKEND_URL",
"layer": "install",
"from": "https://{domain}",
"required": true,
"description": "The same address: the backend is under /api (2.2.0+)"
},
{
"name": "TRUST_PROXY",
"layer": "server",
"required": true,
"description": "Behind frp: loopback"
},
{
"name": "APP_TIMEZONE",
"layer": "install",
"default": "Africa/Johannesburg"
},
{
"name": "HUB_URL",
"layer": "defaults",
"default": "https://hub.crosscode.co.za"
},
{
"name": "HUB_HEARTBEAT_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "HUB_PUBLIC_KEY",
"layer": "defaults",
"description": "Only for a Hub other than hub.crosscode.co.za (staging, a test VM): its licence public key"
},
{
"name": "HUB_LICENCE_KEY",
"layer": "install",
"description": "With it the site registers with the Hub by itself when it first starts"
},
{
"name": "HUB_ADMIN_NAME",
"layer": "install",
"description": "The site admin, sent to the Hub on registering"
},
{
"name": "HUB_ADMIN_EMAIL",
"layer": "install"
},
{
"name": "BACKUP_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "DAILY_SUMMARY_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "SENTRY_DSN",
"layer": "defaults"
}
],
"frontend/.env.production": [
{
"name": "NEXT_PUBLIC_API_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
}
]
}
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@crosscode/events-frontend",
"version": "2.2.0",
"version": "2.3.0",
"private": true,
"scripts": {
"dev": "next dev",
@@ -23,6 +23,7 @@ interface UserItem {
phoneNumber?: string | null;
notificationPreference?: string;
twoFactorEnabled?: boolean;
twoFactorMethod?: "email" | "whatsapp" | "authenticator" | null;
isActive: boolean;
createdAt: string;
updatedAt: string;
@@ -34,6 +35,8 @@ type Role = typeof roleOptions[number];
const notificationPreferenceOptions = ["email", "whatsapp", "both"] as const;
const TWO_FACTOR_LABEL: Record<"email" | "whatsapp" | "authenticator", string> = { email: "email", whatsapp: "WhatsApp", authenticator: "app" };
export default function AdminUsersPage() {
const { user, loading, token } = useAuth();
const router = useRouter();
@@ -418,6 +421,9 @@ export default function AdminUsersPage() {
</td>
<td className="p-2">
<RoleBadge role={u.role as RoleBadgeRole} />
<div className="mt-1 text-xs text-gray-500" title="How their sign-in codes come">
{u.twoFactorEnabled ? `2FA: ${TWO_FACTOR_LABEL[u.twoFactorMethod || "email"]}` : "No 2FA"}
</div>
</td>
<td className="p-2">
<span>{u.phoneNumber || ""}</span>
@@ -14,17 +14,19 @@ import { useCampuses } from "@/hooks/useCampuses";
import { apiV2, apiV2Data } from "@/lib/apiV2";
import { useStateSync } from "@/lib/hooks";
type ActivityEvent = { id: string; type: "login" | "password_changed" | "password_reset" | "two_factor_enabled" | "two_factor_disabled"; device: string | null; createdAt: string };
type ActivityEvent = { id: string; type: "login" | "password_changed" | "password_reset" | "two_factor_enabled" | "two_factor_disabled" | "recovery_code_used" | "recovery_codes_replaced"; device: string | null; createdAt: string };
const ACTIVITY_META: Record<ActivityEvent["type"], { label: (device: string | null) => string; icon: typeof LogIn }> = {
login: { label: d => `Logged in${d ? ` from ${d}` : ""}`, icon: LogIn },
password_changed: { label: () => "Password changed", icon: KeyRound },
password_reset: { label: () => "Password reset via email link", icon: RotateCcw },
two_factor_enabled: { label: () => "Two-factor authentication turned on", icon: ShieldCheck },
recovery_code_used: { label: () => "Logged in with a recovery code", icon: ShieldAlert },
recovery_codes_replaced: { label: () => "New recovery codes made", icon: ShieldCheck },
two_factor_disabled: { label: () => "Two-factor authentication turned off", icon: ShieldOff },
};
const TWO_FACTOR_METHOD_LABEL: Record<TwoFactorMethod, string> = { email: "email", whatsapp: "WhatsApp" };
const TWO_FACTOR_METHOD_LABEL: Record<TwoFactorMethod, string> = { email: "email", whatsapp: "WhatsApp", authenticator: "an authenticator app" };
// Shared bits of the page (outside the component, so they aren't new components each render).
const Alert = ({ msg }: { msg: { type: "ok" | "err"; text: string } }) => (
@@ -185,12 +187,16 @@ export default function UserProfilePage() {
// loaded once per page load, so it wouldn't reflect a change made here.
const [tfEnabled, setTfEnabled] = useState(false);
const [tfMethod, setTfMethod] = useState<TwoFactorMethod>("email");
const [tfStep, setTfStep] = useState<"idle" | "code" | "disable" | "change">("idle");
const [tfStep, setTfStep] = useState<"idle" | "code" | "disable" | "change" | "recovery">("idle");
const [tfPassword, setTfPassword] = useState("");
const [tfCode, setTfCode] = useState("");
const [tfChallenge, setTfChallenge] = useState<TwoFactorChallenge | null>(null);
const [tfMsg, setTfMsg] = useDismissingState<{ type: "ok" | "err"; text: string } | null>(null);
const [tfBusy, setTfBusy] = useState(false);
// 2.3.0: recovery codes just made (shown once), how many are left, and the setup QR code.
const [tfRecovery, setTfRecovery] = useState<string[] | null>(null);
const [tfRecoveryLeft, setTfRecoveryLeft] = useState(0);
const [tfQr, setTfQr] = useState<{ url: string; image: string } | null>(null);
// WhatsApp codes need the number that's actually saved, not one still being typed.
const canWhatsAppCodes = isValidZAPhone(user?.phoneNumber || "");
@@ -201,9 +207,19 @@ export default function UserProfilePage() {
if (user) {
setTfEnabled(!!user.twoFactorEnabled);
setTfMethod(user.twoFactorMethod || "email");
setTfRecoveryLeft((user as { recoveryCodesLeft?: number }).recoveryCodesLeft ?? 0);
}
}, [user]);
// The setup QR code, drawn here from the otpauth:// link (the key never leaves the browser).
useEffect(() => {
const url = tfChallenge?.otpauthUrl;
if (!url) return;
let alive = true;
import("qrcode").then(q => q.toDataURL(url, { margin: 1, width: 200 })).then(image => { if (alive) setTfQr({ url, image }); }).catch(() => {});
return () => { alive = false; };
}, [tfChallenge]);
const resetTwoFactorForm = () => {
setTfStep("idle");
setTfPassword("");
@@ -241,12 +257,14 @@ export default function UserProfilePage() {
const res = await (apiV2Data("post", "/me/two-factor/confirm", {
authToken: token,
body: { challengeId: tfChallenge.challengeId, code: tfCode },
}) as Promise<{ twoFactorMethod: TwoFactorMethod }>);
}) as Promise<{ twoFactorMethod: TwoFactorMethod; recoveryCodes?: string[] }>);
const wasEnabled = tfEnabled;
setTfEnabled(true);
setTfMethod(res.twoFactorMethod);
resetTwoFactorForm();
setTfMsg({ type: "ok", text: wasEnabled ? `Codes will now be sent by ${TWO_FACTOR_METHOD_LABEL[res.twoFactorMethod]}.` : "Two-factor authentication is on." });
setTfRecovery(res.recoveryCodes ?? null);
setTfRecoveryLeft(res.recoveryCodes?.length ?? 0);
setTfMsg({ type: "ok", text: wasEnabled ? `Your codes now come from ${TWO_FACTOR_METHOD_LABEL[res.twoFactorMethod]}.` : "Two-factor authentication is on." });
loadActivity();
} catch (e) {
setTfMsg({ type: "err", text: errorMessage(e) || "Failed to verify the code." });
@@ -255,6 +273,35 @@ export default function UserProfilePage() {
}
};
const newRecoveryCodes = async (e: React.FormEvent) => {
e.preventDefault();
if (!token) return;
setTfBusy(true);
setTfMsg(null);
try {
const res = await (apiV2Data("post", "/me/two-factor/recovery-codes", { authToken: token, body: { password: tfPassword } }) as Promise<{ recoveryCodes: string[] }>);
resetTwoFactorForm();
setTfRecovery(res.recoveryCodes);
setTfRecoveryLeft(res.recoveryCodes.length);
loadActivity();
} catch (e) {
setTfMsg({ type: "err", text: errorMessage(e) || "Couldn't make new recovery codes." });
} finally {
setTfBusy(false);
}
};
const downloadRecovery = () => {
if (!tfRecovery) return;
const text = `Recovery codes for ${user?.email || "your account"} (${window.location.host})\nEach works once, in place of a code, if you lose your phone or app.\n\n${tfRecovery.join("\n")}\n`;
const url = URL.createObjectURL(new Blob([text], { type: "text/plain" }));
const link = document.createElement("a");
link.href = url;
link.download = "recovery-codes.txt";
link.click();
URL.revokeObjectURL(url);
};
const disableTwoFactor = async (e: React.FormEvent) => {
e.preventDefault();
if (!token) return;
@@ -266,6 +313,7 @@ export default function UserProfilePage() {
body: { password: tfPassword },
});
setTfEnabled(false);
setTfRecoveryLeft(0);
resetTwoFactorForm();
setTfMsg({ type: "ok", text: "Two-factor authentication is off." });
loadActivity();
@@ -424,9 +472,20 @@ export default function UserProfilePage() {
{tfStep === "code" && tfChallenge ? (
<form onSubmit={confirmEnableTwoFactor} className="space-y-4">
<p className="text-sm text-gray-500">
We sent a 6-digit code to {tfChallenge.destination} by {TWO_FACTOR_METHOD_LABEL[tfChallenge.method]}. {tfEnabled ? "Enter it to switch your codes to this channel." : "Enter it to finish turning on two-factor authentication."}
</p>
{tfChallenge.method === "authenticator" ? (
<div className="space-y-3 text-sm text-gray-600">
<p>Scan this with an authenticator app (Google Authenticator, Microsoft Authenticator, 1Password, Authy…), then enter the 6-digit code it shows.</p>
{/* eslint-disable-next-line @next/next/no-img-element -- a data: URL drawn in the browser */}
{tfQr && tfQr.url === tfChallenge.otpauthUrl && <img src={tfQr.image} alt="QR code to add this account to an authenticator app" width={200} height={200} className="rounded border" />}
{tfChallenge.secret && (
<p className="text-xs">Can&apos;t scan it? Type this key into the app: <code className="select-all break-all rounded bg-gray-100 px-1 py-0.5 font-mono">{tfChallenge.secret.replace(/(.{4})/g, "$1 ").trim()}</code></p>
)}
</div>
) : (
<p className="text-sm text-gray-500">
We sent a 6-digit code to {tfChallenge.destination} by {TWO_FACTOR_METHOD_LABEL[tfChallenge.method]}. {tfEnabled ? "Enter it to switch your codes to this channel." : "Enter it to finish turning on two-factor authentication."}
</p>
)}
<div>
<label htmlFor="profile-verification-code" className="block text-sm font-medium text-gray-700 mb-1">Verification code</label>
<input
@@ -446,16 +505,32 @@ export default function UserProfilePage() {
Cancel
</button>
<button type="submit" disabled={tfBusy || tfCode.length !== 6} className="px-4 py-2 rounded-lg bg-brand-600 text-white text-sm font-medium hover:bg-brand-700 disabled:opacity-50">
{tfBusy ? "Verifying…" : "Turn on two-factor"}
{tfBusy ? "Verifying…" : tfEnabled ? "Switch" : "Turn on two-factor"}
</button>
</div>
</form>
) : tfEnabled && tfStep !== "change" ? (
<>
<p className="text-sm text-gray-500 mb-4">
<span className="font-medium text-green-700">On.</span> When you log in we send a 6-digit code by{" "}
{TWO_FACTOR_METHOD_LABEL[tfMethod]}. Devices you&apos;ve chosen to trust skip the code for 30 days.
<span className="font-medium text-green-700">On.</span>{" "}
{tfMethod === "authenticator" ? "When you log in, enter the 6-digit code from your authenticator app." : <>When you log in we send a 6-digit code by {TWO_FACTOR_METHOD_LABEL[tfMethod]}.</>}{" "}
Devices you&apos;ve chosen to trust skip the code for 30 days.
</p>
<div className="mb-4 rounded-lg border border-gray-200 bg-gray-50 p-3 text-sm">
<p className="text-gray-700"><span className="font-medium">Recovery codes:</span> {tfRecoveryLeft} of 10 left. Each works once in place of a code if you lose your phone or app.</p>
{tfStep === "recovery" ? (
<form onSubmit={newRecoveryCodes} className="mt-3 flex flex-wrap items-end gap-2">
<div className="flex-1 min-w-[12rem]">
<label htmlFor="profile-recovery-password" className="block text-xs font-medium text-gray-700 mb-1">Confirm with your password</label>
<input id="profile-recovery-password" type="password" className={inputCls} value={tfPassword} onChange={e => setTfPassword(e.target.value)} required autoComplete="current-password" />
</div>
<button type="button" onClick={resetTwoFactorForm} className="px-3 py-2 rounded-lg bg-gray-100 text-sm font-medium hover:bg-gray-200">Cancel</button>
<button type="submit" disabled={tfBusy || !tfPassword} className="px-3 py-2 rounded-lg bg-brand-600 text-white text-sm font-medium hover:bg-brand-700 disabled:opacity-50">{tfBusy ? "Making…" : "Make new codes"}</button>
</form>
) : (
<button onClick={() => { setTfStep("recovery"); setTfPassword(""); setTfMsg(null); }} className="mt-2 text-sm text-brand-700 hover:underline">New recovery codes…</button>
)}
</div>
{tfRequired ? (
<>
<p className="text-xs text-gray-500 mb-3">Required for your role, so it can&apos;t be turned off.</p>
@@ -495,11 +570,12 @@ export default function UserProfilePage() {
<p className="text-sm text-gray-500">
{tfEnabled
? "Choose where your login codes go. We'll send a code there first to check it reaches you."
: "Add a second step to logging in: after your password, we send a 6-digit code that you need to enter."}
: "Add a second step to logging in: after your password, a 6-digit code from an authenticator app, or one we send you."}
</p>
<div>
<label htmlFor="profile-send-my-codes-by" className="block text-sm font-medium text-gray-700 mb-1">Send my codes by</label>
<select id="profile-send-my-codes-by" className={inputCls} value={tfMethod} onChange={e => setTfMethod(e.target.value as TwoFactorMethod)}>
<option value="authenticator">An authenticator app (most secure)</option>
<option value="email">Email</option>
{canWhatsAppCodes && <option value="whatsapp">WhatsApp</option>}
</select>
@@ -518,11 +594,23 @@ export default function UserProfilePage() {
</button>
)}
<button type="submit" disabled={tfBusy || !tfPassword} className="px-4 py-2 rounded-lg bg-brand-600 text-white text-sm font-medium hover:bg-brand-700 disabled:opacity-50">
{tfBusy ? "Sending code…" : "Send verification code"}
{tfBusy ? (tfMethod === "authenticator" ? "Preparing…" : "Sending code…") : tfMethod === "authenticator" ? "Set up the app" : "Send verification code"}
</button>
</div>
</form>
)}
{tfRecovery && (
<div className="mt-4 rounded-lg border border-amber-300 bg-amber-50 p-4 text-sm" role="region" aria-label="Your recovery codes">
<p className="font-medium text-amber-900">Save your recovery codes now</p>
<p className="mt-1 text-amber-900">Each works once, in place of a code, if you lose your phone or app. They won&apos;t be shown again.</p>
<ul className="mt-3 grid grid-cols-2 gap-x-6 gap-y-1 font-mono text-sm sm:grid-cols-3">{tfRecovery.map(c => <li key={c}>{c}</li>)}</ul>
<div className="mt-3 flex flex-wrap gap-2">
<button type="button" onClick={downloadRecovery} className="px-3 py-2 rounded-lg bg-white border border-amber-300 text-sm font-medium hover:bg-amber-100">Download</button>
<button type="button" onClick={() => navigator.clipboard?.writeText(tfRecovery.join("\n"))} className="px-3 py-2 rounded-lg bg-white border border-amber-300 text-sm font-medium hover:bg-amber-100">Copy</button>
<button type="button" onClick={() => setTfRecovery(null)} className="px-3 py-2 rounded-lg bg-amber-600 text-white text-sm font-medium hover:bg-amber-700">I&apos;ve saved them</button>
</div>
</div>
)}
{tfMsg && <Alert msg={tfMsg} />}
</section>
@@ -27,6 +27,8 @@ export function LoginForm() {
// Two-factor step — set once the password is accepted for an account with 2FA on.
const [challenge, setChallenge] = useState<TwoFactorChallenge | null>(null);
const [code, setCode] = useState("");
// 2.3.0: a recovery code instead of the 6-digit code (xxxxx-xxxxx, single use).
const [useRecovery, setUseRecovery] = useState(false);
const [rememberDevice, setRememberDevice] = useState(false);
const [resendIn, setResendIn] = useState(0);
@@ -109,6 +111,7 @@ export function LoginForm() {
const backToPassword = () => {
setChallenge(null);
setCode("");
setUseRecovery(false);
setPassword("");
setError(null);
};
@@ -124,20 +127,24 @@ export function LoginForm() {
</p>
)}
<p className="text-sm text-gray-600 mt-1">
We sent a 6-digit code to {challenge.destination} by {challenge.method === "whatsapp" ? "WhatsApp" : "email"}. It expires in 10 minutes.
{useRecovery
? "Enter one of the recovery codes you saved when you turned on two-factor. Each works once."
: challenge.method === "authenticator"
? "Open your authenticator app and enter the 6-digit code it shows for this site."
: <>We sent a 6-digit code to {challenge.destination} by {challenge.method === "whatsapp" ? "WhatsApp" : "email"}. It expires in 10 minutes.</>}
</p>
</div>
<div>
<label htmlFor="login-code" className="block text-sm font-medium mb-1">Verification code</label>
<label htmlFor="login-code" className="block text-sm font-medium mb-1">{useRecovery ? "Recovery code" : "Verification code"}</label>
<input
id="login-code"
type="text"
inputMode="numeric"
autoComplete="one-time-code"
maxLength={6}
inputMode={useRecovery ? "text" : "numeric"}
autoComplete={useRecovery ? "off" : "one-time-code"}
maxLength={useRecovery ? 11 : 6}
value={code}
onChange={(e) => setCode(e.target.value.replace(/\D/g, ""))}
onChange={(e) => setCode(useRecovery ? e.target.value.toLowerCase().replace(/[^a-z0-9-]/g, "") : e.target.value.replace(/\D/g, ""))}
className="w-full border rounded px-3 py-2 text-lg tracking-widest"
autoFocus
required
@@ -158,7 +165,7 @@ export function LoginForm() {
<button
type="submit"
disabled={loading || code.length !== 6}
disabled={loading || (useRecovery ? code.replace(/-/g, "").length !== 10 : code.length !== 6)}
className="w-full bg-brand-600 hover:bg-brand-700 text-white rounded py-2 disabled:opacity-60"
>
{loading ? "Verifying..." : "Verify and log in"}
@@ -168,14 +175,21 @@ export function LoginForm() {
<button type="button" onClick={backToPassword} className="text-brand-600 hover:underline">
Back to login
</button>
<button
type="button"
onClick={onResend}
disabled={resendIn > 0}
className="text-brand-600 hover:underline disabled:text-gray-400 disabled:no-underline"
>
{resendIn > 0 ? `Resend code in ${resendIn}s` : "Resend code"}
</button>
{!challenge.enrolling && (
<button type="button" onClick={() => { setUseRecovery(!useRecovery); setCode(""); setError(null); }} className="text-brand-600 hover:underline">
{useRecovery ? "Use a code instead" : "Use a recovery code"}
</button>
)}
{challenge.method !== "authenticator" && !useRecovery && (
<button
type="button"
onClick={onResend}
disabled={resendIn > 0}
className="text-brand-600 hover:underline disabled:text-gray-400 disabled:no-underline"
>
{resendIn > 0 ? `Resend code in ${resendIn}s` : "Resend code"}
</button>
)}
</div>
</form>
);
@@ -12950,6 +12950,28 @@
"type": "boolean",
"description": "Their role must use two-step sign-in"
},
"twoFactorMethod": {
"anyOf": [
{
"type": "string",
"enum": [
"email",
"whatsapp",
"authenticator"
]
},
{
"type": "null"
}
],
"description": "How their codes come (2.3.0: authenticator = an app)"
},
"recoveryCodesLeft": {
"type": "integer",
"minimum": -9007199254740991,
"maximum": 9007199254740991,
"description": "Unused recovery codes (0 with two-step sign-in off)"
},
"newsPromptPending": {
"type": "boolean"
},
@@ -12995,6 +13017,8 @@
"homeCampusId",
"createdAt",
"twoFactorRequired",
"twoFactorMethod",
"recoveryCodesLeft",
"newsPromptPending",
"noticePromptPending",
"campusScope"
@@ -47764,7 +47788,8 @@
"type": "string",
"enum": [
"email",
"whatsapp"
"whatsapp",
"authenticator"
]
},
"destination": {
@@ -48395,7 +48420,7 @@
"Me"
],
"summary": "Start turning on two-step sign-in",
"description": "Sends a code to confirm with.\n\n**Who:** Any signed-in person.",
"description": "Sends a code to confirm with, or for `authenticator` (2.3.0) answers with `secret` and `otpauthUrl` to show as a QR code; the app's first code confirms it.\n\n**Who:** Any signed-in person.",
"operationId": "postMeTwo-factorEnable",
"security": [
{
@@ -48414,7 +48439,8 @@
"type": "string",
"enum": [
"email",
"whatsapp"
"whatsapp",
"authenticator"
]
},
"password": {
@@ -48448,12 +48474,27 @@
"type": "string",
"enum": [
"email",
"whatsapp"
"whatsapp",
"authenticator"
]
},
"destination": {
"type": "string",
"description": "Where the code went, partly hidden"
},
"secret": {
"description": "authenticator: the key to type in by hand",
"type": [
"string",
"null"
]
},
"otpauthUrl": {
"description": "authenticator: show as a QR code",
"type": [
"string",
"null"
]
}
},
"required": [
@@ -48549,11 +48590,107 @@
},
"twoFactorMethod": {
"type": "string"
},
"recoveryCodes": {
"type": "array",
"items": {
"type": "string"
},
"description": "Shown once: single-use codes for when the app or phone is lost"
}
},
"required": [
"twoFactorEnabled",
"twoFactorMethod"
"twoFactorMethod",
"recoveryCodes"
],
"additionalProperties": {}
}
},
"required": [
"data"
],
"additionalProperties": false
}
}
}
},
"400": {
"description": "Validation failed (VALIDATION_FAILED)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
},
"401": {
"description": "Not signed in (UNAUTHENTICATED)",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/Error"
}
}
}
}
}
}
},
"/me/two-factor/recovery-codes": {
"post": {
"tags": [
"Me"
],
"summary": "A new set of recovery codes",
"description": "Needs the password. The old codes stop working; the new ones are shown once.\n\n**Who:** Any signed-in person.",
"operationId": "postMeTwo-factorRecovery-codes",
"security": [
{
"bearerAuth": []
}
],
"parameters": [],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"password": {
"type": "string"
}
},
"required": [
"password"
],
"additionalProperties": {}
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"recoveryCodes": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"recoveryCodes"
],
"additionalProperties": {}
}
@@ -48975,6 +49112,28 @@
}
]
},
"twoFactorEnabled": {
"type": [
"boolean",
"null"
]
},
"twoFactorMethod": {
"description": "How their sign-in codes come (2.3.0: authenticator = an app)",
"anyOf": [
{
"type": "string",
"enum": [
"email",
"whatsapp",
"authenticator"
]
},
{
"type": "null"
}
]
},
"homeCampusId": {
"type": [
"string",
@@ -49148,6 +49307,28 @@
}
]
},
"twoFactorEnabled": {
"type": [
"boolean",
"null"
]
},
"twoFactorMethod": {
"description": "How their sign-in codes come (2.3.0: authenticator = an app)",
"anyOf": [
{
"type": "string",
"enum": [
"email",
"whatsapp",
"authenticator"
]
},
{
"type": "null"
}
]
},
"homeCampusId": {
"type": [
"string",
@@ -49385,6 +49566,28 @@
}
]
},
"twoFactorEnabled": {
"type": [
"boolean",
"null"
]
},
"twoFactorMethod": {
"description": "How their sign-in codes come (2.3.0: authenticator = an app)",
"anyOf": [
{
"type": "string",
"enum": [
"email",
"whatsapp",
"authenticator"
]
},
{
"type": "null"
}
]
},
"homeCampusId": {
"type": [
"string",
+97 -4
View File
@@ -4787,7 +4787,7 @@ export interface paths {
put?: never;
/**
* Start turning on two-step sign-in
* @description Sends a code to confirm with.
* @description Sends a code to confirm with, or for `authenticator` (2.3.0) answers with `secret` and `otpauthUrl` to show as a QR code; the app's first code confirms it.
*
* **Who:** Any signed-in person.
*/
@@ -4818,6 +4818,28 @@ export interface paths {
patch?: never;
trace?: never;
};
"/me/two-factor/recovery-codes": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
/**
* A new set of recovery codes
* @description Needs the password. The old codes stop working; the new ones are shown once.
*
* **Who:** Any signed-in person.
*/
post: operations["postMeTwo-factorRecovery-codes"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/me/two-factor/disable": {
parameters: {
query?: never;
@@ -12787,6 +12809,10 @@ export interface operations {
createdAt: string;
/** @description Their role must use two-step sign-in */
twoFactorRequired: boolean;
/** @description How their codes come (2.3.0: authenticator = an app) */
twoFactorMethod: ("email" | "whatsapp" | "authenticator") | null;
/** @description Unused recovery codes (0 with two-step sign-in off) */
recoveryCodesLeft: number;
newsPromptPending: boolean;
noticePromptPending: false | ("first" | "updated");
campusScope: string[] | null;
@@ -28100,7 +28126,7 @@ export interface operations {
data: {
challengeId: string;
/** @enum {string} */
method: "email" | "whatsapp";
method: "email" | "whatsapp" | "authenticator";
/** @description Where the code went, partly hidden */
destination: string;
} & {
@@ -28472,7 +28498,7 @@ export interface operations {
content: {
"application/json": {
/** @enum {string} */
method: "email" | "whatsapp";
method: "email" | "whatsapp" | "authenticator";
password: string;
} & {
[key: string]: unknown;
@@ -28490,9 +28516,13 @@ export interface operations {
data: {
challengeId: string;
/** @enum {string} */
method: "email" | "whatsapp";
method: "email" | "whatsapp" | "authenticator";
/** @description Where the code went, partly hidden */
destination: string;
/** @description authenticator: the key to type in by hand */
secret?: string | null;
/** @description authenticator: show as a QR code */
otpauthUrl?: string | null;
} & {
[key: string]: unknown;
};
@@ -28548,6 +28578,60 @@ export interface operations {
/** @constant */
twoFactorEnabled: true;
twoFactorMethod: string;
/** @description Shown once: single-use codes for when the app or phone is lost */
recoveryCodes: string[];
} & {
[key: string]: unknown;
};
};
};
};
/** @description Validation failed (VALIDATION_FAILED) */
400: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
/** @description Not signed in (UNAUTHENTICATED) */
401: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
};
};
"postMeTwo-factorRecovery-codes": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody: {
content: {
"application/json": {
password: string;
} & {
[key: string]: unknown;
};
};
};
responses: {
/** @description OK */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
data: {
recoveryCodes: string[];
} & {
[key: string]: unknown;
};
@@ -28738,6 +28822,9 @@ export interface operations {
role: "admin" | "supervisor" | "staff" | "user";
isActive?: boolean | null;
notificationPreference?: ("email" | "whatsapp" | "both") | null;
twoFactorEnabled?: boolean | null;
/** @description How their sign-in codes come (2.3.0: authenticator = an app) */
twoFactorMethod?: ("email" | "whatsapp" | "authenticator") | null;
homeCampusId?: string | null;
createdAt?: string | null;
} & {
@@ -28810,6 +28897,9 @@ export interface operations {
role: "admin" | "supervisor" | "staff" | "user";
isActive?: boolean | null;
notificationPreference?: ("email" | "whatsapp" | "both") | null;
twoFactorEnabled?: boolean | null;
/** @description How their sign-in codes come (2.3.0: authenticator = an app) */
twoFactorMethod?: ("email" | "whatsapp" | "authenticator") | null;
homeCampusId?: string | null;
createdAt?: string | null;
/** @description Campuses they're limited to */
@@ -28972,6 +29062,9 @@ export interface operations {
role: "admin" | "supervisor" | "staff" | "user";
isActive?: boolean | null;
notificationPreference?: ("email" | "whatsapp" | "both") | null;
twoFactorEnabled?: boolean | null;
/** @description How their sign-in codes come (2.3.0: authenticator = an app) */
twoFactorMethod?: ("email" | "whatsapp" | "authenticator") | null;
homeCampusId?: string | null;
createdAt?: string | null;
} & {
+4 -1
View File
@@ -26,7 +26,7 @@ export type AuthResponse = AuthUser & {
deviceToken?: string;
};
export type TwoFactorMethod = "email" | "whatsapp";
export type TwoFactorMethod = "email" | "whatsapp" | "authenticator";
// What the login endpoint returns instead of a token when the account has 2FA on:
// a code was sent to `destination` (masked) and must be exchanged for the token.
@@ -37,6 +37,9 @@ export type TwoFactorChallenge = {
destination: string;
// The role requires 2FA and it wasn't on yet: verifying this code turns it on.
enrolling?: boolean;
// 2.3.0, setting up an authenticator app: the key and its otpauth:// link (shown as a QR code).
secret?: string;
otpauthUrl?: string;
};
export function isTwoFactorChallenge(res: unknown): res is TwoFactorChallenge {
+28
View File
@@ -8,6 +8,10 @@ This replaces the old rule "the Hub never deploys" (decisions 10 and 23). The pa
stays: **no site depends on the Hub to keep running**, and everything still goes through the
`crosscode` script (EVENTS-2.2-PLAN §4), which keeps working by hand over SSH.
**Built 2026-11-18** (crosscode-platform branch `feat/agent-1.0`, events-hub branch
`feat/servers-agent`; not merged or released). Waiting for the VM tests (§13). Where the build
differs from this plan, see [Changes while building](#changes-while-building).
## Contents
1. [The shape](#1-the-shape)
@@ -453,6 +457,30 @@ Central check: both ports are open in frps and only on 127.0.0.1.
5. Docs: CHANGELOGs, HANDOVER.md (1d done), `events-hub/deployment.md` (the job-signing key),
a server how-to in `apps/agent/README.md`.
## Changes while building
Made on 2026-11-18, while building; none changes what Joshua or a church sees.
1. **Config encryption** is X25519 + HKDF-SHA256 + AES-256-GCM (`x25519-hkdf-sha256-aes-256-gcm`)
instead of `x25519-xsalsa20-poly1305`: Node's built-in crypto has no XSalsa20, and this needs no
extra library on the servers. Same guarantee: only the server's private key opens it.
2. **Deployments, not new Install fields.** An `Install` row only exists once a site registers
with its licence, so the Hub keeps a `Deployment` per product per server (short name, domain,
ports, status), linked to its Install when the site registers. Installs made from the Hub
register by themselves: `HUB_LICENCE_KEY` in their `.env` (Events, on first start).
3. **The export key** goes to the Hub in the export job's report, so the Hub can email the link;
it's used once and never stored (job rows keep the report without it). The Hub still can't read
the stored file.
4. **Jobs are signed when the agent picks them up**, not when they're planned, so a scheduled job
carries the server configuration as it is at that moment.
5. **Uninstall runs as a batch of two** (export, then uninstall), so the uninstall never runs if
the export failed.
6. **Install behind frp** doesn't use nginx on the church server: frpc tunnels the app's two ports
straight to the central server (`crosscode install --frp`). The ports from the Hub are used for
`local_port` and `remote_port`.
7. **`crosscode install`** refuses a database or role that already exists, so it can never take
over (or, on a failed install, drop) another site's database.
## Decisions
Agreed with Joshua on 2026-11-12 (ROADMAP-v3 decisions 27–37):
+1 -1
View File
@@ -27,7 +27,7 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05.
| 1 | Hub 2.0.0 | **Released** 2026-11-05 (`events-hub` `v2.0.0`; `docs/HUB-2.0-PLAN.md` there). **Not deployed yet:** `events-hub/deployment.md` → "Upgrading to 2.0" (run `check-hub2` first). crosscode-site's address prices are merged, **not deployed yet** |
| 1b | Cross Events 2.2.0: moves here as `apps/events`, `crosscode` script | **In progress.** Planned in [EVENTS-2.2-PLAN.md](./EVENTS-2.2-PLAN.md). **Steps 1–3 done 2026-11-12** (merged into `main`): history merged in under `apps/events` (from `hope-events` `e6c9670`), workspaces and one root lockfile with versions pinned to 2.1.1, same test results as 2.1.1 (711 backend tests, typecheck, both builds). **The `crosscode` script and `adopt` (§4–5) are written** (branch `feat/crosscode-script`, 2026-11-12; shellcheck and 47 bats tests pass, not yet run on a server). **Tagged `events@2.2.0` on 2026-11-17, before the VM testing (Joshua's call), with two additions: AI use in tokens and the backend under `/api` (Hub `v2.0.1` and crosscode-site `v1.2.0` released with them).** **Still to do:** VM testing (§6) of the tagged release, adopting each install, archiving `hope-events` (§7). Until then servers keep deploying from `hope-events` `v2.1.1`, and **Events work happens here** (`hope-events` is frozen; a hotfix there as 2.1.2 is cherry-picked into `apps/events`) |
| 1c | Cross Events 2.3.0: authenticator app 2FA (TOTP + recovery codes) | TOTP not started (`TwoFactorMethod` enum in Events' `prisma/schema.prisma`). |
| 1d | Agent 1.0.0 + Hub 2.1.0 (+ contracts 1.3.0): server agent, installs / scheduled updates / uninstalls from the Hub, server stats | **Planned** 2026-11-12 in [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md) (ROADMAP decisions 27–37). Starts after 1b; can run alongside 1c |
| 1d | Agent 1.0.0 + Hub 2.1.0 (+ contracts 1.3.0): server agent, installs / scheduled updates / uninstalls from the Hub, server stats | **Built 2026-11-18, not released.** crosscode-platform branch `feat/agent-1.0` (contracts 1.3.0, the crosscode script with `--json`, env templates, frpc, restart / export / uninstall / purge, Events env template + export + `HUB_LICENCE_KEY`, `apps/agent`), events-hub branch `feat/servers-agent` (Hub 2.1.0). All unit and script tests pass. **Still to do:** the VM tests (AGENT-1.0-PLAN §13, together with EVENTS-2.2-PLAN §6), then release and roll out (§14). |
| 2 | Cross Events 2.4.0: protocol v2, Linked products, event feed, `packages/seo` | Not started. Events keeps speaking protocol 1 until then |
| 3–7 | Accounts 1.0.0 → Website 1.0.0 → Events 3.0.0 → App 1.0.0 → App 1.x | Not started |
+22 -3
View File
@@ -38,13 +38,28 @@
"name": "@crosscode/accounts",
"version": "0.0.0"
},
"apps/agent": {
"name": "@crosscode/agent",
"version": "1.0.0",
"license": "UNLICENSED",
"dependencies": {
"@crosscode/contracts": "*",
"zod": "^4.6.7"
},
"bin": {
"crosscode-agent": "dist/cli.js"
},
"engines": {
"node": ">=24"
}
},
"apps/app": {
"name": "@crosscode/app",
"version": "0.0.0"
},
"apps/events/backend": {
"name": "@crosscode/events-backend",
"version": "2.2.0",
"version": "2.3.0",
"hasInstallScript": true,
"dependencies": {
"@cj-tech-master/excelts": "^10.2.0",
@@ -91,7 +106,7 @@
},
"apps/events/frontend": {
"name": "@crosscode/events-frontend",
"version": "2.2.0",
"version": "2.3.0",
"dependencies": {
"@dnd-kit/core": "^6.3.1",
"@hookform/resolvers": "^5.2.1",
@@ -876,6 +891,10 @@
"resolved": "apps/accounts",
"link": true
},
"node_modules/@crosscode/agent": {
"resolved": "apps/agent",
"link": true
},
"node_modules/@crosscode/app": {
"resolved": "apps/app",
"link": true
@@ -19633,7 +19652,7 @@
},
"packages/contracts": {
"name": "@crosscode/contracts",
"version": "1.2.0",
"version": "1.3.0",
"license": "UNLICENSED",
"dependencies": {
"zod": "^4.6.7"
+1 -1
View File
@@ -16,7 +16,7 @@
"test:scripts": "bats scripts/tests",
"typecheck": "npm run typecheck --workspaces --if-present",
"lint": "eslint .",
"lint:scripts": "shellcheck -x scripts/crosscode scripts/products/events.sh",
"lint:scripts": "shellcheck -x scripts/crosscode scripts/products/events.sh scripts/lib/report.sh scripts/lib/env.sh scripts/lib/frpc.sh scripts/lib/manage.sh apps/agent/deploy/install.sh apps/agent/deploy/revert.sh",
"check": "npm run lint && npm run lint:scripts && npm run typecheck && npm test && npm run test:scripts && npm run build"
},
"engines": {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@crosscode/contracts",
"version": "1.2.0",
"version": "1.3.0",
"description": "The shared JSON formats used by every Cross Code product: zod schemas, JSON Schema, TypeScript types and sample payloads",
"license": "UNLICENSED",
"type": "commonjs",
@@ -0,0 +1,50 @@
{
"schema": "crosscode.church-export/1",
"exportedAt": "2026-11-20T12:00:00.000Z",
"product": "events",
"productVersion": "2.3.0",
"church": { "name": "Grace Demo Church", "timezone": "Africa/Johannesburg" },
"people": [
{
"ref": "p1",
"name": "Thandi Example",
"email": "thandi@example.org",
"phone": "+27821234567",
"role": "admin",
"createdAt": "2026-01-10T08:00:00.000Z",
"consent": { "email": true, "whatsapp": false, "privacyNoticeAcceptedAt": "2026-01-10T08:00:00.000Z" }
}
],
"events": [
{
"ref": "e1",
"title": "Youth Camp",
"description": "A weekend away.",
"startsAt": "2026-12-04T16:00:00.000Z",
"endsAt": "2026-12-06T12:00:00.000Z",
"location": "Camp Example",
"sessions": [{ "label": "Friday", "startsAt": "2026-12-04T16:00:00.000Z", "endsAt": "2026-12-04T21:00:00.000Z" }],
"ticketTypes": [{ "ref": "t1", "name": "Camper", "priceCents": 60000, "currency": "ZAR" }],
"capacity": 80,
"published": true
}
],
"registrations": [
{
"ref": "r1",
"eventRef": "e1",
"personRef": "p1",
"status": "confirmed",
"createdAt": "2026-11-01T09:00:00.000Z",
"attendees": [{ "name": "Thandi Example", "email": "thandi@example.org", "ticketTypeRef": "t1", "checkedInAt": null }],
"answers": [{ "question": "Dietary needs", "answer": "None" }]
}
],
"payments": [
{ "registrationRef": "r1", "amountCents": 60000, "currency": "ZAR", "method": "card", "status": "paid", "paidAt": "2026-11-01T09:05:00.000Z", "reference": "YOCO-123" }
],
"checkIns": [],
"rosters": [{ "eventRef": "e1", "role": "Kitchen", "personRef": "p1", "startsAt": null, "endsAt": null }],
"settings": { "org_name": "Grace Demo Church" },
"files": [{ "path": "uploads/events/camp.jpg", "bytes": 120400 }]
}
@@ -0,0 +1,56 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-job-config.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"server": {
"default": {},
"type": "object",
"propertyNames": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"additionalProperties": {
"type": "string",
"maxLength": 4000
}
},
"defaults": {
"default": {},
"type": "object",
"propertyNames": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"additionalProperties": {
"type": "string",
"maxLength": 4000
}
},
"install": {
"default": {},
"type": "object",
"propertyNames": {
"type": "string",
"enum": [
"events",
"website",
"app",
"accounts"
]
},
"additionalProperties": {
"type": "object",
"propertyNames": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"additionalProperties": {
"type": "string",
"maxLength": 4000
}
}
}
},
"additionalProperties": false
}
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,55 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-join-request.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"agentApiVersion": {
"type": "number",
"const": 1
},
"token": {
"type": "string",
"pattern": "^[A-Za-z0-9_-]{32,128}$"
},
"mode": {
"type": "string",
"enum": [
"church",
"central"
]
},
"hostname": {
"type": "string",
"maxLength": 253
},
"agentVersion": {
"type": "string",
"maxLength": 40
},
"publicKeys": {
"type": "object",
"properties": {
"ed25519": {
"type": "string",
"pattern": "^[A-Za-z0-9_-]{43}$"
},
"x25519": {
"type": "string",
"pattern": "^[A-Za-z0-9_-]{43}$"
}
},
"required": [
"ed25519",
"x25519"
]
}
},
"required": [
"agentApiVersion",
"token",
"mode",
"hostname",
"agentVersion",
"publicKeys"
]
}
@@ -0,0 +1,31 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-join-response.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"serverId": {
"type": "string",
"pattern": "^srv_[A-Za-z0-9_-]{8,64}$"
},
"hubJobKey": {
"type": "string",
"pattern": "^[A-Za-z0-9_-]{43}$"
},
"pollSeconds": {
"type": "integer",
"minimum": 5,
"maximum": 600
},
"statsSeconds": {
"type": "integer",
"minimum": 15,
"maximum": 3600
}
},
"required": [
"serverId",
"hubJobKey",
"pollSeconds",
"statsSeconds"
]
}
@@ -0,0 +1,39 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-poll-request.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"agentApiVersion": {
"type": "number",
"const": 1
},
"agentVersion": {
"type": "string",
"maxLength": 40
},
"mode": {
"type": "string",
"enum": [
"church",
"central"
]
},
"runningJobId": {
"anyOf": [
{
"type": "string",
"pattern": "^job_[A-Za-z0-9_-]{8,64}$"
},
{
"type": "null"
}
]
}
},
"required": [
"agentApiVersion",
"agentVersion",
"mode",
"runningJobId"
]
}
@@ -0,0 +1,21 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-poll-response.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"jobs": {
"maxItems": 10,
"type": "array",
"items": {}
},
"pollSeconds": {
"type": "integer",
"minimum": 5,
"maximum": 600
}
},
"required": [
"jobs",
"pollSeconds"
]
}
@@ -0,0 +1,136 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-report.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"schema": {
"type": "string",
"const": "crosscode.agent-report/1"
},
"jobId": {
"type": "string",
"pattern": "^job_[A-Za-z0-9_-]{8,64}$"
},
"serverId": {
"type": "string",
"pattern": "^srv_[A-Za-z0-9_-]{8,64}$"
},
"type": {
"type": "string",
"enum": [
"install",
"add-product",
"deploy",
"adopt",
"adopt-undo",
"restart",
"export",
"uninstall",
"purge",
"update-agent",
"check-ports",
"status"
]
},
"status": {
"type": "string",
"enum": [
"succeeded",
"failed",
"rolled-back",
"refused",
"expired"
]
},
"startedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"steps": {
"maxItems": 200,
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 200
},
"startedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"finishedAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"outcome": {
"type": "string",
"enum": [
"ok",
"failed",
"skipped",
"rolled-back"
]
},
"detail": {
"type": "string",
"maxLength": 2000
}
},
"required": [
"name",
"startedAt",
"finishedAt",
"outcome"
]
}
},
"result": {
"default": {},
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {}
},
"message": {
"type": "string",
"maxLength": 2000
},
"log": {
"maxItems": 200,
"type": "array",
"items": {
"type": "string",
"maxLength": 1000
}
}
},
"required": [
"schema",
"jobId",
"serverId",
"type",
"status",
"startedAt",
"finishedAt",
"steps",
"log"
]
}
+296
View File
@@ -0,0 +1,296 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/agent-stats.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"schema": {
"type": "string",
"const": "crosscode.agent-stats/1"
},
"at": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"ram": {
"type": "object",
"properties": {
"totalBytes": {
"type": "number",
"minimum": 0
},
"usedBytes": {
"type": "number",
"minimum": 0
},
"availableBytes": {
"type": "number",
"minimum": 0
},
"swapTotalBytes": {
"type": "number",
"minimum": 0
},
"swapUsedBytes": {
"type": "number",
"minimum": 0
}
},
"required": [
"totalBytes",
"usedBytes",
"availableBytes",
"swapTotalBytes",
"swapUsedBytes"
]
},
"cpu": {
"type": "object",
"properties": {
"percent": {
"type": "number",
"minimum": 0,
"maximum": 100
},
"load1": {
"type": "number",
"minimum": 0
},
"load5": {
"type": "number",
"minimum": 0
},
"load15": {
"type": "number",
"minimum": 0
},
"cores": {
"type": "integer",
"exclusiveMinimum": 0,
"maximum": 9007199254740991
}
},
"required": [
"percent",
"load1",
"load5",
"load15",
"cores"
]
},
"disks": {
"maxItems": 30,
"type": "array",
"items": {
"type": "object",
"properties": {
"mount": {
"type": "string",
"maxLength": 255
},
"totalBytes": {
"type": "number",
"minimum": 0
},
"usedBytes": {
"type": "number",
"minimum": 0
},
"freeBytes": {
"type": "number",
"minimum": 0
}
},
"required": [
"mount",
"totalBytes",
"usedBytes",
"freeBytes"
]
}
},
"uptimeSeconds": {
"type": "number",
"minimum": 0
},
"versions": {
"type": "object",
"properties": {
"os": {
"type": "string",
"maxLength": 200
},
"node": {
"type": "string",
"maxLength": 40
},
"postgres": {
"anyOf": [
{
"type": "string",
"maxLength": 40
},
{
"type": "null"
}
]
},
"frpc": {
"anyOf": [
{
"type": "string",
"maxLength": 40
},
{
"type": "null"
}
]
},
"agent": {
"type": "string",
"maxLength": 40
}
},
"required": [
"os",
"node",
"postgres",
"frpc",
"agent"
]
},
"frpcRunning": {
"type": [
"boolean",
"null"
]
},
"processes": {
"maxItems": 200,
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 100
},
"church": {
"anyOf": [
{
"type": "string",
"pattern": "^[a-z][a-z0-9-]{1,30}$"
},
{
"type": "null"
}
]
},
"product": {
"anyOf": [
{
"type": "string",
"enum": [
"events",
"website",
"app",
"accounts"
]
},
{
"type": "null"
}
]
},
"status": {
"type": "string",
"maxLength": 40
},
"memoryBytes": {
"type": "number",
"minimum": 0
},
"cpuPercent": {
"type": "number",
"minimum": 0
},
"restarts": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"uptimeSeconds": {
"type": "number",
"minimum": 0
}
},
"required": [
"name",
"church",
"product",
"status",
"memoryBytes",
"cpuPercent",
"restarts",
"uptimeSeconds"
]
}
},
"central": {
"anyOf": [
{
"type": "object",
"properties": {
"proxyBindAddr": {
"anyOf": [
{
"type": "string",
"maxLength": 100
},
{
"type": "null"
}
]
},
"allowPorts": {
"anyOf": [
{
"type": "string",
"maxLength": 2000
},
{
"type": "null"
}
]
},
"frpsRunning": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"proxyBindAddr",
"allowPorts",
"frpsRunning"
]
},
{
"type": "null"
}
]
}
},
"required": [
"schema",
"at",
"ram",
"cpu",
"disks",
"uptimeSeconds",
"versions",
"frpcRunning",
"processes"
]
}
@@ -0,0 +1,676 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/church-export.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"schema": {
"type": "string",
"const": "crosscode.church-export/1"
},
"exportedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"product": {
"type": "string",
"enum": [
"events",
"website",
"app",
"accounts"
]
},
"productVersion": {
"type": "string",
"maxLength": 40
},
"church": {
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 300
},
"timezone": {
"anyOf": [
{
"type": "string",
"pattern": "^[A-Za-z_]+(\\/[A-Za-z0-9_+-]+){0,2}$|^UTC$"
},
{
"type": "null"
}
]
}
},
"required": [
"name",
"timezone"
]
},
"people": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ref": {
"type": "string",
"pattern": "^p[0-9]+$"
},
"name": {
"type": "string",
"maxLength": 300
},
"email": {
"anyOf": [
{
"type": "string",
"maxLength": 254
},
{
"type": "null"
}
]
},
"phone": {
"anyOf": [
{
"type": "string",
"maxLength": 40
},
{
"type": "null"
}
]
},
"role": {
"type": "string",
"maxLength": 40
},
"createdAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"consent": {
"type": "object",
"properties": {
"email": {
"type": [
"boolean",
"null"
]
},
"whatsapp": {
"type": [
"boolean",
"null"
]
},
"privacyNoticeAcceptedAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
}
},
"required": [
"email",
"whatsapp",
"privacyNoticeAcceptedAt"
]
}
},
"required": [
"ref",
"name",
"email",
"phone",
"role",
"createdAt",
"consent"
]
}
},
"events": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ref": {
"type": "string",
"pattern": "^e[0-9]+$"
},
"title": {
"type": "string",
"maxLength": 500
},
"description": {
"anyOf": [
{
"type": "string",
"maxLength": 20000
},
{
"type": "null"
}
]
},
"startsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"endsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"location": {
"anyOf": [
{
"type": "string",
"maxLength": 500
},
{
"type": "null"
}
]
},
"sessions": {
"type": "array",
"items": {
"type": "object",
"properties": {
"label": {
"anyOf": [
{
"type": "string",
"maxLength": 300
},
{
"type": "null"
}
]
},
"startsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"endsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
}
},
"required": [
"label",
"startsAt",
"endsAt"
]
}
},
"ticketTypes": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ref": {
"type": "string",
"pattern": "^t[0-9]+$"
},
"name": {
"type": "string",
"maxLength": 300
},
"priceCents": {
"type": "integer",
"minimum": -9007199254740991,
"maximum": 9007199254740991
},
"currency": {
"type": "string",
"minLength": 3,
"maxLength": 3
}
},
"required": [
"ref",
"name",
"priceCents",
"currency"
]
}
},
"capacity": {
"anyOf": [
{
"type": "integer",
"minimum": -9007199254740991,
"maximum": 9007199254740991
},
{
"type": "null"
}
]
},
"published": {
"type": "boolean"
}
},
"required": [
"ref",
"title",
"description",
"startsAt",
"endsAt",
"location",
"sessions",
"ticketTypes",
"capacity",
"published"
]
}
},
"registrations": {
"type": "array",
"items": {
"type": "object",
"properties": {
"ref": {
"type": "string",
"pattern": "^r[0-9]+$"
},
"eventRef": {
"type": "string",
"pattern": "^e[0-9]+$"
},
"personRef": {
"anyOf": [
{
"type": "string",
"pattern": "^p[0-9]+$"
},
{
"type": "null"
}
]
},
"status": {
"type": "string",
"maxLength": 40
},
"createdAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"attendees": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 300
},
"email": {
"anyOf": [
{
"type": "string",
"maxLength": 254
},
{
"type": "null"
}
]
},
"ticketTypeRef": {
"anyOf": [
{
"type": "string",
"pattern": "^t[0-9]+$"
},
{
"type": "null"
}
]
},
"checkedInAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
}
},
"required": [
"name",
"email",
"ticketTypeRef",
"checkedInAt"
]
}
},
"answers": {
"default": [],
"type": "array",
"items": {
"type": "object",
"properties": {
"question": {
"type": "string",
"maxLength": 500
},
"answer": {
"type": "string",
"maxLength": 5000
}
},
"required": [
"question",
"answer"
]
}
}
},
"required": [
"ref",
"eventRef",
"personRef",
"status",
"createdAt",
"attendees"
]
}
},
"payments": {
"type": "array",
"items": {
"type": "object",
"properties": {
"registrationRef": {
"anyOf": [
{
"type": "string",
"pattern": "^r[0-9]+$"
},
{
"type": "null"
}
]
},
"amountCents": {
"type": "integer",
"minimum": -9007199254740991,
"maximum": 9007199254740991
},
"currency": {
"type": "string",
"minLength": 3,
"maxLength": 3
},
"method": {
"type": "string",
"maxLength": 40
},
"status": {
"type": "string",
"maxLength": 40
},
"paidAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"reference": {
"anyOf": [
{
"type": "string",
"maxLength": 200
},
{
"type": "null"
}
]
}
},
"required": [
"registrationRef",
"amountCents",
"currency",
"method",
"status",
"paidAt",
"reference"
]
}
},
"checkIns": {
"type": "array",
"items": {
"type": "object",
"properties": {
"eventRef": {
"type": "string",
"pattern": "^e[0-9]+$"
},
"personRef": {
"anyOf": [
{
"type": "string",
"pattern": "^p[0-9]+$"
},
{
"type": "null"
}
]
},
"name": {
"anyOf": [
{
"type": "string",
"maxLength": 300
},
{
"type": "null"
}
]
},
"at": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
}
},
"required": [
"eventRef",
"personRef",
"name",
"at"
]
}
},
"rosters": {
"type": "array",
"items": {
"type": "object",
"properties": {
"eventRef": {
"anyOf": [
{
"type": "string",
"pattern": "^e[0-9]+$"
},
{
"type": "null"
}
]
},
"role": {
"type": "string",
"maxLength": 200
},
"personRef": {
"anyOf": [
{
"type": "string",
"pattern": "^p[0-9]+$"
},
{
"type": "null"
}
]
},
"startsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"endsAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
}
},
"required": [
"eventRef",
"role",
"personRef",
"startsAt",
"endsAt"
]
}
},
"settings": {
"type": "object",
"propertyNames": {
"type": "string",
"maxLength": 100
},
"additionalProperties": {
"type": "string",
"maxLength": 5000
}
},
"files": {
"type": "array",
"items": {
"type": "object",
"properties": {
"path": {
"type": "string",
"pattern": "^uploads\\/[A-Za-z0-9._/-]+$"
},
"bytes": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
}
},
"required": [
"path",
"bytes"
]
}
}
},
"required": [
"schema",
"exportedAt",
"product",
"productVersion",
"church",
"people",
"events",
"registrations",
"payments",
"checkIns",
"rosters",
"settings",
"files"
]
}
@@ -0,0 +1,74 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/env-template.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"schema": {
"type": "string",
"const": "crosscode.env-template/1"
},
"product": {
"type": "string",
"enum": [
"events",
"website",
"app",
"accounts"
]
},
"files": {
"type": "object",
"propertyNames": {
"type": "string",
"pattern": "^[a-z0-9._/-]+$"
},
"additionalProperties": {
"minItems": 1,
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"pattern": "^[A-Z][A-Z0-9_]*$"
},
"layer": {
"type": "string",
"enum": [
"server",
"defaults",
"install",
"generated"
]
},
"required": {
"default": false,
"type": "boolean"
},
"default": {
"type": "string",
"maxLength": 4000
},
"from": {
"type": "string",
"maxLength": 400
},
"description": {
"type": "string",
"maxLength": 400
}
},
"required": [
"name",
"layer"
]
}
}
}
},
"required": [
"schema",
"product",
"files"
]
}
@@ -0,0 +1,104 @@
{
"$id": "https://c-git.crosscode.co.za/joshua/crosscode-platform/raw/branch/main/packages/contracts/schemas/script-report.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"succeeded",
"failed",
"rolled-back",
"refused",
"expired"
]
},
"startedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"steps": {
"maxItems": 200,
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string",
"maxLength": 200
},
"startedAt": {
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
"finishedAt": {
"anyOf": [
{
"type": "string",
"format": "date-time",
"pattern": "^(?:(?:\\d\\d[2468][048]|\\d\\d[13579][26]|\\d\\d0[48]|[02468][048]00|[13579][26]00)-02-29|\\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\\d|30)|(?:02)-(?:0[1-9]|1\\d|2[0-8])))T(?:(?:[01]\\d|2[0-3]):[0-5]\\d:[0-5]\\d(?:\\.\\d+)?(?:Z))$"
},
{
"type": "null"
}
]
},
"outcome": {
"type": "string",
"enum": [
"ok",
"failed",
"skipped",
"rolled-back"
]
},
"detail": {
"type": "string",
"maxLength": 2000
}
},
"required": [
"name",
"startedAt",
"finishedAt",
"outcome"
]
}
},
"result": {
"default": {},
"type": "object",
"propertyNames": {
"type": "string"
},
"additionalProperties": {}
},
"message": {
"type": "string",
"maxLength": 2000
},
"schema": {
"type": "string",
"const": "crosscode.script-report/1"
},
"command": {
"type": "string",
"maxLength": 40
}
},
"required": [
"status",
"startedAt",
"finishedAt",
"steps",
"schema",
"command"
]
}
@@ -41,6 +41,17 @@ function contracts(c) {
"hub-ai-complete-response": c.hub.aiCompleteResponseSchema,
"hub-error": c.hub.hubErrorSchema,
"health": c.hub.healthSchema,
"agent-job": c.agent.agentJobSchema,
"agent-report": c.agent.agentReportSchema,
"script-report": c.agent.scriptReportSchema,
"agent-stats": c.agent.agentStatsSchema,
"agent-join-request": c.agent.joinRequestSchema,
"agent-join-response": c.agent.joinResponseSchema,
"agent-poll-request": c.agent.pollRequestSchema,
"agent-poll-response": c.agent.pollResponseSchema,
"agent-job-config": c.agent.jobConfigSchema,
"env-template": c.agent.envTemplateSchema,
"church-export": c.churchExportSchema,
};
}
+320
View File
@@ -0,0 +1,320 @@
import { z } from "zod";
import { dateTimeSchema, productSchema } from "./common";
/**
* The server agent (docs/AGENT-1.0-PLAN.md, ROADMAP-v3 phase 1d): what the Hub and the agent on
* each server send each other. The agent only polls (it listens on nothing), runs only the
* `crosscode` script, and only for jobs of the fixed types below, signed by the Hub.
*
* Signing and encryption (Node only) are in `@crosscode/contracts/signing`:
* - jobs: Ed25519 by the Hub over `canonicalJson(job without "signature")`;
* - agent → Hub requests: Ed25519 by the agent over `timestamp + "." + rawBody` ({@link AGENT_HEADERS});
* - the job's `config`: sealed to the server's X25519 key (X25519 + HKDF-SHA256 + AES-256-GCM).
*/
export const AGENT_API_VERSION = 1;
/** Headers on every agent → Hub request (after joining). */
export const AGENT_HEADERS = {
server: "x-crosscode-server",
timestamp: "x-crosscode-timestamp",
signature: "x-crosscode-signature",
} as const;
// ---------------------------------------------------------------------------------------------
// Building blocks
// ---------------------------------------------------------------------------------------------
/** A church's short name: folders, PM2 names, database names, tunnel names (`hope`, `cross-roads`). */
export const shortNameSchema = z.string().regex(/^[a-z][a-z0-9-]{1,30}$/, "Expected a short name: a-z, 0-9 and -, starting with a letter");
/** A release tag, `<product>@x.y.z` (no prereleases: jobs only install releases). */
export const releaseTagSchema = z.string().regex(/^(accounts|events|website|app)@\d+\.\d+\.\d+$/, "Expected <product>@x.y.z");
/** An agent release tag, `agent@x.y.z`. */
export const agentTagSchema = z.string().regex(/^agent@\d+\.\d+\.\d+$/, "Expected agent@x.y.z");
/** A host name (no scheme, no path): `events.hope.co.za`. */
export const domainSchema = z.string().max(253).regex(/^(?=.{1,253}$)([a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,}$/, "Expected a domain like events.hope.co.za");
/** A TCP port the Hub hands out (frp tunnels use the same number on both ends). */
export const portSchema = z.int().min(1024).max(65535);
/** An absolute path on the server without `..`, spaces or shell characters. */
export const serverPathSchema = z.string().max(255).regex(/^\/(?!.*\/\.\.?(\/|$))[A-Za-z0-9._/-]+$/, "Expected an absolute path");
export const productPortsSchema = z.object({ web: portSchema, api: portSchema }).refine((p) => p.web !== p.api, "web and api must differ");
export type ProductPorts = z.infer<typeof productPortsSchema>;
export const AGENT_MODES = ["church", "central"] as const;
export const agentModeSchema = z.enum(AGENT_MODES);
export type AgentMode = z.infer<typeof agentModeSchema>;
// ---------------------------------------------------------------------------------------------
// Jobs (crosscode.agent-job/1)
// ---------------------------------------------------------------------------------------------
/** Each job type and the arguments it takes. Unknown keys are refused, not stripped. */
export const jobArgsSchemas = {
/** `crosscode install <church> --products … --frp --json` */
install: z.strictObject({
church: shortNameSchema,
products: z.array(productSchema).min(1).max(4),
/** Per product: the release (default: the latest). */
versions: z.partialRecord(productSchema, releaseTagSchema).optional(),
/** Per product: where it's served. */
domains: z.partialRecord(productSchema, domainSchema),
/** Per product: the ports from the Hub's port list. */
ports: z.partialRecord(productSchema, productPortsSchema),
}),
/** `crosscode add <church> <product>` (from phase 3). */
"add-product": z.strictObject({ church: shortNameSchema, product: productSchema, version: releaseTagSchema.optional(), domain: domainSchema, ports: productPortsSchema }),
/** `crosscode deploy <church> [products] [--version …] --json` */
deploy: z.strictObject({
church: shortNameSchema,
products: z.array(productSchema).min(1).max(4),
versions: z.array(releaseTagSchema).max(4).optional(),
check: z.boolean().optional(),
}),
/** `crosscode adopt <church> --events-dir <dir> [--version …] [--address …] --json` */
adopt: z.strictObject({ church: shortNameSchema, eventsDir: serverPathSchema, version: releaseTagSchema.optional(), address: domainSchema.optional() }),
"adopt-undo": z.strictObject({ church: shortNameSchema }),
/** `crosscode restart <church> [product] --json` (PM2 reload) */
restart: z.strictObject({ church: shortNameSchema, product: productSchema.optional() }),
/** `crosscode export <church> --out <zip> --json`; the agent encrypts the ZIP and uploads it under `exportId`. */
export: z.strictObject({ church: shortNameSchema, exportId: z.string().regex(/^exp_[A-Za-z0-9_-]{8,64}$/) }),
/** `crosscode uninstall <church> --json` */
uninstall: z.strictObject({ church: shortNameSchema }),
/** `crosscode purge <church> --json` (deletes what uninstall kept for 30 days) */
purge: z.strictObject({ church: shortNameSchema }),
/** The agent updates itself to that tag. */
"update-agent": z.strictObject({ version: agentTagSchema }),
/** Central server only: are these ports allowed, open and bound to 127.0.0.1? */
"check-ports": z.strictObject({ ports: z.array(portSchema).min(1).max(50) }),
/** `crosscode status [<church>] --json` */
status: z.strictObject({ church: shortNameSchema.optional() }),
} as const;
export type JobType = keyof typeof jobArgsSchemas;
export const JOB_TYPES = Object.keys(jobArgsSchemas) as JobType[];
export const jobTypeSchema = z.enum(JOB_TYPES as [JobType, ...JobType[]]);
export type JobArgs<T extends JobType> = z.infer<(typeof jobArgsSchemas)[T]>;
/** The only jobs a central agent runs (§3). */
export const CENTRAL_JOB_TYPES: readonly JobType[] = ["check-ports", "status", "update-agent"];
/** Time limits per job type, in minutes (§3). A job past it is stopped and reported as failed. */
export const JOB_TIME_LIMIT_MINUTES: Record<JobType, number> = {
install: 30, "add-product": 30, deploy: 20, adopt: 30, "adopt-undo": 15, restart: 5, export: 20,
uninstall: 20, purge: 10, "update-agent": 15, "check-ports": 2, status: 3,
};
/** Config sealed to the server's X25519 key (`sealConfig` in `@crosscode/contracts/signing`). */
export const sealedSchema = z.strictObject({
alg: z.literal("x25519-hkdf-sha256-aes-256-gcm"),
/** The sender's one-time X25519 public key, base64url (raw 32 bytes). */
epk: z.string().regex(/^[A-Za-z0-9_-]{43}$/),
/** 12-byte nonce, base64url. */
iv: z.string().regex(/^[A-Za-z0-9_-]{16}$/),
/** Ciphertext with the 16-byte GCM tag appended, base64url. */
ciphertext: z.string().regex(/^[A-Za-z0-9_-]+$/).max(200_000),
});
export type Sealed = z.infer<typeof sealedSchema>;
/**
* What a job's sealed `config` holds once opened (§5): `.env` values by layer, each keyed by the
* variable name. `generated` values are never sent: the server makes them.
*/
export const jobConfigSchema = z.strictObject({
server: z.record(z.string().regex(/^[A-Z][A-Z0-9_]*$/), z.string().max(4000)).default({}),
defaults: z.record(z.string().regex(/^[A-Z][A-Z0-9_]*$/), z.string().max(4000)).default({}),
/** Per product: the install's own values (URLs, timezone, licence key, PORT). */
install: z.partialRecord(productSchema, z.record(z.string().regex(/^[A-Z][A-Z0-9_]*$/), z.string().max(4000))).default({}),
});
export type JobConfig = z.infer<typeof jobConfigSchema>;
const jobBase = {
schema: z.literal("crosscode.agent-job/1"),
id: z.string().regex(/^job_[A-Za-z0-9_-]{8,64}$/),
serverId: z.string().regex(/^srv_[A-Za-z0-9_-]{8,64}$/),
config: sealedSchema.nullable().optional(),
notBefore: dateTimeSchema,
expiresAt: dateTimeSchema,
/** Ed25519 over `canonicalJson(job without "signature")`, base64url. */
signature: z.string().regex(/^[A-Za-z0-9_-]{86}$/),
};
function jobOf<T extends JobType>(type: T) {
return z.strictObject({ ...jobBase, type: z.literal(type), args: jobArgsSchemas[type] });
}
export const agentJobSchema = z.discriminatedUnion("type", [
jobOf("install"), jobOf("add-product"), jobOf("deploy"), jobOf("adopt"), jobOf("adopt-undo"), jobOf("restart"),
jobOf("export"), jobOf("uninstall"), jobOf("purge"), jobOf("update-agent"), jobOf("check-ports"), jobOf("status"),
]);
export type AgentJob = z.infer<typeof agentJobSchema>;
export type UnsignedAgentJob = Omit<AgentJob, "signature">;
// ---------------------------------------------------------------------------------------------
// Reports (crosscode.agent-report/1)
// ---------------------------------------------------------------------------------------------
export const JOB_STATUSES = ["succeeded", "failed", "rolled-back", "refused", "expired"] as const;
export const jobStatusSchema = z.enum(JOB_STATUSES);
export type JobStatus = z.infer<typeof jobStatusSchema>;
export const reportStepSchema = z.object({
name: z.string().max(200),
startedAt: dateTimeSchema,
finishedAt: dateTimeSchema.nullable(),
outcome: z.enum(["ok", "failed", "skipped", "rolled-back"]),
detail: z.string().max(2000).optional(),
});
export type ReportStep = z.infer<typeof reportStepSchema>;
/**
* A job's result. The `crosscode … --json` output is the report without `jobId`, `serverId` and
* `log`; the agent adds those (the log's last 200 lines, filtered for secrets) and signs the request.
*/
export const agentReportSchema = z.object({
schema: z.literal("crosscode.agent-report/1"),
jobId: z.string().regex(/^job_[A-Za-z0-9_-]{8,64}$/),
serverId: z.string().regex(/^srv_[A-Za-z0-9_-]{8,64}$/),
type: jobTypeSchema,
status: jobStatusSchema,
startedAt: dateTimeSchema,
finishedAt: dateTimeSchema,
steps: z.array(reportStepSchema).max(200),
/** What the command found or did, by type (versions, ports, the export's size …). */
result: z.record(z.string(), z.unknown()).default({}),
/** Why a job was refused or failed, in a sentence. */
message: z.string().max(2000).optional(),
log: z.array(z.string().max(1000)).max(200),
});
export type AgentReport = z.infer<typeof agentReportSchema>;
/** What `crosscode … --json` prints: the report's own part. */
export const scriptReportSchema = agentReportSchema.omit({ schema: true, jobId: true, serverId: true, type: true, log: true }).extend({
schema: z.literal("crosscode.script-report/1"),
command: z.string().max(40),
});
export type ScriptReport = z.infer<typeof scriptReportSchema>;
// ---------------------------------------------------------------------------------------------
// Stats (crosscode.agent-stats/1), every 60 s
// ---------------------------------------------------------------------------------------------
const bytes = z.number().nonnegative();
export const processStatSchema = z.object({
name: z.string().max(100),
church: shortNameSchema.nullable(),
product: productSchema.nullable(),
status: z.string().max(40),
memoryBytes: bytes,
cpuPercent: z.number().nonnegative(),
restarts: z.int().nonnegative(),
uptimeSeconds: z.number().nonnegative(),
});
export const centralStatSchema = z.object({
/** `proxy_bind_addr` in frps.ini, null when it isn't set (tunnels then open on every address). */
proxyBindAddr: z.string().max(100).nullable(),
/** `allow_ports` in frps.ini as written, e.g. `20000-29999`. */
allowPorts: z.string().max(2000).nullable(),
frpsRunning: z.boolean().nullable(),
});
export const agentStatsSchema = z.object({
schema: z.literal("crosscode.agent-stats/1"),
at: dateTimeSchema,
ram: z.object({ totalBytes: bytes, usedBytes: bytes, availableBytes: bytes, swapTotalBytes: bytes, swapUsedBytes: bytes }),
cpu: z.object({ percent: z.number().min(0).max(100), load1: z.number().nonnegative(), load5: z.number().nonnegative(), load15: z.number().nonnegative(), cores: z.int().positive() }),
disks: z.array(z.object({ mount: z.string().max(255), totalBytes: bytes, usedBytes: bytes, freeBytes: bytes })).max(30),
uptimeSeconds: z.number().nonnegative(),
versions: z.object({ os: z.string().max(200), node: z.string().max(40), postgres: z.string().max(40).nullable(), frpc: z.string().max(40).nullable(), agent: z.string().max(40) }),
frpcRunning: z.boolean().nullable(),
processes: z.array(processStatSchema).max(200),
central: centralStatSchema.nullable().optional(),
});
export type AgentStats = z.infer<typeof agentStatsSchema>;
// ---------------------------------------------------------------------------------------------
// Agent API (/api/agent/v1/*)
// ---------------------------------------------------------------------------------------------
const base64urlKey = z.string().regex(/^[A-Za-z0-9_-]{43}$/, "Expected a raw 32-byte key, base64url");
/** POST /api/agent/v1/join (unsigned: the one-time token is the proof). */
export const joinRequestSchema = z.object({
agentApiVersion: z.literal(AGENT_API_VERSION),
token: z.string().regex(/^[A-Za-z0-9_-]{32,128}$/),
mode: agentModeSchema,
hostname: z.string().max(253),
agentVersion: z.string().max(40),
/** The agent's Ed25519 key (signs its requests) and X25519 key (receives config). */
publicKeys: z.object({ ed25519: base64urlKey, x25519: base64urlKey }),
});
export const joinResponseSchema = z.object({
serverId: z.string().regex(/^srv_[A-Za-z0-9_-]{8,64}$/),
/** The Hub's job-signing Ed25519 key, pinned by the agent. */
hubJobKey: base64urlKey,
pollSeconds: z.int().min(5).max(600),
statsSeconds: z.int().min(15).max(3600),
});
/** POST /api/agent/v1/poll (signed). */
export const pollRequestSchema = z.object({
agentApiVersion: z.literal(AGENT_API_VERSION),
agentVersion: z.string().max(40),
mode: agentModeSchema,
/** The job being run now, if any (the Hub won't hand out another). */
runningJobId: z.string().regex(/^job_[A-Za-z0-9_-]{8,64}$/).nullable(),
});
export const pollResponseSchema = z.object({
jobs: z.array(z.unknown()).max(10), // each checked with agentJobSchema and the signature by the agent
pollSeconds: z.int().min(5).max(600),
});
/** POST /api/agent/v1/stats (signed): an {@link agentStatsSchema} body. → 204 */
/** POST /api/agent/v1/report (signed): an {@link agentReportSchema} body. → 204 */
/** PUT /api/agent/v1/exports/:exportId (signed): the encrypted ZIP, `application/octet-stream`. → 204 */
// ---------------------------------------------------------------------------------------------
// Env template (crosscode.env-template/1): apps/<product>/env.template.json
// ---------------------------------------------------------------------------------------------
export const ENV_LAYERS = ["server", "defaults", "install", "generated"] as const;
export const envLayerSchema = z.enum(ENV_LAYERS);
export const envVariableSchema = z.object({
name: z.string().regex(/^[A-Z][A-Z0-9_]*$/),
layer: envLayerSchema,
required: z.boolean().default(false),
/** Used when no layer gives a value. */
default: z.string().max(4000).optional(),
/** Built from the install's values: `https://{domain}`, `{apiPort}`. */
from: z.string().max(400).optional(),
description: z.string().max(400).optional(),
});
export type EnvVariable = z.infer<typeof envVariableSchema>;
export const envTemplateSchema = z.object({
schema: z.literal("crosscode.env-template/1"),
product: productSchema,
/** Relative to the product's folder (`apps/<product>`): `backend/.env`, `frontend/.env.production`. */
files: z.record(z.string().regex(/^[a-z0-9._/-]+$/), z.array(envVariableSchema).min(1)),
});
export type EnvTemplate = z.infer<typeof envTemplateSchema>;
/** The variables a layer must provide for a template, that it doesn't (the Hub's check before a job). */
export function missingRequired(template: EnvTemplate, layer: "server" | "defaults", values: Record<string, string | undefined>): string[] {
const missing: string[] = [];
for (const vars of Object.values(template.files)) {
for (const v of vars) {
if (v.layer !== layer || !v.required || v.default !== undefined) continue;
if (!values[v.name] || !String(values[v.name]).trim()) missing.push(v.name);
}
}
return [...new Set(missing)];
}
+24
View File
@@ -0,0 +1,24 @@
/**
* Canonical JSON for signatures: object keys sorted (by UTF-16 code units, as JavaScript sorts),
* no whitespace, `undefined` properties left out. Arrays keep their order. Only plain JSON values
* (no Dates, Maps or functions: convert them first).
*/
export function canonicalJson(value: unknown): string {
if (value === null || typeof value === "boolean" || typeof value === "string") return JSON.stringify(value);
if (typeof value === "number") {
if (!Number.isFinite(value)) throw new Error("canonicalJson: numbers must be finite");
return JSON.stringify(value);
}
if (Array.isArray(value)) return `[${value.map((v) => (v === undefined ? "null" : canonicalJson(v))).join(",")}]`;
if (typeof value === "object") {
const proto = Object.getPrototypeOf(value) as unknown;
if (proto !== Object.prototype && proto !== null) throw new Error("canonicalJson: only plain objects");
const obj = value as Record<string, unknown>;
const entries = Object.keys(obj)
.filter((k) => obj[k] !== undefined)
.sort()
.map((k) => `${JSON.stringify(k)}:${canonicalJson(obj[k])}`);
return `{${entries.join(",")}}`;
}
throw new Error(`canonicalJson: can't encode a ${typeof value}`);
}
+85
View File
@@ -0,0 +1,85 @@
import { z } from "zod";
import { dateTimeSchema, productSchema, timezoneSchema } from "./common";
/**
* crosscode.church-export/1: a church's data when it leaves (docs/AGENT-1.0-PLAN.md §9). Written
* for people, in plain terms with our own field names: no database tables, Prisma models or
* internal ids. Records point at each other with references made for the export (`p12`, `e3`,
* `r40`), which mean nothing outside the file.
*
* Shipped as a ZIP: `export.json` (this), `uploads/` (the files in `files`) and `README.txt`.
*/
const ref = (prefix: string) => z.string().regex(new RegExp(`^${prefix}[0-9]+$`));
const text = (max = 20000) => z.string().max(max);
const when = dateTimeSchema.nullable();
const cents = z.int();
export const exportPersonSchema = z.object({
ref: ref("p"),
name: text(300),
email: text(254).nullable(),
phone: text(40).nullable(),
role: text(40),
createdAt: when,
consent: z.object({ email: z.boolean().nullable(), whatsapp: z.boolean().nullable(), privacyNoticeAcceptedAt: when }),
});
export const exportEventSchema = z.object({
ref: ref("e"),
title: text(500),
description: text().nullable(),
startsAt: when,
endsAt: when,
location: text(500).nullable(),
sessions: z.array(z.object({ label: text(300).nullable(), startsAt: when, endsAt: when })),
ticketTypes: z.array(z.object({ ref: ref("t"), name: text(300), priceCents: cents, currency: z.string().length(3) })),
capacity: z.int().nullable(),
published: z.boolean(),
});
export const exportRegistrationSchema = z.object({
ref: ref("r"),
eventRef: ref("e"),
personRef: ref("p").nullable(),
status: text(40),
createdAt: when,
attendees: z.array(z.object({ name: text(300), email: text(254).nullable(), ticketTypeRef: ref("t").nullable(), checkedInAt: when })),
answers: z.array(z.object({ question: text(500), answer: text(5000) })).default([]),
});
export const exportPaymentSchema = z.object({
registrationRef: ref("r").nullable(),
amountCents: cents,
currency: z.string().length(3),
method: text(40),
status: text(40),
paidAt: when,
reference: text(200).nullable(),
});
export const exportCheckInSchema = z.object({ eventRef: ref("e"), personRef: ref("p").nullable(), name: text(300).nullable(), at: when });
export const exportRosterSchema = z.object({ eventRef: ref("e").nullable(), role: text(200), personRef: ref("p").nullable(), startsAt: when, endsAt: when });
export const churchExportSchema = z.object({
schema: z.literal("crosscode.church-export/1"),
exportedAt: dateTimeSchema,
product: productSchema,
productVersion: z.string().max(40),
church: z.object({ name: text(300), timezone: timezoneSchema.nullable() }),
people: z.array(exportPersonSchema),
events: z.array(exportEventSchema),
registrations: z.array(exportRegistrationSchema),
payments: z.array(exportPaymentSchema),
checkIns: z.array(exportCheckInSchema),
rosters: z.array(exportRosterSchema),
/** The site's own settings that matter to a church (name, contact, branding), never secrets. */
settings: z.record(z.string().max(100), z.string().max(5000)),
/** Uploaded files in the ZIP's `uploads/` folder. */
files: z.array(z.object({ path: z.string().regex(/^uploads\/[A-Za-z0-9._/-]+$/), bytes: z.int().nonnegative() })),
});
export type ChurchExport = z.infer<typeof churchExportSchema>;
/** Keys that look like internal ids anywhere in an export (the export test fails on any). */
export const INTERNAL_ID_KEYS = /^(id|.*Id|uuid|_id)$/;
+3
View File
@@ -9,3 +9,6 @@ export * from "./church";
export * from "./identity";
export * from "./return-to";
export * as hub from "./hub";
export * as agent from "./agent";
export * from "./church-export";
export { canonicalJson } from "./canonical";
+161 -1
View File
@@ -1,4 +1,22 @@
import { createHmac, timingSafeEqual } from "node:crypto";
import {
createCipheriv,
createDecipheriv,
createHmac,
createPrivateKey,
createPublicKey,
diffieHellman,
generateKeyPairSync,
hkdfSync,
randomBytes,
sign as edSign,
timingSafeEqual,
verify as edVerify,
type KeyObject,
} from "node:crypto";
import { AGENT_HEADERS, type Sealed } from "./agent";
import { canonicalJson } from "./canonical";
export { canonicalJson } from "./canonical";
/**
* Signed requests between Cross Code installs (Node only: `@crosscode/contracts/signing`).
@@ -74,3 +92,145 @@ export function verify(opts: { secret: string; timestamp: unknown; signature: un
const given = Buffer.from(signature, "hex");
return timingSafeEqual(expected, given) ? { ok: true } : { ok: false, code: "BAD_SIGNATURE" };
}
// ---------------------------------------------------------------------------------------------
// Server agent (docs/AGENT-1.0-PLAN.md §3–4): Ed25519 signatures, sealed config, export files.
// Keys travel as raw 32-byte base64url strings; private keys stay on their machine as PKCS#8 PEM.
// ---------------------------------------------------------------------------------------------
const b64u = (buf: Uint8Array): string => Buffer.from(buf).toString("base64url");
const unb64u = (s: string): Buffer => Buffer.from(s, "base64url");
const bytesOf = (data: RawBody): Buffer => (typeof data === "string" ? Buffer.from(data) : Buffer.from(data));
export interface KeyPair {
/** Raw 32-byte public key, base64url. */
publicKey: string;
/** PKCS#8 PEM. Never leaves the machine that made it. */
privateKeyPem: string;
}
function rawPublic(key: KeyObject): string {
const jwk = key.export({ format: "jwk" });
if (!jwk.x) throw new Error("Not an OKP key");
return jwk.x;
}
export function generateEd25519(): KeyPair {
const { publicKey, privateKey } = generateKeyPairSync("ed25519");
return { publicKey: rawPublic(publicKey), privateKeyPem: privateKey.export({ format: "pem", type: "pkcs8" }).toString() };
}
export function generateX25519(): KeyPair {
const { publicKey, privateKey } = generateKeyPairSync("x25519");
return { publicKey: rawPublic(publicKey), privateKeyPem: privateKey.export({ format: "pem", type: "pkcs8" }).toString() };
}
/** The raw public key of a PKCS#8 private key (to check a configured key pair matches). */
export function publicKeyOf(privateKeyPem: string): string {
return rawPublic(createPublicKey(createPrivateKey(privateKeyPem)));
}
const okpPublic = (crv: "Ed25519" | "X25519", x: string): KeyObject => createPublicKey({ key: { kty: "OKP", crv, x }, format: "jwk" });
/** Ed25519 signature of some bytes, base64url (86 characters). */
export function ed25519Sign(privateKeyPem: string, data: RawBody): string {
return b64u(edSign(null, bytesOf(data), createPrivateKey(privateKeyPem)));
}
export function ed25519Verify(publicKey: string, data: RawBody, signature: unknown): boolean {
if (typeof signature !== "string" || !/^[A-Za-z0-9_-]{86}$/.test(signature)) return false;
try {
return edVerify(null, bytesOf(data), okpPublic("Ed25519", publicKey), unb64u(signature));
} catch {
return false;
}
}
/** The bytes a job's signature covers: the job without `signature`, as canonical JSON. */
export function jobSigningInput(job: Record<string, unknown>): string {
const rest = { ...job };
delete rest.signature;
return canonicalJson(rest);
}
/** The Hub signs a job (with its job-signing key, not the licence key). */
export function signJob<T extends Record<string, unknown>>(job: T, hubPrivateKeyPem: string): T & { signature: string } {
return { ...job, signature: ed25519Sign(hubPrivateKeyPem, jobSigningInput(job)) };
}
/** The agent checks a job against the Hub key it pinned when it joined. */
export function verifyJob(job: Record<string, unknown>, hubPublicKey: string): boolean {
return ed25519Verify(hubPublicKey, jobSigningInput(job), job.signature);
}
/** Headers for an agent → Hub request. `now` is for tests. */
export function signAgentRequest(opts: { serverId: string; privateKeyPem: string; rawBody: RawBody; now?: Date }): Record<string, string> {
const timestamp = (opts.now ?? new Date()).toISOString();
return {
[AGENT_HEADERS.server]: opts.serverId,
[AGENT_HEADERS.timestamp]: timestamp,
[AGENT_HEADERS.signature]: ed25519Sign(opts.privateKeyPem, Buffer.concat([Buffer.from(`${timestamp}.`), bytesOf(opts.rawBody)])),
};
}
/** The Hub checks an agent request against the key the server joined with. */
export function verifyAgentRequest(opts: { publicKey: string; timestamp: unknown; signature: unknown; rawBody: RawBody; now?: Date; maxSkewMs?: number }): VerifyResult {
if (typeof opts.timestamp !== "string") return { ok: false, code: "BAD_SIGNATURE" };
const sentAt = Date.parse(opts.timestamp);
const now = (opts.now ?? new Date()).getTime();
if (!Number.isFinite(sentAt) || Math.abs(now - sentAt) > (opts.maxSkewMs ?? MAX_CLOCK_SKEW_MS)) return { ok: false, code: "STALE_REQUEST" };
return ed25519Verify(opts.publicKey, Buffer.concat([Buffer.from(`${opts.timestamp}.`), bytesOf(opts.rawBody)]), opts.signature)
? { ok: true }
: { ok: false, code: "BAD_SIGNATURE" };
}
const SEAL_INFO = Buffer.from("crosscode.agent-config/1");
function sealKey(shared: Buffer, epk: string, recipient: string): Buffer {
return Buffer.from(hkdfSync("sha256", shared, Buffer.concat([unb64u(epk), unb64u(recipient)]), SEAL_INFO, 32));
}
/** Seal a JSON value to a server's X25519 key: only that server can open it. */
export function sealConfig(value: unknown, recipientX25519: string): Sealed {
const eph = generateKeyPairSync("x25519");
const epk = rawPublic(eph.publicKey);
const shared = diffieHellman({ privateKey: eph.privateKey, publicKey: okpPublic("X25519", recipientX25519) });
const key = sealKey(shared, epk, recipientX25519);
const iv = randomBytes(12);
const cipher = createCipheriv("aes-256-gcm", key, iv);
const data = Buffer.concat([cipher.update(JSON.stringify(value), "utf8"), cipher.final(), cipher.getAuthTag()]);
return { alg: "x25519-hkdf-sha256-aes-256-gcm", epk, iv: b64u(iv), ciphertext: b64u(data) };
}
/** Open a sealed value with the server's X25519 private key. Throws if it was changed or isn't for this key. */
export function openConfig(sealed: Sealed, x25519PrivateKeyPem: string): unknown {
const priv = createPrivateKey(x25519PrivateKeyPem);
const recipient = rawPublic(createPublicKey(priv));
const shared = diffieHellman({ privateKey: priv, publicKey: okpPublic("X25519", sealed.epk) });
const key = sealKey(shared, sealed.epk, recipient);
const data = unb64u(sealed.ciphertext);
if (data.length < 17) throw new Error("Sealed data is too short");
const decipher = createDecipheriv("aes-256-gcm", key, unb64u(sealed.iv));
decipher.setAuthTag(data.subarray(data.length - 16));
const plain = Buffer.concat([decipher.update(data.subarray(0, data.length - 16)), decipher.final()]);
return JSON.parse(plain.toString("utf8")) as unknown;
}
/**
* An export ZIP encrypted with a fresh random key (§9): `iv (12) ‖ ciphertext ‖ tag (16)`,
* AES-256-GCM, which the browser decrypts with WebCrypto. The key only goes in the link's `#…`.
*/
export function encryptExport(zip: Uint8Array): { key: string; file: Buffer } {
const key = randomBytes(32);
const iv = randomBytes(12);
const cipher = createCipheriv("aes-256-gcm", key, iv);
const file = Buffer.concat([iv, cipher.update(zip), cipher.final(), cipher.getAuthTag()]);
return { key: b64u(key), file };
}
export function decryptExport(file: Uint8Array, key: string): Buffer {
const buf = Buffer.from(file);
const decipher = createDecipheriv("aes-256-gcm", unb64u(key), buf.subarray(0, 12));
decipher.setAuthTag(buf.subarray(buf.length - 16));
return Buffer.concat([decipher.update(buf.subarray(12, buf.length - 16)), decipher.final()]);
}
+174
View File
@@ -0,0 +1,174 @@
import { agent, canonicalJson, churchExportSchema, INTERNAL_ID_KEYS } from "../src";
import {
decryptExport,
encryptExport,
generateEd25519,
generateX25519,
openConfig,
publicKeyOf,
sealConfig,
signAgentRequest,
signJob,
verifyAgentRequest,
verifyJob,
} from "../src/signing";
import { sample } from "./helpers";
const hubKeys = generateEd25519();
const serverKeys = { ed: generateEd25519(), x: generateX25519() };
function unsignedJob(over: Record<string, unknown> = {}) {
return {
schema: "crosscode.agent-job/1",
id: "job_abcdefgh12",
serverId: "srv_abcdefgh12",
type: "deploy",
args: { church: "hope", products: ["events"], versions: ["events@2.4.1"] },
config: null,
notBefore: "2026-11-20T12:00:00.000Z",
expiresAt: "2026-11-20T12:15:00.000Z",
...over,
};
}
describe("canonical JSON", () => {
it("sorts keys at every level and drops undefined", () => {
expect(canonicalJson({ b: 1, a: { d: [3, { z: 1, y: 2 }], c: undefined } })).toBe('{"a":{"d":[3,{"y":2,"z":1}]},"b":1}');
});
it("refuses values that don't round-trip", () => {
expect(() => canonicalJson({ a: new Date() })).toThrow();
expect(() => canonicalJson(Number.NaN)).toThrow();
});
});
describe("jobs", () => {
it("a signed job parses and verifies with the pinned Hub key", () => {
const job = signJob(unsignedJob(), hubKeys.privateKeyPem);
expect(agent.agentJobSchema.safeParse(job).success).toBe(true);
expect(verifyJob(job, hubKeys.publicKey)).toBe(true);
});
it("key order doesn't matter, but any change to the content does", () => {
const job = signJob(unsignedJob(), hubKeys.privateKeyPem);
const reordered = Object.fromEntries(Object.entries(job).reverse());
expect(verifyJob(reordered, hubKeys.publicKey)).toBe(true);
expect(verifyJob({ ...job, args: { ...job.args, church: "other" } }, hubKeys.publicKey)).toBe(false);
expect(verifyJob({ ...job, expiresAt: "2026-12-20T12:15:00.000Z" }, hubKeys.publicKey)).toBe(false);
});
it("a job signed by another key is refused", () => {
const forged = signJob(unsignedJob(), generateEd25519().privateKeyPem);
expect(verifyJob(forged, hubKeys.publicKey)).toBe(false);
expect(verifyJob({ ...unsignedJob(), signature: "x".repeat(86) }, hubKeys.publicKey)).toBe(false);
});
it.each([
["shell characters in a name", { args: { church: "hope; rm -rf /", products: ["events"] } }],
["an unknown product", { args: { church: "hope", products: ["shop"] } }],
["a version that isn't a release tag", { args: { church: "hope", products: ["events"], versions: ["main"] } }],
["an unknown argument", { args: { church: "hope", products: ["events"], command: "id" } }],
["an unknown type", { type: "run", args: { command: "id" } }],
])("refuses %s", (_label, over) => {
const job = signJob(unsignedJob(over), hubKeys.privateKeyPem);
expect(agent.agentJobSchema.safeParse(job).success).toBe(false);
});
it("adopt only takes a plain absolute folder", () => {
const ok = signJob(unsignedJob({ type: "adopt", args: { church: "hope", eventsDir: "/var/www/hope-events" } }), hubKeys.privateKeyPem);
expect(agent.agentJobSchema.safeParse(ok).success).toBe(true);
for (const eventsDir of ["/var/www/../etc", "relative/dir", "/var/www/a b", "/var/www/$(id)"]) {
const bad = signJob(unsignedJob({ type: "adopt", args: { church: "hope", eventsDir } }), hubKeys.privateKeyPem);
expect(agent.agentJobSchema.safeParse(bad).success).toBe(false);
}
});
it("install needs different web and api ports", () => {
const args = { church: "hope", products: ["events"], domains: { events: "events.hope.co.za" }, ports: { events: { web: 20010, api: 20010 } } };
expect(agent.jobArgsSchemas.install.safeParse(args).success).toBe(false);
expect(agent.jobArgsSchemas.install.safeParse({ ...args, ports: { events: { web: 20010, api: 20011 } } }).success).toBe(true);
});
});
describe("agent requests", () => {
const body = JSON.stringify({ agentApiVersion: 1, agentVersion: "1.0.0", mode: "church", runningJobId: null });
const now = new Date("2026-11-20T12:00:00Z");
it("verify with the server's key, within five minutes", () => {
const h = signAgentRequest({ serverId: "srv_abcdefgh12", privateKeyPem: serverKeys.ed.privateKeyPem, rawBody: body, now });
const check = (over: Partial<Parameters<typeof verifyAgentRequest>[0]>) =>
verifyAgentRequest({ publicKey: serverKeys.ed.publicKey, timestamp: h[agent.AGENT_HEADERS.timestamp], signature: h[agent.AGENT_HEADERS.signature], rawBody: body, now, ...over });
expect(check({})).toEqual({ ok: true });
expect(check({ rawBody: `${body} ` })).toEqual({ ok: false, code: "BAD_SIGNATURE" });
expect(check({ publicKey: generateEd25519().publicKey })).toEqual({ ok: false, code: "BAD_SIGNATURE" });
expect(check({ now: new Date(now.getTime() + 6 * 60 * 1000) })).toEqual({ ok: false, code: "STALE_REQUEST" });
});
});
describe("sealed config", () => {
const config = { server: { TRUST_PROXY: "loopback", DB_HOST: "10.0.0.5" }, defaults: { HUB_URL: "https://hub.crosscode.co.za" }, install: { events: { FRONTEND_URL: "https://events.hope.co.za" } } };
it("only the server's key opens it", () => {
const sealed = sealConfig(config, serverKeys.x.publicKey);
expect(agent.sealedSchema.safeParse(sealed).success).toBe(true);
expect(JSON.stringify(sealed)).not.toContain("loopback");
expect(openConfig(sealed, serverKeys.x.privateKeyPem)).toEqual(config);
expect(() => openConfig(sealed, generateX25519().privateKeyPem)).toThrow();
});
it("a changed ciphertext is refused", () => {
const sealed = sealConfig(config, serverKeys.x.publicKey);
const bytes = Buffer.from(sealed.ciphertext, "base64url");
bytes[0] = (bytes[0] ?? 0) ^ 1;
expect(() => openConfig({ ...sealed, ciphertext: bytes.toString("base64url") }, serverKeys.x.privateKeyPem)).toThrow();
});
it("the opened config parses", () => {
expect(agent.jobConfigSchema.safeParse(config).success).toBe(true);
});
it("publicKeyOf matches the generated pair", () => {
expect(publicKeyOf(serverKeys.x.privateKeyPem)).toBe(serverKeys.x.publicKey);
expect(publicKeyOf(serverKeys.ed.privateKeyPem)).toBe(serverKeys.ed.publicKey);
});
});
describe("exports", () => {
it("encrypt and decrypt with the link's key", () => {
const zip = Buffer.from("PK\u0003\u0004 pretend zip");
const { key, file } = encryptExport(zip);
expect(file.includes(zip)).toBe(false);
expect(decryptExport(file, key).equals(zip)).toBe(true);
expect(() => decryptExport(file, encryptExport(zip).key)).toThrow();
});
it("the sample has no internal ids", () => {
const data = sample("church-export.json");
expect(churchExportSchema.safeParse(data).success).toBe(true);
const keys: string[] = [];
const walk = (v: unknown): void => {
if (Array.isArray(v)) v.forEach(walk);
else if (v && typeof v === "object") for (const [k, x] of Object.entries(v)) { keys.push(k); walk(x); }
};
walk(data);
expect(keys.filter((k) => INTERNAL_ID_KEYS.test(k))).toEqual([]);
});
});
describe("env templates", () => {
it("lists the required server and defaults values that are missing", () => {
const template = agent.envTemplateSchema.parse({
schema: "crosscode.env-template/1",
product: "events",
files: {
"backend/.env": [
{ name: "TRUST_PROXY", layer: "server", required: true },
{ name: "DB_HOST", layer: "server", required: true },
{ name: "BACKUP_ENABLED", layer: "defaults", required: true, default: "true" },
{ name: "HUB_URL", layer: "defaults", required: true },
],
},
});
expect(agent.missingRequired(template, "server", { TRUST_PROXY: "loopback" })).toEqual(["DB_HOST"]);
expect(agent.missingRequired(template, "defaults", {})).toEqual(["HUB_URL"]);
});
});
@@ -35,6 +35,7 @@ const SAMPLE_SCHEMA: Record<string, string> = {
"hub-heartbeat-accounts.json": "hub-heartbeat-request",
"hub-heartbeat-response.json": "hub-heartbeat-response",
"hub-licence-payload.json": "hub-licence-payload",
"church-export.json": "church-export",
};
describe("JSON Schema (schemas/)", () => {
+2
View File
@@ -2,6 +2,7 @@ import { readdirSync } from "node:fs";
import type { z } from "zod";
import {
accountsCallSchema,
churchExportSchema,
churchSchema,
eventChangedSchema,
eventFeedSchema,
@@ -34,6 +35,7 @@ const SCHEMAS: Record<string, z.ZodType> = {
"hub-heartbeat-accounts.json": hub.heartbeatRequestSchema,
"hub-heartbeat-response.json": hub.heartbeatResponseSchema,
"hub-licence-payload.json": hub.licencePayloadSchema,
"church-export.json": churchExportSchema,
};
describe("samples", () => {
+57 -6
View File
@@ -24,13 +24,30 @@ if [ -z "${CROSSCODE_HOME:-}" ]; then
CROSSCODE_HOME="$copy" CROSSCODE_FROM="$root" exec bash "$copy/crosscode" "$@"
fi
# Running from the copy: remove it afterwards (only ever a mktemp folder, never a checkout).
if [[ "$CROSSCODE_HOME" == /tmp/crosscode.* ]]; then
trap 'rm -rf "$CROSSCODE_HOME"' EXIT
fi
# Options for every command, taken out of the arguments here: --json (a crosscode.script-report/1
# on stdout at the end, the log on stderr) and --config <file> (the .env values by layer, from
# the agent).
CROSSCODE_JSON=0
CROSSCODE_CONFIG="${CROSSCODE_CONFIG:-}"
args=()
while [ $# -gt 0 ]; do
case "$1" in
--json) CROSSCODE_JSON=1; shift ;;
--config)
[ $# -ge 2 ] || { echo '!!> --config needs a file.' >&2; exit 1; }
CROSSCODE_CONFIG="$2"; shift 2 ;;
*) args+=("$1"); shift ;;
esac
done
set -- "${args[@]+"${args[@]}"}"
export CROSSCODE_CONFIG
# shellcheck source=lib/common.sh
source "$CROSSCODE_HOME/lib/common.sh"
# shellcheck source=lib/env.sh
source "$CROSSCODE_HOME/lib/env.sh"
# shellcheck source=lib/frpc.sh
source "$CROSSCODE_HOME/lib/frpc.sh"
# shellcheck source=lib/deploy.sh
source "$CROSSCODE_HOME/lib/deploy.sh"
# shellcheck source=lib/status.sh
@@ -39,6 +56,32 @@ source "$CROSSCODE_HOME/lib/status.sh"
source "$CROSSCODE_HOME/lib/install.sh"
# shellcheck source=lib/adopt.sh
source "$CROSSCODE_HOME/lib/adopt.sh"
# shellcheck source=lib/manage.sh
source "$CROSSCODE_HOME/lib/manage.sh"
# At the end: the report (with --json), and the copy this runs from is removed (only ever a
# mktemp folder, never a checkout).
on_exit() {
local rc=$?
[ "$CROSSCODE_JSON" != 1 ] || report_emit "$rc"
if [[ "$CROSSCODE_HOME" == /tmp/crosscode.* ]]; then rm -rf "$CROSSCODE_HOME"; fi
return "$rc"
}
trap on_exit EXIT
command="${1:-help}"
[ $# -gt 0 ] && shift
if [ "$CROSSCODE_JSON" = 1 ]; then
# The report goes to the real stdout (fd 3); everything else to stderr.
exec 3>&1 1>&2
report_init "$command"
fi
if [ -n "$CROSSCODE_CONFIG" ]; then
[ -f "$CROSSCODE_CONFIG" ] || die "--config: there's no file $CROSSCODE_CONFIG."
node -e 'JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"))' "$CROSSCODE_CONFIG" 2>/dev/null ||
die "--config: $CROSSCODE_CONFIG isn't JSON."
fi
usage() {
cat <<EOF
@@ -51,19 +94,26 @@ crosscode: Cross Code installs on this server ($(git -C "${CROSSCODE_FROM:-.}" r
crosscode install <church> --products events --domain <address> a new church
crosscode adopt <church> --events-dir <old folder> move an existing Cross Events install here
crosscode adopt <church> --undo go back to the old folder (within 14 days)
crosscode restart <church> [events] reload the PM2 processes
crosscode export <church> --out <file.zip> the church's data and uploads
crosscode uninstall <church> --confirm <church> remove it (a final backup is kept 30 days)
crosscode purge <church> --confirm <church> delete what uninstall kept
crosscode help [<command>] this, or a command's options
Every command takes --json (a report at the end) and --config <file> (settings from the Hub).
Churches live in $CROSSCODE_SITES/<church>/ (church.env, one checkout per product).
EOF
}
command="${1:-help}"
[ $# -gt 0 ] && shift
case "$command" in
deploy) cmd_deploy "$@" ;;
status) cmd_status "$@" ;;
install) cmd_install "$@" ;;
adopt) cmd_adopt "$@" ;;
restart) cmd_restart "$@" ;;
export) cmd_export "$@" ;;
uninstall) cmd_uninstall "$@" ;;
purge) cmd_purge "$@" ;;
add) die "\`crosscode add\` comes with the second product (phase 3)." ;;
help|-h|--help)
case "${1:-}" in
@@ -71,6 +121,7 @@ case "$command" in
status) status_usage ;;
install) install_usage ;;
adopt) adopt_usage ;;
restart|export|uninstall|purge) manage_usage ;;
*) usage ;;
esac ;;
*) usage; die "Unknown command \"$command\"." ;;
+33 -8
View File
@@ -61,6 +61,14 @@ rewrite_ecosystem() {
sed -E "s#([\"'\`])${old}(/|[\"'\`])#\\1${new}\\2#g"
}
# migrations_only_in <backend dir> <other backend dir>: the Prisma migrations the first has and
# the second doesn't, one per line, sorted.
migrations_only_in() {
comm -13 \
<(find "$2/prisma/migrations" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' 2>/dev/null | sort) \
<(find "$1/prisma/migrations" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' 2>/dev/null | sort)
}
# Ignored files in a checkout's folder (the install's own data, not in git), one per line.
ignored_files_in() { git_in "$1" ls-files --others --ignored --exclude-standard -- "$2"; }
@@ -169,11 +177,16 @@ adopt_run() {
step "Installing and building $version (the old install keeps serving)"
events_install_deps "$new" "$user"
events_build "$new" "$user"
local expected
local expected newmig
expected="$(events_expected_version "$new")"
if events_pending_migrations "$new" "$user"; then
die "$version has database migrations that aren't applied (or the database can't be read). adopt only moves the install; run it at a 2.2.x without migrations. Remove $new; nothing else changed."
# Migrations the release has and the old install doesn't (2.2.0 has one: AI tokens). They run
# in the switch-over, after the backup, while neither version is running. Anything else
# pending (or a database that can't be read) stops here, before anything changes.
newmig="$(migrations_only_in "$app/backend" "$old/backend")"
if events_pending_migrations "$new" "$user" && [ -z "$newmig" ]; then
die "The database has migrations pending that $version doesn't explain (or it can't be read). Remove $new; nothing else changed."
fi
[ -z "$newmig" ] || info "Migrations to apply in the switch-over: $(echo "$newmig" | tr '\n' ' ')"
# ── 6. Switch over: the only downtime ─────────────────────────────────────────────────────
step "Switching over"
@@ -186,6 +199,10 @@ adopt_run() {
(
set -e
run_as "$user" "$new" pm2 delete "$backend" "$frontend"
if [ -n "$newmig" ]; then
info "Migrations"
events_migrate "$new" "$user"
fi
run_as "$user" "$app" pm2 start ecosystem.config.js
run_as "$user" "$app" pm2 save
wait_for_version "$(events_health_url)" "$expected" 60
@@ -208,6 +225,9 @@ adopt_run() {
church_env_set EVENTS_OLD_DIR "$old$ADOPT_SUFFIX"
church_env_set EVENTS_OLD_REF "$oldref"
church_env_set EVENTS_ADOPTED_AT "$(date +%F)"
# The migrations adopt applied: adopt --undo may go back past these (they only add to the
# database, which the old release ignores), but not past any applied later.
[ -z "$newmig" ] || church_env_set EVENTS_ADOPT_MIGRATIONS "$(echo "$newmig" | tr '\n' ' ' | sed 's/ $//')"
church_env_write "$(church_env_path "$church")"
step "Adopted: $church runs $version from $new"
info "The old folder is $old$ADOPT_SUFFIX, kept $ADOPT_KEEP_DAYS days for \`crosscode adopt $church --undo\`."
@@ -246,11 +266,16 @@ adopt_undo() {
[ ! -e "$orig" ] || die "$orig exists again; move it away first."
step "Checks"
# The database must still suit 2.1.x: no migrations the old release doesn't have.
extra="$(comm -13 \
<(find "$oldkept/backend/prisma/migrations" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort) \
<(find "$app/backend/prisma/migrations" -mindepth 1 -maxdepth 1 -type d -printf '%f\n' | sort))"
[ -z "$extra" ] || die "$(checkout_ref "$new" events) has database migrations $oldref doesn't know ($(echo "$extra" | tr '\n' ' ')). Going back would break it; restore a backup instead."
# The database must still suit 2.1.x: no migrations the old release doesn't have, apart from
# the ones adopt itself applied (they only add to the database, which 2.1.x ignores).
local adopted m
adopted=" $(pvar events ADOPT_MIGRATIONS) "
extra=""
while IFS= read -r m; do
[ -n "$m" ] && [[ "$adopted" != *" $m "* ]] && extra+="$m "
done < <(migrations_only_in "$app/backend" "$oldkept/backend")
[ -z "$extra" ] || die "$(checkout_ref "$new" events) has database migrations $oldref doesn't know ($extra), applied after the move. Going back would break it; restore a backup instead."
[ "$adopted" = " " ] || info "The columns added by $(pvar events ADOPT_MIGRATIONS) stay in the database; $oldref ignores them."
step "Database backup"
events_backup "$new" "$user"
+31 -2
View File
@@ -14,10 +14,12 @@ CROSSCODE_NODE_MIN=20
CROSSCODE_NODE_WARN=22
# ── Logging ──────────────────────────────────────────────────────────────────────────────────
step() { printf '==> %s\n' "$*"; }
# shellcheck source=report.sh
source "$(dirname "${BASH_SOURCE[0]}")/report.sh"
step() { printf '==> %s\n' "$*"; report_line step "$(now_iso)" "$*"; }
info() { printf ' %s\n' "$*"; }
warn() { printf '!!> %s\n' "$*" >&2; }
die() { printf '!!> %s\n' "$*" >&2; exit 1; }
die() { printf '!!> %s\n' "$*" >&2; report_line message "$*"; exit 1; }
# ── Small checks ─────────────────────────────────────────────────────────────────────────────
require_root() {
@@ -64,8 +66,35 @@ upper() { printf '%s' "$1" | tr '[:lower:]-' '[:upper:]_'; }
# ── Running things as the app user ───────────────────────────────────────────────────────────
# run_as <user> <dir> <command…>: runs the command in <dir> as <user> (directly when we already
# are that user), with that user's HOME so PM2 and npm use their own folders.
# pm2_service: makes sure the pm2 daemon of user $1 runs under its own systemd unit (pm2-<user>),
# never as a child of whatever ran this script. A daemon started from a command (the agent's job,
# an SSH session) belongs to that command's service and dies with it: stopping or updating the
# agent took down every site it had installed (found in the VM test). Done once per user per run.
PM2_SERVICE_DONE=" "
pm2_service() {
local user="$1" home pid n
case "$PM2_SERVICE_DONE" in *" $user "*) return 0 ;; esac
PM2_SERVICE_DONE="$PM2_SERVICE_DONE$user "
{ command -v systemctl >/dev/null && [ -d /run/systemd/system ] && [ "$(id -u)" = 0 ]; } || return 0
home="$(getent passwd "$user" | cut -d: -f6)"
systemctl cat "pm2-$user" >/dev/null 2>&1 || env PATH="$PATH:/usr/bin" pm2 startup systemd -u "$user" --hp "$home" >/dev/null
systemctl is-active -q "pm2-$user" && return 0
pid="$(cat "$home/.pm2/pm2.pid" 2>/dev/null || true)"
if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
# A daemon outside its unit: keep its list (an empty one doesn't replace the saved list),
# stop it, and let the unit bring the list back. Its sites are down for a few seconds.
n="$(sudo -u "$user" -H -- pm2 jlist 2>/dev/null | node -e 'let s = ""; process.stdin.on("data", (d) => (s += d)).on("end", () => { try { console.log(JSON.parse(s).length); } catch { console.log(0); } })')"
if [ "${n:-0}" -gt 0 ]; then sudo -u "$user" -H -- pm2 save --force >/dev/null 2>&1 || true; fi
sudo -u "$user" -H -- pm2 kill >/dev/null 2>&1 || true
info "PM2 for $user now runs under systemd (pm2-$user)."
fi
systemctl start "pm2-$user" || warn "pm2-$user didn't start: check systemctl status pm2-$user."
}
run_as() {
local user="$1" dir="$2"
# Any PM2 command runs against the daemon in its own unit (see pm2_service).
if [ "$3" = pm2 ]; then pm2_service "$user"; fi
shift 2
if [ "$(id -un)" = "$user" ]; then
(cd -- "$dir" && "$@")
+44 -3
View File
@@ -133,7 +133,7 @@ deploy_church() {
if [ -z "$dir" ] || [ ! -d "$dir/.git" ]; then warn "$p: $dir isn't a checkout (church.env $(upper "$p")_DIR)."; failed=1; continue; fi
if [ -z "$user" ] || ! id "$user" >/dev/null 2>&1; then warn "$p: no app user \"$user\" (church.env $(upper "$p")_USER)."; failed=1; continue; fi
DIR[$p]="$dir"; USR[$p]="$user"
if ! run_as "$user" "$dir" git fetch --tags --quiet origin; then
if ! run_as "$user" "$dir" git fetch --tags --force --quiet origin; then
warn "$p: couldn't fetch from origin."; failed=1; continue
fi
PREV[$p]="$(checkout_ref "$dir" "$p")"
@@ -193,6 +193,26 @@ deploy_church() {
done
[ "$failed" = 0 ] || die "Requirements not met for $church. Nothing was changed."
# New settings in the target release's env template (AGENT-1.0-PLAN §5): a new required one
# with no value anywhere stops the deploy here, before anything is touched.
local tpl out
local -a vars
for p in "${products[@]}"; do
[ -n "${TVER[$p]}" ] || continue
tpl="$(mktemp)"
if git_in "${DIR[$p]}" show "${TARGET[$p]}:apps/$p/env.template.json" > "$tpl" 2>/dev/null; then
mapfile -t vars < <("${p}_env_vars")
if out="$(env_apply_template "$tpl" "$(pvar "$p" DIR)/apps/$p" "$p" "${TARGET[$p]}" --check "${vars[@]}" 2>&1)"; then
[ -z "$out" ] || printf '%s\n' "$out" | sed "s/^/ $p: /"
else
warn "$p ${TVER[$p]}: $out"
failed=1
fi
fi
rm -f "$tpl"
done
[ "$failed" = 0 ] || die "Settings missing for $church (set them in the Hub's server configuration or defaults). Nothing was changed."
for p in "${products[@]}"; do
if [ "${PREV[$p]}" = "${TARGET[$p]}" ]; then
info "$p: already on ${PREV[$p]}."
@@ -218,8 +238,17 @@ deploy_church() {
echo
step "Summary for $church"
for p in "${products[@]}"; do info "$(printf '%-9s %s' "$p" "${RESULT[$p]}")"; done
[ "$stopped" = 0 ] || return 1
local json="" rolled=0
for p in "${products[@]}"; do
info "$(printf '%-9s %s' "$p" "${RESULT[$p]}")"
json+="${json:+,}\"$p\":{\"from\":\"${PREV[$p]}\",\"to\":\"${TARGET[$p]}\",\"result\":\"${RESULT[$p]//\"/\'}\"}"
[[ "${RESULT[$p]}" != *"rolled back to"*"running again"* && "${RESULT[$p]}" != *"rolled back to"*"never stopped"* ]] || rolled=1
done
report_result_json products "{$json}"
if [ "$stopped" = 1 ]; then
[ "$rolled" = 0 ] || report_status rolled-back
return 1
fi
}
# deploy_product <product> <checkout> <user> <previous ref> <target ref>: sets DEPLOY_OK and
@@ -238,6 +267,12 @@ deploy_product() {
echo checkout > "$stage_file"
step "$p: checking out $target"
run_as "$user" "$dir" git checkout -q --detach "$target"
if [ -f "$dir/apps/$p/env.template.json" ]; then
step "$p: settings new in this release"
local -a vars
mapfile -t vars < <("${p}_env_vars")
env_apply "$dir/apps/$p" "$p" "$target" "${vars[@]}" | sed 's/^/ /'
fi
step "$p: installing"
"${p}_install_deps" "$dir" "$user"
step "$p: migrations"
@@ -268,6 +303,12 @@ deploy_product() {
return 0
fi
warn "$p: deploy failed. Going back to $prev (database migrations aren't undone; the backup has the database as it was)."
# A migration that failed is recorded as failed, and Prisma then refuses every later deploy
# (P3009) until it's resolved. Mark this release's failed migrations as rolled back, while its
# checkout is still in place (found in the VM test).
if declare -F "${p}_resolve_failed_migrations" >/dev/null; then
"${p}_resolve_failed_migrations" "$dir" "$user" "$prev" || warn "$p: couldn't mark the failed migration as rolled back; the next deploy will say so."
fi
set +e
(
set -e
+108
View File
@@ -0,0 +1,108 @@
# shellcheck shell=bash
# The job's config and the database (docs/AGENT-1.0-PLAN.md §5).
#
# CROSSCODE_CONFIG is a JSON file with the .env values by layer ({ server, defaults, install }),
# given with --config by the agent (it opens the sealed config from the Hub, writes it mode 600
# and removes it afterwards). By hand there's usually none, and the script's own defaults apply.
#
# Database: the server layer can name a PostgreSQL server (DB_HOST, DB_PORT) and an admin
# connection that may create databases and roles (DB_ADMIN_URL). Without DB_ADMIN_URL the local
# PostgreSQL is used through `sudo -u postgres`, as before.
CROSSCODE_CONFIG="${CROSSCODE_CONFIG:-}"
# cfg <layer> <KEY> [product]: a value from the config, "" when there's none.
cfg() {
[ -n "$CROSSCODE_CONFIG" ] && [ -f "$CROSSCODE_CONFIG" ] || return 0
node -e '
const c = JSON.parse(require("fs").readFileSync(process.argv[1], "utf8"));
const [layer, key, product] = process.argv.slice(2);
const v = layer === "install" ? ((c.install || {})[product] || {})[key] : (c[layer] || {})[key];
if (v !== undefined && v !== null) process.stdout.write(String(v));' "$CROSSCODE_CONFIG" "$1" "$2" "${3:-}"
}
# env_template <product dir> <product>: the release's template, or the copy shipped with the
# script (for releases from before templates, e.g. events@2.2.0).
env_template() {
if [ -f "$1/env.template.json" ]; then echo "$1/env.template.json"; else echo "$CROSSCODE_HOME/products/$2.env.template.json"; fi
}
# env_apply <product dir> <product> <release> [envtpl options…]: writes or appends the .env files.
# Generated values come in as CROSSCODE_GEN_<NAME> environment variables.
env_apply() {
local app="$1" product="$2" release="$3"
shift 3
local -a conf=()
[ -n "$CROSSCODE_CONFIG" ] && conf=(--config "$CROSSCODE_CONFIG")
node "$CROSSCODE_HOME/lib/envtpl.cjs" --template "$(env_template "$app" "$product")" --dir "$app" \
--product "$product" --release "$release" "${conf[@]+"${conf[@]}"}" "$@"
}
# env_apply_template <template file> <product dir> <product> <release> [envtpl options…]: as
# env_apply, with a template from elsewhere (deploy checks the target release's before checkout).
env_apply_template() {
local tpl="$1" app="$2" product="$3" release="$4"
shift 4
local -a conf=()
[ -n "$CROSSCODE_CONFIG" ] && conf=(--config "$CROSSCODE_CONFIG")
node "$CROSSCODE_HOME/lib/envtpl.cjs" --template "$tpl" --dir "$app" \
--product "$product" --release "$release" "${conf[@]+"${conf[@]}"}" "$@"
}
db_host() { local h; h="$(cfg server DB_HOST)"; printf '%s' "${h:-localhost}"; }
db_port() { local p; p="$(cfg server DB_PORT)"; printf '%s' "${p:-5432}"; }
# db_admin <psql args…>: psql as the database admin. SQL on stdin or with -c.
db_admin() {
local url
url="$(cfg server DB_ADMIN_URL)"
if [ -z "$url" ]; then
sudo -u postgres psql -v ON_ERROR_STOP=1 -q "$@"
return
fi
# The URL goes in as PG* variables, so the password isn't in the process list.
local -a pg
# shellcheck disable=SC2016 # JavaScript, not shell
mapfile -t pg < <(node -e '
const u = new URL(process.argv[1]);
if (!/^postgres(ql)?:$/.test(u.protocol)) process.exit(2);
const out = { PGHOST: u.hostname, PGPORT: u.port || "5432", PGUSER: decodeURIComponent(u.username), PGPASSWORD: decodeURIComponent(u.password), PGDATABASE: decodeURIComponent(u.pathname.slice(1)) || "postgres" };
const ssl = u.searchParams.get("sslmode"); if (ssl) out.PGSSLMODE = ssl;
for (const [k, v] of Object.entries(out)) console.log(`${k}=${v}`);' "$url") || die "DB_ADMIN_URL in the server's configuration isn't a postgresql:// URL."
env "${pg[@]}" psql -v ON_ERROR_STOP=1 -q "$@"
}
# db_create <name> <password>: the role and database (both named <name>), or a new password for
# an existing role. Names are checked by the caller (a-z, 0-9, _).
db_create() {
local db="$1" pass="$2"
[[ "$db" =~ ^[a-z][a-z0-9_]{0,62}$ ]] || die "Bad database name $db."
db_admin <<SQL
DO \$\$ BEGIN
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = '$db') THEN
CREATE ROLE "$db" LOGIN PASSWORD '$pass';
ELSE
ALTER ROLE "$db" WITH LOGIN PASSWORD '$pass';
END IF;
END \$\$;
SQL
db_admin -tAc "SELECT 1 FROM pg_database WHERE datname = '$db'" | grep -q 1 ||
db_admin -c "CREATE DATABASE \"$db\" OWNER \"$db\""
}
# db_exists <name>: a database or a role with that name exists.
db_exists() {
local n="$1" out
[[ "$n" =~ ^[a-z][a-z0-9_]{0,62}$ ]] || die "Bad database name $n."
out="$(db_admin -tAc "SELECT 1 FROM pg_roles WHERE rolname = '$n' UNION ALL SELECT 1 FROM pg_database WHERE datname = '$n'")" ||
die "Couldn't reach PostgreSQL on $(db_host) to check for $n."
[ -n "$out" ]
}
# db_drop <name>: the database and its role (after a backup by the caller).
db_drop() {
local db="$1"
[[ "$db" =~ ^[a-z][a-z0-9_]{0,62}$ ]] || die "Bad database name $db."
db_admin -c "DROP DATABASE IF EXISTS \"$db\" WITH (FORCE)"
db_admin -c "DROP ROLE IF EXISTS \"$db\""
}
+164
View File
@@ -0,0 +1,164 @@
#!/usr/bin/env node
// Builds a product's .env files from its env template (crosscode.env-template/1,
// docs/AGENT-1.0-PLAN.md §5). Used by `crosscode install` (files don't exist yet: written whole)
// and `crosscode deploy` (files exist: only variables they don't have are appended, with a
// comment; existing lines are never changed or removed).
//
// node envtpl.cjs --template <file> --dir <product dir> [--config <job config JSON>]
// [--product events] [--var domain=… --var apiPort=…] [--fallback NAME=value]
// [--release events@x.y.z] [--check]
//
// Values by layer: generated → environment variables CROSSCODE_GEN_<NAME> (so secrets never
// appear in arguments or files other than the .env); server / defaults → the config's layers;
// install → the config's install[product], else the variable's `from` filled with --var values.
// Then --fallback, then the template's `default`. A required variable with no value stops
// everything before any file is written (exit 3, the names on stderr).
// --check: only says what would be added. Prints names only, never values.
'use strict';
const fs = require('fs');
const path = require('path');
function parseArgs(argv) {
const out = { vars: {}, fallback: {}, check: false };
for (let i = 0; i < argv.length; i++) {
const a = argv[i];
const next = () => {
if (i + 1 >= argv.length) throw new Error(`${a} needs a value`);
return argv[++i];
};
if (a === '--template') out.template = next();
else if (a === '--dir') out.dir = next();
else if (a === '--config') out.config = next();
else if (a === '--product') out.product = next();
else if (a === '--release') out.release = next();
else if (a === '--check') out.check = true;
else if (a === '--var' || a === '--fallback') {
const v = next();
const eq = v.indexOf('=');
if (eq < 1) throw new Error(`${a} ${v}: expected NAME=value`);
out[a === '--var' ? 'vars' : 'fallback'][v.slice(0, eq)] = v.slice(eq + 1);
} else throw new Error(`Unknown option ${a}`);
}
if (!out.template || !out.dir) throw new Error('--template and --dir are required');
return out;
}
/** KEY names a .env file already sets (comments and blank lines ignored, `export KEY=` allowed). */
function keysIn(text) {
const keys = new Set();
for (const line of text.split(/\r?\n/)) {
const m = /^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=/.exec(line);
if (m) keys.add(m[1]);
}
return keys;
}
function fill(from, vars) {
let missing = false;
const value = from.replace(/\{([A-Za-z]+)\}/g, (_, k) => {
if (vars[k] === undefined || vars[k] === '') missing = true;
return vars[k] ?? '';
});
return missing ? undefined : value;
}
/** A value as a .env line: bare when safe, otherwise double-quoted (dotenv's rules). */
function line(name, value) {
if (/[\r\n]/.test(value)) throw new Error(`${name}: values can't contain newlines`);
return /^[A-Za-z0-9_./:@,+=%-]*$/.test(value) ? `${name}=${value}` : `${name}="${value.replace(/\\/g, '\\\\').replace(/"/g, '\\"')}"`;
}
function resolve(variable, ctx) {
let value;
const nonEmpty = v => (v === undefined || v === null || String(v) === '' ? undefined : String(v));
switch (variable.layer) {
case 'generated': value = nonEmpty(process.env[`CROSSCODE_GEN_${variable.name}`]); break;
case 'server': value = nonEmpty(ctx.config.server?.[variable.name]); break;
case 'defaults': value = nonEmpty(ctx.config.defaults?.[variable.name]); break;
case 'install':
value = nonEmpty(ctx.config.install?.[ctx.product]?.[variable.name]);
if (value === undefined && variable.from) value = nonEmpty(fill(variable.from, ctx.vars));
break;
default: throw new Error(`${variable.name}: unknown layer ${variable.layer}`);
}
if (value === undefined) value = nonEmpty(ctx.fallback[variable.name]);
if (value === undefined) value = nonEmpty(variable.default);
return value;
}
function plan(opts) {
const template = JSON.parse(fs.readFileSync(opts.template, 'utf8'));
if (template.schema !== 'crosscode.env-template/1') throw new Error(`${opts.template} isn't a crosscode.env-template/1`);
const config = opts.config ? JSON.parse(fs.readFileSync(opts.config, 'utf8')) : {};
const ctx = { config, vars: opts.vars, fallback: opts.fallback, product: opts.product || template.product };
const files = [];
const missing = [];
for (const [rel, variables] of Object.entries(template.files)) {
if (rel.includes('..') || path.isAbsolute(rel)) throw new Error(`Bad file name in the template: ${rel}`);
const file = path.join(opts.dir, rel);
const exists = fs.existsSync(file);
const have = exists ? keysIn(fs.readFileSync(file, 'utf8')) : new Set();
const add = [];
for (const v of variables) {
if (have.has(v.name)) continue;
const value = resolve(v, ctx);
if (value === undefined) {
if (v.required) missing.push(`${v.name} (${v.layer}, ${rel})`);
continue;
}
add.push({ name: v.name, value });
}
files.push({ rel, file, exists, add });
}
return { files, missing };
}
function write({ files }, release) {
const date = new Date().toISOString().slice(0, 10);
for (const f of files) {
if (!f.add.length) continue;
const lines = f.add.map(a => line(a.name, a.value));
let text;
if (!f.exists) {
text = `# Written by crosscode from env.template.json${release ? ` (${release})` : ''}, ${date}.\n${lines.join('\n')}\n`;
} else {
const current = fs.readFileSync(f.file, 'utf8');
text = `${current}${current === '' || current.endsWith('\n') ? '' : '\n'}\n# Added by crosscode ${date}${release ? ` (${release})` : ''}\n${lines.join('\n')}\n`;
}
fs.mkdirSync(path.dirname(f.file), { recursive: true });
const tmp = `${f.file}.crosscode-${process.pid}`;
fs.writeFileSync(tmp, text, { mode: 0o600 });
if (f.exists) {
const st = fs.statSync(f.file);
try { fs.chownSync(tmp, st.uid, st.gid); } catch { /* not root: same owner anyway */ }
fs.chmodSync(tmp, st.mode & 0o777);
}
fs.renameSync(tmp, f.file);
}
}
function main() {
let opts;
try {
opts = parseArgs(process.argv.slice(2));
} catch (e) {
console.error(`envtpl: ${e.message}`);
process.exit(2);
}
const result = plan(opts);
if (result.missing.length) {
console.error(`Required settings with no value: ${result.missing.join(', ')}`);
process.exit(3);
}
for (const f of result.files) {
if (!f.add.length) continue;
const names = f.add.map(a => a.name).join(', ');
console.log(`${f.rel}: ${opts.check ? 'would add' : f.exists ? 'added' : 'wrote'} ${names}`);
}
if (!opts.check) write(result, opts.release);
}
if (require.main === module) main();
module.exports = { parseArgs, keysIn, fill, line, plan, write };
+147
View File
@@ -0,0 +1,147 @@
# shellcheck shell=bash
# frpc tunnels for church installs (docs/AGENT-1.0-PLAN.md §6). The script only ever appends
# marked blocks to frpc.ini and removes its own blocks again; nothing outside a block is changed,
# except two admin lines added once to [common] so `frpc reload` can pick up new tunnels without
# dropping the others. Every change: a backup, a temporary file, `frpc verify`, then the move.
FRPC_INI="${FRPC_INI:-/etc/frp/frpc.ini}"
FRPC_BACKUPS="${FRPC_BACKUPS:-/etc/frp/backups}"
FRPC_BIN="${FRPC_BIN:-frpc}"
frpc_begin() { printf '# >>> crosscode:%s:%s' "$1" "$2"; }
frpc_end() { printf '# <<< crosscode:%s:%s' "$1" "$2"; }
# frpc_block <church> <product> <web port> <api port>: the block (no leading blank line).
frpc_block() {
local church="$1" product="$2" web="$3" api="$4" name
name="$church-$product"
cat <<EOF
$(frpc_begin "$church" "$product") (added by crosscode $(date +%F), do not edit by hand)
[$name-web]
type = tcp
local_ip = 127.0.0.1
local_port = $web
remote_port = $web
[$name-api]
type = tcp
local_ip = 127.0.0.1
local_port = $api
remote_port = $api
$(frpc_end "$church" "$product")
EOF
}
frpc_check_format() {
[ -f "$FRPC_INI" ] || die "No $FRPC_INI: is frpc installed and set up on this server?"
if [ -f "$(dirname "$FRPC_INI")/frpc.toml" ] && ! grep -q '^\[common\]' "$FRPC_INI"; then
die "This server's frpc uses frpc.toml; crosscode only edits frpc.ini so far."
fi
}
frpc_has_block() { grep -qF "$(frpc_begin "$1" "$2")" "$FRPC_INI" 2>/dev/null; }
# frpc_ports <church> <product>: the web and api ports in its block ("web api"), "" if none.
frpc_ports() {
awk -v b="$(frpc_begin "$1" "$2")" -v e="$(frpc_end "$1" "$2")" '
index($0, b) == 1 { inb = 1; next }
index($0, e) == 1 { inb = 0 }
inb && /^\[.*-web\]/ { w = 1; a = 0 }
inb && /^\[.*-api\]/ { a = 1; w = 0 }
inb && /^remote_port/ { split($0, p, "="); gsub(/ /, "", p[2]); if (w) web = p[2]; if (a) api = p[2] }
END { if (web != "" || api != "") print web " " api }
' "$FRPC_INI" 2>/dev/null
}
# with_admin_lines <frpc.ini>: the file, with admin_addr / admin_port added to [common] if it has
# no admin_addr there (an existing one is left as it is).
with_admin_lines() {
if awk '/^\[common\]/ { c = 1; next } /^\[/ { c = 0 } c && /^[[:space:]]*admin_addr[[:space:]]*=/ { f = 1 } END { exit !f }' "$1"; then
cat "$1"
else
awk '{ print } /^\[common\][[:space:]]*$/ && !done { print "admin_addr = 127.0.0.1"; print "admin_port = 7400"; done = 1 }' "$1"
fi
}
# frpc_replace <new file>: checks and moves a new frpc.ini into place (backup first).
frpc_replace() {
local new="$1" stamp
if ! "$FRPC_BIN" verify -c "$new" >/dev/null 2>&1; then
rm -f "$new"
die "frpc verify refused the new frpc.ini, so it wasn't changed."
fi
stamp="$(date +%Y%m%d%H%M%S)"
mkdir -p "$FRPC_BACKUPS"
cp -p "$FRPC_INI" "$FRPC_BACKUPS/frpc.ini.$stamp"
chmod --reference="$FRPC_INI" "$new" 2>/dev/null || chmod 600 "$new"
mv -f "$new" "$FRPC_INI"
}
# frpc_add <church> <product> <web> <api>: appends the block (no change if it's there already).
frpc_add() {
local church="$1" product="$2" web="$3" api="$4" new name
frpc_check_format
if frpc_has_block "$church" "$product"; then
info "frpc: $church:$product is already in $FRPC_INI."
return 0
fi
name="$church-$product"
if grep -qE "^\[($name-web|$name-api)\]" "$FRPC_INI"; then
die "frpc: a tunnel named $name-web or $name-api is already in $FRPC_INI (outside crosscode's blocks)."
fi
if grep -qE "^[[:space:]]*(local_port|remote_port)[[:space:]]*=[[:space:]]*($web|$api)[[:space:]]*$" "$FRPC_INI"; then
die "frpc: port $web or $api is already used in $FRPC_INI."
fi
new="$(mktemp "$(dirname "$FRPC_INI")/.frpc.ini.XXXXXX")"
with_admin_lines "$FRPC_INI" > "$new"
# The admin lines were added just now: the running frpc only reads them when it starts.
grep -q "^admin_addr" "$FRPC_INI" || FRPC_NEEDS_RESTART=1
# Keep the file's last line intact: only add a newline if it had none.
[ -z "$(tail -c 1 "$new")" ] || echo >> "$new"
{ echo; frpc_block "$church" "$product" "$web" "$api"; } >> "$new"
frpc_replace "$new"
info "frpc: added $name-web ($web) and $name-api ($api)."
}
# frpc_remove <church> <product>: removes its block (and the blank line added before it).
frpc_remove() {
local church="$1" product="$2" new
frpc_check_format
frpc_has_block "$church" "$product" || { info "frpc: no block for $church:$product."; return 0; }
new="$(mktemp "$(dirname "$FRPC_INI")/.frpc.ini.XXXXXX")"
# Drops the block's lines and the one blank line straight before its begin marker.
awk -v b="$(frpc_begin "$church" "$product")" -v e="$(frpc_end "$church" "$product")" '
{ lines[NR] = $0 }
END {
start = 0; stop = 0
for (i = 1; i <= NR; i++) if (index(lines[i], b) == 1) { start = i; break }
for (i = start; i <= NR; i++) if (index(lines[i], e) == 1) { stop = i; break }
if (start > 1 && lines[start - 1] == "") start--
for (i = 1; i <= NR; i++) if (i < start || i > stop) print lines[i]
}
' "$FRPC_INI" > "$new"
frpc_replace "$new"
info "frpc: removed $church-$product."
}
FRPC_NEEDS_RESTART=0
# frpc_reload: picks up the changed frpc.ini. Normally a reload (no other tunnel drops). The first
# time the admin lines were added, or if the reload fails, frpc is restarted instead: its other
# tunnels drop for a few seconds (found in the VM test: a reload can't work before the restart).
frpc_reload() {
if [ "$FRPC_NEEDS_RESTART" != 1 ] && "$FRPC_BIN" reload -c "$FRPC_INI" >/dev/null 2>&1; then
return 0
fi
if command -v systemctl >/dev/null && systemctl list-unit-files frpc.service >/dev/null 2>&1 && systemctl cat frpc >/dev/null 2>&1; then
if [ "$FRPC_NEEDS_RESTART" = 1 ]; then
warn "frpc: restarting it once so it reads the admin lines (its other tunnels drop for a few seconds; later changes reload without this)."
else
warn "frpc: reload didn't work, so restarting it (its other tunnels drop for a few seconds)."
fi
report_result frpcRestarted true
systemctl restart frpc && FRPC_NEEDS_RESTART=0 && return 0
fi
warn "frpc couldn't be reloaded or restarted: restart it by hand (systemctl restart frpc) to pick up the change."
return 1
}
+142 -79
View File
@@ -1,26 +1,34 @@
# shellcheck shell=bash
# crosscode install: a new church on this server (from hope-events' scripts/install.sh 2.1.x).
# Events only in 2.2.0; Accounts, Website and App follow as their products/*.sh are added.
# crosscode install: a new church on this server (from hope-events' scripts/install.sh 2.1.x;
# behind frp from the Hub in agent 1.0, docs/AGENT-1.0-PLAN.md §7).
# Events only so far; Accounts, Website and App follow as their products/*.sh are added.
install_usage() {
cat <<'EOF'
crosscode install <church> --products events --domain <address> [options]
A new church: system packages (Node 24, PostgreSQL, PM2, nginx, certbot) if missing, then per
product an app user, database, checkout, .env, PM2 processes, nginx site and certificate.
Writes /var/www/sites/<church>/church.env. DNS must already point here (it prints the records).
A new church: system packages if missing, then per product an app user, database, checkout,
.env files (from the release's env.template.json), PM2 processes, and either an nginx site with
a certificate (a server with its own address) or frpc tunnels (--frp, behind the central
server). Writes /var/www/sites/<church>/church.env. If a step fails, everything this run created
is removed again.
--products events the products to install (only events in this version)
--domain <address> the church's address: hopefamily.co.za or hopefamily.crosscode.co.za;
Events is served on events.<address>
--events-domain <domain> serve Events somewhere else instead
--email <email> for Let's Encrypt notices (required unless --no-ssl)
--version events@x.y.z that release (default: the latest)
--main the latest main (staging churches only)
--user <name> the Linux user the apps run as (default: cc-<church>)
--api-port <n> Events backend port (default: the first free one from 5000)
--web-port <n> Events frontend port (default: the first free one from 3000)
--frp behind frp: add tunnels to frpc.ini (ports must be given, and are
used for the tunnel too); no nginx or certificate here
--email <email> for Let's Encrypt notices (required unless --frp or --no-ssl)
--no-ssl no certificate (behind another proxy that does HTTPS)
--config <file> the .env values by layer (JSON; given by the agent)
--keep-on-failure leave what was made if a step fails (to look into it)
--json print a report (JSON) at the end; the log goes to stderr
EOF
}
@@ -44,8 +52,30 @@ free_port() {
random_secret() { openssl rand -base64 64 | tr -d '/+=\n' | head -c "${1:-48}"; }
# ── Undo: what this run made, undone in reverse if it fails ──────────────────────────────────
INSTALL_UNDO=""
undo_add() { [ -z "$INSTALL_UNDO" ] || printf '%s\n' "$*" >> "$INSTALL_UNDO"; }
install_undo() {
local kind a b
step "Removing what this install made"
tac "$INSTALL_UNDO" | while read -r kind a b; do
case "$kind" in
pm2) # shellcheck disable=SC2086 # b is a space-separated list of PM2 names
run_as "$a" / pm2 delete $b >/dev/null 2>&1 || true
run_as "$a" / pm2 save --force >/dev/null 2>&1 || true
info "PM2: $b removed" ;;
frpc) frpc_remove "$a" "$b" && frpc_reload || warn "frpc block $a:$b not removed" ;;
nginx) rm -f "/etc/nginx/sites-enabled/$a" "/etc/nginx/sites-available/$a"; systemctl reload nginx 2>/dev/null || true; info "nginx: $a removed" ;;
db) db_drop "$a" >/dev/null 2>&1 && info "database $a removed" || warn "database $a not removed" ;;
dir) rm -rf -- "$a" && info "$a removed" ;;
user) pkill -u "$a" 2>/dev/null || true; userdel -r "$a" >/dev/null 2>&1 && info "user $a removed" || true ;;
esac
done
}
cmd_install() {
local church="" products="" address="" events_domain="" email="" version="" main=0 user="" api="" web="" ssl=1
local church="" products="" address="" events_domain="" email="" version="" main=0 user="" api="" web="" ssl=1 frp=0 keep=0
while [ $# -gt 0 ]; do
case "$1" in
--products) products="$(printf '%s' "$2" | tr ',' ' ')"; shift 2 ;;
@@ -58,6 +88,8 @@ cmd_install() {
--api-port) api="$2"; shift 2 ;;
--web-port) web="$2"; shift 2 ;;
--no-ssl) ssl=0; shift ;;
--frp) frp=1; ssl=0; shift ;;
--keep-on-failure) keep=1; shift ;;
-h|--help) install_usage; return 0 ;;
-*) die "Unknown option $1 (crosscode install --help)." ;;
*) [ -z "$church" ] || die "One church at a time."; church="$1"; shift ;;
@@ -73,43 +105,84 @@ cmd_install() {
done
[[ "$address" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?\.[a-z]{2,}$ ]] || die "--domain: give the church's address, e.g. hopefamily.co.za."
events_domain="${events_domain:-events.$address}"
[ "$ssl" = 0 ] || [ -n "$email" ] || die "--email is required for the certificate (or use --no-ssl)."
[[ "$events_domain" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?\.[a-z]{2,}$ ]] || die "--events-domain: expected a domain."
[ "$ssl" = 0 ] || [ -n "$email" ] || die "--email is required for the certificate (or use --frp or --no-ssl)."
[ -z "$version" ] || [[ "$version" =~ ^events@[0-9]+\.[0-9]+\.[0-9]+(-[0-9A-Za-z.-]+)?$ ]] || die "--version: expected events@x.y.z."
[ "$main" = 0 ] || [ -z "$version" ] || die "Use --main or --version, not both."
if [ "$frp" = 1 ]; then
[ -n "$api" ] && [ -n "$web" ] || die "--frp needs --api-port and --web-port (from the Hub's port list)."
fi
user="${user:-cc-$church}"
user="${user:0:32}"
require_root
[ ! -f "$(church_env_path "$church")" ] || die "$church is already installed here (crosscode status $church). New products for it come with \`crosscode add\` (phase 3)."
[ ! -e "$(church_dir "$church")" ] || die "$(church_dir "$church") exists without a church.env. Move it away first."
local taken
mapfile -t taken < <(used_ports)
if [ -n "$api" ] && printf '%s\n' "${taken[@]}" | grep -qx "$api"; then die "Port $api is already in use here."; fi
if [ -n "$web" ] && printf '%s\n' "${taken[@]}" | grep -qx "$web"; then die "Port $web is already in use here."; fi
api="${api:-$(free_port 5000 "${taken[@]}")}"
web="${web:-$(free_port 3000 "${taken[@]}" "$api")}"
[[ "$api" =~ ^[0-9]+$ && "$web" =~ ^[0-9]+$ && "$api" != "$web" ]] || die "Ports must be two different numbers."
if [ "$frp" = 1 ]; then
command -v "$FRPC_BIN" >/dev/null || die "frpc isn't installed here ($FRPC_BIN)."
frpc_check_format
fi
report_result church "$church"
report_result_json ports "{\"events\":{\"web\":$web,\"api\":$api}}"
report_result_json domains "{\"events\":\"$events_domain\"}"
install_system_packages "$ssl"
step "App user ($user)"
id "$user" >/dev/null 2>&1 || useradd --create-home --shell /bin/bash "$user"
CENV=()
church_env_set CHURCH "$church"
church_env_set PRODUCTS "events"
church_env_set ADDRESS "$address"
install_events "$church" "$user" "$events_domain" "$api" "$web" "$version" "$main" "$ssl" "$email"
church_env_write "$(church_env_path "$church")"
INSTALL_UNDO="$(mktemp)"
local rc
set +e
(
set -e
install_system_packages "$ssl" "$frp"
step "App user ($user)"
if ! id "$user" >/dev/null 2>&1; then
useradd --create-home --shell /bin/bash "$user"
undo_add user "$user"
fi
CENV=()
church_env_set CHURCH "$church"
church_env_set PRODUCTS "events"
church_env_set ADDRESS "$address"
install_events "$church" "$user" "$events_domain" "$api" "$web" "$version" "$main" "$ssl" "$email" "$frp"
church_env_write "$(church_env_path "$church")"
)
rc=$?
set -e
if [ "$rc" != 0 ]; then
if [ "$keep" = 1 ]; then
warn "Install failed; --keep-on-failure: what was made is left in place ($(church_dir "$church"))."
else
install_undo
fi
rm -f "$INSTALL_UNDO"
die "Install of $church failed."
fi
rm -f "$INSTALL_UNDO"
step "Done"
info "DNS (if not set yet): $events_domain → this server (A record, or a CNAME)."
info "Open https://$events_domain/setup to create the admin account, then enter the licence key."
if [ "$frp" = 1 ]; then
info "Tunnels: $church-events-web → $web, $church-events-api → $api. Set up the site on the central server."
else
info "DNS (if not set yet): $events_domain → this server (A record, or a CNAME)."
fi
info "Open https://$events_domain/setup to create the admin account."
info "Updates: crosscode deploy $church events"
}
install_system_packages() {
local ssl="$1"
local ssl="$1" frp="$2"
step "System packages"
export DEBIAN_FRONTEND=noninteractive
apt-get update -q
apt-get install -y -q curl git rsync nginx postgresql postgresql-client openssl build-essential ca-certificates gnupg
apt-get install -y -q curl git rsync zip postgresql-client openssl build-essential ca-certificates gnupg
# A database here, unless the server's configuration names another PostgreSQL server.
[ -n "$(cfg server DB_ADMIN_URL)" ] || apt-get install -y -q postgresql
[ "$frp" = 1 ] || apt-get install -y -q nginx
[ "$ssl" = 0 ] || apt-get install -y -q certbot python3-certbot-nginx
# Node 24 is the LTS line supported until April 2028.
if ! command -v node >/dev/null || [ "$(node_major)" -lt 24 ]; then
@@ -121,33 +194,30 @@ install_system_packages() {
}
install_events() {
local church="$1" user="$2" domain="$3" api="$4" web="$5" version="$6" main="$7" ssl="$8" email="$9"
local dir app db pass url name scheme
local church="$1" user="$2" domain="$3" api="$4" web="$5" version="$6" main="$7" ssl="$8" email="$9" frp="${10}"
local dir app db pass name ref
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/events.sh"
dir="$(church_dir "$church")/events"
app="$dir/apps/events"
db="$(printf '%s_events' "$church" | tr '-' '_')"
name="$church-events"
scheme=https
[ "$ssl" = 1 ] || scheme=http
url="$scheme://$domain"
church_env_set EVENTS_DIR "$dir"
church_env_set EVENTS_DOMAIN "$domain"
church_env_set EVENTS_PM2 "$name-backend $name-frontend"
church_env_set EVENTS_PORTS "$api $web"
church_env_set EVENTS_DB "$db"
church_env_set EVENTS_USER "$user"
[ "$frp" = 0 ] || church_env_set EVENTS_FRP 1
step "Events: code ($dir)"
mkdir -p "$(church_dir "$church")"
undo_add dir "$(church_dir "$church")"
chmod 755 "$CROSSCODE_SITES" "$(church_dir "$church")"
if [ ! -d "$dir/.git" ]; then
mkdir -p "$dir"
chown "$user:$user" "$dir"
run_as "$user" "$dir" git clone --quiet "$CROSSCODE_REPO" .
fi
run_as "$user" "$dir" git fetch --tags --quiet origin
mkdir -p "$dir"
chown "$user:$user" "$dir"
run_as "$user" "$dir" git clone --quiet "$CROSSCODE_REPO" .
run_as "$user" "$dir" git fetch --tags --force --quiet origin
if [ "$main" = 1 ]; then
version=origin/main
elif [ -z "$version" ]; then
@@ -155,47 +225,31 @@ install_events() {
[ -n "$version" ] || die "No events@x.y.z release tags yet (use --version or --main)."
fi
run_as "$user" "$dir" git checkout -q --detach "$version"
info "At $(checkout_ref "$dir" events)"
ref="$(checkout_ref "$dir" events)"
info "At $ref"
report_result_json versions "{\"events\":\"$ref\"}"
step "Events: database ($db)"
# Re-running keeps the password already in backend/.env; otherwise a new one.
pass=""
if [ -f "$app/backend/.env" ]; then
pass="$(env_value "$app/backend/.env" DATABASE_URL | sed -nE 's#^postgresql://[^:]+:([^@]+)@.*#\1#p')"
fi
pass="${pass:-$(random_secret 32)}"
sudo -u postgres psql -v ON_ERROR_STOP=1 -q <<SQL
DO \$\$ BEGIN
IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = '$db') THEN
CREATE ROLE "$db" LOGIN PASSWORD '$pass';
ELSE
ALTER ROLE "$db" WITH LOGIN PASSWORD '$pass';
END IF;
END \$\$;
SQL
sudo -u postgres psql -tAc "SELECT 1 FROM pg_database WHERE datname = '$db'" | grep -q 1 ||
sudo -u postgres createdb -O "$db" "$db"
step "Events: database ($db on $(db_host))"
# Never reuse (or later drop) a database or role that something else may be using.
! db_exists "$db" || die "A database or role named $db already exists on $(db_host). Remove it first if it's left over, or choose another church name."
pass="$(random_secret 32)"
undo_add db "$db"
db_create "$db" "$pass"
step "Events: configuration"
if [ ! -f "$app/backend/.env" ]; then
cat > "$app/backend/.env" <<ENV
NODE_ENV=production
PORT=$api
DATABASE_URL=postgresql://$db:$pass@localhost:5432/$db
JWT_SECRET=$(random_secret 64)
FRONTEND_URL=$url
APP_BASE_URL=$url
BACKEND_URL=$url
TRUST_PROXY=1
APP_TIMEZONE=Africa/Johannesburg
ENV
info "Wrote backend/.env (keep JWT_SECRET: it also encrypts stored secrets)."
else
info "backend/.env exists: kept."
fi
[ -f "$app/frontend/.env.production" ] || echo "NEXT_PUBLIC_API_URL=$url" > "$app/frontend/.env.production"
local trust=1
[ "$frp" = 0 ] || trust=loopback
(
export CROSSCODE_GEN_NODE_ENV=production
export CROSSCODE_GEN_JWT_SECRET
CROSSCODE_GEN_JWT_SECRET="$(random_secret 64)"
export CROSSCODE_GEN_DATABASE_URL="postgresql://$db:$pass@$(db_host):$(db_port)/$db"
env_apply "$app" events "$ref" --var "domain=$domain" --var "apiPort=$api" --var "webPort=$web" --var "church=$church" \
--fallback "TRUST_PROXY=$trust"
) | sed 's/^/ /'
[ "${PIPESTATUS[0]}" = 0 ] || die "The .env files couldn't be written (see above)."
chown "$user:$user" "$app/backend/.env" "$app/frontend/.env.production"
chmod 600 "$app/backend/.env"
chmod 600 "$app/backend/.env" "$app/frontend/.env.production"
events_ensure_pm2_config "$dir" "$user"
step "Events: install, migrate, build"
@@ -204,20 +258,28 @@ ENV
events_build "$dir" "$user"
step "Events: PM2"
undo_add pm2 "$user" "$name-backend $name-frontend"
run_as "$user" "$app" pm2 startOrReload ecosystem.config.js
run_as "$user" "$app" pm2 save
env PATH="$PATH:/usr/bin" pm2 startup systemd -u "$user" --hp "$(getent passwd "$user" | cut -d: -f6)" >/dev/null
step "Events: nginx ($domain)"
install_events_nginx "$name" "$domain" "$api" "$web"
if [ "$ssl" = 1 ]; then
step "Events: certificate"
certbot --nginx -d "$domain" --non-interactive --agree-tos -m "$email" --redirect
if [ "$frp" = 1 ]; then
step "Events: frpc tunnels"
frpc_add "$church" events "$web" "$api"
undo_add frpc "$church" events
frpc_reload
else
step "Events: nginx ($domain)"
install_events_nginx "$name" "$domain" "$api" "$web"
undo_add nginx "$name"
if [ "$ssl" = 1 ]; then
step "Events: certificate"
certbot --nginx -d "$domain" --non-interactive --agree-tos -m "$email" --redirect
fi
fi
step "Events: checking"
wait_for_version "$(events_health_url)" "$(events_expected_version "$dir")" 60 ||
warn "The backend isn't answering yet: sudo -u $user pm2 logs"
step "Events: checking it answers"
wait_for_version "$(events_health_url)" "$(events_expected_version "$dir")" 90 ||
die "The backend didn't answer with its version (sudo -u $user pm2 logs $name-backend)."
}
install_events_nginx() {
@@ -230,6 +292,7 @@ server {
client_max_body_size 25m;
# The API, uploaded files, health and API docs go to the backend; everything else to the site.
# (From Events 2.2.0 everything is under /api; the other paths are for older releases.)
location ~ ^/(api|uploads|health|docs)(/|\$) {
proxy_pass http://127.0.0.1:$api;
proxy_set_header Host \$host;
+205
View File
@@ -0,0 +1,205 @@
# shellcheck shell=bash
# crosscode restart / export / uninstall / purge (docs/AGENT-1.0-PLAN.md §4 and §9).
CROSSCODE_REMOVED="${CROSSCODE_REMOVED:-$CROSSCODE_SITES/.removed}"
manage_usage() {
cat <<'EOF'
crosscode restart <church> [events] reload the products' PM2 processes
crosscode export <church> --out <file.zip> the church's data (crosscode.church-export/1) and uploads
crosscode uninstall <church> --confirm <church>
final backup and uploads kept in
/var/www/sites/.removed/<church>-<date>/ (30 days),
then PM2, tunnels, database and folders removed
crosscode purge <church> --confirm <church> delete what uninstall kept
All take --json (a report at the end; the log goes to stderr).
EOF
}
# The church's products that this script can handle, in deploy order.
manage_products() {
local p
# shellcheck disable=SC2046 # PRODUCTS is a space-separated list
for p in $(in_product_order $(cenv PRODUCTS)); do
is_supported_product "$p" && echo "$p"
done
return 0
}
cmd_restart() {
local church="" p
local -a named=()
while [ $# -gt 0 ]; do
case "$1" in
-h|--help) manage_usage; return 0 ;;
-*) die "Unknown option $1 (crosscode help restart)." ;;
*) if [ -z "$church" ]; then church="$1"; else is_known_product "$1" || die "Unknown product \"$1\"."; named+=("$1"); fi; shift ;;
esac
done
[ -n "$church" ] || die "Name the church."
require_root
load_church "$church"
[ "${#named[@]}" -gt 0 ] || mapfile -t named < <(manage_products)
for p in "${named[@]}"; do
church_has_product "$p" || die "$church has no $p."
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/$p.sh"
step "$p: reloading PM2"
"${p}_reload" "$(pvar "$p" DIR)" "$(pvar "$p" USER)"
step "$p: waiting for it to answer"
wait_for_version "$("${p}_health_url")" "$("${p}_expected_version" "$(pvar "$p" DIR)")" 60
done
}
cmd_export() {
local church="" out="" p tmp dir user bytes
while [ $# -gt 0 ]; do
case "$1" in
--out) [ $# -ge 2 ] || die "--out needs a file."; out="$2"; shift 2 ;;
-h|--help) manage_usage; return 0 ;;
-*) die "Unknown option $1 (crosscode help export)." ;;
*) [ -z "$church" ] || die "One church at a time."; church="$1"; shift ;;
esac
done
[ -n "$church" ] || die "Name the church."
[[ "$out" == /*.zip ]] || die "--out: an absolute path ending in .zip."
[ ! -e "$out" ] || die "$out exists already."
require_root
load_church "$church"
command -v zip >/dev/null || die "zip isn't installed (apt-get install zip)."
tmp="$(mktemp -d)"
chmod 755 "$tmp"
local rc
set +e
(
set -e
for p in $(manage_products); do
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/$p.sh"
dir="$(pvar "$p" DIR)"; user="$(pvar "$p" USER)"
step "$p: exporting the data"
mkdir -p "$tmp/$p"
chown "$user:$user" "$tmp/$p"
"${p}_export" "$dir" "$user" "$tmp/$p"
step "$p: adding the uploads"
"${p}_copy_uploads" "$dir" "$tmp/$p/uploads"
done
step "Zipping"
# One product: its folder is the ZIP's top level (export.json, uploads/, README.txt).
if [ "$(manage_products | wc -l)" = 1 ]; then
(cd "$tmp/$(manage_products)" && zip -qr "$out" .)
else
(cd "$tmp" && zip -qr "$out" .)
fi
chmod 600 "$out"
)
rc=$?
set -e
rm -rf "$tmp"
[ "$rc" = 0 ] || { rm -f "$out"; die "Export of $church failed."; }
bytes="$(stat -c %s "$out")"
report_result file "$out"
report_result_json bytes "$bytes"
step "Exported $church to $out ($bytes bytes)"
}
cmd_uninstall() {
local church="" confirm="" p dir user db kept old name
while [ $# -gt 0 ]; do
case "$1" in
--confirm) [ $# -ge 2 ] || die "--confirm needs the church's name."; confirm="$2"; shift 2 ;;
-h|--help) manage_usage; return 0 ;;
-*) die "Unknown option $1 (crosscode help uninstall)." ;;
*) [ -z "$church" ] || die "One church at a time."; church="$1"; shift ;;
esac
done
[ -n "$church" ] || die "Name the church."
[ "$confirm" = "$church" ] || die "Uninstalling removes $church's sites and databases: add --confirm $church."
require_root
load_church "$church"
kept="$CROSSCODE_REMOVED/$church-$(date +%Y%m%d-%H%M%S)"
mkdir -p "$kept"
chmod 700 "$CROSSCODE_REMOVED" "$kept"
report_result kept "$kept"
# 1. Keep a final backup, the uploads and the .env files (the backup needs JWT_SECRET to read
# stored secrets) for 30 days. Nothing is removed until this has worked for every product.
for p in $(manage_products); do
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/$p.sh"
dir="$(pvar "$p" DIR)"; user="$(pvar "$p" USER)"
step "$p: final backup"
"${p}_backup" "$dir" "$user"
step "$p: keeping the backup, uploads and settings in $kept/$p"
"${p}_keep" "$dir" "$kept/$p"
done
# 2. Remove, last product first.
local -a order
mapfile -t order < <(manage_products | tac)
for p in "${order[@]}"; do
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/$p.sh"
dir="$(pvar "$p" DIR)"; user="$(pvar "$p" USER)"; db="$(pvar "$p" DB)"
step "$p: stopping and removing the PM2 processes"
# shellcheck disable=SC2046 # the PM2 names are a space-separated list
run_as "$user" / pm2 delete $("${p}_pm2_names") >/dev/null 2>&1 || true
run_as "$user" / pm2 save --force >/dev/null 2>&1 || true
if [ -f "$FRPC_INI" ] && frpc_has_block "$church" "$p"; then
step "$p: removing the frpc tunnels"
frpc_remove "$church" "$p"
frpc_reload
fi
name="$church-$p"
if [ -f "/etc/nginx/sites-available/$name" ]; then
step "$p: removing the nginx site"
rm -f "/etc/nginx/sites-enabled/$name" "/etc/nginx/sites-available/$name"
if nginx -t >/dev/null 2>&1; then systemctl reload nginx; else warn "nginx didn't reload: check nginx -t."; fi
fi
if [ -n "$db" ]; then
step "$p: dropping the database $db"
db_drop "$db"
fi
old="$(pvar "$p" OLD_DIR)"
if [ -n "$old" ] && [ -d "$old" ]; then rm -rf -- "$old"; info "Removed the old folder $old."; fi
done
step "Removing $(church_dir "$church")"
user="$(pvar events USER)"
rm -rf -- "$(church_dir "$church")"
# The app user goes too when crosscode made it for this church (cc-<church>).
if [ "$user" = "cc-$church" ] && id "$user" >/dev/null 2>&1; then
systemctl stop "pm2-$user" >/dev/null 2>&1 || true
pm2 unstartup systemd -u "$user" >/dev/null 2>&1 || true
pkill -u "$user" 2>/dev/null || true
userdel -r "$user" >/dev/null 2>&1 || warn "The user $user couldn't be removed (userdel -r $user)."
fi
step "Uninstalled $church. Kept until purged (30 days): $kept"
}
cmd_purge() {
local church="" confirm="" d n=0
while [ $# -gt 0 ]; do
case "$1" in
--confirm) [ $# -ge 2 ] || die "--confirm needs the church's name."; confirm="$2"; shift 2 ;;
-h|--help) manage_usage; return 0 ;;
-*) die "Unknown option $1 (crosscode help purge)." ;;
*) [ -z "$church" ] || die "One church at a time."; church="$1"; shift ;;
esac
done
[ -n "$church" ] || die "Name the church."
valid_slug "$church" || die "\"$church\" isn't a church name."
[ "$confirm" = "$church" ] || die "Purging deletes $church's last backup for good: add --confirm $church."
require_root
[ ! -f "$(church_env_path "$church")" ] || die "$church is still installed here; purge only deletes what uninstall kept."
for d in "$CROSSCODE_REMOVED/$church"-*; do
[ -d "$d" ] || continue
[[ "$(basename "$d")" =~ ^${church}-[0-9]{8}-[0-9]{6}$ ]] || continue
step "Deleting $d"
rm -rf -- "$d"
n=$((n + 1))
done
report_result_json deleted "$n"
[ "$n" -gt 0 ] || info "Nothing kept for $church."
}
+78
View File
@@ -0,0 +1,78 @@
# shellcheck shell=bash
# --json reports (docs/AGENT-1.0-PLAN.md §4): with --json, every command prints one
# crosscode.script-report/1 object on stdout when it ends, and its usual output goes to stderr.
# The agent reads the object, adds the log lines and sends it to the Hub.
#
# What goes in: each step() as a step (the next step, or the end, closes it), report_result
# key/values, report_status (rolled-back), and the last die() message. Only names and outcomes:
# never values from .env files.
REPORT_FILE=""
REPORT_COMMAND=""
REPORT_STARTED=""
now_iso() { date -u +%Y-%m-%dT%H:%M:%S.000Z; }
# report_init <command>: starts collecting (only with --json).
report_init() {
REPORT_FILE="$(mktemp "${TMPDIR:-/tmp}/crosscode-report.XXXXXX")"
REPORT_COMMAND="$1"
REPORT_STARTED="$(now_iso)"
}
# report_line <kind> <fields…>: one tab-separated line (tabs and newlines in fields become spaces).
report_line() {
[ -n "$REPORT_FILE" ] || return 0
local out="$1" f
shift
for f in "$@"; do
f="${f//$'\t'/ }"
out+=$'\t'"${f//$'\n'/ }"
done
printf '%s\n' "$out" >> "$REPORT_FILE"
}
# report_result <key> <value>: a string. report_result_json <key> <json>: any JSON value.
report_result() { report_line result "$1" "$2"; }
report_result_json() { report_line resultjson "$1" "$2"; }
# report_status rolled-back: how the command ended, when it isn't just succeeded / failed.
report_status() { report_line status "$1"; }
# report_step_outcome <outcome> [detail]: the outcome of the step that's running (default ok).
report_step_outcome() { report_line outcome "$1" "${2:-}"; }
# report_emit <exit code>: prints the report on fd 3 (the real stdout) and removes the file.
report_emit() {
[ -n "$REPORT_FILE" ] || return 0
node -e '
const fs = require("fs");
const [file, command, startedAt, code] = process.argv.slice(1);
const finishedAt = new Date().toISOString().replace(/\.\d{3}Z$/, ".000Z");
const steps = [], result = {};
let status = null, message;
for (const line of fs.readFileSync(file, "utf8").split("\n")) {
if (!line) continue;
const [kind, a = "", b = ""] = line.split("\t");
if (kind === "step") {
const last = steps[steps.length - 1];
if (last && !last.finishedAt) last.finishedAt = a;
steps.push({ name: b.slice(0, 200), startedAt: a, finishedAt: null, outcome: "ok" });
} else if (kind === "outcome" && steps.length) {
const last = steps[steps.length - 1];
last.outcome = a;
if (b) last.detail = b.slice(0, 2000);
} else if (kind === "result") result[a] = b;
else if (kind === "resultjson") { try { result[a] = JSON.parse(b); } catch { result[a] = b; } }
else if (kind === "status") status = a;
else if (kind === "message") message = a.slice(0, 2000);
}
const last = steps[steps.length - 1];
if (last && !last.finishedAt) last.finishedAt = finishedAt;
const failed = code !== "0";
if (failed && last && last.outcome === "ok") last.outcome = "failed";
const out = { schema: "crosscode.script-report/1", command, status: status || (failed ? "failed" : "succeeded"), startedAt, finishedAt, steps, result };
if (message) out.message = message;
process.stdout.write(JSON.stringify(out) + "\n");
' "$REPORT_FILE" "$REPORT_COMMAND" "$REPORT_STARTED" "$1" >&3 || true
rm -f "$REPORT_FILE"
REPORT_FILE=""
}
+13 -1
View File
@@ -31,24 +31,32 @@ cmd_status() {
[ "${#churches[@]}" -gt 0 ] || { info "No churches on this server ($CROSSCODE_SITES/*/church.env)."; return 0; }
fi
local c
STATUS_JSON=""
for c in "${churches[@]}"; do status_church "$c" "$fetch"; done
report_result_json churches "[$STATUS_JSON]"
}
STATUS_JSON=""
# jstr <text>: a JSON string.
jstr() { node -e 'process.stdout.write(JSON.stringify(process.argv[1]))' "$1"; }
status_church() {
local church="$1" fetch="$2" p dir user ref latest url json version db jobs backup line
load_church "$church"
echo
step "$church ($(cenv ADDRESS '(no address)'))"
local pj="" hj
# shellcheck disable=SC2046 # PRODUCTS is a space-separated list of names
for p in $(in_product_order $(cenv PRODUCTS)); do
dir="$(pvar "$p" DIR)"
user="$(pvar "$p" USER)"
if [ ! -d "$dir/.git" ]; then
info "$(printf '%-9s %s' "$p" "missing checkout: $dir")"
pj+="${pj:+,}{\"product\":\"$p\",\"ref\":null,\"latest\":null,\"health\":null,\"problem\":\"missing checkout\"}"
continue
fi
if [ "$fetch" = 1 ]; then
run_as "$user" "$dir" timeout 20 git fetch --tags --quiet origin 2>/dev/null || true
run_as "$user" "$dir" timeout 20 git fetch --tags --force --quiet origin 2>/dev/null || true
fi
ref="$(checkout_ref "$dir" "$p")"
latest="$(latest_tag "$dir" "$p")"
@@ -56,6 +64,7 @@ status_church() {
[ -n "$latest" ] && [ "$ref" != "$latest" ] && line="$line (update available)"
info "$line"
hj=null
if is_supported_product "$p"; then
# shellcheck source=/dev/null
source "$CROSSCODE_HOME/products/$p.sh"
@@ -66,12 +75,15 @@ status_church() {
jobs="$(printf '%s' "$json" | json_get jobs)"
backup="$(printf '%s' "$json" | json_get lastBackupAt)"
info " health: version ${version:-?}, database ${db:-?}, jobs ${jobs:-?}, last backup ${backup:-never}"
hj="$json"
else
warn " $p isn't answering on $url (pm2 status; pm2 logs)."
fi
fi
pj+="${pj:+,}{\"product\":\"$p\",\"ref\":$(jstr "$ref"),\"latest\":$(jstr "$latest"),\"health\":$hj,\"ports\":$(jstr "$(pvar "$p" PORTS)"),\"domain\":$(jstr "$(pvar "$p" DOMAIN)")}"
status_old_folder "$p"
done
STATUS_JSON+="${STATUS_JSON:+,}{\"church\":\"$church\",\"address\":$(jstr "$(cenv ADDRESS)"),\"products\":[$pj]}"
}
# After `adopt`, the old folder is kept for 14 days so `adopt --undo` can go back to it.
+114
View File
@@ -0,0 +1,114 @@
{
"schema": "crosscode.env-template/1",
"product": "events",
"files": {
"backend/.env": [
{
"name": "NODE_ENV",
"layer": "generated",
"required": true,
"description": "production"
},
{
"name": "PORT",
"layer": "install",
"from": "{apiPort}",
"required": true,
"description": "The backend's port (from the Hub's port list)"
},
{
"name": "DATABASE_URL",
"layer": "generated",
"required": true,
"description": "Made on the server with a new password; never leaves it"
},
{
"name": "JWT_SECRET",
"layer": "generated",
"required": true,
"description": "Also encrypts stored secrets: never change it"
},
{
"name": "FRONTEND_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
},
{
"name": "APP_BASE_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
},
{
"name": "BACKEND_URL",
"layer": "install",
"from": "https://{domain}",
"required": true,
"description": "The same address: the backend is under /api (2.2.0+)"
},
{
"name": "TRUST_PROXY",
"layer": "server",
"required": true,
"description": "Behind frp: loopback"
},
{
"name": "APP_TIMEZONE",
"layer": "install",
"default": "Africa/Johannesburg"
},
{
"name": "HUB_URL",
"layer": "defaults",
"default": "https://hub.crosscode.co.za"
},
{
"name": "HUB_HEARTBEAT_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "HUB_PUBLIC_KEY",
"layer": "defaults",
"description": "Only for a Hub other than hub.crosscode.co.za (staging, a test VM): its licence public key"
},
{
"name": "HUB_LICENCE_KEY",
"layer": "install",
"description": "With it the site registers with the Hub by itself when it first starts"
},
{
"name": "HUB_ADMIN_NAME",
"layer": "install",
"description": "The site admin, sent to the Hub on registering"
},
{
"name": "HUB_ADMIN_EMAIL",
"layer": "install"
},
{
"name": "BACKUP_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "DAILY_SUMMARY_ENABLED",
"layer": "defaults",
"default": "true"
},
{
"name": "SENTRY_DSN",
"layer": "defaults"
}
],
"frontend/.env.production": [
{
"name": "NEXT_PUBLIC_API_URL",
"layer": "install",
"from": "https://{domain}",
"required": true
}
]
}
}
+65
View File
@@ -77,3 +77,68 @@ events_reload() {
events_ensure_pm2_config "$1" "$2"
run_as "$2" "$(events_app "$1")" pm2 reload ecosystem.config.js --update-env
}
# The church's data as crosscode.church-export/1 into <out dir> (owned by the app user). Releases
# from before the export (events@2.2.0) don't have the script.
events_export() {
local dir="$1" user="$2" out="$3" app
app="$(events_app "$dir")"
[ -f "$app/backend/scripts/export-church.ts" ] ||
die "This release of Events ($(checkout_ref "$dir" events)) has no export yet. Update it first."
run_as "$user" "$app/backend" npx tsx scripts/export-church.ts --out-dir "$out"
}
# Uploaded files into <dest> (WhatsApp ticket PDFs waiting to be sent are left out).
events_copy_uploads() {
local src
src="$(events_app "$1")/backend/public/uploads"
mkdir -p "$2"
[ -d "$src" ] || return 0
rsync -a --exclude tickets-temp/ "$src/" "$2/"
}
# What uninstall keeps for 30 days in <dest>: the backups (the last is the final one), the
# uploads, and the .env files (the backup's stored secrets are encrypted with its JWT_SECRET).
events_keep() {
local app dest="$2"
app="$(events_app "$1")"
mkdir -p "$dest"
[ ! -d "$app/backend/backups" ] || rsync -a "$app/backend/backups/" "$dest/backups/"
[ ! -d "$app/backend/public/uploads" ] || tar -czf "$dest/uploads.tar.gz" -C "$app/backend/public" uploads
cp -p "$app/backend/.env" "$dest/backend.env" 2>/dev/null || true
cp -p "$app/frontend/.env.production" "$dest/frontend.env.production" 2>/dev/null || true
chmod -R go-rwx "$dest"
}
# --var arguments for the env template (envtpl.cjs), from church.env.
events_env_vars() {
printf '%s\n' --var "domain=$(pvar events DOMAIN)" --var "apiPort=$(events_api_port)" --var "webPort=$(events_web_port)" --var "church=$(cenv CHURCH)"
}
# After a failed deploy, before going back to <previous ref>: migrations Prisma recorded as failed
# that the previous release doesn't have are marked as rolled back, so the next deploy isn't
# blocked (P3009). The database itself isn't changed: the backup taken first has it as it was.
events_resolve_failed_migrations() {
local dir="$1" user="$2" prev="$3" backend name
backend="$(events_app "$dir")/backend"
local -a failed
# shellcheck disable=SC2016 # JavaScript, not shell
mapfile -t failed < <(run_as "$user" "$backend" node -e '
require("dotenv").config();
const { Client } = require("pg");
const c = new Client({ connectionString: process.env.DATABASE_URL });
c.connect()
.then(() => c.query("select migration_name from _prisma_migrations where finished_at is null and rolled_back_at is null"))
.then(r => { r.rows.forEach(x => console.log(x.migration_name)); return c.end(); })
.catch(e => { console.error(e.message); process.exit(1); });')
for name in "${failed[@]}"; do
[ -n "$name" ] || continue
if git_in "$dir" cat-file -e "$prev:apps/events/backend/prisma/migrations/$name/migration.sql" 2>/dev/null; then
warn "events: $name failed but $prev has it too; left for a look (npx prisma migrate status)."
continue
fi
run_as "$user" "$backend" npx prisma migrate resolve --rolled-back "$name" >/dev/null
info "events: marked the failed migration $name as rolled back."
report_result "resolvedMigration" "$name"
done
}
+38
View File
@@ -209,3 +209,41 @@ cc() { bash "$CROSSCODE_HOME/crosscode" "$@"; }
[ "$status" -eq 1 ]
[[ "$output" == *'No church "nowhere"'* ]]
}
@test "deploy --json: a rolled-back release is reported as rolled-back, with each product's result" {
out="$(FAIL_HEALTH=2.2.1 cc deploy hope events --version events@2.2.1 --json 2>/dev/null || true)"
node -e '
const r = JSON.parse(process.argv[1]);
if (r.schema !== "crosscode.script-report/1" || r.status !== "rolled-back") throw new Error(r.status);
if (r.result.products.events.to !== "events@2.2.1" || r.result.products.events.from !== "events@2.2.0") throw new Error("products");
if (!r.steps.some(s => s.name.includes("reloading PM2"))) throw new Error("steps");' "$out"
[ "$(on hope)" = events@2.2.0 ]
}
@test "a release with a new required setting and no value for it stops at the checks" {
echo 'events_env_vars() { printf "%s\n" --var domain=events.hope.co.za --var apiPort=5000; }' >> "$CROSSCODE_HOME/products/events.sh"
mkdir -p "$CROSSCODE_SITES/hope/events/apps/events/backend"
printf 'DATABASE_URL=x\n' > "$CROSSCODE_SITES/hope/events/apps/events/backend/.env"
# As in the real repo, .env files are ignored by git.
echo .env >> "$CROSSCODE_SITES/hope/events/.git/info/exclude"
cat > "$TMP/origin/apps/events/env.template.json" <<'TPL'
{ "schema": "crosscode.env-template/1", "product": "events", "files": { "backend/.env": [
{ "name": "DATABASE_URL", "layer": "generated", "required": true },
{ "name": "NEW_SERVER_SETTING", "layer": "server", "required": true },
{ "name": "NEW_DEFAULT", "layer": "defaults", "default": "on" } ] } }
TPL
release 2.2.2 "## 2.2.2"
run cc deploy hope events --version events@2.2.2
[ "$status" -eq 1 ]
[[ "$output" == *"NEW_SERVER_SETTING (server, backend/.env)"* ]]
[[ "$output" == *"Nothing was changed"* ]]
[ "$(on hope)" = events@2.2.0 ]
echo '{ "server": { "NEW_SERVER_SETTING": "yes" } }' > "$TMP/config.json"
run cc deploy hope events --version events@2.2.2 --config "$TMP/config.json"
[ "$status" -eq 0 ]
[ "$(on hope)" = events@2.2.2 ]
grep -qx 'DATABASE_URL=x' "$CROSSCODE_SITES/hope/events/apps/events/backend/.env"
grep -qx 'NEW_SERVER_SETTING=yes' "$CROSSCODE_SITES/hope/events/apps/events/backend/.env"
grep -qx 'NEW_DEFAULT=on' "$CROSSCODE_SITES/hope/events/apps/events/backend/.env"
}
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bats
# .env files from a product's env template (lib/envtpl.cjs, AGENT-1.0-PLAN §5): written whole on
# install, only appended to on deploy, nothing written when a required value is missing.
setup() {
TMP="$BATS_TEST_TMPDIR"
TPL_JS="$BATS_TEST_DIRNAME/../lib/envtpl.cjs"
APP="$TMP/app"
mkdir -p "$APP/backend" "$APP/frontend"
cat > "$TMP/template.json" <<'EOF'
{ "schema": "crosscode.env-template/1", "product": "events", "files": {
"backend/.env": [
{ "name": "DATABASE_URL", "layer": "generated", "required": true },
{ "name": "PORT", "layer": "install", "from": "{apiPort}", "required": true },
{ "name": "FRONTEND_URL", "layer": "install", "from": "https://{domain}", "required": true },
{ "name": "TRUST_PROXY", "layer": "server", "required": true },
{ "name": "HUB_URL", "layer": "defaults", "default": "https://hub.crosscode.co.za" },
{ "name": "SENTRY_DSN", "layer": "defaults" },
{ "name": "HUB_LICENCE_KEY", "layer": "install" }
],
"frontend/.env.production": [
{ "name": "NEXT_PUBLIC_API_URL", "layer": "install", "from": "https://{domain}", "required": true }
] } }
EOF
cat > "$TMP/config.json" <<'EOF'
{ "server": { "TRUST_PROXY": "loopback" }, "defaults": {}, "install": { "events": { "HUB_LICENCE_KEY": "CEH-AAAA-BBBB" } } }
EOF
export CROSSCODE_GEN_DATABASE_URL='postgresql://hope_events:s3cr3t@localhost:5432/hope_events'
}
envtpl() { node "$TPL_JS" --template "$TMP/template.json" --dir "$APP" --product events --var domain=events.hope.co.za --var apiPort=20011 "$@"; }
@test "install: writes every file from the layers, mode 600, values never printed" {
run envtpl --config "$TMP/config.json" --release events@2.3.0
[ "$status" -eq 0 ]
[[ "$output" == *"backend/.env: wrote DATABASE_URL, PORT, FRONTEND_URL, TRUST_PROXY, HUB_URL, HUB_LICENCE_KEY"* ]]
[[ "$output" != *"s3cr3t"* && "$output" != *"CEH-AAAA"* ]]
grep -qx 'DATABASE_URL=postgresql://hope_events:s3cr3t@localhost:5432/hope_events' "$APP/backend/.env"
grep -qx 'PORT=20011' "$APP/backend/.env"
grep -qx 'FRONTEND_URL=https://events.hope.co.za' "$APP/backend/.env"
grep -qx 'TRUST_PROXY=loopback' "$APP/backend/.env"
grep -qx 'HUB_URL=https://hub.crosscode.co.za' "$APP/backend/.env"
! grep -q SENTRY_DSN "$APP/backend/.env"
grep -qx 'NEXT_PUBLIC_API_URL=https://events.hope.co.za' "$APP/frontend/.env.production"
# File modes only mean something on Linux (the servers); Windows file systems ignore them.
[ "$(uname -s)" != Linux ] || [ "$(stat -c %a "$APP/backend/.env")" = 600 ]
}
@test "a required value missing anywhere: nothing is written" {
run envtpl --release events@2.3.0
[ "$status" -eq 3 ]
[[ "$output" == *"TRUST_PROXY (server, backend/.env)"* ]]
[ ! -e "$APP/backend/.env" ]
[ ! -e "$APP/frontend/.env.production" ]
}
@test "--fallback fills a value no layer gives" {
run envtpl --fallback TRUST_PROXY=1
[ "$status" -eq 0 ]
grep -qx 'TRUST_PROXY=1' "$APP/backend/.env"
}
@test "deploy: only new variables are appended; existing lines stay byte for byte" {
printf '# by hand\nDATABASE_URL=postgresql://keep\nPORT=5000\nFRONTEND_URL="https://old.example"\nTRUST_PROXY=1\nCUSTOM=x\n' > "$APP/backend/.env"
cp "$APP/backend/.env" "$TMP/before"
echo 'NEXT_PUBLIC_API_URL=https://old.example' > "$APP/frontend/.env.production"
run envtpl --config "$TMP/config.json" --release events@2.4.0
[ "$status" -eq 0 ]
[[ "$output" == *"backend/.env: added HUB_URL, HUB_LICENCE_KEY"* ]]
[[ "$output" != *"frontend"* ]]
head -c "$(stat -c %s "$TMP/before")" "$APP/backend/.env" | cmp - "$TMP/before"
grep -q '^# Added by crosscode .* (events@2.4.0)$' "$APP/backend/.env"
grep -qx 'HUB_URL=https://hub.crosscode.co.za' "$APP/backend/.env"
[ "$(grep -c '^PORT=' "$APP/backend/.env")" = 1 ]
}
@test "--check says what would be added and changes nothing" {
printf 'DATABASE_URL=x\nPORT=1\nFRONTEND_URL=y\nTRUST_PROXY=1\n' > "$APP/backend/.env"
cp "$APP/backend/.env" "$TMP/before"
run envtpl --check
[ "$status" -eq 0 ]
[[ "$output" == *"backend/.env: would add HUB_URL"* ]]
cmp "$APP/backend/.env" "$TMP/before"
[ ! -e "$APP/frontend/.env.production" ]
}
@test "values with spaces or quotes are quoted the way dotenv reads them" {
run node -e 'const t = require(process.argv[1]); console.log(t.line("A", "plain/value:1")); console.log(t.line("B", "two words \"q\""))' "$TPL_JS"
[ "${lines[0]}" = 'A=plain/value:1' ]
[ "${lines[1]}" = 'B="two words \"q\""' ]
}
@test "a file name that leaves the product folder is refused" {
echo '{ "schema": "crosscode.env-template/1", "product": "events", "files": { "../evil": [ { "name": "A", "layer": "defaults", "default": "1" } ] } }' > "$TMP/bad.json"
run node "$TPL_JS" --template "$TMP/bad.json" --dir "$APP"
[ "$status" -ne 0 ]
[ ! -e "$TMP/evil" ]
}
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env bats
# frpc.ini blocks (lib/frpc.sh, AGENT-1.0-PLAN §6): only appended, removed by their markers,
# everything written by hand unchanged byte for byte, nothing changed when `frpc verify` refuses.
setup() {
TMP="$BATS_TEST_TMPDIR"
# shellcheck source=../lib/common.sh
source "$BATS_TEST_DIRNAME/../lib/common.sh"
# shellcheck source=../lib/frpc.sh
source "$BATS_TEST_DIRNAME/../lib/frpc.sh"
FRPC_INI="$TMP/frp/frpc.ini"
FRPC_BACKUPS="$TMP/frp/backups"
mkdir -p "$TMP/frp" "$TMP/bin"
# A stub frpc: verify fails when FRPC_REFUSE is set, reload logs.
cat > "$TMP/bin/frpc" <<'EOF'
#!/usr/bin/env bash
case "$1" in
verify) [ -z "${FRPC_REFUSE:-}" ] ;;
reload) echo reload >> "$(dirname "$3")/reloads" ;;
esac
EOF
chmod +x "$TMP/bin/frpc"
FRPC_BIN="$TMP/bin/frpc"
cat > "$FRPC_INI" <<'EOF'
[common]
server_addr = 203.0.113.10
server_port = 7000
token = hand-written
# the church's own tunnel, written by hand
[events-frontend]
type = tcp
local_ip = 127.0.0.1
local_port = 3000
remote_port = 3000
EOF
cp "$FRPC_INI" "$TMP/original"
}
@test "adds a marked block at the end, and the admin lines to [common] once" {
frpc_add hope events 20010 20011
grep -qx '# >>> crosscode:hope:events (added by crosscode .*' "$FRPC_INI" || grep -q '^# >>> crosscode:hope:events' "$FRPC_INI"
grep -qx '\[hope-events-web\]' "$FRPC_INI"
grep -qx 'remote_port = 20011' "$FRPC_INI"
[ "$(grep -c '^admin_addr = 127.0.0.1$' "$FRPC_INI")" = 1 ]
[ "$(sed -n 2p "$FRPC_INI")" = "admin_addr = 127.0.0.1" ]
frpc_add bridge events 20020 20021
[ "$(grep -c '^admin_addr' "$FRPC_INI")" = 1 ]
[ "$(frpc_ports hope events)" = "20010 20011" ]
[ "$(ls "$FRPC_BACKUPS" | wc -l)" -ge 1 ]
}
@test "the hand-written lines stay exactly as they were" {
frpc_add hope events 20010 20011
# Everything except the two admin lines and the appended block is the original.
grep -v '^admin_' "$FRPC_INI" | head -n "$(wc -l < "$TMP/original")" | cmp - "$TMP/original"
}
@test "removing one block leaves the other blocks and the hand-written lines untouched" {
frpc_add hope events 20010 20011
cp "$FRPC_INI" "$TMP/with-hope"
frpc_add bridge events 20020 20021
frpc_remove bridge events
cmp "$FRPC_INI" "$TMP/with-hope"
frpc_remove hope events
grep -v '^admin_' "$FRPC_INI" | cmp - "$TMP/original"
}
@test "adding the same block twice changes nothing" {
frpc_add hope events 20010 20011
cp "$FRPC_INI" "$TMP/once"
run frpc_add hope events 20010 20011
[ "$status" -eq 0 ]
cmp "$FRPC_INI" "$TMP/once"
}
@test "a port or tunnel name that's already used is refused" {
run frpc_add other events 3000 20031
[ "$status" -eq 1 ]
[[ "$output" == *"port 3000 or 20031 is already used"* ]]
cmp "$FRPC_INI" "$TMP/original"
}
@test "a failed frpc verify leaves the file untouched" {
FRPC_REFUSE=1
export FRPC_REFUSE
run frpc_add hope events 20010 20011
[ "$status" -eq 1 ]
[[ "$output" == *"frpc verify refused"* ]]
cmp "$FRPC_INI" "$TMP/original"
[ -z "$(find "$TMP/frp" -name '.frpc.ini.*')" ]
}
@test "an admin_addr that's already set is left alone" {
sed -i 's/^server_port = 7000$/server_port = 7000\nadmin_addr = 0.0.0.0/' "$FRPC_INI"
frpc_add hope events 20010 20011
[ "$(grep -c '^admin_addr' "$FRPC_INI")" = 1 ]
grep -qx 'admin_addr = 0.0.0.0' "$FRPC_INI"
}
@test "a file without a final newline keeps its last line" {
printf '[common]\nserver_addr = x' > "$FRPC_INI"
frpc_add hope events 20010 20011
grep -qx 'server_addr = x' "$FRPC_INI"
}
@test "the first time (admin lines just added) frpc is restarted; afterwards it's reloaded" {
cat > "$TMP/bin/systemctl" <<'STUB'
#!/usr/bin/env bash
echo "systemctl $*" >> "$(dirname "$0")/../calls"
STUB
chmod +x "$TMP/bin/systemctl"
PATH="$TMP/bin:$PATH"
FRPC_NEEDS_RESTART=0
frpc_add hope events 20010 20011
[ "$FRPC_NEEDS_RESTART" = 1 ]
run frpc_reload
[ "$status" -eq 0 ]
grep -q "systemctl restart frpc" "$TMP/calls"
[[ "$output" == *"restarting it once"* ]]
# Admin lines now there: the next change reloads.
: > "$TMP/calls"
FRPC_NEEDS_RESTART=0
frpc_add bridge events 20020 20021
[ "$FRPC_NEEDS_RESTART" = 0 ]
frpc_reload
! grep -q "restart frpc" "$TMP/calls"
grep -q reload "$TMP/frp/reloads"
}
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env bats
# --json reports (lib/report.sh) and the argument checks of the new commands (what needs a server
# is tested on a VM).
bats_require_minimum_version 1.5.0
setup() {
export CROSSCODE_NO_UPDATE=1
CC="$BATS_TEST_DIRNAME/../crosscode"
}
json() { node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{const v=process.argv[1].split(".").reduce((o,k)=>o==null?o:o[k],JSON.parse(s));process.stdout.write(typeof v==="object"?JSON.stringify(v):String(v))})' "$1"; }
@test "--json: one report on stdout, the log on stderr" {
run --separate-stderr bash "$CC" help --json
[ "$status" -eq 0 ]
[ "$(printf '%s' "$output" | json schema)" = "crosscode.script-report/1" ]
[ "$(printf '%s' "$output" | json status)" = "succeeded" ]
[ "$(printf '%s' "$output" | json command)" = "help" ]
[[ "$stderr" == *"crosscode deploy <church>"* ]]
}
@test "--json on a failure: status failed and the reason" {
run --separate-stderr bash "$CC" uninstall hope --json
[ "$status" -eq 1 ]
[ "$(printf '%s' "$output" | json status)" = "failed" ]
[[ "$(printf '%s' "$output" | json message)" == *"--confirm hope"* ]]
}
@test "--config must be an existing JSON file" {
run bash "$CC" status --config /nonexistent.json
[ "$status" -eq 1 ]
[[ "$output" == *"no file /nonexistent.json"* ]]
echo 'not json' > "$BATS_TEST_TMPDIR/c.json"
run bash "$CC" status --config "$BATS_TEST_TMPDIR/c.json"
[ "$status" -eq 1 ]
[[ "$output" == *"isn't JSON"* ]]
}
@test "uninstall and purge need --confirm with the church's own name" {
run bash "$CC" uninstall hope
[ "$status" -eq 1 ]
[[ "$output" == *"add --confirm hope"* ]]
run bash "$CC" uninstall hope --confirm other
[ "$status" -eq 1 ]
run bash "$CC" purge hope
[ "$status" -eq 1 ]
[[ "$output" == *"add --confirm hope"* ]]
}
@test "export needs an absolute .zip path" {
run bash "$CC" export hope --out relative.zip
[ "$status" -eq 1 ]
[[ "$output" == *"absolute path ending in .zip"* ]]
}
@test "install --frp needs the ports from the Hub" {
run bash "$CC" install hope --products events --domain hope.co.za --frp
[ "$status" -eq 1 ]
[[ "$output" == *"--frp needs --api-port and --web-port"* ]]
}
@test "help lists the new commands" {
run bash "$CC" help
[[ "$output" == *"crosscode uninstall <church> --confirm <church>"* ]]
[[ "$output" == *"--json"* ]]
run bash "$CC" help export
[[ "$output" == *"crosscode.church-export/1"* ]]
}