diff --git a/apps/events/docs/HANDOVER.md b/apps/events/docs/HANDOVER.md index 9de8e95..ee26a3b 100644 --- a/apps/events/docs/HANDOVER.md +++ b/apps/events/docs/HANDOVER.md @@ -1,6 +1,8 @@ # Handover — where things stand and how to carry on -Last updated: 2026-11-05, after **2.1.1** (deployed). **This file covers Cross Events only** +Last updated: 2026-11-17, after **2.3.0** (released, tested on a VM, not deployed yet; servers +still run 2.1.1 until they're adopted, see the root [ROLLOUT-1.0.md](../../../docs/ROLLOUT-1.0.md)). +**This file covers Cross Events only** (what's been done, where the code lives, known gaps). **The plan for every product and where each phase stands are in the monorepo's root `docs/`** → [ROADMAP-v3.md](../../../docs/ROADMAP-v3.md), @@ -95,13 +97,13 @@ encrypted settings unreadable or break existing installs. ## Next up 1. **Roadmap v3**, across every product: see `crosscode-platform/docs/HANDOVER.md` ("Where each - phase stands", "Next up"). For Cross Events that means **2.2.0** (the move into the monorepo as - `apps/events`, with the `crosscode` script; `crosscode-platform/docs/EVENTS-2.2-PLAN.md`), then - **2.3.0** (authenticator app 2FA, `TwoFactorMethod` in `prisma/schema.prisma`) and **2.4.0** + phase stands", "Next up"). **2.2.0** (the move into the monorepo) and **2.3.0** (authenticator + app 2FA and recovery codes, plus what installs from the Hub need) are released (2026-11-17). + Next for Cross Events is **rolling them out** (each install adopted from the Hub), then **2.4.0** (protocol v2, Linked products, the event feed). - - **Deploying until 2.2.0:** `bash scripts/deploy.sh --check`, then - `bash scripts/deploy.sh v` on each server (installs ask through Request update as - usual). From 2.2.0 on: `crosscode deploy events`. + - **Deploying:** installs still on 2.1.1 move with `adopt` (Hub → Servers → Installs → Link + existing → Move onto the monorepo, or `crosscode adopt` by hand), never `deploy`. After that, + updates are planned on the Hub (Updates), or `crosscode deploy events` by hand. - **AI for a church:** set the licence's monthly AI cap above R0 on the Hub (Hub 1.6.0+); with no cap the buttons don't show. - Testing AI locally: run the Hub (`../events-hub`, port 5200), set Settings → AI there, give a @@ -119,6 +121,27 @@ encrypted settings unreadable or break existing installs. Languages (Afrikaans, §10) and Wallet passes (§7) are removed from scope (2026-10-28). +## Where two-factor lives (2.3.0) + +- `backend/src/utils/twoFactor.js`: every method (email, WhatsApp, `authenticator`), challenges, + recovery codes (10 per set, stored as HMACs, single use). `backend/src/utils/totp.js`: RFC 6238 + with a last-step check (`User.totpLastStep`), so a code can't be used twice. +- **`totpSecret` never goes to the browser:** `LOGIN_USER_SELECT` doesn't include it (that selection + is sent back on sign-in), so `createChallenge` reads it from the database itself. +- Frontend: the profile page draws the QR code in the browser (a dynamic `import("qrcode")`); + `LoginForm.tsx` has "Use a recovery code". Tests: `tests/twoFactorAuthenticator.test.js`. + +## Where the Hub-install pieces live (2.3.0) + +- `env.template.json` (copied to `scripts/products/events.env.template.json` at the monorepo root; + keep them the same): every `.env` variable with its layer (`server`, `defaults`, `install`, + `generated`). The Hub has its own copy (`events-hub/backend/src/utils/envTemplates/events.json`). + A new variable goes in all three. +- `backend/scripts/export-church.ts` (`src/services/churchExport.ts`): the church's data as + `crosscode.church-export/1`, for `crosscode export` and uninstalls. No internal ids. +- `backend/src/hub/client.js` `connectFromEnv()`: a site the Hub installed registers by itself on + first start from `HUB_LICENCE_KEY`. + ## Working on this repo - **Release routine:** branch `feat/` → commit → bump with diff --git a/docs/AGENT-1.0-PLAN.md b/docs/AGENT-1.0-PLAN.md index 1d8a638..dc4590b 100644 --- a/docs/AGENT-1.0-PLAN.md +++ b/docs/AGENT-1.0-PLAN.md @@ -8,9 +8,9 @@ This replaces the old rule "the Hub never deploys" (decisions 10 and 23). The pa stays: **no site depends on the Hub to keep running**, and everything still goes through the `crosscode` script (EVENTS-2.2-PLAN §4), which keeps working by hand over SSH. -**Built 2026-11-18** (crosscode-platform branch `feat/agent-1.0`, events-hub branch -`feat/servers-agent`; not merged or released). Waiting for the VM tests (§13). Where the build -differs from this plan, see [Changes while building](#changes-while-building). +**Built, VM tested and released 2026-11-17** (`agent@1.0.0`, `contracts@1.3.0`, Hub `v2.1.0`; with +`events@2.3.0`). Not rolled out yet: the steps are in [ROLLOUT-1.0.md](./ROLLOUT-1.0.md). Where the +build differs from this plan, see [Changes while building](#changes-while-building). ## Contents @@ -459,7 +459,7 @@ Central check: both ports are open in frps and only on 127.0.0.1. ## Changes while building -Made on 2026-11-18, while building; none changes what Joshua or a church sees. +Made on 2026-11-17, while building and in the VM test; none changes what Joshua or a church sees. 1. **Config encryption** is X25519 + HKDF-SHA256 + AES-256-GCM (`x25519-hkdf-sha256-aes-256-gcm`) instead of `x25519-xsalsa20-poly1305`: Node's built-in crypto has no XSalsa20, and this needs no @@ -480,6 +480,18 @@ Made on 2026-11-18, while building; none changes what Joshua or a church sees. `local_port` and `remote_port`. 7. **`crosscode install`** refuses a database or role that already exists, so it can never take over (or, on a failed install, drop) another site's database. +8. **PM2 runs under its own systemd unit** (`pm2-`) for every site user, set up by the + `crosscode` script before any PM2 command. Found in the VM test: a PM2 daemon started from the + agent's job belonged to the agent's service, so stopping or updating the agent stopped the + sites. The agent's stats never start a PM2 daemon. +9. **frpc is restarted once**, the first time its admin lines are added (a running frpc only reads + them when it starts); later changes reload it. Every tunnel on that server drops for a few + seconds that first time. +10. **The VM test** (§13) also fixed: `adopt` applies a release's new migrations; a failed + migration is marked rolled back so later deploys aren't blocked; the agent install builds with + the root's tools; tags are fetched with `--force`; the Hub saves the servers settings, lets a + failed or removed install's short name be used again, and links a Hub-installed site to its + registration so banners reach it. The full list is in [HANDOVER.md](./HANDOVER.md). ## Decisions diff --git a/docs/EVENTS-2.2-PLAN.md b/docs/EVENTS-2.2-PLAN.md index ccea050..f5291a5 100644 --- a/docs/EVENTS-2.2-PLAN.md +++ b/docs/EVENTS-2.2-PLAN.md @@ -252,6 +252,10 @@ database or its name. ## 6. Testing +**Done 2026-11-17** on a WSL Ubuntu 24.04 VM (steps 1–7 below; step 8 in the agent test). All +pass after two fixes: `adopt` refused a release with new migrations (it now applies them in the +switch-over), and a failed migration blocked later deploys (now marked rolled back). + - **Events, unchanged behaviour:** backend Jest, `tsc --noEmit`, frontend build and the Playwright e2e run, all from the monorepo. They should give the same counts as the 2.1.1 baseline. diff --git a/docs/HANDOVER.md b/docs/HANDOVER.md index cd1a660..77680b1 100644 --- a/docs/HANDOVER.md +++ b/docs/HANDOVER.md @@ -1,21 +1,21 @@ # Handover — Cross Code, all products -Last updated: 2026-11-12. **The plan is [ROADMAP-v3.md](./ROADMAP-v3.md)** (decided 2026-11-03, +Last updated: 2026-11-17. **The plan is [ROADMAP-v3.md](./ROADMAP-v3.md)** (decided 2026-11-03, decisions table near the end). This file says where each phase stands and what comes next. Product-specific notes (where the code lives, known gaps) stay with each product: Cross Events' -are in `hope-events/docs/HANDOVER.md` (`apps/events/docs/HANDOVER.md` from Events 2.2.0), and -the Hub's are in `events-hub/docs/`. +are in [`apps/events/docs/HANDOVER.md`](../apps/events/docs/HANDOVER.md), and the Hub's are in +`events-hub/docs/` and `events-hub/deployment.md`. The docs for every product are here in `docs/`: the roadmap, the Hub protocol -([hub-protocol.md](./hub-protocol.md), with a copy in `events-hub`), this handover and the plans -for each phase. They moved here from `hope-events/docs/` on 2026-11-05. +([hub-protocol.md](./hub-protocol.md), with a copy in `events-hub`), this handover, the plans +for each phase and the rollout checklist ([ROLLOUT-1.0.md](./ROLLOUT-1.0.md)). ## Repos | Repo | What | |------|------| -| `joshua/crosscode-platform` (this one) | Contracts, the apps (Accounts, Website, App, and Events from 2.2.0), shared packages, the `crosscode` script (from Events 2.2.0), the server agent (`apps/agent`, phase 1d), and these docs | -| `joshua/hope-events` | Cross Events up to 2.1.x. Archived after the 2.2.0 move | +| `joshua/crosscode-platform` (this one) | Contracts, the apps (Events from 2.2.0; Accounts, Website and App to come), shared packages, the `crosscode` script (`scripts/`), the server agent (`apps/agent`), and these docs | +| `joshua/hope-events` | Cross Events up to 2.1.x. Frozen; archived once every install is adopted (EVENTS-2.2-PLAN §7) | | `joshua/events-hub` | Cross Code Hub. Its own server, its own `scripts/deploy.sh` | | `joshua/crosscode-site` | crosscode.co.za. Updated at the end of every phase that changes what it says (ROADMAP §9) | @@ -23,26 +23,44 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05. | Phase | What | State | |-------|------|-------| -| 0 | contracts 1.0.0 / 1.1.0 | **Released** 2026-11-04 (`contracts@1.0.0`, `contracts@1.1.0`). Not published to a registry (decision 22): the Hub gets a packed release committed to its repo | -| 1 | Hub 2.0.0 | **Released** 2026-11-05 (`events-hub` `v2.0.0`; `docs/HUB-2.0-PLAN.md` there). **Not deployed yet:** `events-hub/deployment.md` → "Upgrading to 2.0" (run `check-hub2` first). crosscode-site's address prices are merged, **not deployed yet** | -| 1b | Cross Events 2.2.0: moves here as `apps/events`, `crosscode` script | **In progress.** Planned in [EVENTS-2.2-PLAN.md](./EVENTS-2.2-PLAN.md). **Steps 1–3 done 2026-11-12** (merged into `main`): history merged in under `apps/events` (from `hope-events` `e6c9670`), workspaces and one root lockfile with versions pinned to 2.1.1, same test results as 2.1.1 (711 backend tests, typecheck, both builds). **The `crosscode` script and `adopt` (§4–5) are written** (branch `feat/crosscode-script`, 2026-11-12; shellcheck and 47 bats tests pass, not yet run on a server). **Tagged `events@2.2.0` on 2026-11-17, before the VM testing (Joshua's call), with two additions: AI use in tokens and the backend under `/api` (Hub `v2.0.1` and crosscode-site `v1.2.0` released with them).** **Still to do:** VM testing (§6) of the tagged release, adopting each install, archiving `hope-events` (§7). Until then servers keep deploying from `hope-events` `v2.1.1`, and **Events work happens here** (`hope-events` is frozen; a hotfix there as 2.1.2 is cherry-picked into `apps/events`) | -| 1c | Cross Events 2.3.0: authenticator app 2FA (TOTP + recovery codes) | TOTP not started (`TwoFactorMethod` enum in Events' `prisma/schema.prisma`). | -| 1d | Agent 1.0.0 + Hub 2.1.0 (+ contracts 1.3.0): server agent, installs / scheduled updates / uninstalls from the Hub, server stats | **Built 2026-11-18, not released.** crosscode-platform branch `feat/agent-1.0` (contracts 1.3.0, the crosscode script with `--json`, env templates, frpc, restart / export / uninstall / purge, Events env template + export + `HUB_LICENCE_KEY`, `apps/agent`), events-hub branch `feat/servers-agent` (Hub 2.1.0). All unit and script tests pass. **Still to do:** the VM tests (AGENT-1.0-PLAN §13, together with EVENTS-2.2-PLAN §6), then release and roll out (§14). | +| 0 | contracts 1.0.0 / 1.1.0 | **Released** 2026-11-04. Not published to a registry (decision 22): the Hub gets a packed release committed to its repo (`backend/vendor/`) | +| 1 | Hub 2.0.x | **Released** (`v2.0.0` 2026-11-05, `v2.0.1` AI tokens and `v2.0.2` refunds / voiding refunded invoices, 2026-11-17). **Not deployed yet:** `events-hub/deployment.md` → "Upgrading to 2.0" (run `check-hub2` first) | +| 1b | Cross Events 2.2.0: moved here as `apps/events`, the `crosscode` script | **Released** 2026-11-17 (`events@2.2.0`, with AI use in tokens and the backend under `/api`). **VM tested 2026-11-17** (EVENTS-2.2-PLAN §6, all passing after fixes). **Not rolled out:** servers still run `hope-events` `v2.1.1` until each install is adopted | +| 1c | Cross Events 2.3.0: authenticator app 2FA (TOTP + recovery codes) | **Released** 2026-11-17 (`events@2.3.0`). crosscode-site `v1.3.0` lists it (merged and tagged, **not deployed**) | +| 1d | Agent 1.0.0 + Hub 2.1.0 (+ contracts 1.3.0) | **Released** 2026-11-17 (`agent@1.0.0`, `contracts@1.3.0`, Hub `v2.1.0`). **VM tested** end to end (AGENT-1.0-PLAN §13: joins, installs behind frp, scheduled / batch / missed updates, uninstall with export and purge, forged and replayed jobs). **Not rolled out** | | 2 | Cross Events 2.4.0: protocol v2, Linked products, event feed, `packages/seo` | Not started. Events keeps speaking protocol 1 until then | | 3–7 | Accounts 1.0.0 → Website 1.0.0 → Events 3.0.0 → App 1.0.0 → App 1.x | Not started | ## Next up -1. **Deploy Hub 2.0.0** and **crosscode-site** (the address prices). This finishes phase 1 and is - needed before 2.2.0, because Hub 1.7 can't see `events@…` tags. -2. **Cross Events 2.2.0**: [EVENTS-2.2-PLAN.md](./EVENTS-2.2-PLAN.md). Decided 2026-11-05: - `adopt` builds next to the running install and switches over, and `adopt --undo` is built - (ROADMAP decisions 25–26). -3. **Agent 1.0.0 + Hub 2.1.0** (phase 1d): [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md). Before the - first install through it: check `proxy_bind_addr = 127.0.0.1` in `frps.ini` on the central - server (worth doing now: without it, tunnel ports such as `:5000` may be reachable straight from - the internet), and set each server's database settings and `TRUST_PROXY` in the Hub. -4. **Cross Events 2.3.0** (authenticator app 2FA), then 2.4.0, as in ROADMAP §9. +1. **Roll out**, in the order in [ROLLOUT-1.0.md](./ROLLOUT-1.0.md): Hub 2.1.0 (make and back up + the job-signing key first) → the central server's agent (check `proxy_bind_addr = 127.0.0.1`) + → each church server's agent → **adopt** the demo site, then each church (Link existing → Move + onto the monorepo) → a throwaway New install to prove FASTPANEL, DNS and Let's Encrypt. +2. **Deploy crosscode-site `v1.3.0`** (secure sign-in in the features and the brochure). +3. **Archive `hope-events`** once every install reports 2.2.0 or later (EVENTS-2.2-PLAN §7). +4. **Cross Events 2.4.0** (phase 2), as in ROADMAP §9. + +## The VM test (2026-11-17) + +Everything above was run on a WSL Ubuntu 24.04 VM before release: a church server, a "central" +network namespace running frps with `proxy_bind_addr`, the Hub (`feat/servers-agent`) and local +git mirrors standing in for c-git. It found nine bugs, all fixed with tests before tagging. None +lost data, because each failed safely: + +1. `adopt` refused a release with new migrations (it now applies them in the switch-over). +2. A failed migration blocked every later deploy (Prisma P3009; now marked rolled back). +3. The agent install skipped the root's build tools (`npm ci --include-workspace-root`). +4. A failed install's short name couldn't be used again. +5. The Hub didn't save the servers settings (install defaults, port range, central IP). +6. frpc only reads its admin lines when it starts (restarted once the first time). +7. **A site's PM2 belonged to the agent's service**, so stopping or updating the agent took down + every site it had installed. PM2 now always runs under `pm2-`. +8. Update and closing banners never reached Hub-installed sites (registration now links them). +9. A moved tag blocked a church's deploys (tags are fetched with `--force`). + +Not covered by the VM: FASTPANEL, real DNS, Let's Encrypt. Hence the throwaway install in the +rollout. ## Things to remember across phases @@ -51,10 +69,20 @@ for each phase. They moved here from `hope-events/docs/` on 2026-11-05. `backend/src/utils/billing/tiers.js`) in the same phase, so the site and the Hub never disagree. - **A phase isn't done until crosscode-site matches it** (ROADMAP §9, "Updating crosscode-site"). - Servers stay **PM2 + nginx** (no Docker), with one database per product per church, under - `/var/www/sites//`. + `/var/www/sites//`. Each site user's PM2 runs as a systemd unit, + `pm2-`; the `crosscode` script makes sure of it before any PM2 command. +- **The agent pins the Hub's job key** when it joins. Back up `events-hub/backend/keys/` (licence + and job keys); replacing the job key means every server joins again. +- **A new `.env` variable** goes in three env templates: `apps//env.template.json`, + `scripts/products/.env.template.json` and the Hub's `envTemplates/.json`. +- **A new Events install never takes over an existing database**: moving an existing site is + `adopt`, not a New install pointed at the old database. - **Prisma in the monorepo:** each app other than Events sets its own generator `output`, so hoisted clients don't overwrite each other (EVENTS-2.2-PLAN §2). - **AI for a church:** set the licence's monthly AI cap above R0 on the Hub; with no cap the - buttons don't show (rules in Events' handover, "AI rules"). -- **Releasing:** branch → merge → tag (`@`) → push, with a CHANGELOG entry, and - `npm run check` first. Pushes to c-git often fail authentication the first time; retry once. + buttons don't show (rules in Events' handover, "AI rules"). Churches see tokens; only the Hub + shows rand (markup 15% by default). +- **Releasing:** branch → merge → tag (`@` here, `v` in events-hub and + crosscode-site) → push, with a CHANGELOG entry, and `npm run check` first. Pushes to c-git often + fail authentication the first time; retry once. A push can also hang waiting on a login prompt + after it has already gone through: check with `git ls-remote` before pushing again. diff --git a/docs/ROADMAP-v3.md b/docs/ROADMAP-v3.md index be45bc1..0b60409 100644 --- a/docs/ROADMAP-v3.md +++ b/docs/ROADMAP-v3.md @@ -613,9 +613,9 @@ Planned in [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md). In short: |-------|---------|------|----------------| | 0 | contracts 1.0.0 | `crosscode.event/1`, `crosscode.church/1`, `crosscode.identity/1`, Hub protocol v2: schemas, generated JSON Schema, docs, sample payloads, the monorepo set up. **Released 2026-11-04** | Everything else depends on them | | 1 | **Hub 2.0.0** | Cross Code Hub: products, licence per product, per-customer billing with bundles, protocol v2 (v1 still accepted), sibling discovery, AI usage per product, **releases per product with requirements** (§5), contracts as a committed file (§2). **Released 2026-11-05** (crosscode-site address prices merged) | Must exist before a second product is sold, and before Events releases come from the monorepo | -| 1b | **Cross Events 2.2.0** | **Moves into the monorepo** as `apps/events`, with its git history (merged in, not copied); its packages become workspaces; the **`crosscode` script** (`deploy`, `status`, `install`, `adopt`; §8) replaces `scripts/deploy.sh` and `scripts/install.sh`; every install moved with `crosscode adopt`; `hope-events` archived. Planned with no feature changes; two small ones were added on 2026-11-17 (decided by Joshua): **AI use shown in tokens**, never rand, outside the Hub (needs Hub 2.0.1), and **everything the backend serves under `/api`**, so a site and its API share one address and one proxy rule. **Tagged 2026-11-17, before the VM testing** | Before more products, so every product is released and deployed the same way from the start | -| 1c | **Cross Events 2.3.0** | **Authenticator app 2FA** (TOTP) next to email and WhatsApp, with recovery codes; admins can see which method each staff account uses. | Independent and useful now; the Accounts migration later carries it over | -| 1d | **Agent 1.0.0 + Hub 2.1.0** (contracts 1.3.0) | **Server agent** on every server (polls only, signed jobs, runs the `crosscode` script); Hub: Servers page with stats and alerts, server configuration, frp port list, New install with a choice of server, scheduled / emergency / batch updates with notice to the church, uninstall with an encrypted JSON export, emailed reports. [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md) | Right after 1b (it runs the `crosscode` script), so every new church from then on is installed from the Hub. Can run alongside 1c | +| 1b | **Cross Events 2.2.0** | **Moves into the monorepo** as `apps/events`, with its git history (merged in, not copied); its packages become workspaces; the **`crosscode` script** (`deploy`, `status`, `install`, `adopt`; §8) replaces `scripts/deploy.sh` and `scripts/install.sh`; every install moved with `crosscode adopt`; `hope-events` archived. Planned with no feature changes; two small ones were added on 2026-11-17 (decided by Joshua): **AI use shown in tokens**, never rand, outside the Hub (needs Hub 2.0.1), and **everything the backend serves under `/api`**, so a site and its API share one address and one proxy rule. **Tagged 2026-11-17, before the VM testing; VM tested the same day** | Before more products, so every product is released and deployed the same way from the start | +| 1c | **Cross Events 2.3.0** | **Authenticator app 2FA** (TOTP) next to email and WhatsApp, with recovery codes; admins can see which method each staff account uses. **Released 2026-11-17** (`events@2.3.0`) | Independent and useful now; the Accounts migration later carries it over | +| 1d | **Agent 1.0.0 + Hub 2.1.0** (contracts 1.3.0) | **Server agent** on every server (polls only, signed jobs, runs the `crosscode` script); Hub: Servers page with stats and alerts, server configuration, frp port list, New install with a choice of server, scheduled / emergency / batch updates with notice to the church, uninstall with an encrypted JSON export, emailed reports. [AGENT-1.0-PLAN.md](./AGENT-1.0-PLAN.md). **Released 2026-11-17** after the VM test; rollout in [ROLLOUT-1.0.md](./ROLLOUT-1.0.md) | Right after 1b (it runs the `crosscode` script), so every new church from then on is installed from the Hub. Can run alongside 1c | | 2 | **Cross Events 2.4.0** | Reports `product: "events"` (protocol v2), **Linked products**, the event feed (publish + show events from siblings), `packages/seo` (§11) with canonical links for imported events | Small; ready for when the Website exists | | 3 | **Accounts 1.0.0** | OIDC provider, sign-in, 2FA (email, WhatsApp, authenticator app), password reset, consent and notice, export / delete fan-out, admin, install script | The Website is built on it from day one | | 4 | **Church Website 1.0.0** | Public site with the **four templates** (§6), admin, simple events + feed, SEO (§11), Hope Family Church as the first church, then a second test church on a different template to prove it repeats | More churches need a website than an app; it also covers the "website only" case |