Root HANDOVER: phases 1b–1d released and VM tested, the nine bugs the VM test found, rollout next. Events HANDOVER: 2.3.0, where two-factor and the Hub-install pieces live. AGENT-1.0-PLAN and EVENTS-2.2-PLAN: tested and released, the VM changes; ROADMAP §9 marked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ls1vMdni7EfytGYqQTprME
305 lines
18 KiB
Markdown
305 lines
18 KiB
Markdown
# Cross Events 2.2.0 — moving into the monorepo
|
||
|
||
Phase **1b** of [ROADMAP-v3.md](./ROADMAP-v3.md) (§8 "Releases and updates", §9; decisions 21, 23,
|
||
24). Written 2026-11-05.
|
||
|
||
**What 2.2.0 is:** Cross Events moves from `joshua/hope-events` into this repo as `apps/events`,
|
||
with its git history, and the **`crosscode` script** replaces `scripts/deploy.sh` and
|
||
`scripts/install.sh`. **It changes no features.** If something breaks, the cause is the move.
|
||
Churches see one line in What's new ("Nothing changes for you").
|
||
|
||
**Done when:**
|
||
|
||
- `events@2.2.0` is tagged here, and the Hub lists it under Cross Events.
|
||
- Every existing install has been moved with `crosscode adopt` and reports 2.2.0.
|
||
- `hope-events` is archived.
|
||
- The test results match 2.1.1's.
|
||
|
||
## Before starting
|
||
|
||
- **Hub 2.0.0 is deployed.** Its release sync already reads `events@…` tags from this repo, with
|
||
notes in `apps/events/WHATS-NEW.md` (`events-hub/backend/src/utils/releaseSync.js`). It also
|
||
keeps the `hope-events` `v…` source for older releases. Hub 1.7 can't see `events@` tags.
|
||
- `hope-events` `main` is at 2.1.1 with nothing in flight, and **frozen** from step 1 on. A fix
|
||
needed during the move goes in on the `hope-events` side as 2.1.2 and is merged in again
|
||
(step 1 can be re-run).
|
||
- **Baseline:** the result counts of 2.1.1's backend Jest run, frontend build and Playwright e2e
|
||
run, so step 6 has something to compare against.
|
||
- `git-filter-repo` installed on the dev machine (`pip install git-filter-repo`).
|
||
|
||
## 1. History: merge it in, with paths rewritten
|
||
|
||
On a fresh clone, so the working repo is never rewritten:
|
||
|
||
```bash
|
||
git clone https://c-git.crosscode.co.za/joshua/hope-events.git events-move
|
||
cd events-move
|
||
git tag -l | xargs git tag -d # v… tags stay in the archived repo
|
||
git filter-repo --to-subdirectory-filter apps/events
|
||
cd ../crosscode-platform
|
||
git checkout -b move/events
|
||
git remote add events-move ../events-move && git fetch events-move main
|
||
git merge --allow-unrelated-histories events-move/main \
|
||
-m "Cross Events moves into the monorepo (apps/events), from hope-events <last commit hash>"
|
||
git remote remove events-move
|
||
```
|
||
|
||
- **Why `filter-repo` rather than `git subtree add`:** every old commit then already has its
|
||
`apps/events/…` path, so `git log` and `git blame` on a file show its whole history without
|
||
`--follow`. Commit hashes change. The archived repo keeps the originals, and the merge message
|
||
names the last one.
|
||
- **Old tags aren't brought over.** Bare `v2.1.1` tags would be ambiguous in a monorepo, and the
|
||
Hub reads them from the archived repo. Product tags start at `events@2.2.0`.
|
||
|
||
## 2. Workspaces and the lockfile
|
||
|
||
- Root `package.json` → `"workspaces": ["packages/*", "apps/*", "!apps/events",
|
||
"apps/events/backend", "apps/events/frontend"]`. **Done 2026-11-12, and changed from the first
|
||
plan:** `apps/events` itself is **not** a workspace. npm doesn't create `node_modules` for a
|
||
workspace that sits inside another workspace's folder, so any package it nested under
|
||
`apps/events/frontend/node_modules` (e.g. `@hookform/resolvers`) was silently missing. Its dev
|
||
tools (`@playwright/test`, `@axe-core/playwright`, `concurrently`) moved to the root
|
||
`devDependencies`; its scripts still run from `apps/events` (npm finds the root's
|
||
`node_modules/.bin`). Its `build` and `test` scripts are renamed `build:all` and `test:backend`
|
||
so nothing would run twice if it became a workspace again.
|
||
- **`allowScripts` moved to the root** `package.json` (npm 11 ignores it in a workspace), with
|
||
the same choices as before: Prisma, its engines, `@sentry/cli` and `unrs-resolver` allowed;
|
||
`@parcel/watcher`, `@scarf/scarf` and `esbuild` not.
|
||
- Rename the packages so workspace names are unique: `@crosscode/events` (was `hope-events`),
|
||
`@crosscode/events-backend`, `@crosscode/events-frontend`. Nothing reads the names. The Hub
|
||
reads the backend's `version` (`backend/src/hub/report.js`), and that stays. All three go to
|
||
`2.2.0`.
|
||
- **One lockfile at the root.** Delete `backend/package-lock.json` and
|
||
`frontend/package-lock.json`, then run `npm install` at the root. **Versions must not drift:**
|
||
compare `npm ls --all` before and after (next, react, prisma, @prisma/client, express,
|
||
typescript, jest, ts-jest, @sentry/nextjs, pg-boss, and anything else resolved differently).
|
||
Pin anything that moved with `overrides`. Keep the existing `shell-quote` override at the root.
|
||
**Done 2026-11-12:** every direct dependency of the backend and frontend that moved is pinned
|
||
to its 2.1.1 version in the root `overrides` (34 of them, e.g. `next` 16.4.3, `nodemailer`
|
||
10.3.0, the Radix packages), with `@sentry/node` pinned **only inside
|
||
`@crosscode/events-backend`** (10.71.0), since the frontend's `@sentry/nextjs` uses 10.78.0 and
|
||
Sentry packages must match. Not pinned: `eslint` (the root lint declares `^10.13.0`, so npm
|
||
refuses an override; 10.13 → 10.14, lint only), and about 50 transitive packages per app on
|
||
newer patch / minor versions (mostly build tooling: Babel, Rollup, browserslist). They can't
|
||
all be pinned, because the backend and frontend no longer have separate trees. The test
|
||
comparison (§6) is the check. Once 2.2.0 is out, these pins can be loosened in a normal
|
||
release.
|
||
- **Hoisting traps to check:**
|
||
- Prisma's `prisma-client-js` generates into the hoisted `node_modules/.prisma/client`. That's
|
||
fine while Events is the only Prisma app. **Accounts (phase 3) must use its own generator
|
||
`output`**, otherwise the two overwrite each other locally. On servers each checkout builds
|
||
only one app, so the problem doesn't arise there. Events keeps the default, so no imports
|
||
change in 2.2.0.
|
||
- Next.js 16 finds the workspace root from the lockfile. Once the nested lockfiles are gone,
|
||
there's no "multiple lockfiles" warning, and `next build` traces files from the repo root.
|
||
Check that `.next` still builds to `apps/events/frontend/.next`, and that `NEXT_DIST_DIR`
|
||
(e2e) still works.
|
||
- `@types/node` 20, TypeScript 5, Jest 30 and ESLint 10 are the same majors on both sides.
|
||
- **Lint:** the root `eslint .` ignores `apps/events/**` (add it to `globalIgnores`). Events
|
||
keeps its own configs: `frontend/eslint.config.mjs`, plus `tsc` for the backend. Moving Events
|
||
onto the root lint is a later cleanup, not part of 2.2.0.
|
||
- **Root `npm run check`** runs Events' scripts through `--workspaces --if-present`: backend
|
||
`test` and `typecheck`, frontend `build`. Make sure the backend tests run with no database, as
|
||
they do today. If any need one, give them their own script so the root `check` stays
|
||
self-contained.
|
||
|
||
## 3. Inside `apps/events`
|
||
|
||
Paths mostly carry over, because everything in Events is relative to its own folder:
|
||
|
||
- **`.gitignore`** moves to `apps/events/.gitignore`. Git reads a nested `.gitignore` relative to
|
||
its own folder, so `/ecosystem.config.js`, `backend/public/uploads/`, `backend/backups/` and
|
||
`backend/data/*.json` keep working unchanged. Check with `git status --ignored` on a copy of a
|
||
real install.
|
||
- **`ecosystem.config.js`** uses `__dirname + '/backend'`. Placed in `apps/events/`, it needs no
|
||
change. `adopt` rewrites only installs whose copy has absolute `cwd` paths.
|
||
- **Playwright** (`apps/events/playwright.config.ts`) keeps its `cwd: "backend"` / `"frontend"`.
|
||
Run it from `apps/events`.
|
||
- **Docs:** `ROADMAP-v3.md` and `hub-protocol.md` already live in the root `docs/`. Remove their
|
||
pointer stubs from `apps/events/docs/` in the merge. Code comments that cite
|
||
`docs/hub-protocol.md` then point at the root copy. `apps/events/docs/HANDOVER.md` keeps the
|
||
Events-only parts (where the code lives, known gaps), and the cross-product status lives in the
|
||
root [HANDOVER.md](./HANDOVER.md).
|
||
- **`CHANGELOG.md` and `WHATS-NEW.md`** stay in `apps/events/` (the Hub reads
|
||
`apps/events/WHATS-NEW.md`). The 2.2.0 entry in `WHATS-NEW.md` has no `— requires`.
|
||
- **`README.md` and `DEPLOYMENT.md`:** rewrite the deploy and install sections for `crosscode`.
|
||
The nginx and PM2 background stays.
|
||
- `scripts/deploy.sh` and `scripts/install.sh` are **removed** from `apps/events`. The archived
|
||
repo keeps them for installs that haven't moved yet.
|
||
|
||
## 4. The `crosscode` script
|
||
|
||
It lives in this repo's root `scripts/`. It's bash, like today's scripts, and is checked with
|
||
`shellcheck`. On a server it's a checkout of `main` in `/opt/crosscode/tools`, updated with
|
||
`git pull --ff-only` at the start of every run, then re-run from a copy (as `deploy.sh` does
|
||
today, so a checkout never changes the running script).
|
||
|
||
```
|
||
scripts/
|
||
crosscode entry: self-update, re-run from a copy, dispatch, `help`
|
||
lib/common.sh logging, root/app-user checks, church.env read/write, tag lookup
|
||
lib/deploy.sh checks → per product in order → summary
|
||
lib/install.sh new church (from today's install.sh)
|
||
lib/adopt.sh existing Events install → monorepo
|
||
lib/status.sh
|
||
products/events.sh the product's hooks (below); accounts.sh, website.sh, app.sh follow
|
||
tests/ bats tests for the parts that don't need a server
|
||
```
|
||
|
||
**Product hooks.** Each `products/<name>.sh` defines `backup`, `install_deps` (`npm ci
|
||
--include=dev -w apps/events/backend -w apps/events/frontend`), `migrate`, `build`, `pm2_names`,
|
||
`health_url` and `rollback_build`. With hooks, adding Accounts in phase 3 means adding a file,
|
||
not rewriting the deploy. For Events they come from today's `deploy.sh`: the backup through
|
||
`backupUtils.runBackup()`, `prisma migrate deploy`, the backend `tsc` build and `next build`.
|
||
|
||
**`church.env`** (`/var/www/sites/<church>/church.env`, mode 600, owned by root):
|
||
|
||
```
|
||
CHURCH=hopefamily
|
||
PRODUCTS="events" # accounts website app as they're added
|
||
ADDRESS=hopefamily.co.za # or hopefamily.crosscode.co.za
|
||
EVENTS_DIR=/var/www/sites/hopefamily/events
|
||
EVENTS_DOMAIN=events.hopefamily.co.za
|
||
EVENTS_PM2="hope-events-backend hope-events-frontend"
|
||
EVENTS_PORTS="5000 3000" # api web
|
||
EVENTS_DB=hope_events
|
||
EVENTS_USER=hopeevents
|
||
```
|
||
|
||
Secrets stay in each product's own `.env`, never in `church.env`.
|
||
|
||
**Commands in 2.2.0:**
|
||
|
||
| Command | In 2.2.0 |
|
||
|---------|----------|
|
||
| `deploy <church> [events]` | Yes. Accounts is skipped while a church has none (§8). Tags are `events@x.y.z`, newest by version (`git tag -l 'events@*' --sort=-v:refname`) |
|
||
| `deploy … --version events@x.y.z`, `--main`, `--check` | Yes |
|
||
| `deploy --every-church …` | Yes (it's a loop over `/var/www/sites/*/church.env`) |
|
||
| `status [<church>]` | Yes: version from `/health`, latest tag, DB, jobs, last backup |
|
||
| `install <church> --products events --domain …` | Yes, for Events only (from today's `install.sh`: Node 24, PostgreSQL, PM2, nginx, certbot); it writes `church.env` |
|
||
| `adopt <church> --events-dir …` | Yes (section 5) |
|
||
| `adopt <church> --undo` | Yes (decision 26): goes back to the old folder and the 2.1.x it ran, within the days the old folder is kept |
|
||
| `add <church> <product>` | **Later** (phase 3), when there's a second product to add |
|
||
|
||
**Built 2026-11-12** (branch `feat/crosscode-script`), as above, with these details settled while
|
||
writing it:
|
||
|
||
- **Run as root.** `church.env` is root-only (mode 600), so every command needs root; each app's
|
||
commands (git, npm, Prisma, PM2) run as its app user through `sudo -u`.
|
||
- `church.env` is read as data, never sourced: only `CHURCH`, `PRODUCTS`, `ADDRESS` and
|
||
`<PRODUCT>_*` keys are taken.
|
||
- `--allow-local-changes` replaces `ALLOW_LOCAL_CHANGES=1` (the variable still works). `--check`
|
||
never passes with changes.
|
||
- "Latest" only counts plain `events@x.y.z` tags, so test tags such as `events@2.2.0-test` are
|
||
only ever deployed with `--version`.
|
||
- `--every-church` stops at the first church that fails (its products roll back as usual), and
|
||
names the churches it didn't reach.
|
||
- `install`: the app user defaults to `cc-<church>`, PM2 names to `<church>-events-backend` /
|
||
`-frontend`, the database to `<church>_events`, and ports to the first free ones from 5000
|
||
(API) and 3000 (web), counting every church's `church.env` and anything listening.
|
||
- `adopt` writes `church.env` only once the switch has worked, and refuses to start if nginx
|
||
refers to the old folder by path (it gets renamed). It also records `EVENTS_OLD_DIR`,
|
||
`EVENTS_OLD_REF` and `EVENTS_ADOPTED_AT` for `--undo` and `status`.
|
||
- `adopt --undo` refuses if the new checkout has migrations the old release doesn't have, and
|
||
moves `/var/www/sites/<church>` aside to `.undone-<church>-<time>` (not deleted), so `adopt`
|
||
can be run again.
|
||
- Tests: `npm run lint:scripts` (shellcheck) and `npm run test:scripts` (bats: 47 tests,
|
||
including deploys against a local git origin with stub hooks for the checks, requirements,
|
||
order, rollback and `--every-church`). Both are part of `npm run check`.
|
||
|
||
**Requirement check** (decision 24): before anything is touched, read the target version's
|
||
heading in `apps/<product>/WHATS-NEW.md`. Refuse the deploy if a `— requires <product> <version>`
|
||
isn't met by the installed sibling or by this same run. Nothing in 2.2.0 requires anything, but
|
||
the check is built and tested now.
|
||
|
||
**A deploy, per product** (as `deploy.sh` today): refuse if files were changed by hand →
|
||
database backup → `git fetch --tags` and check out the tag (detached) → install and build that
|
||
app only → migrations → `pm2 reload` → poll `/health` until it reports the new version (60 s).
|
||
On failure that product goes back to its previous tag and rebuilds, and the run stops. The
|
||
summary lists each product as updated, rolled back or not reached.
|
||
|
||
## 5. `crosscode adopt` (once per existing install)
|
||
|
||
**Decided 2026-11-05 (ROADMAP decision 25):** build the new checkout **next to** the running
|
||
install, then switch over, instead of moving the running folder. Moving the folder underneath running PM2
|
||
processes breaks anything they read from disk later (Next.js pages, uploads), so the site would
|
||
be down for the whole build. Building next to it keeps the old install untouched, which makes
|
||
rolling back simply "start the old one again".
|
||
|
||
1. **Checks:** the old folder is a `hope-events` checkout at a `v2.1.x` tag with no files changed
|
||
by hand, and `/var/www/sites/<church>/events` doesn't exist (or is already an adopted
|
||
checkout, in which case stop: nothing to do).
|
||
2. **Read what the install already has:** the PM2 names and ports from `ecosystem.config.js`, and
|
||
the database name from `backend/.env` `DATABASE_URL`. Write `church.env`.
|
||
3. **New checkout** of this repo at `events@2.2.0` in `/var/www/sites/<church>/events`, owned by
|
||
the app user.
|
||
4. **Copy the install's own files** into `apps/events/`: `backend/.env`, `frontend/.env*`,
|
||
`ecosystem.config.js` (with absolute `cwd`s rewritten), `backend/data/*.json`, and
|
||
`backend/backups/`. Then `rsync -a backend/public/uploads/`, which may be large; the old site
|
||
is still serving meanwhile.
|
||
5. **Build** (`npm ci -w …`, backend build, `next build`). The old install keeps running.
|
||
6. **Switch over** (the only downtime, a few seconds): database backup → a final
|
||
`rsync -a --delete` of the uploads and `data/*.json` → `prisma migrate status` must show
|
||
nothing pending (2.2.0 adds no migrations; refuse if it does) → `pm2 delete` the two names →
|
||
`pm2 start ecosystem.config.js` from the new folder → `pm2 save` → `/health` reports 2.2.0.
|
||
7. **On failure at step 6:** `pm2 delete` → `pm2 start` from the old folder → `pm2 save`. Nothing
|
||
in the old folder was changed, so it comes back exactly as it was.
|
||
8. **Keep the old folder**, renamed `<old>.pre-2.2.0`, for 14 days. `adopt --undo` uses it.
|
||
After that, `crosscode status` reminds you to remove it.
|
||
|
||
The nginx config doesn't change (it only proxies to ports), and neither do the PM2 names, the
|
||
database or its name.
|
||
|
||
## 6. Testing
|
||
|
||
**Done 2026-11-17** on a WSL Ubuntu 24.04 VM (steps 1–7 below; step 8 in the agent test). All
|
||
pass after two fixes: `adopt` refused a release with new migrations (it now applies them in the
|
||
switch-over), and a failed migration blocked later deploys (now marked rolled back).
|
||
|
||
- **Events, unchanged behaviour:** backend Jest, `tsc --noEmit`, frontend build and the
|
||
Playwright e2e run, all from the monorepo. They should give the same counts as the 2.1.1
|
||
baseline.
|
||
- **Script:** `shellcheck`, plus bats tests for tag lookup, the `— requires` parsing,
|
||
`church.env` reading and writing, and the rewriting of `ecosystem.config.js`.
|
||
- **On a throwaway Ubuntu 24.04 VM** (e.g. multipass), in this order:
|
||
1. `install.sh` from the archived repo at `v2.1.1`. Add demo data and a few uploads.
|
||
2. `crosscode adopt` → 2.2.0 works, the uploads are there, and the database is unchanged.
|
||
3. `adopt --undo` → back on 2.1.1. Then `adopt` again.
|
||
4. `crosscode deploy <church> events --version events@2.2.0-test` with a broken migration on a
|
||
test tag → it rolls back to 2.2.0 and the site stays up.
|
||
5. A file changed by hand → `deploy --check` refuses it.
|
||
6. A second church with `crosscode install` on the same VM → separate databases, `.env` files,
|
||
uploads and PM2 names. `deploy --every-church` updates both.
|
||
7. `crosscode status` for both.
|
||
8. The Hub (local, port 5200) sees the install's heartbeat as 2.2.0, and `events@2.2.0` under
|
||
Releases.
|
||
|
||
## 7. Rolling it out
|
||
|
||
1. **`hope-events`, last commit on `main`:** README "Moved to `crosscode-platform/apps/events`;
|
||
this repo only keeps 2.1.x for installs that haven't moved". No tag.
|
||
2. **Here:** merge `move/events` → `npm run check` → tag `events@2.2.0` → push with tags. Remove
|
||
any test tags first.
|
||
3. **Hub:** run release sync. Check that Cross Events shows 2.2.0 and the older `v…` releases
|
||
are still listed.
|
||
4. **Servers:** set up `/opt/crosscode/tools`. Adopt a staging copy first, then each real
|
||
install one at a time, checking `crosscode status` after each. Installs on 2.1.1 see 2.2.0 on
|
||
the Hub. Their "Request update" is answered with `adopt`, not `deploy`.
|
||
5. **When every install reports 2.2.0:** archive `hope-events` on Gitea (read-only; cloning and
|
||
fetching still work, so `adopt --undo` and old installs keep working).
|
||
6. **crosscode-site:** nothing changes (ROADMAP §9).
|
||
7. **Docs:** a 2.2.0 entry in `apps/events/CHANGELOG.md` and the root `CHANGELOG.md`, and the
|
||
root [HANDOVER.md](./HANDOVER.md) updated with 1b done and 1c (2.3.0, authenticator app 2FA)
|
||
next.
|
||
|
||
## Decisions
|
||
|
||
1. `adopt` builds next to the old install and switches over (section 5); ROADMAP §8 updated to
|
||
match (decision 25).
|
||
2. `adopt --undo` is built (decision 26).
|
||
|
||
## Still open
|
||
|
||
- Do any Events backend tests need a database, which would affect the root `npm run check`?
|
||
(Answered while doing section 2.)
|