diff --git a/CHANGELOG.md b/CHANGELOG.md index 602082d..f6cbb29 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,21 @@ and this project follows [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Added + +- Registration funnel stats (first-party, no third-party analytics), showing where people drop off between seeing an event and paying. + - A new cookie notice asks visitors once. Only those who choose "Allow" are counted, using a random anonymous visit id. + - We record the event page view and the start of registration. The submit and paid stages come from the registration itself. + - Only the stage, the traffic source and the device type are stored. No name, IP address or User-Agent is kept. +- New report, **Registration funnel**: counts and conversion % per stage over a date range, a daily trend, and breakdowns by traffic source, device and ticket option. + - Selecting several events compares them. + - Sources come from `utm_source` links and from short links, which now add `?src=` when redirecting. +- A "Conversion (past month)" tile on the supervisor and admin dashboards covers live events. +- Raw visit records are rolled up into daily totals after 90 days by a daily job (`ANALYTICS_ROLLUP_ENABLED=false` to turn it off). +- The privacy policy's cookies section now describes these statistics. +- New `/api/analytics` endpoints. + - Requires the `add_funnel_analytics` database migration. + ## [1.21.0] - 2026-10-19 ### Added diff --git a/FEATURES.md b/FEATURES.md index 77a3a92..58616d9 100644 --- a/FEATURES.md +++ b/FEATURES.md @@ -171,6 +171,17 @@ Items & Pricing → Sections → Form → Visibility → Attachments**. Each payment records **who recorded it** as well as who it's for. - **Refunds** — negative payments against an original payment, netted against the original method. +- **Registration funnel stats** — first-party and consent-based, with no third-party analytics. + - A cookie notice asks once. Only visitors who allow it get a random visit id. + - We record event page views and registration starts. Submits and payments come + from the registration itself. + - Each record holds only the stage, the traffic source (utm_source, or the short-link + alias, which short links now pass as `?src=`) and the device type. + - The Registration funnel report shows counts and conversion per stage over a date + range, a daily trend, and breakdowns by source, device and ticket option. + - A "Conversion (past month)" tile on the supervisor/admin dashboards covers live + events. + - Raw rows are rolled up into daily totals after 90 days. - **Post-event feedback** — a Feedback step in the event wizard. - Questions use the form builder, with new **star rating (1–5)** and **multiple choice** types (also available on attendee forms). @@ -404,7 +415,7 @@ a viewer with its own filters, stat tiles and bar charts. | Category | Reports | |----------|---------| | Financial | Payments between dates · Revenue summary (by method) · Revenue detailed · Finance report (revenue & costs, after discounts) · Profit report · Discounts (coupon codes) | -| Registration | Attendees per event (grouped by option) · Registration type counts · Master orders breakdown · Registration status breakdown (count by registration or ticket quantity) · Parental consent status · Medical & emergency list · Feedback summary (response rate, average ratings, choice breakdowns, comments; compare events) | +| Registration | Registration funnel (views → started → submitted → paid, daily trend, by source / device / ticket option) · Attendees per event (grouped by option) · Registration type counts · Master orders breakdown · Registration status breakdown (count by registration or ticket quantity) · Parental consent status · Medical & emergency list · Feedback summary (response rate, average ratings, choice breakdowns, comments; compare events) | | Ticketing | Ticket usage summary | | Donations | Donations breakdown (used / unused per event, plus the sponsorship pool) | | Cashup | Cashup reconciliation · Cashup audit trail | @@ -477,6 +488,7 @@ falling back to a general guide. | Scheduled email/WhatsApp worker | Polls every 30 s (`SCHEDULED_EMAILS_INTERVAL_MS`) and purges sent jobs after 24 h | | Daily event summary email | 07:00 daily | | Waitlist: expire unclaimed offers, offer freed places | Every 10 minutes (`WAITLIST_JOB_ENABLED=false` to disable) | +| Funnel analytics: roll raw visit rows older than 90 days into daily totals | Daily (`ANALYTICS_ROLLUP_ENABLED=false` to disable) | | Post-event feedback forms and their 3-day reminder | Every 15 minutes (`FEEDBACK_JOB_ENABLED=false` to disable) | | Parental consent reminders, and deleting medical details after events | 09:00 daily (`CONSENT_JOBS_ENABLED=false` to disable) | | Database backup (`pg_dump`, gzipped, 14-day retention) | 02:00 daily (`BACKUP_ENABLED=false` to disable) | diff --git a/backend/API_DOCUMENTATION.md b/backend/API_DOCUMENTATION.md index 7a0d7a3..6e26cfc 100644 --- a/backend/API_DOCUMENTATION.md +++ b/backend/API_DOCUMENTATION.md @@ -26,6 +26,7 @@ Tokens are issued by `POST /api/users/login`. - [Rooms & groups](#rooms--groups) - [Sponsor a spot](#sponsor-a-spot) - [Post-event feedback](#post-event-feedback) +- [Registration funnel](#registration-funnel) - [Payments](#payments) - [Tickets](#tickets) - [Broadcasts (Email)](#broadcasts-email) @@ -283,6 +284,22 @@ For unauthenticated (guest) registration on events with `requiresAuth: false`: --- +## Registration funnel + +| Method | Path | Auth | Description | +|--------|------|------|-------------| +| POST | `/api/analytics/track` | public (rate-limited) | `{ sessionId, eventId, stage: 'view'\|'start', source? }` → always `204`. Bots are ignored; device type comes from the User-Agent | +| GET | `/api/analytics/funnel?eventIds=&from=&to=` | supervisor+ | `{ totals, conversion, trend, bySource, byDevice, byOption, perEvent }` | + +How the stages are recorded: +- `POST /api/registrations` accepts `analyticsSessionId`. It's stored on the registration and records the `submit` stage. +- `paid` is read from that registration's status. +- Counts are sessions that reached a stage or a later one. + +Data handling: +- Sessions inactive for 90 days are rolled up into `AnalyticsDaily` by a daily job (`ANALYTICS_ROLLUP_ENABLED`). +- `/api/stats/overview` now includes `funnel: { views, paid, conversion }` for live events over the past 30 days. + ## Post-event feedback | Method | Path | Auth | Description | diff --git a/backend/prisma/migrations/20261019180220_add_funnel_analytics/migration.sql b/backend/prisma/migrations/20261019180220_add_funnel_analytics/migration.sql new file mode 100644 index 0000000..4172326 --- /dev/null +++ b/backend/prisma/migrations/20261019180220_add_funnel_analytics/migration.sql @@ -0,0 +1,47 @@ +-- AlterTable +ALTER TABLE "Registration" ADD COLUMN "analyticsSessionId" TEXT; + +-- CreateTable +CREATE TABLE "AnalyticsEvent" ( + "id" TEXT NOT NULL, + "sessionId" TEXT NOT NULL, + "eventId" TEXT NOT NULL, + "stage" TEXT NOT NULL, + "source" TEXT, + "device" TEXT, + "createdAt" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP, + + CONSTRAINT "AnalyticsEvent_pkey" PRIMARY KEY ("id") +); + +-- CreateTable +CREATE TABLE "AnalyticsDaily" ( + "id" TEXT NOT NULL, + "eventId" TEXT NOT NULL, + "day" DATE NOT NULL, + "stage" TEXT NOT NULL, + "source" TEXT NOT NULL DEFAULT 'direct', + "device" TEXT NOT NULL DEFAULT 'unknown', + "sessions" INTEGER NOT NULL DEFAULT 0, + + CONSTRAINT "AnalyticsDaily_pkey" PRIMARY KEY ("id") +); + +-- CreateIndex +CREATE INDEX "AnalyticsEvent_eventId_createdAt_idx" ON "AnalyticsEvent"("eventId", "createdAt"); + +-- CreateIndex +CREATE UNIQUE INDEX "AnalyticsEvent_sessionId_eventId_stage_key" ON "AnalyticsEvent"("sessionId", "eventId", "stage"); + +-- CreateIndex +CREATE INDEX "AnalyticsDaily_eventId_day_idx" ON "AnalyticsDaily"("eventId", "day"); + +-- CreateIndex +CREATE UNIQUE INDEX "AnalyticsDaily_eventId_day_stage_source_device_key" ON "AnalyticsDaily"("eventId", "day", "stage", "source", "device"); + +-- AddForeignKey +ALTER TABLE "AnalyticsEvent" ADD CONSTRAINT "AnalyticsEvent_eventId_fkey" FOREIGN KEY ("eventId") REFERENCES "Event"("id") ON DELETE CASCADE ON UPDATE CASCADE; + +-- AddForeignKey +ALTER TABLE "AnalyticsDaily" ADD CONSTRAINT "AnalyticsDaily_eventId_fkey" FOREIGN KEY ("eventId") REFERENCES "Event"("id") ON DELETE CASCADE ON UPDATE CASCADE; + diff --git a/backend/prisma/schema.prisma b/backend/prisma/schema.prisma index ac4ff54..a65f1da 100644 --- a/backend/prisma/schema.prisma +++ b/backend/prisma/schema.prisma @@ -153,6 +153,8 @@ model Event { sponsorshipGoalSpots Int? showSponsorshipProgress Boolean @default(true) feedbackForm FeedbackForm? + analyticsEvents AnalyticsEvent[] + analyticsDaily AnalyticsDaily[] allocationSets AllocationSet[] waitlistOfferHours Int @default(24) waitlistPartialOffers Boolean @default(true) @@ -266,6 +268,7 @@ model Registration { discountTotal Float @default(0) // Where named attendees' tickets go: all to the payer, or each to the attendee. ticketDelivery TicketDelivery @default(payer) + analyticsSessionId String? createdAt DateTime @default(now()) updatedAt DateTime @updatedAt user User @relation(fields: [userId], references: [id], onDelete: Restrict) @@ -851,6 +854,39 @@ enum TicketDelivery { // One named person on a group registration, for one main-ticket place. Gets their // own quantity-1 ticket and their own attendee form response. +/// First-party funnel tracking (only after cookie consent): one row per anonymous +/// session, event and stage ('view' | 'start' | 'submit'). "Paid" comes from the +/// registration that carries the session id. Rows older than 90 days are rolled up +/// into AnalyticsDaily and deleted. +model AnalyticsEvent { + id String @id @default(uuid()) + sessionId String + eventId String + stage String + source String? + device String? + createdAt DateTime @default(now()) + event Event @relation(fields: [eventId], references: [id], onDelete: Cascade) + + @@unique([sessionId, eventId, stage]) + @@index([eventId, createdAt]) +} + +/// Rolled-up funnel counts: sessions that reached `stage`, per day, source and device. +model AnalyticsDaily { + id String @id @default(uuid()) + eventId String + day DateTime @db.Date + stage String + source String @default("direct") + device String @default("unknown") + sessions Int @default(0) + event Event @relation(fields: [eventId], references: [id], onDelete: Cascade) + + @@unique([eventId, day, stage, source, device]) + @@index([eventId, day]) +} + /// Post-event survey. Sent at sendAt to checked-in attendees (or everyone registered), /// with one reminder 3 days later. anonymous responses aren't linked to a person. model FeedbackForm { diff --git a/backend/src/controllers/analyticsController.js b/backend/src/controllers/analyticsController.js new file mode 100644 index 0000000..1e6eac4 --- /dev/null +++ b/backend/src/controllers/analyticsController.js @@ -0,0 +1,44 @@ +const prisma = require('../config/db'); +const { safeErrorMessage } = require('../utils/errorUtils'); +const { track, funnelFor } = require('../utils/analytics'); + +// @desc Record a funnel step (event page view, registration started) +// @route POST /api/analytics/track { sessionId, eventId, stage: 'view'|'start', source? } +// @access Public — only sent by browsers whose visitor accepted cookies +const trackStage = async (req, res) => { + try { + const { sessionId, eventId, stage, source } = req.body || {}; + await track({ sessionId, eventId, stage, source, userAgent: req.headers['user-agent'] }); + } catch (e) { + console.warn('[analytics] track failed:', e?.message || e); + } + // Always a quiet 204 — tracking must never get in a visitor's way. + res.status(204).end(); +}; + +// @desc Registration funnel for one or more events +// @route GET /api/analytics/funnel?eventIds=a,b&from=YYYY-MM-DD&to=YYYY-MM-DD +// @access Private/Supervisor +const getFunnel = async (req, res) => { + try { + const eventIds = String(req.query.eventIds || '').split(',').map(s => s.trim()).filter(Boolean).slice(0, 50); + if (eventIds.length === 0) { res.status(400); throw new Error('Choose at least one event'); } + const events = await prisma.event.findMany({ where: { id: { in: eventIds } }, select: { id: true, title: true } }); + const total = await funnelFor(eventIds, { from: req.query.from, to: req.query.to }); + // Per event too, so events can be compared. + const perEvent = []; + if (events.length > 1) { + for (const ev of events) { + const f = await funnelFor([ev.id], { from: req.query.from, to: req.query.to }); + perEvent.push({ eventId: ev.id, title: ev.title, totals: f.totals, conversion: f.conversion }); + } + } else if (events[0]) { + perEvent.push({ eventId: events[0].id, title: events[0].title, totals: total.totals, conversion: total.conversion }); + } + res.json({ ...total, perEvent }); + } catch (error) { + res.status(res.statusCode === 200 ? 400 : res.statusCode).json({ message: safeErrorMessage(error) }); + } +}; + +module.exports = { trackStage, getFunnel }; diff --git a/backend/src/controllers/registrationController.js b/backend/src/controllers/registrationController.js index 465afd7..9b281f0 100644 --- a/backend/src/controllers/registrationController.js +++ b/backend/src/controllers/registrationController.js @@ -110,7 +110,7 @@ async function checkVariantStock(variant, requestedQty, excludeEntryId = null) { // @access Private const createRegistration = async (req, res) => { try { - const { eventId, options, guestName, guestEmail, guestPhone, couponCode, attendees, ticketDelivery, waitlistToken } = req.body; + const { eventId, options, guestName, guestEmail, guestPhone, couponCode, attendees, ticketDelivery, waitlistToken, analyticsSessionId } = req.body; // Check if event exists const canIncludeVariants = !!(prisma && prisma.optionVariant && typeof prisma.optionVariant.findMany === 'function'); @@ -503,6 +503,12 @@ const createRegistration = async (req, res) => { } })(); + // Funnel: this registration came from a tracked (cookie-consented) visit. + if (analyticsSessionId) { + const { recordSubmit } = require('../utils/analytics'); + recordSubmit(registration, analyticsSessionId).catch(e => console.warn('[analytics] submit not recorded:', e?.message || e)); + } + res.status(201).json({ ...attachComputedTotals(registration), ...(couponError ? { couponError } : {}) }); } catch (error) { res.status(400).json({ message: error.message }); diff --git a/backend/src/controllers/statsController.js b/backend/src/controllers/statsController.js index 4ecab53..e4af3d3 100644 --- a/backend/src/controllers/statsController.js +++ b/backend/src/controllers/statsController.js @@ -267,7 +267,21 @@ const getOverviewStats = async (req, res) => { ticketsSold: ticketSumById[r.eventId] || 0, })); + // Funnel conversion (event page views → paid) across live events, last 30 days. + let funnel = null; + try { + const live = await prisma.event.findMany({ where: { isActive: true, endDate: { gte: now } }, select: { id: true } }); + if (live.length) { + const { funnelFor } = require('../utils/analytics'); + const f = await funnelFor(live.map(e => e.id), { from: new Date(now.getTime() - 30 * 864e5).toISOString() }); + if (f.totals.view > 0) funnel = { views: f.totals.view, paid: f.totals.paid, conversion: f.conversion.paid }; + } + } catch (e) { + console.warn('[stats] funnel tile failed:', e?.message || e); + } + res.json({ + funnel, activeEvents, registrations: { thisMonth: thisRegistrations, lastMonth: lastRegistrations, pctChange: pctChange(thisRegistrations, lastRegistrations) }, ticketsSold: { thisMonth: thisTicketsSold, lastMonth: lastTicketsSold, pctChange: pctChange(thisTicketsSold, lastTicketsSold) }, diff --git a/backend/src/index.js b/backend/src/index.js index 7d18f86..3d50593 100644 --- a/backend/src/index.js +++ b/backend/src/index.js @@ -182,6 +182,7 @@ app.use('/api/scan-locks', require('./routes/scanLockRoutes')); app.use('/api/allocations', require('./routes/allocationRoutes')); app.use('/api/sponsorships', require('./routes/sponsorshipRoutes')); app.use('/api/feedback', require('./routes/feedbackRoutes')); +app.use('/api/analytics', require('./routes/analyticsRoutes')); app.use('/api/backups', backupRoutes); // Pre-warm the settings cache so synchronous helpers have DB values from startup @@ -638,6 +639,11 @@ app.get('/docs', async (req, res) => { ]}, ]}, + { title: 'Registration funnel', base: '/api/analytics', endpoints: [ + { method:'POST', path:'/api/analytics/track', auth:'Public (rate-limited)', desc:'Record a funnel step from a cookie-consented browser (always 204)', + request:{ body:{ sessionId:'random-uuid', eventId:'ev-uuid-...', stage:'view', source:'camp-2025' }}}, + { method:'GET', path:'/api/analytics/funnel', auth:'supervisor+', desc:'Funnel for ?eventIds=a,b&from=&to= — totals, conversion, daily trend, by source/device/option' }, + ]}, { title: 'Post-event feedback', base: '/api/feedback', endpoints: [ { method:'GET', path:'/api/feedback/event/:eventId', auth:'supervisor+', desc:'An event\'s feedback form, with invite and response counts' }, { method:'PUT', path:'/api/feedback/event/:eventId', auth:'supervisor+', desc:'Create/update the feedback form (questions lock once answered)', @@ -1399,6 +1405,29 @@ app.listen(PORT, () => { console.warn('[waitlist] Not scheduled:', e?.message || e); } + // Funnel analytics — once a day, roll raw rows older than 90 days into daily totals. + try { + const enabled = String(process.env.ANALYTICS_ROLLUP_ENABLED || 'true').toLowerCase() !== 'false'; + if (enabled) { + const { rollupOldEvents } = require('./utils/analytics'); + const run = async () => { + try { + const n = await rollupOldEvents(); + if (n) console.log(`[analytics] Rolled up ${n} session(s) older than 90 days.`); + } catch (e) { + console.warn('[analytics] Rollup failed:', e?.message || e); + } + }; + setTimeout(run, 5 * 60 * 1000); + setInterval(run, 24 * 60 * 60 * 1000); + console.log('[analytics] Rollup daily.'); + } else { + console.log('[analytics] Rollup disabled by env ANALYTICS_ROLLUP_ENABLED=false'); + } + } catch (e) { + console.warn('[analytics] Not scheduled:', e?.message || e); + } + // Feedback job — send post-event feedback forms at their send time, and the one // reminder 3 days later. try { diff --git a/backend/src/routes/analyticsRoutes.js b/backend/src/routes/analyticsRoutes.js new file mode 100644 index 0000000..2ff8751 --- /dev/null +++ b/backend/src/routes/analyticsRoutes.js @@ -0,0 +1,18 @@ +const express = require('express'); +const rateLimit = require('express-rate-limit'); +const router = express.Router(); +const { trackStage, getFunnel } = require('../controllers/analyticsController'); +const { protect, supervisor } = require('../middleware/authMiddleware'); + +const trackLimiter = rateLimit({ + windowMs: 60 * 1000, + max: 60, + standardHeaders: true, + legacyHeaders: false, + handler: (req, res) => res.status(204).end(), // silently drop floods +}); + +router.post('/track', trackLimiter, trackStage); +router.get('/funnel', protect, supervisor, getFunnel); + +module.exports = router; diff --git a/backend/src/utils/analytics.js b/backend/src/utils/analytics.js new file mode 100644 index 0000000..999df2f --- /dev/null +++ b/backend/src/utils/analytics.js @@ -0,0 +1,220 @@ +/** + * Registration funnel — first-party, consent-based tracking. + * + * The browser only tracks after the visitor accepts cookies: it keeps a random session + * id and reports 'view' (event page) and 'start' (registration page). 'submit' is + * recorded here when a registration arrives carrying that session id, and 'paid' is + * read from that registration's status, so no personal data is stored with the events — + * just a stage, a traffic source and a device type (worked out from the User-Agent, + * which isn't kept). + * + * Counts are "reached": a session counts at a stage if it got that far or further, so a + * visitor who went straight to the registration page still counts as a view. Raw rows + * older than 90 days are rolled up into AnalyticsDaily and deleted. + */ + +const prisma = require('../config/db'); + +const STAGES = ['view', 'start', 'submit', 'paid']; +const TRACKED = ['view', 'start']; +const RAW_DAYS = 90; +const SESSION_RE = /^[a-zA-Z0-9-]{16,64}$/; + +const dayOf = (d) => new Date(d).toISOString().slice(0, 10); + +// ─── Pure helpers ───────────────────────────────────────────────────────────── + +const validSession = (sid) => typeof sid === 'string' && SESSION_RE.test(sid); + +/** 'mobile' | 'tablet' | 'desktop', or null for bots. Pure. */ +function deviceFromUA(ua) { + const s = String(ua || ''); + if (!s || /bot|crawl|spider|slurp|preview|headless|lighthouse/i.test(s)) return null; + if (/iPad|Tablet|PlayBook|Silk|(Android(?!.*Mobile))/i.test(s)) return 'tablet'; + if (/Mobi|iPhone|iPod|Android|Windows Phone/i.test(s)) return 'mobile'; + return 'desktop'; +} + +/** A traffic source label: utm_source / short-link alias, tidied; 'direct' when none. Pure. */ +function cleanSource(src) { + const s = String(src || '').trim().toLowerCase().replace(/[^a-z0-9_.-]/g, '').slice(0, 50); + return s || 'direct'; +} + +/** + * One entry per session: the furthest stage reached, its source/device, and the day each + * stage was reached. Pure. + * @param {Array} events - [{ sessionId, stage, createdAt, source, device }] + * @param {Set} paidSessions - session ids whose registration is paid + */ +function sessionsFrom(events, paidSessions = new Set()) { + const map = new Map(); + for (const e of [...events].sort((a, b) => new Date(a.createdAt) - new Date(b.createdAt))) { + const s = map.get(e.sessionId) || { sessionId: e.sessionId, reach: -1, source: null, device: null, days: {} }; + const idx = STAGES.indexOf(e.stage); + if (idx > s.reach) s.reach = idx; + if (!s.days[e.stage]) s.days[e.stage] = dayOf(e.createdAt); + if (!s.source && e.source) s.source = e.source; + if (!s.device && e.device) s.device = e.device; + map.set(e.sessionId, s); + } + for (const s of map.values()) { + if (paidSessions.has(s.sessionId) && s.reach >= 2) { s.reach = 3; s.days.paid = s.days.submit; } + // A stage reached without its own event (e.g. no 'view' for a direct link) takes the + // day of the next stage that was recorded. + for (let i = STAGES.length - 1; i >= 0; i--) { + if (i <= s.reach && !s.days[STAGES[i]]) s.days[STAGES[i]] = s.days[STAGES[i + 1]]; + } + s.source = s.source || 'direct'; + s.device = s.device || 'unknown'; + } + return [...map.values()]; +} + +/** Rows for AnalyticsDaily: sessions reaching each stage per day/source/device. Pure. */ +function rollupRows(sessions) { + const by = new Map(); + for (const s of sessions) { + for (let i = 0; i <= s.reach; i++) { + const key = [s.days[STAGES[i]], STAGES[i], s.source, s.device].join('|'); + by.set(key, (by.get(key) || 0) + 1); + } + } + return [...by.entries()].map(([key, n]) => { + const [day, stage, source, device] = key.split('|'); + return { day, stage, source, device, sessions: n }; + }); +} + +const emptyCounts = () => ({ view: 0, start: 0, submit: 0, paid: 0 }); + +/** + * The funnel report from raw sessions plus rolled-up rows. Pure. + * @returns {{ totals, conversion, trend, bySource, byDevice }} + */ +function funnelReport(sessions, daily = []) { + const rows = [...rollupRows(sessions), ...daily.map(d => ({ ...d, day: typeof d.day === 'string' ? d.day.slice(0, 10) : dayOf(d.day) }))]; + const totals = emptyCounts(); + const trend = new Map(), bySource = new Map(), byDevice = new Map(); + const add = (map, key, stage, n) => { const c = map.get(key) || emptyCounts(); c[stage] += n; map.set(key, c); }; + for (const r of rows) { + totals[r.stage] += r.sessions; + add(trend, r.day, r.stage, r.sessions); + add(bySource, r.source, r.stage, r.sessions); + add(byDevice, r.device, r.stage, r.sessions); + } + const pct = (a, b) => (b ? Math.round((a / b) * 1000) / 10 : null); + const list = (map) => [...map.entries()].map(([key, c]) => ({ key, ...c, conversion: pct(c.paid, c.view) })).sort((a, b) => b.view - a.view); + return { + totals, + conversion: { + start: pct(totals.start, totals.view), submit: pct(totals.submit, totals.view), paid: pct(totals.paid, totals.view), + startToSubmit: pct(totals.submit, totals.start), submitToPaid: pct(totals.paid, totals.submit), + }, + trend: [...trend.entries()].sort(([a], [b]) => a.localeCompare(b)).map(([day, c]) => ({ day, ...c })), + bySource: list(bySource), + byDevice: list(byDevice), + }; +} + +// ─── Database ───────────────────────────────────────────────────────────────── + +/** Records a 'view' or 'start'. Repeats from the same session are ignored. */ +async function track({ sessionId, eventId, stage, source, userAgent }) { + if (!validSession(sessionId) || !TRACKED.includes(stage) || typeof eventId !== 'string') return false; + const device = deviceFromUA(userAgent); + if (!device) return false; // bots + const event = await prisma.event.findUnique({ where: { id: eventId }, select: { id: true } }); + if (!event) return false; + await prisma.analyticsEvent.createMany({ data: [{ sessionId, eventId, stage, source: cleanSource(source), device }], skipDuplicates: true }); + return true; +} + +/** 'submit' for a registration made on the public registration page. */ +async function recordSubmit(registration, sessionId) { + if (!validSession(sessionId) || !registration?.id) return; + const first = await prisma.analyticsEvent.findFirst({ where: { sessionId, eventId: registration.eventId }, orderBy: { createdAt: 'asc' } }); + await prisma.registration.updateMany({ where: { id: registration.id, analyticsSessionId: null }, data: { analyticsSessionId: sessionId } }); + await prisma.analyticsEvent.createMany({ + data: [{ sessionId, eventId: registration.eventId, stage: 'submit', source: first?.source || 'direct', device: first?.device || null }], + skipDuplicates: true, + }); +} + +async function paidSessionsFor(eventIds, sessionIds) { + if (!sessionIds.length) return new Set(); + const regs = await prisma.registration.findMany({ + where: { eventId: { in: eventIds }, status: 'paid', analyticsSessionId: { in: sessionIds } }, + select: { analyticsSessionId: true }, + }); + return new Set(regs.map(r => r.analyticsSessionId)); +} + +/** + * The funnel for some events between two dates (by the day each stage was reached). + * Adds a by-ticket-option breakdown from the tracked registrations. + */ +async function funnelFor(eventIds, { from, to } = {}) { + const range = {}; + if (from) range.gte = new Date(from); + if (to) { const t = new Date(to); t.setHours(23, 59, 59, 999); range.lte = t; } + const where = { eventId: { in: eventIds }, ...(from || to ? { createdAt: range } : {}) }; + const events = await prisma.analyticsEvent.findMany({ where, select: { sessionId: true, stage: true, createdAt: true, source: true, device: true, eventId: true } }); + // A session is per event (the same browser can look at several events). + const keyed = events.map(e => ({ ...e, sessionId: `${e.eventId}:${e.sessionId}` })); + const paidRaw = await paidSessionsFor(eventIds, [...new Set(events.map(e => e.sessionId))]); + const paid = new Set(events.filter(e => paidRaw.has(e.sessionId)).map(e => `${e.eventId}:${e.sessionId}`)); + const daily = await prisma.analyticsDaily.findMany({ + where: { eventId: { in: eventIds }, ...(from || to ? { day: range } : {}) }, + select: { day: true, stage: true, source: true, device: true, sessions: true }, + }); + const report = funnelReport(sessionsFrom(keyed, paid), daily); + + const regs = await prisma.registration.findMany({ + where: { eventId: { in: eventIds }, analyticsSessionId: { not: null }, status: { not: 'cancelled' }, ...(from || to ? { createdAt: range } : {}) }, + select: { status: true, registrationOptions: { select: { quantity: true, eventOption: { select: { name: true } }, variant: { select: { name: true } } } } }, + }); + const byOption = new Map(); + for (const r of regs) { + for (const ro of r.registrationOptions) { + const key = ro.variant ? `${ro.eventOption.name} — ${ro.variant.name}` : ro.eventOption.name; + const c = byOption.get(key) || { key, submitted: 0, paid: 0 }; + c.submitted += 1; + if (r.status === 'paid') c.paid += 1; + byOption.set(key, c); + } + } + return { ...report, byOption: [...byOption.values()].sort((a, b) => b.submitted - a.submitted) }; +} + +/** Rolls up sessions whose last activity is older than 90 days, then deletes their rows. */ +async function rollupOldEvents(now = new Date()) { + const cutoff = new Date(now.getTime() - RAW_DAYS * 864e5); + const groups = await prisma.analyticsEvent.groupBy({ by: ['eventId', 'sessionId'], _max: { createdAt: true }, having: { createdAt: { _max: { lt: cutoff } } } }); + if (groups.length === 0) return 0; + const byEvent = new Map(); + for (const g of groups) byEvent.set(g.eventId, [...(byEvent.get(g.eventId) || []), g.sessionId]); + let rolled = 0; + for (const [eventId, sessionIds] of byEvent) { + for (let i = 0; i < sessionIds.length; i += 500) { + const batch = sessionIds.slice(i, i + 500); + const events = await prisma.analyticsEvent.findMany({ where: { eventId, sessionId: { in: batch } } }); + const paid = await paidSessionsFor([eventId], batch); + const rows = rollupRows(sessionsFrom(events, paid)); + await prisma.$transaction(async (tx) => { + for (const r of rows) { + await tx.analyticsDaily.upsert({ + where: { eventId_day_stage_source_device: { eventId, day: new Date(r.day), stage: r.stage, source: r.source, device: r.device } }, + update: { sessions: { increment: r.sessions } }, + create: { eventId, day: new Date(r.day), stage: r.stage, source: r.source, device: r.device, sessions: r.sessions }, + }); + } + await tx.analyticsEvent.deleteMany({ where: { eventId, sessionId: { in: batch } } }); + }); + rolled += batch.length; + } + } + return rolled; +} + +module.exports = { STAGES, RAW_DAYS, validSession, deviceFromUA, cleanSource, sessionsFrom, rollupRows, funnelReport, track, recordSubmit, funnelFor, rollupOldEvents }; diff --git a/backend/tests/analytics.test.js b/backend/tests/analytics.test.js new file mode 100644 index 0000000..9fbdbb1 --- /dev/null +++ b/backend/tests/analytics.test.js @@ -0,0 +1,70 @@ +const { deviceFromUA, cleanSource, sessionsFrom, rollupRows, funnelReport } = require('../src/utils/analytics'); + +const ev = (sessionId, stage, day, extra = {}) => ({ sessionId, stage, createdAt: `${day}T10:00:00Z`, ...extra }); + +describe('deviceFromUA', () => { + test('mobile, tablet, desktop and bots', () => { + expect(deviceFromUA('Mozilla/5.0 (iPhone; CPU iPhone OS 17_0) Mobile/15E148')).toBe('mobile'); + expect(deviceFromUA('Mozilla/5.0 (Linux; Android 14; SM-X200) Safari')).toBe('tablet'); + expect(deviceFromUA('Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/120')).toBe('desktop'); + expect(deviceFromUA('Googlebot/2.1')).toBeNull(); + expect(deviceFromUA('')).toBeNull(); + }); +}); + +describe('cleanSource', () => { + test('tidies, defaulting to direct', () => { + expect(cleanSource(' Facebook ')).toBe('facebook'); + expect(cleanSource('camp-2025