Deployment guide for installs behind an frp tunnel
DEPLOYMENT.md: install with --no-ssl, switch URLs to https, TRUST_PROXY, nginx and /etc/frp/frpc.ini for TLS on the VPS or in frpc (https2http), checking visitor IPs, updates and troubleshooting. Linked from the README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,342 @@
|
||||
# Deploying Hope Events behind frp
|
||||
|
||||
This guide is for a server that is **not reachable from the internet directly**: it runs
|
||||
`frpc`, which opens a tunnel to `frps` on a public server (the "VPS"). Visitors reach the VPS,
|
||||
and frp carries the traffic through to this server. The frp client config lives at
|
||||
**`/etc/frp/frpc.ini`**.
|
||||
|
||||
For a server with a public IP, `scripts/install.sh` does everything (see the README). Behind frp
|
||||
it needs a few changes, because:
|
||||
|
||||
- **HTTPS can't be set up by the installer.** DNS points at the VPS, not at this server, so
|
||||
`certbot --nginx` can't work the usual way. HTTPS ends either on the VPS or in `frpc`.
|
||||
- **There are more proxies in the way.** The app must be told which of them to trust, or it sees
|
||||
every visitor as `127.0.0.1` (rate limiting then treats everyone as one person, and PayFast
|
||||
payment notifications fail their IP check).
|
||||
- **The app must know the site is HTTPS** even though nginx on this server only speaks HTTP;
|
||||
otherwise it builds `http://` links (event images, payment return URLs).
|
||||
|
||||
## 1. How the traffic flows
|
||||
|
||||
Pick the setup that matches how your other frp sites get HTTPS.
|
||||
|
||||
**A. HTTPS ends on the VPS** (nginx/Caddy on the VPS holds the certificate) — recommended if the
|
||||
VPS already does this for other sites:
|
||||
|
||||
```
|
||||
visitor ──https──▶ VPS nginx/Caddy ──http──▶ frps (vhost_http_port) ══tunnel══▶ frpc ──▶ nginx :80 ──▶ app
|
||||
```
|
||||
|
||||
**B. HTTPS ends in frpc** (the VPS only runs frps; the certificate lives on this server and
|
||||
frpc's `https2http` plugin uses it):
|
||||
|
||||
```
|
||||
visitor ──https──▶ frps (vhost_https_port) ══tunnel══▶ frpc https2http ──▶ nginx 127.0.0.1:8080 ──▶ app
|
||||
visitor ──http───▶ frps (vhost_http_port) ══tunnel══▶ frpc ──▶ nginx :80 (certificate check + redirect to https)
|
||||
```
|
||||
|
||||
On this server, nginx sends `/api`, `/uploads`, `/health` and `/docs` to the backend (port 5000)
|
||||
and everything else to the Next.js site (port 3000), the same as a normal install.
|
||||
|
||||
## 2. Before you start
|
||||
|
||||
- Ubuntu 22.04 or 24.04 on this server, with `frpc` already installed and connected (it runs as
|
||||
a systemd service, usually `frpc.service`).
|
||||
- The site's DNS record (e.g. `events.yourchurch.org`) points at the **VPS**.
|
||||
- On the VPS, `frps.ini` has `vhost_http_port` set (and `vhost_https_port` for setup B). Note
|
||||
the values; you need them below.
|
||||
- The licence key for this site from the Hope Events Hub.
|
||||
|
||||
## 3. Install the app
|
||||
|
||||
On this server, as root, run the installer **without** its certificate step:
|
||||
|
||||
```bash
|
||||
curl -fsSLO https://c-git.crosscode.co.za/joshua/hope-events/raw/branch/main/scripts/install.sh
|
||||
sudo bash install.sh --domain events.yourchurch.org --no-ssl --version v1.41.2
|
||||
```
|
||||
|
||||
It installs Node 20, PostgreSQL, PM2 and nginx, creates the database, writes the config, builds
|
||||
and starts the app (in `/opt/hope-events`, run by the `hopeevents` user). Use `--name`, `--dir`,
|
||||
`--web-port` and `--api-port` if other apps already use those names or ports; adjust the ports
|
||||
below to match.
|
||||
|
||||
### Switch the URLs to https
|
||||
|
||||
With `--no-ssl` the installer writes `http://` addresses. Change them to `https://`:
|
||||
|
||||
```bash
|
||||
cd /opt/hope-events
|
||||
sudo sed -i 's#=http://#=https://#' backend/.env frontend/.env.production
|
||||
grep -E 'URL=' backend/.env frontend/.env.production # all should now start with https://
|
||||
```
|
||||
|
||||
While `backend/.env` is open, set which proxies to trust (all of them run on loopback in both
|
||||
setups; see [step 7](#7-check-that-visitor-addresses-come-through) to confirm):
|
||||
|
||||
```bash
|
||||
sudo sed -i 's#^TRUST_PROXY=.*#TRUST_PROXY=loopback#' backend/.env
|
||||
```
|
||||
|
||||
`NEXT_PUBLIC_API_URL` is baked in when the site is built, so rebuild the frontend and restart:
|
||||
|
||||
```bash
|
||||
sudo -u hopeevents -H bash -lc 'cd /opt/hope-events/frontend && npm run build'
|
||||
sudo -u hopeevents -H bash -lc 'cd /opt/hope-events && pm2 reload ecosystem.config.js --update-env'
|
||||
```
|
||||
|
||||
Keep `JWT_SECRET` in `backend/.env` as it is: it also encrypts the secrets stored in the
|
||||
database (SMTP, payment keys).
|
||||
|
||||
## 4. nginx on this server
|
||||
|
||||
The installer wrote `/etc/nginx/sites-available/hope-events`. Replace it with the version for
|
||||
your setup, changing the domain (and the ports if you changed them).
|
||||
|
||||
### Setup A (HTTPS on the VPS)
|
||||
|
||||
```nginx
|
||||
# Shows the forwarded-for chain in the access log (see step 7). Safe to keep.
|
||||
log_format hope_proxy '$remote_addr xff="$http_x_forwarded_for" "$request" $status';
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name events.yourchurch.org;
|
||||
client_max_body_size 25m;
|
||||
access_log /var/log/nginx/hope-events.access.log hope_proxy;
|
||||
|
||||
# Visitors only reach this through the VPS over HTTPS, so tell the app so.
|
||||
location ~ ^/(api|uploads|health|docs)(/|$) {
|
||||
proxy_pass http://127.0.0.1:5000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
}
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The VPS nginx/Caddy must redirect plain HTTP to HTTPS and pass the `Host` header through to frps
|
||||
unchanged (frps picks the site by its host name). For nginx on the VPS, roughly:
|
||||
|
||||
```nginx
|
||||
server {
|
||||
listen 443 ssl;
|
||||
server_name events.yourchurch.org;
|
||||
# ssl_certificate / ssl_certificate_key from certbot on the VPS
|
||||
client_max_body_size 25m;
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:8080; # frps vhost_http_port
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Setup B (HTTPS in frpc)
|
||||
|
||||
Port 80 only answers certificate checks and redirects everything else; the real site is on
|
||||
`127.0.0.1:8080`, which only frpc's `https2http` plugin reaches.
|
||||
|
||||
```nginx
|
||||
log_format hope_proxy '$remote_addr xff="$http_x_forwarded_for" "$request" $status';
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name events.yourchurch.org;
|
||||
location /.well-known/acme-challenge/ { root /var/www/certbot; }
|
||||
location / { return 301 https://$host$request_uri; }
|
||||
}
|
||||
|
||||
server {
|
||||
listen 127.0.0.1:8080;
|
||||
server_name events.yourchurch.org;
|
||||
client_max_body_size 25m;
|
||||
access_log /var/log/nginx/hope-events.access.log hope_proxy;
|
||||
|
||||
location ~ ^/(api|uploads|health|docs)(/|$) {
|
||||
proxy_pass http://127.0.0.1:5000;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
}
|
||||
location / {
|
||||
proxy_pass http://127.0.0.1:3000;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto https;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then, for either setup:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /var/www/certbot # setup B only
|
||||
sudo nginx -t && sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
## 5. frpc: `/etc/frp/frpc.ini`
|
||||
|
||||
Back it up first, then **add** a section for the site below the existing ones. Leave `[common]`
|
||||
(server address, port, token) and the other sections as they are. Section names must be unique
|
||||
across everything connected to your frps.
|
||||
|
||||
```bash
|
||||
sudo cp /etc/frp/frpc.ini /etc/frp/frpc.ini.bak-$(date +%F)
|
||||
sudo nano /etc/frp/frpc.ini
|
||||
```
|
||||
|
||||
### Setup A
|
||||
|
||||
```ini
|
||||
[hope-events-http]
|
||||
type = http
|
||||
local_ip = 127.0.0.1
|
||||
local_port = 80
|
||||
custom_domains = events.yourchurch.org
|
||||
```
|
||||
|
||||
### Setup B
|
||||
|
||||
```ini
|
||||
# Plain HTTP: certificate checks and the redirect to https
|
||||
[hope-events-http]
|
||||
type = http
|
||||
local_ip = 127.0.0.1
|
||||
local_port = 80
|
||||
custom_domains = events.yourchurch.org
|
||||
|
||||
# HTTPS: frpc holds the certificate and passes plain HTTP to nginx on 127.0.0.1:8080
|
||||
[hope-events-https]
|
||||
type = https
|
||||
custom_domains = events.yourchurch.org
|
||||
plugin = https2http
|
||||
plugin_local_addr = 127.0.0.1:8080
|
||||
plugin_crt_path = /etc/letsencrypt/live/events.yourchurch.org/fullchain.pem
|
||||
plugin_key_path = /etc/letsencrypt/live/events.yourchurch.org/privkey.pem
|
||||
plugin_header_X-Forwarded-Proto = https
|
||||
```
|
||||
|
||||
Restart frpc and check it registered the new sections without errors:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart frpc
|
||||
sudo journalctl -u frpc -n 30 --no-pager # look for "start proxy success" for each section
|
||||
```
|
||||
|
||||
> Newer frp releases (0.52+) prefer `frpc.toml`, but still read `.ini`. Don't mix the two: if
|
||||
> your service already uses `frpc.ini`, keep adding to it.
|
||||
|
||||
### Certificate (setup B only)
|
||||
|
||||
Request it through the tunnel with the webroot method (not `--nginx`, which would rewrite the
|
||||
nginx config), then have renewals restart frpc so it loads the new certificate:
|
||||
|
||||
```bash
|
||||
sudo apt-get install -y certbot
|
||||
sudo certbot certonly --webroot -w /var/www/certbot -d events.yourchurch.org \
|
||||
-m you@yourchurch.org --agree-tos --non-interactive
|
||||
echo 'systemctl restart frpc' | sudo tee /etc/letsencrypt/renewal-hooks/deploy/restart-frpc.sh
|
||||
sudo chmod +x /etc/letsencrypt/renewal-hooks/deploy/restart-frpc.sh
|
||||
sudo systemctl restart frpc
|
||||
```
|
||||
|
||||
frpc needs to be able to read the certificate files (it does when it runs as root, the usual
|
||||
setup). If the `[hope-events-https]` section fails to start, check `journalctl -u frpc`.
|
||||
|
||||
## 6. Finish in the browser
|
||||
|
||||
1. Open `https://events.yourchurch.org/health` — it should answer with the version.
|
||||
2. Open `https://events.yourchurch.org/setup`: create the admin account.
|
||||
3. **Site settings → Licence**: enter the licence key to connect to the Hub. The site checks in
|
||||
every 15 minutes from then on (outgoing HTTPS to `events-hub.crosscode.co.za`, which frp
|
||||
doesn't affect).
|
||||
4. Configure SMTP, payments (Yoco/PayFast webhooks use the `https://` `BACKEND_URL`), branding,
|
||||
etc. in Site settings.
|
||||
|
||||
## 7. Check that visitor addresses come through
|
||||
|
||||
Open the site from your phone (on mobile data, so it isn't your own network), then on the
|
||||
server:
|
||||
|
||||
```bash
|
||||
sudo tail -n 5 /var/log/nginx/hope-events.access.log
|
||||
```
|
||||
|
||||
The `xff="…"` part is what the app sees, with nginx's own entry added at the end:
|
||||
|
||||
- `xff="<your phone's IP>, 127.0.0.1"` (only `127.0.0.1` after the real address): good,
|
||||
`TRUST_PROXY=loopback` is right.
|
||||
- `xff="<your phone's IP>, 203.0.113.10"` (a non-local address after it, e.g. the VPS's): add
|
||||
that address: `TRUST_PROXY=loopback,203.0.113.10`, then
|
||||
`pm2 reload ecosystem.config.js --update-env`.
|
||||
- Your phone's IP doesn't appear at all: frp isn't passing the visitor's address on (can happen
|
||||
with setup B on some frp versions). Rate limiting and the PayFast IP check will then misbehave.
|
||||
Fix it by moving TLS to the VPS (setup A), or with frp's `proxy_protocol_version = v2` on the
|
||||
frpc section plus nginx's `listen … proxy_protocol` and `set_real_ip_from 127.0.0.1;
|
||||
real_ip_header proxy_protocol;`.
|
||||
|
||||
The backend logs the setting it uses at startup: `pm2 logs hope-events-backend | grep trust`.
|
||||
|
||||
## 8. Updating
|
||||
|
||||
Nothing about frp changes for updates. On the server:
|
||||
|
||||
```bash
|
||||
cd /opt/hope-events
|
||||
sudo -u hopeevents -H bash scripts/deploy.sh v1.41.2 # or no tag for the latest main
|
||||
```
|
||||
|
||||
It backs up the database, checks out the release, installs, migrates, builds and reloads PM2,
|
||||
and goes back to the previous version if anything fails first. `backend/.env`,
|
||||
`frontend/.env.production`, `ecosystem.config.js` and the nginx/frpc configs are not in git, so
|
||||
updates keep your changes from this guide.
|
||||
|
||||
### Deploying from CI over SSH
|
||||
|
||||
The Gitea workflow deploys on a `v*` tag by SSHing to `DEPLOY_HOST`. Behind frp, that needs an
|
||||
SSH tunnel too, e.g. in `frpc.ini`:
|
||||
|
||||
```ini
|
||||
[hope-events-ssh]
|
||||
type = tcp
|
||||
local_ip = 127.0.0.1
|
||||
local_port = 22
|
||||
remote_port = 6022
|
||||
```
|
||||
|
||||
Then use the VPS as `DEPLOY_HOST`. The workflow's `ssh` call (`.gitea/workflows/ci.yml`) doesn't
|
||||
take a port yet, so add `-p 6022` there (or a `DEPLOY_PORT` secret) before relying on it. Only
|
||||
open this if you need it: it exposes this server's SSH on the VPS. Use key-only login. Deploying
|
||||
by hand (above) needs none of this.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Likely cause |
|
||||
| --- | --- |
|
||||
| frps shows "404 Not Found" page | The `custom_domains` value doesn't match the address typed, or frpc didn't start that section (`journalctl -u frpc`). |
|
||||
| 502 Bad Gateway | nginx is up but the app isn't: `sudo -u hopeevents pm2 ls` and `pm2 logs`. |
|
||||
| Site loads but login/API calls fail (CORS) | `FRONTEND_URL` in `backend/.env` is still `http://`, or the frontend wasn't rebuilt after changing `NEXT_PUBLIC_API_URL`. |
|
||||
| Images or links use `http://` (browser "mixed content" warnings) | nginx isn't sending `X-Forwarded-Proto https`, or `TRUST_PROXY` isn't set. |
|
||||
| "Too many requests" for everyone at once | All visitors look like one address: see step 7. |
|
||||
| PayFast payments stay pending | Same as above: the PayFast notification's IP check sees the proxy, not PayFast. |
|
||||
| Uploads over ~1 MB fail | `client_max_body_size` missing on this server's nginx or the VPS's. |
|
||||
| Redirect loop (setup B) | The site's `server` block is on port 80 instead of `127.0.0.1:8080`, or `plugin_local_addr` points at port 80. |
|
||||
@@ -114,6 +114,8 @@ app. Then open `https://<domain>/setup`: create the admin account and enter the
|
||||
to connect the site to the Hope Events Hub. Several installs can share a server with different
|
||||
`--name`, `--dir`, `--web-port` and `--api-port`. Run `--help` for all options.
|
||||
|
||||
Behind an frp tunnel (no public IP on the server)? See [DEPLOYMENT.md](DEPLOYMENT.md).
|
||||
|
||||
### Updating
|
||||
|
||||
To update a running install, on the server in the repo folder:
|
||||
|
||||
Reference in New Issue
Block a user