JoeBot/HOA_Financial_Platform
1
0
Fork 0
Code Issues Pull Requests 2 Actions Packages Projects Releases Wiki Activity

refactor: remove Docker nginx from production, use host nginx directly

Browse Source 
The production stack no longer runs a Docker nginx container. Instead,
the host-level nginx handles SSL termination AND request routing:
  /api/* → 127.0.0.1:3000 (backend)
  /*     → 127.0.0.1:3001 (frontend)

Changes:
- docker-compose.prod.yml: set nginx replicas to 0, expose backend and
  frontend on 127.0.0.1 only (loopback)
- nginx/host-production.conf: new ready-to-copy host nginx config with
  SSL, rate limiting, proxy buffering, and AI endpoint timeouts
- docs/DEPLOYMENT.md: rewritten production deployment and SSL sections
  to reflect the simplified single-nginx architecture

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
olsch01
2026-03-02 20:08:32 -05:00
parent d526025926
commit 2c215353d4
3 changed files with 199 additions and 284 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docker-compose.prod.yml
View File

@@ -2,29 +2,31 @@
# docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
#
# What this changes from the base (dev) config:
# - Disables the Docker nginx container (host nginx handles routing + SSL)
# - Backend: production Dockerfile (compiled JS, no watch, no devDeps)
# - Frontend: production Dockerfile (static build served by nginx on port 3001)
# - Backend + Frontend bound to 127.0.0.1 only (host nginx proxies to them)
# - No source-code volume mounts (uses baked-in built code)
# - Memory limits and health checks on backend
# - Tuned PostgreSQL for production workloads
# - Restart policies for reliability
#
# SSL/TLS is handled at the host level (e.g., host nginx + certbot).
# The Docker nginx container listens internally on port 80, mapped to
# host port 8080 so it doesn't conflict with the host reverse proxy.
# SSL/TLS and request routing are handled by the host-level nginx.
# See nginx/host-production.conf for a ready-to-use reference config.
services:
nginx:
ports:
- "8080:80" # override: avoid conflict with host nginx
volumes:
- ./nginx/production.conf:/etc/nginx/conf.d/default.conf:ro
restart: unless-stopped
# Disabled in production — host nginx handles routing + SSL directly.
# The dev-only Docker nginx is still used by the base docker-compose.yml.
deploy:
replicas: 0
backend:
build:
context: ./backend
dockerfile: Dockerfile # production Dockerfile (compiled JS)
ports:
- "127.0.0.1:3000:3000" # loopback only — host nginx proxies here
volumes: [] # override: no source mounts in prod
environment:
- DATABASE_URL=${DATABASE_URL}
@@ -53,6 +55,8 @@ services:
build:
context: ./frontend
dockerfile: Dockerfile # production Dockerfile (static nginx)
ports:
- "127.0.0.1:3001:3001" # loopback only — host nginx proxies here
volumes: [] # override: no source mounts in prod
environment:
- NODE_ENV=production

364
docs/DEPLOYMENT.md
View File

@@ -30,8 +30,9 @@ On the **target server**, ensure the following are installed:
| Git | 2.x |
| `psql` (client) | 15+ *(optional, for manual DB work)* |
The app runs five containers — nginx, backend (NestJS), frontend (Vite/React),
PostgreSQL 15, and Redis 7. Total memory footprint is roughly **1–2 GB** idle.
The app runs four containers in production — backend (NestJS), frontend
(React/nginx), PostgreSQL 15, and Redis 7. A fifth nginx container is used
in dev mode only. Total memory footprint is roughly **1–2 GB** idle.
For SSL, the server must also have:
- A **public hostname** with a DNS A record pointing to the server's IP
@@ -99,7 +100,7 @@ This will:
- Build the backend and frontend images
- Pull `postgres:15-alpine`, `redis:7-alpine`, and `nginx:alpine`
- Initialize the PostgreSQL database with the shared schema (`db/init/00-init.sql`)
- Start all five services on the `hoanet` bridge network
- Start all services on the `hoanet` bridge network
### 4. Wait for healthy services
@@ -107,7 +108,7 @@ This will:
docker compose ps
```
All five containers should show `Up` (postgres and redis should also show
All containers should show `Up` (postgres and redis should also show
`(healthy)`). If the backend is restarting, check logs:
```bash
@@ -156,7 +157,7 @@ local development but will fail under even light production load.
| Backend | `nest start --watch` (ts-node, file watcher) | Compiled JS, clustered across CPU cores |
| DB pooling | None (new connection per query) | Pool of 30 reusable connections |
| Postgres | Default config (100 connections) | Tuned: 200 connections, optimized buffers |
| Nginx | Basic proxy | Keepalive upstreams, buffering, rate limiting |
| Nginx | Docker nginx routes all traffic | Disabled — host nginx routes directly |
| Restart | None | `unless-stopped` on all services |
### Deploy for production
@@ -171,56 +172,42 @@ nano .env
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
```
> **Port mapping:** The production overlay maps the Docker nginx to **host
> port 8080** (not 80) so it doesn't conflict with a host-level reverse
> proxy. If you're NOT using a host reverse proxy, you can override this:
> ```bash
> # Direct access on port 80 (no host reverse proxy)
> docker compose -f docker-compose.yml -f docker-compose.prod.yml \
> up -d --build --scale nginx=0
> # Then manually: docker compose ... run -d -p 80:80 nginx
> ```
The production overlay **disables the Docker nginx container** — request routing
and SSL are handled by the host-level nginx. Backend and frontend are exposed
on `127.0.0.1` only (loopback), so they aren't publicly accessible without the
host nginx in front.
### Host reverse proxy setup (recommended)
### Host nginx setup (required for production)
In production, SSL termination is typically handled by a **host-level nginx**
(or Caddy, Traefik, etc.) that proxies to the Docker stack on port 8080:
A ready-to-use host nginx config is included at `nginx/host-production.conf`.
It handles SSL termination, request routing, rate limiting, proxy buffering,
and extended timeouts for AI endpoints.
```nginx
# /etc/nginx/sites-available/app.yourdomain.com
server {
listen 80;
server_name app.yourdomain.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name app.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/app.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/app.yourdomain.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
```
Then enable and reload:
```bash
# Copy the reference config
sudo cp nginx/host-production.conf /etc/nginx/sites-available/app.yourdomain.com
# Edit the hostname (replace all instances of app.yourdomain.com)
sudo sed -i 's/app.yourdomain.com/YOUR_HOSTNAME/g' \
/etc/nginx/sites-available/app.yourdomain.com
# Enable the site
sudo ln -s /etc/nginx/sites-available/app.yourdomain.com /etc/nginx/sites-enabled/
sudo certbot --nginx -d app.yourdomain.com # if using certbot on host
# Get an SSL certificate (certbot modifies the config automatically)
sudo certbot --nginx -d YOUR_HOSTNAME
# Test and reload
sudo nginx -t && sudo systemctl reload nginx
```
The host config routes traffic directly to the Docker services:
- `/api/*` → `http://127.0.0.1:3000` (NestJS backend)
- `/` → `http://127.0.0.1:3001` (React frontend served by nginx)
> See `nginx/host-production.conf` for the full config including rate limiting,
> proxy buffering, and extended AI endpoint timeouts.
> **Tip:** Create a shell alias to avoid typing the compose files every time:
> ```bash
> echo 'alias dc="docker compose -f docker-compose.yml -f docker-compose.prod.yml"' >> ~/.bashrc
@@ -241,11 +228,12 @@ sudo nginx -t && sudo systemctl reload nginx
- Served by a lightweight nginx container (not Vite)
- Static assets cached with immutable headers (Vite filename hashing)
**Nginx (`nginx/production.conf`)**
- Keepalive connections to upstream services (connection reuse)
- Proxy buffering to prevent 502s during slow responses
**Host Nginx (`nginx/host-production.conf`)**
- SSL termination + HTTP→HTTPS redirect (via certbot on host)
- Rate limiting on API routes (10 req/s per IP, burst 30)
- Proper timeouts tuned per endpoint type
- Proxy buffering to prevent 502s during slow responses
- Extended timeouts for AI endpoints (180s for investment/health-score calls)
- Routes `/api/*` → backend:3000, `/` → frontend:3001
**PostgreSQL**
- `max_connections=200` (up from default 100)
@@ -270,233 +258,58 @@ Kubernetes replicas.
## SSL with Certbot (Let's Encrypt)
This section walks through enabling HTTPS using the included nginx container
and a Certbot sidecar. The process has three phases: obtain the certificate,
switch nginx to SSL, and set up auto-renewal.
SSL is handled entirely at the host level using certbot with the host nginx.
No Docker containers are involved in SSL termination.
### Files involved
### Prerequisites
| File | Purpose |
|------|---------|
| `nginx/default.conf` | HTTP-only config (development / initial state) |
| `nginx/certbot-init.conf` | Temporary config used only during initial cert request |
| `nginx/ssl.conf` | Full SSL config (HTTP→HTTPS redirect + TLS termination) |
| `docker-compose.ssl.yml` | Compose override that adds port 443, certbot volumes, and renewal service |
- A public hostname with DNS pointing to this server
- Ports 80 and 443 open in the firewall
- Host nginx installed: `sudo apt install nginx` (Ubuntu/Debian)
- Certbot installed: `sudo apt install certbot python3-certbot-nginx`
### Overview
### Obtain a certificate
```
Phase 1 — Obtain certificate
┌────────────────────────────────────────────────┐
│ nginx serves certbot-init.conf on port 80 │
│ certbot answers ACME challenge → gets cert │
└────────────────────────────────────────────────┘
Phase 2 — Enable SSL
┌────────────────────────────────────────────────┐
│ nginx switches to ssl.conf │
│ port 80 → 301 redirect to 443 │
│ port 443 → TLS termination → backend/frontend │
└────────────────────────────────────────────────┘
Phase 3 — Auto-renewal
┌────────────────────────────────────────────────┐
│ certbot container checks every 12 hours │
│ renews if cert expires within 30 days │
│ nginx reloads via cron to pick up new cert │
└────────────────────────────────────────────────┘
```
---
### Phase 1 — Obtain the certificate
Throughout this section, replace `staging.example.com` with your actual
hostname and `you@example.com` with your real email.
#### Step 1: Edit `nginx/ssl.conf` with your hostname
If you followed the "Host nginx setup" section above, certbot was already
run as part of that process. If not:
```bash
cd /opt/hoa-ledgeriq
sed -i 's/staging.example.com/YOUR_HOSTNAME/g' nginx/ssl.conf
# Ensure the host nginx config is in place first
sudo certbot --nginx -d YOUR_HOSTNAME
```
This updates the three places where the hostname appears: the `server_name`
directive and the two certificate paths.
Certbot will:
1. Verify domain ownership via an ACME challenge on port 80
2. Obtain the certificate from Let's Encrypt
3. Automatically modify the nginx config to enable SSL
4. Set up an HTTP → HTTPS redirect
#### Step 2: Swap in the temporary certbot-init config
Nginx can't start with `ssl.conf` yet because the certificates don't exist.
Use the minimal HTTP-only config that only serves the ACME challenge:
### Verify HTTPS
```bash
# Back up the current config
cp nginx/default.conf nginx/default.conf.bak
# Use the certbot init config temporarily
cp nginx/certbot-init.conf nginx/default.conf
```
#### Step 3: Start the stack with the SSL compose override
```bash
docker compose -f docker-compose.yml -f docker-compose.ssl.yml up -d --build
```
This starts all services plus the certbot container, exposes port 443,
and creates the shared volumes for certificates and ACME challenges.
#### Step 4: Request the certificate
```bash
docker compose -f docker-compose.yml -f docker-compose.ssl.yml \
run --rm certbot certonly \
--webroot \
--webroot-path /var/www/certbot \
--email you@example.com \
--agree-tos \
--no-eff-email \
-d YOUR_HOSTNAME
```
You should see output ending with:
```
Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/YOUR_HOSTNAME/fullchain.pem
Key is saved at: /etc/letsencrypt/live/YOUR_HOSTNAME/privkey.pem
```
> **Troubleshooting:** If certbot fails with a connection error, ensure:
> - DNS for `YOUR_HOSTNAME` resolves to this server's public IP
> - Port 80 is open (firewall, security group, cloud provider)
> - No other process is bound to port 80
---
### Phase 2 — Switch to full SSL
#### Step 5: Activate the SSL nginx config
```bash
cp nginx/ssl.conf nginx/default.conf
```
#### Step 6: Restart nginx to load the certificate
```bash
docker compose -f docker-compose.yml -f docker-compose.ssl.yml \
exec nginx nginx -s reload
```
#### Step 7: Verify HTTPS
```bash
# Should follow redirect and return HTML
# Should return 200 with SSL
curl -I https://YOUR_HOSTNAME
# Should return 301 redirect
# Should return 301 redirect to HTTPS
curl -I http://YOUR_HOSTNAME
```
Expected `http://` response:
### Auto-renewal
```
HTTP/1.1 301 Moved Permanently
Location: https://YOUR_HOSTNAME/
```
The app is now accessible at `https://YOUR_HOSTNAME` with a valid
Let's Encrypt certificate. All HTTP requests redirect to HTTPS.
---
### Phase 3 — Auto-renewal
The `certbot` service defined in `docker-compose.ssl.yml` already runs a
renewal loop (checks every 12 hours, renews if < 30 days remain).
However, nginx needs to be told to reload when a new certificate is issued.
Add a host-level cron job that reloads nginx daily:
Certbot installs a systemd timer (or cron job) that checks for renewal
twice daily. Verify it's active:
```bash
# Add to the server's crontab (as root)
crontab -e
sudo systemctl status certbot.timer
```
Add this line:
```cron
0 4 * * * cd /opt/hoa-ledgeriq && docker compose -f docker-compose.yml -f docker-compose.ssl.yml exec -T nginx nginx -s reload > /dev/null 2>&1
```
This reloads nginx at 4 AM daily. Since reload is graceful (no downtime),
this is safe to run even when no new cert was issued.
> **Alternative:** If you prefer not to use cron, you can add a
> `--deploy-hook` to the certbot command that reloads nginx only when a
> renewal actually happens:
> ```bash
> docker compose -f docker-compose.yml -f docker-compose.ssl.yml \
> run --rm certbot certonly \
> --webroot \
> --webroot-path /var/www/certbot \
> --email you@example.com \
> --agree-tos \
> --no-eff-email \
> --deploy-hook "nginx -s reload" \
> -d YOUR_HOSTNAME
> ```
---
### Day-to-day usage with SSL
Once SSL is set up, **always** use both compose files when running commands:
To test renewal without actually renewing:
```bash
# Start / restart
docker compose -f docker-compose.yml -f docker-compose.ssl.yml up -d
# Rebuild after code changes
docker compose -f docker-compose.yml -f docker-compose.ssl.yml up -d --build
# View logs
docker compose -f docker-compose.yml -f docker-compose.ssl.yml logs -f nginx
# Shortcut: create an alias
echo 'alias dc="docker compose -f docker-compose.yml -f docker-compose.ssl.yml"' >> ~/.bashrc
source ~/.bashrc
dc up -d --build # much easier
sudo certbot renew --dry-run
```
### Reverting to HTTP-only (development)
If you need to go back to plain HTTP (e.g., local development):
```bash
# Restore the original HTTP-only config
cp nginx/default.conf.bak nginx/default.conf
# Run without the SSL override
docker compose up -d
```
---
### Quick reference: SSL file map
```
nginx/
├── default.conf ← active config (copy ssl.conf here for HTTPS)
├── default.conf.bak ← backup of original HTTP-only config
├── certbot-init.conf ← temporary, used only during initial cert request
└── ssl.conf ← full SSL config (edit hostname before using)
docker-compose.yml ← base stack (HTTP only, ports 80)
docker-compose.ssl.yml ← SSL overlay (adds 443, certbot, volumes)
```
Certbot automatically reloads nginx after a successful renewal.
---
@@ -667,8 +480,8 @@ docker compose exec -T postgres psql \
### Quick health checks
```bash
# Backend is responding (use https:// if SSL is enabled)
curl -s http://localhost/api/auth/login | head -c 100
# Backend is responding
curl -s http://localhost:3000/api/auth/login | head -c 100
# Database is accessible
docker compose exec -T postgres psql -U hoafinance -d hoafinance \
@@ -704,7 +517,9 @@ curl -sI http://YOUR_HOSTNAME | grep -E 'HTTP|Location'
docker compose logs -f # all services
docker compose logs -f backend # backend only
docker compose logs -f postgres # database only
docker compose logs -f nginx # nginx access/error log
docker compose logs -f frontend # frontend nginx
sudo tail -f /var/log/nginx/access.log # host nginx access log
sudo tail -f /var/log/nginx/error.log # host nginx error log
```
---
@@ -747,26 +562,23 @@ docker compose logs -f nginx # nginx access/error log
│ (PG 15) │ │ (Redis 7) │
└────────────┘ └───────────┘
Production (with host reverse proxy):
┌──────────────────────┐
Browser ─────────► │ Host nginx :80/:443 │ ← SSL termination
└────────┬─────────────┘
▼
┌──────────────────┐
│ Docker nginx:8080│ ← proxy to services
└────────┬─────────┘
┌──────────┴──────────┐
▼ ▼
┌──────────────┐ ┌──────────────┐
│ backend :3000│ │frontend :3001│
│ (compiled) │ │ (static nginx)│
└──────┬───────┘ └──────────────┘
┌────┴────┐
▼ ▼
┌────────────┐ ┌───────────┐
│postgres:5432│ │redis :6379│
│ (PG 15) │ │ (Redis 7) │
└────────────┘ └───────────┘
Production (host nginx handles SSL + routing):
┌────────────────────────────────┐
Browser ─────────► │ Host nginx :80/:443 (SSL) │
│ /api/* → 127.0.0.1:3000 │
│ /* → 127.0.0.1:3001 │
└────────┬───────────┬───────────┘
▼ ▼
┌──────────────┐ ┌──────────────┐
│ backend :3000│ │frontend :3001│
│ (compiled) │ │ (static nginx)│
└──────┬───────┘ └──────────────┘
┌────┴────┐
▼ ▼
┌────────────┐ ┌───────────┐
│postgres:5432│ │redis :6379│
│ (PG 15) │ │ (Redis 7) │
└────────────┘ └───────────┘
```
**Multi-tenant isolation:** Each HOA organization gets its own PostgreSQL

99
nginx/host-production.conf Normal file
View File

@@ -0,0 +1,99 @@
# HOA LedgerIQ — Host-level nginx config (production)
#
# Copy this file to /etc/nginx/sites-available/app.yourdomain.com
# and symlink to /etc/nginx/sites-enabled/:
#
# sudo cp nginx/host-production.conf /etc/nginx/sites-available/app.yourdomain.com
# sudo ln -s /etc/nginx/sites-available/app.yourdomain.com /etc/nginx/sites-enabled/
# sudo nginx -t && sudo systemctl reload nginx
#
# Then obtain an SSL certificate:
# sudo certbot --nginx -d app.yourdomain.com
#
# Replace "app.yourdomain.com" with your actual hostname throughout this file.
# --- Rate limiting ---
# 10 requests/sec per IP for API routes (shared memory zone: 10 MB ≈ 160k IPs)
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;
# --- HTTP → HTTPS redirect ---
server {
listen 80;
server_name app.yourdomain.com;
# Let certbot answer ACME challenges
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
# Everything else → HTTPS
location / {
return 301 https://$host$request_uri;
}
}
# --- Main HTTPS server ---
server {
listen 443 ssl;
server_name app.yourdomain.com;
# SSL certificates (managed by certbot)
ssl_certificate /etc/letsencrypt/live/app.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/app.yourdomain.com/privkey.pem;
# Modern TLS settings
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384';
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# --- Proxy defaults ---
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# Buffer settings — prevent 502s when backend is slow to respond
proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 8 16k;
proxy_busy_buffers_size 32k;
# --- API routes → NestJS backend (port 3000) ---
location /api/ {
limit_req zone=api_limit burst=30 nodelay;
proxy_pass http://127.0.0.1:3000;
proxy_read_timeout 30s;
proxy_connect_timeout 5s;
proxy_send_timeout 15s;
}
# AI endpoints — longer timeouts (LLM calls can take 30-120s)
location /api/investment-planning/recommendations {
proxy_pass http://127.0.0.1:3000;
proxy_read_timeout 180s;
proxy_connect_timeout 10s;
proxy_send_timeout 30s;
}
location /api/health-scores/calculate {
proxy_pass http://127.0.0.1:3000;
proxy_read_timeout 180s;
proxy_connect_timeout 10s;
proxy_send_timeout 30s;
}
# --- Frontend → React SPA served by nginx (port 3001) ---
location / {
proxy_pass http://127.0.0.1:3001;
proxy_read_timeout 10s;
proxy_connect_timeout 5s;
proxy_cache_bypass $http_upgrade;
}
}