From 31e44cc737513c300547c7f5457db099c3fbf7a9 Mon Sep 17 00:00:00 2001 From: ilia Date: Wed, 5 Aug 2026 17:05:21 -0400 Subject: [PATCH] Scrub active admin guides for plainer tone Rewrite docs/USER_GUIDE.md, docs/DEPLOYMENT.md, docs/DEPLOY_FROM_SCRATCH.md, and DEPLOYMENT_CHECKLIST.md to drop AI-generated tells: bold-colon feature walls, Title Case headings, em dashes, and rule-of-three lists. Facts (commands, paths, ports, URLs) are unchanged. Historical scratch docs (PHOTO_VIEWER_*, FIX_*, *_REVIEW, *_PLAN, etc.) and the cPanel-specific runbooks are untouched. --- DEPLOYMENT_CHECKLIST.md | 20 +- docs/DEPLOYMENT.md | 210 +++++------- docs/DEPLOY_FROM_SCRATCH.md | 106 +++--- docs/USER_GUIDE.md | 658 +++++++++++++----------------------- 4 files changed, 385 insertions(+), 609 deletions(-) diff --git a/DEPLOYMENT_CHECKLIST.md b/DEPLOYMENT_CHECKLIST.md index 4f2a6b8..0d0edc8 100644 --- a/DEPLOYMENT_CHECKLIST.md +++ b/DEPLOYMENT_CHECKLIST.md @@ -1,8 +1,8 @@ -# Deployment Checklist +# Deployment checklist After pulling from Git, configure the following server-specific settings: -## 1. Environment Files (gitignored - safe to modify) +## 1. Environment files (gitignored, safe to modify) ### Root `.env` ```bash @@ -33,7 +33,7 @@ NEXTAUTH_SECRET=your-secret-key-here AUTH_URL=http://:3001 # Password reset / email verification emails (lib/email.ts). Without these, -# "Forgot password" and "Confirm your email" never actually send mail — see +# "Forgot password" and "Confirm your email" never actually send mail, see # issue #56. Pick ONE provider path: # Option A: SMTP (primary; falls back to Resend automatically on failure) @@ -59,7 +59,7 @@ RESEND_FROM_NAME=PunimTag Viewer Verify email sending works with `cd viewer-frontend && npx tsx scripts/test-email-sending.ts` after setting these. -## 2. PM2 Configuration +## 2. PM2 configuration Copy the template and customize for your server: @@ -72,23 +72,23 @@ Edit `ecosystem.config.js` and update: - All `error_file` and `out_file` paths to your user's home directory - `PYTHONPATH` and `PATH` environment variables -## 3. System Configuration (One-time setup) +## 3. System configuration (one-time setup) -### Firewall Rules +### Firewall rules ```bash sudo ufw allow 3000/tcp # Admin frontend sudo ufw allow 3001/tcp # Viewer frontend sudo ufw allow 8000/tcp # Backend API ``` -### Database Setup -Create admin user in auth database: +### Database setup +Create the admin user in the auth database: ```bash cd viewer-frontend npx tsx scripts/fix-admin-user.ts ``` -## 4. Build Frontends +## 4. Build frontends ```bash # Admin frontend @@ -103,7 +103,7 @@ npm run prisma:generate:all npm run build ``` -## 5. Start Services +## 5. Start services ```bash pm2 start ecosystem.config.js diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 62fd087..76782bf 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -8,44 +8,39 @@ Last reviewed: January 6, 2026. ## Table of contents 1. [Prerequisites](#prerequisites) -2. [Development Server Deployment](#development-server-deployment) -3. [Production Deployment](#production-deployment) -4. [Environment Configuration](#environment-configuration) -5. [Service Management](#service-management) +2. [Development server deployment](#development-server-deployment) +3. [Production deployment](#production-deployment) +4. [Environment configuration](#environment-configuration) +5. [Service management](#service-management) 6. [Troubleshooting](#troubleshooting) --- ## Prerequisites -### Server Requirements +### Server requirements -- **Operating System**: Linux (Ubuntu 20.04+ recommended) -- **Python**: 3.12 or higher -- **Node.js**: 18+ and npm 9+ -- **PostgreSQL**: 12+ (required) -- **Redis**: 6+ (required for background jobs) -- **System Memory**: Minimum 4GB RAM (8GB+ recommended for large photo collections) -- **Disk Space**: Sufficient space for photos and database +- Linux (Ubuntu 20.04+ recommended) +- Python 3.12 or higher +- Node.js 18+ and npm 9+ +- PostgreSQL 12+ (required) +- Redis 6+ (required for background jobs) +- At least 4GB RAM (8GB+ recommended for large photo collections) +- Enough disk space for photos and the database -### Development Server Information +### Development server information -**Development Server:** -- **Host**: -- **User**: appuser -- **Password**: [Contact administrator for password] +Development server: host ``, user `appuser`. Contact the +administrator for the password. -**Development Database:** -- **Host**: `` -- **Port**: 5432 -- **User**: -- **Password**: [Contact administrator for password] +Development database: host ``, port 5432, user ``. +Contact the administrator for the password. --- ## Development server deployment -### Step 1: Build Applications +### Step 1: build applications On your local machine or CI/CD pipeline: @@ -60,12 +55,10 @@ npm run build:all npm run deploy:dev ``` -This will: -- Build admin-frontend for production -- Build viewer-frontend for production -- Prepare deployment package +This builds admin-frontend and viewer-frontend for production and prepares +the deployment package. -### Step 2: Transfer Files to Server +### Step 2: transfer files to server ```bash # Create deployment directory structure @@ -83,7 +76,7 @@ scp -r viewer-frontend/.next/* appuser@:/opt/punimtag/viewer-front scp .env.example appuser@:/opt/punimtag/.env ``` -### Step 3: Server Setup +### Step 3: server setup SSH into the development server: @@ -92,7 +85,7 @@ ssh appuser@ cd /opt/punimtag ``` -#### Install System Dependencies +#### Install system dependencies ```bash # Update package list @@ -112,9 +105,9 @@ source venv/bin/activate pip install -r requirements.txt ``` -#### Configure Environment +#### Configure environment -Edit `.env` file: +Edit the `.env` file: ```bash nano .env @@ -143,17 +136,15 @@ API_HOST=0.0.0.0 API_PORT=8000 ``` -#### Configure Frontend Environment +#### Configure frontend environment -**Admin Frontend:** -Create `admin-frontend/.env`: +Admin frontend: create `admin-frontend/.env`: ```bash VITE_API_URL=http://:8000 ``` -**Viewer Frontend:** -Create `viewer-frontend/.env`: +Viewer frontend: create `viewer-frontend/.env`: ```bash DATABASE_URL=postgresql://:[PASSWORD]@:5432/punimtag @@ -170,11 +161,11 @@ npx prisma generate cd .. ``` -### Step 4: Set Up Systemd Services +### Step 4: set up systemd services Create systemd service files for automatic startup: -#### Backend API Service +#### Backend API service ```bash sudo nano /etc/systemd/system/punimtag-api.service @@ -199,7 +190,7 @@ RestartSec=10 WantedBy=multi-user.target ``` -#### RQ Worker Service +#### RQ worker service ```bash sudo nano /etc/systemd/system/punimtag-worker.service @@ -224,7 +215,7 @@ RestartSec=10 WantedBy=multi-user.target ``` -#### Admin Frontend Service (using PM2 or nginx) +#### Admin frontend service (using PM2 or nginx) For production, use nginx to serve the built frontend. Alternatively, use PM2: @@ -236,7 +227,7 @@ pm2 save pm2 startup ``` -#### Viewer Frontend Service +#### Viewer frontend service ```bash cd /opt/punimtag/viewer-frontend @@ -244,7 +235,7 @@ pm2 start npm --name punimtag-viewer -- start pm2 save ``` -### Step 5: Start Services +### Step 5: start services ```bash # Enable and start services @@ -258,58 +249,35 @@ sudo systemctl status punimtag-api sudo systemctl status punimtag-worker ``` -### Step 6: Verify Deployment +### Step 6: verify deployment -1. **Check API Health:** - ```bash - curl http://:8000/api/v1/health - ``` - -2. **Check API Documentation:** - Open browser: `http://:8000/docs` - -3. **Access Admin Frontend:** - Open browser: `http://:3000` - -4. **Access Viewer Frontend:** - Open browser: `http://:3001` +1. Check API health: `curl http://:8000/api/v1/health` +2. Check API documentation: open `http://:8000/docs` in a browser. +3. Access the admin frontend: open `http://:3000`. +4. Access the viewer frontend: open `http://:3001`. --- ## Production deployment -### Additional Production Considerations +### Additional production considerations -1. **Security:** - - Change all default passwords - - Use strong JWT secrets - - Enable HTTPS with SSL certificates - - Configure firewall rules - - Restrict database access - - Use read-only database users where possible +Security: change all default passwords, use strong JWT secrets, enable +HTTPS with SSL certificates, configure firewall rules, restrict database +access, and use read-only database users where possible. -2. **Performance:** - - Use nginx as reverse proxy - - Enable gzip compression - - Configure caching headers - - Use CDN for static assets (optional) - - Set up database connection pooling - - Configure Redis persistence +Performance: use nginx as a reverse proxy, enable gzip compression, +configure caching headers, use a CDN for static assets if you want one, set +up database connection pooling, and configure Redis persistence. -3. **Monitoring:** - - Set up log aggregation - - Configure health check endpoints - - Set up alerting for service failures - - Monitor database performance - - Track API response times +Monitoring: set up log aggregation, configure health check endpoints, set up +alerting for service failures, and monitor database performance and API +response times. -4. **Backup:** - - Automated database backups - - Photo storage backups - - Configuration file backups - - Disaster recovery plan +Backup: automate database backups, photo storage backups, and configuration +file backups, and have a disaster recovery plan. -### Nginx Configuration Example +### Nginx configuration example ```nginx # /etc/nginx/sites-available/punimtag @@ -362,9 +330,10 @@ server { ## Environment configuration -### Required Environment Variables +### Required environment variables + +Backend (`.env`): -**Backend (.env):** ```bash # Database (PostgreSQL - required) DATABASE_URL=postgresql+psycopg2://user:password@host:5432/punimtag @@ -386,12 +355,14 @@ API_HOST=0.0.0.0 API_PORT=8000 ``` -**Admin Frontend (admin-frontend/.env):** +Admin frontend (`admin-frontend/.env`): + ```bash VITE_API_URL=http://your-api-url:8000 ``` -**Viewer Frontend (viewer-frontend/.env):** +Viewer frontend (`viewer-frontend/.env`): + ```bash DATABASE_URL=postgresql://user:password@host:5432/punimtag DATABASE_URL_AUTH=postgresql://user:password@host:5432/punimtag_auth @@ -403,7 +374,7 @@ NEXTAUTH_SECRET=your-nextauth-secret ## Service management -### Systemd Commands +### Systemd commands ```bash # Start services @@ -427,7 +398,7 @@ sudo journalctl -u punimtag-api -f sudo journalctl -u punimtag-worker -f ``` -### PM2 Commands +### PM2 commands ```bash # Start services @@ -454,46 +425,36 @@ pm2 monit ## Troubleshooting -### Common Issues +### Common issues -1. **Database Connection Errors:** - - Verify PostgreSQL is running - - Check connection string in `.env` - - Verify database exists and user has permissions - - Check firewall rules +Database connection errors: verify PostgreSQL is running, check the +connection string in `.env`, verify the database exists and the user has +permissions, and check firewall rules. -2. **Redis Connection Errors:** - - Verify Redis is running: `redis-cli ping` - - Check Redis URL in `.env` - - Verify Redis is accessible +Redis connection errors: verify Redis is running (`redis-cli ping`), check +the Redis URL in `.env`, and verify Redis is reachable. -3. **API Not Starting:** - - Check logs: `sudo journalctl -u punimtag-api -n 50` - - Verify Python virtual environment is activated - - Check PYTHONPATH is set correctly - - Verify all dependencies are installed +API not starting: check logs (`sudo journalctl -u punimtag-api -n 50`), +verify the Python virtual environment is activated, check that `PYTHONPATH` +is set correctly, and verify all dependencies are installed. -4. **Frontend Build Errors:** - - Clear node_modules and reinstall - - Check Node.js version (18+) - - Verify all environment variables are set +Frontend build errors: clear `node_modules` and reinstall, check the +Node.js version (18+), and verify all environment variables are set. -5. **Worker Not Processing Jobs:** - - Check worker logs: `sudo journalctl -u punimtag-worker -f` - - Verify Redis connection - - Check job queue: `redis-cli` - - Restart worker service +Worker not processing jobs: check worker logs +(`sudo journalctl -u punimtag-worker -f`), verify the Redis connection, +check the job queue (`redis-cli`), and restart the worker service. -### Log Locations +### Log locations -- **Systemd logs**: `sudo journalctl -u ` -- **PM2 logs**: `pm2 logs` -- **Application logs**: `/opt/punimtag/logs/` (if configured) +- Systemd logs: `sudo journalctl -u ` +- PM2 logs: `pm2 logs` +- Application logs: `/opt/punimtag/logs/` (if configured) -### Health Checks +### Health checks ```bash -# API Health +# API health curl http://localhost:8000/api/v1/health # Redis @@ -505,7 +466,7 @@ psql -h -U -d punimtag -c "SELECT 1;" --- -## Deployment Checklist +## Deployment checklist - [ ] Build all frontends - [ ] Transfer files to server @@ -523,8 +484,7 @@ psql -h -U -d punimtag -c "SELECT 1;" --- -**For additional help, see:** -- `README.md` - Main documentation -- `docs/ARCHITECTURE.md` - System architecture -- `docs/USER_GUIDE.md` - User guide +For additional help, see `README.md` (main documentation), +`docs/ARCHITECTURE.md` (system architecture), and `docs/USER_GUIDE.md` +(user guide). diff --git a/docs/DEPLOY_FROM_SCRATCH.md b/docs/DEPLOY_FROM_SCRATCH.md index c938f7a..bd80ea4 100644 --- a/docs/DEPLOY_FROM_SCRATCH.md +++ b/docs/DEPLOY_FROM_SCRATCH.md @@ -1,27 +1,22 @@ -# Deploying PunimTag (From Scratch, Simple) +# Deploying PunimTag from scratch (simple) -> **Deploying via CPanel?** See [`DEPLOY_CPANEL.md`](./DEPLOY_CPANEL.md) for CPanel-specific instructions. +> Deploying via cPanel? See [`DEPLOY_CPANEL.md`](./DEPLOY_CPANEL.md) for cPanel-specific instructions. -This guide is for a **fresh install** where the databases do **not** need to be migrated. -You will start with **empty PostgreSQL databases** and deploy the app from a copy of the repo -(e.g., downloaded from **SharePoint**). +This guide is for a fresh install where the databases do not need to be +migrated: you start with empty PostgreSQL databases and deploy the app from +a copy of the repo (for example, downloaded from SharePoint). -PunimTag is a monorepo with: -- **Backend**: FastAPI (`backend/`) on port **8000** -- **Admin**: React/Vite (`admin-frontend/`) on port **3000** -- **Viewer**: Next.js (`viewer-frontend/`) on port **3001** -- **Jobs**: Redis + RQ worker (`backend/worker.py`) +PunimTag is a monorepo with a FastAPI backend (`backend/`) on port 8000, a +React/Vite admin app (`admin-frontend/`) on port 3000, a Next.js viewer +(`viewer-frontend/`) on port 3001, and a Redis + RQ worker for jobs +(`backend/worker.py`). --- -## Prerequisites (One-time) +## Prerequisites (one-time) -On the server you deploy to, install: -- **Python 3.12+** -- **Node.js 18+** and npm -- **PostgreSQL 12+** -- **Redis 6+** -- **PM2** (`npm i -g pm2`) +On the server you deploy to, install Python 3.12+, Node.js 18+ and npm, +PostgreSQL 12+, Redis 6+, and PM2 (`npm i -g pm2`). Also make sure the server has: - A path for uploaded photos (example: `/punimtag/data/uploads`) @@ -48,7 +43,7 @@ Notes: - If you manage Postgres on a separate host, you only need `postgresql-client` on this server. - If you install Postgres locally, install `postgresql` (server) too, not just the client. -### Firewall Rules (One-time setup) +### Firewall rules (one-time setup) Configure firewall to allow access to the application ports: @@ -58,13 +53,14 @@ sudo ufw allow 3001/tcp # Viewer frontend sudo ufw allow 8000/tcp # Backend API ``` -### PostgreSQL Remote Connection Setup (if using remote database) +### PostgreSQL remote connection setup (if using a remote database) -If your PostgreSQL database is on a **separate server** from the application, you need to configure PostgreSQL to accept remote connections. +If your PostgreSQL database is on a separate server from the application, +you need to configure PostgreSQL to accept remote connections. -**On the PostgreSQL database server:** +On the PostgreSQL database server: -1. **Edit `pg_hba.conf`** to allow connections from your application server: +1. Edit `pg_hba.conf` to allow connections from your application server: ```bash sudo nano /etc/postgresql/*/main/pg_hba.conf ``` @@ -78,7 +74,7 @@ If your PostgreSQL database is on a **separate server** from the application, yo Replace `` with your actual application server IP address. Replace `md5` with `scram-sha-256` if your PostgreSQL version uses that (PostgreSQL 14+). -2. **Edit `postgresql.conf`** to listen on network interfaces: +2. Edit `postgresql.conf` to listen on network interfaces: ```bash sudo nano /etc/postgresql/*/main/postgresql.conf ``` @@ -90,26 +86,26 @@ If your PostgreSQL database is on a **separate server** from the application, yo # listen_addresses = 'localhost,' # Replace with your DB server IP ``` -3. **Restart PostgreSQL** to apply changes: +3. Restart PostgreSQL to apply changes: ```bash sudo systemctl restart postgresql ``` -4. **Configure firewall** on the database server to allow PostgreSQL connections: +4. Configure the firewall on the database server to allow PostgreSQL connections: ```bash sudo ufw allow from to any port 5432 # Replace with your app server IP # OR allow from all (less secure): # sudo ufw allow 5432/tcp ``` -5. **Test the connection** from the application server: +5. Test the connection from the application server: ```bash psql -h -U punim_dev_user -d postgres ``` Replace `` with your database server IP and `punim_dev_user` with your database username. -**Note:** If PostgreSQL is on the same server as the application, you can skip this step and use `localhost` in your connection strings. +Note: if PostgreSQL is on the same server as the application, you can skip this step and use `localhost` in your connection strings. --- @@ -137,7 +133,7 @@ The script will: If you prefer manual steps, continue below. -## Step 1 — Put the code on the server +## Step 1: put the code on the server If you received the code via SharePoint: 1. Download the repo ZIP from SharePoint. @@ -152,7 +148,7 @@ sudo chown -R $USER:$USER /opt/punimtag --- -## Step 2 — Create environment files (rename `_example` → real) +## Step 2: create environment files (rename `_example` to real) ### 2.1 Root env: `/opt/punimtag/.env` @@ -163,7 +159,7 @@ cd /opt/punimtag cp .env.example .env ``` -2. Edit `.env` and set the real values. The template includes **at least**: +2. Edit `.env` and set the real values. The template includes at least: ```bash # PostgreSQL (main database) @@ -184,11 +180,11 @@ PHOTO_STORAGE_DIR=/opt/punimtag/data/uploads REDIS_URL=redis://127.0.0.1:6379/0 ``` -**Important:** If using a **remote PostgreSQL server**, ensure you've completed the "PostgreSQL Remote Connection Setup" steps in the Prerequisites section above before configuring these connection strings. +**Important:** if using a remote PostgreSQL server, complete the "PostgreSQL remote connection setup" steps in the prerequisites section above before configuring these connection strings. Notes: -- The backend **auto-creates tables** on first run if they are missing. -- The backend will also attempt to create the databases **if** the configured Postgres user has +- The backend auto-creates tables on first run if they are missing. +- The backend will also attempt to create the databases if the configured Postgres user has privileges (otherwise create the DBs manually). ### 2.2 Admin env: `/opt/punimtag/admin-frontend/.env` @@ -202,18 +198,18 @@ cp .env.example .env 2. Edit `.env`: -**For direct access (no reverse proxy):** +For direct access (no reverse proxy): ```bash VITE_API_URL=http://YOUR_SERVER_IP_OR_DOMAIN:8000 ``` -**For reverse proxy setup (HTTPS via Caddy/nginx):** +For reverse proxy setup (HTTPS via Caddy/nginx): ```bash # Leave empty to use relative paths - API calls will go through the same proxy VITE_API_URL= ``` -**Important:** When using a reverse proxy (Caddy/nginx) with HTTPS, set `VITE_API_URL` to empty. This allows the frontend to use relative API paths that work correctly with the proxy, avoiding mixed content errors. +**Important:** when using a reverse proxy (Caddy/nginx) with HTTPS, set `VITE_API_URL` to empty. This lets the frontend use relative API paths that work correctly with the proxy, avoiding mixed content errors. ### 2.3 Viewer env: `/opt/punimtag/viewer-frontend/.env` @@ -244,7 +240,7 @@ AUTH_URL=http://YOUR_SERVER_IP_OR_DOMAIN:3001 --- -## Step 3 — Install dependencies +## Step 3: install dependencies From the repo root: @@ -264,7 +260,7 @@ npm ci --- -## Step 4 — Initialize the viewer Prisma clients +## Step 4: initialize the viewer Prisma clients The viewer uses Prisma clients for both DBs. @@ -275,7 +271,7 @@ npm run prisma:generate:all --- -## Step 5 — Create the auth DB tables + admin user +## Step 5: create the auth DB tables and admin user The auth DB schema is set up by the viewer scripts. @@ -291,7 +287,7 @@ npx tsx scripts/fix-admin-user.ts --- -## Step 6 — Build frontends +## Step 6: build frontends Build the frontend applications for production: @@ -310,7 +306,7 @@ The viewer frontend build creates an optimized Next.js production build. --- -## Step 7 — Configure PM2 +## Step 7: configure PM2 This repo includes a PM2 config template. If `ecosystem.config.js` doesn't exist, copy it from the example: @@ -326,7 +322,7 @@ Edit `ecosystem.config.js` and update: --- -## Step 8 — Start the services (PM2) +## Step 8: start the services (PM2) Start all services using PM2: @@ -344,7 +340,7 @@ pm2 startup --- -## Step 9 — First-run DB initialization (automatic) +## Step 9: first-run DB initialization (automatic) On first startup, the backend will connect to Postgres and create missing tables automatically. @@ -362,21 +358,21 @@ curl -sS http://127.0.0.1:3001/api/health --- -## Step 10 — Open the apps +## Step 10: open the apps -- **Admin**: `http://YOUR_SERVER:3000` -- **Viewer**: `http://YOUR_SERVER:3001` -- **API docs**: `http://YOUR_SERVER:8000/docs` +- Admin: `http://YOUR_SERVER:3000` +- Viewer: `http://YOUR_SERVER:3001` +- API docs: `http://YOUR_SERVER:8000/docs` --- -## Step 11 — Reverse Proxy Setup (HTTPS via Caddy/nginx) +## Step 11: reverse proxy setup (HTTPS via Caddy/nginx) -If you're using a reverse proxy (Caddy, nginx, etc.) to serve the application over HTTPS, configure it to route `/api/*` requests to the backend **before** serving static files. +If you're using a reverse proxy (Caddy, nginx, etc.) to serve the application over HTTPS, configure it to route `/api/*` requests to the backend before serving static files. -The proxy must forward `/api/*` requests to the backend (port 8000) **before** trying to serve static files. +The proxy must forward `/api/*` requests to the backend (port 8000) before trying to serve static files. -#### Caddy Configuration +#### Caddy configuration Update your Caddyfile on the proxy server: @@ -412,7 +408,7 @@ your-admin-domain.com { } ``` -**Important:** The `handle /api/*` block **must come before** the general `reverse_proxy` directive. +**Important:** the `handle /api/*` block must come before the general `reverse_proxy` directive. After updating: ```bash @@ -423,7 +419,7 @@ caddy validate --config /path/to/Caddyfile sudo systemctl reload caddy ``` -#### Nginx Configuration +#### Nginx configuration ```nginx server { @@ -458,7 +454,7 @@ sudo nginx -t sudo systemctl reload nginx ``` -### Environment Variable Setup +### Environment variable setup When using a reverse proxy, ensure `admin-frontend/.env` has: @@ -474,7 +470,7 @@ This allows the frontend to use relative API paths (`/api/v1/...`) that work cor ### API requests return HTML instead of JSON -1. Ensure your reverse proxy (Caddy/nginx) routes `/api/*` requests to the backend **before** serving static files (see Step 11 above). +1. Ensure your reverse proxy (Caddy/nginx) routes `/api/*` requests to the backend before serving static files (see Step 11 above). 2. Verify `admin-frontend/.env` has `VITE_API_URL=` (empty) when using a proxy. 3. Rebuild the frontend after changing `.env`: `cd admin-frontend && npm run build && pm2 restart punimtag-admin` diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index b8d41e8..ded6210 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -17,457 +17,277 @@ Viewer browsing is separate (port 3001). ### First-time setup -1. **Start the Application** - - Ensure Redis is running - - Start the backend API server - - Start the frontend development server - - Open your browser to `http://localhost:3000` - -2. **Login** - - Default credentials: - - Use credentials from `.env` (`ADMIN_USERNAME` / `ADMIN_PASSWORD`) - - Change these credentials before any shared deploy. - -3. **Initial Workflow** - - **Scan** → Import your photos - - **Process** → Detect faces in photos - - **Identify** → Name the people in your photos - - **Auto-Match** → Automatically match similar faces to previously identified people - - **Search** → Find photos by people, dates, or tags - - **Tags** → Tag photos and manage tags +1. Start the application: make sure Redis is running, then start the backend + API server and the frontend dev server, and open `http://localhost:3000`. +2. Log in with the credentials from `.env` (`ADMIN_USERNAME` / `ADMIN_PASSWORD`). + Change these before any shared deploy. +3. The usual first workflow is Scan (import photos), Process (detect faces), + Identify (name the people in your photos), Auto-Match (match similar faces + to people you've already identified), Search (find photos by people, dates, + or tags), and Tags (tag photos and manage tags). ## Navigation -The application uses a **left sidebar navigation** with the following pages: +The left sidebar has these pages: -- **Dashboard** - Overview and statistics -- **Scan** - Import photos from folders or upload files -- **Process** - Detect and process faces in photos -- **Identify** - Manually identify people in faces -- **Auto-Match** - Automatically match similar faces to previously identified faces -- **Search** - Search and filter photos -- **Modify** - Edit person information -- **Tags** - Tag photos and manage photo tags -- **Faces Maintenance** - Manage face data -- **Settings** - Application settings +- Dashboard: overview and statistics +- Scan: import photos from folders or upload files +- Process: detect and process faces in photos +- Identify: manually identify people in faces +- Auto-Match: automatically match similar faces to previously identified faces +- Search: search and filter photos +- Modify: edit person information +- Tags: tag photos and manage photo tags +- Faces Maintenance: manage face data +- Settings: application settings ## Pages -### Login Page +### Login page -Purpose: Authenticate and access the application +Authenticates you and gets you into the app: username/password, JWT-based, +with an automatic redirect to the dashboard after login. -Features: -- Username and password login -- JWT-based authentication -- Automatic redirect to dashboard after login +How to use it: +1. Enter your username (`ADMIN_USERNAME` from `.env`). +2. Enter your password (`ADMIN_PASSWORD` from `.env`). +3. Click "Login". +4. You'll land on the Dashboard. -How to use: -1. Enter your username (`ADMIN_USERNAME` from `.env`) -2. Enter your password (`ADMIN_PASSWORD` from `.env`) -3. Click "Login" button -4. You'll be redirected to the Dashboard - -Notes: -- Session persists until logout -- Default credentials are for development only -- Change credentials in production for security +The session persists until logout. Default credentials are for development +only; change them before production use. ### Dashboard -Purpose: Overview of your photo collection and statistics - -Features: -- Collection statistics -- Quick access to main features -- Recent activity summary - -How to use: -- View statistics about your photo collection -- Navigate to other pages using the sidebar -- Monitor overall system status - -Status: Basic implementation - more stats may be added later - - -### Scan Page - -Purpose: Import photos into your collection from folders or upload files - -Features: -- **Folder Selection**: Browse and select folders containing photos -- **Recursive Scanning**: Option to scan subfolders recursively -- **Duplicate Detection**: Automatically detects and skips duplicate photos -- **Real-time Progress**: Live progress tracking during import - -How to use: - -**Folder Scan** -1. Click "Browse Folder" button -2. Select a folder containing photos -3. Toggle "Recursive" if you want to include subfolders -4. Click "Start Scan" button -5. Monitor progress in the progress bar -6. View results (photos added, existing photos skipped) - -**What Happens**: -- Photos are copied to `data/uploads` directory -- EXIF metadata is extracted (date taken, orientation, etc.) -- Duplicate detection by checksum -- Photos are added to database -- Faces are NOT detected yet (use Process page for that) - -**Tips**: -- Large folders may take time - be patient! - - -### Process Page - -Purpose: Detect faces in imported photos and generate face encodings - -Features: -- **face detection method used - `retinaface` - Best accuracy, medium speed - -- **Face recognition model used - `ArcFace` - Best accuracy, medium speed -- **Batch Size**: Configure how many photos to process at once -- **Real-time Progress**: Live progress tracking -- **Job Cancellation**: Stop processing if needed - -How to use: -1. Optionally set **Batch Size** (leave empty for default) -2. Click "Start Processing" button -3. Monitor progress: - - Progress bar shows overall completion - - Photo count shows photos processed - - Face count shows faces detected and stored -4. Wait for completion or click "Stop Processing" to cancel - -**What Happens**: -- DeepFace analyzes each unprocessed photo -- Faces are detected and located -- 512-dimensional face encodings are generated -- Face metadata is stored (confidence, quality, location) -- Photos are marked as processed - -**Tips**: -- You can cancel and resume later - - -### Identify Page - -Purpose: Manually identify people in detected faces - -Features: -- **Face Navigation**: Browse through unidentified faces -- **Person Creation**: Create new person records -- **Similar Faces Panel**: View similar faces for comparison -- **Confidence Display**: See match confidence percentages -- **Date Filtering**: Filter faces by date taken or processed -- **Unique Faces Filter**: Hide duplicate faces of same person -- **Face Information**: View face metadata (confidence, quality, detector/model) - -How to use: - -**Basic Identification**: -1. Navigate to Identify page -2. View the current face on the left panel -3. Select existing person from the drop down list or create new person below -4. Enter person information: - - First Name (required) - - Last Name (required) - - Middle Name (optional) - - Maiden Name (optional) - - Date of Birth (optional) - - Email (optional) - - Phone (optional) -5. Click "Identify" button to identify the face - -**Using Similar Faces**: -1. Toggle "Compare" checkbox to show similar faces -2. View similar faces in the right panel -3. See confidence percentages (color-coded) -4. Select similar faces to bulk identify with the current left face -5. Click "Identify" button to bulk identify left and all selected on the right faces to that person. - -**Navigation**: -- Use "Next" button to move to next unidentified face -- Use "Back" button to go to previous face -- Use filters to narrow down faces to identify - -**Unique Faces Filter**: -- Check "Unique faces only" to hide duplicates -- Shows only one representative face per person -- Similar faces panel remains unfiltered - -**Confidence Colors**: -- **80%+** = Very High (Almost Certain) -- **70%+** = High (Likely Match) -- **60%+** = Medium (Possible Match) -- **50%+** = Low (Questionable) -- **<50%** = Very Low (Unlikely) - -**Tips**: -- Use similar faces to identify groups of photos -- Date filtering helps focus on specific time periods -- Unique faces filter reduces clutter -- Confidence scores help prioritize identification - - -### Auto-Match Page - -Purpose: Automatically match unidentified faces to identified people - -Features: -- **Person-Centric View**: Shows identified person on left, matches on right -- **Checkbox Selection**: Select which faces to identify -- **Confidence Display**: Color-coded match confidence -- **Batch Identification**: Identify multiple faces at once -- **Navigation**: Move between different people - -How to use: - -**Manual Match Workflow**: -1. Navigate to Auto-Match page -2. Faces load automatically on page load -3. View identified person on the left panel -4. View matching unidentified faces on the right panel -5. Check boxes next to faces you want to identify -6. Click "Save changes for [Person Name]" button -7. Use "Next" and "Back" buttons to navigate between people - -***Automatic Match Workflow**: -1. Navigate to Auto-Match page -2. Click Run Auto-Match button -3. All unidentified faces will be matched to identified faces based on the following clitiria: -simillarity higher than 70% -picture quality higher than 50% -profile faces are excluded for better accuracy - -**Understanding Matches**: -- Left panel shows an identified person with their reference face -- Right panel shows unidentified faces that match this person -- Confidence percentages show match quality -- Only faces above threshold are shown - -**Selecting Faces**: -- Check boxes to select faces to identify -- Uncheck boxes to deselect -- Previously identified faces are pre-checked -- Selections persist when navigating between people - -**Saving Changes**: -- Click "Save changes" button in left panel -- Only checked faces are identified -- Changes are saved immediately -- You can go back and uncheck faces to unidentify them - -**Navigation**: -- "Next" button moves to next person (disabled on last person) -- "Back" button moves to previous person (disabled on first person) -- Navigation preserves checkbox selections - -**Tips**: -- Review matches carefully before saving -- Use confidence scores to guide decisions -- You can correct mistakes by going back and unchecking -- High confidence matches (>70%) are usually accurate - - -### Search Page - -Purpose: Search and filter photos by various criteria - -Features: -- **People Filter**: Filter by identified people -- **Date Filter**: Filter by date taken or date added -- **Tag Filter**: Filter by photo tags -- **Folder Filter**: Filter by source folder -- **Photo Grid**: Virtualized grid of matching photos -- **Pagination**: Navigate through search results - -How to use: - -**Basic Search**: -1. Navigate to Search page -2. Use filter dropdowns to set criteria: - - **People**: Select one or more people - - **Date Taken**: Set date range - - **Date Added**: Set date range - - **Tags**: Select one or more tags - - **Folder**: Select source folder -3. Click "Search" or filters apply automatically -4. View matching photos in the grid -5. Click on photos to view details - -**Filters**: -- Multiple filters can be combined -- All filters work together (AND logic) -- Clear individual filters or reset all - -**Photo Grid**: -- Virtualized for performance with large result sets -- Scroll to load more photos -- Click photos for details or actions - -**Tips**: -- Combine filters for precise searches -- Use date ranges to find photos from specific periods -- Tag filtering helps find themed photos -- People filter is most useful after identification - - -### Modify Page - -Purpose: Edit person information and manage person records - -Features: -- **Person Selection**: Choose person to edit -- **Information Editing**: Update names and date of birth -- **Face Management**: View and manage person's faces -- **Person Deletion**: Remove person records (with confirmation) - -How to use: - -**Editing Person Information**: -1. Navigate to Modify page -2. Select person from dropdown -3. Edit information fields: - - First Name - - Last Name - - Middle Name - - Maiden Name - - Date of Birth -4. Click "Save Changes" button - -**Managing Faces**: -- View all faces associated with person -- See face thumbnails -- Remove faces from person if needed (unmatch) - -**Tips**: -- Use this to correct mistakes in identification -- Update names if you learn more information -- Be careful with deletion - it's permanent - - -### Tags Page - -Purpose: Manage photo tags and tag-photo relationships - -Features: -- **Tag List**: View all existing tags -- **Tag Creation**: Create new tags -- **Tag Editing**: Edit tag names -- **Tag Deletion**: Remove tags -- **Photo-Tag Linkage**: Assign tags to photos - -How to use: - -**Creating Tags**: -1. Navigate to Tags page -2. Click "Manage Tags" button -3. Enter new tag name -4. Click "Add tag" button - -**Managing Tags**: -- Click "Manage Tags" button -- View all tags in a list -- Edit tag names by clicking edit button -- Delete tags by selecting it's check box and clicking "Delete selected tags" button -- **Warning**: Deleting a tag removes it from all photos - -**Assigning Tags to Photos**: -- Select photos from Tags page(or from Search page) -- Use tag assignment interface -- Tags can be assigned to multiple photos at once by selecting multiple photos and clicking "Tag Selected Photos" button -- Tags can be assigned to all photos in a specific folder at once by clicking the linkage icon next to the folder name -- Tags can be assigned to a single photo by clicking the linkage icon on the right of each photo - -**Tips**: -- Use descriptive tag names -- Create tags for events, locations, themes -- Tags help organize and find photos later -Note: Tags are case insensitive!********* - -### Faces Maintenance Page - -Purpose: Remove unwanted faces - mainly due to low quality face detections - -Features: -- **Face List**: View all faces in database -- **Face Filtering**: Filter quality -- **Face Deletion**: Remove unwanted faces -- **Bulk Operations**: Perform actions on multiple faces +Shows an overview of your photo collection: collection statistics, quick +links to the main features, and a recent activity summary. Use it to check +overall system status and jump to other pages from the sidebar. -How to use: +This is a basic implementation; more stats may be added later. -**Viewing Faces**: -1. Navigate to Faces Maintenance page -2. View list of all faces -3. See face thumbnails and metadata -4. Filter faces by quality (delete faces with low quality) -**Deleting Faces**: -- Select faces to delete -- Click "Delete Selected" button -- Confirm deletion -- **Warning**: Deletion is permanent +### Scan page -**Bulk Operations**: -- Select multiple faces -- Perform bulk actions (delete, unidentify, etc.) -- Useful for cleaning up database - -**Tips**: -- Remove low-quality face detections -- Regular maintenance keeps database clean +Imports photos into your collection from folders or uploaded files. It scans +subfolders recursively if you ask it to, skips duplicates automatically, and +shows live progress during import. +How to use it: +1. Click "Browse Folder". +2. Select a folder containing photos. +3. Toggle "Recursive" if you want to include subfolders. +4. Click "Start Scan". +5. Watch the progress bar. +6. Check the results (photos added, existing photos skipped). + +What happens during a scan: photos are copied to `data/uploads`, EXIF +metadata is extracted (date taken, orientation, etc.), duplicates are caught +by checksum, and photos are added to the database. Faces are not detected +yet; that happens on the Process page. -### Settings Page +Large folders take time, so plan for that. -Purpose: Configure application settings and preferences +### Process page +Detects faces in imported photos and generates face encodings. It uses +RetinaFace for face detection and ArcFace for face recognition, both chosen +for best accuracy at medium speed. You can set a batch size, watch progress +live, and cancel a job if needed. -### Complete Workflow: Import and Identify Photos - -1. **Scan Photos** - - Go to Scan page - - Browse folder or upload files - - Wait for import to complete +How to use it: +1. Optionally set a batch size (leave empty for the default). +2. Click "Start Processing". +3. Watch progress: the progress bar shows overall completion, the photo + count shows photos processed, and the face count shows faces detected + and stored. +4. Wait for it to finish, or click "Stop Processing" to cancel. -2. **Process Faces** - - Go to Process page - - Select detector and model (or use defaults) - - Click "Start Processing" - - Wait for face detection to complete +What happens: DeepFace analyzes each unprocessed photo, detects and locates +faces, generates 512-dimensional face encodings, stores face metadata +(confidence, quality, location), and marks photos as processed. -3. **Identify People** - - Go to Identify page - - Browse through faces - - Create person records and identify faces - - Use similar faces to help identification +You can cancel a run and resume later without losing progress. -4. **Auto-Match Remaining** - - Go to Auto-Match page - - Review automatic matches - - Select faces to identify - - Save changes -5. **Search and Organize** - - Go to Search page - - Filter by people, dates, or tags - - Find specific photos - - Assign tags for organization +### Identify page +For manually identifying people in detected faces. It lets you browse +unidentified faces, create new person records, compare against a similar +faces panel, see match confidence percentages, filter by date, hide +duplicate faces of the same person, and view face metadata (confidence, +quality, detector/model). -**Last Updated**: October 2025 -**Version**: 1.0 -**Application**: PunimTag Web +Basic identification: +1. Go to the Identify page. +2. Look at the current face in the left panel. +3. Pick an existing person from the dropdown, or create a new one below it. +4. Enter person details: first name and last name are required; middle name, + maiden name, date of birth, email, and phone are optional. +5. Click "Identify". +Using similar faces: +1. Toggle "Compare" to show similar faces. +2. Look at similar faces in the right panel, with color-coded confidence + percentages. +3. Select similar faces to bulk-identify along with the current one. +4. Click "Identify" to apply the identification to the left face and any + selected faces on the right. + +Navigation: "Next" moves to the next unidentified face, "Back" goes to the +previous one, and filters narrow down which faces you see. + +The "Unique faces only" filter hides duplicates and shows one representative +face per person; the similar faces panel stays unfiltered. + +Confidence colors: 80%+ is very high (almost certain), 70%+ is high (likely +match), 60%+ is medium (possible match), 50%+ is low (questionable), and +below 50% is very low (unlikely). + +Tips: use similar faces to identify groups of photos at once, use date +filtering to focus on a specific period, and use confidence scores to +prioritize which faces to work on first. + + +### Auto-Match page + +Automatically matches unidentified faces to people you've already +identified. It shows the identified person on the left and their matches on +the right, with checkboxes to select which faces to identify, color-coded +confidence, and navigation between people. + +Manual match workflow: +1. Go to the Auto-Match page. +2. Faces load automatically. +3. The identified person shows on the left panel, with matching unidentified + faces on the right. +4. Check the boxes next to faces you want to identify. +5. Click "Save changes for [Person Name]". +6. Use "Next" and "Back" to move between people. + +Automatic match workflow: +1. Go to the Auto-Match page. +2. Click "Run Auto-Match". +3. All unidentified faces are matched to identified faces using these + criteria: similarity above 70%, picture quality above 50%, and profile + faces excluded (they hurt accuracy). + +The left panel shows the identified person's reference face; the right +panel shows unidentified faces that match them, with confidence percentages. +Only faces above the threshold are shown. + +Checkboxes select which faces to identify. Previously identified faces are +pre-checked, and selections persist as you navigate between people. Saving +only applies to checked faces, applies immediately, and can be undone by +going back and unchecking. + +Review matches carefully before saving; confidence above 70% is usually +accurate, but check anyway. + + +### Search page + +Searches and filters photos by person, date, tag, or folder. Results show in +a virtualized grid with pagination. + +How to use it: +1. Go to the Search page. +2. Set filters: people (one or more), date taken, date added, tags (one or + more), and folder. +3. Click "Search", or let filters apply automatically. +4. Browse matching photos in the grid. +5. Click a photo for details. + +Filters combine with AND logic, and you can clear individual filters or +reset all of them. The grid is virtualized so it stays responsive with large +result sets; scroll to load more, click photos for details or actions. + +People filtering is most useful once you've done some identification work. + + +### Modify page + +Edits person information and manages person records: choose a person, edit +their name fields and date of birth, view and manage their faces, or delete +the person record (with confirmation). + +Editing person information: +1. Go to the Modify page. +2. Select a person from the dropdown. +3. Edit first name, last name, middle name, maiden name, or date of birth. +4. Click "Save Changes". + +Managing faces: view all faces associated with the person, see thumbnails, +and remove faces from the person if needed (unmatch). + +Use this page to correct identification mistakes or update names as you +learn more. Deletion is permanent, so be careful with it. + + +### Tags page + +Manages photo tags and which photos they're linked to: a tag list, tag +creation and editing, tag deletion, and assigning tags to photos. + +Creating tags: +1. Go to the Tags page. +2. Click "Manage Tags". +3. Enter a new tag name. +4. Click "Add tag". + +Managing tags: click "Manage Tags" to see the full list, edit names with the +edit button, or delete a tag by checking its box and clicking "Delete +selected tags". Deleting a tag removes it from all photos. + +Assigning tags to photos: select photos from the Tags page or the Search +page, then use the tag assignment interface. You can tag multiple selected +photos at once with "Tag Selected Photos", tag every photo in a folder via +the linkage icon next to the folder name, or tag a single photo via the +linkage icon next to that photo. + +Use descriptive names and create tags for events, locations, or themes so +they're easier to find later. Tags are case-insensitive. + + +### Faces Maintenance page + +For removing unwanted faces, mainly low-quality detections: a full face +list, quality filtering, deletion, and bulk operations. + +Viewing faces: +1. Go to the Faces Maintenance page. +2. Browse the list of all faces, with thumbnails and metadata. +3. Filter by quality to find faces worth deleting. + +Deleting faces: select the faces, click "Delete Selected", and confirm. +Deletion is permanent. + +Bulk operations let you act on multiple faces at once (delete, unidentify, +etc.), which is useful for general database cleanup. + + +### Settings page + +Configures application settings and preferences. + + +### Complete workflow: import and identify photos + +1. Scan photos: go to Scan, browse a folder or upload files, wait for the + import to finish. +2. Process faces: go to Process, pick a detector and model (or use the + defaults), click "Start Processing", and wait for detection to finish. +3. Identify people: go to Identify, browse faces, create person records, + and use similar faces to help. +4. Auto-match remaining: go to Auto-Match, review the automatic matches, + select faces to identify, and save. +5. Search and organize: go to Search, filter by people, dates, or tags, find + what you need, and assign tags for organization. + + +Last updated October 2025 (version 1.0, PunimTag Web). *For technical details and development information, see the main README.md* -