ilia/punimtag
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Update architecture documentation and add auto-match load analysis

Browse Source 
This commit updates the `ARCHITECTURE.md` file to reflect the transition to a web-based application, including new features and system overview. Additionally, it introduces `AUTOMATCH_LOAD_ANALYSIS.md`, detailing performance issues with the Auto-Match page and recommendations for optimizations. A new document, `CONFIDENCE_CALIBRATION_SUMMARY.md`, is also added to explain the implementation of a confidence calibration system for face recognition, ensuring more accurate match probabilities. These updates enhance the project's documentation and provide insights for future improvements.
This commit is contained in:
Tanya
2026-01-06 13:11:30 -05:00
parent 3ec0da1573
commit 845b3f3b87
25 changed files with 7544 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/DEPLOYMENT.md
+531
View File
@@ -0,0 +1,531 @@
# PunimTag Deployment Guide
**Last Updated:** January 6, 2026
This guide covers deployment of PunimTag to development and production environments.
---
## Table of Contents
1. [Prerequisites](#prerequisites)
2. [Development Server Deployment](#development-server-deployment)
3. [Production Deployment](#production-deployment)
4. [Environment Configuration](#environment-configuration)
5. [Service Management](#service-management)
6. [Troubleshooting](#troubleshooting)
---
## Prerequisites
### Server Requirements
- **Operating System**: Linux (Ubuntu 20.04+ recommended)
- **Python**: 3.12 or higher
- **Node.js**: 18+ and npm 9+
- **PostgreSQL**: 12+ (required)
- **Redis**: 6+ (required for background jobs)
- **System Memory**: Minimum 4GB RAM (8GB+ recommended for large photo collections)
- **Disk Space**: Sufficient space for photos and database
### Development Server Information
**Development Server:**
- **Host**: 10.0.10.121
- **User**: appuser
- **Password**: C0caC0la
**Development Database:**
- **Host**: 10.0.10.181
- **Port**: 5432
- **User**: ladmin
- **Password**: C0caC0la
---
## Development Server Deployment
### Step 1: Build Applications
On your local machine or CI/CD pipeline:
```bash
# Install dependencies (if not already done)
npm run install:all
# Build all frontends
npm run build:all
# Or use the deployment script
npm run deploy:dev
```
This will:
- Build admin-frontend for production
- Build viewer-frontend for production
- Prepare deployment package
### Step 2: Transfer Files to Server
```bash
# Create deployment directory structure
ssh appuser@10.0.10.121 "mkdir -p /opt/punimtag/{backend,admin-frontend,viewer-frontend,data/uploads,logs}"
# Transfer backend
scp -r backend/* appuser@10.0.10.121:/opt/punimtag/backend/
scp requirements.txt appuser@10.0.10.121:/opt/punimtag/
# Transfer built frontends
scp -r admin-frontend/dist/* appuser@10.0.10.121:/opt/punimtag/admin-frontend/
scp -r viewer-frontend/.next/* appuser@10.0.10.121:/opt/punimtag/viewer-frontend/
# Transfer configuration files
scp .env.example appuser@10.0.10.121:/opt/punimtag/.env
```
### Step 3: Server Setup
SSH into the development server:
```bash
ssh appuser@10.0.10.121
cd /opt/punimtag
```
#### Install System Dependencies
```bash
# Update package list
sudo apt update
# Install PostgreSQL client (if not already installed)
sudo apt install -y postgresql-client
# Install Redis (if not already installed)
sudo apt install -y redis-server
sudo systemctl start redis-server
sudo systemctl enable redis-server
# Install Python dependencies
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
#### Configure Environment
Edit `.env` file:
```bash
nano .env
```
Set the following variables:
```bash
# Development Database
DATABASE_URL=postgresql+psycopg2://ladmin:C0caC0la@10.0.10.181:5432/punimtag
DATABASE_URL_AUTH=postgresql+psycopg2://ladmin:C0caC0la@10.0.10.181:5432/punimtag_auth
# JWT Secrets (change in production!)
SECRET_KEY=dev-secret-key-change-in-production
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin
# Photo storage
PHOTO_STORAGE_DIR=/opt/punimtag/data/uploads
# Redis
REDIS_URL=redis://localhost:6379/0
# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
```
#### Configure Frontend Environment
**Admin Frontend:**
Create `admin-frontend/.env`:
```bash
VITE_API_URL=http://10.0.10.121:8000
```
**Viewer Frontend:**
Create `viewer-frontend/.env`:
```bash
DATABASE_URL=postgresql://ladmin:C0caC0la@10.0.10.181:5432/punimtag
DATABASE_URL_AUTH=postgresql://ladmin:C0caC0la@10.0.10.181:5432/punimtag_auth
NEXTAUTH_URL=http://10.0.10.121:3001
NEXTAUTH_SECRET=dev-secret-key-change-in-production
```
Generate Prisma client:
```bash
cd viewer-frontend
npx prisma generate
cd ..
```
### Step 4: Set Up Systemd Services
Create systemd service files for automatic startup:
#### Backend API Service
```bash
sudo nano /etc/systemd/system/punimtag-api.service
```
```ini
[Unit]
Description=PunimTag API Server
After=network.target postgresql.service redis.service
[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/punimtag
Environment="PATH=/opt/punimtag/venv/bin"
Environment="PYTHONPATH=/opt/punimtag"
ExecStart=/opt/punimtag/venv/bin/uvicorn backend.app:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
#### RQ Worker Service
```bash
sudo nano /etc/systemd/system/punimtag-worker.service
```
```ini
[Unit]
Description=PunimTag RQ Worker
After=network.target postgresql.service redis.service
[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/punimtag
Environment="PATH=/opt/punimtag/venv/bin"
Environment="PYTHONPATH=/opt/punimtag"
ExecStart=/opt/punimtag/venv/bin/python -m rq worker --url redis://localhost:6379/0
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.target
```
#### Admin Frontend Service (using PM2 or nginx)
For production, use nginx to serve the built frontend. Alternatively, use PM2:
```bash
npm install -g pm2
cd /opt/punimtag/admin-frontend
pm2 serve dist 3000 --name punimtag-admin
pm2 save
pm2 startup
```
#### Viewer Frontend Service
```bash
cd /opt/punimtag/viewer-frontend
pm2 start npm --name punimtag-viewer -- start
pm2 save
```
### Step 5: Start Services
```bash
# Enable and start services
sudo systemctl enable punimtag-api
sudo systemctl enable punimtag-worker
sudo systemctl start punimtag-api
sudo systemctl start punimtag-worker
# Check status
sudo systemctl status punimtag-api
sudo systemctl status punimtag-worker
```
### Step 6: Verify Deployment
1. **Check API Health:**
```bash
curl http://10.0.10.121:8000/api/v1/health
```
2. **Check API Documentation:**
Open browser: `http://10.0.10.121:8000/docs`
3. **Access Admin Frontend:**
Open browser: `http://10.0.10.121:3000`
4. **Access Viewer Frontend:**
Open browser: `http://10.0.10.121:3001`
---
## Production Deployment
### Additional Production Considerations
1. **Security:**
- Change all default passwords
- Use strong JWT secrets
- Enable HTTPS with SSL certificates
- Configure firewall rules
- Restrict database access
- Use read-only database users where possible
2. **Performance:**
- Use nginx as reverse proxy
- Enable gzip compression
- Configure caching headers
- Use CDN for static assets (optional)
- Set up database connection pooling
- Configure Redis persistence
3. **Monitoring:**
- Set up log aggregation
- Configure health check endpoints
- Set up alerting for service failures
- Monitor database performance
- Track API response times
4. **Backup:**
- Automated database backups
- Photo storage backups
- Configuration file backups
- Disaster recovery plan
### Nginx Configuration Example
```nginx
# /etc/nginx/sites-available/punimtag
# Admin Frontend
server {
listen 80;
server_name admin.punimtag.example.com;
root /opt/punimtag/admin-frontend/dist;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /api {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
# Viewer Frontend
server {
listen 80;
server_name viewer.punimtag.example.com;
location / {
proxy_pass http://127.0.0.1:3001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
# API (optional, if exposing directly)
server {
listen 80;
server_name api.punimtag.example.com;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}
```
---
## Environment Configuration
### Required Environment Variables
**Backend (.env):**
```bash
# Database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://user:password@host:5432/punimtag
DATABASE_URL_AUTH=postgresql+psycopg2://user:password@host:5432/punimtag_auth
# JWT Configuration
SECRET_KEY=your-secret-key-here
ADMIN_USERNAME=admin
ADMIN_PASSWORD=secure-password
# Photo Storage
PHOTO_STORAGE_DIR=/path/to/photos
# Redis
REDIS_URL=redis://localhost:6379/0
# API Configuration
API_HOST=0.0.0.0
API_PORT=8000
```
**Admin Frontend (admin-frontend/.env):**
```bash
VITE_API_URL=http://your-api-url:8000
```
**Viewer Frontend (viewer-frontend/.env):**
```bash
DATABASE_URL=postgresql://user:password@host:5432/punimtag
DATABASE_URL_AUTH=postgresql://user:password@host:5432/punimtag_auth
NEXTAUTH_URL=http://your-viewer-url:3001
NEXTAUTH_SECRET=your-nextauth-secret
```
---
## Service Management
### Systemd Commands
```bash
# Start services
sudo systemctl start punimtag-api
sudo systemctl start punimtag-worker
# Stop services
sudo systemctl stop punimtag-api
sudo systemctl stop punimtag-worker
# Restart services
sudo systemctl restart punimtag-api
sudo systemctl restart punimtag-worker
# Check status
sudo systemctl status punimtag-api
sudo systemctl status punimtag-worker
# View logs
sudo journalctl -u punimtag-api -f
sudo journalctl -u punimtag-worker -f
```
### PM2 Commands
```bash
# Start services
pm2 start punimtag-admin
pm2 start punimtag-viewer
# Stop services
pm2 stop punimtag-admin
pm2 stop punimtag-viewer
# Restart services
pm2 restart punimtag-admin
pm2 restart punimtag-viewer
# View logs
pm2 logs punimtag-admin
pm2 logs punimtag-viewer
# Monitor
pm2 monit
```
---
## Troubleshooting
### Common Issues
1. **Database Connection Errors:**
- Verify PostgreSQL is running
- Check connection string in `.env`
- Verify database exists and user has permissions
- Check firewall rules
2. **Redis Connection Errors:**
- Verify Redis is running: `redis-cli ping`
- Check Redis URL in `.env`
- Verify Redis is accessible
3. **API Not Starting:**
- Check logs: `sudo journalctl -u punimtag-api -n 50`
- Verify Python virtual environment is activated
- Check PYTHONPATH is set correctly
- Verify all dependencies are installed
4. **Frontend Build Errors:**
- Clear node_modules and reinstall
- Check Node.js version (18+)
- Verify all environment variables are set
5. **Worker Not Processing Jobs:**
- Check worker logs: `sudo journalctl -u punimtag-worker -f`
- Verify Redis connection
- Check job queue: `redis-cli`
- Restart worker service
### Log Locations
- **Systemd logs**: `sudo journalctl -u <service-name>`
- **PM2 logs**: `pm2 logs`
- **Application logs**: `/opt/punimtag/logs/` (if configured)
### Health Checks
```bash
# API Health
curl http://localhost:8000/api/v1/health
# Redis
redis-cli ping
# PostgreSQL
psql -h 10.0.10.181 -U ladmin -d punimtag -c "SELECT 1;"
```
---
## Deployment Checklist
- [ ] Build all frontends
- [ ] Transfer files to server
- [ ] Install system dependencies
- [ ] Set up Python virtual environment
- [ ] Configure environment variables
- [ ] Set up systemd services
- [ ] Configure nginx (production)
- [ ] Set up SSL certificates (production)
- [ ] Configure firewall rules
- [ ] Test all services
- [ ] Set up monitoring
- [ ] Configure backups
- [ ] Document deployment process
---
**For additional help, see:**
- `README.md` - Main documentation
- `docs/ARCHITECTURE.md` - System architecture
- `docs/USER_GUIDE.md` - User guide