PunimTag Web

Modern Photo Management and Facial Recognition System

A fast, simple, and modern web application for organizing and tagging photos using state-of-the-art DeepFace AI with ArcFace recognition model.

---

🎯 Features

• 🌐 Web-Based: Modern React frontend with FastAPI backend
• 🔥 DeepFace AI: State-of-the-art face detection with RetinaFace and ArcFace models
• 🎯 Superior Accuracy: 512-dimensional embeddings (4x more detailed than face_recognition)
• ⚙️ Multiple Detectors: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors
• 🎨 Flexible Models: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models
• 👤 Person Identification: Identify and tag people across your photo collection
• 🤖 Smart Auto-Matching: Intelligent face matching with quality scoring and cosine similarity
• 📊 Confidence Calibration: Empirical-based confidence scores for realistic match probabilities
• 🔍 Advanced Search: Search by people, dates, tags, and folders
• 🏷️ Tag Management: Organize photos with hierarchical tags
• ⚡ Batch Processing: Process thousands of photos efficiently
• 🎯 Unique Faces Filter: Hide duplicate faces to focus on unique individuals
• 🔄 Real-time Updates: Live progress tracking and job status updates
• 🔒 Privacy-First: All data stored locally, no cloud dependencies

---

🚀 Quick Start

Prerequisites

• Python 3.12 or higher (with pip)
• Node.js 18+ and npm
• PostgreSQL (for production, optional for development with SQLite)
• Redis (for background job processing)

Note: The automated installation script (./install.sh) will install PostgreSQL and Redis automatically on Ubuntu/Debian systems.

Installation

Option 1: Automated Installation (Recommended for Linux/Ubuntu/Debian)

The automated installation script will install all system dependencies, Python packages, frontend dependencies, and set up databases:

# Clone the repository
git clone <repository-url>
cd punimtag

# Run the installation script
./install.sh

The script will:

• ✅ Check prerequisites (Python 3.12+, Node.js 18+)
• ✅ Install system dependencies (PostgreSQL, Redis) on Ubuntu/Debian
• ✅ Set up PostgreSQL databases (main + auth)
• ✅ Create Python virtual environment
• ✅ Install all Python dependencies
• ✅ Install all frontend dependencies
• ✅ Create .env configuration files
• ✅ Create necessary data directories

Note: On macOS or other systems, the script will skip system dependency installation. You'll need to install PostgreSQL and Redis manually.

Option 2: Manual Installation

# Clone the repository
git clone <repository-url>
cd punimtag

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Install frontend dependencies
cd frontend
npm install
cd ..

Database Setup

PostgreSQL (Default - Network Database): The application is configured to use PostgreSQL by default. The application requires two separate databases:

1. Main database (punimtag) - Stores photos, faces, people, tags, and backend user accounts
2. Auth database (punimtag_auth) - Stores frontend website user accounts and moderation data

Both database connections are configured via the .env file.

Install PostgreSQL (if not installed):

# On Ubuntu/Debian:
sudo apt update && sudo apt install -y postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Or use the automated setup script:
./scripts/setup_postgresql.sh

Create Main Database and User:

sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';"
sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;"

Create Auth Database (for frontend website user accounts):

sudo -u postgres psql -c "CREATE DATABASE punimtag_auth OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag;"

Note: The auth database (punimtag_auth) stores user accounts for the frontend website, separate from the main application database. Both databases are required for full functionality.

Grant DELETE Permissions on Auth Database Tables: If you encounter permission errors when trying to delete records from the auth database (e.g., when using "Clear database" in the admin panel), grant DELETE permissions:

# Grant DELETE permission on all auth database tables
sudo -u postgres psql -d punimtag_auth << 'EOF'
GRANT DELETE ON TABLE pending_photos TO punimtag;
GRANT DELETE ON TABLE users TO punimtag;
GRANT DELETE ON TABLE pending_identifications TO punimtag;
GRANT DELETE ON TABLE inappropriate_photo_reports TO punimtag;
EOF

# Or grant on a single table:
sudo -u postgres psql -d punimtag_auth -c "GRANT DELETE ON TABLE pending_photos TO punimtag;"

Alternatively, use the automated script (requires sudo password):

./scripts/grant_auth_db_delete_permission.sh

Configuration: The .env file contains both database connection strings:

# Main application database
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth database (for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

Automatic Initialization: The database and all tables are automatically created on first startup. No manual migration is needed!

The web application will:

• Connect to PostgreSQL using the .env configuration
• Create all required tables with the correct schema on startup
• Match the desktop version schema exactly for compatibility

Manual Setup (Optional): If you need to reset the database or create it manually:

source venv/bin/activate
export PYTHONPATH=/home/ladmin/Code/punimtag
# Recreate all tables from models
python scripts/recreate_tables_web.py

SQLite (Alternative - Local Database): To use SQLite instead of PostgreSQL, comment out or remove the DATABASE_URL line in .env, or set it to:

DATABASE_URL=sqlite:///data/punimtag.db

Database Schema: The web version uses the exact same schema as the desktop version for full compatibility:

• photos - Photo metadata (path, filename, date_taken, processed, media_type)
• people - Person records (first_name, last_name, middle_name, maiden_name, date_of_birth)
• faces - Face detections (encoding, location, quality_score, face_confidence, exif_orientation, excluded)
• person_encodings - Person face encodings for matching
• tags - Tag definitions
• phototaglinkage - Photo-tag relationships (with linkage_type)
• users - Backend user accounts (with password hashing, roles, permissions)
• photo_person_linkage - Direct photo-person associations (for videos)
• role_permissions - Role-based permission matrix

Auth Database Schema: The separate auth database (punimtag_auth) stores frontend website user accounts:

• users - Frontend website user accounts (email, password_hash, is_active)
• pending_photos - Photos pending moderation
• pending_identifications - Face identifications pending approval
• inappropriate_photo_reports - Reported photos for review

Running the Application

Prerequisites:

• PostgreSQL must be installed and running (see Database Setup section above)

• Redis must be installed and running (for background jobs)

  Install Redis (if not installed):

  # On Ubuntu/Debian:
  sudo apt update && sudo apt install -y redis-server
  sudo systemctl start redis-server
  sudo systemctl enable redis-server  # Auto-start on boot
  
  # On macOS with Homebrew:
  brew install redis
  brew services start redis
  
  # Verify Redis is running:
  redis-cli ping  # Should respond with "PONG"
  

  Start Redis (if installed but not running):

  # On Linux:
  sudo systemctl start redis-server
  
  # Or run directly:
  redis-server
  

Option 1: Manual Start (Recommended for Development)

Terminal 1 - Backend API:

cd /home/ladmin/Code/punimtag
source venv/bin/activate
export PYTHONPATH=/home/ladmin/Code/punimtag
uvicorn src.web.app:app --host 127.0.0.1 --port 8000

You should see:

✅ Database already initialized (7 tables exist)
✅ RQ worker started in background subprocess (PID: ...)
INFO:     Started server process
INFO:     Uvicorn running on http://127.0.0.1:8000

Terminal 2 - Frontend:

cd /home/ladmin/Code/punimtag/frontend
npm run dev

You should see:

VITE v5.4.21  ready in 811 ms
➜  Local:   http://localhost:3000/

Option 2: Using Helper Script (Backend + Worker)

Terminal 1 - Backend API + Worker:

cd /home/ladmin/Code/punimtag
./run_api_with_worker.sh

This script will:

• Check if Redis is running (start it if needed)
• Start the RQ worker in the background
• Start the FastAPI server
• Handle cleanup on Ctrl+C

Terminal 2 - Frontend:

cd /home/ladmin/Code/punimtag/frontend
npm run dev

Access the Application

1. Open your browser to http://localhost:3000
2. Login with default credentials:
   • Username: admin
   • Password: admin
3. API documentation available at http://127.0.0.1:8000/docs

Troubleshooting

Port 8000 already in use:

# Find and kill the process using port 8000
lsof -i :8000
kill <PID>

# Or use pkill
pkill -f "uvicorn.*app"

Port 3000 already in use:

# Find and kill the process using port 3000
lsof -i :3000
kill <PID>

# Or change the port in frontend/vite.config.ts

Redis not running:

# Start Redis
sudo systemctl start redis-server
# Or
redis-server

Database issues:

# Recreate all tables (WARNING: This will delete all data!)
cd /home/ladmin/Code/punimtag
source venv/bin/activate
export PYTHONPATH=/home/ladmin/Code/punimtag
python scripts/recreate_tables_web.py

Important Notes

• The database and tables are automatically created on first startup - no manual setup needed!
• The RQ worker starts automatically in a background subprocess when the API server starts
• Make sure Redis is running first, or the worker won't start
• Worker names are unique to avoid conflicts when restarting
• Photo uploads are stored in data/uploads (configurable via PHOTO_STORAGE_DIR env var)
• DeepFace models download automatically on first use (can take 5-10 minutes, ~100MB)
• First run is slower due to model downloads (subsequent runs are faster)

---

📖 Documentation

• Architecture: System design and technical details

🏗️ Project Structure

punimtag/
├── src/                    # Source code
│   ├── web/               # Web backend
│   │   ├── api/           # API routers
│   │   ├── db/            # Database models and session
│   │   ├── schemas/       # Pydantic models
│   │   └── services/      # Business logic services
│   └── core/              # Legacy desktop business logic
├── frontend/               # React frontend
│   ├── src/
│   │   ├── api/           # API client
│   │   ├── components/    # React components
│   │   ├── context/       # React contexts (Auth)
│   │   ├── hooks/         # Custom hooks
│   │   └── pages/         # Page components
│   └── package.json
├── tests/                  # Test suite
├── docs/                   # Documentation
├── data/                   # Application data (database, images)
├── alembic/                # Database migrations
└── deploy/                 # Docker deployment configs

---

📊 Current Status

Foundations

Backend:

• ✅ FastAPI application with CORS middleware
• ✅ Health, version, and metrics endpoints
• ✅ JWT authentication (login, refresh, user info)
• ✅ Job management endpoints (RQ/Redis integration)
• ✅ SQLAlchemy models for all entities
• ✅ Alembic migrations configured and applied
• ✅ Database initialized (SQLite default, PostgreSQL supported)
• ✅ RQ worker auto-start (starts automatically with API server)
• ✅ Pending linkage moderation API for user tag suggestions

Frontend:

• ✅ React + Vite + TypeScript setup
• ✅ Tailwind CSS configured
• ✅ Authentication flow with login page
• ✅ Protected routes with auth context
• ✅ Navigation layout (left sidebar + top bar)
• ✅ All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings)
• ✅ User Tagged Photos moderation tab for approving/denying pending tag linkages

Database:

• ✅ All tables created automatically on startup: photos, faces, people, person_encodings, tags, phototaglinkage
• ✅ Schema matches desktop version exactly for full compatibility
• ✅ Indices configured for performance
• ✅ SQLite database at data/punimtag.db (auto-created if missing)

Image Ingestion & Processing

Backend:

• ✅ Photo import service with checksum computation
• ✅ EXIF date extraction and image metadata
• ✅ Folder scanning with recursive option
• ✅ File upload support
• ✅ Background job processing with RQ
• ✅ Real-time job progress via SSE (Server-Sent Events)
• ✅ Duplicate detection (by path and checksum)
• ✅ Photo storage configuration (PHOTO_STORAGE_DIR)
• ✅ DeepFace pipeline integration
• ✅ Face detection (RetinaFace, MTCNN, OpenCV, SSD)
• ✅ Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)
• ✅ Face processing service with configurable detectors/models
• ✅ EXIF orientation handling
• ✅ Face quality scoring and validation
• ✅ Batch processing with progress tracking
• ✅ Job cancellation support

Frontend:

• ✅ Scan tab UI with folder selection
• ✅ Drag-and-drop file upload
• ✅ Recursive scan toggle
• ✅ Real-time job progress with progress bar
• ✅ Job status monitoring (SSE integration)
• ✅ Results display (added/existing counts)
• ✅ Error handling and user feedback
• ✅ Process tab UI with configuration controls
• ✅ Detector/model selection dropdowns
• ✅ Batch size configuration
• ✅ Start/Stop processing controls
• ✅ Processing progress display with photo count
• ✅ Results summary (faces detected, faces stored)
• ✅ Job cancellation support

Worker:

• ✅ RQ worker auto-starts with API server
• ✅ Unique worker names to avoid conflicts
• ✅ Graceful shutdown handling
• ✅ String-based function paths for reliable serialization

Identify Workflow & Auto-Match

Backend:

• ✅ Identify face endpoints with person creation
• ✅ Auto-match engine with similarity thresholds
• ✅ Unidentified faces management and filtering
• ✅ Person creation and linking
• ✅ Batch identification support
• ✅ Similar faces search with cosine similarity
• ✅ Confidence calibration system (empirical-based)
• ✅ Face unmatch/removal functionality
• ✅ Batch similarity calculations

Frontend:

• ✅ Identify page UI with face navigation
• ✅ Person creation and editing
• ✅ Similar faces panel with confidence display
• ✅ Auto-Match page with person-centric view
• ✅ Checkbox selection for batch identification
• ✅ Confidence percentages with color coding
• ✅ Unique faces filter (hide duplicates)
• ✅ Date filtering for faces
• ✅ Real-time face matching and display

PSearch & Tags

Backend:

• ✅ Search endpoints with filters (people, dates, tags, folders)
• ✅ Tag management endpoints (create, update, delete)
• ✅ Photo-tag linkage system
• ✅ Advanced filtering and querying
• ✅ Photo grid endpoints with pagination

Frontend:

• ✅ Search page with advanced filters
• ✅ Tag management UI
• ✅ Photo grid with virtualized rendering
• ✅ Filter by people, dates, tags, and folders
• ✅ Search results display

---

🔧 Configuration

Database

PostgreSQL (Default - Network Database): The application uses PostgreSQL by default, configured via the .env file:

# Main application database
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth database (for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

SQLite (Alternative - Local Database): To use SQLite instead, comment out or remove the DATABASE_URL line in .env, or set:

DATABASE_URL=sqlite:///data/punimtag.db

Note: When using SQLite, the auth database (DATABASE_URL_AUTH) should still be configured as PostgreSQL if you need frontend website user authentication features. The auth database is optional but required for full multi-user functionality.

Environment Variables

Configuration is managed via the .env file in the project root. A .env.example template is provided.

Required Configuration:

# Database (PostgreSQL by default)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth Database (for frontend website user accounts - separate from main database)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

# JWT Secrets (change in production!)
SECRET_KEY=dev-secret-key-change-in-production

# Single-user credentials (change in production!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin

# Photo storage directory (default: data/uploads)
PHOTO_STORAGE_DIR=data/uploads

Frontend Configuration: Create a .env file in the frontend/ directory:

# Backend API URL (must be accessible from browsers)
VITE_API_URL=http://127.0.0.1:8000

Note: The .env file is automatically loaded by the application using python-dotenv. Environment variables can also be set directly in your shell if preferred.

---

---

🔄 Phase 5: Polish & Release (In Progress)

• Performance optimization
• Accessibility improvements
• Production deployment
• Documentation updates

---

🏗️ Architecture

Backend:

• Framework: FastAPI (Python 3.12+)
• Database: PostgreSQL (default, network), SQLite (optional, local)
• ORM: SQLAlchemy 2.0
• Configuration: Environment variables via .env file (python-dotenv)
• Jobs: Redis + RQ
• Auth: JWT (python-jose)

Frontend:

• Framework: React 18 + TypeScript
• Build Tool: Vite
• Styling: Tailwind CSS
• State: React Query + Context API
• Routing: React Router

Deployment:

• Docker Compose for local development
• Containerized services for production

---

📦 Dependencies

Backend:

• fastapi==0.115.0
• uvicorn[standard]==0.30.6
• pydantic==2.9.1
• SQLAlchemy==2.0.36
• alembic==1.13.2
• python-jose[cryptography]==3.3.0
• redis==5.0.8
• rq==1.16.2
• psycopg2-binary==2.9.9 (PostgreSQL driver)
• python-multipart==0.0.9 (file uploads)
• python-dotenv==1.0.0 (environment variables)
• bcrypt==4.1.2 (password hashing)
• deepface>=0.0.79
• tensorflow>=2.13.0
• opencv-python>=4.8.0
• retina-face>=0.0.13
• numpy>=1.21.0
• pillow>=8.0.0

Frontend:

• react==18.2.0
• react-router-dom==6.20.0
• @tanstack/react-query==5.8.4
• axios==1.6.2
• tailwindcss==3.3.5

---

🔒 Security

• JWT-based authentication with refresh tokens
• Password hashing with bcrypt
• CORS configured for development (restrict in production)
• SQL injection prevention via SQLAlchemy ORM
• Input validation via Pydantic schemas
• Separate auth database for frontend website user accounts

⚠️ Note: Default credentials (admin/admin) are for development only. Change in production!

---

🐛 Known Limitations

• Multi-user support with role-based permissions (single-user mode deprecated)
• SQLite for development (PostgreSQL recommended for production)
• GPU acceleration not yet implemented (CPU-only for now)
• Large databases (>50K photos) may require optimization
• DeepFace model downloads on first use (can take 5-10 minutes, ~100MB)
• Face processing is CPU-intensive (~2-3x slower than face_recognition, but more accurate)
• First run is slower due to model downloads (subsequent runs are faster)

---

📝 License

[Add your license here]

---

👥 Authors

PunimTag Development Team

---

🙏 Acknowledgments

• DeepFace library by Sefik Ilkin Serengil - Modern face recognition framework
• ArcFace - Additive Angular Margin Loss for Deep Face Recognition
• RetinaFace - State-of-the-art face detection
• TensorFlow, React, FastAPI, and all open-source contributors

---

📧 Support

For questions or issues:

1. Check documentation in docs/

---

Made with ❤️ for photo enthusiasts