PunimTag Web

Modern Photo Management and Facial Recognition System

A fast, simple, and modern web application for organizing and tagging photos using state-of-the-art DeepFace AI with ArcFace recognition model.

Monorepo Structure: This project contains both the admin interface (React) and viewer interface (Next.js) in a unified repository for easier maintenance and setup.

---

🎯 Features

• 🌐 Web-Based: Modern React frontend with FastAPI backend
• 🔥 DeepFace AI: State-of-the-art face detection with RetinaFace and ArcFace models
• 🎯 Superior Accuracy: 512-dimensional embeddings (4x more detailed than face_recognition)
• ⚙️ Multiple Detectors: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors
• 🎨 Flexible Models: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models
• 👤 Person Identification: Identify and tag people across your photo collection
• 🤖 Smart Auto-Matching: Intelligent face matching with quality scoring and cosine similarity
• 📊 Confidence Calibration: Empirical-based confidence scores for realistic match probabilities
• 🔍 Advanced Search: Search by people, dates, tags, and folders
• 🏷️ Tag Management: Organize photos with hierarchical tags
• ⚡ Batch Processing: Process thousands of photos efficiently
• 🎯 Unique Faces Filter: Hide duplicate faces to focus on unique individuals
• 🔄 Real-time Updates: Live progress tracking and job status updates
• 🔒 Privacy-First: All data stored locally, no cloud dependencies

---

🚀 Quick Start

Prerequisites

• Python 3.12 or higher (with pip)
• Node.js 18+ and npm
• PostgreSQL (for production, optional for development with SQLite)
• Redis (for background job processing)

Note: The automated installation script (./install.sh) will install PostgreSQL and Redis automatically on Ubuntu/Debian systems.

Installation

Option 1: Automated Installation (Recommended for Linux/Ubuntu/Debian)

The automated installation script will install all system dependencies, Python packages, frontend dependencies, and set up databases:

# Clone the repository
git clone <repository-url>
cd punimtag

# Run the installation script
./install.sh

The script will:

• ✅ Check prerequisites (Python 3.12+, Node.js 18+)
• ✅ Install system dependencies (PostgreSQL, Redis) on Ubuntu/Debian
• ✅ Set up PostgreSQL databases (main + auth)
• ✅ Create Python virtual environment
• ✅ Install all Python dependencies
• ✅ Install all frontend dependencies (admin-frontend and viewer-frontend)
• ✅ Create .env configuration files
• ✅ Create necessary data directories

Note: After installation, you'll need to generate Prisma clients for the viewer-frontend:

cd viewer-frontend
npx prisma generate

Note: On macOS or other systems, the script will skip system dependency installation. You'll need to install PostgreSQL and Redis manually.

Option 2: Manual Installation

# Clone the repository
git clone <repository-url>
cd punimtag

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Install frontend dependencies
cd admin-frontend
npm install
cd ../viewer-frontend
npm install
# Generate Prisma clients for viewer-frontend (after setting up .env)
npx prisma generate
cd ..

Database Setup

Database Configuration: The application uses two separate databases:

1. Main database (punimtag) - Stores photos, faces, people, tags, and backend user accounts
   • Default: SQLite at data/punimtag.db (for development)
   • Optional: PostgreSQL (for production)
2. Auth database (punimtag_auth) - Stores frontend website user accounts and moderation data
   • Required: PostgreSQL (always uses PostgreSQL)

Both database connections are configured via the .env file.

Install PostgreSQL (if not installed):

# On Ubuntu/Debian:
sudo apt update && sudo apt install -y postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Or use the automated setup script:
./scripts/setup_postgresql.sh

Create Main Database and User:

sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';"
sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;"

Create Auth Database (for frontend website user accounts):

sudo -u postgres psql -c "CREATE DATABASE punimtag_auth OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag;"

Note: The auth database (punimtag_auth) stores user accounts for the frontend website, separate from the main application database. Both databases are required for full functionality.

Grant DELETE Permissions on Auth Database Tables: If you encounter permission errors when trying to delete records from the auth database (e.g., when using "Clear database" in the admin panel), grant DELETE permissions:

# Grant DELETE permission on all auth database tables
sudo -u postgres psql -d punimtag_auth << 'EOF'
GRANT DELETE ON TABLE pending_photos TO punimtag;
GRANT DELETE ON TABLE users TO punimtag;
GRANT DELETE ON TABLE pending_identifications TO punimtag;
GRANT DELETE ON TABLE inappropriate_photo_reports TO punimtag;
EOF

# Or grant on a single table:
sudo -u postgres psql -d punimtag_auth -c "GRANT DELETE ON TABLE pending_photos TO punimtag;"

Alternatively, use the automated script (requires sudo password):

./scripts/grant_auth_db_delete_permission.sh

Configuration: The .env file in the project root contains database connection strings:

# Main application database (SQLite - default for development)
DATABASE_URL=sqlite:///data/punimtag.db

# Auth database (PostgreSQL - always required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

Automatic Initialization: The database and all tables are automatically created on first startup. No manual migration is needed!

The web application will:

• Connect to the database using the .env configuration
• Create all required tables with the correct schema on startup
• Match the desktop version schema exactly for compatibility

Note: The main database uses SQLite by default for easier development. For production, you can switch to PostgreSQL by updating DATABASE_URL in .env.

SQLite (Default - Local Database): The main database uses SQLite by default for development. The .env file should contain:

# Main database (SQLite - default for development)
DATABASE_URL=sqlite:///data/punimtag.db

# Or use absolute path:
# DATABASE_URL=file:/home/ladmin/code/punimtag/data/punimtag.db

PostgreSQL (Optional - for Production): To use PostgreSQL for the main database instead, set:

DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

Database Schema: The web version uses the exact same schema as the desktop version for full compatibility:

• photos - Photo metadata (path, filename, date_taken, processed, media_type)
• people - Person records (first_name, last_name, middle_name, maiden_name, date_of_birth)
• faces - Face detections (encoding, location, quality_score, face_confidence, exif_orientation, excluded)
• person_encodings - Person face encodings for matching
• tags - Tag definitions
• phototaglinkage - Photo-tag relationships (with linkage_type)
• users - Backend user accounts (with password hashing, roles, permissions)
• photo_person_linkage - Direct photo-person associations (for videos)
• role_permissions - Role-based permission matrix

Auth Database Schema: The separate auth database (punimtag_auth) stores frontend website user accounts:

• users - Frontend website user accounts (email, password_hash, is_active)
• pending_photos - Photos pending moderation
• pending_identifications - Face identifications pending approval
• inappropriate_photo_reports - Reported photos for review

Running the Application

Prerequisites:

• PostgreSQL must be installed and running (see Database Setup section above)

• Redis must be installed and running (for background jobs)

  Install Redis (if not installed):

  # On Ubuntu/Debian:
  sudo apt update && sudo apt install -y redis-server
  sudo systemctl start redis-server
  sudo systemctl enable redis-server  # Auto-start on boot
  
  # On macOS with Homebrew:
  brew install redis
  brew services start redis
  
  # Verify Redis is running:
  redis-cli ping  # Should respond with "PONG"
  

  Start Redis (if installed but not running):

  # On Linux:
  sudo systemctl start redis-server
  
  # Or run directly:
  redis-server
  

Option 1: Using Helper Scripts (Recommended)

Terminal 1 - Backend API + Worker:

cd punimtag
./run_api_with_worker.sh

This script will:

• Check if Redis is running (start it if needed)
• Ensure database schema is up to date
• Start the RQ worker in the background
• Start the FastAPI server
• Handle cleanup on Ctrl+C

You should see:

✅ Database schema ready
🚀 Starting RQ worker...
🚀 Starting FastAPI server...
✅ Server running on http://127.0.0.1:8000
✅ Worker running (PID: ...)
✅ API running (PID: ...)

Alternative: Start backend only (without worker):

cd punimtag
./start_backend.sh

Stop the backend:

cd punimtag
./stop_backend.sh

Terminal 2 - Admin Frontend:

cd punimtag/admin-frontend
npm run dev

You should see:

VITE v5.4.21  ready in 811 ms
➜  Local:   http://localhost:3000/

Terminal 3 - Viewer Frontend (Optional):

cd punimtag/viewer-frontend
# Generate Prisma clients (only needed once or after schema changes)
npx prisma generate
npm run dev

You should see:

▲ Next.js 16.1.1 (Turbopack)
- Local:        http://localhost:3001/

Option 2: Manual Start

Terminal 1 - Backend API:

cd punimtag
source venv/bin/activate
export PYTHONPATH="$(pwd)"
python3 -m uvicorn backend.app:app --host 127.0.0.1 --port 8000 --reload

Note: If you encounter warnings about "Electron/Chromium" when running uvicorn, use python3 -m uvicorn instead, or use the helper scripts above.

Terminal 2 - Admin Frontend:

cd punimtag/admin-frontend
npm run dev

Terminal 3 - Viewer Frontend (Optional):

cd punimtag/viewer-frontend
npx prisma generate  # Only needed once or after schema changes
npm run dev

Access the Applications

1. Admin Interface: Open your browser to http://localhost:3000
   • Login with default credentials:
     • Username: admin
     • Password: admin
2. Viewer Interface (Optional): Open your browser to http://localhost:3001
   • Public photo viewing interface
   • Separate authentication system
3. API Documentation: Available at http://127.0.0.1:8000/docs

Troubleshooting

Port 8000 already in use:

# Use the stop script
cd punimtag
./stop_backend.sh

# Or manually find and kill the process
lsof -i :8000
kill <PID>

# Or use pkill
pkill -f "uvicorn.*backend.app"

Port 3000 already in use:

# Find and kill the process using port 3000
lsof -i :3000
kill <PID>

# Or change the port in admin-frontend/vite.config.ts

Redis not running:

# Start Redis
sudo systemctl start redis-server
# Or
redis-server

# Verify Redis is running
redis-cli ping  # Should respond with "PONG"

Worker module not found error: If you see ModuleNotFoundError: No module named 'backend':

• Make sure you're using the helper scripts (./run_api_with_worker.sh or ./start_backend.sh)
• These scripts set PYTHONPATH correctly
• If running manually, ensure export PYTHONPATH="$(pwd)" is set

Python/Cursor interception warnings: If you see warnings about "Electron/Chromium" when running uvicorn:

• Use python3 -m uvicorn instead of just uvicorn
• Or use the helper scripts which handle this automatically

Database issues:

# The database is automatically created on first startup
# If you need to reset it, delete the database file:
rm data/punimtag.db

# The schema will be recreated on next startup

Viewer frontend shows 0 photos:

• Make sure the database has photos (import them via admin frontend)
• Verify DATABASE_URL in viewer-frontend/.env points to the correct database
• Ensure Prisma client is generated: cd viewer-frontend && npx prisma generate
• Check that photos are marked as processed: true in the database

Important Notes

• The database and tables are automatically created on first startup - no manual setup needed!
• The RQ worker starts automatically in a background subprocess when the API server starts
• Make sure Redis is running first, or the worker won't start
• Worker names are unique to avoid conflicts when restarting
• Photo uploads are stored in data/uploads (configurable via PHOTO_STORAGE_DIR env var)
• DeepFace models download automatically on first use (can take 5-10 minutes, ~100MB)
• First run is slower due to model downloads (subsequent runs are faster)

---

📖 Documentation

• Architecture: System design and technical details

🏗️ Project Structure

punimtag/
├── backend/                # FastAPI backend
│   ├── api/               # API routers
│   ├── db/                # Database models and session
│   ├── schemas/           # Pydantic models
│   ├── services/          # Business logic services
│   ├── constants/         # Constants and configuration
│   ├── utils/             # Utility functions
│   ├── app.py             # FastAPI application
│   └── worker.py          # RQ worker for background jobs
├── admin-frontend/         # React admin interface
│   ├── src/
│   │   ├── api/           # API client
│   │   ├── components/    # React components
│   │   ├── context/       # React contexts (Auth)
│   │   ├── hooks/         # Custom hooks
│   │   └── pages/         # Page components
│   └── package.json
├── viewer-frontend/        # Next.js viewer interface
│   ├── app/               # Next.js app router
│   ├── components/         # React components
│   ├── lib/               # Utilities and database
│   ├── prisma/            # Prisma schemas
│   └── package.json
├── src/                    # Legacy desktop code
│   └── core/              # Legacy desktop business logic
├── tests/                  # Test suite
├── docs/                   # Documentation
├── data/                   # Application data (database, images)
├── scripts/                # Utility scripts
├── deploy/                 # Docker deployment configs
└── package.json            # Root package.json for monorepo

---

📊 Current Status

Foundations

Backend:

• ✅ FastAPI application with CORS middleware
• ✅ Health, version, and metrics endpoints
• ✅ JWT authentication (login, refresh, user info)
• ✅ Job management endpoints (RQ/Redis integration)
• ✅ SQLAlchemy models for all entities
• ✅ Alembic migrations configured and applied
• ✅ Database initialized (SQLite default, PostgreSQL supported)
• ✅ RQ worker auto-start (starts automatically with API server)
• ✅ Pending linkage moderation API for user tag suggestions

Frontend:

• ✅ React + Vite + TypeScript setup
• ✅ Tailwind CSS configured
• ✅ Authentication flow with login page
• ✅ Protected routes with auth context
• ✅ Navigation layout (left sidebar + top bar)
• ✅ All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings)
• ✅ User Tagged Photos moderation tab for approving/denying pending tag linkages

Database:

• ✅ All tables created automatically on startup: photos, faces, people, person_encodings, tags, phototaglinkage
• ✅ Schema matches desktop version exactly for full compatibility
• ✅ Indices configured for performance
• ✅ SQLite database at data/punimtag.db (auto-created if missing, default for development)
• ✅ PostgreSQL support for production deployments
• ✅ Separate auth database (PostgreSQL) for frontend user accounts

Image Ingestion & Processing

Backend:

• ✅ Photo import service with checksum computation
• ✅ EXIF date extraction and image metadata
• ✅ Folder scanning with recursive option
• ✅ File upload support
• ✅ Background job processing with RQ
• ✅ Real-time job progress via SSE (Server-Sent Events)
• ✅ Duplicate detection (by path and checksum)
• ✅ Photo storage configuration (PHOTO_STORAGE_DIR)
• ✅ DeepFace pipeline integration
• ✅ Face detection (RetinaFace, MTCNN, OpenCV, SSD)
• ✅ Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)
• ✅ Face processing service with configurable detectors/models
• ✅ EXIF orientation handling
• ✅ Face quality scoring and validation
• ✅ Batch processing with progress tracking
• ✅ Job cancellation support

Frontend:

• ✅ Scan tab UI with folder selection
• ✅ Drag-and-drop file upload
• ✅ Recursive scan toggle
• ✅ Real-time job progress with progress bar
• ✅ Job status monitoring (SSE integration)
• ✅ Results display (added/existing counts)
• ✅ Error handling and user feedback
• ✅ Process tab UI with configuration controls
• ✅ Detector/model selection dropdowns
• ✅ Batch size configuration
• ✅ Start/Stop processing controls
• ✅ Processing progress display with photo count
• ✅ Results summary (faces detected, faces stored)
• ✅ Job cancellation support

Worker:

• ✅ RQ worker auto-starts with API server
• ✅ Unique worker names to avoid conflicts
• ✅ Graceful shutdown handling
• ✅ String-based function paths for reliable serialization

Identify Workflow & Auto-Match

Backend:

• ✅ Identify face endpoints with person creation
• ✅ Auto-match engine with similarity thresholds
• ✅ Unidentified faces management and filtering
• ✅ Person creation and linking
• ✅ Batch identification support
• ✅ Similar faces search with cosine similarity
• ✅ Confidence calibration system (empirical-based)
• ✅ Face unmatch/removal functionality
• ✅ Batch similarity calculations

Frontend:

• ✅ Identify page UI with face navigation
• ✅ Person creation and editing
• ✅ Similar faces panel with confidence display
• ✅ Auto-Match page with person-centric view
• ✅ Checkbox selection for batch identification
• ✅ Confidence percentages with color coding
• ✅ Unique faces filter (hide duplicates)
• ✅ Date filtering for faces
• ✅ Real-time face matching and display

PSearch & Tags

Backend:

• ✅ Search endpoints with filters (people, dates, tags, folders)
• ✅ Tag management endpoints (create, update, delete)
• ✅ Photo-tag linkage system
• ✅ Advanced filtering and querying
• ✅ Photo grid endpoints with pagination

Frontend:

• ✅ Search page with advanced filters
• ✅ Tag management UI
• ✅ Photo grid with virtualized rendering
• ✅ Filter by people, dates, tags, and folders
• ✅ Search results display

---

🔧 Configuration

Database

SQLite (Default - Local Database): The main database uses SQLite by default for development, configured via the .env file:

# Main application database (SQLite - default)
DATABASE_URL=sqlite:///data/punimtag.db

# Auth database (PostgreSQL - always required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

PostgreSQL (Optional - for Production): To use PostgreSQL for the main database instead:

DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

Note: The auth database (DATABASE_URL_AUTH) always uses PostgreSQL and is required for frontend website user authentication features.

Environment Variables

Configuration is managed via the .env file in the project root. A .env.example template is provided.

Required Configuration:

# Database (SQLite by default for development)
DATABASE_URL=sqlite:///data/punimtag.db

# Auth Database (PostgreSQL - always required for frontend website user accounts)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

# JWT Secrets (change in production!)
SECRET_KEY=dev-secret-key-change-in-production

# Single-user credentials (change in production!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin

# Photo storage directory (default: data/uploads)
PHOTO_STORAGE_DIR=data/uploads

Admin Frontend Configuration: Create a .env file in the admin-frontend/ directory:

# Backend API URL (must be accessible from browsers)
VITE_API_URL=http://127.0.0.1:8000

Viewer Frontend Configuration: Create a .env file in the viewer-frontend/ directory:

# Main database connection (SQLite - matches backend default)
# Use absolute path for SQLite
DATABASE_URL=file:/home/ladmin/code/punimtag/data/punimtag.db

# Auth database connection (PostgreSQL - always required)
DATABASE_URL_AUTH=postgresql://punimtag:punimtag_password@localhost:5432/punimtag_auth

# Write-capable database connection (optional, falls back to DATABASE_URL if not set)
DATABASE_URL_WRITE=file:/home/ladmin/code/punimtag/data/punimtag.db

# NextAuth configuration
NEXTAUTH_URL=http://localhost:3001
NEXTAUTH_SECRET=dev-secret-key-change-in-production

Generate Prisma Clients: After setting up the .env file, generate the Prisma clients:

cd viewer-frontend
npx prisma generate

Important: The viewer frontend uses SQLite for the main database (matching the backend default). The Prisma schema is configured for SQLite. If you change the backend to PostgreSQL, you'll need to:

1. Update viewer-frontend/prisma/schema.prisma to use provider = "postgresql"
2. Update DATABASE_URL in viewer-frontend/.env to the PostgreSQL connection string
3. Run npx prisma generate again

Note: The viewer frontend uses the same database as the backend by default. For production deployments, you may want to create separate read-only and write users for better security.

Note: The .env file is automatically loaded by the application using python-dotenv. Environment variables can also be set directly in your shell if preferred.

---

---

🔄 Phase 5: Polish & Release (In Progress)

• Performance optimization
• Accessibility improvements
• Production deployment
• Documentation updates

---

🏗️ Architecture

Backend:

• Framework: FastAPI (Python 3.12+)
• Database: PostgreSQL (default, network), SQLite (optional, local)
• ORM: SQLAlchemy 2.0
• Configuration: Environment variables via .env file (python-dotenv)
• Jobs: Redis + RQ
• Auth: JWT (python-jose)

Frontend:

• Framework: React 18 + TypeScript
• Build Tool: Vite
• Styling: Tailwind CSS
• State: React Query + Context API
• Routing: React Router

Deployment:

• Docker Compose for local development
• Containerized services for production

---

📦 Dependencies

Backend:

• fastapi==0.115.0
• uvicorn[standard]==0.30.6
• pydantic==2.9.1
• SQLAlchemy==2.0.36
• alembic==1.13.2
• python-jose[cryptography]==3.3.0
• redis==5.0.8
• rq==1.16.2
• psycopg2-binary==2.9.9 (PostgreSQL driver)
• python-multipart==0.0.9 (file uploads)
• python-dotenv==1.0.0 (environment variables)
• bcrypt==4.1.2 (password hashing)
• deepface>=0.0.79
• tensorflow>=2.13.0
• opencv-python>=4.8.0
• retina-face>=0.0.13
• numpy>=1.21.0
• pillow>=8.0.0

Frontend:

• react==18.2.0
• react-router-dom==6.20.0
• @tanstack/react-query==5.8.4
• axios==1.6.2
• tailwindcss==3.3.5

---

🔒 Security

• JWT-based authentication with refresh tokens
• Password hashing with bcrypt
• CORS configured for development (restrict in production)
• SQL injection prevention via SQLAlchemy ORM
• Input validation via Pydantic schemas
• Separate auth database for frontend website user accounts

⚠️ Note: Default credentials (admin/admin) are for development only. Change in production!

---

🐛 Known Limitations

• Multi-user support with role-based permissions (single-user mode deprecated)
• SQLite for development (PostgreSQL recommended for production)
• GPU acceleration not yet implemented (CPU-only for now)
• Large databases (>50K photos) may require optimization
• DeepFace model downloads on first use (can take 5-10 minutes, ~100MB)
• Face processing is CPU-intensive (~2-3x slower than face_recognition, but more accurate)
• First run is slower due to model downloads (subsequent runs are faster)

---

📝 License

[Add your license here]

---

👥 Authors

PunimTag Development Team

---

🙏 Acknowledgments

• DeepFace library by Sefik Ilkin Serengil - Modern face recognition framework
• ArcFace - Additive Angular Margin Loss for Deep Face Recognition
• RetinaFace - State-of-the-art face detection
• TensorFlow, React, FastAPI, and all open-source contributors

---

📧 Support

For questions or issues:

1. Check documentation in docs/

---

Made with ❤️ for photo enthusiasts