# PunimTag Web **Modern Photo Management and Facial Recognition System** A fast, simple, and modern web application for organizing and tagging photos using state-of-the-art DeepFace AI with ArcFace recognition model. **Monorepo Structure:** This project contains both the admin interface (React) and viewer interface (Next.js) in a unified repository for easier maintenance and setup. --- ## 🎯 Features - **🌐 Web-Based**: Modern React frontend with FastAPI backend - **🔥 DeepFace AI**: State-of-the-art face detection with RetinaFace and ArcFace models - **🎯 Superior Accuracy**: 512-dimensional embeddings (4x more detailed than face_recognition) - **⚙️ Multiple Detectors**: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors - **🎨 Flexible Models**: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models - **👤 Person Identification**: Identify and tag people across your photo collection - **🤖 Smart Auto-Matching**: Intelligent face matching with quality scoring and cosine similarity - **📊 Confidence Calibration**: Empirical-based confidence scores for realistic match probabilities - **🔍 Advanced Search**: Search by people, dates, tags, and folders - **🏷️ Tag Management**: Organize photos with hierarchical tags - **⚡ Batch Processing**: Process thousands of photos efficiently - **🎯 Unique Faces Filter**: Hide duplicate faces to focus on unique individuals - **🔄 Real-time Updates**: Live progress tracking and job status updates - **🌐 Network Path Support**: Browse and scan folders on network shares (UNC paths on Windows, mounted shares on Linux) - **📁 Native Folder Picker**: Browse button uses native OS folder picker with full absolute path support - **🔒 Privacy-First**: All data stored locally, no cloud dependencies --- ## 🚀 Quick Start ### Prerequisites - **Python 3.12 or higher** (with pip) - **Node.js 18+ and npm** - **PostgreSQL** (required for both development and production) - **Redis** (for background job processing) - **Python tkinter** (for native folder picker in Scan tab) **Note:** The automated installation script (`./install.sh`) will install PostgreSQL, Redis, and Python tkinter automatically on Ubuntu/Debian systems. ### Installation #### Option 1: Automated Installation (Recommended for Linux/Ubuntu/Debian) The automated installation script will install all system dependencies, Python packages, frontend dependencies, and set up databases: ```bash # Clone the repository git clone cd punimtag # Run the installation script ./install.sh ``` The script will: - ✅ Check prerequisites (Python 3.12+, Node.js 18+) - ✅ Install system dependencies (PostgreSQL, Redis, Python tkinter) on Ubuntu/Debian - ✅ Set up PostgreSQL databases (main + auth) - ✅ Create Python virtual environment - ✅ Install all Python dependencies - ✅ Install all frontend dependencies (admin-frontend and viewer-frontend) - ✅ Create `.env` configuration files - ✅ Create necessary data directories **Note:** After installation, you'll need to generate Prisma clients for the viewer-frontend: ```bash cd viewer-frontend npx prisma generate ``` **Note:** On macOS or other systems, the script will skip system dependency installation. You'll need to install PostgreSQL, Redis, and Python tkinter manually. **Installing tkinter manually:** - **Ubuntu/Debian:** `sudo apt install python3-tk` - **RHEL/CentOS:** `sudo yum install python3-tkinter` - **macOS:** Usually included with Python, but if missing: `brew install python-tk` (if using Homebrew Python) - **Windows:** Usually included with Python installation #### Option 2: Manual Installation ```bash # Clone the repository git clone cd punimtag # Create and activate virtual environment python -m venv venv source venv/bin/activate # On Windows: venv\Scripts\activate # Install Python dependencies pip install -r requirements.txt # Install frontend dependencies cd admin-frontend npm install cd ../viewer-frontend npm install # Generate Prisma clients for viewer-frontend (after setting up .env) npx prisma generate cd .. ``` ### Database Setup **Database Configuration:** The application uses **two separate PostgreSQL databases**: 1. **Main database** (`punimtag`) - Stores photos, faces, people, tags, and backend user accounts - **Required: PostgreSQL** 2. **Auth database** (`punimtag_auth`) - Stores frontend website user accounts and moderation data - **Required: PostgreSQL** Both database connections are configured via the `.env` file. **Install PostgreSQL (if not installed):** ```bash # On Ubuntu/Debian: sudo apt update && sudo apt install -y postgresql postgresql-contrib sudo systemctl start postgresql sudo systemctl enable postgresql # Or use the automated setup script: ./scripts/setup_postgresql.sh ``` **Create Main Database and User:** ```bash sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';" sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;" sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;" ``` **Create Auth Database (for frontend website user accounts):** ```bash sudo -u postgres psql -c "CREATE DATABASE punimtag_auth OWNER punimtag;" sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag;" ``` **Note:** The auth database (`punimtag_auth`) stores user accounts for the frontend website, separate from the main application database. Both databases are required for full functionality. **Grant DELETE Permissions on Auth Database Tables:** If you encounter permission errors when trying to delete records from the auth database (e.g., when using "Clear database" in the admin panel), grant DELETE permissions: ```bash # Grant DELETE permission on all auth database tables sudo -u postgres psql -d punimtag_auth << 'EOF' GRANT DELETE ON TABLE pending_photos TO punimtag; GRANT DELETE ON TABLE users TO punimtag; GRANT DELETE ON TABLE pending_identifications TO punimtag; GRANT DELETE ON TABLE inappropriate_photo_reports TO punimtag; EOF # Or grant on a single table: sudo -u postgres psql -d punimtag_auth -c "GRANT DELETE ON TABLE pending_photos TO punimtag;" ``` Alternatively, use the automated script (requires sudo password): ```bash ./scripts/grant_auth_db_delete_permission.sh ``` **Configuration:** The `.env` file in the project root contains database connection strings: ```bash # Main application database (PostgreSQL - required) DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag # Auth database (PostgreSQL - required for frontend website users) DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth ``` **Automatic Initialization:** The database and all tables are automatically created on first startup. No manual migration is needed! The web application will: - Connect to the database using the `.env` configuration - Create all required tables with the correct schema on startup - Match the desktop version schema exactly for compatibility **Database Schema:** The web version uses the **exact same schema** as the desktop version for full compatibility: - `photos` - Photo metadata (path, filename, date_taken, processed, media_type) - `people` - Person records (first_name, last_name, middle_name, maiden_name, date_of_birth) - `faces` - Face detections (encoding, location, quality_score, face_confidence, exif_orientation, excluded) - `person_encodings` - Person face encodings for matching - `tags` - Tag definitions - `phototaglinkage` - Photo-tag relationships (with linkage_type) - `users` - Backend user accounts (with password hashing, roles, permissions) - `photo_person_linkage` - Direct photo-person associations (for videos) - `role_permissions` - Role-based permission matrix **Auth Database Schema:** The separate auth database (`punimtag_auth`) stores frontend website user accounts: - `users` - Frontend website user accounts (email, password_hash, is_active) - `pending_photos` - Photos pending moderation - `pending_identifications` - Face identifications pending approval - `inappropriate_photo_reports` - Reported photos for review ### Running the Application **Prerequisites:** - **PostgreSQL** must be installed and running (see Database Setup section above) - **Redis** must be installed and running (for background jobs) **Install Redis (if not installed):** ```bash # On Ubuntu/Debian: sudo apt update && sudo apt install -y redis-server sudo systemctl start redis-server sudo systemctl enable redis-server # Auto-start on boot # On macOS with Homebrew: brew install redis brew services start redis # Verify Redis is running: redis-cli ping # Should respond with "PONG" ``` **Start Redis (if installed but not running):** ```bash # On Linux: sudo systemctl start redis-server # Or run directly: redis-server ``` #### Option 1: Using Helper Scripts (Recommended) **Terminal 1 - Backend API + Worker:** ```bash cd punimtag ./run_api_with_worker.sh ``` This script will: - Check if Redis is running (start it if needed) - Ensure database schema is up to date - Start the RQ worker in the background - Start the FastAPI server - Handle cleanup on Ctrl+C You should see: ``` ✅ Database schema ready 🚀 Starting RQ worker... 🚀 Starting FastAPI server... ✅ Server running on http://127.0.0.1:8000 ✅ Worker running (PID: ...) ✅ API running (PID: ...) ``` **Alternative: Start backend only (without worker):** ```bash cd punimtag ./start_backend.sh ``` **Stop the backend:** ```bash cd punimtag ./stop_backend.sh ``` **Terminal 2 - Admin Frontend:** ```bash cd punimtag/admin-frontend npm run dev ``` You should see: ``` VITE v5.4.21 ready in 811 ms ➜ Local: http://localhost:3000/ ``` **Terminal 3 - Viewer Frontend (Optional):** ```bash cd punimtag/viewer-frontend # Generate Prisma clients (only needed once or after schema changes) npx prisma generate npm run dev ``` You should see: ``` ▲ Next.js 16.1.1 (Turbopack) - Local: http://localhost:3001/ ``` #### Option 2: Manual Start **Terminal 1 - Backend API:** ```bash cd punimtag source venv/bin/activate export PYTHONPATH="$(pwd)" python3 -m uvicorn backend.app:app --host 127.0.0.1 --port 8000 --reload ``` **Note:** If you encounter warnings about "Electron/Chromium" when running `uvicorn`, use `python3 -m uvicorn` instead, or use the helper scripts above. **Terminal 2 - Admin Frontend:** ```bash cd punimtag/admin-frontend npm run dev ``` **Terminal 3 - Viewer Frontend (Optional):** ```bash cd punimtag/viewer-frontend npx prisma generate # Only needed once or after schema changes npm run dev ``` #### Access the Applications 1. **Admin Interface**: Open your browser to **http://localhost:3000** - Login with default credentials: - Username: `admin` - Password: `admin` 2. **Viewer Interface** (Optional): Open your browser to **http://localhost:3001** - Public photo viewing interface - Separate authentication system 3. **API Documentation**: Available at **http://127.0.0.1:8000/docs** #### Troubleshooting **Port 8000 already in use:** ```bash # Use the stop script cd punimtag ./stop_backend.sh # Or manually find and kill the process lsof -i :8000 kill # Or use pkill pkill -f "uvicorn.*backend.app" ``` **Port 3000 already in use:** ```bash # Find and kill the process using port 3000 lsof -i :3000 kill # Or change the port in admin-frontend/vite.config.ts ``` **Redis not running:** ```bash # Start Redis sudo systemctl start redis-server # Or redis-server # Verify Redis is running redis-cli ping # Should respond with "PONG" ``` **Worker module not found error:** If you see `ModuleNotFoundError: No module named 'backend'`: - Make sure you're using the helper scripts (`./run_api_with_worker.sh` or `./start_backend.sh`) - These scripts set PYTHONPATH correctly - If running manually, ensure `export PYTHONPATH="$(pwd)"` is set **Python/Cursor interception warnings:** If you see warnings about "Electron/Chromium" when running `uvicorn`: - Use `python3 -m uvicorn` instead of just `uvicorn` - Or use the helper scripts which handle this automatically **Database issues:** ```bash # The database is automatically created on first startup # If you need to reset it, delete the database file: rm data/punimtag.db # The schema will be recreated on next startup ``` **Browse button returns 503 error or doesn't show folder picker:** This indicates that Python tkinter is not available. Install it: ```bash # Ubuntu/Debian: sudo apt install python3-tk # RHEL/CentOS: sudo yum install python3-tkinter # Verify installation: python3 -c "import tkinter; print('tkinter available')" ``` **Note:** If running on a remote server without a display, you may need to set the DISPLAY environment variable or use X11 forwarding: ```bash export DISPLAY=:0 # Or for X11 forwarding: export DISPLAY=localhost:10.0 ``` **Viewer frontend shows 0 photos:** - Make sure the database has photos (import them via admin frontend) - Verify `DATABASE_URL` in `viewer-frontend/.env` points to the correct database - Ensure Prisma client is generated: `cd viewer-frontend && npx prisma generate` - Check that photos are marked as `processed: true` in the database #### Important Notes - The database and tables are **automatically created on first startup** - no manual setup needed! - The RQ worker starts automatically in a background subprocess when the API server starts - Make sure Redis is running first, or the worker won't start - Worker names are unique to avoid conflicts when restarting - Photo uploads are stored in `data/uploads` (configurable via `PHOTO_STORAGE_DIR` env var) - **DeepFace models download automatically on first use** (can take 5-10 minutes, ~100MB) - First run is slower due to model downloads (subsequent runs are faster) --- ## 📖 Documentation - **[Architecture](docs/ARCHITECTURE.md)**: System design and technical details * ## 🏗️ Project Structure ``` punimtag/ ├── backend/ # FastAPI backend │ ├── api/ # API routers │ ├── db/ # Database models and session │ ├── schemas/ # Pydantic models │ ├── services/ # Business logic services │ ├── constants/ # Constants and configuration │ ├── utils/ # Utility functions │ ├── app.py # FastAPI application │ └── worker.py # RQ worker for background jobs ├── admin-frontend/ # React admin interface │ ├── src/ │ │ ├── api/ # API client │ │ ├── components/ # React components │ │ ├── context/ # React contexts (Auth) │ │ ├── hooks/ # Custom hooks │ │ └── pages/ # Page components │ └── package.json ├── viewer-frontend/ # Next.js viewer interface │ ├── app/ # Next.js app router │ ├── components/ # React components │ ├── lib/ # Utilities and database │ ├── prisma/ # Prisma schemas │ └── package.json ├── src/ # Legacy desktop code │ └── core/ # Legacy desktop business logic ├── tests/ # Test suite ├── docs/ # Documentation ├── data/ # Application data (database, images) ├── scripts/ # Utility scripts ├── deploy/ # Docker deployment configs └── package.json # Root package.json for monorepo ``` --- ## 📊 Current Status ### Foundations **Backend:** - ✅ FastAPI application with CORS middleware - ✅ Health, version, and metrics endpoints - ✅ JWT authentication (login, refresh, user info) - ✅ Job management endpoints (RQ/Redis integration) - ✅ SQLAlchemy models for all entities - ✅ Alembic migrations configured and applied - ✅ Database initialized (PostgreSQL required) - ✅ RQ worker auto-start (starts automatically with API server) - ✅ Pending linkage moderation API for user tag suggestions **Frontend:** - ✅ React + Vite + TypeScript setup - ✅ Tailwind CSS configured - ✅ Authentication flow with login page - ✅ Protected routes with auth context - ✅ Navigation layout (left sidebar + top bar) - ✅ All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings) - ✅ User Tagged Photos moderation tab for approving/denying pending tag linkages **Database:** - ✅ All tables created automatically on startup: `photos`, `faces`, `people`, `person_encodings`, `tags`, `phototaglinkage` - ✅ Schema matches desktop version exactly for full compatibility - ✅ Indices configured for performance - ✅ PostgreSQL database (required for both development and production) - ✅ Separate auth database (PostgreSQL) for frontend user accounts ### Image Ingestion & Processing **Backend:** - ✅ Photo import service with checksum computation - ✅ EXIF date extraction and image metadata - ✅ Folder scanning with recursive option - ✅ File upload support - ✅ Background job processing with RQ - ✅ Real-time job progress via SSE (Server-Sent Events) - ✅ Duplicate detection (by path and checksum) - ✅ Photo storage configuration (`PHOTO_STORAGE_DIR`) - ✅ **DeepFace pipeline integration** - ✅ **Face detection (RetinaFace, MTCNN, OpenCV, SSD)** - ✅ **Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)** - ✅ **Face processing service with configurable detectors/models** - ✅ **EXIF orientation handling** - ✅ **Face quality scoring and validation** - ✅ **Batch processing with progress tracking** - ✅ **Job cancellation support** **Frontend:** - ✅ Scan tab UI with folder selection - ✅ **Native folder picker (Browse button)** - Uses tkinter for native OS folder selection - ✅ **Network path support** - Handles UNC paths (Windows: `\\server\share\folder`) and mounted network shares (Linux: `/mnt/nfs-share/photos`) - ✅ **Full absolute path handling** - Automatically normalizes and validates paths - ✅ Drag-and-drop file upload - ✅ Recursive scan toggle - ✅ Real-time job progress with progress bar - ✅ Job status monitoring (SSE integration) - ✅ Results display (added/existing counts) - ✅ Error handling and user feedback - ✅ **Process tab UI with configuration controls** - ✅ **Detector/model selection dropdowns** - ✅ **Batch size configuration** - ✅ **Start/Stop processing controls** - ✅ **Processing progress display with photo count** - ✅ **Results summary (faces detected, faces stored)** - ✅ **Job cancellation support** **Worker:** - ✅ RQ worker auto-starts with API server - ✅ Unique worker names to avoid conflicts - ✅ Graceful shutdown handling - ✅ **String-based function paths for reliable serialization** ### Identify Workflow & Auto-Match **Backend:** - ✅ Identify face endpoints with person creation - ✅ Auto-match engine with similarity thresholds - ✅ Unidentified faces management and filtering - ✅ Person creation and linking - ✅ Batch identification support - ✅ Similar faces search with cosine similarity - ✅ Confidence calibration system (empirical-based) - ✅ Face unmatch/removal functionality - ✅ Batch similarity calculations **Frontend:** - ✅ Identify page UI with face navigation - ✅ Person creation and editing - ✅ Similar faces panel with confidence display - ✅ Auto-Match page with person-centric view - ✅ Checkbox selection for batch identification - ✅ Confidence percentages with color coding - ✅ Unique faces filter (hide duplicates) - ✅ Date filtering for faces - ✅ Real-time face matching and display ### PSearch & Tags **Backend:** - ✅ Search endpoints with filters (people, dates, tags, folders) - ✅ Tag management endpoints (create, update, delete) - ✅ Photo-tag linkage system - ✅ Advanced filtering and querying - ✅ Photo grid endpoints with pagination **Frontend:** - ✅ Search page with advanced filters - ✅ Tag management UI - ✅ Photo grid with virtualized rendering - ✅ Filter by people, dates, tags, and folders - ✅ Search results display --- ## 🔧 Configuration ### Database **PostgreSQL (Required):** Both databases use PostgreSQL. Configure via the `.env` file: ```bash # Main application database (PostgreSQL - required) DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag # Auth database (PostgreSQL - required for frontend website users) DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth ``` ### Environment Variables Configuration is managed via the `.env` file in the project root. A `.env.example` template is provided. **Required Configuration:** ```bash # Main Database (PostgreSQL - required) DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag # Auth Database (PostgreSQL - required for frontend website user accounts) DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth # JWT Secrets (change in production!) SECRET_KEY=dev-secret-key-change-in-production # Single-user credentials (change in production!) ADMIN_USERNAME=admin ADMIN_PASSWORD=admin # Photo storage directory (default: data/uploads) PHOTO_STORAGE_DIR=data/uploads ``` **Admin Frontend Configuration:** Create a `.env` file in the `admin-frontend/` directory: ```bash # Backend API URL (must be accessible from browsers) VITE_API_URL=http://127.0.0.1:8000 ``` **Viewer Frontend Configuration:** Create a `.env` file in the `viewer-frontend/` directory: ```bash # Main database connection (PostgreSQL - required) DATABASE_URL=postgresql://punimtag:punimtag_password@localhost:5432/punimtag # Auth database connection (PostgreSQL - required) DATABASE_URL_AUTH=postgresql://punimtag:punimtag_password@localhost:5432/punimtag_auth # Write-capable database connection (optional, falls back to DATABASE_URL if not set) DATABASE_URL_WRITE=postgresql://punimtag:punimtag_password@localhost:5432/punimtag # NextAuth configuration NEXTAUTH_URL=http://localhost:3001 NEXTAUTH_SECRET=dev-secret-key-change-in-production ``` **Generate Prisma Clients:** After setting up the `.env` file, generate the Prisma clients: ```bash cd viewer-frontend npx prisma generate ``` **Important:** The viewer frontend uses **PostgreSQL** for the main database (matching the backend). The Prisma schema is configured for PostgreSQL. **Note:** The viewer frontend uses the same database as the backend by default. For production deployments, you may want to create separate read-only and write users for better security. **Note:** The `.env` file is automatically loaded by the application using `python-dotenv`. Environment variables can also be set directly in your shell if preferred. --- --- ### 🔄 Phase 5: Polish & Release (In Progress) - Performance optimization - Accessibility improvements - Production deployment - Documentation updates --- ## 🏗️ Architecture **Backend:** - **Framework**: FastAPI (Python 3.12+) - **Database**: PostgreSQL (required) - **ORM**: SQLAlchemy 2.0 - **Configuration**: Environment variables via `.env` file (python-dotenv) - **Jobs**: Redis + RQ - **Auth**: JWT (python-jose) **Frontend:** - **Framework**: React 18 + TypeScript - **Build Tool**: Vite - **Styling**: Tailwind CSS - **State**: React Query + Context API - **Routing**: React Router **Deployment:** - Docker Compose for local development - Containerized services for production --- ## 📦 Dependencies **Backend:** - `fastapi==0.115.0` - `uvicorn[standard]==0.30.6` - `pydantic==2.9.1` - `SQLAlchemy==2.0.36` - `alembic==1.13.2` - `python-jose[cryptography]==3.3.0` - `redis==5.0.8` - `rq==1.16.2` - `psycopg2-binary==2.9.9` (PostgreSQL driver) - `python-multipart==0.0.9` (file uploads) - `python-dotenv==1.0.0` (environment variables) - `bcrypt==4.1.2` (password hashing) - `deepface>=0.0.79` - `tensorflow>=2.13.0` - `opencv-python>=4.8.0` - `retina-face>=0.0.13` - `numpy>=1.21.0` - `pillow>=8.0.0` **Frontend:** - `react==18.2.0` - `react-router-dom==6.20.0` - `@tanstack/react-query==5.8.4` - `axios==1.6.2` - `tailwindcss==3.3.5` --- ## 🔒 Security - JWT-based authentication with refresh tokens - Password hashing with bcrypt - CORS configured for development (restrict in production) - SQL injection prevention via SQLAlchemy ORM - Input validation via Pydantic schemas - Separate auth database for frontend website user accounts **⚠️ Note**: Default credentials (`admin`/`admin`) are for development only. Change in production! --- ## 🐛 Known Limitations - Multi-user support with role-based permissions (single-user mode deprecated) - PostgreSQL for both development and production - GPU acceleration not yet implemented (CPU-only for now) - Large databases (>50K photos) may require optimization - DeepFace model downloads on first use (can take 5-10 minutes, ~100MB) - Face processing is CPU-intensive (~2-3x slower than face_recognition, but more accurate) - First run is slower due to model downloads (subsequent runs are faster) --- ## 📝 License [Add your license here] --- ## 👥 Authors PunimTag Development Team --- ## 🙏 Acknowledgments - **DeepFace** library by Sefik Ilkin Serengil - Modern face recognition framework - **ArcFace** - Additive Angular Margin Loss for Deep Face Recognition - **RetinaFace** - State-of-the-art face detection - TensorFlow, React, FastAPI, and all open-source contributors --- ## 📧 Support For questions or issues: 1. Check documentation in `docs/` --- **Made with ❤️ for photo enthusiasts**