punimtag/README.md

# PunimTag Web

**Modern Photo Management and Facial Recognition System**

A fast, simple, and modern web application for organizing and tagging photos using state-of-the-art DeepFace AI with ArcFace recognition model.

**Monorepo Structure:** This project contains both the admin interface (React) and viewer interface (Next.js) in a unified repository for easier maintenance and setup.

---

## 🎯 Features

- **🌐 Web-Based**: Modern React frontend with FastAPI backend
- **🔥 DeepFace AI**: State-of-the-art face detection with RetinaFace and ArcFace models
- **🎯 Superior Accuracy**: 512-dimensional embeddings (4x more detailed than face_recognition)
- **⚙️ Multiple Detectors**: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors
- **🎨 Flexible Models**: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models
- **👤 Person Identification**: Identify and tag people across your photo collection
- **🤖 Smart Auto-Matching**: Intelligent face matching with quality scoring and cosine similarity
- **📊 Confidence Calibration**: Empirical-based confidence scores for realistic match probabilities
- **🔍 Advanced Search**: Search by people, dates, tags, and folders
- **🏷️ Tag Management**: Organize photos with hierarchical tags
- **⚡ Batch Processing**: Process thousands of photos efficiently
- **🎯 Unique Faces Filter**: Hide duplicate faces to focus on unique individuals
- **🔄 Real-time Updates**: Live progress tracking and job status updates
- **🔒 Privacy-First**: All data stored locally, no cloud dependencies

---

## 🚀 Quick Start

### Prerequisites

- **Python 3.12 or higher** (with pip)
- **Node.js 18+ and npm**
- **PostgreSQL** (required for both development and production)
- **Redis** (for background job processing)

**Note:** The automated installation script (`./install.sh`) will install PostgreSQL and Redis automatically on Ubuntu/Debian systems.

### Installation

#### Option 1: Automated Installation (Recommended for Linux/Ubuntu/Debian)

The automated installation script will install all system dependencies, Python packages, frontend dependencies, and set up databases:

```bash
# Clone the repository
git clone <repository-url>
cd punimtag

# Run the installation script
./install.sh
```

The script will:
- ✅ Check prerequisites (Python 3.12+, Node.js 18+)
- ✅ Install system dependencies (PostgreSQL, Redis) on Ubuntu/Debian
- ✅ Set up PostgreSQL databases (main + auth)
- ✅ Create Python virtual environment
- ✅ Install all Python dependencies
- ✅ Install all frontend dependencies (admin-frontend and viewer-frontend)
- ✅ Create `.env` configuration files
- ✅ Create necessary data directories

**Note:** After installation, you'll need to generate Prisma clients for the viewer-frontend:
```bash
cd viewer-frontend
npx prisma generate
```

**Note:** On macOS or other systems, the script will skip system dependency installation. You'll need to install PostgreSQL and Redis manually.

#### Option 2: Manual Installation

```bash
# Clone the repository
git clone <repository-url>
cd punimtag

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Install frontend dependencies
cd admin-frontend
npm install
cd ../viewer-frontend
npm install
# Generate Prisma clients for viewer-frontend (after setting up .env)
npx prisma generate
cd ..
```

### Database Setup

**Database Configuration:**
The application uses **two separate PostgreSQL databases**:
1. **Main database** (`punimtag`) - Stores photos, faces, people, tags, and backend user accounts
   - **Required: PostgreSQL**
2. **Auth database** (`punimtag_auth`) - Stores frontend website user accounts and moderation data
   - **Required: PostgreSQL**

Both database connections are configured via the `.env` file.

**Install PostgreSQL (if not installed):**
```bash
# On Ubuntu/Debian:
sudo apt update && sudo apt install -y postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql

# Or use the automated setup script:
./scripts/setup_postgresql.sh
```

**Create Main Database and User:**
```bash
sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';"
sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;"
```

**Create Auth Database (for frontend website user accounts):**
```bash
sudo -u postgres psql -c "CREATE DATABASE punimtag_auth OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag;"
```

**Note:** The auth database (`punimtag_auth`) stores user accounts for the frontend website, separate from the main application database. Both databases are required for full functionality.

**Grant DELETE Permissions on Auth Database Tables:**
If you encounter permission errors when trying to delete records from the auth database (e.g., when using "Clear database" in the admin panel), grant DELETE permissions:

```bash
# Grant DELETE permission on all auth database tables
sudo -u postgres psql -d punimtag_auth << 'EOF'
GRANT DELETE ON TABLE pending_photos TO punimtag;
GRANT DELETE ON TABLE users TO punimtag;
GRANT DELETE ON TABLE pending_identifications TO punimtag;
GRANT DELETE ON TABLE inappropriate_photo_reports TO punimtag;
EOF

# Or grant on a single table:
sudo -u postgres psql -d punimtag_auth -c "GRANT DELETE ON TABLE pending_photos TO punimtag;"
```

Alternatively, use the automated script (requires sudo password):
```bash
./scripts/grant_auth_db_delete_permission.sh
```

**Configuration:**
The `.env` file in the project root contains database connection strings:
```bash
# Main application database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth database (PostgreSQL - required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth
```

**Automatic Initialization:**
The database and all tables are automatically created on first startup. No manual migration is needed!

The web application will:
- Connect to the database using the `.env` configuration
- Create all required tables with the correct schema on startup
- Match the desktop version schema exactly for compatibility

**Database Schema:**
The web version uses the **exact same schema** as the desktop version for full compatibility:
- `photos` - Photo metadata (path, filename, date_taken, processed, media_type)
- `people` - Person records (first_name, last_name, middle_name, maiden_name, date_of_birth)
- `faces` - Face detections (encoding, location, quality_score, face_confidence, exif_orientation, excluded)
- `person_encodings` - Person face encodings for matching
- `tags` - Tag definitions
- `phototaglinkage` - Photo-tag relationships (with linkage_type)
- `users` - Backend user accounts (with password hashing, roles, permissions)
- `photo_person_linkage` - Direct photo-person associations (for videos)
- `role_permissions` - Role-based permission matrix

**Auth Database Schema:**
The separate auth database (`punimtag_auth`) stores frontend website user accounts:
- `users` - Frontend website user accounts (email, password_hash, is_active)
- `pending_photos` - Photos pending moderation
- `pending_identifications` - Face identifications pending approval
- `inappropriate_photo_reports` - Reported photos for review

### Running the Application

**Prerequisites:**
- **PostgreSQL** must be installed and running (see Database Setup section above)
- **Redis** must be installed and running (for background jobs)
  
  **Install Redis (if not installed):**
  ```bash
  # On Ubuntu/Debian:
  sudo apt update && sudo apt install -y redis-server
  sudo systemctl start redis-server
  sudo systemctl enable redis-server  # Auto-start on boot
  
  # On macOS with Homebrew:
  brew install redis
  brew services start redis
  
  # Verify Redis is running:
  redis-cli ping  # Should respond with "PONG"
  ```
  
  **Start Redis (if installed but not running):**
  ```bash
  # On Linux:
  sudo systemctl start redis-server
  
  # Or run directly:
  redis-server
  ```

#### Option 1: Using Helper Scripts (Recommended)

**Terminal 1 - Backend API + Worker:**
```bash
cd punimtag
./run_api_with_worker.sh
```

This script will:
- Check if Redis is running (start it if needed)
- Ensure database schema is up to date
- Start the RQ worker in the background
- Start the FastAPI server
- Handle cleanup on Ctrl+C

You should see:
```
✅ Database schema ready
🚀 Starting RQ worker...
🚀 Starting FastAPI server...
✅ Server running on http://127.0.0.1:8000
✅ Worker running (PID: ...)
✅ API running (PID: ...)
```

**Alternative: Start backend only (without worker):**
```bash
cd punimtag
./start_backend.sh
```

**Stop the backend:**
```bash
cd punimtag
./stop_backend.sh
```

**Terminal 2 - Admin Frontend:**
```bash
cd punimtag/admin-frontend
npm run dev
```

You should see:
```
VITE v5.4.21  ready in 811 ms
➜  Local:   http://localhost:3000/
```

**Terminal 3 - Viewer Frontend (Optional):**
```bash
cd punimtag/viewer-frontend
# Generate Prisma clients (only needed once or after schema changes)
npx prisma generate
npm run dev
```

You should see:
```
▲ Next.js 16.1.1 (Turbopack)
- Local:        http://localhost:3001/
```

#### Option 2: Manual Start

**Terminal 1 - Backend API:**
```bash
cd punimtag
source venv/bin/activate
export PYTHONPATH="$(pwd)"
python3 -m uvicorn backend.app:app --host 127.0.0.1 --port 8000 --reload
```

**Note:** If you encounter warnings about "Electron/Chromium" when running `uvicorn`, use `python3 -m uvicorn` instead, or use the helper scripts above.

**Terminal 2 - Admin Frontend:**
```bash
cd punimtag/admin-frontend
npm run dev
```

**Terminal 3 - Viewer Frontend (Optional):**
```bash
cd punimtag/viewer-frontend
npx prisma generate  # Only needed once or after schema changes
npm run dev
```

#### Access the Applications

1. **Admin Interface**: Open your browser to **http://localhost:3000**
   - Login with default credentials:
     - Username: `admin`
     - Password: `admin`
2. **Viewer Interface** (Optional): Open your browser to **http://localhost:3001**
   - Public photo viewing interface
   - Separate authentication system
3. **API Documentation**: Available at **http://127.0.0.1:8000/docs**

#### Troubleshooting

**Port 8000 already in use:**
```bash
# Use the stop script
cd punimtag
./stop_backend.sh

# Or manually find and kill the process
lsof -i :8000
kill <PID>

# Or use pkill
pkill -f "uvicorn.*backend.app"
```

**Port 3000 already in use:**
```bash
# Find and kill the process using port 3000
lsof -i :3000
kill <PID>

# Or change the port in admin-frontend/vite.config.ts
```

**Redis not running:**
```bash
# Start Redis
sudo systemctl start redis-server
# Or
redis-server

# Verify Redis is running
redis-cli ping  # Should respond with "PONG"
```

**Worker module not found error:**
If you see `ModuleNotFoundError: No module named 'backend'`:
- Make sure you're using the helper scripts (`./run_api_with_worker.sh` or `./start_backend.sh`)
- These scripts set PYTHONPATH correctly
- If running manually, ensure `export PYTHONPATH="$(pwd)"` is set

**Python/Cursor interception warnings:**
If you see warnings about "Electron/Chromium" when running `uvicorn`:
- Use `python3 -m uvicorn` instead of just `uvicorn`
- Or use the helper scripts which handle this automatically

**Database issues:**
```bash
# The database is automatically created on first startup
# If you need to reset it, delete the database file:
rm data/punimtag.db

# The schema will be recreated on next startup
```

**Viewer frontend shows 0 photos:**
- Make sure the database has photos (import them via admin frontend)
- Verify `DATABASE_URL` in `viewer-frontend/.env` points to the correct database
- Ensure Prisma client is generated: `cd viewer-frontend && npx prisma generate`
- Check that photos are marked as `processed: true` in the database

#### Important Notes

- The database and tables are **automatically created on first startup** - no manual setup needed!
- The RQ worker starts automatically in a background subprocess when the API server starts
- Make sure Redis is running first, or the worker won't start
- Worker names are unique to avoid conflicts when restarting
- Photo uploads are stored in `data/uploads` (configurable via `PHOTO_STORAGE_DIR` env var)
- **DeepFace models download automatically on first use** (can take 5-10 minutes, ~100MB)
- First run is slower due to model downloads (subsequent runs are faster)

---

## 📖 Documentation

- **[Architecture](docs/ARCHITECTURE.md)**: System design and technical details


*

## 🏗️ Project Structure

```
punimtag/
├── backend/                # FastAPI backend
│   ├── api/               # API routers
│   ├── db/                # Database models and session
│   ├── schemas/           # Pydantic models
│   ├── services/          # Business logic services
│   ├── constants/         # Constants and configuration
│   ├── utils/             # Utility functions
│   ├── app.py             # FastAPI application
│   └── worker.py          # RQ worker for background jobs
├── admin-frontend/         # React admin interface
│   ├── src/
│   │   ├── api/           # API client
│   │   ├── components/    # React components
│   │   ├── context/       # React contexts (Auth)
│   │   ├── hooks/         # Custom hooks
│   │   └── pages/         # Page components
│   └── package.json
├── viewer-frontend/        # Next.js viewer interface
│   ├── app/               # Next.js app router
│   ├── components/         # React components
│   ├── lib/               # Utilities and database
│   ├── prisma/            # Prisma schemas
│   └── package.json
├── src/                    # Legacy desktop code
│   └── core/              # Legacy desktop business logic
├── tests/                  # Test suite
├── docs/                   # Documentation
├── data/                   # Application data (database, images)
├── scripts/                # Utility scripts
├── deploy/                 # Docker deployment configs
└── package.json            # Root package.json for monorepo
```

---

## 📊 Current Status

###  Foundations 

**Backend:**
- ✅ FastAPI application with CORS middleware
- ✅ Health, version, and metrics endpoints
- ✅ JWT authentication (login, refresh, user info)
- ✅ Job management endpoints (RQ/Redis integration)
- ✅ SQLAlchemy models for all entities
- ✅ Alembic migrations configured and applied
- ✅ Database initialized (PostgreSQL required)
- ✅ RQ worker auto-start (starts automatically with API server)
- ✅ Pending linkage moderation API for user tag suggestions

**Frontend:**
- ✅ React + Vite + TypeScript setup
- ✅ Tailwind CSS configured
- ✅ Authentication flow with login page
- ✅ Protected routes with auth context
- ✅ Navigation layout (left sidebar + top bar)
- ✅ All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings)
- ✅ User Tagged Photos moderation tab for approving/denying pending tag linkages

**Database:**
- ✅ All tables created automatically on startup: `photos`, `faces`, `people`, `person_encodings`, `tags`, `phototaglinkage`
- ✅ Schema matches desktop version exactly for full compatibility
- ✅ Indices configured for performance
- ✅ PostgreSQL database (required for both development and production)
- ✅ Separate auth database (PostgreSQL) for frontend user accounts

###  Image Ingestion & Processing 

**Backend:**
- ✅ Photo import service with checksum computation
- ✅ EXIF date extraction and image metadata
- ✅ Folder scanning with recursive option
- ✅ File upload support
- ✅ Background job processing with RQ
- ✅ Real-time job progress via SSE (Server-Sent Events)
- ✅ Duplicate detection (by path and checksum)
- ✅ Photo storage configuration (`PHOTO_STORAGE_DIR`)
- ✅ **DeepFace pipeline integration**
- ✅ **Face detection (RetinaFace, MTCNN, OpenCV, SSD)**
- ✅ **Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)**
- ✅ **Face processing service with configurable detectors/models**
- ✅ **EXIF orientation handling**
- ✅ **Face quality scoring and validation**
- ✅ **Batch processing with progress tracking**
- ✅ **Job cancellation support**

**Frontend:**
- ✅ Scan tab UI with folder selection
- ✅ Drag-and-drop file upload
- ✅ Recursive scan toggle
- ✅ Real-time job progress with progress bar
- ✅ Job status monitoring (SSE integration)
- ✅ Results display (added/existing counts)
- ✅ Error handling and user feedback
- ✅ **Process tab UI with configuration controls**
- ✅ **Detector/model selection dropdowns**
- ✅ **Batch size configuration**
- ✅ **Start/Stop processing controls**
- ✅ **Processing progress display with photo count**
- ✅ **Results summary (faces detected, faces stored)**
- ✅ **Job cancellation support**

**Worker:**
- ✅ RQ worker auto-starts with API server
- ✅ Unique worker names to avoid conflicts
- ✅ Graceful shutdown handling
- ✅ **String-based function paths for reliable serialization**

### Identify Workflow & Auto-Match 

**Backend:**
- ✅ Identify face endpoints with person creation
- ✅ Auto-match engine with similarity thresholds
- ✅ Unidentified faces management and filtering
- ✅ Person creation and linking
- ✅ Batch identification support
- ✅ Similar faces search with cosine similarity
- ✅ Confidence calibration system (empirical-based)
- ✅ Face unmatch/removal functionality
- ✅ Batch similarity calculations

**Frontend:**
- ✅ Identify page UI with face navigation
- ✅ Person creation and editing
- ✅ Similar faces panel with confidence display
- ✅ Auto-Match page with person-centric view
- ✅ Checkbox selection for batch identification
- ✅ Confidence percentages with color coding
- ✅ Unique faces filter (hide duplicates)
- ✅ Date filtering for faces
- ✅ Real-time face matching and display

### PSearch & Tags 

**Backend:**
- ✅ Search endpoints with filters (people, dates, tags, folders)
- ✅ Tag management endpoints (create, update, delete)
- ✅ Photo-tag linkage system
- ✅ Advanced filtering and querying
- ✅ Photo grid endpoints with pagination

**Frontend:**
- ✅ Search page with advanced filters
- ✅ Tag management UI
- ✅ Photo grid with virtualized rendering
- ✅ Filter by people, dates, tags, and folders
- ✅ Search results display

---

## 🔧 Configuration

### Database

**PostgreSQL (Required):**
Both databases use PostgreSQL. Configure via the `.env` file:
```bash
# Main application database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth database (PostgreSQL - required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth
```

### Environment Variables

Configuration is managed via the `.env` file in the project root. A `.env.example` template is provided.

**Required Configuration:**
```bash
# Main Database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag

# Auth Database (PostgreSQL - required for frontend website user accounts)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth

# JWT Secrets (change in production!)
SECRET_KEY=dev-secret-key-change-in-production

# Single-user credentials (change in production!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin

# Photo storage directory (default: data/uploads)
PHOTO_STORAGE_DIR=data/uploads
```

**Admin Frontend Configuration:**
Create a `.env` file in the `admin-frontend/` directory:
```bash
# Backend API URL (must be accessible from browsers)
VITE_API_URL=http://127.0.0.1:8000
```

**Viewer Frontend Configuration:**
Create a `.env` file in the `viewer-frontend/` directory:
```bash
# Main database connection (PostgreSQL - required)
DATABASE_URL=postgresql://punimtag:punimtag_password@localhost:5432/punimtag

# Auth database connection (PostgreSQL - required)
DATABASE_URL_AUTH=postgresql://punimtag:punimtag_password@localhost:5432/punimtag_auth

# Write-capable database connection (optional, falls back to DATABASE_URL if not set)
DATABASE_URL_WRITE=postgresql://punimtag:punimtag_password@localhost:5432/punimtag

# NextAuth configuration
NEXTAUTH_URL=http://localhost:3001
NEXTAUTH_SECRET=dev-secret-key-change-in-production
```

**Generate Prisma Clients:**
After setting up the `.env` file, generate the Prisma clients:
```bash
cd viewer-frontend
npx prisma generate
```

**Important:** The viewer frontend uses **PostgreSQL** for the main database (matching the backend). The Prisma schema is configured for PostgreSQL.

**Note:** The viewer frontend uses the same database as the backend by default. For production deployments, you may want to create separate read-only and write users for better security.

**Note:** The `.env` file is automatically loaded by the application using `python-dotenv`. Environment variables can also be set directly in your shell if preferred.

---


---

### 🔄 Phase 5: Polish & Release (In Progress)
- Performance optimization
- Accessibility improvements
- Production deployment
- Documentation updates

---

## 🏗️ Architecture

**Backend:**
- **Framework**: FastAPI (Python 3.12+)
- **Database**: PostgreSQL (required)
- **ORM**: SQLAlchemy 2.0
- **Configuration**: Environment variables via `.env` file (python-dotenv)
- **Jobs**: Redis + RQ
- **Auth**: JWT (python-jose)

**Frontend:**
- **Framework**: React 18 + TypeScript
- **Build Tool**: Vite
- **Styling**: Tailwind CSS
- **State**: React Query + Context API
- **Routing**: React Router

**Deployment:**
- Docker Compose for local development
- Containerized services for production

---

## 📦 Dependencies

**Backend:**
- `fastapi==0.115.0`
- `uvicorn[standard]==0.30.6`
- `pydantic==2.9.1`
- `SQLAlchemy==2.0.36`
- `alembic==1.13.2`
- `python-jose[cryptography]==3.3.0`
- `redis==5.0.8`
- `rq==1.16.2`
- `psycopg2-binary==2.9.9` (PostgreSQL driver)
- `python-multipart==0.0.9` (file uploads)
- `python-dotenv==1.0.0` (environment variables)
- `bcrypt==4.1.2` (password hashing)
- `deepface>=0.0.79`
- `tensorflow>=2.13.0`
- `opencv-python>=4.8.0`
- `retina-face>=0.0.13`
- `numpy>=1.21.0`
- `pillow>=8.0.0`

**Frontend:**
- `react==18.2.0`
- `react-router-dom==6.20.0`
- `@tanstack/react-query==5.8.4`
- `axios==1.6.2`
- `tailwindcss==3.3.5`

---

## 🔒 Security

- JWT-based authentication with refresh tokens
- Password hashing with bcrypt
- CORS configured for development (restrict in production)
- SQL injection prevention via SQLAlchemy ORM
- Input validation via Pydantic schemas
- Separate auth database for frontend website user accounts

**⚠️ Note**: Default credentials (`admin`/`admin`) are for development only. Change in production!

---

## 🐛 Known Limitations

- Multi-user support with role-based permissions (single-user mode deprecated)
- PostgreSQL for both development and production
- GPU acceleration not yet implemented (CPU-only for now)
- Large databases (>50K photos) may require optimization
- DeepFace model downloads on first use (can take 5-10 minutes, ~100MB)
- Face processing is CPU-intensive (~2-3x slower than face_recognition, but more accurate)
- First run is slower due to model downloads (subsequent runs are faster)

---

## 📝 License

[Add your license here]

---

## 👥 Authors

PunimTag Development Team

---



## 🙏 Acknowledgments

- **DeepFace** library by Sefik Ilkin Serengil - Modern face recognition framework
- **ArcFace** - Additive Angular Margin Loss for Deep Face Recognition
- **RetinaFace** - State-of-the-art face detection
- TensorFlow, React, FastAPI, and all open-source contributors

---

## 📧 Support

For questions or issues:
1. Check documentation in `docs/`


---

**Made with ❤️ for photo enthusiasts**