# PunimTag Web Rebuild Plan (Greenfield)

Version: 2.0  
Last Updated: October 31, 2025

---

## 1) Direction and Principles

- **Greenfield rewrite**: Build PunimTag as a web application from scratch.
- **No backward compatibility**: The existing desktop code and database will not be reused.
- **Fresh database**: The new system initializes a new database; prior data will be erased.
- **Simple**: Minimal clicks, clean UI, frictionless first-run onboarding.
- **Fast**: Snappy interactions (<100ms perceived), heavy tasks offloaded to jobs.
- **Modern**: Type-safe APIs, real-time progress, responsive design, dark/light modes.

---

## 2) Target Architecture (Web-Native)

### Backend
- **Framework**: FastAPI (Python 3.12+)
- **Database**: PostgreSQL (primary) using async driver (asyncpg) and SQLAlchemy 2.0 ORM
- **Jobs**: Redis + RQ (or Celery) for background tasks (scan/process/auto-match/thumbnailing)
- **Storage**: Local disk for images (phase 1); S3-compatible storage option (phase 2)
- **Auth**: JWT (access/refresh) with optional single-user mode
- **Schemas**: Pydantic for request/response models; OpenAPI-first
- **Images**: Streaming endpoints, range support, signed URLs for originals

### Frontend
- **Framework**: React + Vite + TypeScript
- **Design**: Tailwind CSS + Headless UI/Radix; dark/light theme toggle
- **State**: React Query for server state; Zustand for local UI state
- **Routing**: React Router with code-splitting; keyboard-first UX
- **Performance**: Virtualized grids, skeletons, optimistic updates

### DevOps / Deployment
- **Local**: Docker Compose (web, worker, db, redis, frontend)
- **Prod**: Containers behind Traefik/Nginx, HTTPS, zero-downtime deploys
- **Observability**: Structured logs, request/job IDs, metrics endpoint, health checks

---

## 3) Domain Model (Matches Desktop Schema)

The web version uses the **exact same schema** as the desktop version for full compatibility:

- `photos` (id, path, filename, date_added, date_taken DATE, processed)
- `people` (id, first_name, last_name, middle_name, maiden_name, date_of_birth, created_date)
- `faces` (id, photo_id, person_id, encoding BLOB, location TEXT, confidence REAL, quality_score REAL, is_primary_encoding, detector_backend, model_name, face_confidence REAL, exif_orientation)
- `person_encodings` (id, person_id, face_id, encoding BLOB, quality_score REAL, detector_backend, model_name, created_date)
- `tags` (id, tag_name, created_date)
- `phototaglinkage` (linkage_id, photo_id, tag_id, linkage_type, created_date)
- Indices for common queries (date_taken, quality_score, person, photo, tag)

**Note:** 
- Encodings stored as binary (BLOB) - float32 arrays from DeepFace ArcFace (512 dimensions)
- Location stored as JSON text string matching desktop format: `"{'x': x, 'y': y, 'w': w, 'h': h}"`
- Quality scores stored as REAL (0.0-1.0 range)
- Schema matches desktop version exactly for data portability

---

## 4) Core Features (Web MVP)

- Import photos (upload UI and/or server-side folder ingest)
- EXIF parsing, metadata extraction
- Face detection and embeddings (DeepFace ArcFace + RetinaFace)
- **Identify workflow** (manual identification - one face at a time, assign to existing/new person)
- **Auto-match workflow** (automated bulk matching - matches unidentified faces to identified people with tolerance thresholds)
- **Modify Identified workflow** (edit person information, unmatch faces from people, undo/save changes)
- Tags: CRUD and bulk tagging
- Search: by people, tags, date range, folder; infinite scroll grid
- Thumbnail generation and caching (100×100, 256×256)
- Real-time job progress (SSE/WebSocket)

**Note:** Identify, Auto-Match, and Modify Identified are **separate features** with distinct UIs and workflows:
- **Identify**: Manual process, one unidentified face at a time, user assigns to person
- **Auto-Match**: Automated process, shows identified person on left, matched unidentified faces on right, bulk accept/reject
- **Modify Identified**: Edit identified people, view faces per person, unmatch faces, edit person information (names, date of birth)

---

## 5) API Design (v1)

Base URL: `/api/v1`

- Auth: POST `/auth/login`, POST `/auth/refresh`, GET `/auth/me`
- Photos: GET `/photos`, POST `/photos/import` (upload or folder), GET `/photos/{id}`, GET `/photos/{id}/image`
- Faces: POST `/faces/process`, GET `/faces/unidentified`, POST `/faces/{id}/identify`, POST `/faces/auto-match`, POST `/faces/{id}/unmatch`
- People: GET `/people`, POST `/people`, GET `/people/{id}`, PUT `/people/{id}`, GET `/people/{id}/faces`
- Tags: GET `/tags`, POST `/tags`, POST `/photos/{id}/tags`
- Jobs: GET `/jobs/{id}`, GET `/jobs/stream` (SSE)

All responses are typed; errors use structured codes. Pagination with `page`/`page_size` (default 50, max 200).

---

## 6) UX and Navigation

- Shell layout: left nav + top bar; routes: Dashboard, Scan, Process, Search, Identify, Auto-Match, Modify, Tags, Settings
- **Identify flow**: Manual identification - one face at a time, keyboard (J/K), Enter to accept, quick-create person, similar faces for comparison
- **Auto-Match flow**: Automated bulk matching - identified person on left, matched faces on right, bulk accept/reject, tolerance threshold configuration
- **Modify Identified flow**: People list with search, face grid per person, edit person info, unmatch faces, undo/save changes
- Search: virtualized grid, filters drawer, saved filters
- Tags: inline edit, bulk apply from grid selection
- Settings: detector/model selection, thresholds, storage paths

Visual design: neutral palette, clear hierarchy, low-chrome UI, subtle motion (<150ms).

---

## 7) Performance Strategy

- Background jobs for heavy ops, job progress surfaced in UI
- ETag/Cache-Control for images; pre-generate thumbnails lazily
- Server-side pagination and ordering; stable sorts
- Connection pooling for DB; prepared statements; indices
- Optional GPU support later; batch processing tuned by environment

---

## 8) Security & Privacy

- JWT auth; role-ready design; CSRF-safe patterns if cookie mode later
- Validate and sanitize image paths, content-type checks
- Principle of least privilege for workers
- HTTPS in production; signed URLs for original image access

---

## 9) Delivery Plan (No Migration, Full Rebuild)

### Phase 1: Foundations (1–2 weeks)
- Create repositories and base directories: `src/web`, `frontend`, `deploy`
- Initialize FastAPI app structure:
  - `src/web/app.py` (app factory, CORS, middleware)
  - `src/web/api` (routers: `auth`, `photos`, `faces`, `tags`, `people`, `jobs`, `health`)
  - `src/web/schemas` (Pydantic models)
  - `src/web/db` (SQLAlchemy models, session management)
  - `src/web/services` (application services—no DL/GUI logic)
- Database setup:
  - Add SQLAlchemy models matching desktop schema: `photos`, `faces`, `people`, `person_encodings`, `tags`, `phototaglinkage`
  - Configure Alembic; generate initial migration; enable UUID/created_at defaults
  - Auto-create tables on startup (via `Base.metadata.create_all()` in lifespan)
  - Connect to PostgreSQL via env (`DATABASE_URL`); enable connection pooling
  - Note: Schema matches desktop version exactly for compatibility
- Auth foundation:
  - Implement JWT issuance/refresh; `/auth/login`, `/auth/refresh`, `/auth/me`
  - Single-user bootstrap via env secrets; password hashing
- Jobs subsystem:
  - Add Redis + RQ; define job schema/status; `/jobs/{id}` endpoint
  - Worker entrypoint `src/web/worker.py` with graceful shutdown
- Developer experience:
  - Create Docker Compose with services: `api`, `worker`, `db`, `redis`, `frontend`, `proxy`
  - Logging: request IDs middleware; structured logs (JSON)
  - Health endpoints: `/health`, `/version`, `/metrics` (basic)
- Frontend scaffold:
  - Vite + React + TypeScript + Tailwind; base layout (left nav + top bar)
  - Auth flow (login page, token storage), API client with interceptors
  - Routes: Dashboard (placeholder), Scan (placeholder), Process (placeholder), Search (placeholder), Identify (placeholder), Auto-Match (placeholder), Modify (placeholder), Tags (placeholder), Settings (placeholder)

### Phase 2: Image Ingestion & Face Processing (2–3 weeks) ✅ **COMPLETE**
- Image ingestion:
  - ✅ Backend: `/photos/import` supports folder ingest and file upload
  - ✅ Store originals on disk; create DB rows
  - ✅ Generate thumbnails lazily (worker job)
- ✅ **Scan tab UI:**
  - ✅ Folder selection (browse or drag-and-drop)
  - ✅ Upload progress indicators
  - ✅ Recursive scan toggle
  - ✅ Display scan results (count of photos added)
  - ✅ Trigger background job for photo import
  - ✅ Real-time job status with SSE progress updates
- ✅ **DeepFace pipeline:**
  - ✅ Worker task: detect faces (RetinaFace), compute embeddings (ArcFace); store embeddings as binary
  - ✅ Persist face bounding boxes (location as JSON), confidence (face_confidence), quality_score; link to photos
  - ✅ Configurable batch size and thresholds via settings
  - ✅ EXIF orientation handling
  - ✅ Face quality scoring and validation
- ✅ **Process tab UI:**
  - ✅ Start/stop face processing controls
  - ✅ Batch size configuration
  - ✅ Detector/model selection dropdowns (RetinaFace, MTCNN, OpenCV, SSD / ArcFace, Facenet, etc.)
  - ✅ Processing progress bar with current photo count
  - ✅ Job status display (pending, running, completed, failed)
  - ✅ Results summary (faces detected, processing time)
  - ✅ Error handling and retry mechanism
  - ✅ Job cancellation support
  - ✅ Real-time progress updates via SSE
- Progress & observability:
  - ✅ Expose job progress via SSE `/jobs/stream`; show per-task progress bars
  - ✅ Add worker metrics and error surfacing in UI

### Phase 3: Identify Workflow (2–3 weeks)
**Manual identification of unidentified faces - one face at a time**

- **Backend APIs:**
  - `GET /api/v1/faces/unidentified` - Get unidentified faces with filters (quality, date ranges, sorting)
  - `GET /api/v1/faces/{id}/similar` - Get similar faces for comparison
  - `POST /api/v1/faces/{id}/identify` - Assign face to existing person or create new person
  - `GET /api/v1/people` - List all people (for dropdown selection)
  - `POST /api/v1/people` - Create new person
  - Implement person creation and linking, plus `person_encodings` insertions (matches desktop schema)

- **Frontend Identify UI:**
  - Left panel: Current unidentified face display with photo info
  - Right panel: Similar faces grid for comparison (if matches found)
  - Filters: Quality slider (min quality %), date ranges (date_from, date_to, date_processed_from, date_processed_to)
  - Sort options: By quality, by date taken, by date processed
  - Batch size configuration
  - Person assignment form:
    - Dropdown to select existing person
    - "Create New Person" button/modal with name fields
    - Accept/Skip buttons
  - Keyboard navigation: J/K to navigate between faces, Enter to accept
  - Progress indicator: Current face X of Y
  - Photo info display: filename, date taken, path
  - Optimistic updates with server confirmation
  - Inline toasts for success/error feedback

### Phase 4: Auto-Match Workflow (1–2 weeks)
**Automated bulk matching of unidentified faces to identified people**

- **Backend APIs:**
  - `POST /api/v1/faces/auto-match` - Start auto-match process with tolerance threshold
  - `GET /api/v1/faces/auto-match/{match_id}` - Get match results for an identified person
  - `POST /api/v1/faces/auto-match/{match_id}/accept` - Accept bulk matches for a person
  - `POST /api/v1/faces/auto-match/{match_id}/reject` - Reject bulk matches for a person
  - Auto-match engine: For each identified person, find all unidentified faces that match using cosine similarity
  - Quality filters and similarity thresholds (tolerance)
  - Batch processing with progress tracking

- **Frontend Auto-Match UI:**
  - Configuration: Tolerance threshold input (0.0-1.0, lower = stricter)
  - "Start Auto-Match" button to begin process
  - Two-panel layout:
    - Left panel: Identified person with best quality face, person name, stats (X faces already identified)
    - Right panel: Grid of matched unidentified faces for this person
  - Each match shows: Face thumbnail, similarity score/percentage, photo info, checkbox for selection
  - Bulk actions: "Accept Selected", "Accept All", "Reject Selected", "Reject All" buttons
  - Navigation: Previous/Next buttons to navigate through identified people
  - Progress indicator: Person X of Y, N faces matched for current person
  - Match quality display: Similarity percentage for each match
  - Optimistic updates with server confirmation
  - Job-based processing: Auto-match runs as background job with progress tracking

### Phase 5: Modify Identified Workflow (1–2 weeks)
**Edit identified people, view faces per person, unmatch faces, edit person information**

- **Backend APIs:**
  - `GET /api/v1/people` - Get all identified people with face counts (filtered by last name if provided)
  - `GET /api/v1/people/{id}/faces` - Get all faces for a specific person with photo info
  - `PUT /api/v1/people/{id}` - Update person information (first_name, last_name, middle_name, maiden_name, date_of_birth)
  - `POST /api/v1/faces/{id}/unmatch` - Unmatch a face from its person (set person_id to NULL)
  - `POST /api/v1/faces/batch-unmatch` - Batch unmatch multiple faces (accepts array of face IDs)
  - Implement person update with validation (require first_name and last_name)
  - Support filtering people by last name (case-insensitive search)
  - Return face thumbnails and photo metadata for face grid display

- **Frontend Modify Identified UI:**
  - Two-panel layout:
    - Left panel: People list with search/filter by last name
    - Right panel: Face grid for selected person
  - People list features:
    - Search input for filtering by last name (case-insensitive)
    - Display person name with face count: "John Doe (15)"
    - Click person to load their faces in right panel
    - Edit button (✏️) next to each person to open edit dialog
  - Face grid features:
    - Responsive grid layout (adapts to panel width)
    - Face thumbnails with photo icon overlay
    - "Unmatch" button for each face
    - Visual feedback when faces are unmatched (hidden until save)
  - Edit person dialog:
    - Form fields: First name, Last name, Middle name, Maiden name, Date of birth
    - Date picker for date of birth (calendar widget)
    - Validation: Require first_name and last_name
    - Save/Cancel buttons with keyboard shortcuts (Enter to save, Escape to cancel)
  - Change management:
    - Track unmatched faces (temporary state, not persisted until save)
    - "Undo changes" button (undoes all unmatched faces for current person)
    - "Save changes" button (persists all unmatched faces to database)
    - "Exit Edit Identified" button (warns if unsaved changes exist)
  - Optimistic updates with server confirmation
  - Inline toasts for success/error feedback
  - Keyboard navigation support

### Phase 6: Search & Tags (1–2 weeks)
- Search APIs:
  - `/photos` with filters: person_ids, tag_ids, date_from/to, min_quality, sort, page/page_size
  - Stable sorting (date_taken desc, id tie-breaker); efficient indices
- Frontend Search UI:
  - Virtualized masonry/grid with infinite scroll
  - Filters drawer (people, tags, date range, quality); saved searches
  - Photo detail drawer with faces, tags, EXIF; open original
- Tagging:
  - Endpoints: `/tags` CRUD, `/photos/{id}/tags` bulk add/remove
  - Frontend: multi-select in grid and bulk tag apply; inline tag editor
- People management:
  - People list with search; merge people flow (optional)
  - Display representative face and stats per person

### Phase 7: Polish & Release (1–2 weeks)
- Performance polish:
  - HTTP caching headers for images; prefetch thumbnails
  - DB indices review; query timing; N+1 checks; connection pool tuning
  - Web vitals: TTFB, LCP, INP; code-splitting; image lazy-loading
- Accessibility & design:
  - Keyboard access for all actions; focus outlines; ARIA roles
  - High-contrast theme; color tokens; reduced motion option
  - Consistent spacing/typography; empty states and error states
- Security & hardening:
  - Rate limits on mutating endpoints; payload size limits; input validation
  - JWT rotation; secure cookies mode option; HTTPS enforcement behind proxy
  - Signed URLs for originals; restrict path traversal; content-type checks
- Testing & quality:
  - API contract tests, worker unit tests, E2E happy-path (Cypress/Playwright)
  - Load test ingestion and processing pipeline (k6)
  - >80% coverage target for API/workers; lint/type-check CI gates
- Packaging & deployment:
  - Production Dockerfiles (multi-stage), SBOMs
  - Compose/Helm manifests; Traefik/Nginx config with TLS
  - Runbook docs: operations, backups, environment variables

---

## 10) Success Criteria

- Users complete import → process → identify → auto-match → modify → tag → search entirely on the web
- Identify workflow: Manual identification of faces one-at-a-time with similar faces comparison
- Auto-match workflow: Automated bulk matching with tolerance thresholds and bulk accept/reject
- Modify Identified workflow: Edit person information, view faces per person, unmatch faces with undo/save functionality
- Smooth UX with live job progress; no perceived UI blocking
- Clean, documented OpenAPI; type-safe FE client; >80% test coverage (API + workers)
- Production deployment artifacts ready (Docker, env, reverse proxy)

---

## 11) First Sprint Checklist

- [ ] Initialize repo structure for web (`src/web`, `frontend/`)
- [ ] FastAPI app with `/health`, `/version`, `/auth/*`, `/jobs/*`
- [ ] PostgreSQL schema + migrations (Alembic)
- [ ] Redis + worker container; background job wiring
- [ ] Frontend scaffold with auth, layout, routing, theme
- [ ] CI pipeline: lint, type-check, tests, build



