ilia/punimtag
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: scrub homelab IPs and default credentials, reduce emoji noise

Browse Source
This commit is contained in:
ilia
2026-07-26 15:29:13 -04:00
parent 5f8f2e0bee
commit 86d2b476b0
2 changed files with 152 additions and 154 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+150 -152
View File
@@ -8,28 +8,28 @@ A fast, simple, and modern web application for organizing and tagging photos usi
---
## 🎯 Features
## Features
- **🌐 Web-Based**: Modern React frontend with FastAPI backend
- **🔥 DeepFace AI**: State-of-the-art face detection with RetinaFace and ArcFace models
- **🎯 Superior Accuracy**: 512-dimensional embeddings (4x more detailed than face_recognition)
- **⚙️ Multiple Detectors**: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors
- **🎨 Flexible Models**: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models
- **👤 Person Identification**: Identify and tag people across your photo collection
- **🤖 Smart Auto-Matching**: Intelligent face matching with quality scoring and cosine similarity
- **📊 Confidence Calibration**: Empirical-based confidence scores for realistic match probabilities
- **🔍 Advanced Search**: Search by people, dates, tags, and folders
- **🏷️ Tag Management**: Organize photos with hierarchical tags
- **⚡ Batch Processing**: Process thousands of photos efficiently
- **🎯 Unique Faces Filter**: Hide duplicate faces to focus on unique individuals
- **🔄 Real-time Updates**: Live progress tracking and job status updates
- **🌐 Network Path Support**: Browse and scan folders on network shares (UNC paths on Windows, mounted shares on Linux)
- **📁 Native Folder Picker**: Browse button uses native OS folder picker with full absolute path support
- **🔒 Privacy-First**: All data stored locally, no cloud dependencies
- **Web-Based**: Modern React frontend with FastAPI backend
- **DeepFace AI**: State-of-the-art face detection with RetinaFace and ArcFace models
- **Superior Accuracy**: 512-dimensional embeddings (4x more detailed than face_recognition)
- **Multiple Detectors**: Choose from RetinaFace, MTCNN, OpenCV, or SSD detectors
- **Flexible Models**: Select ArcFace, Facenet, Facenet512, or VGG-Face recognition models
- **Person Identification**: Identify and tag people across your photo collection
- **Smart Auto-Matching**: Intelligent face matching with quality scoring and cosine similarity
- **Confidence Calibration**: Empirical-based confidence scores for realistic match probabilities
- **Advanced Search**: Search by people, dates, tags, and folders
- **Tag Management**: Organize photos with hierarchical tags
- **Batch Processing**: Process thousands of photos efficiently
- **Unique Faces Filter**: Hide duplicate faces to focus on unique individuals
- **Real-time Updates**: Live progress tracking and job status updates
- **Network Path Support**: Browse and scan folders on network shares (UNC paths on Windows, mounted shares on Linux)
- **Native Folder Picker**: Browse button uses native OS folder picker with full absolute path support
- **Privacy-First**: All data stored locally, no cloud dependencies
---
## 🚀 Quick Start
## Quick Start
### Prerequisites
@@ -57,14 +57,14 @@ cd punimtag
```
The script will:
- ✅ Check prerequisites (Python 3.12+, Node.js 18+)
- ✅ Install system dependencies (PostgreSQL, Redis, Python tkinter) on Ubuntu/Debian
- ✅ Set up PostgreSQL databases (main + auth)
- ✅ Create Python virtual environment
- ✅ Install all Python dependencies
- ✅ Install all frontend dependencies (admin-frontend and viewer-frontend)
- ✅ Create `.env` configuration files
- ✅ Create necessary data directories
- Check prerequisites (Python 3.12+, Node.js 18+)
- Install system dependencies (PostgreSQL, Redis, Python tkinter) on Ubuntu/Debian
- Set up PostgreSQL databases (main + auth)
- Create Python virtual environment
- Install all Python dependencies
- Install all frontend dependencies (admin-frontend and viewer-frontend)
- Create `.env` configuration files
- Create necessary data directories
**Note:** After installation, you'll need to generate Prisma clients for the viewer-frontend:
```bash
@@ -120,23 +120,23 @@ Both database connections are configured via the `.env` file.
For development, you can use the shared development PostgreSQL server:
**Dev PostgreSQL Server:**
- **Host**: 10.0.10.179 (postgresQA / pve201 VM 109; legacy `.181` is dead)
- **Host**: `<db-host>`
- **Port**: 5432
- **User**: ladmin
- **Password**: [Contact administrator for password]
**Development Server:**
- **Host**: 10.0.10.121
- **Host**: `<backend-host>`
- **User**: appuser
- **Password**: [Contact administrator for password]
Configure your `.env` file for development:
```bash
# Main database (dev)
DATABASE_URL=postgresql+psycopg2://ladmin:[PASSWORD]@10.0.10.179:5432/punimtag
DATABASE_URL=postgresql+psycopg2://ladmin:[PASSWORD]@<db-host>:5432/punimtag
# Auth database (dev)
DATABASE_URL_AUTH=postgresql+psycopg2://ladmin:[PASSWORD]@10.0.10.179:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql+psycopg2://ladmin:[PASSWORD]@<db-host>:5432/punimtag_auth
```
**Install PostgreSQL (if not installed):**
@@ -152,7 +152,7 @@ sudo systemctl enable postgresql
**Create Main Database and User:**
```bash
sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';"
sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD '<choose-a-password>';"
sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;"
```
@@ -192,19 +192,19 @@ The `.env` file in the project root contains database connection strings:
**Local Development:**
```bash
# Main application database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
DATABASE_URL=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag
# Auth database (PostgreSQL - required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag_auth
```
**Development Server:**
```bash
# Main database (dev PostgreSQL server)
DATABASE_URL=postgresql+psycopg2://ladmin:[PASSWORD]@10.0.10.179:5432/punimtag
DATABASE_URL=postgresql+psycopg2://ladmin:[PASSWORD]@<db-host>:5432/punimtag
# Auth database (dev PostgreSQL server)
DATABASE_URL_AUTH=postgresql+psycopg2://ladmin:[PASSWORD]@10.0.10.179:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql+psycopg2://ladmin:[PASSWORD]@<db-host>:5432/punimtag_auth
```
**Automatic Initialization:**
@@ -355,9 +355,7 @@ npm run dev
#### Access the Applications
1. **Admin Interface**: Open your browser to **http://localhost:3000**
- Login with default credentials:
- Username: `admin`
- Password: `admin`
- Log in with the credentials configured in your `.env` (`ADMIN_USERNAME` / `ADMIN_PASSWORD`)
2. **Viewer Interface** (Optional): Open your browser to **http://localhost:3001**
- Public photo viewing interface
- Separate authentication system
@@ -457,14 +455,14 @@ export DISPLAY=localhost:10.0
---
## 📖 Documentation
## Documentation
- **[Architecture](docs/ARCHITECTURE.md)**: System design and technical details
*
## 🏗️ Project Structure
## Project Structure
```
punimtag/
@@ -503,125 +501,125 @@ punimtag/
---
## 📊 Current Status
## Current Status
### Foundations
**Backend:**
- ✅ FastAPI application with CORS middleware
- ✅ Health, version, and metrics endpoints
- ✅ JWT authentication (login, refresh, user info)
- ✅ Job management endpoints (RQ/Redis integration)
- ✅ SQLAlchemy models for all entities
- ✅ Alembic migrations configured and applied
- ✅ Database initialized (PostgreSQL required)
- ✅ RQ worker auto-start (starts automatically with API server)
- ✅ Pending linkage moderation API for user tag suggestions
- FastAPI application with CORS middleware
- Health, version, and metrics endpoints
- JWT authentication (login, refresh, user info)
- Job management endpoints (RQ/Redis integration)
- SQLAlchemy models for all entities
- Alembic migrations configured and applied
- Database initialized (PostgreSQL required)
- RQ worker auto-start (starts automatically with API server)
- Pending linkage moderation API for user tag suggestions
**Frontend:**
- ✅ React + Vite + TypeScript setup
- ✅ Tailwind CSS configured
- ✅ Authentication flow with login page
- ✅ Protected routes with auth context
- ✅ Navigation layout (left sidebar + top bar)
- ✅ All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings)
- ✅ User Tagged Photos moderation tab for approving/denying pending tag linkages
- React + Vite + TypeScript setup
- Tailwind CSS configured
- Authentication flow with login page
- Protected routes with auth context
- Navigation layout (left sidebar + top bar)
- All page routes (Dashboard, Scan, Process, Search, Identify, Auto-Match, Tags, Settings)
- User Tagged Photos moderation tab for approving/denying pending tag linkages
**Database:**
- ✅ All tables created automatically on startup: `photos`, `faces`, `people`, `person_encodings`, `tags`, `phototaglinkage`
- ✅ Schema matches desktop version exactly for full compatibility
- ✅ Indices configured for performance
- ✅ PostgreSQL database (required for both development and production)
- ✅ Separate auth database (PostgreSQL) for frontend user accounts
- All tables created automatically on startup: `photos`, `faces`, `people`, `person_encodings`, `tags`, `phototaglinkage`
- Schema matches desktop version exactly for full compatibility
- Indices configured for performance
- PostgreSQL database (required for both development and production)
- Separate auth database (PostgreSQL) for frontend user accounts
### Image Ingestion & Processing
**Backend:**
- ✅ Photo import service with checksum computation
- ✅ EXIF date extraction and image metadata
- ✅ Folder scanning with recursive option
- ✅ File upload support
- ✅ Background job processing with RQ
- ✅ Real-time job progress via SSE (Server-Sent Events)
- ✅ Duplicate detection (by path and checksum)
- ✅ Photo storage configuration (`PHOTO_STORAGE_DIR`)
- ✅ **DeepFace pipeline integration**
- ✅ **Face detection (RetinaFace, MTCNN, OpenCV, SSD)**
- ✅ **Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)**
- ✅ **Face processing service with configurable detectors/models**
- ✅ **EXIF orientation handling**
- ✅ **Face quality scoring and validation**
- ✅ **Batch processing with progress tracking**
- ✅ **Job cancellation support**
- Photo import service with checksum computation
- EXIF date extraction and image metadata
- Folder scanning with recursive option
- File upload support
- Background job processing with RQ
- Real-time job progress via SSE (Server-Sent Events)
- Duplicate detection (by path and checksum)
- Photo storage configuration (`PHOTO_STORAGE_DIR`)
- **DeepFace pipeline integration**
- **Face detection (RetinaFace, MTCNN, OpenCV, SSD)**
- **Face embeddings computation (ArcFace, Facenet, Facenet512, VGG-Face)**
- **Face processing service with configurable detectors/models**
- **EXIF orientation handling**
- **Face quality scoring and validation**
- **Batch processing with progress tracking**
- **Job cancellation support**
**Frontend:**
- ✅ Scan tab UI with folder selection
- ✅ **Native folder picker (Browse button)** - Uses tkinter for native OS folder selection
- ✅ **Network path support** - Handles UNC paths (Windows: `\\server\share\folder`) and mounted network shares (Linux: `/mnt/nfs-share/photos`)
- ✅ **Full absolute path handling** - Automatically normalizes and validates paths
- ✅ Drag-and-drop file upload
- ✅ Recursive scan toggle
- ✅ Real-time job progress with progress bar
- ✅ Job status monitoring (SSE integration)
- ✅ Results display (added/existing counts)
- ✅ Error handling and user feedback
- ✅ **Process tab UI with configuration controls**
- ✅ **Detector/model selection dropdowns**
- ✅ **Batch size configuration**
- ✅ **Start/Stop processing controls**
- ✅ **Processing progress display with photo count**
- ✅ **Results summary (faces detected, faces stored)**
- ✅ **Job cancellation support**
- Scan tab UI with folder selection
- **Native folder picker (Browse button)** - Uses tkinter for native OS folder selection
- **Network path support** - Handles UNC paths (Windows: `\\server\share\folder`) and mounted network shares (Linux: `/mnt/nfs-share/photos`)
- **Full absolute path handling** - Automatically normalizes and validates paths
- Drag-and-drop file upload
- Recursive scan toggle
- Real-time job progress with progress bar
- Job status monitoring (SSE integration)
- Results display (added/existing counts)
- Error handling and user feedback
- **Process tab UI with configuration controls**
- **Detector/model selection dropdowns**
- **Batch size configuration**
- **Start/Stop processing controls**
- **Processing progress display with photo count**
- **Results summary (faces detected, faces stored)**
- **Job cancellation support**
**Worker:**
- ✅ RQ worker auto-starts with API server
- ✅ Unique worker names to avoid conflicts
- ✅ Graceful shutdown handling
- ✅ **String-based function paths for reliable serialization**
- RQ worker auto-starts with API server
- Unique worker names to avoid conflicts
- Graceful shutdown handling
- **String-based function paths for reliable serialization**
### Identify Workflow & Auto-Match
**Backend:**
- ✅ Identify face endpoints with person creation
- ✅ Auto-match engine with similarity thresholds
- ✅ Unidentified faces management and filtering
- ✅ Person creation and linking
- ✅ Batch identification support
- ✅ Similar faces search with cosine similarity
- ✅ Confidence calibration system (empirical-based)
- ✅ Face unmatch/removal functionality
- ✅ Batch similarity calculations
- Identify face endpoints with person creation
- Auto-match engine with similarity thresholds
- Unidentified faces management and filtering
- Person creation and linking
- Batch identification support
- Similar faces search with cosine similarity
- Confidence calibration system (empirical-based)
- Face unmatch/removal functionality
- Batch similarity calculations
**Frontend:**
- ✅ Identify page UI with face navigation
- ✅ Person creation and editing
- ✅ Similar faces panel with confidence display
- ✅ Auto-Match page with person-centric view
- ✅ Checkbox selection for batch identification
- ✅ Confidence percentages with color coding
- ✅ Unique faces filter (hide duplicates)
- ✅ Date filtering for faces
- ✅ Real-time face matching and display
- Identify page UI with face navigation
- Person creation and editing
- Similar faces panel with confidence display
- Auto-Match page with person-centric view
- Checkbox selection for batch identification
- Confidence percentages with color coding
- Unique faces filter (hide duplicates)
- Date filtering for faces
- Real-time face matching and display
### PSearch & Tags
**Backend:**
- ✅ Search endpoints with filters (people, dates, tags, folders)
- ✅ Tag management endpoints (create, update, delete)
- ✅ Photo-tag linkage system
- ✅ Advanced filtering and querying
- ✅ Photo grid endpoints with pagination
- Search endpoints with filters (people, dates, tags, folders)
- Tag management endpoints (create, update, delete)
- Photo-tag linkage system
- Advanced filtering and querying
- Photo grid endpoints with pagination
**Frontend:**
- ✅ Search page with advanced filters
- ✅ Tag management UI
- ✅ Photo grid with virtualized rendering
- ✅ Filter by people, dates, tags, and folders
- ✅ Search results display
- Search page with advanced filters
- Tag management UI
- Photo grid with virtualized rendering
- Filter by people, dates, tags, and folders
- Search results display
---
## 🔧 Configuration
## Configuration
### Database
@@ -629,10 +627,10 @@ punimtag/
Both databases use PostgreSQL. Configure via the `.env` file:
```bash
# Main application database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
DATABASE_URL=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag
# Auth database (PostgreSQL - required for frontend website users)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag_auth
```
### Environment Variables
@@ -642,17 +640,17 @@ Configuration is managed via the `.env` file in the project root. A `.env.exampl
**Required Configuration:**
```bash
# Main Database (PostgreSQL - required)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
DATABASE_URL=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag
# Auth Database (PostgreSQL - required for frontend website user accounts)
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:<choose-a-password>@localhost:5432/punimtag_auth
# JWT Secrets (change in production!)
SECRET_KEY=dev-secret-key-change-in-production
# Single-user credentials (change in production!)
# Single-user credentials (set your own values!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin
ADMIN_PASSWORD=<choose-a-password>
# Photo storage directory (default: data/uploads)
PHOTO_STORAGE_DIR=data/uploads
@@ -669,13 +667,13 @@ VITE_API_URL=http://127.0.0.1:8000
Create a `.env` file in the `viewer-frontend/` directory:
```bash
# Main database connection (PostgreSQL - required)
DATABASE_URL=postgresql://punimtag:punimtag_password@localhost:5432/punimtag
DATABASE_URL=postgresql://punimtag:<choose-a-password>@localhost:5432/punimtag
# Auth database connection (PostgreSQL - required)
DATABASE_URL_AUTH=postgresql://punimtag:punimtag_password@localhost:5432/punimtag_auth
DATABASE_URL_AUTH=postgresql://punimtag:<choose-a-password>@localhost:5432/punimtag_auth
# Write-capable database connection (optional, falls back to DATABASE_URL if not set)
DATABASE_URL_WRITE=postgresql://punimtag:punimtag_password@localhost:5432/punimtag
DATABASE_URL_WRITE=postgresql://punimtag:<choose-a-password>@localhost:5432/punimtag
# NextAuth configuration
NEXTAUTH_URL=http://localhost:3001
@@ -700,7 +698,7 @@ npx prisma generate
---
### 🔄 Phase 5: Polish & Release (In Progress)
### Phase 5: Polish & Release (In Progress)
- Performance optimization
- Accessibility improvements
- Production deployment
@@ -708,7 +706,7 @@ npx prisma generate
---
## 🏗️ Architecture
## Architecture
**Backend:**
- **Framework**: FastAPI (Python 3.12+)
@@ -731,7 +729,7 @@ npx prisma generate
---
## 📦 Dependencies
## Dependencies
**Backend:**
- `fastapi==0.115.0`
@@ -762,7 +760,7 @@ npx prisma generate
---
## 🔒 Security
## Security
- JWT-based authentication with refresh tokens
- Password hashing with bcrypt
@@ -771,11 +769,11 @@ npx prisma generate
- Input validation via Pydantic schemas
- Separate auth database for frontend website user accounts
**⚠️ Note**: Default credentials (`admin`/`admin`) are for development only. Change in production!
**Note**: Set strong values for `ADMIN_USERNAME`, `ADMIN_PASSWORD`, and `SECRET_KEY` before deploying.
---
## 🐛 Known Limitations
## Known Limitations
- Multi-user support with role-based permissions (single-user mode deprecated)
- PostgreSQL for both development and production
@@ -787,13 +785,13 @@ npx prisma generate
---
## 📝 License
## License
[Add your license here]
---
## 👥 Authors
## Authors
PunimTag Development Team
@@ -801,7 +799,7 @@ PunimTag Development Team
## 🙏 Acknowledgments
## Acknowledgments
- **DeepFace** library by Sefik Ilkin Serengil - Modern face recognition framework
- **ArcFace** - Additive Angular Margin Loss for Deep Face Recognition
@@ -810,19 +808,19 @@ PunimTag Development Team
---
## 🚀 Deployment
## Deployment
### Development Server Deployment
The project includes scripts for deploying to the development server.
**Development Server:**
- **Host**: 10.0.10.121
- **Host**: `<backend-host>`
- **User**: appuser
- **Password**: [Contact administrator for password]
**Development Database:**
- **Host**: 10.0.10.179 (postgresQA / pve201 VM 109; legacy `.181` is dead)
- **Host**: `<db-host>`
- **Port**: 5432
- **User**: ladmin
- **Password**: [Contact administrator for password]
@@ -854,7 +852,7 @@ The deployment script will:
2. **Transfer files to server:**
```bash
# Transfer backend and built frontends
scp -r backend admin-frontend/dist viewer-frontend/.next appuser@10.0.10.121:/path/to/deployment
scp -r backend admin-frontend/dist viewer-frontend/.next appuser@<backend-host>:/path/to/deployment
```
3. **Set up environment on server:**
@@ -884,7 +882,7 @@ See `docs/DEPLOYMENT.md` for complete production deployment guide.
---
## 📧 Support
## Support
For questions or issues:
1. Check documentation in `docs/`
@@ -893,6 +891,6 @@ For questions or issues:
---
**Made with ❤️ for photo enthusiasts**
**Made for photo enthusiasts**