docs: Update README and add run script for Redis and RQ worker integration
This commit enhances the README with detailed instructions for installing and starting Redis, including commands for various operating systems. It clarifies the automatic startup of the RQ worker with the FastAPI server and updates the project status for Phase 2 features. Additionally, a new script `run_api_with_worker.sh` is introduced to streamline the process of starting the FastAPI server alongside the RQ worker, ensuring a smoother setup for users. The worker now has a unique name to prevent conflicts during execution.
This commit is contained in:
tanyar09
@@ -65,22 +65,33 @@ This creates the SQLite database at `data/punimtag.db` (default). For PostgreSQL
|
||||
### Running the Application
|
||||
|
||||
**Prerequisites:**
|
||||
- Redis must be running (for background jobs)
|
||||
- Redis must be installed and running (for background jobs)
|
||||
|
||||
**Install Redis (if not installed):**
|
||||
```bash
|
||||
# Check if Redis is running
|
||||
redis-cli ping
|
||||
# If not running, start Redis:
|
||||
# On Linux: sudo systemctl start redis
|
||||
# On macOS with Homebrew: brew services start redis
|
||||
# Or run directly: redis-server
|
||||
# On Ubuntu/Debian:
|
||||
sudo apt update && sudo apt install -y redis-server
|
||||
sudo systemctl start redis-server
|
||||
sudo systemctl enable redis-server # Auto-start on boot
|
||||
|
||||
# On macOS with Homebrew:
|
||||
brew install redis
|
||||
brew services start redis
|
||||
|
||||
# Verify Redis is running:
|
||||
redis-cli ping # Should respond with "PONG"
|
||||
```
|
||||
|
||||
**Start Redis (if installed but not running):**
|
||||
```bash
|
||||
# On Linux:
|
||||
sudo systemctl start redis-server
|
||||
|
||||
# Or run directly:
|
||||
redis-server
|
||||
```
|
||||
|
||||
**Terminal 1 - Redis (if not running as service):**
|
||||
```bash
|
||||
redis-server
|
||||
```
|
||||
|
||||
**Terminal 2 - Backend API:**
|
||||
**Terminal 2 - Backend API (automatically starts RQ worker):**
|
||||
```bash
|
||||
cd /home/ladmin/Code/punimtag
|
||||
source venv/bin/activate
|
||||
@@ -88,15 +99,16 @@ export PYTHONPATH=/home/ladmin/Code/punimtag
|
||||
uvicorn src.web.app:app --host 127.0.0.1 --port 8000
|
||||
```
|
||||
|
||||
**Terminal 3 - RQ Worker (required for photo import):**
|
||||
```bash
|
||||
cd /home/ladmin/Code/punimtag
|
||||
source venv/bin/activate
|
||||
export PYTHONPATH=/home/ladmin/Code/punimtag
|
||||
python -m src.web.worker
|
||||
You should see:
|
||||
```
|
||||
✅ RQ worker started in background subprocess (PID: ...)
|
||||
INFO: Started server process
|
||||
INFO: Uvicorn running on http://127.0.0.1:8000
|
||||
```
|
||||
|
||||
**Terminal 4 - Frontend:**
|
||||
**Note:** The RQ worker automatically starts in a background subprocess when the API starts. You'll see a confirmation message with the worker PID. If Redis isn't running, you'll see a warning message.
|
||||
|
||||
**Terminal 3 - Frontend:**
|
||||
```bash
|
||||
cd /home/ladmin/Code/punimtag/frontend
|
||||
npm run dev
|
||||
@@ -108,7 +120,11 @@ Then open your browser to **http://localhost:3000**
|
||||
- Username: `admin`
|
||||
- Password: `admin`
|
||||
|
||||
**Note:** The RQ worker (Terminal 3) is required for background photo import jobs. Without it, jobs will remain in "Pending" status.
|
||||
**Note:**
|
||||
- The RQ worker starts automatically in a background subprocess when the API server starts
|
||||
- Make sure Redis is running first, or the worker won't start
|
||||
- Worker names are unique to avoid conflicts when restarting
|
||||
- Photo uploads are stored in `data/uploads` (configurable via `PHOTO_STORAGE_DIR` env var)
|
||||
|
||||
---
|
||||
|
||||
@@ -119,6 +135,13 @@ Then open your browser to **http://localhost:3000**
|
||||
- **[Phase 1 Status](docs/PHASE1_FOUNDATION_STATUS.md)**: Phase 1 implementation status
|
||||
- **[Phase 1 Checklist](docs/PHASE1_CHECKLIST.md)**: Complete Phase 1 checklist
|
||||
|
||||
**Phase 2 Features:**
|
||||
- Photo import via folder scan or file upload
|
||||
- Background processing with progress tracking
|
||||
- Real-time job status updates (SSE)
|
||||
- Duplicate detection by checksum
|
||||
- EXIF metadata extraction
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ Project Structure
|
||||
@@ -158,10 +181,10 @@ punimtag/
|
||||
- ✅ Health, version, and metrics endpoints
|
||||
- ✅ JWT authentication (login, refresh, user info)
|
||||
- ✅ Job management endpoints (RQ/Redis integration)
|
||||
- ✅ API routers for photos, faces, people, tags (placeholders)
|
||||
- ✅ SQLAlchemy models for all entities
|
||||
- ✅ Alembic migrations configured and applied
|
||||
- ✅ Database initialized (SQLite default, PostgreSQL supported)
|
||||
- ✅ RQ worker auto-start (starts automatically with API server)
|
||||
|
||||
**Frontend:**
|
||||
- ✅ React + Vite + TypeScript setup
|
||||
@@ -176,13 +199,40 @@ punimtag/
|
||||
- ✅ Indices configured for performance
|
||||
- ✅ SQLite database at `data/punimtag.db`
|
||||
|
||||
### Next: Phase 2 - Processing & Identify
|
||||
### Phase 2: Image Ingestion & Scan Tab ✅ **COMPLETE**
|
||||
|
||||
- Photo import (folder scan and upload)
|
||||
- Face detection and processing pipeline
|
||||
**Backend:**
|
||||
- ✅ Photo import service with checksum computation
|
||||
- ✅ EXIF date extraction and image metadata
|
||||
- ✅ Folder scanning with recursive option
|
||||
- ✅ File upload support
|
||||
- ✅ Background job processing with RQ
|
||||
- ✅ Real-time job progress via SSE (Server-Sent Events)
|
||||
- ✅ Duplicate detection (by path and checksum)
|
||||
- ✅ Photo storage configuration (`PHOTO_STORAGE_DIR`)
|
||||
|
||||
**Frontend:**
|
||||
- ✅ Scan tab UI with folder selection
|
||||
- ✅ Drag-and-drop file upload
|
||||
- ✅ Recursive scan toggle
|
||||
- ✅ Real-time job progress with progress bar
|
||||
- ✅ Job status monitoring (SSE integration)
|
||||
- ✅ Results display (added/existing counts)
|
||||
- ✅ Error handling and user feedback
|
||||
|
||||
**Worker:**
|
||||
- ✅ RQ worker auto-starts with API server
|
||||
- ✅ Unique worker names to avoid conflicts
|
||||
- ✅ Graceful shutdown handling
|
||||
|
||||
### Next: Phase 3 - Face Processing & Identify
|
||||
|
||||
- DeepFace pipeline integration
|
||||
- Face detection (RetinaFace, MTCNN, OpenCV, SSD)
|
||||
- Face embeddings computation (ArcFace, Facenet, etc.)
|
||||
- Identify workflow UI
|
||||
- Auto-match engine
|
||||
- Scan and Process tab implementations
|
||||
- Process tab implementation
|
||||
|
||||
---
|
||||
|
||||
@@ -213,6 +263,9 @@ SECRET_KEY=your-secret-key-here
|
||||
# Single-user credentials (change in production!)
|
||||
ADMIN_USERNAME=admin
|
||||
ADMIN_PASSWORD=admin
|
||||
|
||||
# Photo storage directory (default: data/uploads)
|
||||
PHOTO_STORAGE_DIR=data/uploads
|
||||
```
|
||||
|
||||
---
|
||||
@@ -242,20 +295,26 @@ npm test
|
||||
- Database setup
|
||||
- Basic API endpoints
|
||||
|
||||
### 🔄 Phase 2: Processing & Identify (In Progress)
|
||||
- Photo import (scan/upload)
|
||||
- DeepFace processing pipeline
|
||||
### ✅ Phase 2: Image Ingestion & Scan Tab (Complete)
|
||||
- ✅ Photo import (folder scan and file upload)
|
||||
- ✅ Background job processing with RQ
|
||||
- ✅ Real-time progress tracking via SSE
|
||||
- ✅ Scan tab UI implementation
|
||||
- ✅ Duplicate detection and metadata extraction
|
||||
|
||||
### 🔄 Phase 3: Processing & Identify (In Progress)
|
||||
- Face detection and processing pipeline (DeepFace)
|
||||
- Identify workflow UI
|
||||
- Auto-match engine
|
||||
- Scan and Process tabs
|
||||
- Process tab implementation
|
||||
|
||||
### 📋 Phase 3: Search & Tags
|
||||
### 📋 Phase 4: Search & Tags
|
||||
- Search endpoints with filters
|
||||
- Tag management UI
|
||||
- Virtualized photo grid
|
||||
- Advanced filtering
|
||||
|
||||
### 🎨 Phase 4: Polish & Release
|
||||
### 🎨 Phase 5: Polish & Release
|
||||
- Performance optimization
|
||||
- Accessibility improvements
|
||||
- Production deployment
|
||||
|
||||
Reference in New Issue
Block a user