ilia/punimtag
1
0
Fork 0
Code Issues 9 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: Add PostgreSQL support and configuration setup for PunimTag

Browse Source 
This commit introduces PostgreSQL as the default database for the PunimTag application, along with a new `.env.example` file for configuration. A setup script for PostgreSQL has been added to automate the installation and database creation process. The README has been updated to reflect these changes, including instructions for setting up PostgreSQL and using the `.env` file for configuration. Additionally, the database session management has been enhanced to support PostgreSQL connection pooling. Documentation has been updated accordingly.
This commit is contained in:
tanyar09 2025-11-14 12:44:12 -05:00
parent c661aeeda6
commit 8caa9e192b
6 changed files with 400 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
.env.example Normal file
View File

@ -0,0 +1,16 @@
# Database Configuration
# PostgreSQL (for network database)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
# Or use SQLite for local development (default if DATABASE_URL not set)
# DATABASE_URL=sqlite:///data/punimtag.db
# Photo Storage
PHOTO_STORAGE_DIR=data/uploads
# JWT Secrets (change in production!)
SECRET_KEY=your-secret-key-here-change-in-production
# Single-user credentials (change in production!)
ADMIN_USERNAME=admin
ADMIN_PASSWORD=admin

64
README.md
View File

@ -55,11 +55,38 @@ cd ..
### Database Setup
**PostgreSQL (Default - Network Database):**
The application is configured to use PostgreSQL by default. The database connection is configured via the `.env` file.
**Install PostgreSQL (if not installed):**
```bash
# On Ubuntu/Debian:
sudo apt update && sudo apt install -y postgresql postgresql-contrib
sudo systemctl start postgresql
sudo systemctl enable postgresql
# Or use the automated setup script:
./scripts/setup_postgresql.sh
```
**Create Database and User:**
```bash
sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';"
sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;"
sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;"
```
**Configuration:**
The `.env` file contains the database connection string:
```bash
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
```
**Automatic Initialization:**
The database and all tables are automatically created on first startup. No manual migration is needed!
The web application will:
- Create the database file at `data/punimtag.db` (SQLite default) if it doesn't exist
- Connect to PostgreSQL using the `.env` configuration
- Create all required tables with the correct schema on startup
- Match the desktop version schema exactly for compatibility
@ -72,10 +99,10 @@ export PYTHONPATH=/home/ladmin/Code/punimtag
python scripts/recreate_tables_web.py
```
**PostgreSQL (Production):**
Set the `DATABASE_URL` environment variable:
**SQLite (Alternative - Local Database):**
To use SQLite instead of PostgreSQL, comment out or remove the `DATABASE_URL` line in `.env`, or set it to:
```bash
export DATABASE_URL=postgresql+psycopg2://user:password@host:port/database
DATABASE_URL=sqlite:///data/punimtag.db
```
**Database Schema:**
@ -90,7 +117,8 @@ The web version uses the **exact same schema** as the desktop version for full c
### Running the Application
**Prerequisites:**
- Redis must be installed and running (for background jobs)
- **PostgreSQL** must be installed and running (see Database Setup section above)
- **Redis** must be installed and running (for background jobs)
**Install Redis (if not installed):**
```bash
@ -375,25 +403,29 @@ punimtag/
### Database
**SQLite (Default for Development):**
**PostgreSQL (Default - Network Database):**
The application uses PostgreSQL by default, configured via the `.env` file:
```bash
# Default location: data/punimtag.db
# No configuration needed
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
```
**PostgreSQL (Production):**
**SQLite (Alternative - Local Database):**
To use SQLite instead, comment out or remove the `DATABASE_URL` line in `.env`, or set:
```bash
export DATABASE_URL=postgresql+psycopg2://user:password@host:port/database
DATABASE_URL=sqlite:///data/punimtag.db
```
### Environment Variables
Configuration is managed via the `.env` file in the project root. A `.env.example` template is provided.
**Required Configuration:**
```bash
# Database (optional, defaults to SQLite)
DATABASE_URL=sqlite:///data/punimtag.db
# Database (PostgreSQL by default)
DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag
# JWT Secrets (change in production!)
SECRET_KEY=your-secret-key-here
SECRET_KEY=dev-secret-key-change-in-production
# Single-user credentials (change in production!)
ADMIN_USERNAME=admin
@ -403,6 +435,8 @@ ADMIN_PASSWORD=admin
PHOTO_STORAGE_DIR=data/uploads
```
**Note:** The `.env` file is automatically loaded by the application using `python-dotenv`. Environment variables can also be set directly in your shell if preferred.
---
@ -420,9 +454,9 @@ PHOTO_STORAGE_DIR=data/uploads
**Backend:**
- **Framework**: FastAPI (Python 3.12+)
- **Database**: SQLite (dev), PostgreSQL (production)
- **Database**: PostgreSQL (default, network), SQLite (optional, local)
- **ORM**: SQLAlchemy 2.0
- **Migrations**: Alembic
- **Configuration**: Environment variables via `.env` file (python-dotenv)
- **Jobs**: Redis + RQ
- **Auth**: JWT (python-jose)

1
requirements.txt
View File

@ -7,6 +7,7 @@ redis==5.0.8
rq==1.16.2
python-jose[cryptography]==3.3.0
python-multipart==0.0.9
python-dotenv==1.0.0
# PunimTag Dependencies - DeepFace Implementation
# Core Dependencies
numpy>=1.21.0

253
scripts/debug_pose_classification.py Executable file
View File

@ -0,0 +1,253 @@
#!/usr/bin/env python3
"""Debug pose classification for identified faces
This script helps identify why poses might be incorrectly classified.
It shows detailed pose information and can recalculate poses from photos.
"""
import sys
import os
import json
from typing import Optional, List, Tuple
# Add project root to path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from src.web.db.models import Face, Person, Photo
from src.web.db.session import get_database_url
from src.utils.pose_detection import PoseDetector
def analyze_pose_classification(
face_id: Optional[int] = None,
person_id: Optional[int] = None,
recalculate: bool = False,
) -> None:
"""Analyze pose classification for identified faces.
Args:
face_id: Specific face ID to check (None = all identified faces)
person_id: Specific person ID to check (None = all persons)
recalculate: If True, recalculate pose from photo to verify classification
"""
db_url = get_database_url()
print(f"Connecting to database: {db_url}")
engine = create_engine(db_url)
Session = sessionmaker(bind=engine)
session = Session()
try:
# Build query
query = (
session.query(Face, Person, Photo)
.join(Person, Face.person_id == Person.id)
.join(Photo, Face.photo_id == Photo.id)
.filter(Face.person_id.isnot(None))
)
if face_id:
query = query.filter(Face.id == face_id)
if person_id:
query = query.filter(Person.id == person_id)
faces = query.order_by(Person.id, Face.id).all()
if not faces:
print("No identified faces found matching criteria.")
return
print(f"\n{'='*80}")
print(f"Found {len(faces)} identified face(s)")
print(f"{'='*80}\n")
pose_detector = None
if recalculate:
try:
pose_detector = PoseDetector()
print("Pose detector initialized for recalculation\n")
except Exception as e:
print(f"Warning: Could not initialize pose detector: {e}")
print("Skipping recalculation\n")
recalculate = False
for face, person, photo in faces:
person_name = f"{person.first_name} {person.last_name}"
print(f"{'='*80}")
print(f"Face ID: {face.id}")
print(f"Person: {person_name} (ID: {person.id})")
print(f"Photo: {photo.filename}")
print(f"Photo Path: {photo.path}")
print(f"{'-'*80}")
# Current stored pose information
print("STORED POSE INFORMATION:")
print(f" Pose Mode: {face.pose_mode}")
print(f" Yaw Angle: {face.yaw_angle:.2f}°" if face.yaw_angle is not None else " Yaw Angle: None")
print(f" Pitch Angle: {face.pitch_angle:.2f}°" if face.pitch_angle is not None else " Pitch Angle: None")
print(f" Roll Angle: {face.roll_angle:.2f}°" if face.roll_angle is not None else " Roll Angle: None")
print(f" Face Confidence: {face.face_confidence:.3f}")
print(f" Quality Score: {face.quality_score:.3f}")
# Parse location
try:
location = json.loads(face.location) if isinstance(face.location, str) else face.location
print(f" Location: {location}")
except:
print(f" Location: {face.location}")
# Analyze classification
print(f"\nPOSE CLASSIFICATION ANALYSIS:")
yaw = face.yaw_angle
pitch = face.pitch_angle
roll = face.roll_angle
if yaw is not None:
abs_yaw = abs(yaw)
print(f" Yaw: {yaw:.2f}° (absolute: {abs_yaw:.2f}°)")
if abs_yaw < 30.0:
expected_mode = "frontal"
print(f" → Expected: {expected_mode} (yaw < 30°)")
elif yaw <= -30.0:
expected_mode = "profile_left"
print(f" → Expected: {expected_mode} (yaw <= -30°, face turned left)")
elif yaw >= 30.0:
expected_mode = "profile_right"
print(f" → Expected: {expected_mode} (yaw >= 30°, face turned right)")
else:
expected_mode = "unknown"
print(f" → Expected: {expected_mode} (edge case)")
if face.pose_mode != expected_mode:
print(f" ⚠️ MISMATCH: Stored pose_mode='{face.pose_mode}' but expected '{expected_mode}'")
else:
print(f" ✓ Classification matches expected mode")
else:
print(f" Yaw: None (cannot determine pose from yaw)")
print(f" ⚠️ Warning: Yaw angle is missing, pose classification may be unreliable")
# Recalculate if requested
if recalculate and pose_detector and photo.path and os.path.exists(photo.path):
print(f"\nRECALCULATING POSE FROM PHOTO:")
try:
pose_faces = pose_detector.detect_pose_faces(photo.path)
if not pose_faces:
print(" No faces detected in photo")
else:
# Try to match face by location
face_location = location if isinstance(location, dict) else json.loads(face.location) if isinstance(face.location, str) else {}
face_x = face_location.get('x', 0)
face_y = face_location.get('y', 0)
face_w = face_location.get('w', 0)
face_h = face_location.get('h', 0)
face_center_x = face_x + face_w / 2
face_center_y = face_y + face_h / 2
best_match = None
best_distance = float('inf')
for pose_face in pose_faces:
pose_area = pose_face.get('facial_area', {})
if isinstance(pose_area, dict):
pose_x = pose_area.get('x', 0)
pose_y = pose_area.get('y', 0)
pose_w = pose_area.get('w', 0)
pose_h = pose_area.get('h', 0)
pose_center_x = pose_x + pose_w / 2
pose_center_y = pose_y + pose_h / 2
# Calculate distance between centers
distance = ((face_center_x - pose_center_x) ** 2 +
(face_center_y - pose_center_y) ** 2) ** 0.5
if distance < best_distance:
best_distance = distance
best_match = pose_face
if best_match:
recalc_yaw = best_match.get('yaw_angle')
recalc_pitch = best_match.get('pitch_angle')
recalc_roll = best_match.get('roll_angle')
recalc_face_width = best_match.get('face_width')
recalc_pose_mode = best_match.get('pose_mode')
print(f" Recalculated Yaw: {recalc_yaw:.2f}°" if recalc_yaw is not None else " Recalculated Yaw: None")
print(f" Recalculated Pitch: {recalc_pitch:.2f}°" if recalc_pitch is not None else " Recalculated Pitch: None")
print(f" Recalculated Roll: {recalc_roll:.2f}°" if recalc_roll is not None else " Recalculated Roll: None")
print(f" Face Width: {recalc_face_width:.2f}px" if recalc_face_width is not None else " Face Width: None")
print(f" Recalculated Pose Mode: {recalc_pose_mode}")
# Compare
if recalc_pose_mode != face.pose_mode:
print(f" ⚠️ MISMATCH: Stored='{face.pose_mode}' vs Recalculated='{recalc_pose_mode}'")
if recalc_yaw is not None and face.yaw_angle is not None:
# Convert Decimal to float for comparison
stored_yaw = float(face.yaw_angle)
yaw_diff = abs(recalc_yaw - stored_yaw)
if yaw_diff > 1.0: # More than 1 degree difference
print(f" ⚠️ Yaw difference: {yaw_diff:.2f}°")
else:
print(" Could not match face location to detected faces")
except Exception as e:
print(f" Error recalculating: {e}")
import traceback
traceback.print_exc()
print()
print(f"{'='*80}")
print("Analysis complete")
print(f"{'='*80}\n")
finally:
session.close()
def main():
"""Main entry point"""
import argparse
parser = argparse.ArgumentParser(
description="Debug pose classification for identified faces"
)
parser.add_argument(
"--face-id",
type=int,
help="Specific face ID to check"
)
parser.add_argument(
"--person-id",
type=int,
help="Specific person ID to check"
)
parser.add_argument(
"--recalculate",
action="store_true",
help="Recalculate pose from photo to verify classification"
)
args = parser.parse_args()
try:
analyze_pose_classification(
face_id=args.face_id,
person_id=args.person_id,
recalculate=args.recalculate,
)
except Exception as e:
print(f"❌ Error: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()

59
scripts/setup_postgresql.sh Executable file
View File

@ -0,0 +1,59 @@
#!/bin/bash
# Setup script for PostgreSQL database for PunimTag
set -e
echo "🔧 Setting up PostgreSQL for PunimTag..."
# Check if PostgreSQL is installed
if ! command -v psql &> /dev/null; then
echo "📦 Installing PostgreSQL..."
sudo apt update
sudo apt install -y postgresql postgresql-contrib
echo "✅ PostgreSQL installed"
else
echo "✅ PostgreSQL is already installed"
fi
# Start PostgreSQL service
echo "🚀 Starting PostgreSQL service..."
sudo systemctl start postgresql
sudo systemctl enable postgresql
# Create database and user
echo "📝 Creating database and user..."
sudo -u postgres psql << EOF
-- Create user if it doesn't exist
DO \$\$
BEGIN
IF NOT EXISTS (SELECT FROM pg_user WHERE usename = 'punimtag') THEN
CREATE USER punimtag WITH PASSWORD 'punimtag_password';
END IF;
END
\$\$;
-- Create database if it doesn't exist
SELECT 'CREATE DATABASE punimtag OWNER punimtag'
WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'punimtag')\gexec
-- Grant privileges
GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;
\q
EOF
echo "✅ Database and user created"
echo ""
echo "📋 Database connection details:"
echo " Host: localhost"
echo " Port: 5432"
echo " Database: punimtag"
echo " User: punimtag"
echo " Password: punimtag_password"
echo ""
echo "✅ PostgreSQL setup complete!"
echo ""
echo "Next steps:"
echo "1. Install python-dotenv: pip install python-dotenv"
echo "2. The .env file is already configured with the connection string"
echo "3. Run your application - it will connect to PostgreSQL automatically"

23
src/web/db/session.py
View File

@ -1,10 +1,16 @@
from __future__ import annotations
from pathlib import Path
from typing import Generator
from dotenv import load_dotenv
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
# Load environment variables from .env file if it exists
env_path = Path(__file__).parent.parent.parent.parent / ".env"
load_dotenv(dotenv_path=env_path)
def get_database_url() -> str:
"""Fetch database URL from environment or defaults."""
@ -22,7 +28,22 @@ database_url = get_database_url()
connect_args = {}
if database_url.startswith("sqlite"):
connect_args = {"check_same_thread": False}
engine = create_engine(database_url, pool_pre_ping=True, future=True, connect_args=connect_args)
# PostgreSQL connection pool settings
pool_kwargs = {"pool_pre_ping": True}
if database_url.startswith("postgresql"):
pool_kwargs.update({
"pool_size": 10,
"max_overflow": 20,
"pool_recycle": 3600,
})
engine = create_engine(
database_url,
future=True,
connect_args=connect_args,
**pool_kwargs
)
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, future=True)