diff --git a/README.md b/README.md index c7071ff..815ad4a 100644 --- a/README.md +++ b/README.md @@ -29,12 +29,42 @@ A fast, simple, and modern web application for organizing and tagging photos usi ### Prerequisites -- Python 3.12 or higher -- Node.js 18+ and npm -- Virtual environment (recommended) +- **Python 3.12 or higher** (with pip) +- **Node.js 18+ and npm** +- **PostgreSQL** (for production, optional for development with SQLite) +- **Redis** (for background job processing) + +**Note:** The automated installation script (`./install.sh`) will install PostgreSQL and Redis automatically on Ubuntu/Debian systems. ### Installation +#### Option 1: Automated Installation (Recommended for Linux/Ubuntu/Debian) + +The automated installation script will install all system dependencies, Python packages, frontend dependencies, and set up databases: + +```bash +# Clone the repository +git clone +cd punimtag + +# Run the installation script +./install.sh +``` + +The script will: +- ✅ Check prerequisites (Python 3.12+, Node.js 18+) +- ✅ Install system dependencies (PostgreSQL, Redis) on Ubuntu/Debian +- ✅ Set up PostgreSQL databases (main + auth) +- ✅ Create Python virtual environment +- ✅ Install all Python dependencies +- ✅ Install all frontend dependencies +- ✅ Create `.env` configuration files +- ✅ Create necessary data directories + +**Note:** On macOS or other systems, the script will skip system dependency installation. You'll need to install PostgreSQL and Redis manually. + +#### Option 2: Manual Installation + ```bash # Clone the repository git clone @@ -56,7 +86,11 @@ cd .. ### Database Setup **PostgreSQL (Default - Network Database):** -The application is configured to use PostgreSQL by default. The database connection is configured via the `.env` file. +The application is configured to use PostgreSQL by default. The application requires **two separate databases**: +1. **Main database** (`punimtag`) - Stores photos, faces, people, tags, and backend user accounts +2. **Auth database** (`punimtag_auth`) - Stores frontend website user accounts and moderation data + +Both database connections are configured via the `.env` file. **Install PostgreSQL (if not installed):** ```bash @@ -69,13 +103,21 @@ sudo systemctl enable postgresql ./scripts/setup_postgresql.sh ``` -**Create Database and User:** +**Create Main Database and User:** ```bash sudo -u postgres psql -c "CREATE USER punimtag WITH PASSWORD 'punimtag_password';" sudo -u postgres psql -c "CREATE DATABASE punimtag OWNER punimtag;" sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag;" ``` +**Create Auth Database (for frontend website user accounts):** +```bash +sudo -u postgres psql -c "CREATE DATABASE punimtag_auth OWNER punimtag;" +sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag;" +``` + +**Note:** The auth database (`punimtag_auth`) stores user accounts for the frontend website, separate from the main application database. Both databases are required for full functionality. + **Grant DELETE Permissions on Auth Database Tables:** If you encounter permission errors when trying to delete records from the auth database (e.g., when using "Clear database" in the admin panel), grant DELETE permissions: @@ -98,9 +140,13 @@ Alternatively, use the automated script (requires sudo password): ``` **Configuration:** -The `.env` file contains the database connection string: +The `.env` file contains both database connection strings: ```bash +# Main application database DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag + +# Auth database (for frontend website users) +DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth ``` **Automatic Initialization:** @@ -128,12 +174,22 @@ DATABASE_URL=sqlite:///data/punimtag.db **Database Schema:** The web version uses the **exact same schema** as the desktop version for full compatibility: -- `photos` - Photo metadata (path, filename, date_taken, processed) +- `photos` - Photo metadata (path, filename, date_taken, processed, media_type) - `people` - Person records (first_name, last_name, middle_name, maiden_name, date_of_birth) -- `faces` - Face detections (encoding, location, quality_score, face_confidence, exif_orientation) +- `faces` - Face detections (encoding, location, quality_score, face_confidence, exif_orientation, excluded) - `person_encodings` - Person face encodings for matching - `tags` - Tag definitions - `phototaglinkage` - Photo-tag relationships (with linkage_type) +- `users` - Backend user accounts (with password hashing, roles, permissions) +- `photo_person_linkage` - Direct photo-person associations (for videos) +- `role_permissions` - Role-based permission matrix + +**Auth Database Schema:** +The separate auth database (`punimtag_auth`) stores frontend website user accounts: +- `users` - Frontend website user accounts (email, password_hash, is_active) +- `pending_photos` - Photos pending moderation +- `pending_identifications` - Face identifications pending approval +- `inappropriate_photo_reports` - Reported photos for review ### Running the Application @@ -429,7 +485,11 @@ punimtag/ **PostgreSQL (Default - Network Database):** The application uses PostgreSQL by default, configured via the `.env` file: ```bash +# Main application database DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag + +# Auth database (for frontend website users) +DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth ``` **SQLite (Alternative - Local Database):** @@ -438,6 +498,8 @@ To use SQLite instead, comment out or remove the `DATABASE_URL` line in `.env`, DATABASE_URL=sqlite:///data/punimtag.db ``` +**Note:** When using SQLite, the auth database (`DATABASE_URL_AUTH`) should still be configured as PostgreSQL if you need frontend website user authentication features. The auth database is optional but required for full multi-user functionality. + ### Environment Variables Configuration is managed via the `.env` file in the project root. A `.env.example` template is provided. @@ -447,6 +509,9 @@ Configuration is managed via the `.env` file in the project root. A `.env.exampl # Database (PostgreSQL by default) DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag +# Auth Database (for frontend website user accounts - separate from main database) +DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth + # JWT Secrets (change in production!) SECRET_KEY=dev-secret-key-change-in-production @@ -458,6 +523,13 @@ ADMIN_PASSWORD=admin PHOTO_STORAGE_DIR=data/uploads ``` +**Frontend Configuration:** +Create a `.env` file in the `frontend/` directory: +```bash +# Backend API URL (must be accessible from browsers) +VITE_API_URL=http://127.0.0.1:8000 +``` + **Note:** The `.env` file is automatically loaded by the application using `python-dotenv`. Environment variables can also be set directly in your shell if preferred. --- @@ -507,8 +579,16 @@ PHOTO_STORAGE_DIR=data/uploads - `python-jose[cryptography]==3.3.0` - `redis==5.0.8` - `rq==1.16.2` +- `psycopg2-binary==2.9.9` (PostgreSQL driver) +- `python-multipart==0.0.9` (file uploads) +- `python-dotenv==1.0.0` (environment variables) +- `bcrypt==4.1.2` (password hashing) - `deepface>=0.0.79` - `tensorflow>=2.13.0` +- `opencv-python>=4.8.0` +- `retina-face>=0.0.13` +- `numpy>=1.21.0` +- `pillow>=8.0.0` **Frontend:** - `react==18.2.0` @@ -522,10 +602,11 @@ PHOTO_STORAGE_DIR=data/uploads ## 🔒 Security - JWT-based authentication with refresh tokens -- Password hashing (to be implemented in production) +- Password hashing with bcrypt - CORS configured for development (restrict in production) - SQL injection prevention via SQLAlchemy ORM - Input validation via Pydantic schemas +- Separate auth database for frontend website user accounts **⚠️ Note**: Default credentials (`admin`/`admin`) are for development only. Change in production! @@ -533,9 +614,8 @@ PHOTO_STORAGE_DIR=data/uploads ## 🐛 Known Limitations -- Single-user mode only (multi-user support planned) +- Multi-user support with role-based permissions (single-user mode deprecated) - SQLite for development (PostgreSQL recommended for production) -- No password hashing yet (plain text comparison - fix before production) - GPU acceleration not yet implemented (CPU-only for now) - Large databases (>50K photos) may require optimization - DeepFace model downloads on first use (can take 5-10 minutes, ~100MB) diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..8b684fa --- /dev/null +++ b/install.sh @@ -0,0 +1,375 @@ +#!/bin/bash +# Comprehensive installation script for PunimTag Web +# Installs all system dependencies, Python packages, and frontend dependencies + +set -e + +# Colors for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Project root directory +PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$PROJECT_ROOT" + +echo -e "${BLUE}╔══════════════════════════════════════════════════════════╗${NC}" +echo -e "${BLUE}║ PunimTag Web - Comprehensive Installation Script ║${NC}" +echo -e "${BLUE}╚══════════════════════════════════════════════════════════╝${NC}" +echo "" + +# Function to check if command exists +command_exists() { + command -v "$1" >/dev/null 2>&1 +} + +# Function to check Python version +check_python_version() { + if command_exists python3; then + PYTHON_VERSION=$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))') + PYTHON_MAJOR=$(echo "$PYTHON_VERSION" | cut -d. -f1) + PYTHON_MINOR=$(echo "$PYTHON_VERSION" | cut -d. -f2) + + if [ "$PYTHON_MAJOR" -lt 3 ] || ([ "$PYTHON_MAJOR" -eq 3 ] && [ "$PYTHON_MINOR" -lt 12 ]); then + echo -e "${RED}❌ Python 3.12 or higher is required. Found: Python $PYTHON_VERSION${NC}" + return 1 + else + echo -e "${GREEN}✅ Python $PYTHON_VERSION found${NC}" + return 0 + fi + else + echo -e "${RED}❌ Python 3 is not installed${NC}" + return 1 + fi +} + +# Function to check Node.js version +check_node_version() { + if command_exists node; then + NODE_VERSION=$(node -v | sed 's/v//') + NODE_MAJOR=$(echo "$NODE_VERSION" | cut -d. -f1) + + if [ "$NODE_MAJOR" -lt 18 ]; then + echo -e "${RED}❌ Node.js 18 or higher is required. Found: Node.js $NODE_VERSION${NC}" + return 1 + else + echo -e "${GREEN}✅ Node.js $NODE_VERSION found${NC}" + return 0 + fi + else + echo -e "${RED}❌ Node.js is not installed${NC}" + return 1 + fi +} + +# Detect OS +detect_os() { + if [[ "$OSTYPE" == "linux-gnu"* ]]; then + if [ -f /etc/os-release ]; then + . /etc/os-release + OS=$ID + else + OS="linux" + fi + elif [[ "$OSTYPE" == "darwin"* ]]; then + OS="macos" + else + OS="unknown" + fi + echo "$OS" +} + +# Install system dependencies (Linux only) +install_system_dependencies() { + local os=$(detect_os) + + if [[ "$os" != "ubuntu" && "$os" != "debian" ]]; then + echo -e "${YELLOW}⚠️ Automatic system dependency installation is only supported on Ubuntu/Debian${NC}" + echo -e "${YELLOW} Please install PostgreSQL and Redis manually for other systems${NC}" + return 0 + fi + + echo -e "${BLUE}📦 Installing system dependencies...${NC}" + + # Update package list + sudo apt update + + # Install PostgreSQL + if ! command_exists psql; then + echo -e "${BLUE} Installing PostgreSQL...${NC}" + sudo apt install -y postgresql postgresql-contrib + sudo systemctl start postgresql + sudo systemctl enable postgresql + echo -e "${GREEN} ✅ PostgreSQL installed${NC}" + else + echo -e "${GREEN} ✅ PostgreSQL already installed${NC}" + fi + + # Install Redis + if ! command_exists redis-cli; then + echo -e "${BLUE} Installing Redis...${NC}" + sudo apt install -y redis-server + sudo systemctl start redis-server + sudo systemctl enable redis-server + echo -e "${GREEN} ✅ Redis installed${NC}" + else + echo -e "${GREEN} ✅ Redis already installed${NC}" + fi + + # Verify Redis is running + if redis-cli ping >/dev/null 2>&1; then + echo -e "${GREEN} ✅ Redis is running${NC}" + else + echo -e "${YELLOW} ⚠️ Redis is not running. Starting Redis...${NC}" + sudo systemctl start redis-server || redis-server --daemonize yes + fi +} + +# Setup PostgreSQL databases +setup_databases() { + echo -e "${BLUE}🗄️ Setting up PostgreSQL databases...${NC}" + + if ! command_exists psql; then + echo -e "${YELLOW}⚠️ PostgreSQL is not installed. Skipping database setup.${NC}" + echo -e "${YELLOW} Please install PostgreSQL and run: ./scripts/setup_postgresql.sh${NC}" + return 0 + fi + + # Start PostgreSQL if not running + if ! sudo systemctl is-active --quiet postgresql; then + echo -e "${BLUE} Starting PostgreSQL service...${NC}" + sudo systemctl start postgresql + fi + + # Create user and databases + echo -e "${BLUE} Creating database user and databases...${NC}" + sudo -u postgres psql << 'EOF' || true +-- Create user if it doesn't exist +DO $$ +BEGIN + IF NOT EXISTS (SELECT FROM pg_user WHERE usename = 'punimtag') THEN + CREATE USER punimtag WITH PASSWORD 'punimtag_password'; + END IF; +END +$$; + +-- Create main database if it doesn't exist +SELECT 'CREATE DATABASE punimtag OWNER punimtag' +WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'punimtag')\gexec + +-- Create auth database if it doesn't exist +SELECT 'CREATE DATABASE punimtag_auth OWNER punimtag' +WHERE NOT EXISTS (SELECT FROM pg_database WHERE datname = 'punimtag_auth')\gexec + +-- Grant privileges +GRANT ALL PRIVILEGES ON DATABASE punimtag TO punimtag; +GRANT ALL PRIVILEGES ON DATABASE punimtag_auth TO punimtag; +\q +EOF + + echo -e "${GREEN} ✅ Databases created${NC}" +} + +# Create Python virtual environment +setup_python_venv() { + echo -e "${BLUE}🐍 Setting up Python virtual environment...${NC}" + + if [ ! -d "venv" ]; then + echo -e "${BLUE} Creating virtual environment...${NC}" + python3 -m venv venv + echo -e "${GREEN} ✅ Virtual environment created${NC}" + else + echo -e "${GREEN} ✅ Virtual environment already exists${NC}" + fi + + # Activate virtual environment + source venv/bin/activate + + # Upgrade pip + echo -e "${BLUE} Upgrading pip...${NC}" + pip install --upgrade pip --quiet + + echo -e "${GREEN} ✅ Python environment ready${NC}" +} + +# Install Python dependencies +install_python_dependencies() { + echo -e "${BLUE}📦 Installing Python dependencies...${NC}" + + if [ ! -f "requirements.txt" ]; then + echo -e "${RED}❌ requirements.txt not found${NC}" + return 1 + fi + + source venv/bin/activate + pip install -r requirements.txt + + echo -e "${GREEN} ✅ Python dependencies installed${NC}" +} + +# Install frontend dependencies +install_frontend_dependencies() { + echo -e "${BLUE}📦 Installing frontend dependencies...${NC}" + + if [ ! -d "frontend" ]; then + echo -e "${RED}❌ frontend directory not found${NC}" + return 1 + fi + + cd frontend + + if [ ! -f "package.json" ]; then + echo -e "${RED}❌ package.json not found${NC}" + cd .. + return 1 + fi + + echo -e "${BLUE} Running npm install...${NC}" + npm install + echo -e "${GREEN} ✅ Frontend dependencies installed${NC}" + + cd .. +} + +# Create .env file if it doesn't exist +create_env_file() { + echo -e "${BLUE}⚙️ Setting up environment configuration...${NC}" + + if [ ! -f ".env" ]; then + echo -e "${BLUE} Creating .env file...${NC}" + cat > .env << 'EOF' +# Database (PostgreSQL by default) +DATABASE_URL=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag + +# Auth Database (for frontend website user accounts - separate from main database) +DATABASE_URL_AUTH=postgresql+psycopg2://punimtag:punimtag_password@localhost:5432/punimtag_auth + +# JWT Secrets (change in production!) +SECRET_KEY=dev-secret-key-change-in-production + +# Single-user credentials (change in production!) +ADMIN_USERNAME=admin +ADMIN_PASSWORD=admin + +# Photo storage directory (default: data/uploads) +PHOTO_STORAGE_DIR=data/uploads +EOF + echo -e "${GREEN} ✅ .env file created${NC}" + echo -e "${YELLOW} ⚠️ Please review and update .env file with your configuration${NC}" + else + echo -e "${GREEN} ✅ .env file already exists${NC}" + fi + + # Create frontend .env file if it doesn't exist + if [ ! -f "frontend/.env" ]; then + echo -e "${BLUE} Creating frontend/.env file...${NC}" + cat > frontend/.env << 'EOF' +# Backend API URL (must be accessible from browsers) +VITE_API_URL=http://127.0.0.1:8000 +EOF + echo -e "${GREEN} ✅ frontend/.env file created${NC}" + else + echo -e "${GREEN} ✅ frontend/.env file already exists${NC}" + fi +} + +# Create data directories +create_data_directories() { + echo -e "${BLUE}📁 Creating data directories...${NC}" + + mkdir -p data/uploads + mkdir -p data/thumbnails + + echo -e "${GREEN} ✅ Data directories created${NC}" +} + +# Main installation function +main() { + echo -e "${BLUE}🔍 Checking prerequisites...${NC}" + echo "" + + # Check Python + if ! check_python_version; then + echo -e "${RED}❌ Please install Python 3.12 or higher${NC}" + exit 1 + fi + + # Check Node.js + if ! check_node_version; then + echo -e "${RED}❌ Please install Node.js 18 or higher${NC}" + exit 1 + fi + + # Check npm + if ! command_exists npm; then + echo -e "${RED}❌ npm is not installed${NC}" + exit 1 + else + echo -e "${GREEN}✅ npm found${NC}" + fi + + echo "" + echo -e "${BLUE}🚀 Starting installation...${NC}" + echo "" + + # Install system dependencies (Linux only) + install_system_dependencies + echo "" + + # Setup databases + setup_databases + echo "" + + # Setup Python environment + setup_python_venv + echo "" + + # Install Python dependencies + install_python_dependencies + echo "" + + # Install frontend dependencies + install_frontend_dependencies + echo "" + + # Create .env files + create_env_file + echo "" + + # Create data directories + create_data_directories + echo "" + + # Success message + echo -e "${GREEN}╔══════════════════════════════════════════════════════════╗${NC}" + echo -e "${GREEN}║ ✅ Installation Complete! ║${NC}" + echo -e "${GREEN}╚══════════════════════════════════════════════════════════╝${NC}" + echo "" + echo -e "${BLUE}📋 Next Steps:${NC}" + echo "" + echo -e "1. ${YELLOW}Review and update .env file${NC} (if needed):" + echo -e " ${BLUE} nano .env${NC}" + echo "" + echo -e "2. ${YELLOW}Activate the virtual environment:${NC}" + echo -e " ${BLUE} source venv/bin/activate${NC}" + echo "" + echo -e "3. ${YELLOW}Start the backend API:${NC}" + echo -e " ${BLUE} export PYTHONPATH=$PROJECT_ROOT${NC}" + echo -e " ${BLUE} uvicorn src.web.app:app --host 127.0.0.1 --port 8000${NC}" + echo "" + echo -e "4. ${YELLOW}In another terminal, start the frontend:${NC}" + echo -e " ${BLUE} cd frontend${NC}" + echo -e " ${BLUE} npm run dev${NC}" + echo "" + echo -e "5. ${YELLOW}Access the application:${NC}" + echo -e " ${BLUE} http://localhost:3000${NC}" + echo "" + echo -e "${GREEN}For more information, see README.md${NC}" + echo "" +} + +# Run main function +main +