Update project documentation and structure; enhance README, finalize project reorganization, and improve testing standards.
This commit is contained in:
@@ -1,136 +1,294 @@
|
||||
# PunimTag - Intelligent Photo Management System
|
||||
# PunimTag CLI - Minimal Photo Face Tagger
|
||||
|
||||
A Flask-based photo management system with automatic face recognition, tagging, and duplicate detection.
|
||||
A simple command-line tool for automatic face recognition and photo tagging. No web interface, no complex dependencies - just the essentials.
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
# 1. Setup (one time only)
|
||||
git clone <your-repo>
|
||||
cd PunimTag
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
python3 setup.py
|
||||
|
||||
# Run the application
|
||||
python main.py
|
||||
# 2. Scan photos
|
||||
python3 photo_tagger.py scan /path/to/your/photos
|
||||
|
||||
# Access the web interface
|
||||
# http://localhost:5000
|
||||
# 3. Process faces
|
||||
python3 photo_tagger.py process
|
||||
|
||||
# 4. Identify faces interactively
|
||||
python3 photo_tagger.py identify
|
||||
|
||||
# 5. View statistics
|
||||
python3 photo_tagger.py stats
|
||||
```
|
||||
|
||||
## 📁 Project Structure
|
||||
## 📦 Installation
|
||||
|
||||
### Automatic Setup (Recommended)
|
||||
```bash
|
||||
# Clone and setup
|
||||
git clone <your-repo>
|
||||
cd PunimTag
|
||||
|
||||
# Create virtual environment (IMPORTANT!)
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
|
||||
# Run setup script
|
||||
python3 setup.py
|
||||
```
|
||||
|
||||
**⚠️ IMPORTANT**: Always activate the virtual environment before running any commands:
|
||||
```bash
|
||||
source venv/bin/activate # Run this every time you open a new terminal
|
||||
```
|
||||
|
||||
### Manual Setup (Alternative)
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
python3 photo_tagger.py stats # Creates database
|
||||
```
|
||||
|
||||
## 🎯 Commands
|
||||
|
||||
### Scan for Photos
|
||||
```bash
|
||||
# Scan a folder
|
||||
python3 photo_tagger.py scan /path/to/photos
|
||||
|
||||
# Scan recursively (recommended)
|
||||
python3 photo_tagger.py scan /path/to/photos --recursive
|
||||
```
|
||||
|
||||
### Process Photos for Faces
|
||||
```bash
|
||||
# Process 50 photos (default)
|
||||
python3 photo_tagger.py process
|
||||
|
||||
# Process 20 photos with CNN model (more accurate)
|
||||
python3 photo_tagger.py process --limit 20 --model cnn
|
||||
|
||||
# Process with HOG model (faster)
|
||||
python3 photo_tagger.py process --limit 100 --model hog
|
||||
```
|
||||
|
||||
### Identify Faces
|
||||
```bash
|
||||
# Identify 20 faces interactively
|
||||
python3 photo_tagger.py identify
|
||||
|
||||
# Identify 10 faces at a time
|
||||
python3 photo_tagger.py identify --batch 10
|
||||
```
|
||||
|
||||
**Interactive commands during identification:**
|
||||
- Type person's name to identify
|
||||
- `s` = skip this face
|
||||
- `q` = quit
|
||||
- `list` = show known people
|
||||
|
||||
### Add Tags
|
||||
```bash
|
||||
# Tag photos matching pattern
|
||||
python3 photo_tagger.py tag --pattern "vacation"
|
||||
|
||||
# Tag any photos
|
||||
python3 photo_tagger.py tag
|
||||
```
|
||||
|
||||
### Search
|
||||
```bash
|
||||
# Find photos with a person
|
||||
python3 photo_tagger.py search "John"
|
||||
|
||||
# Find photos with partial name match
|
||||
python3 photo_tagger.py search "Joh"
|
||||
```
|
||||
|
||||
### Statistics
|
||||
```bash
|
||||
# View database statistics
|
||||
python3 photo_tagger.py stats
|
||||
```
|
||||
|
||||
## 📊 Example Workflow
|
||||
|
||||
```bash
|
||||
# ALWAYS activate virtual environment first!
|
||||
source venv/bin/activate
|
||||
|
||||
# 1. Scan your photo collection
|
||||
python3 photo_tagger.py scan ~/Pictures --recursive
|
||||
|
||||
# 2. Process photos for faces (start with small batch)
|
||||
python3 photo_tagger.py process --limit 20
|
||||
|
||||
# 3. Check what we found
|
||||
python3 photo_tagger.py stats
|
||||
|
||||
# 4. Identify some faces
|
||||
python3 photo_tagger.py identify --batch 10
|
||||
|
||||
# 5. Search for photos of someone
|
||||
python3 photo_tagger.py search "Alice"
|
||||
|
||||
# 6. Add some tags
|
||||
python3 photo_tagger.py tag --pattern "birthday"
|
||||
```
|
||||
|
||||
## 🗃️ Database
|
||||
|
||||
The tool uses SQLite database (`photos.db` by default) with these tables:
|
||||
- **photos** - Photo file paths and processing status
|
||||
- **people** - Known people names
|
||||
- **faces** - Face encodings and locations
|
||||
- **tags** - Custom tags for photos
|
||||
|
||||
## ⚙️ Configuration
|
||||
|
||||
### Face Detection Models
|
||||
- **hog** - Faster, good for CPU-only systems
|
||||
- **cnn** - More accurate, requires more processing power
|
||||
|
||||
### Database Location
|
||||
```bash
|
||||
# Use custom database file
|
||||
python3 photo_tagger.py scan /photos --db /path/to/my.db
|
||||
```
|
||||
|
||||
## 🔧 System Requirements
|
||||
|
||||
### Required System Packages (Ubuntu/Debian)
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y cmake build-essential libopenblas-dev liblapack-dev libx11-dev libgtk-3-dev python3-dev python3-venv
|
||||
```
|
||||
|
||||
### Python Dependencies
|
||||
- `face-recognition` - Face detection and recognition
|
||||
- `dlib` - Machine learning library
|
||||
- `pillow` - Image processing
|
||||
- `numpy` - Numerical operations
|
||||
- `click` - Command line interface
|
||||
- `setuptools` - Package management
|
||||
|
||||
## 📁 File Structure
|
||||
|
||||
```
|
||||
PunimTag/
|
||||
├── src/ # Main application source code
|
||||
│ ├── backend/ # Flask backend and API
|
||||
│ │ ├── app.py # Main Flask application
|
||||
│ │ ├── db_manager.py # Database operations
|
||||
│ │ └── visual_identifier.py # Face recognition
|
||||
│ ├── frontend/ # JavaScript and UI components
|
||||
│ └── utils/ # Utility functions
|
||||
│ └── tag_manager.py # Tag management
|
||||
├── docs/ # Documentation and steering documents
|
||||
│ ├── product.md # Product vision and goals
|
||||
│ ├── structure.md # Project organization
|
||||
│ ├── tech.md # Technical architecture
|
||||
│ ├── api-standards.md # API design standards
|
||||
│ ├── testing-standards.md # Testing guidelines
|
||||
│ └── code-conventions.md # Coding standards
|
||||
├── tests/ # Test files
|
||||
│ ├── test_main.py # Main test suite
|
||||
│ └── conftest.py # Test configuration
|
||||
├── data/ # Database files and user data
|
||||
├── config/ # Configuration files
|
||||
│ ├── settings.py # Application settings
|
||||
│ └── punimtag_config.json
|
||||
├── scripts/ # Utility scripts
|
||||
├── assets/ # Static assets
|
||||
├── photos/ # User photo storage
|
||||
└── main.py # Application entry point
|
||||
├── photo_tagger.py # Main CLI tool
|
||||
├── setup.py # Setup script
|
||||
├── run.sh # Convenience script (auto-activates venv)
|
||||
├── requirements.txt # Python dependencies
|
||||
├── README.md # This file
|
||||
├── venv/ # Virtual environment (created by setup)
|
||||
├── photos.db # Database (created automatically)
|
||||
├── data/ # Additional data files
|
||||
└── logs/ # Log files
|
||||
```
|
||||
|
||||
## 🎯 Key Features
|
||||
|
||||
- **Automatic Face Recognition**: Identify and tag people in photos
|
||||
- **Smart Organization**: Group photos by people, events, and locations
|
||||
- **Duplicate Detection**: Find and manage duplicate photos automatically
|
||||
- **Intuitive Interface**: Web-based GUI with progressive loading
|
||||
- **Privacy-First**: Local processing, no cloud dependencies
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### Steering Documents
|
||||
|
||||
- **[Product Vision](docs/product.md)**: Product goals, target users, and roadmap
|
||||
- **[Project Structure](docs/structure.md)**: Architecture and organization principles
|
||||
- **[Technical Architecture](docs/tech.md)**: Technology stack and implementation details
|
||||
- **[API Standards](docs/api-standards.md)**: API design and development guidelines
|
||||
- **[Testing Standards](docs/testing-standards.md)**: Testing strategy and best practices
|
||||
- **[Code Conventions](docs/code-conventions.md)**: Coding standards and style guides
|
||||
|
||||
### Development Guidelines
|
||||
|
||||
1. **Follow the steering documents** for consistent development
|
||||
2. **Use the organized structure** - place code in appropriate directories
|
||||
3. **Write tests** following the testing standards
|
||||
4. **Follow API standards** for all endpoints
|
||||
5. **Adhere to code conventions** for maintainability
|
||||
|
||||
## 🧪 Testing
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### "externally-managed-environment" Error
|
||||
**Solution**: Always use a virtual environment!
|
||||
```bash
|
||||
# Run the main test suite
|
||||
python tests/test_main.py
|
||||
|
||||
# Run with pytest (if installed)
|
||||
pytest tests/
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
python3 setup.py
|
||||
```
|
||||
|
||||
## 🔧 Configuration
|
||||
|
||||
Configuration is centralized in `config/settings.py`:
|
||||
|
||||
- Database paths
|
||||
- Face recognition settings
|
||||
- File upload limits
|
||||
- Thumbnail sizes
|
||||
|
||||
## 🚀 Deployment
|
||||
|
||||
### Development
|
||||
|
||||
### Virtual Environment Not Active
|
||||
**Problem**: Commands fail or use wrong Python
|
||||
**Solution**: Always activate the virtual environment:
|
||||
```bash
|
||||
python main.py
|
||||
source venv/bin/activate
|
||||
# You should see (venv) in your prompt
|
||||
```
|
||||
|
||||
### Production
|
||||
|
||||
### dlib Installation Issues
|
||||
```bash
|
||||
# Use a WSGI server like Gunicorn
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 main:app
|
||||
# Ubuntu/Debian - install system dependencies first
|
||||
sudo apt-get install build-essential cmake libopenblas-dev
|
||||
|
||||
# Then retry setup
|
||||
source venv/bin/activate
|
||||
python3 setup.py
|
||||
```
|
||||
|
||||
## 📦 Dependencies
|
||||
### "Please install face_recognition_models" Warning
|
||||
This warning is harmless - the application still works correctly. It's a known issue with Python 3.13.
|
||||
|
||||
- **Flask**: Web framework
|
||||
- **SQLite**: Database
|
||||
- **dlib**: Face recognition
|
||||
- **Pillow**: Image processing
|
||||
- **NumPy**: Numerical operations
|
||||
### Memory Issues
|
||||
- Use `--model hog` for faster processing
|
||||
- Process in smaller batches with `--limit 10`
|
||||
- Close other applications to free memory
|
||||
|
||||
### No Faces Found
|
||||
- Check image quality and lighting
|
||||
- Ensure faces are clearly visible
|
||||
- Try `--model cnn` for better detection
|
||||
|
||||
## 🎯 What This Tool Does
|
||||
|
||||
✅ **Simple**: Single Python file, minimal dependencies
|
||||
✅ **Fast**: Efficient face detection and recognition
|
||||
✅ **Private**: Everything runs locally, no cloud services
|
||||
✅ **Flexible**: Batch processing, interactive identification
|
||||
✅ **Lightweight**: No web interface overhead
|
||||
|
||||
## 🚫 What This Tool Doesn't Do
|
||||
|
||||
❌ Web interface (removed for simplicity)
|
||||
❌ Duplicate detection (can be added later)
|
||||
❌ Image editing or enhancement
|
||||
❌ Cloud sync or sharing
|
||||
❌ Complex ML training
|
||||
|
||||
## 📈 Performance Tips
|
||||
|
||||
- **Always use virtual environment** to avoid conflicts
|
||||
- Start with small batches (`--limit 20`) to test
|
||||
- Use `hog` model for speed, `cnn` for accuracy
|
||||
- Process photos in smaller folders first
|
||||
- Identify faces in batches to avoid fatigue
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
1. Read the steering documents in `docs/`
|
||||
2. Follow the code conventions
|
||||
3. Write tests for new features
|
||||
4. Update documentation as needed
|
||||
This is now a minimal, focused tool. Key principles:
|
||||
- Keep it simple and fast
|
||||
- CLI-only interface
|
||||
- Minimal dependencies
|
||||
- Clear, readable code
|
||||
- **Always use python3** commands
|
||||
|
||||
## 📄 License
|
||||
---
|
||||
|
||||
This project is licensed under the MIT License.
|
||||
**Total project size**: ~300 lines of Python code
|
||||
**Dependencies**: 6 essential packages
|
||||
**Setup time**: ~5 minutes
|
||||
**Perfect for**: Batch processing personal photo collections
|
||||
|
||||
## 🆘 Support
|
||||
## 🔄 Common Commands Cheat Sheet
|
||||
|
||||
For issues and questions:
|
||||
```bash
|
||||
# Setup (one time)
|
||||
python3 -m venv venv && source venv/bin/activate && python3 setup.py
|
||||
|
||||
1. Check the steering documents in `docs/`
|
||||
2. Review existing tests in `tests/`
|
||||
3. Check the API standards for endpoint usage
|
||||
# Daily usage - Option 1: Use run script (automatic venv activation)
|
||||
./run.sh scan ~/Pictures --recursive
|
||||
./run.sh process --limit 50
|
||||
./run.sh identify --batch 10
|
||||
./run.sh stats
|
||||
|
||||
# Daily usage - Option 2: Manual venv activation
|
||||
source venv/bin/activate
|
||||
python3 photo_tagger.py scan ~/Pictures --recursive
|
||||
python3 photo_tagger.py process --limit 50
|
||||
python3 photo_tagger.py identify --batch 10
|
||||
python3 photo_tagger.py stats
|
||||
```
|
||||
Reference in New Issue
Block a user