POTE/DEPLOYMENT_AND_AUTOMATION.md

# POTE Deployment & Automation Guide

## 🎯 Quick Answer to Your Questions

### After Deployment, What Happens?

**By default: NOTHING automatic happens.** You need to set up automation.

The deployed system is:
- ✅ Running (database, code installed)
- ✅ Accessible via SSH at your Proxmox IP
- ❌ NOT fetching data automatically
- ❌ NOT sending reports automatically
- ❌ NOT monitoring markets automatically

**You must either:**
1. **Run scripts manually** when you want updates, OR
2. **Set up automation** (5 minutes, see below)

---

## 🚀 Option 1: Automated Email Reports (Recommended)

### What You Get

- **Daily reports at 6 AM** (or your chosen time)
  - New congressional trades from yesterday
  - Market alerts (unusual activity)
  - Suspicious timing detections
  
- **Weekly reports on Sundays**
  - Most active officials & securities
  - Repeat offenders (consistent suspicious timing)
  - Pattern analysis

- **Sent to your email** (Gmail, SendGrid, or custom SMTP)

### 5-Minute Setup

SSH to your deployed POTE server:

```bash
ssh poteapp@your-proxmox-ip
cd ~/pote
```

Run the interactive setup:

```bash
./scripts/setup_cron.sh
```

Follow prompts:
1. Enter your email address
2. Choose report time (default: 6 AM)
3. Done! ✅

**See full guide:** [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md)

---

## 📍 Option 2: Access Reports via IP (No Email)

If you don't want email, you can:

### SSH to Server and View Reports

```bash
ssh poteapp@your-proxmox-ip
cd ~/pote
source venv/bin/activate

# Generate report to stdout
python scripts/send_daily_report.py --to dummy@example.com --save-to-file /tmp/report.txt
cat /tmp/report.txt
```

### Access Saved Reports

If you set up automation (even without email), reports are saved to:

```bash
# SSH to server
ssh poteapp@your-proxmox-ip

# View reports
ls -lh ~/logs/
cat ~/logs/daily_report_$(date +%Y%m%d).txt
cat ~/logs/weekly_report_$(date +%Y%m%d).txt
```

### Run Scripts Manually via SSH

```bash
ssh poteapp@your-proxmox-ip
cd ~/pote
source venv/bin/activate

# Fetch new data
python scripts/fetch_congressional_trades.py
python scripts/monitor_market.py --scan
python scripts/analyze_disclosure_timing.py --recent 7

# View results in database
python scripts/health_check.py
```

---

## 🌐 Option 3: Build a Web Interface (Future)

Currently, POTE is **command-line only**. No web UI yet.

**To add a web interface, you would need to:**
1. Build a FastAPI backend (expose read-only endpoints)
2. Build a React/Streamlit frontend
3. Access via `http://your-proxmox-ip:8000`

This is **not implemented yet**, but it's on the roadmap (Phase 3).

For now, use SSH or email reports.

---

## 🔄 Do You Need CI/CD Pipelines?

### What the Pipeline Does

The included CI/CD pipeline (`.github/workflows/ci.yml`) runs on **every git push**:

1. ✅ Lint & test (93 tests)
2. ✅ Security scanning
3. ✅ Dependency scanning
4. ✅ Docker build test

### Should You Use It?

**YES, if you:**
- Are actively developing POTE
- Want to catch bugs before deployment
- Want automated testing on every commit
- Use GitHub or Gitea for version control

**NO, if you:**
- Just want to use POTE as-is
- Don't plan to modify the code
- Don't have a CI/CD runner set up

### How to Use the Pipeline

The pipeline is **GitHub Actions / Gitea Actions compatible**.

#### For GitHub:
Push your code to GitHub. The workflow runs automatically.

#### For Gitea (Your Setup):
1. Ensure Gitea Actions runner is installed
2. Push to your Gitea repo
3. Workflow runs automatically

#### For Local Testing:
```bash
# Run tests locally
pytest tests/ -v

# Run linters
ruff check src/ tests/
black --check src/ tests/
mypy src/

# Build Docker image
docker build -t pote:test .
```

---

## 📊 Comparison of Options

| Method | Pros | Cons | Best For |
|--------|------|------|----------|
| **Automated Email** | ✅ Convenient<br>✅ No SSH needed<br>✅ Daily/weekly updates | ❌ Requires SMTP setup | Most users |
| **SSH + Manual Scripts** | ✅ Full control<br>✅ No email needed | ❌ Manual work<br>❌ Must remember to run | Power users |
| **Saved Reports (SSH access)** | ✅ Automated<br>✅ No email | ❌ Must SSH to view | Users without email |
| **Web Interface** | ✅ User-friendly | ❌ Not implemented yet | Future |

---

## 🛠️ Your Ansible Pipeline

Your existing Ansible CI/CD pipeline is **NOT directly usable** for POTE because:

1. **Language mismatch**: Your pipeline is for Node.js/Ansible; POTE is Python
2. **Different tooling**: POTE uses pytest, ruff, mypy (not npm, ansible-lint)
3. **Different structure**: POTE uses different security scanners

### What You CAN Reuse

**Concepts from your pipeline that ARE used in POTE's CI/CD:**

- ✅ Security scanning (Trivy, Bandit instead of Gitleaks)
- ✅ Dependency scanning (Trivy instead of npm audit)
- ✅ SAST scanning (Bandit instead of Semgrep)
- ✅ Container scanning (Docker build test)
- ✅ Workflow summary generation

**The POTE pipeline (`.github/workflows/ci.yml`) already includes all of these!**

### If You Want to Integrate with Your Gitea

Your Gitea server can run the POTE pipeline using Gitea Actions:

1. Push POTE to your Gitea
2. Gitea Actions will detect `.github/workflows/ci.yml`
3. Workflow runs automatically (if runner is configured)

**No modifications needed** - it's already compatible!

---

## 📧 Email Setup Examples

### Gmail (Most Common)

In your deployed `.env` file:

```env
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASSWORD=your-16-char-app-password
FROM_EMAIL=pote-reports@gmail.com
REPORT_RECIPIENTS=your-email@example.com
```

**Important**: Use an [App Password](https://myaccount.google.com/apppasswords), NOT your regular password!

### SendGrid

```env
SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_USER=apikey
SMTP_PASSWORD=SG.your-api-key-here
FROM_EMAIL=noreply@yourdomain.com
REPORT_RECIPIENTS=admin@yourdomain.com
```

### Custom Mail Server

```env
SMTP_HOST=mail.yourdomain.com
SMTP_PORT=587
SMTP_USER=username
SMTP_PASSWORD=password
FROM_EMAIL=pote@yourdomain.com
REPORT_RECIPIENTS=admin@yourdomain.com
```

---

## ✅ Recommended Setup for Most Users

1. **Deploy to Proxmox** (5 min)
   ```bash
   bash scripts/proxmox_setup.sh
   ```

2. **Set up automated email reports** (5 min)
   ```bash
   ssh poteapp@your-ip
   cd ~/pote
   ./scripts/setup_cron.sh
   ```

3. **Done!** You'll receive:
   - Daily reports at 6 AM
   - Weekly reports on Sundays
   - All reports also saved to `~/logs/`

4. **(Optional)** Set up CI/CD if you plan to develop:
   - Push to GitHub/Gitea
   - Pipeline runs automatically

---

## 🔍 Monitoring & Health Checks

### Add System Health Monitoring

Edit crontab:

```bash
crontab -e
```

Add health check (every 6 hours):

```cron
0 */6 * * * /home/poteapp/pote/venv/bin/python /home/poteapp/pote/scripts/health_check.py >> /home/poteapp/logs/health.log 2>&1
```

Check health anytime:

```bash
ssh poteapp@your-ip
cd ~/pote
source venv/bin/activate
python scripts/health_check.py
```

---

## 📚 Full Documentation

- **Automation Setup**: [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md) ⭐
- **Deployment**: [`PROXMOX_QUICKSTART.md`](PROXMOX_QUICKSTART.md) ⭐
- **Usage**: [`QUICKSTART.md`](QUICKSTART.md) ⭐
- **Detailed Automation Guide**: [`docs/12_automation_and_reporting.md`](docs/12_automation_and_reporting.md)
- **Monitoring System**: [`MONITORING_SYSTEM_COMPLETE.md`](MONITORING_SYSTEM_COMPLETE.md)

---

## 🎉 Summary

**After deployment:**
- Reports are NOT sent automatically by default
- You must set up automation OR run scripts manually
- **Recommended: 5-minute automation setup** with `./scripts/setup_cron.sh`
- Reports can be emailed OR saved to `~/logs/` for SSH access
- CI/CD pipeline is included and ready for GitHub/Gitea
- Your Ansible pipeline is not directly usable, but concepts are already implemented

**Most users should:**
1. Deploy to Proxmox
2. Run `./scripts/setup_cron.sh`
3. Receive daily/weekly email reports
4. Done! 🚀