POTE/DEPLOYMENT_AND_AUTOMATION.md

POTE Deployment & Automation Guide

🎯 Quick Answer to Your Questions

After Deployment, What Happens?

By default: NOTHING automatic happens. You need to set up automation.

The deployed system is:

• ✅ Running (database, code installed)
• ✅ Accessible via SSH at your Proxmox IP
• ❌ NOT fetching data automatically
• ❌ NOT sending reports automatically
• ❌ NOT monitoring markets automatically

You must either:

1. Run scripts manually when you want updates, OR
2. Set up automation (5 minutes, see below)

---

🚀 Option 1: Automated Email Reports (Recommended)

What You Get

• Daily reports at 6 AM (or your chosen time)

  • New congressional trades from yesterday
  • Market alerts (unusual activity)
  • Suspicious timing detections

• Weekly reports on Sundays

  • Most active officials & securities
  • Repeat offenders (consistent suspicious timing)
  • Pattern analysis

• Sent to your email (Gmail, SendGrid, or custom SMTP)

5-Minute Setup

SSH to your deployed POTE server:

ssh poteapp@your-proxmox-ip
cd ~/pote

Run the interactive setup:

./scripts/setup_cron.sh

Follow prompts:

1. Enter your email address
2. Choose report time (default: 6 AM)
3. Done! ✅

See full guide: AUTOMATION_QUICKSTART.md

---

📍 Option 2: Access Reports via IP (No Email)

If you don't want email, you can:

SSH to Server and View Reports

ssh poteapp@your-proxmox-ip
cd ~/pote
source venv/bin/activate

# Generate report to stdout
python scripts/send_daily_report.py --to dummy@example.com --save-to-file /tmp/report.txt
cat /tmp/report.txt

Access Saved Reports

If you set up automation (even without email), reports are saved to:

# SSH to server
ssh poteapp@your-proxmox-ip

# View reports
ls -lh ~/logs/
cat ~/logs/daily_report_$(date +%Y%m%d).txt
cat ~/logs/weekly_report_$(date +%Y%m%d).txt

Run Scripts Manually via SSH

ssh poteapp@your-proxmox-ip
cd ~/pote
source venv/bin/activate

# Fetch new data
python scripts/fetch_congressional_trades.py
python scripts/monitor_market.py --scan
python scripts/analyze_disclosure_timing.py --recent 7

# View results in database
python scripts/health_check.py

---

🌐 Option 3: Build a Web Interface (Future)

Currently, POTE is command-line only. No web UI yet.

To add a web interface, you would need to:

1. Build a FastAPI backend (expose read-only endpoints)
2. Build a React/Streamlit frontend
3. Access via http://your-proxmox-ip:8000

This is not implemented yet, but it's on the roadmap (Phase 3).

For now, use SSH or email reports.

---

🔄 Do You Need CI/CD Pipelines?

What the Pipeline Does

The included CI/CD pipeline (.github/workflows/ci.yml) runs on every git push:

1. ✅ Lint & test (93 tests)
2. ✅ Security scanning
3. ✅ Dependency scanning
4. ✅ Docker build test

Should You Use It?

YES, if you:

• Are actively developing POTE
• Want to catch bugs before deployment
• Want automated testing on every commit
• Use GitHub or Gitea for version control

NO, if you:

• Just want to use POTE as-is
• Don't plan to modify the code
• Don't have a CI/CD runner set up

How to Use the Pipeline

The pipeline is GitHub Actions / Gitea Actions compatible.

For GitHub:

Push your code to GitHub. The workflow runs automatically.

For Gitea (Your Setup):

1. Ensure Gitea Actions runner is installed
2. Push to your Gitea repo
3. Workflow runs automatically

For Local Testing:

# Run tests locally
pytest tests/ -v

# Run linters
ruff check src/ tests/
black --check src/ tests/
mypy src/

# Build Docker image
docker build -t pote:test .

---

📊 Comparison of Options

Method	Pros	Cons	Best For
Automated Email	✅ Convenient ✅ No SSH needed ✅ Daily/weekly updates	❌ Requires SMTP setup	Most users
SSH + Manual Scripts	✅ Full control ✅ No email needed	❌ Manual work ❌ Must remember to run	Power users
Saved Reports (SSH access)	✅ Automated ✅ No email	❌ Must SSH to view	Users without email
Web Interface	✅ User-friendly	❌ Not implemented yet	Future

---

🛠️ Your Ansible Pipeline

Your existing Ansible CI/CD pipeline is NOT directly usable for POTE because:

1. Language mismatch: Your pipeline is for Node.js/Ansible; POTE is Python
2. Different tooling: POTE uses pytest, ruff, mypy (not npm, ansible-lint)
3. Different structure: POTE uses different security scanners

What You CAN Reuse

Concepts from your pipeline that ARE used in POTE's CI/CD:

• ✅ Security scanning (Trivy, Bandit instead of Gitleaks)
• ✅ Dependency scanning (Trivy instead of npm audit)
• ✅ SAST scanning (Bandit instead of Semgrep)
• ✅ Container scanning (Docker build test)
• ✅ Workflow summary generation

The POTE pipeline (.github/workflows/ci.yml) already includes all of these!

If You Want to Integrate with Your Gitea

Your Gitea server can run the POTE pipeline using Gitea Actions:

1. Push POTE to your Gitea
2. Gitea Actions will detect .github/workflows/ci.yml
3. Workflow runs automatically (if runner is configured)

No modifications needed - it's already compatible!

---

📧 Email Setup Examples

Gmail (Most Common)

In your deployed .env file:

SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASSWORD=your-16-char-app-password
FROM_EMAIL=pote-reports@gmail.com
REPORT_RECIPIENTS=your-email@example.com

Important: Use an App Password, NOT your regular password!

SendGrid

SMTP_HOST=smtp.sendgrid.net
SMTP_PORT=587
SMTP_USER=apikey
SMTP_PASSWORD=SG.your-api-key-here
FROM_EMAIL=noreply@yourdomain.com
REPORT_RECIPIENTS=admin@yourdomain.com

Custom Mail Server

SMTP_HOST=mail.yourdomain.com
SMTP_PORT=587
SMTP_USER=username
SMTP_PASSWORD=password
FROM_EMAIL=pote@yourdomain.com
REPORT_RECIPIENTS=admin@yourdomain.com

---

✅ Recommended Setup for Most Users

1. Deploy to Proxmox (5 min)

   bash scripts/proxmox_setup.sh
   

2. Set up automated email reports (5 min)

   ssh poteapp@your-ip
   cd ~/pote
   ./scripts/setup_cron.sh
   

3. Done! You'll receive:

   • Daily reports at 6 AM
   • Weekly reports on Sundays
   • All reports also saved to ~/logs/

4. (Optional) Set up CI/CD if you plan to develop:

   • Push to GitHub/Gitea
   • Pipeline runs automatically

---

🔍 Monitoring & Health Checks

Add System Health Monitoring

Edit crontab:

crontab -e

Add health check (every 6 hours):

0 */6 * * * /home/poteapp/pote/venv/bin/python /home/poteapp/pote/scripts/health_check.py >> /home/poteapp/logs/health.log 2>&1

Check health anytime:

ssh poteapp@your-ip
cd ~/pote
source venv/bin/activate
python scripts/health_check.py

---

📚 Full Documentation

• Automation Setup: AUTOMATION_QUICKSTART.md ⭐
• Deployment: PROXMOX_QUICKSTART.md ⭐
• Usage: QUICKSTART.md ⭐
• Detailed Automation Guide: docs/12_automation_and_reporting.md
• Monitoring System: MONITORING_SYSTEM_COMPLETE.md

---

🎉 Summary

After deployment:

• Reports are NOT sent automatically by default
• You must set up automation OR run scripts manually
• Recommended: 5-minute automation setup with ./scripts/setup_cron.sh
• Reports can be emailed OR saved to ~/logs/ for SSH access
• CI/CD pipeline is included and ready for GitHub/Gitea
• Your Ansible pipeline is not directly usable, but concepts are already implemented

Most users should:

1. Deploy to Proxmox
2. Run ./scripts/setup_cron.sh
3. Receive daily/weekly email reports
4. Done! 🚀