Add comprehensive deployment and automation FAQ
Answers user's questions: - What happens after deployment? (nothing automatic by default) - How to get reports? (3 options: email, SSH, future web UI) - Where are reports sent? (email or saved to ~/logs/) - Do you need to check IP? (depends on setup method) - Can we setup email reports? (YES! 5-minute setup) - Do we need CI/CD pipelines? (optional, but included) - Can we use existing Ansible pipeline? (concepts reused, not directly) This document ties everything together and provides clear next steps.
This commit is contained in:
ilia
parent
0d8d85adc1
commit
d8f723bafb
338
DEPLOYMENT_AND_AUTOMATION.md
Normal file
338
DEPLOYMENT_AND_AUTOMATION.md
Normal file
@ -0,0 +1,338 @@
|
||||
# POTE Deployment & Automation Guide
|
||||
|
||||
## 🎯 Quick Answer to Your Questions
|
||||
|
||||
### After Deployment, What Happens?
|
||||
|
||||
**By default: NOTHING automatic happens.** You need to set up automation.
|
||||
|
||||
The deployed system is:
|
||||
- ✅ Running (database, code installed)
|
||||
- ✅ Accessible via SSH at your Proxmox IP
|
||||
- ❌ NOT fetching data automatically
|
||||
- ❌ NOT sending reports automatically
|
||||
- ❌ NOT monitoring markets automatically
|
||||
|
||||
**You must either:**
|
||||
1. **Run scripts manually** when you want updates, OR
|
||||
2. **Set up automation** (5 minutes, see below)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Option 1: Automated Email Reports (Recommended)
|
||||
|
||||
### What You Get
|
||||
|
||||
- **Daily reports at 6 AM** (or your chosen time)
|
||||
- New congressional trades from yesterday
|
||||
- Market alerts (unusual activity)
|
||||
- Suspicious timing detections
|
||||
|
||||
- **Weekly reports on Sundays**
|
||||
- Most active officials & securities
|
||||
- Repeat offenders (consistent suspicious timing)
|
||||
- Pattern analysis
|
||||
|
||||
- **Sent to your email** (Gmail, SendGrid, or custom SMTP)
|
||||
|
||||
### 5-Minute Setup
|
||||
|
||||
SSH to your deployed POTE server:
|
||||
|
||||
```bash
|
||||
ssh poteapp@your-proxmox-ip
|
||||
cd ~/pote
|
||||
```
|
||||
|
||||
Run the interactive setup:
|
||||
|
||||
```bash
|
||||
./scripts/setup_cron.sh
|
||||
```
|
||||
|
||||
Follow prompts:
|
||||
1. Enter your email address
|
||||
2. Choose report time (default: 6 AM)
|
||||
3. Done! ✅
|
||||
|
||||
**See full guide:** [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md)
|
||||
|
||||
---
|
||||
|
||||
## 📍 Option 2: Access Reports via IP (No Email)
|
||||
|
||||
If you don't want email, you can:
|
||||
|
||||
### SSH to Server and View Reports
|
||||
|
||||
```bash
|
||||
ssh poteapp@your-proxmox-ip
|
||||
cd ~/pote
|
||||
source venv/bin/activate
|
||||
|
||||
# Generate report to stdout
|
||||
python scripts/send_daily_report.py --to dummy@example.com --save-to-file /tmp/report.txt
|
||||
cat /tmp/report.txt
|
||||
```
|
||||
|
||||
### Access Saved Reports
|
||||
|
||||
If you set up automation (even without email), reports are saved to:
|
||||
|
||||
```bash
|
||||
# SSH to server
|
||||
ssh poteapp@your-proxmox-ip
|
||||
|
||||
# View reports
|
||||
ls -lh ~/logs/
|
||||
cat ~/logs/daily_report_$(date +%Y%m%d).txt
|
||||
cat ~/logs/weekly_report_$(date +%Y%m%d).txt
|
||||
```
|
||||
|
||||
### Run Scripts Manually via SSH
|
||||
|
||||
```bash
|
||||
ssh poteapp@your-proxmox-ip
|
||||
cd ~/pote
|
||||
source venv/bin/activate
|
||||
|
||||
# Fetch new data
|
||||
python scripts/fetch_congressional_trades.py
|
||||
python scripts/monitor_market.py --scan
|
||||
python scripts/analyze_disclosure_timing.py --recent 7
|
||||
|
||||
# View results in database
|
||||
python scripts/health_check.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌐 Option 3: Build a Web Interface (Future)
|
||||
|
||||
Currently, POTE is **command-line only**. No web UI yet.
|
||||
|
||||
**To add a web interface, you would need to:**
|
||||
1. Build a FastAPI backend (expose read-only endpoints)
|
||||
2. Build a React/Streamlit frontend
|
||||
3. Access via `http://your-proxmox-ip:8000`
|
||||
|
||||
This is **not implemented yet**, but it's on the roadmap (Phase 3).
|
||||
|
||||
For now, use SSH or email reports.
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Do You Need CI/CD Pipelines?
|
||||
|
||||
### What the Pipeline Does
|
||||
|
||||
The included CI/CD pipeline (`.github/workflows/ci.yml`) runs on **every git push**:
|
||||
|
||||
1. ✅ Lint & test (93 tests)
|
||||
2. ✅ Security scanning
|
||||
3. ✅ Dependency scanning
|
||||
4. ✅ Docker build test
|
||||
|
||||
### Should You Use It?
|
||||
|
||||
**YES, if you:**
|
||||
- Are actively developing POTE
|
||||
- Want to catch bugs before deployment
|
||||
- Want automated testing on every commit
|
||||
- Use GitHub or Gitea for version control
|
||||
|
||||
**NO, if you:**
|
||||
- Just want to use POTE as-is
|
||||
- Don't plan to modify the code
|
||||
- Don't have a CI/CD runner set up
|
||||
|
||||
### How to Use the Pipeline
|
||||
|
||||
The pipeline is **GitHub Actions / Gitea Actions compatible**.
|
||||
|
||||
#### For GitHub:
|
||||
Push your code to GitHub. The workflow runs automatically.
|
||||
|
||||
#### For Gitea (Your Setup):
|
||||
1. Ensure Gitea Actions runner is installed
|
||||
2. Push to your Gitea repo
|
||||
3. Workflow runs automatically
|
||||
|
||||
#### For Local Testing:
|
||||
```bash
|
||||
# Run tests locally
|
||||
pytest tests/ -v
|
||||
|
||||
# Run linters
|
||||
ruff check src/ tests/
|
||||
black --check src/ tests/
|
||||
mypy src/
|
||||
|
||||
# Build Docker image
|
||||
docker build -t pote:test .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Comparison of Options
|
||||
|
||||
| Method | Pros | Cons | Best For |
|
||||
|--------|------|------|----------|
|
||||
| **Automated Email** | ✅ Convenient<br>✅ No SSH needed<br>✅ Daily/weekly updates | ❌ Requires SMTP setup | Most users |
|
||||
| **SSH + Manual Scripts** | ✅ Full control<br>✅ No email needed | ❌ Manual work<br>❌ Must remember to run | Power users |
|
||||
| **Saved Reports (SSH access)** | ✅ Automated<br>✅ No email | ❌ Must SSH to view | Users without email |
|
||||
| **Web Interface** | ✅ User-friendly | ❌ Not implemented yet | Future |
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Your Ansible Pipeline
|
||||
|
||||
Your existing Ansible CI/CD pipeline is **NOT directly usable** for POTE because:
|
||||
|
||||
1. **Language mismatch**: Your pipeline is for Node.js/Ansible; POTE is Python
|
||||
2. **Different tooling**: POTE uses pytest, ruff, mypy (not npm, ansible-lint)
|
||||
3. **Different structure**: POTE uses different security scanners
|
||||
|
||||
### What You CAN Reuse
|
||||
|
||||
**Concepts from your pipeline that ARE used in POTE's CI/CD:**
|
||||
|
||||
- ✅ Security scanning (Trivy, Bandit instead of Gitleaks)
|
||||
- ✅ Dependency scanning (Trivy instead of npm audit)
|
||||
- ✅ SAST scanning (Bandit instead of Semgrep)
|
||||
- ✅ Container scanning (Docker build test)
|
||||
- ✅ Workflow summary generation
|
||||
|
||||
**The POTE pipeline (`.github/workflows/ci.yml`) already includes all of these!**
|
||||
|
||||
### If You Want to Integrate with Your Gitea
|
||||
|
||||
Your Gitea server can run the POTE pipeline using Gitea Actions:
|
||||
|
||||
1. Push POTE to your Gitea
|
||||
2. Gitea Actions will detect `.github/workflows/ci.yml`
|
||||
3. Workflow runs automatically (if runner is configured)
|
||||
|
||||
**No modifications needed** - it's already compatible!
|
||||
|
||||
---
|
||||
|
||||
## 📧 Email Setup Examples
|
||||
|
||||
### Gmail (Most Common)
|
||||
|
||||
In your deployed `.env` file:
|
||||
|
||||
```env
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASSWORD=your-16-char-app-password
|
||||
FROM_EMAIL=pote-reports@gmail.com
|
||||
REPORT_RECIPIENTS=your-email@example.com
|
||||
```
|
||||
|
||||
**Important**: Use an [App Password](https://myaccount.google.com/apppasswords), NOT your regular password!
|
||||
|
||||
### SendGrid
|
||||
|
||||
```env
|
||||
SMTP_HOST=smtp.sendgrid.net
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=apikey
|
||||
SMTP_PASSWORD=SG.your-api-key-here
|
||||
FROM_EMAIL=noreply@yourdomain.com
|
||||
REPORT_RECIPIENTS=admin@yourdomain.com
|
||||
```
|
||||
|
||||
### Custom Mail Server
|
||||
|
||||
```env
|
||||
SMTP_HOST=mail.yourdomain.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=username
|
||||
SMTP_PASSWORD=password
|
||||
FROM_EMAIL=pote@yourdomain.com
|
||||
REPORT_RECIPIENTS=admin@yourdomain.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ Recommended Setup for Most Users
|
||||
|
||||
1. **Deploy to Proxmox** (5 min)
|
||||
```bash
|
||||
bash scripts/proxmox_setup.sh
|
||||
```
|
||||
|
||||
2. **Set up automated email reports** (5 min)
|
||||
```bash
|
||||
ssh poteapp@your-ip
|
||||
cd ~/pote
|
||||
./scripts/setup_cron.sh
|
||||
```
|
||||
|
||||
3. **Done!** You'll receive:
|
||||
- Daily reports at 6 AM
|
||||
- Weekly reports on Sundays
|
||||
- All reports also saved to `~/logs/`
|
||||
|
||||
4. **(Optional)** Set up CI/CD if you plan to develop:
|
||||
- Push to GitHub/Gitea
|
||||
- Pipeline runs automatically
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Monitoring & Health Checks
|
||||
|
||||
### Add System Health Monitoring
|
||||
|
||||
Edit crontab:
|
||||
|
||||
```bash
|
||||
crontab -e
|
||||
```
|
||||
|
||||
Add health check (every 6 hours):
|
||||
|
||||
```cron
|
||||
0 */6 * * * /home/poteapp/pote/venv/bin/python /home/poteapp/pote/scripts/health_check.py >> /home/poteapp/logs/health.log 2>&1
|
||||
```
|
||||
|
||||
Check health anytime:
|
||||
|
||||
```bash
|
||||
ssh poteapp@your-ip
|
||||
cd ~/pote
|
||||
source venv/bin/activate
|
||||
python scripts/health_check.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📚 Full Documentation
|
||||
|
||||
- **Automation Setup**: [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md) ⭐
|
||||
- **Deployment**: [`PROXMOX_QUICKSTART.md`](PROXMOX_QUICKSTART.md) ⭐
|
||||
- **Usage**: [`QUICKSTART.md`](QUICKSTART.md) ⭐
|
||||
- **Detailed Automation Guide**: [`docs/12_automation_and_reporting.md`](docs/12_automation_and_reporting.md)
|
||||
- **Monitoring System**: [`MONITORING_SYSTEM_COMPLETE.md`](MONITORING_SYSTEM_COMPLETE.md)
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Summary
|
||||
|
||||
**After deployment:**
|
||||
- Reports are NOT sent automatically by default
|
||||
- You must set up automation OR run scripts manually
|
||||
- **Recommended: 5-minute automation setup** with `./scripts/setup_cron.sh`
|
||||
- Reports can be emailed OR saved to `~/logs/` for SSH access
|
||||
- CI/CD pipeline is included and ready for GitHub/Gitea
|
||||
- Your Ansible pipeline is not directly usable, but concepts are already implemented
|
||||
|
||||
**Most users should:**
|
||||
1. Deploy to Proxmox
|
||||
2. Run `./scripts/setup_cron.sh`
|
||||
3. Receive daily/weekly email reports
|
||||
4. Done! 🚀
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user