Add documentation for Ansible separation and customization
NEW FILES: ========== 📄 CUSTOMIZATION_CHECKLIST.md - Complete list of everything that needs customization - Organized by priority: Critical, Important, Optional - Covers .env, Ansible, Gitea secrets, email, etc. - Quick action checklist for deployment 📄 ANSIBLE_HANDOFF.md - Guide for integrating POTE with existing Ansible system - Explains what Ansible needs to know - Variable reference and secrets management - Multi-environment deployment strategy - Example playbook and testing instructions 📄 MOVE_ANSIBLE_TO_SEPARATE_REPO.md - Explains why ansible/ should be in infrastructure repo - Step-by-step migration guide - Final directory structure for both repos - Benefits and workflow after migration KEY INSIGHT: ============ The ansible/ directory doesn't belong in the POTE app repo because: - Ansible runs BEFORE the app exists (creates container, deploys app) - Creates circular dependency (Ansible clones repo that contains Ansible) - Should live in centralized infrastructure repository NEXT STEPS: =========== 1. Review CUSTOMIZATION_CHECKLIST.md for deployment config 2. Copy ansible/ to infrastructure repo 3. Remove ansible/ from POTE repo (keep handoff docs) 4. Deploy via centralized Ansible system
This commit is contained in:
parent
d2ae095fcf
commit
e9fd12d949
376
ANSIBLE_HANDOFF.md
Normal file
376
ANSIBLE_HANDOFF.md
Normal file
@ -0,0 +1,376 @@
|
|||||||
|
# POTE Ansible Integration - Handoff Guide
|
||||||
|
|
||||||
|
## 🎯 Purpose
|
||||||
|
|
||||||
|
This document explains what your Ansible auto-configure system needs to know to deploy POTE across different environments.
|
||||||
|
|
||||||
|
**Note:** The Ansible code in `ansible/` directory should be **moved to your Ansible repository**, not kept here in the POTE app repo.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📦 What to Move to Your Ansible Repository
|
||||||
|
|
||||||
|
### Files to Extract from POTE Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Copy these to your Ansible repository:
|
||||||
|
ansible/
|
||||||
|
├── README.md → roles/pote/README.md
|
||||||
|
├── roles/pote/defaults/main.yml → roles/pote/defaults/main.yml
|
||||||
|
├── group_vars/
|
||||||
|
│ ├── all.yml → group_vars/all.yml (merge with existing)
|
||||||
|
│ ├── development.yml → group_vars/development.yml (merge)
|
||||||
|
│ ├── staging.yml → group_vars/staging.yml (merge)
|
||||||
|
│ └── production.yml → group_vars/production.yml (merge)
|
||||||
|
├── inventory.example.yml → inventory.example.yml (reference)
|
||||||
|
└── vault.example.yml → vault.example.yml (reference)
|
||||||
|
```
|
||||||
|
|
||||||
|
### What to Keep in POTE Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Keep these for deployment reference:
|
||||||
|
docs/08_proxmox_deployment.md # Manual deployment guide
|
||||||
|
scripts/proxmox_setup.sh # Bootstrap script (if needed)
|
||||||
|
CUSTOMIZATION_CHECKLIST.md # Configuration reference
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔧 Integration with Your Ansible System
|
||||||
|
|
||||||
|
### 1. Create POTE Role in Your Ansible Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In your Ansible repository:
|
||||||
|
roles/
|
||||||
|
└── pote/
|
||||||
|
├── defaults/
|
||||||
|
│ └── main.yml # Copy from POTE repo
|
||||||
|
├── tasks/
|
||||||
|
│ └── main.yml # Create based on requirements below
|
||||||
|
├── templates/
|
||||||
|
│ └── env.j2 # Create for .env file
|
||||||
|
├── handlers/
|
||||||
|
│ └── main.yml # Restart services, etc.
|
||||||
|
└── README.md # Copy from POTE repo
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Required Ansible Tasks
|
||||||
|
|
||||||
|
Your `roles/pote/tasks/main.yml` should handle:
|
||||||
|
|
||||||
|
#### Phase 1: System Preparation
|
||||||
|
- Create application user (`poteapp`)
|
||||||
|
- Install system dependencies (Python 3.11, PostgreSQL, build-essential, etc.)
|
||||||
|
- Configure PostgreSQL database and user
|
||||||
|
|
||||||
|
#### Phase 2: Application Deployment
|
||||||
|
- Clone POTE repository from Git (using SSH key)
|
||||||
|
- Create Python virtual environment
|
||||||
|
- Install Python dependencies
|
||||||
|
- Create `.env` file from template
|
||||||
|
- Run database migrations (`alembic upgrade head`)
|
||||||
|
|
||||||
|
#### Phase 3: Automation Setup
|
||||||
|
- Create log directories
|
||||||
|
- Install cron jobs for daily/weekly reports
|
||||||
|
- Set up health check monitoring
|
||||||
|
- Configure email reporting
|
||||||
|
|
||||||
|
#### Phase 4: Security & Cleanup
|
||||||
|
- Set proper file permissions
|
||||||
|
- Secure `.env` file (600)
|
||||||
|
- Clean up temporary files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Variables Your Ansible System Needs
|
||||||
|
|
||||||
|
### Minimal Required Variables (3)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Absolute minimum to deploy POTE:
|
||||||
|
pote_git_repo: "gitea@10.0.30.169:ilia/POTE.git"
|
||||||
|
pote_git_branch: "main" # or "dev", "qa" per environment
|
||||||
|
vault_smtp_password: "{{ vault_smtp_password }}"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Recommended Variables (Environment-Specific)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Per-environment configuration:
|
||||||
|
environment: "production" # or "development", "staging"
|
||||||
|
pote_git_branch: "main" # "dev" for development, "qa" for staging
|
||||||
|
|
||||||
|
# Database
|
||||||
|
db_name: "potedb" # "potedb_dev" for dev, "potedb_qa" for staging
|
||||||
|
db_user: "poteuser"
|
||||||
|
db_password: "{{ vault_db_password_prod }}"
|
||||||
|
|
||||||
|
# Email
|
||||||
|
smtp_host: "mail.levkin.ca"
|
||||||
|
smtp_port: 587
|
||||||
|
smtp_user: "test@levkin.ca"
|
||||||
|
from_email: "test@levkin.ca"
|
||||||
|
smtp_password: "{{ vault_smtp_password }}"
|
||||||
|
report_recipients: "ilia@levkin.ca" # Your actual email
|
||||||
|
|
||||||
|
# Monitoring
|
||||||
|
market_tickers: "NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX"
|
||||||
|
alert_severity: 5
|
||||||
|
|
||||||
|
# Automation
|
||||||
|
pote_enable_cron: true
|
||||||
|
pote_daily_report_time: "0 6" # 6:00 AM
|
||||||
|
pote_weekly_report_time: "0 8" # 8:00 AM Sunday
|
||||||
|
```
|
||||||
|
|
||||||
|
### Full Variable List
|
||||||
|
|
||||||
|
See `ansible/roles/pote/defaults/main.yml` for all 200+ variables with defaults.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔐 Secrets Management
|
||||||
|
|
||||||
|
### Required Secrets in Ansible Vault
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# In your Ansible vault (group_vars/all/vault.yml):
|
||||||
|
vault_git_ssh_key: |
|
||||||
|
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||||
|
your_ssh_private_key_for_gitea
|
||||||
|
-----END OPENSSH PRIVATE KEY-----
|
||||||
|
|
||||||
|
vault_ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2E..."
|
||||||
|
|
||||||
|
vault_smtp_password: "your_email_password"
|
||||||
|
|
||||||
|
vault_db_password_dev: "dev_password_123"
|
||||||
|
vault_db_password_qa: "qa_password_123"
|
||||||
|
vault_db_password_prod: "strong_production_password"
|
||||||
|
|
||||||
|
# Optional:
|
||||||
|
vault_quiverquant_key: ""
|
||||||
|
vault_fmp_key: ""
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎭 Multi-Environment Strategy
|
||||||
|
|
||||||
|
### Branch → Environment Mapping
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
development:
|
||||||
|
git_branch: dev
|
||||||
|
db_name: potedb_dev
|
||||||
|
log_level: DEBUG
|
||||||
|
cron_enabled: false # Manual testing
|
||||||
|
|
||||||
|
staging:
|
||||||
|
git_branch: qa
|
||||||
|
db_name: potedb_qa
|
||||||
|
log_level: INFO
|
||||||
|
cron_enabled: true
|
||||||
|
|
||||||
|
production:
|
||||||
|
git_branch: main
|
||||||
|
db_name: potedb
|
||||||
|
log_level: WARNING
|
||||||
|
cron_enabled: true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deployment Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Deploy to development
|
||||||
|
ansible-playbook deploy-pote.yml --limit development
|
||||||
|
|
||||||
|
# Deploy to staging
|
||||||
|
ansible-playbook deploy-pote.yml --limit staging
|
||||||
|
|
||||||
|
# Deploy to production
|
||||||
|
ansible-playbook deploy-pote.yml --limit production
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 Example Playbook
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
# deploy-pote.yml
|
||||||
|
- name: Deploy POTE Application
|
||||||
|
hosts: all
|
||||||
|
become: yes
|
||||||
|
|
||||||
|
pre_tasks:
|
||||||
|
- name: Ensure base OS role has run
|
||||||
|
debug:
|
||||||
|
msg: "Assuming base_os role has already configured the system"
|
||||||
|
|
||||||
|
roles:
|
||||||
|
- role: pote
|
||||||
|
tags: ['pote', 'deploy']
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 Deployment Workflow
|
||||||
|
|
||||||
|
### Initial Deployment (New Server)
|
||||||
|
|
||||||
|
1. **Provision LXC container** in Proxmox
|
||||||
|
2. **Run base_os role** (your existing role)
|
||||||
|
3. **Run pote role** (new role)
|
||||||
|
4. **Verify deployment** with health check
|
||||||
|
|
||||||
|
### Updates (Existing Server)
|
||||||
|
|
||||||
|
1. **Pull latest code** from Git
|
||||||
|
2. **Update dependencies** if needed
|
||||||
|
3. **Run migrations** (`alembic upgrade head`)
|
||||||
|
4. **Restart services** if needed
|
||||||
|
|
||||||
|
### Rollback (If Needed)
|
||||||
|
|
||||||
|
1. **Checkout previous Git tag/commit**
|
||||||
|
2. **Run migrations down** if needed
|
||||||
|
3. **Restore database backup** if needed
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🧪 Testing the Integration
|
||||||
|
|
||||||
|
### Dry Run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Test without making changes
|
||||||
|
ansible-playbook deploy-pote.yml --limit development --check --diff
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verify Deployment
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# SSH into server and run:
|
||||||
|
cd ~/pote
|
||||||
|
source venv/bin/activate
|
||||||
|
python scripts/health_check.py
|
||||||
|
|
||||||
|
# Check cron jobs:
|
||||||
|
crontab -l | grep POTE
|
||||||
|
|
||||||
|
# Check logs:
|
||||||
|
tail -f ~/logs/daily_run.log
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📊 What POTE Provides for Ansible
|
||||||
|
|
||||||
|
### 1. Configuration Template
|
||||||
|
|
||||||
|
The `.env` file template your Ansible should generate:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Database
|
||||||
|
DATABASE_URL=postgresql://{{ db_user }}:{{ db_password }}@localhost:5432/{{ db_name }}
|
||||||
|
|
||||||
|
# Email
|
||||||
|
SMTP_HOST={{ smtp_host }}
|
||||||
|
SMTP_PORT={{ smtp_port }}
|
||||||
|
SMTP_USER={{ smtp_user }}
|
||||||
|
SMTP_PASSWORD={{ smtp_password }}
|
||||||
|
FROM_EMAIL={{ from_email }}
|
||||||
|
REPORT_RECIPIENTS={{ report_recipients }}
|
||||||
|
|
||||||
|
# Monitoring
|
||||||
|
MARKET_MONITOR_TICKERS={{ market_tickers }}
|
||||||
|
ALERT_MIN_SEVERITY={{ alert_severity }}
|
||||||
|
|
||||||
|
# Logging
|
||||||
|
LOG_LEVEL={{ log_level }}
|
||||||
|
LOG_FILE={{ log_file }}
|
||||||
|
|
||||||
|
# Optional API Keys
|
||||||
|
QUIVERQUANT_API_KEY={{ quiverquant_api_key | default('') }}
|
||||||
|
FMP_API_KEY={{ fmp_api_key | default('') }}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Health Check Endpoint
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Your Ansible can verify deployment with:
|
||||||
|
python scripts/health_check.py
|
||||||
|
# Exit code 0 = healthy, non-zero = issues
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Database Migrations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Your Ansible should run after code updates:
|
||||||
|
alembic upgrade head
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Idempotency
|
||||||
|
|
||||||
|
All POTE scripts are idempotent:
|
||||||
|
- Re-running data fetches won't duplicate records
|
||||||
|
- Migrations are safe to re-run
|
||||||
|
- Cron setup checks for existing jobs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Quick Start for Your Ansible Team
|
||||||
|
|
||||||
|
1. **Copy** `ansible/` directory from POTE repo to your Ansible repo
|
||||||
|
2. **Customize** `group_vars/` files with your values
|
||||||
|
3. **Create** `group_vars/all/vault.yml` with secrets
|
||||||
|
4. **Encrypt** vault: `ansible-vault encrypt group_vars/all/vault.yml`
|
||||||
|
5. **Update** `inventory.yml` with your server IPs
|
||||||
|
6. **Test** deployment to dev: `ansible-playbook deploy-pote.yml --limit development`
|
||||||
|
7. **Verify** with health check
|
||||||
|
8. **Deploy** to staging, then production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔗 Dependencies
|
||||||
|
|
||||||
|
### System Packages (via apt)
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
system_packages:
|
||||||
|
- python3.11
|
||||||
|
- python3.11-venv
|
||||||
|
- python3.11-dev
|
||||||
|
- python3-pip
|
||||||
|
- postgresql
|
||||||
|
- postgresql-contrib
|
||||||
|
- libpq-dev
|
||||||
|
- build-essential
|
||||||
|
- git
|
||||||
|
- curl
|
||||||
|
- htop
|
||||||
|
```
|
||||||
|
|
||||||
|
### Python Packages (via pip)
|
||||||
|
|
||||||
|
Installed automatically via `pip install -e .` from `pyproject.toml`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Support
|
||||||
|
|
||||||
|
- **POTE Configuration:** See `CUSTOMIZATION_CHECKLIST.md`
|
||||||
|
- **Ansible Variables:** See `ansible/roles/pote/defaults/main.yml`
|
||||||
|
- **Deployment Issues:** Check `scripts/health_check.py` output
|
||||||
|
- **Email Issues:** See `EMAIL_SETUP.md`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated:** December 2025
|
||||||
|
**POTE Version:** 0.1.0
|
||||||
|
**Target Ansible Version:** 2.9+
|
||||||
|
|
||||||
311
CUSTOMIZATION_CHECKLIST.md
Normal file
311
CUSTOMIZATION_CHECKLIST.md
Normal file
@ -0,0 +1,311 @@
|
|||||||
|
# POTE Customization Checklist
|
||||||
|
|
||||||
|
**Everything you need to change from generic defaults to your specific deployment.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔴 CRITICAL - Must Change (Security & Functionality)
|
||||||
|
|
||||||
|
### 1. Email Configuration (`.env` file)
|
||||||
|
```bash
|
||||||
|
# Current generic values:
|
||||||
|
SMTP_HOST=mail.levkin.ca # ✅ Already yours
|
||||||
|
SMTP_PORT=587 # ✅ OK (standard)
|
||||||
|
SMTP_USER=test@levkin.ca # ✅ Already yours
|
||||||
|
SMTP_PASSWORD=your_password_here # 🔴 CHANGE THIS
|
||||||
|
FROM_EMAIL=test@levkin.ca # ✅ Already yours
|
||||||
|
REPORT_RECIPIENTS=admin@localhost # 🔴 CHANGE THIS to your email
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Update `SMTP_PASSWORD` and `REPORT_RECIPIENTS` in `.env`
|
||||||
|
|
||||||
|
### 2. Database Password (`.env` file)
|
||||||
|
```bash
|
||||||
|
# Current generic value:
|
||||||
|
DATABASE_URL=postgresql://poteuser:changeme123@localhost:5432/potedb
|
||||||
|
|
||||||
|
# 🔴 CHANGE "changeme123" to a strong password
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Choose a strong password and update in:
|
||||||
|
- `.env` file
|
||||||
|
- PostgreSQL (if already created): `ALTER USER poteuser PASSWORD 'your_new_password';`
|
||||||
|
|
||||||
|
### 3. Git Repository (Ansible: `ansible/group_vars/all.yml`)
|
||||||
|
```yaml
|
||||||
|
# Current value:
|
||||||
|
pote_git_repo: "gitea@10.0.30.169:ilia/POTE.git"
|
||||||
|
|
||||||
|
# 🔴 This is YOUR Gitea repo, but verify:
|
||||||
|
# - IP address: 10.0.30.169 (is this correct?)
|
||||||
|
# - Username: ilia (is this correct?)
|
||||||
|
# - Repo name: POTE (is this correct?)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Verify or update the Git repo URL
|
||||||
|
|
||||||
|
### 4. SSH Keys (Ansible: `ansible/vault.example.yml`)
|
||||||
|
```yaml
|
||||||
|
# 🔴 MUST CREATE vault.yml with your actual keys:
|
||||||
|
vault_git_ssh_key: |
|
||||||
|
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||||
|
your_ssh_private_key_here # 🔴 ADD YOUR KEY
|
||||||
|
-----END OPENSSH PRIVATE KEY-----
|
||||||
|
|
||||||
|
vault_ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." # 🔴 ADD YOUR KEY
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:**
|
||||||
|
1. Copy `ansible/vault.example.yml` → `ansible/group_vars/all/vault.yml`
|
||||||
|
2. Add your SSH keys
|
||||||
|
3. Encrypt: `ansible-vault encrypt ansible/group_vars/all/vault.yml`
|
||||||
|
|
||||||
|
### 5. Server IP Addresses (Ansible: `ansible/inventory.example.yml`)
|
||||||
|
```yaml
|
||||||
|
# Current values:
|
||||||
|
development:
|
||||||
|
hosts:
|
||||||
|
pote-dev:
|
||||||
|
ansible_host: 10.0.10.100 # 🔴 CHANGE to your dev server IP
|
||||||
|
|
||||||
|
staging:
|
||||||
|
hosts:
|
||||||
|
pote-qa:
|
||||||
|
ansible_host: 10.0.10.101 # 🔴 CHANGE to your QA server IP
|
||||||
|
|
||||||
|
production:
|
||||||
|
hosts:
|
||||||
|
pote-prod:
|
||||||
|
ansible_host: 10.0.10.95 # 🔴 CHANGE to your prod server IP (or keep if correct)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Update all IP addresses to match your Proxmox LXC containers
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟡 IMPORTANT - Should Change (Gitea Secrets)
|
||||||
|
|
||||||
|
### 6. Gitea Secrets (for CI/CD pipelines)
|
||||||
|
|
||||||
|
**Location:** Gitea Web UI → Repository Settings → Secrets
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Required secrets:
|
||||||
|
DB_PASSWORD=changeme123 # 🟡 CHANGE to match your DB password
|
||||||
|
SMTP_PASSWORD=your_password_here # 🟡 CHANGE to your email password
|
||||||
|
SMTP_HOST=mail.levkin.ca # ✅ Already yours
|
||||||
|
SMTP_USER=test@levkin.ca # ✅ Already yours
|
||||||
|
FROM_EMAIL=test@levkin.ca # ✅ Already yours
|
||||||
|
|
||||||
|
# For deployment workflow (if using):
|
||||||
|
PROXMOX_SSH_KEY=<your_private_key> # 🟡 ADD your SSH private key
|
||||||
|
PROXMOX_HOST=10.0.10.95 # 🟡 CHANGE to your server IP
|
||||||
|
PROXMOX_USER=poteapp # 🟡 CHANGE if using different user
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Add/update secrets in Gitea:
|
||||||
|
1. Go to `https://git.levkin.ca/ilia/POTE/settings/secrets`
|
||||||
|
2. Add each secret listed above
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🟢 OPTIONAL - Customize for Your Needs
|
||||||
|
|
||||||
|
### 7. Email Recipients (Multiple locations)
|
||||||
|
|
||||||
|
**`.env` file:**
|
||||||
|
```bash
|
||||||
|
REPORT_RECIPIENTS=admin@localhost # 🟢 Change to your email(s), comma-separated
|
||||||
|
```
|
||||||
|
|
||||||
|
**`scripts/automated_daily_run.sh`:**
|
||||||
|
```bash
|
||||||
|
REPORT_RECIPIENTS="${REPORT_RECIPIENTS:-admin@localhost}" # 🟢 Change default
|
||||||
|
```
|
||||||
|
|
||||||
|
**`scripts/setup_cron.sh`:**
|
||||||
|
- Will prompt you interactively for email address
|
||||||
|
|
||||||
|
**Action:** Update to your preferred email(s) for reports
|
||||||
|
|
||||||
|
### 8. Market Monitoring Tickers (`.env` and Ansible)
|
||||||
|
|
||||||
|
**`.env` file:**
|
||||||
|
```bash
|
||||||
|
MARKET_MONITOR_TICKERS=NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX
|
||||||
|
# 🟢 Customize this list based on what Congress trades most
|
||||||
|
```
|
||||||
|
|
||||||
|
**`ansible/group_vars/all.yml`:**
|
||||||
|
```yaml
|
||||||
|
market_tickers: "NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX"
|
||||||
|
# 🟢 Should match .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Research most-traded congressional stocks and update list
|
||||||
|
|
||||||
|
### 9. Alert Severity Threshold (`.env` and Ansible)
|
||||||
|
|
||||||
|
**`.env` file:**
|
||||||
|
```bash
|
||||||
|
ALERT_MIN_SEVERITY=5 # 🟢 1-10, lower = more sensitive
|
||||||
|
```
|
||||||
|
|
||||||
|
**`ansible/group_vars/all.yml`:**
|
||||||
|
```yaml
|
||||||
|
alert_severity: 5 # 🟢 Should match .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Adjust based on how many alerts you want (5 is moderate)
|
||||||
|
|
||||||
|
### 10. Cron Schedule Times (Ansible)
|
||||||
|
|
||||||
|
**`ansible/group_vars/development.yml`, `staging.yml`, `production.yml`:**
|
||||||
|
```yaml
|
||||||
|
pote_daily_report_time: "0 6" # 🟢 6:00 AM - change to your preference
|
||||||
|
pote_weekly_report_time: "0 8" # 🟢 8:00 AM Sunday - change to your preference
|
||||||
|
pote_health_check_time: "*/30 * * * *" # 🟢 Every 30 min - change to your preference
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Adjust times based on your timezone and preferences
|
||||||
|
|
||||||
|
### 11. Log Level (`.env` and Ansible)
|
||||||
|
|
||||||
|
**`.env` file:**
|
||||||
|
```bash
|
||||||
|
LOG_LEVEL=INFO # 🟢 DEBUG, INFO, WARNING, ERROR
|
||||||
|
```
|
||||||
|
|
||||||
|
**`ansible/group_vars/all.yml`:**
|
||||||
|
```yaml
|
||||||
|
log_level: "INFO" # 🟢 Should match .env
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Use `DEBUG` for development, `INFO` for production
|
||||||
|
|
||||||
|
### 12. Application User (Ansible)
|
||||||
|
|
||||||
|
**`ansible/group_vars/all.yml`:**
|
||||||
|
```yaml
|
||||||
|
appuser_name: "poteapp" # 🟢 Change if you want a different username
|
||||||
|
```
|
||||||
|
|
||||||
|
**`scripts/proxmox_setup.sh`:**
|
||||||
|
```bash
|
||||||
|
APP_USER="poteapp" # 🟢 Should match Ansible
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Keep as `poteapp` unless you have a specific naming convention
|
||||||
|
|
||||||
|
### 13. Database Names (Ansible - per environment)
|
||||||
|
|
||||||
|
**`ansible/group_vars/development.yml`:**
|
||||||
|
```yaml
|
||||||
|
db_name: "potedb_dev" # 🟢 OK as-is
|
||||||
|
```
|
||||||
|
|
||||||
|
**`ansible/group_vars/staging.yml`:**
|
||||||
|
```yaml
|
||||||
|
db_name: "potedb_qa" # 🟢 OK as-is
|
||||||
|
```
|
||||||
|
|
||||||
|
**`ansible/group_vars/production.yml`:**
|
||||||
|
```yaml
|
||||||
|
db_name: "potedb" # 🟢 OK as-is
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Keep defaults unless you have a specific naming convention
|
||||||
|
|
||||||
|
### 14. Backup Retention (Ansible)
|
||||||
|
|
||||||
|
**`ansible/roles/pote/defaults/main.yml`:**
|
||||||
|
```yaml
|
||||||
|
pote_backup_retention_days: 90 # 🟢 Adjust based on disk space
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Increase for production (180+ days), decrease for dev (30 days)
|
||||||
|
|
||||||
|
### 15. API Keys (`.env` - if you get paid APIs)
|
||||||
|
|
||||||
|
**`.env` file:**
|
||||||
|
```bash
|
||||||
|
QUIVERQUANT_API_KEY= # 🟢 Optional paid service
|
||||||
|
FMP_API_KEY= # 🟢 Optional paid service
|
||||||
|
```
|
||||||
|
|
||||||
|
**Action:** Add keys if you subscribe to these services (not required)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📋 Summary: Quick Action List
|
||||||
|
|
||||||
|
### Immediate (Before First Deployment)
|
||||||
|
1. ✅ Update `.env`: `SMTP_PASSWORD`, `REPORT_RECIPIENTS`, `DATABASE_URL` password
|
||||||
|
2. ✅ Create `ansible/group_vars/all/vault.yml` with your SSH keys
|
||||||
|
3. ✅ Encrypt vault: `ansible-vault encrypt ansible/group_vars/all/vault.yml`
|
||||||
|
4. ✅ Update `ansible/inventory.yml` with your server IPs
|
||||||
|
5. ✅ Add Gitea secrets: `DB_PASSWORD`, `SMTP_PASSWORD`
|
||||||
|
|
||||||
|
### Before Production Use
|
||||||
|
6. ✅ Verify Git repo URL in `ansible/group_vars/all.yml`
|
||||||
|
7. ✅ Customize `MARKET_MONITOR_TICKERS` based on research
|
||||||
|
8. ✅ Adjust cron times for your timezone
|
||||||
|
9. ✅ Set appropriate log levels per environment
|
||||||
|
|
||||||
|
### Optional Enhancements
|
||||||
|
10. ⭐ Add paid API keys if you subscribe
|
||||||
|
11. ⭐ Adjust alert sensitivity based on testing
|
||||||
|
12. ⭐ Customize backup retention per environment
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔍 How to Find These Files
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Configuration files:
|
||||||
|
.env # Main config (not in git)
|
||||||
|
src/pote/config.py # Python config loader
|
||||||
|
|
||||||
|
# Ansible files:
|
||||||
|
ansible/inventory.yml # Server IPs (copy from .example.yml)
|
||||||
|
ansible/group_vars/all.yml # Common variables
|
||||||
|
ansible/group_vars/all/vault.yml # Secrets (create from vault.example.yml)
|
||||||
|
ansible/group_vars/development.yml # Dev environment
|
||||||
|
ansible/group_vars/staging.yml # QA environment
|
||||||
|
ansible/group_vars/production.yml # Prod environment
|
||||||
|
ansible/roles/pote/defaults/main.yml # All default variables
|
||||||
|
|
||||||
|
# Scripts:
|
||||||
|
scripts/automated_daily_run.sh # Daily automation
|
||||||
|
scripts/automated_weekly_run.sh # Weekly automation
|
||||||
|
scripts/setup_cron.sh # Cron setup
|
||||||
|
scripts/proxmox_setup.sh # Initial server setup
|
||||||
|
|
||||||
|
# CI/CD:
|
||||||
|
.github/workflows/ci.yml # CI pipeline
|
||||||
|
.github/workflows/deploy.yml # Deployment pipeline
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚨 Security Reminders
|
||||||
|
|
||||||
|
1. **NEVER commit `.env` to git** - it's in `.gitignore`
|
||||||
|
2. **NEVER commit unencrypted `vault.yml`** - always use `ansible-vault encrypt`
|
||||||
|
3. **NEVER put real passwords in example files** - use placeholders
|
||||||
|
4. **ALWAYS use strong passwords** - minimum 16 characters, mixed case, numbers, symbols
|
||||||
|
5. **ALWAYS use Gitea secrets** for CI/CD - never hardcode in workflow files
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📞 Need Help?
|
||||||
|
|
||||||
|
- **Ansible Vault:** `ansible-vault --help`
|
||||||
|
- **Gitea Secrets:** Repository Settings → Secrets → Actions
|
||||||
|
- **Environment Variables:** See `src/pote/config.py` for all available settings
|
||||||
|
- **Testing Config:** Run `python scripts/health_check.py` after changes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Last Updated:** December 2025
|
||||||
|
|
||||||
248
MOVE_ANSIBLE_TO_SEPARATE_REPO.md
Normal file
248
MOVE_ANSIBLE_TO_SEPARATE_REPO.md
Normal file
@ -0,0 +1,248 @@
|
|||||||
|
# Move Ansible Code to Separate Repository
|
||||||
|
|
||||||
|
## ❌ Problem
|
||||||
|
|
||||||
|
The `ansible/` directory in this repository doesn't make sense here because:
|
||||||
|
|
||||||
|
1. **Ansible runs BEFORE the app exists** - it creates the container and deploys the app
|
||||||
|
2. **Ansible belongs in your infrastructure repo** - not in the application repo
|
||||||
|
3. **Chicken-and-egg problem** - Ansible needs to clone this repo, but the Ansible code is inside it
|
||||||
|
|
||||||
|
## ✅ Solution
|
||||||
|
|
||||||
|
Move the Ansible code to your **Ansible/infrastructure repository** where your other roles live.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚚 Migration Steps
|
||||||
|
|
||||||
|
### Step 1: Copy Ansible Code to Your Ansible Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From your Ansible repository:
|
||||||
|
cd /path/to/your/ansible-repo
|
||||||
|
|
||||||
|
# Copy the POTE role
|
||||||
|
cp -r /path/to/pote/ansible/roles/pote roles/
|
||||||
|
|
||||||
|
# Merge group_vars (don't overwrite existing files!)
|
||||||
|
# Manually merge these into your existing group_vars:
|
||||||
|
# - ansible/group_vars/all.yml
|
||||||
|
# - ansible/group_vars/development.yml
|
||||||
|
# - ansible/group_vars/staging.yml
|
||||||
|
# - ansible/group_vars/production.yml
|
||||||
|
|
||||||
|
# Copy examples for reference
|
||||||
|
cp /path/to/pote/ansible/inventory.example.yml examples/pote-inventory.example.yml
|
||||||
|
cp /path/to/pote/ansible/vault.example.yml examples/pote-vault.example.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 2: Remove Ansible Code from POTE Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From the POTE repository:
|
||||||
|
cd /path/to/pote
|
||||||
|
|
||||||
|
# Remove the ansible directory
|
||||||
|
git rm -r ansible/
|
||||||
|
|
||||||
|
# Remove Ansible-specific docs (keep handoff guide)
|
||||||
|
git rm ANSIBLE_INTEGRATION.md
|
||||||
|
|
||||||
|
# Keep these files:
|
||||||
|
# - ANSIBLE_HANDOFF.md (reference for Ansible team)
|
||||||
|
# - CUSTOMIZATION_CHECKLIST.md (configuration reference)
|
||||||
|
# - docs/08_proxmox_deployment.md (manual deployment guide)
|
||||||
|
|
||||||
|
# Commit the changes
|
||||||
|
git add -A
|
||||||
|
git commit -m "Move Ansible code to infrastructure repository
|
||||||
|
|
||||||
|
The ansible/ directory has been moved to the Ansible infrastructure
|
||||||
|
repository where it belongs. This repo now only contains:
|
||||||
|
- Application code
|
||||||
|
- Deployment documentation (ANSIBLE_HANDOFF.md)
|
||||||
|
- Configuration reference (CUSTOMIZATION_CHECKLIST.md)
|
||||||
|
- Manual deployment guide (docs/08_proxmox_deployment.md)
|
||||||
|
|
||||||
|
Ansible code now lives in: [your-ansible-repo]/roles/pote/"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Update Your Ansible Repo
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# In your Ansible repository:
|
||||||
|
|
||||||
|
# Create the playbook
|
||||||
|
cat > playbooks/deploy-pote.yml << 'EOF'
|
||||||
|
---
|
||||||
|
- name: Deploy POTE Application
|
||||||
|
hosts: pote_servers
|
||||||
|
become: yes
|
||||||
|
|
||||||
|
roles:
|
||||||
|
- role: pote
|
||||||
|
tags: ['pote', 'deploy']
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Update inventory (add POTE servers)
|
||||||
|
# Edit inventory/production.yml, inventory/staging.yml, etc.
|
||||||
|
|
||||||
|
# Create/update vault with POTE secrets
|
||||||
|
ansible-vault edit group_vars/all/vault.yml
|
||||||
|
# Add: vault_smtp_password, vault_git_ssh_key, vault_db_password_*, etc.
|
||||||
|
|
||||||
|
# Test deployment
|
||||||
|
ansible-playbook playbooks/deploy-pote.yml --limit development --check
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📁 Final Directory Structure
|
||||||
|
|
||||||
|
### POTE Application Repo (this repo)
|
||||||
|
|
||||||
|
```
|
||||||
|
pote/
|
||||||
|
├── src/pote/ # Application code
|
||||||
|
├── tests/ # Tests
|
||||||
|
├── scripts/ # Deployment scripts
|
||||||
|
├── docs/ # Documentation
|
||||||
|
├── .github/workflows/ # CI/CD pipelines
|
||||||
|
├── ANSIBLE_HANDOFF.md # ✅ Keep - Reference for Ansible team
|
||||||
|
├── CUSTOMIZATION_CHECKLIST.md # ✅ Keep - Configuration guide
|
||||||
|
├── MOVE_ANSIBLE_TO_SEPARATE_REPO.md # ✅ Keep - This file
|
||||||
|
└── pyproject.toml # Python dependencies
|
||||||
|
```
|
||||||
|
|
||||||
|
### Your Ansible Infrastructure Repo
|
||||||
|
|
||||||
|
```
|
||||||
|
ansible-infrastructure/
|
||||||
|
├── roles/
|
||||||
|
│ ├── base_os/ # Your existing role
|
||||||
|
│ ├── pote/ # ✅ NEW - Moved from POTE repo
|
||||||
|
│ │ ├── defaults/main.yml
|
||||||
|
│ │ ├── tasks/main.yml
|
||||||
|
│ │ ├── templates/env.j2
|
||||||
|
│ │ └── handlers/main.yml
|
||||||
|
│ └── ...other roles...
|
||||||
|
├── playbooks/
|
||||||
|
│ ├── deploy-pote.yml # ✅ NEW - POTE deployment playbook
|
||||||
|
│ └── ...other playbooks...
|
||||||
|
├── group_vars/
|
||||||
|
│ ├── all.yml # Merged POTE variables
|
||||||
|
│ ├── all/vault.yml # Added POTE secrets
|
||||||
|
│ ├── development.yml # Merged POTE dev config
|
||||||
|
│ ├── staging.yml # Merged POTE staging config
|
||||||
|
│ └── production.yml # Merged POTE prod config
|
||||||
|
├── inventory/
|
||||||
|
│ ├── production.yml # Added POTE servers
|
||||||
|
│ ├── staging.yml # Added POTE servers
|
||||||
|
│ └── development.yml # Added POTE servers
|
||||||
|
└── examples/
|
||||||
|
├── pote-inventory.example.yml
|
||||||
|
└── pote-vault.example.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🎯 Benefits of This Approach
|
||||||
|
|
||||||
|
1. **Cleaner Separation** - Application code separate from infrastructure code
|
||||||
|
2. **No Circular Dependency** - Ansible can clone the app repo without needing to be inside it
|
||||||
|
3. **Centralized Infrastructure** - All your Ansible roles in one place
|
||||||
|
4. **Easier Management** - Update infrastructure without touching app code
|
||||||
|
5. **Standard Practice** - Follows industry best practices
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🔄 Workflow After Migration
|
||||||
|
|
||||||
|
### Deploying POTE
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# From your Ansible repository:
|
||||||
|
ansible-playbook playbooks/deploy-pote.yml --limit production
|
||||||
|
```
|
||||||
|
|
||||||
|
### Updating POTE Application
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Option 1: Via Ansible (recommended)
|
||||||
|
ansible-playbook playbooks/deploy-pote.yml --limit production --tags update
|
||||||
|
|
||||||
|
# Option 2: Via Gitea Actions (CI/CD)
|
||||||
|
# Push to main branch → Gitea Actions runs tests → Auto-deploy
|
||||||
|
|
||||||
|
# Option 3: Manual (SSH to server)
|
||||||
|
cd ~/pote
|
||||||
|
git pull origin main
|
||||||
|
source venv/bin/activate
|
||||||
|
pip install -e .
|
||||||
|
alembic upgrade head
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 📝 What to Tell Your Ansible Team
|
||||||
|
|
||||||
|
Send them:
|
||||||
|
1. **ANSIBLE_HANDOFF.md** - Complete integration guide
|
||||||
|
2. **CUSTOMIZATION_CHECKLIST.md** - What needs to be configured
|
||||||
|
3. **ansible/** directory - The actual role code
|
||||||
|
|
||||||
|
They need to:
|
||||||
|
1. Copy the role to their Ansible repo
|
||||||
|
2. Merge the group_vars into their existing structure
|
||||||
|
3. Add POTE servers to their inventory
|
||||||
|
4. Add POTE secrets to their vault
|
||||||
|
5. Test deployment to dev environment
|
||||||
|
6. Deploy to staging, then production
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ Important Notes
|
||||||
|
|
||||||
|
### Don't Delete These from POTE Repo
|
||||||
|
|
||||||
|
Keep these files for reference:
|
||||||
|
- `ANSIBLE_HANDOFF.md` - Ansible integration guide
|
||||||
|
- `CUSTOMIZATION_CHECKLIST.md` - Configuration reference
|
||||||
|
- `docs/08_proxmox_deployment.md` - Manual deployment guide
|
||||||
|
- `scripts/proxmox_setup.sh` - Bootstrap script (if needed)
|
||||||
|
|
||||||
|
### Do Delete These from POTE Repo
|
||||||
|
|
||||||
|
Remove these (they belong in Ansible repo):
|
||||||
|
- `ansible/` directory (entire thing)
|
||||||
|
- `ANSIBLE_INTEGRATION.md` (redundant with ANSIBLE_HANDOFF.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🚀 Ready to Move?
|
||||||
|
|
||||||
|
Run these commands when ready:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Backup first (just in case)
|
||||||
|
cp -r ansible /tmp/pote-ansible-backup
|
||||||
|
|
||||||
|
# 2. Remove from POTE repo
|
||||||
|
git rm -r ansible/
|
||||||
|
git rm ANSIBLE_INTEGRATION.md
|
||||||
|
git add -A
|
||||||
|
git commit -m "Move Ansible code to infrastructure repository"
|
||||||
|
git push origin dev
|
||||||
|
|
||||||
|
# 3. Copy to your Ansible repo (do this manually)
|
||||||
|
# See "Step 1: Copy Ansible Code to Your Ansible Repo" above
|
||||||
|
|
||||||
|
# 4. Test deployment from Ansible repo
|
||||||
|
# See "Step 3: Update Your Ansible Repo" above
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Questions?** See `ANSIBLE_HANDOFF.md` for complete integration details.
|
||||||
|
|
||||||
Loading…
x
Reference in New Issue
Block a user