Add documentation for Ansible separation and customization
NEW FILES: ========== 📄 CUSTOMIZATION_CHECKLIST.md - Complete list of everything that needs customization - Organized by priority: Critical, Important, Optional - Covers .env, Ansible, Gitea secrets, email, etc. - Quick action checklist for deployment 📄 ANSIBLE_HANDOFF.md - Guide for integrating POTE with existing Ansible system - Explains what Ansible needs to know - Variable reference and secrets management - Multi-environment deployment strategy - Example playbook and testing instructions 📄 MOVE_ANSIBLE_TO_SEPARATE_REPO.md - Explains why ansible/ should be in infrastructure repo - Step-by-step migration guide - Final directory structure for both repos - Benefits and workflow after migration KEY INSIGHT: ============ The ansible/ directory doesn't belong in the POTE app repo because: - Ansible runs BEFORE the app exists (creates container, deploys app) - Creates circular dependency (Ansible clones repo that contains Ansible) - Should live in centralized infrastructure repository NEXT STEPS: =========== 1. Review CUSTOMIZATION_CHECKLIST.md for deployment config 2. Copy ansible/ to infrastructure repo 3. Remove ansible/ from POTE repo (keep handoff docs) 4. Deploy via centralized Ansible system
This commit is contained in:
ilia
parent
d2ae095fcf
commit
e9fd12d949
376
ANSIBLE_HANDOFF.md
Normal file
376
ANSIBLE_HANDOFF.md
Normal file
@ -0,0 +1,376 @@
|
||||
# POTE Ansible Integration - Handoff Guide
|
||||
|
||||
## 🎯 Purpose
|
||||
|
||||
This document explains what your Ansible auto-configure system needs to know to deploy POTE across different environments.
|
||||
|
||||
**Note:** The Ansible code in `ansible/` directory should be **moved to your Ansible repository**, not kept here in the POTE app repo.
|
||||
|
||||
---
|
||||
|
||||
## 📦 What to Move to Your Ansible Repository
|
||||
|
||||
### Files to Extract from POTE Repo
|
||||
|
||||
```bash
|
||||
# Copy these to your Ansible repository:
|
||||
ansible/
|
||||
├── README.md → roles/pote/README.md
|
||||
├── roles/pote/defaults/main.yml → roles/pote/defaults/main.yml
|
||||
├── group_vars/
|
||||
│ ├── all.yml → group_vars/all.yml (merge with existing)
|
||||
│ ├── development.yml → group_vars/development.yml (merge)
|
||||
│ ├── staging.yml → group_vars/staging.yml (merge)
|
||||
│ └── production.yml → group_vars/production.yml (merge)
|
||||
├── inventory.example.yml → inventory.example.yml (reference)
|
||||
└── vault.example.yml → vault.example.yml (reference)
|
||||
```
|
||||
|
||||
### What to Keep in POTE Repo
|
||||
|
||||
```bash
|
||||
# Keep these for deployment reference:
|
||||
docs/08_proxmox_deployment.md # Manual deployment guide
|
||||
scripts/proxmox_setup.sh # Bootstrap script (if needed)
|
||||
CUSTOMIZATION_CHECKLIST.md # Configuration reference
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Integration with Your Ansible System
|
||||
|
||||
### 1. Create POTE Role in Your Ansible Repo
|
||||
|
||||
```bash
|
||||
# In your Ansible repository:
|
||||
roles/
|
||||
└── pote/
|
||||
├── defaults/
|
||||
│ └── main.yml # Copy from POTE repo
|
||||
├── tasks/
|
||||
│ └── main.yml # Create based on requirements below
|
||||
├── templates/
|
||||
│ └── env.j2 # Create for .env file
|
||||
├── handlers/
|
||||
│ └── main.yml # Restart services, etc.
|
||||
└── README.md # Copy from POTE repo
|
||||
```
|
||||
|
||||
### 2. Required Ansible Tasks
|
||||
|
||||
Your `roles/pote/tasks/main.yml` should handle:
|
||||
|
||||
#### Phase 1: System Preparation
|
||||
- Create application user (`poteapp`)
|
||||
- Install system dependencies (Python 3.11, PostgreSQL, build-essential, etc.)
|
||||
- Configure PostgreSQL database and user
|
||||
|
||||
#### Phase 2: Application Deployment
|
||||
- Clone POTE repository from Git (using SSH key)
|
||||
- Create Python virtual environment
|
||||
- Install Python dependencies
|
||||
- Create `.env` file from template
|
||||
- Run database migrations (`alembic upgrade head`)
|
||||
|
||||
#### Phase 3: Automation Setup
|
||||
- Create log directories
|
||||
- Install cron jobs for daily/weekly reports
|
||||
- Set up health check monitoring
|
||||
- Configure email reporting
|
||||
|
||||
#### Phase 4: Security & Cleanup
|
||||
- Set proper file permissions
|
||||
- Secure `.env` file (600)
|
||||
- Clean up temporary files
|
||||
|
||||
---
|
||||
|
||||
## 📋 Variables Your Ansible System Needs
|
||||
|
||||
### Minimal Required Variables (3)
|
||||
|
||||
```yaml
|
||||
# Absolute minimum to deploy POTE:
|
||||
pote_git_repo: "gitea@10.0.30.169:ilia/POTE.git"
|
||||
pote_git_branch: "main" # or "dev", "qa" per environment
|
||||
vault_smtp_password: "{{ vault_smtp_password }}"
|
||||
```
|
||||
|
||||
### Recommended Variables (Environment-Specific)
|
||||
|
||||
```yaml
|
||||
# Per-environment configuration:
|
||||
environment: "production" # or "development", "staging"
|
||||
pote_git_branch: "main" # "dev" for development, "qa" for staging
|
||||
|
||||
# Database
|
||||
db_name: "potedb" # "potedb_dev" for dev, "potedb_qa" for staging
|
||||
db_user: "poteuser"
|
||||
db_password: "{{ vault_db_password_prod }}"
|
||||
|
||||
# Email
|
||||
smtp_host: "mail.levkin.ca"
|
||||
smtp_port: 587
|
||||
smtp_user: "test@levkin.ca"
|
||||
from_email: "test@levkin.ca"
|
||||
smtp_password: "{{ vault_smtp_password }}"
|
||||
report_recipients: "ilia@levkin.ca" # Your actual email
|
||||
|
||||
# Monitoring
|
||||
market_tickers: "NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX"
|
||||
alert_severity: 5
|
||||
|
||||
# Automation
|
||||
pote_enable_cron: true
|
||||
pote_daily_report_time: "0 6" # 6:00 AM
|
||||
pote_weekly_report_time: "0 8" # 8:00 AM Sunday
|
||||
```
|
||||
|
||||
### Full Variable List
|
||||
|
||||
See `ansible/roles/pote/defaults/main.yml` for all 200+ variables with defaults.
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Secrets Management
|
||||
|
||||
### Required Secrets in Ansible Vault
|
||||
|
||||
```yaml
|
||||
# In your Ansible vault (group_vars/all/vault.yml):
|
||||
vault_git_ssh_key: |
|
||||
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||
your_ssh_private_key_for_gitea
|
||||
-----END OPENSSH PRIVATE KEY-----
|
||||
|
||||
vault_ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2E..."
|
||||
|
||||
vault_smtp_password: "your_email_password"
|
||||
|
||||
vault_db_password_dev: "dev_password_123"
|
||||
vault_db_password_qa: "qa_password_123"
|
||||
vault_db_password_prod: "strong_production_password"
|
||||
|
||||
# Optional:
|
||||
vault_quiverquant_key: ""
|
||||
vault_fmp_key: ""
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎭 Multi-Environment Strategy
|
||||
|
||||
### Branch → Environment Mapping
|
||||
|
||||
```yaml
|
||||
development:
|
||||
git_branch: dev
|
||||
db_name: potedb_dev
|
||||
log_level: DEBUG
|
||||
cron_enabled: false # Manual testing
|
||||
|
||||
staging:
|
||||
git_branch: qa
|
||||
db_name: potedb_qa
|
||||
log_level: INFO
|
||||
cron_enabled: true
|
||||
|
||||
production:
|
||||
git_branch: main
|
||||
db_name: potedb
|
||||
log_level: WARNING
|
||||
cron_enabled: true
|
||||
```
|
||||
|
||||
### Deployment Commands
|
||||
|
||||
```bash
|
||||
# Deploy to development
|
||||
ansible-playbook deploy-pote.yml --limit development
|
||||
|
||||
# Deploy to staging
|
||||
ansible-playbook deploy-pote.yml --limit staging
|
||||
|
||||
# Deploy to production
|
||||
ansible-playbook deploy-pote.yml --limit production
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Example Playbook
|
||||
|
||||
```yaml
|
||||
---
|
||||
# deploy-pote.yml
|
||||
- name: Deploy POTE Application
|
||||
hosts: all
|
||||
become: yes
|
||||
|
||||
pre_tasks:
|
||||
- name: Ensure base OS role has run
|
||||
debug:
|
||||
msg: "Assuming base_os role has already configured the system"
|
||||
|
||||
roles:
|
||||
- role: pote
|
||||
tags: ['pote', 'deploy']
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Deployment Workflow
|
||||
|
||||
### Initial Deployment (New Server)
|
||||
|
||||
1. **Provision LXC container** in Proxmox
|
||||
2. **Run base_os role** (your existing role)
|
||||
3. **Run pote role** (new role)
|
||||
4. **Verify deployment** with health check
|
||||
|
||||
### Updates (Existing Server)
|
||||
|
||||
1. **Pull latest code** from Git
|
||||
2. **Update dependencies** if needed
|
||||
3. **Run migrations** (`alembic upgrade head`)
|
||||
4. **Restart services** if needed
|
||||
|
||||
### Rollback (If Needed)
|
||||
|
||||
1. **Checkout previous Git tag/commit**
|
||||
2. **Run migrations down** if needed
|
||||
3. **Restore database backup** if needed
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing the Integration
|
||||
|
||||
### Dry Run
|
||||
|
||||
```bash
|
||||
# Test without making changes
|
||||
ansible-playbook deploy-pote.yml --limit development --check --diff
|
||||
```
|
||||
|
||||
### Verify Deployment
|
||||
|
||||
```bash
|
||||
# SSH into server and run:
|
||||
cd ~/pote
|
||||
source venv/bin/activate
|
||||
python scripts/health_check.py
|
||||
|
||||
# Check cron jobs:
|
||||
crontab -l | grep POTE
|
||||
|
||||
# Check logs:
|
||||
tail -f ~/logs/daily_run.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 What POTE Provides for Ansible
|
||||
|
||||
### 1. Configuration Template
|
||||
|
||||
The `.env` file template your Ansible should generate:
|
||||
|
||||
```bash
|
||||
# Database
|
||||
DATABASE_URL=postgresql://{{ db_user }}:{{ db_password }}@localhost:5432/{{ db_name }}
|
||||
|
||||
# Email
|
||||
SMTP_HOST={{ smtp_host }}
|
||||
SMTP_PORT={{ smtp_port }}
|
||||
SMTP_USER={{ smtp_user }}
|
||||
SMTP_PASSWORD={{ smtp_password }}
|
||||
FROM_EMAIL={{ from_email }}
|
||||
REPORT_RECIPIENTS={{ report_recipients }}
|
||||
|
||||
# Monitoring
|
||||
MARKET_MONITOR_TICKERS={{ market_tickers }}
|
||||
ALERT_MIN_SEVERITY={{ alert_severity }}
|
||||
|
||||
# Logging
|
||||
LOG_LEVEL={{ log_level }}
|
||||
LOG_FILE={{ log_file }}
|
||||
|
||||
# Optional API Keys
|
||||
QUIVERQUANT_API_KEY={{ quiverquant_api_key | default('') }}
|
||||
FMP_API_KEY={{ fmp_api_key | default('') }}
|
||||
```
|
||||
|
||||
### 2. Health Check Endpoint
|
||||
|
||||
```bash
|
||||
# Your Ansible can verify deployment with:
|
||||
python scripts/health_check.py
|
||||
# Exit code 0 = healthy, non-zero = issues
|
||||
```
|
||||
|
||||
### 3. Database Migrations
|
||||
|
||||
```bash
|
||||
# Your Ansible should run after code updates:
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
### 4. Idempotency
|
||||
|
||||
All POTE scripts are idempotent:
|
||||
- Re-running data fetches won't duplicate records
|
||||
- Migrations are safe to re-run
|
||||
- Cron setup checks for existing jobs
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Quick Start for Your Ansible Team
|
||||
|
||||
1. **Copy** `ansible/` directory from POTE repo to your Ansible repo
|
||||
2. **Customize** `group_vars/` files with your values
|
||||
3. **Create** `group_vars/all/vault.yml` with secrets
|
||||
4. **Encrypt** vault: `ansible-vault encrypt group_vars/all/vault.yml`
|
||||
5. **Update** `inventory.yml` with your server IPs
|
||||
6. **Test** deployment to dev: `ansible-playbook deploy-pote.yml --limit development`
|
||||
7. **Verify** with health check
|
||||
8. **Deploy** to staging, then production
|
||||
|
||||
---
|
||||
|
||||
## 🔗 Dependencies
|
||||
|
||||
### System Packages (via apt)
|
||||
|
||||
```yaml
|
||||
system_packages:
|
||||
- python3.11
|
||||
- python3.11-venv
|
||||
- python3.11-dev
|
||||
- python3-pip
|
||||
- postgresql
|
||||
- postgresql-contrib
|
||||
- libpq-dev
|
||||
- build-essential
|
||||
- git
|
||||
- curl
|
||||
- htop
|
||||
```
|
||||
|
||||
### Python Packages (via pip)
|
||||
|
||||
Installed automatically via `pip install -e .` from `pyproject.toml`
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support
|
||||
|
||||
- **POTE Configuration:** See `CUSTOMIZATION_CHECKLIST.md`
|
||||
- **Ansible Variables:** See `ansible/roles/pote/defaults/main.yml`
|
||||
- **Deployment Issues:** Check `scripts/health_check.py` output
|
||||
- **Email Issues:** See `EMAIL_SETUP.md`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** December 2025
|
||||
**POTE Version:** 0.1.0
|
||||
**Target Ansible Version:** 2.9+
|
||||
|
||||
311
CUSTOMIZATION_CHECKLIST.md
Normal file
311
CUSTOMIZATION_CHECKLIST.md
Normal file
@ -0,0 +1,311 @@
|
||||
# POTE Customization Checklist
|
||||
|
||||
**Everything you need to change from generic defaults to your specific deployment.**
|
||||
|
||||
---
|
||||
|
||||
## 🔴 CRITICAL - Must Change (Security & Functionality)
|
||||
|
||||
### 1. Email Configuration (`.env` file)
|
||||
```bash
|
||||
# Current generic values:
|
||||
SMTP_HOST=mail.levkin.ca # ✅ Already yours
|
||||
SMTP_PORT=587 # ✅ OK (standard)
|
||||
SMTP_USER=test@levkin.ca # ✅ Already yours
|
||||
SMTP_PASSWORD=your_password_here # 🔴 CHANGE THIS
|
||||
FROM_EMAIL=test@levkin.ca # ✅ Already yours
|
||||
REPORT_RECIPIENTS=admin@localhost # 🔴 CHANGE THIS to your email
|
||||
```
|
||||
|
||||
**Action:** Update `SMTP_PASSWORD` and `REPORT_RECIPIENTS` in `.env`
|
||||
|
||||
### 2. Database Password (`.env` file)
|
||||
```bash
|
||||
# Current generic value:
|
||||
DATABASE_URL=postgresql://poteuser:changeme123@localhost:5432/potedb
|
||||
|
||||
# 🔴 CHANGE "changeme123" to a strong password
|
||||
```
|
||||
|
||||
**Action:** Choose a strong password and update in:
|
||||
- `.env` file
|
||||
- PostgreSQL (if already created): `ALTER USER poteuser PASSWORD 'your_new_password';`
|
||||
|
||||
### 3. Git Repository (Ansible: `ansible/group_vars/all.yml`)
|
||||
```yaml
|
||||
# Current value:
|
||||
pote_git_repo: "gitea@10.0.30.169:ilia/POTE.git"
|
||||
|
||||
# 🔴 This is YOUR Gitea repo, but verify:
|
||||
# - IP address: 10.0.30.169 (is this correct?)
|
||||
# - Username: ilia (is this correct?)
|
||||
# - Repo name: POTE (is this correct?)
|
||||
```
|
||||
|
||||
**Action:** Verify or update the Git repo URL
|
||||
|
||||
### 4. SSH Keys (Ansible: `ansible/vault.example.yml`)
|
||||
```yaml
|
||||
# 🔴 MUST CREATE vault.yml with your actual keys:
|
||||
vault_git_ssh_key: |
|
||||
-----BEGIN OPENSSH PRIVATE KEY-----
|
||||
your_ssh_private_key_here # 🔴 ADD YOUR KEY
|
||||
-----END OPENSSH PRIVATE KEY-----
|
||||
|
||||
vault_ssh_public_key: "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC..." # 🔴 ADD YOUR KEY
|
||||
```
|
||||
|
||||
**Action:**
|
||||
1. Copy `ansible/vault.example.yml` → `ansible/group_vars/all/vault.yml`
|
||||
2. Add your SSH keys
|
||||
3. Encrypt: `ansible-vault encrypt ansible/group_vars/all/vault.yml`
|
||||
|
||||
### 5. Server IP Addresses (Ansible: `ansible/inventory.example.yml`)
|
||||
```yaml
|
||||
# Current values:
|
||||
development:
|
||||
hosts:
|
||||
pote-dev:
|
||||
ansible_host: 10.0.10.100 # 🔴 CHANGE to your dev server IP
|
||||
|
||||
staging:
|
||||
hosts:
|
||||
pote-qa:
|
||||
ansible_host: 10.0.10.101 # 🔴 CHANGE to your QA server IP
|
||||
|
||||
production:
|
||||
hosts:
|
||||
pote-prod:
|
||||
ansible_host: 10.0.10.95 # 🔴 CHANGE to your prod server IP (or keep if correct)
|
||||
```
|
||||
|
||||
**Action:** Update all IP addresses to match your Proxmox LXC containers
|
||||
|
||||
---
|
||||
|
||||
## 🟡 IMPORTANT - Should Change (Gitea Secrets)
|
||||
|
||||
### 6. Gitea Secrets (for CI/CD pipelines)
|
||||
|
||||
**Location:** Gitea Web UI → Repository Settings → Secrets
|
||||
|
||||
```bash
|
||||
# Required secrets:
|
||||
DB_PASSWORD=changeme123 # 🟡 CHANGE to match your DB password
|
||||
SMTP_PASSWORD=your_password_here # 🟡 CHANGE to your email password
|
||||
SMTP_HOST=mail.levkin.ca # ✅ Already yours
|
||||
SMTP_USER=test@levkin.ca # ✅ Already yours
|
||||
FROM_EMAIL=test@levkin.ca # ✅ Already yours
|
||||
|
||||
# For deployment workflow (if using):
|
||||
PROXMOX_SSH_KEY=<your_private_key> # 🟡 ADD your SSH private key
|
||||
PROXMOX_HOST=10.0.10.95 # 🟡 CHANGE to your server IP
|
||||
PROXMOX_USER=poteapp # 🟡 CHANGE if using different user
|
||||
```
|
||||
|
||||
**Action:** Add/update secrets in Gitea:
|
||||
1. Go to `https://git.levkin.ca/ilia/POTE/settings/secrets`
|
||||
2. Add each secret listed above
|
||||
|
||||
---
|
||||
|
||||
## 🟢 OPTIONAL - Customize for Your Needs
|
||||
|
||||
### 7. Email Recipients (Multiple locations)
|
||||
|
||||
**`.env` file:**
|
||||
```bash
|
||||
REPORT_RECIPIENTS=admin@localhost # 🟢 Change to your email(s), comma-separated
|
||||
```
|
||||
|
||||
**`scripts/automated_daily_run.sh`:**
|
||||
```bash
|
||||
REPORT_RECIPIENTS="${REPORT_RECIPIENTS:-admin@localhost}" # 🟢 Change default
|
||||
```
|
||||
|
||||
**`scripts/setup_cron.sh`:**
|
||||
- Will prompt you interactively for email address
|
||||
|
||||
**Action:** Update to your preferred email(s) for reports
|
||||
|
||||
### 8. Market Monitoring Tickers (`.env` and Ansible)
|
||||
|
||||
**`.env` file:**
|
||||
```bash
|
||||
MARKET_MONITOR_TICKERS=NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX
|
||||
# 🟢 Customize this list based on what Congress trades most
|
||||
```
|
||||
|
||||
**`ansible/group_vars/all.yml`:**
|
||||
```yaml
|
||||
market_tickers: "NVDA,TSLA,AAPL,MSFT,GOOGL,META,AMZN,AMD,INTC,NFLX"
|
||||
# 🟢 Should match .env
|
||||
```
|
||||
|
||||
**Action:** Research most-traded congressional stocks and update list
|
||||
|
||||
### 9. Alert Severity Threshold (`.env` and Ansible)
|
||||
|
||||
**`.env` file:**
|
||||
```bash
|
||||
ALERT_MIN_SEVERITY=5 # 🟢 1-10, lower = more sensitive
|
||||
```
|
||||
|
||||
**`ansible/group_vars/all.yml`:**
|
||||
```yaml
|
||||
alert_severity: 5 # 🟢 Should match .env
|
||||
```
|
||||
|
||||
**Action:** Adjust based on how many alerts you want (5 is moderate)
|
||||
|
||||
### 10. Cron Schedule Times (Ansible)
|
||||
|
||||
**`ansible/group_vars/development.yml`, `staging.yml`, `production.yml`:**
|
||||
```yaml
|
||||
pote_daily_report_time: "0 6" # 🟢 6:00 AM - change to your preference
|
||||
pote_weekly_report_time: "0 8" # 🟢 8:00 AM Sunday - change to your preference
|
||||
pote_health_check_time: "*/30 * * * *" # 🟢 Every 30 min - change to your preference
|
||||
```
|
||||
|
||||
**Action:** Adjust times based on your timezone and preferences
|
||||
|
||||
### 11. Log Level (`.env` and Ansible)
|
||||
|
||||
**`.env` file:**
|
||||
```bash
|
||||
LOG_LEVEL=INFO # 🟢 DEBUG, INFO, WARNING, ERROR
|
||||
```
|
||||
|
||||
**`ansible/group_vars/all.yml`:**
|
||||
```yaml
|
||||
log_level: "INFO" # 🟢 Should match .env
|
||||
```
|
||||
|
||||
**Action:** Use `DEBUG` for development, `INFO` for production
|
||||
|
||||
### 12. Application User (Ansible)
|
||||
|
||||
**`ansible/group_vars/all.yml`:**
|
||||
```yaml
|
||||
appuser_name: "poteapp" # 🟢 Change if you want a different username
|
||||
```
|
||||
|
||||
**`scripts/proxmox_setup.sh`:**
|
||||
```bash
|
||||
APP_USER="poteapp" # 🟢 Should match Ansible
|
||||
```
|
||||
|
||||
**Action:** Keep as `poteapp` unless you have a specific naming convention
|
||||
|
||||
### 13. Database Names (Ansible - per environment)
|
||||
|
||||
**`ansible/group_vars/development.yml`:**
|
||||
```yaml
|
||||
db_name: "potedb_dev" # 🟢 OK as-is
|
||||
```
|
||||
|
||||
**`ansible/group_vars/staging.yml`:**
|
||||
```yaml
|
||||
db_name: "potedb_qa" # 🟢 OK as-is
|
||||
```
|
||||
|
||||
**`ansible/group_vars/production.yml`:**
|
||||
```yaml
|
||||
db_name: "potedb" # 🟢 OK as-is
|
||||
```
|
||||
|
||||
**Action:** Keep defaults unless you have a specific naming convention
|
||||
|
||||
### 14. Backup Retention (Ansible)
|
||||
|
||||
**`ansible/roles/pote/defaults/main.yml`:**
|
||||
```yaml
|
||||
pote_backup_retention_days: 90 # 🟢 Adjust based on disk space
|
||||
```
|
||||
|
||||
**Action:** Increase for production (180+ days), decrease for dev (30 days)
|
||||
|
||||
### 15. API Keys (`.env` - if you get paid APIs)
|
||||
|
||||
**`.env` file:**
|
||||
```bash
|
||||
QUIVERQUANT_API_KEY= # 🟢 Optional paid service
|
||||
FMP_API_KEY= # 🟢 Optional paid service
|
||||
```
|
||||
|
||||
**Action:** Add keys if you subscribe to these services (not required)
|
||||
|
||||
---
|
||||
|
||||
## 📋 Summary: Quick Action List
|
||||
|
||||
### Immediate (Before First Deployment)
|
||||
1. ✅ Update `.env`: `SMTP_PASSWORD`, `REPORT_RECIPIENTS`, `DATABASE_URL` password
|
||||
2. ✅ Create `ansible/group_vars/all/vault.yml` with your SSH keys
|
||||
3. ✅ Encrypt vault: `ansible-vault encrypt ansible/group_vars/all/vault.yml`
|
||||
4. ✅ Update `ansible/inventory.yml` with your server IPs
|
||||
5. ✅ Add Gitea secrets: `DB_PASSWORD`, `SMTP_PASSWORD`
|
||||
|
||||
### Before Production Use
|
||||
6. ✅ Verify Git repo URL in `ansible/group_vars/all.yml`
|
||||
7. ✅ Customize `MARKET_MONITOR_TICKERS` based on research
|
||||
8. ✅ Adjust cron times for your timezone
|
||||
9. ✅ Set appropriate log levels per environment
|
||||
|
||||
### Optional Enhancements
|
||||
10. ⭐ Add paid API keys if you subscribe
|
||||
11. ⭐ Adjust alert sensitivity based on testing
|
||||
12. ⭐ Customize backup retention per environment
|
||||
|
||||
---
|
||||
|
||||
## 🔍 How to Find These Files
|
||||
|
||||
```bash
|
||||
# Configuration files:
|
||||
.env # Main config (not in git)
|
||||
src/pote/config.py # Python config loader
|
||||
|
||||
# Ansible files:
|
||||
ansible/inventory.yml # Server IPs (copy from .example.yml)
|
||||
ansible/group_vars/all.yml # Common variables
|
||||
ansible/group_vars/all/vault.yml # Secrets (create from vault.example.yml)
|
||||
ansible/group_vars/development.yml # Dev environment
|
||||
ansible/group_vars/staging.yml # QA environment
|
||||
ansible/group_vars/production.yml # Prod environment
|
||||
ansible/roles/pote/defaults/main.yml # All default variables
|
||||
|
||||
# Scripts:
|
||||
scripts/automated_daily_run.sh # Daily automation
|
||||
scripts/automated_weekly_run.sh # Weekly automation
|
||||
scripts/setup_cron.sh # Cron setup
|
||||
scripts/proxmox_setup.sh # Initial server setup
|
||||
|
||||
# CI/CD:
|
||||
.github/workflows/ci.yml # CI pipeline
|
||||
.github/workflows/deploy.yml # Deployment pipeline
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Security Reminders
|
||||
|
||||
1. **NEVER commit `.env` to git** - it's in `.gitignore`
|
||||
2. **NEVER commit unencrypted `vault.yml`** - always use `ansible-vault encrypt`
|
||||
3. **NEVER put real passwords in example files** - use placeholders
|
||||
4. **ALWAYS use strong passwords** - minimum 16 characters, mixed case, numbers, symbols
|
||||
5. **ALWAYS use Gitea secrets** for CI/CD - never hardcode in workflow files
|
||||
|
||||
---
|
||||
|
||||
## 📞 Need Help?
|
||||
|
||||
- **Ansible Vault:** `ansible-vault --help`
|
||||
- **Gitea Secrets:** Repository Settings → Secrets → Actions
|
||||
- **Environment Variables:** See `src/pote/config.py` for all available settings
|
||||
- **Testing Config:** Run `python scripts/health_check.py` after changes
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** December 2025
|
||||
|
||||
248
MOVE_ANSIBLE_TO_SEPARATE_REPO.md
Normal file
248
MOVE_ANSIBLE_TO_SEPARATE_REPO.md
Normal file
@ -0,0 +1,248 @@
|
||||
# Move Ansible Code to Separate Repository
|
||||
|
||||
## ❌ Problem
|
||||
|
||||
The `ansible/` directory in this repository doesn't make sense here because:
|
||||
|
||||
1. **Ansible runs BEFORE the app exists** - it creates the container and deploys the app
|
||||
2. **Ansible belongs in your infrastructure repo** - not in the application repo
|
||||
3. **Chicken-and-egg problem** - Ansible needs to clone this repo, but the Ansible code is inside it
|
||||
|
||||
## ✅ Solution
|
||||
|
||||
Move the Ansible code to your **Ansible/infrastructure repository** where your other roles live.
|
||||
|
||||
---
|
||||
|
||||
## 🚚 Migration Steps
|
||||
|
||||
### Step 1: Copy Ansible Code to Your Ansible Repo
|
||||
|
||||
```bash
|
||||
# From your Ansible repository:
|
||||
cd /path/to/your/ansible-repo
|
||||
|
||||
# Copy the POTE role
|
||||
cp -r /path/to/pote/ansible/roles/pote roles/
|
||||
|
||||
# Merge group_vars (don't overwrite existing files!)
|
||||
# Manually merge these into your existing group_vars:
|
||||
# - ansible/group_vars/all.yml
|
||||
# - ansible/group_vars/development.yml
|
||||
# - ansible/group_vars/staging.yml
|
||||
# - ansible/group_vars/production.yml
|
||||
|
||||
# Copy examples for reference
|
||||
cp /path/to/pote/ansible/inventory.example.yml examples/pote-inventory.example.yml
|
||||
cp /path/to/pote/ansible/vault.example.yml examples/pote-vault.example.yml
|
||||
```
|
||||
|
||||
### Step 2: Remove Ansible Code from POTE Repo
|
||||
|
||||
```bash
|
||||
# From the POTE repository:
|
||||
cd /path/to/pote
|
||||
|
||||
# Remove the ansible directory
|
||||
git rm -r ansible/
|
||||
|
||||
# Remove Ansible-specific docs (keep handoff guide)
|
||||
git rm ANSIBLE_INTEGRATION.md
|
||||
|
||||
# Keep these files:
|
||||
# - ANSIBLE_HANDOFF.md (reference for Ansible team)
|
||||
# - CUSTOMIZATION_CHECKLIST.md (configuration reference)
|
||||
# - docs/08_proxmox_deployment.md (manual deployment guide)
|
||||
|
||||
# Commit the changes
|
||||
git add -A
|
||||
git commit -m "Move Ansible code to infrastructure repository
|
||||
|
||||
The ansible/ directory has been moved to the Ansible infrastructure
|
||||
repository where it belongs. This repo now only contains:
|
||||
- Application code
|
||||
- Deployment documentation (ANSIBLE_HANDOFF.md)
|
||||
- Configuration reference (CUSTOMIZATION_CHECKLIST.md)
|
||||
- Manual deployment guide (docs/08_proxmox_deployment.md)
|
||||
|
||||
Ansible code now lives in: [your-ansible-repo]/roles/pote/"
|
||||
```
|
||||
|
||||
### Step 3: Update Your Ansible Repo
|
||||
|
||||
```bash
|
||||
# In your Ansible repository:
|
||||
|
||||
# Create the playbook
|
||||
cat > playbooks/deploy-pote.yml << 'EOF'
|
||||
---
|
||||
- name: Deploy POTE Application
|
||||
hosts: pote_servers
|
||||
become: yes
|
||||
|
||||
roles:
|
||||
- role: pote
|
||||
tags: ['pote', 'deploy']
|
||||
EOF
|
||||
|
||||
# Update inventory (add POTE servers)
|
||||
# Edit inventory/production.yml, inventory/staging.yml, etc.
|
||||
|
||||
# Create/update vault with POTE secrets
|
||||
ansible-vault edit group_vars/all/vault.yml
|
||||
# Add: vault_smtp_password, vault_git_ssh_key, vault_db_password_*, etc.
|
||||
|
||||
# Test deployment
|
||||
ansible-playbook playbooks/deploy-pote.yml --limit development --check
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 Final Directory Structure
|
||||
|
||||
### POTE Application Repo (this repo)
|
||||
|
||||
```
|
||||
pote/
|
||||
├── src/pote/ # Application code
|
||||
├── tests/ # Tests
|
||||
├── scripts/ # Deployment scripts
|
||||
├── docs/ # Documentation
|
||||
├── .github/workflows/ # CI/CD pipelines
|
||||
├── ANSIBLE_HANDOFF.md # ✅ Keep - Reference for Ansible team
|
||||
├── CUSTOMIZATION_CHECKLIST.md # ✅ Keep - Configuration guide
|
||||
├── MOVE_ANSIBLE_TO_SEPARATE_REPO.md # ✅ Keep - This file
|
||||
└── pyproject.toml # Python dependencies
|
||||
```
|
||||
|
||||
### Your Ansible Infrastructure Repo
|
||||
|
||||
```
|
||||
ansible-infrastructure/
|
||||
├── roles/
|
||||
│ ├── base_os/ # Your existing role
|
||||
│ ├── pote/ # ✅ NEW - Moved from POTE repo
|
||||
│ │ ├── defaults/main.yml
|
||||
│ │ ├── tasks/main.yml
|
||||
│ │ ├── templates/env.j2
|
||||
│ │ └── handlers/main.yml
|
||||
│ └── ...other roles...
|
||||
├── playbooks/
|
||||
│ ├── deploy-pote.yml # ✅ NEW - POTE deployment playbook
|
||||
│ └── ...other playbooks...
|
||||
├── group_vars/
|
||||
│ ├── all.yml # Merged POTE variables
|
||||
│ ├── all/vault.yml # Added POTE secrets
|
||||
│ ├── development.yml # Merged POTE dev config
|
||||
│ ├── staging.yml # Merged POTE staging config
|
||||
│ └── production.yml # Merged POTE prod config
|
||||
├── inventory/
|
||||
│ ├── production.yml # Added POTE servers
|
||||
│ ├── staging.yml # Added POTE servers
|
||||
│ └── development.yml # Added POTE servers
|
||||
└── examples/
|
||||
├── pote-inventory.example.yml
|
||||
└── pote-vault.example.yml
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Benefits of This Approach
|
||||
|
||||
1. **Cleaner Separation** - Application code separate from infrastructure code
|
||||
2. **No Circular Dependency** - Ansible can clone the app repo without needing to be inside it
|
||||
3. **Centralized Infrastructure** - All your Ansible roles in one place
|
||||
4. **Easier Management** - Update infrastructure without touching app code
|
||||
5. **Standard Practice** - Follows industry best practices
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Workflow After Migration
|
||||
|
||||
### Deploying POTE
|
||||
|
||||
```bash
|
||||
# From your Ansible repository:
|
||||
ansible-playbook playbooks/deploy-pote.yml --limit production
|
||||
```
|
||||
|
||||
### Updating POTE Application
|
||||
|
||||
```bash
|
||||
# Option 1: Via Ansible (recommended)
|
||||
ansible-playbook playbooks/deploy-pote.yml --limit production --tags update
|
||||
|
||||
# Option 2: Via Gitea Actions (CI/CD)
|
||||
# Push to main branch → Gitea Actions runs tests → Auto-deploy
|
||||
|
||||
# Option 3: Manual (SSH to server)
|
||||
cd ~/pote
|
||||
git pull origin main
|
||||
source venv/bin/activate
|
||||
pip install -e .
|
||||
alembic upgrade head
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 What to Tell Your Ansible Team
|
||||
|
||||
Send them:
|
||||
1. **ANSIBLE_HANDOFF.md** - Complete integration guide
|
||||
2. **CUSTOMIZATION_CHECKLIST.md** - What needs to be configured
|
||||
3. **ansible/** directory - The actual role code
|
||||
|
||||
They need to:
|
||||
1. Copy the role to their Ansible repo
|
||||
2. Merge the group_vars into their existing structure
|
||||
3. Add POTE servers to their inventory
|
||||
4. Add POTE secrets to their vault
|
||||
5. Test deployment to dev environment
|
||||
6. Deploy to staging, then production
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ Important Notes
|
||||
|
||||
### Don't Delete These from POTE Repo
|
||||
|
||||
Keep these files for reference:
|
||||
- `ANSIBLE_HANDOFF.md` - Ansible integration guide
|
||||
- `CUSTOMIZATION_CHECKLIST.md` - Configuration reference
|
||||
- `docs/08_proxmox_deployment.md` - Manual deployment guide
|
||||
- `scripts/proxmox_setup.sh` - Bootstrap script (if needed)
|
||||
|
||||
### Do Delete These from POTE Repo
|
||||
|
||||
Remove these (they belong in Ansible repo):
|
||||
- `ansible/` directory (entire thing)
|
||||
- `ANSIBLE_INTEGRATION.md` (redundant with ANSIBLE_HANDOFF.md)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Ready to Move?
|
||||
|
||||
Run these commands when ready:
|
||||
|
||||
```bash
|
||||
# 1. Backup first (just in case)
|
||||
cp -r ansible /tmp/pote-ansible-backup
|
||||
|
||||
# 2. Remove from POTE repo
|
||||
git rm -r ansible/
|
||||
git rm ANSIBLE_INTEGRATION.md
|
||||
git add -A
|
||||
git commit -m "Move Ansible code to infrastructure repository"
|
||||
git push origin dev
|
||||
|
||||
# 3. Copy to your Ansible repo (do this manually)
|
||||
# See "Step 1: Copy Ansible Code to Your Ansible Repo" above
|
||||
|
||||
# 4. Test deployment from Ansible repo
|
||||
# See "Step 3: Update Your Ansible Repo" above
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Questions?** See `ANSIBLE_HANDOFF.md` for complete integration details.
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user