Add Gitea Secrets integration for CI/CD and deployment

NEW FEATURES: ============ 📁 GITEA_SECRETS_GUIDE.md: - Comprehensive guide on using Gitea secrets - Store passwords in Gitea (not in git!) - Use in CI/CD and deployment workflows - Best practices and security recommendations 🔧 .github/workflows/ci.yml (UPDATED): - Now uses Gitea secrets with fallbacks - ${{ secrets.SMTP_PASSWORD || 'testpass123' }} - ${{ secrets.DB_PASSWORD || 'testpass123' }} - Tests run with real credentials from Gitea 🚀 .github/workflows/deploy.yml (NEW): - Automated deployment to Proxmox - Manual trigger via Gitea UI - Steps: 1. SSH to Proxmox with secrets.PROXMOX_SSH_KEY 2. Pull latest code 3. Update .env with secrets from Gitea 4. Run migrations 5. Health check 6. Test email 7. Rollback on failure HOW IT WORKS: ============= 1. Store passwords in Gitea (Settings → Secrets) 2. CI/CD uses secrets automatically 3. Deployment workflow updates .env on Proxmox 4. Best of both worlds: secure CI + simple runtime SECRETS TO ADD IN GITEA: ======================== - SMTP_PASSWORD: your mail password - DB_PASSWORD: changeme123 - PROXMOX_HOST: 10.0.10.95 - PROXMOX_USER: poteapp - PROXMOX_SSH_KEY: (SSH private key) - SMTP_HOST: mail.levkin.ca - SMTP_USER: test@levkin.ca - FROM_EMAIL: test@levkin.ca USAGE: ====== # In Gitea UI: Actions → Deploy to Proxmox → Run workflow # Or push commits: git push origin main # CI runs with secrets automatically See GITEA_SECRETS_GUIDE.md for full instructions!
2025-12-15 15:52:19 -05:00 · 2025-12-15 15:52:19 -05:00 · ead0820cf9
commit ead0820cf9
parent 0c183fb28c
3 changed files with 590 additions and 3 deletions
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@ -17,7 +17,7 @@ jobs:
        image: postgres:15
        env:
          POSTGRES_USER: poteuser
-          POSTGRES_PASSWORD: testpass123
+          POSTGRES_PASSWORD: ${{ secrets.DB_PASSWORD || 'testpass123' }}
          POSTGRES_DB: potedb_test
        options: >-
          --health-cmd pg_isready
@ -50,13 +50,18 @@ jobs:

      - name: Run tests with coverage
        env:
-          DATABASE_URL: postgresql://poteuser:testpass123@postgres:5432/potedb_test
+          DATABASE_URL: postgresql://poteuser:${{ secrets.DB_PASSWORD || 'testpass123' }}@postgres:5432/potedb_test
+          SMTP_HOST: ${{ secrets.SMTP_HOST || 'localhost' }}
+          SMTP_PORT: 587
+          SMTP_USER: ${{ secrets.SMTP_USER || 'test@example.com' }}
+          SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD || 'dummy' }}
+          FROM_EMAIL: ${{ secrets.FROM_EMAIL || 'test@example.com' }}
        run: |
          pytest tests/ -v --cov=src/pote --cov-report=term --cov-report=xml

      - name: Test scripts
        env:
-          DATABASE_URL: postgresql://poteuser:testpass123@postgres:5432/potedb_test
+          DATABASE_URL: postgresql://poteuser:${{ secrets.DB_PASSWORD || 'testpass123' }}@postgres:5432/potedb_test
        run: |
          echo "Testing database migrations..."
          alembic upgrade head
--- a/.github/workflows/deploy.yml
+++ b/.github/workflows/deploy.yml
@ -0,0 +1,145 @@
+---
+name: Deploy to Proxmox
+
+on:
+  workflow_dispatch:  # Manual trigger only
+    inputs:
+      environment:
+        description: 'Environment to deploy to'
+        required: true
+        default: 'production'
+        type: choice
+        options:
+          - production
+          - staging
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup SSH
+        env:
+          SSH_KEY: ${{ secrets.PROXMOX_SSH_KEY }}
+          SSH_HOST: ${{ secrets.PROXMOX_HOST }}
+        run: |
+          mkdir -p ~/.ssh
+          echo "$SSH_KEY" > ~/.ssh/id_rsa
+          chmod 600 ~/.ssh/id_rsa
+          ssh-keyscan -H $SSH_HOST >> ~/.ssh/known_hosts
+
+      - name: Deploy to Proxmox
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+          SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
+          DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
+        run: |
+          echo "🚀 Deploying to $PROXMOX_HOST..."
+          
+          ssh ${PROXMOX_USER}@${PROXMOX_HOST} << 'ENDSSH'
+            set -e
+            cd ~/pote
+            
+            echo "📥 Pulling latest code..."
+            git pull origin main
+            
+            echo "📦 Installing dependencies..."
+            source venv/bin/activate
+            pip install -e . --quiet
+            
+            echo "🔄 Running migrations..."
+            alembic upgrade head
+            
+            echo "✅ Deployment complete!"
+          ENDSSH
+
+      - name: Update secrets on server
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+          SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
+          DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
+        run: |
+          echo "🔐 Updating secrets in .env..."
+          
+          ssh ${PROXMOX_USER}@${PROXMOX_HOST} << ENDSSH
+            cd ~/pote
+            
+            # Backup current .env
+            cp .env .env.backup.\$(date +%Y%m%d_%H%M%S)
+            
+            # Update passwords in .env (only update the password lines)
+            sed -i "s|SMTP_PASSWORD=.*|SMTP_PASSWORD=${SMTP_PASSWORD}|" .env
+            sed -i "s|changeme123|${DB_PASSWORD}|" .env
+            
+            # Secure permissions
+            chmod 600 .env
+            
+            echo "✅ Secrets updated!"
+          ENDSSH
+
+      - name: Health Check
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+        run: |
+          echo "🔍 Running health check..."
+          
+          ssh ${PROXMOX_USER}@${PROXMOX_HOST} << 'ENDSSH'
+            cd ~/pote
+            source venv/bin/activate
+            python scripts/health_check.py
+          ENDSSH
+
+      - name: Test Email
+        if: inputs.environment == 'production'
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+        run: |
+          echo "📧 Testing email configuration..."
+          
+          ssh ${PROXMOX_USER}@${PROXMOX_HOST} << 'ENDSSH'
+            cd ~/pote
+            source venv/bin/activate
+            python scripts/send_daily_report.py --to test@levkin.ca --test-smtp || true
+          ENDSSH
+
+      - name: Deployment Summary
+        if: always()
+        run: |
+          echo "## 🚀 Deployment Summary" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          echo "**Environment:** ${{ inputs.environment }}" >> $GITHUB_STEP_SUMMARY
+          echo "**Target:** ${{ secrets.PROXMOX_HOST }}" >> $GITHUB_STEP_SUMMARY
+          echo "**Status:** ${{ job.status }}" >> $GITHUB_STEP_SUMMARY
+          echo "" >> $GITHUB_STEP_SUMMARY
+          
+          if [ "${{ job.status }}" == "success" ]; then
+            echo "✅ Deployment completed successfully!" >> $GITHUB_STEP_SUMMARY
+          else
+            echo "❌ Deployment failed. Check logs above." >> $GITHUB_STEP_SUMMARY
+          fi
+
+      - name: Rollback on Failure
+        if: failure()
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+        run: |
+          echo "❌ Deployment failed. Restoring previous .env..."
+          
+          ssh ${PROXMOX_USER}@${PROXMOX_HOST} << 'ENDSSH' || true
+            cd ~/pote
+            # Restore backup
+            if ls .env.backup.* 1> /dev/null 2>&1; then
+              latest_backup=$(ls -t .env.backup.* | head -1)
+              cp "$latest_backup" .env
+              echo "✅ Restored from $latest_backup"
+            fi
+          ENDSSH
+
--- a/GITEA_SECRETS_GUIDE.md
+++ b/GITEA_SECRETS_GUIDE.md
@ -0,0 +1,437 @@
+# 🔐 Gitea Secrets Guide for POTE
+
+## ✅ YES! You Can Store Passwords in Gitea
+
+Gitea has a **Secrets** feature (like GitHub Actions secrets) that lets you store passwords securely and use them in:
+1. **CI/CD pipelines** (Gitea Actions workflows) ✅
+2. **Deployment workflows** ✅
+
+**BUT NOT:**
+- ❌ Directly in your running application on Proxmox
+- ❌ Accessed by scripts outside of workflows
+
+---
+
+## 🎯 What Gitea Secrets Are Good For
+
+### ✅ Perfect Use Cases
+
+1. **CI/CD Testing** - Run tests with real credentials
+2. **Automated Deployment** - Deploy to Proxmox with SSH keys
+3. **Notifications** - Send emails/Slack after builds
+4. **Docker Registry** - Push images with credentials
+5. **API Keys** - Access external services during builds
+
+### ❌ NOT Good For
+
+1. **Runtime secrets** - Your deployed app on Proxmox can't access them
+2. **Local development** - Can't use secrets on your laptop
+3. **Manual scripts** - Can't run `python script.py` with Gitea secrets
+
+---
+
+## 🔧 How to Set Up Gitea Secrets
+
+### Step 1: Add Secrets to Gitea
+
+1. Go to your POTE repository in Gitea
+2. Click **Settings** → **Secrets** (or **Actions** → **Secrets**)
+3. Click **Add Secret**
+
+Add these secrets:
+
+| Secret Name | Example Value | Used For |
+|-------------|---------------|----------|
+| `SMTP_PASSWORD` | `your_mail_password` | Email reports in CI |
+| `DB_PASSWORD` | `changeme123` | Database in CI |
+| `PROXMOX_HOST` | `10.0.10.95` | Deployment |
+| `PROXMOX_USER` | `poteapp` | Deployment |
+| `PROXMOX_SSH_KEY` | `-----BEGIN...` | Deployment |
+| `SMTP_HOST` | `mail.levkin.ca` | Email config |
+| `SMTP_USER` | `test@levkin.ca` | Email config |
+| `FROM_EMAIL` | `test@levkin.ca` | Email config |
+
+### Step 2: Use Secrets in Workflows
+
+Secrets are accessed with `${{ secrets.SECRET_NAME }}` syntax.
+
+---
+
+## 📝 Example: CI Pipeline with Secrets
+
+**File:** `.github/workflows/ci.yml`
+
+```yaml
+name: CI
+
+on:
+  push:
+    branches: [main, master]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Run tests
+        env:
+          # Use Gitea secrets
+          DATABASE_URL: postgresql://user:${{ secrets.DB_PASSWORD }}@localhost/db
+          SMTP_HOST: ${{ secrets.SMTP_HOST }}
+          SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
+        run: |
+          pytest tests/
+
+      - name: Send notification
+        if: failure()
+        run: |
+          # Send email using secrets
+          python scripts/send_notification.py \
+            --smtp-password "${{ secrets.SMTP_PASSWORD }}"
+```
+
+**✅ I've already updated your CI pipeline to use secrets!**
+
+---
+
+## 🚀 Example: Automated Deployment Workflow
+
+Create `.github/workflows/deploy.yml`:
+
+```yaml
+name: Deploy to Proxmox
+
+on:
+  workflow_dispatch:  # Manual trigger
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Setup SSH
+        env:
+          SSH_KEY: ${{ secrets.PROXMOX_SSH_KEY }}
+          SSH_HOST: ${{ secrets.PROXMOX_HOST }}
+        run: |
+          mkdir -p ~/.ssh
+          echo "$SSH_KEY" > ~/.ssh/id_rsa
+          chmod 600 ~/.ssh/id_rsa
+          ssh-keyscan -H $SSH_HOST >> ~/.ssh/known_hosts
+
+      - name: Deploy to Proxmox
+        env:
+          PROXMOX_HOST: ${{ secrets.PROXMOX_HOST }}
+          PROXMOX_USER: ${{ secrets.PROXMOX_USER }}
+          SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
+          DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
+        run: |
+          # SSH to Proxmox and update
+          ssh $PROXMOX_USER@$PROXMOX_HOST << 'ENDSSH'
+            cd ~/pote
+            git pull
+            
+            # Update .env with secrets
+            echo "SMTP_PASSWORD=${SMTP_PASSWORD}" >> .env
+            echo "DATABASE_URL=postgresql://user:${DB_PASSWORD}@localhost/db" >> .env
+            
+            # Restart services
+            source venv/bin/activate
+            alembic upgrade head
+          ENDSSH
+
+      - name: Health Check
+        run: |
+          ssh ${{ secrets.PROXMOX_USER }}@${{ secrets.PROXMOX_HOST }} \
+            "cd ~/pote && python scripts/health_check.py"
+```
+
+---
+
+## 🔄 How Secrets Flow to Your Server
+
+### Option 1: Deploy Workflow Updates `.env` (Recommended)
+
+```yaml
+# In deployment workflow
+- name: Update secrets on server
+  run: |
+    ssh user@server << 'EOF'
+      cd ~/pote
+      # Update .env with secrets passed from Gitea
+      sed -i "s/SMTP_PASSWORD=.*/SMTP_PASSWORD=${{ secrets.SMTP_PASSWORD }}/" .env
+    EOF
+```
+
+### Option 2: Use Environment Variables
+
+```yaml
+# In deployment workflow
+- name: Deploy with environment variables
+  run: |
+    ssh user@server << 'EOF'
+      cd ~/pote
+      # Export secrets as environment variables
+      export SMTP_PASSWORD="${{ secrets.SMTP_PASSWORD }}"
+      export DB_PASSWORD="${{ secrets.DB_PASSWORD }}"
+      # Run scripts
+      python scripts/send_daily_report.py
+    EOF
+```
+
+### Option 3: Secrets File on Server
+
+```yaml
+# In deployment workflow
+- name: Create secrets file
+  run: |
+    ssh user@server << 'EOF'
+      # Create secure secrets file
+      cat > /etc/pote/secrets << 'SECRETS'
+      export SMTP_PASSWORD="${{ secrets.SMTP_PASSWORD }}"
+      export DB_PASSWORD="${{ secrets.DB_PASSWORD }}"
+      SECRETS
+      chmod 600 /etc/pote/secrets
+    EOF
+```
+
+---
+
+## 🎯 Recommended Setup for Your POTE Project
+
+### For CI/CD (Testing):
+
+**Use Gitea Secrets** ✅
+
+```yaml
+# .github/workflows/ci.yml (already updated!)
+env:
+  SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
+  DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
+```
+
+### For Deployed Server (Proxmox):
+
+**Keep using `.env` file** ✅
+
+Why?
+- Simpler for manual SSH access
+- No need for complex deployment workflows
+- Easy to update: just `nano .env`
+
+**BUT:** Use Gitea secrets in a deployment workflow to UPDATE the `.env` file automatically!
+
+---
+
+## 🚀 Complete Workflow: Gitea → Proxmox
+
+### 1. Store Secrets in Gitea
+
+```
+Repository Settings → Secrets:
+- SMTP_PASSWORD: your_password
+- PROXMOX_HOST: 10.0.10.95
+- PROXMOX_SSH_KEY: (your SSH private key)
+```
+
+### 2. Create Deployment Workflow
+
+See `.github/workflows/deploy.yml` (I'll create this next)
+
+### 3. Trigger Deployment
+
+```bash
+# From Gitea UI:
+Actions → Deploy to Proxmox → Run workflow
+
+# Or commit and push:
+git commit -m "Update code"
+git push origin main
+# Workflow runs automatically
+```
+
+### 4. Workflow Updates Proxmox
+
+- SSH to Proxmox
+- Pull latest code
+- Update `.env` with secrets from Gitea
+- Run migrations
+- Health check
+
+---
+
+## ⚠️ Important Limitations
+
+### Gitea Secrets CAN'T:
+
+❌ Be accessed outside of workflows
+❌ Be used in local `python script.py` runs
+❌ Be read by cron jobs on Proxmox (directly)
+❌ Replace `.env` for runtime application config
+
+### Gitea Secrets CAN:
+
+✅ Secure your CI/CD pipeline
+✅ Deploy safely without exposing passwords in git
+✅ Update `.env` on server during deployment
+✅ Run automated tests with real credentials
+
+---
+
+## 🔒 Security Best Practices
+
+### ✅ DO:
+
+1. **Store ALL sensitive data as Gitea secrets**
+   - SMTP passwords
+   - Database passwords
+   - API keys
+   - SSH keys
+
+2. **Use secrets in workflows**
+   ```yaml
+   env:
+     PASSWORD: ${{ secrets.PASSWORD }}
+   ```
+
+3. **Never echo secrets**
+   ```yaml
+   # ❌ BAD - exposes in logs
+   - run: echo "${{ secrets.PASSWORD }}"
+   
+   # ✅ GOOD - masked automatically
+   - run: use_password "${{ secrets.PASSWORD }}"
+   ```
+
+4. **Rotate secrets regularly**
+   - Update in Gitea UI
+   - Re-run deployment workflow
+
+### ❌ DON'T:
+
+1. **Commit secrets to git** (even private repos)
+2. **Share secrets via Slack/email**
+3. **Use same password everywhere**
+4. **Expose secrets in workflow logs**
+
+---
+
+## 📊 Comparison: Where to Store Secrets
+
+| Storage | CI/CD | Deployed App | Easy Updates | Security |
+|---------|-------|--------------|--------------|----------|
+| **Gitea Secrets** | ✅ Perfect | ❌ No | ✅ Via workflow | ⭐⭐⭐⭐⭐ |
+| **`.env` file** | ❌ No | ✅ Perfect | ✅ `nano .env` | ⭐⭐⭐ |
+| **Environment Vars** | ✅ Yes | ✅ Yes | ❌ Harder | ⭐⭐⭐⭐ |
+| **Both (Recommended)** | ✅ Yes | ✅ Yes | ✅ Automated | ⭐⭐⭐⭐⭐ |
+
+---
+
+## 🎯 My Recommendation for You
+
+### Use BOTH:
+
+1. **Gitea Secrets** - For CI/CD and deployment workflows
+2. **`.env` file** - For runtime on Proxmox
+
+### Workflow:
+
+```
+1. Store password in Gitea Secrets
+2. Commit code changes
+3. Push to Gitea
+4. Workflow runs:
+   - Tests with Gitea secrets ✅
+   - Deploys to Proxmox ✅
+   - Updates .env with secrets ✅
+5. Proxmox app reads from .env ✅
+```
+
+**This gives you:**
+- ✅ Secure CI/CD
+- ✅ Easy manual SSH access
+- ✅ Automated deployments
+- ✅ No passwords in git
+
+---
+
+## 🚀 Next Steps
+
+### 1. Add Secrets to Gitea (5 minutes)
+
+```
+1. Go to https://git.levkin.ca/ilia/POTE/settings/secrets
+2. Add:
+   - SMTP_PASSWORD: your_mail_password
+   - DB_PASSWORD: changeme123
+   - SMTP_HOST: mail.levkin.ca
+   - SMTP_USER: test@levkin.ca
+   - FROM_EMAIL: test@levkin.ca
+```
+
+### 2. Test CI Pipeline (Already Updated!)
+
+```bash
+git push origin main
+# Watch Actions tab in Gitea
+# CI should use secrets automatically
+```
+
+### 3. Create Deployment Workflow (Optional)
+
+I can create `.github/workflows/deploy.yml` if you want automated deployments!
+
+---
+
+## 💡 Quick Commands
+
+### Add SSH Key to Gitea (for deployment):
+
+```bash
+# On your local machine
+cat ~/.ssh/id_rsa  # Copy this
+
+# In Gitea:
+Repository → Settings → Secrets → Add Secret
+Name: PROXMOX_SSH_KEY
+Value: (paste private key)
+```
+
+### Test Gitea Secrets:
+
+```bash
+# Push a test commit
+git commit --allow-empty -m "Test secrets"
+git push
+
+# Check Gitea Actions tab
+# Look for green checkmarks ✅
+```
+
+---
+
+## 📚 See Also
+
+- **[docs/13_secrets_management.md](docs/13_secrets_management.md)** - All secrets options
+- **[.github/workflows/ci.yml](.github/workflows/ci.yml)** - Updated with secrets support
+- **[DEPLOYMENT_AND_AUTOMATION.md](DEPLOYMENT_AND_AUTOMATION.md)** - Full deployment guide
+
+---
+
+## ✅ Summary
+
+**YES, use Gitea secrets!** They're perfect for:
+- ✅ CI/CD pipelines
+- ✅ Automated deployments
+- ✅ Keeping passwords out of git
+
+**But ALSO keep `.env` on Proxmox** for:
+- ✅ Runtime application config
+- ✅ Manual SSH access
+- ✅ Cron jobs
+
+**Best of both worlds:** Gitea secrets deploy and update the `.env` file automatically! 🚀