ISSUE:
======
- CI job stuck waiting for tzdata timezone configuration
- Interactive prompt blocks non-interactive CI environment
- Job running for 30+ minutes waiting for user input
FIX:
====
- Set DEBIAN_FRONTEND=noninteractive in all apt-get steps
- Pre-configure timezone to UTC before package installation
- Prevents interactive prompts during package configuration
CHANGES:
========
- Install Node.js step: Added DEBIAN_FRONTEND=noninteractive
- Install Java step: Added DEBIAN_FRONTEND=noninteractive
- Install Python 3.11 step: Added DEBIAN_FRONTEND=noninteractive + TZ=UTC
- Set timezone symlink before apt-get install to avoid tzdata prompt
This ensures all package installations run non-interactively.
ISSUE:
======
- Ubuntu 22.04 comes with Python 3.10
- Project requires Python >=3.11 (pyproject.toml)
- Error: Package 'pote' requires a different Python: 3.10.12 not in '>=3.11'
FIX:
====
- Install Python 3.11 from deadsnakes PPA
- Install pip for Python 3.11
- Use python3.11 -m pip explicitly for all pip operations
- Use python3.11 -m pytest for test execution
CHANGES:
========
- Install software-properties-common for add-apt-repository
- Add deadsnakes PPA for Python 3.11
- Install python3.11, python3.11-dev, python3.11-venv, python3.11-distutils
- Install pip via get-pip.py for Python 3.11
- Update all pip/pytest commands to use python3.11 -m
This ensures the sonar-analysis job uses Python 3.11 as required.
FIXES:
======
1. ✅ Added httpx to dependencies
- Required by house_watcher.py
- Fixes: ModuleNotFoundError: No module named 'httpx'
2. ✅ Install Python/pip in sonar-analysis job
- Ubuntu container doesn't have Python by default
- Fixes: pip: not found error
3. ✅ Remove README.md from Dockerfile
- README.md is excluded by .dockerignore
- Not needed for container to run
- Fixes: Docker build error
CHANGES:
========
- pyproject.toml: Added httpx>=0.24 to dependencies
- ci.yml: Added Python/pip installation step before coverage
- Dockerfile: Removed README.md copy (excluded by .dockerignore)
- .dockerignore: Kept *.md exclusion (docs not needed in container)
All CI jobs should now pass!
ISSUE:
======
Gitea Actions containers need Node.js to run actions/checkout@v4
Error: 'exec: "node": executable file not found in $PATH'
FIX:
====
Added Node.js installation step before checkout in all jobs:
For Debian/Ubuntu containers (python:3.11-bullseye, ubuntu:22.04):
apt-get update && apt-get install -y curl git
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs
For Alpine containers (gitleaks, trivy):
apk add --no-cache nodejs npm curl git
JOBS FIXED:
===========
✅ lint-and-test
✅ secret-scanning
✅ security-scan
✅ dependency-scan
✅ sast-scan
✅ container-scan
✅ docker-build-test (no container, already works)
This is a common requirement for Gitea Actions when using
custom containers.
REMOVED:
========
❌ ANSIBLE_HANDOFF.md - Ansible team will get from git history
❌ ANSIBLE_TECHNICAL_REFERENCE.md - Ansible team will get from git history
❌ MOVE_ANSIBLE_TO_SEPARATE_REPO.md - Migration complete, no longer needed
❌ SETUP_COMPLETE.md - Outdated summary
❌ TESTING_STATUS.md - Outdated status
RATIONALE:
==========
Ansible code has been removed from this repo and belongs in
infrastructure repo. These docs were for the migration process
and are no longer relevant to the POTE application repo.
Ansible team can retrieve these files from git history if needed:
git show d40b412:ANSIBLE_HANDOFF.md
git show d40b412:ANSIBLE_TECHNICAL_REFERENCE.md
KEPT:
=====
✅ CUSTOMIZATION_CHECKLIST.md - Still useful for config reference
✅ CI_PIPELINE_COMPLETE.md - Current CI documentation
✅ All quickstart guides
✅ All deployment docs
NEW FILES:
==========
📄 CUSTOMIZATION_CHECKLIST.md
- Complete list of everything that needs customization
- Organized by priority: Critical, Important, Optional
- Covers .env, Ansible, Gitea secrets, email, etc.
- Quick action checklist for deployment
📄 ANSIBLE_HANDOFF.md
- Guide for integrating POTE with existing Ansible system
- Explains what Ansible needs to know
- Variable reference and secrets management
- Multi-environment deployment strategy
- Example playbook and testing instructions
📄 MOVE_ANSIBLE_TO_SEPARATE_REPO.md
- Explains why ansible/ should be in infrastructure repo
- Step-by-step migration guide
- Final directory structure for both repos
- Benefits and workflow after migration
KEY INSIGHT:
============
The ansible/ directory doesn't belong in the POTE app repo because:
- Ansible runs BEFORE the app exists (creates container, deploys app)
- Creates circular dependency (Ansible clones repo that contains Ansible)
- Should live in centralized infrastructure repository
NEXT STEPS:
===========
1. Review CUSTOMIZATION_CHECKLIST.md for deployment config
2. Copy ansible/ to infrastructure repo
3. Remove ansible/ from POTE repo (keep handoff docs)
4. Deploy via centralized Ansible system
✅ POTE SETUP COMPLETE!
What's Done:
- Branch strategy (main/qa/dev)
- Branch protection with CI checks
- Complete CI/CD pipeline
- Comprehensive documentation
- Email reporting system
- Market monitoring (3 phases)
- Ready for Ansible integration
Next Steps:
1. Merge documentation to main
2. Add Gitea secrets
3. Test the workflow
4. Set up email automation
Total Achievement:
- 93 tests passing
- Professional dev workflow
- Production-ready system
NEW FEATURES:
============
📁 GITEA_SECRETS_GUIDE.md:
- Comprehensive guide on using Gitea secrets
- Store passwords in Gitea (not in git!)
- Use in CI/CD and deployment workflows
- Best practices and security recommendations
🔧 .github/workflows/ci.yml (UPDATED):
- Now uses Gitea secrets with fallbacks
- ${{ secrets.SMTP_PASSWORD || 'testpass123' }}
- ${{ secrets.DB_PASSWORD || 'testpass123' }}
- Tests run with real credentials from Gitea
🚀 .github/workflows/deploy.yml (NEW):
- Automated deployment to Proxmox
- Manual trigger via Gitea UI
- Steps:
1. SSH to Proxmox with secrets.PROXMOX_SSH_KEY
2. Pull latest code
3. Update .env with secrets from Gitea
4. Run migrations
5. Health check
6. Test email
7. Rollback on failure
HOW IT WORKS:
=============
1. Store passwords in Gitea (Settings → Secrets)
2. CI/CD uses secrets automatically
3. Deployment workflow updates .env on Proxmox
4. Best of both worlds: secure CI + simple runtime
SECRETS TO ADD IN GITEA:
========================
- SMTP_PASSWORD: your mail password
- DB_PASSWORD: changeme123
- PROXMOX_HOST: 10.0.10.95
- PROXMOX_USER: poteapp
- PROXMOX_SSH_KEY: (SSH private key)
- SMTP_HOST: mail.levkin.ca
- SMTP_USER: test@levkin.ca
- FROM_EMAIL: test@levkin.ca
USAGE:
======
# In Gitea UI:
Actions → Deploy to Proxmox → Run workflow
# Or push commits:
git push origin main
# CI runs with secrets automatically
See GITEA_SECRETS_GUIDE.md for full instructions!
Covers 6 options for storing passwords securely:
1. .env file (current, good for personal use)
2. Environment variables (better for production)
3. Separate secrets file
4. Docker secrets
5. HashiCorp Vault (enterprise)
6. Git secrets (CI/CD only)
Recommendation: Current .env setup is fine for personal/research use
Improvement: chmod 600 .env (done)
Includes security checklist, rotation procedures, and testing
One-page reference with:
- Your specific configuration (test@levkin.ca)
- 3-step setup (30 seconds + 1 min + 2 min)
- Quick commands for deployed server
- Troubleshooting checklist
- Current status tracker
- Next actions
Everything ready - just needs password in .env
Answers user's questions:
- What happens after deployment? (nothing automatic by default)
- How to get reports? (3 options: email, SSH, future web UI)
- Where are reports sent? (email or saved to ~/logs/)
- Do you need to check IP? (depends on setup method)
- Can we setup email reports? (YES! 5-minute setup)
- Do we need CI/CD pipelines? (optional, but included)
- Can we use existing Ansible pipeline? (concepts reused, not directly)
This document ties everything together and provides clear next steps.
New Features:
- Watchlist system for tracking specific Congress members
- Trading report generation with multiple formats
- Pre-market-close automated updates (3 PM)
New Scripts:
- scripts/fetch_congress_members.py: Manage watchlist
* 29 known active traders (curated list)
* Optional ProPublica API integration (all 535 members)
* Create/view/manage watchlist
- scripts/generate_trading_report.py: Generate trading reports
* Filter by watchlist or show all
* Multiple formats: text, HTML, JSON
* Summary statistics (buys/sells, top tickers)
* Color-coded output (🟢 BUY, 🔴 SELL)
- scripts/pre_market_close_update.sh: 3 PM automation
* Quick fetch of latest trades
* Enrichment of new securities
* Generate and display report
* Saves to reports/ directory
Documentation:
- WATCHLIST_GUIDE.md: Complete guide
* List of 29 known active traders
* How to create/customize watchlist
* Schedule options (pre-market, post-market)
* Email setup (optional)
* FAQ and examples
Known Active Traders Include:
Senate: Tuberville, Rand Paul, Mark Warner, Rick Scott
House: Pelosi, Crenshaw, MTG, Gottheimer, Brian Higgins
Use Cases:
✅ Daily reports at 3 PM (1 hour before close)
✅ See what Congress bought/sold recently
✅ Track specific members you care about
✅ Export to HTML/JSON for further analysis
Critical Fixes:
- Fixed Price model query to use security_id join with Security
- Added Security import to returns.py module
- Fixed all test fixtures to use test_db_session correctly
- Added AAPL price data to sample_prices fixture
New Tests:
- tests/test_analytics_integration.py: 10 comprehensive integration tests
* Real-world scenarios with synthetic price data
* Return calculations, benchmark comparisons, performance metrics
* Edge cases: missing data, sell trades, disclosure timing
Documentation:
- LOCAL_TEST_GUIDE.md: Complete guide for local testing
* How to test before deploying
* Current data status (live vs fixtures)
* Multiple options for getting real data
* Common issues and fixes
Test Results:
✅ All 55 tests passing
✅ Analytics fully functional
✅ Ready for deployment
Live Data Status:
❌ House Stock Watcher API still down (external issue)
✅ Manual CSV import works
✅ yfinance for prices works
✅ Can use system NOW with manual data
- Replace 'sudo -u' with 'su -' throughout the script
- Works perfectly in LXC containers running as root (no sudo installed)
- Also works on regular VMs/servers where sudo is available
- Fixes all remaining: sudo: command not found errors
