Use python3 instead of unavailable python3.11 packages, drop apt step that failed on runner image, and replace buildx with docker build so the test step can run the built image.
Remove Python job containers so checkout@v4 can run (needs Node), install deps on ubuntu-latest for security-scan, and stop excluding README.md from Docker build context.
Switch HouseWatcherClient to public STOCK Act JSON (legacy housestockwatcher.com is down), add httpx/scikit-learn to pyproject, wire SMTP settings through Settings, fix get_session() as a context manager, and use savepoints in TradeLoader so one bad row does not roll back the batch.
NEW FEATURES:
============
📁 GITEA_SECRETS_GUIDE.md:
- Comprehensive guide on using Gitea secrets
- Store passwords in Gitea (not in git!)
- Use in CI/CD and deployment workflows
- Best practices and security recommendations
🔧 .github/workflows/ci.yml (UPDATED):
- Now uses Gitea secrets with fallbacks
- ${{ secrets.SMTP_PASSWORD || 'testpass123' }}
- ${{ secrets.DB_PASSWORD || 'testpass123' }}
- Tests run with real credentials from Gitea
🚀 .github/workflows/deploy.yml (NEW):
- Automated deployment to Proxmox
- Manual trigger via Gitea UI
- Steps:
1. SSH to Proxmox with secrets.PROXMOX_SSH_KEY
2. Pull latest code
3. Update .env with secrets from Gitea
4. Run migrations
5. Health check
6. Test email
7. Rollback on failure
HOW IT WORKS:
=============
1. Store passwords in Gitea (Settings → Secrets)
2. CI/CD uses secrets automatically
3. Deployment workflow updates .env on Proxmox
4. Best of both worlds: secure CI + simple runtime
SECRETS TO ADD IN GITEA:
========================
- SMTP_PASSWORD: your mail password
- DB_PASSWORD: changeme123
- PROXMOX_HOST: 10.0.10.95
- PROXMOX_USER: poteapp
- PROXMOX_SSH_KEY: (SSH private key)
- SMTP_HOST: mail.levkin.ca
- SMTP_USER: test@levkin.ca
- FROM_EMAIL: test@levkin.ca
USAGE:
======
# In Gitea UI:
Actions → Deploy to Proxmox → Run workflow
# Or push commits:
git push origin main
# CI runs with secrets automatically
See GITEA_SECRETS_GUIDE.md for full instructions!
Covers 6 options for storing passwords securely:
1. .env file (current, good for personal use)
2. Environment variables (better for production)
3. Separate secrets file
4. Docker secrets
5. HashiCorp Vault (enterprise)
6. Git secrets (CI/CD only)
Recommendation: Current .env setup is fine for personal/research use
Improvement: chmod 600 .env (done)
Includes security checklist, rotation procedures, and testing
One-page reference with:
- Your specific configuration (test@levkin.ca)
- 3-step setup (30 seconds + 1 min + 2 min)
- Quick commands for deployed server
- Troubleshooting checklist
- Current status tracker
- Next actions
Everything ready - just needs password in .env
Answers user's questions:
- What happens after deployment? (nothing automatic by default)
- How to get reports? (3 options: email, SSH, future web UI)
- Where are reports sent? (email or saved to ~/logs/)
- Do you need to check IP? (depends on setup method)
- Can we setup email reports? (YES! 5-minute setup)
- Do we need CI/CD pipelines? (optional, but included)
- Can we use existing Ansible pipeline? (concepts reused, not directly)
This document ties everything together and provides clear next steps.
New Features:
- Watchlist system for tracking specific Congress members
- Trading report generation with multiple formats
- Pre-market-close automated updates (3 PM)
New Scripts:
- scripts/fetch_congress_members.py: Manage watchlist
* 29 known active traders (curated list)
* Optional ProPublica API integration (all 535 members)
* Create/view/manage watchlist
- scripts/generate_trading_report.py: Generate trading reports
* Filter by watchlist or show all
* Multiple formats: text, HTML, JSON
* Summary statistics (buys/sells, top tickers)
* Color-coded output (🟢 BUY, 🔴 SELL)
- scripts/pre_market_close_update.sh: 3 PM automation
* Quick fetch of latest trades
* Enrichment of new securities
* Generate and display report
* Saves to reports/ directory
Documentation:
- WATCHLIST_GUIDE.md: Complete guide
* List of 29 known active traders
* How to create/customize watchlist
* Schedule options (pre-market, post-market)
* Email setup (optional)
* FAQ and examples
Known Active Traders Include:
Senate: Tuberville, Rand Paul, Mark Warner, Rick Scott
House: Pelosi, Crenshaw, MTG, Gottheimer, Brian Higgins
Use Cases:
✅ Daily reports at 3 PM (1 hour before close)
✅ See what Congress bought/sold recently
✅ Track specific members you care about
✅ Export to HTML/JSON for further analysis
Critical Fixes:
- Fixed Price model query to use security_id join with Security
- Added Security import to returns.py module
- Fixed all test fixtures to use test_db_session correctly
- Added AAPL price data to sample_prices fixture
New Tests:
- tests/test_analytics_integration.py: 10 comprehensive integration tests
* Real-world scenarios with synthetic price data
* Return calculations, benchmark comparisons, performance metrics
* Edge cases: missing data, sell trades, disclosure timing
Documentation:
- LOCAL_TEST_GUIDE.md: Complete guide for local testing
* How to test before deploying
* Current data status (live vs fixtures)
* Multiple options for getting real data
* Common issues and fixes
Test Results:
✅ All 55 tests passing
✅ Analytics fully functional
✅ Ready for deployment
Live Data Status:
❌ House Stock Watcher API still down (external issue)
✅ Manual CSV import works
✅ yfinance for prices works
✅ Can use system NOW with manual data
- Replace 'sudo -u' with 'su -' throughout the script
- Works perfectly in LXC containers running as root (no sudo installed)
- Also works on regular VMs/servers where sudo is available
- Fixes all remaining: sudo: command not found errors
ilia
ilia