Remove Python job containers so checkout@v4 can run (needs Node), install deps on ubuntu-latest for security-scan, and stop excluding README.md from Docker build context.
Switch HouseWatcherClient to public STOCK Act JSON (legacy housestockwatcher.com is down), add httpx/scikit-learn to pyproject, wire SMTP settings through Settings, fix get_session() as a context manager, and use savepoints in TradeLoader so one bad row does not roll back the batch.
NEW FEATURES:
============
📁 GITEA_SECRETS_GUIDE.md:
- Comprehensive guide on using Gitea secrets
- Store passwords in Gitea (not in git!)
- Use in CI/CD and deployment workflows
- Best practices and security recommendations
🔧 .github/workflows/ci.yml (UPDATED):
- Now uses Gitea secrets with fallbacks
- ${{ secrets.SMTP_PASSWORD || 'testpass123' }}
- ${{ secrets.DB_PASSWORD || 'testpass123' }}
- Tests run with real credentials from Gitea
🚀 .github/workflows/deploy.yml (NEW):
- Automated deployment to Proxmox
- Manual trigger via Gitea UI
- Steps:
1. SSH to Proxmox with secrets.PROXMOX_SSH_KEY
2. Pull latest code
3. Update .env with secrets from Gitea
4. Run migrations
5. Health check
6. Test email
7. Rollback on failure
HOW IT WORKS:
=============
1. Store passwords in Gitea (Settings → Secrets)
2. CI/CD uses secrets automatically
3. Deployment workflow updates .env on Proxmox
4. Best of both worlds: secure CI + simple runtime
SECRETS TO ADD IN GITEA:
========================
- SMTP_PASSWORD: your mail password
- DB_PASSWORD: changeme123
- PROXMOX_HOST: 10.0.10.95
- PROXMOX_USER: poteapp
- PROXMOX_SSH_KEY: (SSH private key)
- SMTP_HOST: mail.levkin.ca
- SMTP_USER: test@levkin.ca
- FROM_EMAIL: test@levkin.ca
USAGE:
======
# In Gitea UI:
Actions → Deploy to Proxmox → Run workflow
# Or push commits:
git push origin main
# CI runs with secrets automatically
See GITEA_SECRETS_GUIDE.md for full instructions!
Covers 6 options for storing passwords securely:
1. .env file (current, good for personal use)
2. Environment variables (better for production)
3. Separate secrets file
4. Docker secrets
5. HashiCorp Vault (enterprise)
6. Git secrets (CI/CD only)
Recommendation: Current .env setup is fine for personal/research use
Improvement: chmod 600 .env (done)
Includes security checklist, rotation procedures, and testing
One-page reference with:
- Your specific configuration (test@levkin.ca)
- 3-step setup (30 seconds + 1 min + 2 min)
- Quick commands for deployed server
- Troubleshooting checklist
- Current status tracker
- Next actions
Everything ready - just needs password in .env
Answers user's questions:
- What happens after deployment? (nothing automatic by default)
- How to get reports? (3 options: email, SSH, future web UI)
- Where are reports sent? (email or saved to ~/logs/)
- Do you need to check IP? (depends on setup method)
- Can we setup email reports? (YES! 5-minute setup)
- Do we need CI/CD pipelines? (optional, but included)
- Can we use existing Ansible pipeline? (concepts reused, not directly)
This document ties everything together and provides clear next steps.
New Features:
- Watchlist system for tracking specific Congress members
- Trading report generation with multiple formats
- Pre-market-close automated updates (3 PM)
New Scripts:
- scripts/fetch_congress_members.py: Manage watchlist
* 29 known active traders (curated list)
* Optional ProPublica API integration (all 535 members)
* Create/view/manage watchlist
- scripts/generate_trading_report.py: Generate trading reports
* Filter by watchlist or show all
* Multiple formats: text, HTML, JSON
* Summary statistics (buys/sells, top tickers)
* Color-coded output (🟢 BUY, 🔴 SELL)
- scripts/pre_market_close_update.sh: 3 PM automation
* Quick fetch of latest trades
* Enrichment of new securities
* Generate and display report
* Saves to reports/ directory
Documentation:
- WATCHLIST_GUIDE.md: Complete guide
* List of 29 known active traders
* How to create/customize watchlist
* Schedule options (pre-market, post-market)
* Email setup (optional)
* FAQ and examples
Known Active Traders Include:
Senate: Tuberville, Rand Paul, Mark Warner, Rick Scott
House: Pelosi, Crenshaw, MTG, Gottheimer, Brian Higgins
Use Cases:
✅ Daily reports at 3 PM (1 hour before close)
✅ See what Congress bought/sold recently
✅ Track specific members you care about
✅ Export to HTML/JSON for further analysis
Critical Fixes:
- Fixed Price model query to use security_id join with Security
- Added Security import to returns.py module
- Fixed all test fixtures to use test_db_session correctly
- Added AAPL price data to sample_prices fixture
New Tests:
- tests/test_analytics_integration.py: 10 comprehensive integration tests
* Real-world scenarios with synthetic price data
* Return calculations, benchmark comparisons, performance metrics
* Edge cases: missing data, sell trades, disclosure timing
Documentation:
- LOCAL_TEST_GUIDE.md: Complete guide for local testing
* How to test before deploying
* Current data status (live vs fixtures)
* Multiple options for getting real data
* Common issues and fixes
Test Results:
✅ All 55 tests passing
✅ Analytics fully functional
✅ Ready for deployment
Live Data Status:
❌ House Stock Watcher API still down (external issue)
✅ Manual CSV import works
✅ yfinance for prices works
✅ Can use system NOW with manual data
- Replace 'sudo -u' with 'su -' throughout the script
- Works perfectly in LXC containers running as root (no sudo installed)
- Also works on regular VMs/servers where sudo is available
- Fixes all remaining: sudo: command not found errors
ilia
ilia