Features Added: ============== 📧 EMAIL REPORTING SYSTEM: - EmailReporter: Send reports via SMTP (Gmail, SendGrid, custom) - ReportGenerator: Generate daily/weekly summaries with HTML/text formatting - Configurable via .env (SMTP_HOST, SMTP_PORT, etc.) - Scripts: send_daily_report.py, send_weekly_report.py 🤖 AUTOMATED RUNS: - automated_daily_run.sh: Full daily ETL pipeline + reporting - automated_weekly_run.sh: Weekly pattern analysis + reports - setup_cron.sh: Interactive cron job setup (5-minute setup) - Logs saved to ~/logs/ with automatic cleanup 🔍 HEALTH CHECKS: - health_check.py: System health monitoring - Checks: DB connection, data freshness, counts, recent alerts - JSON output for programmatic use - Exit codes for monitoring integration 🚀 CI/CD PIPELINE: - .github/workflows/ci.yml: Full CI/CD pipeline - GitHub Actions / Gitea Actions compatible - Jobs: lint & test, security scan, dependency scan, Docker build - PostgreSQL service for integration tests - 93 tests passing in CI 📚 COMPREHENSIVE DOCUMENTATION: - AUTOMATION_QUICKSTART.md: 5-minute email setup guide - docs/12_automation_and_reporting.md: Full automation guide - Updated README.md with automation links - Deployment → Production workflow guide 🛠️ IMPROVEMENTS: - All shell scripts made executable - Environment variable examples in .env.example - Report logs saved with timestamps - 30-day log retention with auto-cleanup - Health checks can be scheduled via cron WHAT THIS ENABLES: ================== After deployment, users can: 1. Set up automated daily/weekly email reports (5 min) 2. Receive HTML+text emails with: - New trades, market alerts, suspicious timing - Weekly patterns, rankings, repeat offenders 3. Monitor system health automatically 4. Run full CI/CD pipeline on every commit 5. Deploy with confidence (tests + security scans) USAGE: ====== # One-time setup (on deployed server) ./scripts/setup_cron.sh # Or manually send reports python scripts/send_daily_report.py --to user@example.com python scripts/send_weekly_report.py --to user@example.com # Check system health python scripts/health_check.py See AUTOMATION_QUICKSTART.md for full instructions. 93 tests passing | Full CI/CD | Email reports ready
9.9 KiB
POTE Watchlist & Trading Reports
🎯 Get Trading Reports 1 Hour Before Market Close
Quick Setup
# 1. Create watchlist of officials to monitor
python scripts/fetch_congress_members.py --create
# 2. Setup cron job for 3 PM ET (1 hour before close)
crontab -e
# Add this line:
0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
# Save and exit
What happens at 3 PM daily:
- Fetches latest trade disclosures
- Enriches new securities
- Generates report showing what was bought/sold
- Saves to
reports/trading_report_YYYYMMDD.txt
📋 Watchlist System
Who's on the Default Watchlist?
29 known active traders based on 2023-2024 public reporting:
Top Senate Traders:
- Tommy Tuberville (R-AL) - Very active trader
- Rand Paul (R-KY) - Consistent activity
- Mark Warner (D-VA) - Tech sector focus
- Rick Scott (R-FL) - High volume
Top House Traders:
- Nancy Pelosi (D-CA) - Tech stocks, options
- Dan Crenshaw (R-TX) - Energy, defense
- Marjorie Taylor Greene (R-GA) - Various sectors
- Josh Gottheimer (D-NJ) - Financial services
- Brian Higgins (D-NY) - High frequency
[See full list: python scripts/fetch_congress_members.py]
🔧 Managing Your Watchlist
View Current Watchlist
python scripts/fetch_congress_members.py --list
Create/Reset Watchlist
python scripts/fetch_congress_members.py --create
Creates config/watchlist.json with default 29 active traders.
Customize Watchlist
Edit config/watchlist.json:
[
{
"name": "Nancy Pelosi",
"chamber": "House",
"party": "Democrat",
"state": "CA"
},
{
"name": "Tommy Tuberville",
"chamber": "Senate",
"party": "Republican",
"state": "AL"
}
]
Add anyone you want to track!
Get ALL Members (Not Just Active Traders)
# 1. Get free API key from ProPublica:
# https://www.propublica.org/datastore/api/propublica-congress-api
# 2. Edit scripts/fetch_congress_members.py
# Add your API key
# 3. Run:
python scripts/fetch_congress_members.py --propublica
This fetches all 535 members of Congress (100 Senate + 435 House).
📊 Generating Reports
Manual Report Generation
# Report from last 7 days (watchlist only)
python scripts/generate_trading_report.py --days 7 --watchlist-only
# Report from last 30 days (all officials)
python scripts/generate_trading_report.py --days 30
# Save to file
python scripts/generate_trading_report.py --output report.txt
# HTML format (for email)
python scripts/generate_trading_report.py --format html --output report.html
# JSON format (for programmatic use)
python scripts/generate_trading_report.py --format json --output report.json
Example Report Output
================================================================================
CONGRESSIONAL TRADING REPORT
5 New Trades
Generated: 2024-12-15
================================================================================
────────────────────────────────────────────────────────────────────────────────
👤 Nancy Pelosi (D-CA, House)
────────────────────────────────────────────────────────────────────────────────
Side Ticker Company Sector Value Trade Date Filed
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
🟢 BUY NVDA NVIDIA Corporation Technology $15,001 - $50,000 2024-11-15 2024-12-01
🔴 SELL MSFT Microsoft Corporation Technology $50,001 - $100,000 2024-11-20 2024-12-01
────────────────────────────────────────────────────────────────────────────────
👤 Tommy Tuberville (R-AL, Senate)
────────────────────────────────────────────────────────────────────────────────
Side Ticker Company Sector Value Trade Date Filed
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
🟢 BUY SPY SPDR S&P 500 ETF Financial $100,001 - $250,000 2024-11-18 2024-12-02
🟢 BUY AAPL Apple Inc. Technology $50,001 - $100,000 2024-11-22 2024-12-02
🔴 SELL TSLA Tesla, Inc. Automotive $15,001 - $50,000 2024-11-25 2024-12-02
================================================================================
📊 SUMMARY
================================================================================
Total Trades: 5
Buys: 3
Sells: 2
Unique Officials: 2
Unique Tickers: 5
Top Tickers:
NVDA - 1 trades
MSFT - 1 trades
SPY - 1 trades
AAPL - 1 trades
TSLA - 1 trades
================================================================================
⏰ Automated Schedule Options
Option 1: Pre-Market Close (3 PM ET) - Recommended
crontab -e
# Add: 0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
Why 3 PM?
- 1 hour before market close (4 PM ET)
- Time to review report and make decisions
- Disclosures often appear during business hours
- Weekdays only (no weekends)
Option 2: Pre-Market Open (8 AM ET)
crontab -e
# Add: 0 8 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
Why 8 AM?
- 30 minutes before market opens (9:30 AM ET)
- Catch overnight filings
- Review before trading day
Option 3: After Market Close (5 PM ET)
crontab -e
# Add: 0 17 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
Why 5 PM?
- After market closes
- No trading pressure
- Full day of potential filings captured
Option 4: Multiple Times Per Day
crontab -e
# Add: 0 8,15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
Runs at 8 AM and 3 PM daily (weekdays).
📧 Email Reports (Optional)
Setup Email Notifications
Edit scripts/pre_market_close_update.sh, add at the end:
# Send email with report
if [ -f "$REPORT_FILE" ]; then
mail -s "POTE Trading Report $(date +%Y-%m-%d)" \
your-email@example.com < "$REPORT_FILE"
fi
Requires:
sudo apt install mailutils
# Configure SMTP settings in /etc/postfix/main.cf
HTML Email Reports
python scripts/generate_trading_report.py \
--format html \
--output /tmp/report.html
# Send HTML email
python scripts/send_email.py /tmp/report.html your-email@example.com
🎯 Typical Workflow
Daily Routine (3 PM ET)
-
Automated run at 3 PM
- Script fetches latest disclosures
- Generates report
-
You receive report showing:
- What was bought/sold
- By whom
- When (transaction date)
- Value ranges
-
You review and decide:
- Research the stocks mentioned
- Consider your own investment strategy
- Remember: 30-45 day lag (old trades)
-
Important:
- This is for research/transparency
- Not investment advice
- Trades are 30-45 days old by law
🔍 Finding More Officials
Public Resources
High-Volume Traders:
Official Sources:
- House Clerk: https://disclosures.house.gov/
- Senate: https://efdsearch.senate.gov/
News Coverage:
- "Unusual Whales" on Twitter/X
- Financial news sites
- ProPublica investigations
Research Specific Committees
Members of certain committees tend to trade more:
Senate:
- Banking Committee (financial regulations)
- Armed Services (defense contracts)
- Energy Committee (energy stocks)
House:
- Financial Services
- Energy and Commerce
- Armed Services
Add committee members to your watchlist.
📈 Example Cron Setup
# Edit crontab
crontab -e
# Add these lines:
# Pre-market report (8 AM ET weekdays)
0 8 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
# Weekly full update (Sunday night)
0 0 * * 0 /home/poteapp/pote/scripts/daily_fetch.sh
# Save and exit
This gives you:
- Daily reports at 8 AM (weekdays)
- Weekly full system update (prices, analytics)
🚀 Quick Start Summary
# 1. Create watchlist
python scripts/fetch_congress_members.py --create
# 2. Test report generation
python scripts/generate_trading_report.py --days 7 --watchlist-only
# 3. Setup automation (3 PM daily)
crontab -e
# Add: 0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh
# 4. Check logs
tail -f logs/pre_market_*.log
# 5. View reports
cat reports/trading_report_$(date +%Y%m%d).txt
❓ FAQ
Q: Why are all the trades old (30-45 days)?
A: Federal law (STOCK Act) gives Congress 30-45 days to file. This is normal.
Q: Can I track specific senators/representatives?
A: Yes! Edit config/watchlist.json and add anyone.
Q: Where's the full list of Congress members?
A: Use --propublica option with a free API key to get all 535 members.
Q: Can I get alerts for specific stocks?
A: Yes! Modify generate_trading_report.py to filter by ticker.
Q: What if House Stock Watcher is down?
A: Reports will show existing data. Use CSV import for new data manually.
Q: Can I track past trades?
A: Yes! Adjust --days parameter: --days 365 for full year.
Ready to start tracking?
python scripts/fetch_congress_members.py --create
python scripts/generate_trading_report.py --watchlist-only