POTE/WATCHLIST_GUIDE.md
ilia 0d8d85adc1 Add complete automation, reporting, and CI/CD system
Features Added:
==============

📧 EMAIL REPORTING SYSTEM:
- EmailReporter: Send reports via SMTP (Gmail, SendGrid, custom)
- ReportGenerator: Generate daily/weekly summaries with HTML/text formatting
- Configurable via .env (SMTP_HOST, SMTP_PORT, etc.)
- Scripts: send_daily_report.py, send_weekly_report.py

🤖 AUTOMATED RUNS:
- automated_daily_run.sh: Full daily ETL pipeline + reporting
- automated_weekly_run.sh: Weekly pattern analysis + reports
- setup_cron.sh: Interactive cron job setup (5-minute setup)
- Logs saved to ~/logs/ with automatic cleanup

🔍 HEALTH CHECKS:
- health_check.py: System health monitoring
- Checks: DB connection, data freshness, counts, recent alerts
- JSON output for programmatic use
- Exit codes for monitoring integration

🚀 CI/CD PIPELINE:
- .github/workflows/ci.yml: Full CI/CD pipeline
- GitHub Actions / Gitea Actions compatible
- Jobs: lint & test, security scan, dependency scan, Docker build
- PostgreSQL service for integration tests
- 93 tests passing in CI

📚 COMPREHENSIVE DOCUMENTATION:
- AUTOMATION_QUICKSTART.md: 5-minute email setup guide
- docs/12_automation_and_reporting.md: Full automation guide
- Updated README.md with automation links
- Deployment → Production workflow guide

🛠️ IMPROVEMENTS:
- All shell scripts made executable
- Environment variable examples in .env.example
- Report logs saved with timestamps
- 30-day log retention with auto-cleanup
- Health checks can be scheduled via cron

WHAT THIS ENABLES:
==================
After deployment, users can:
1. Set up automated daily/weekly email reports (5 min)
2. Receive HTML+text emails with:
   - New trades, market alerts, suspicious timing
   - Weekly patterns, rankings, repeat offenders
3. Monitor system health automatically
4. Run full CI/CD pipeline on every commit
5. Deploy with confidence (tests + security scans)

USAGE:
======
# One-time setup (on deployed server)
./scripts/setup_cron.sh

# Or manually send reports
python scripts/send_daily_report.py --to user@example.com
python scripts/send_weekly_report.py --to user@example.com

# Check system health
python scripts/health_check.py

See AUTOMATION_QUICKSTART.md for full instructions.

93 tests passing | Full CI/CD | Email reports ready
2025-12-15 15:34:31 -05:00

9.9 KiB

POTE Watchlist & Trading Reports

🎯 Get Trading Reports 1 Hour Before Market Close

Quick Setup

# 1. Create watchlist of officials to monitor
python scripts/fetch_congress_members.py --create

# 2. Setup cron job for 3 PM ET (1 hour before close)
crontab -e

# Add this line:
0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

# Save and exit

What happens at 3 PM daily:

  1. Fetches latest trade disclosures
  2. Enriches new securities
  3. Generates report showing what was bought/sold
  4. Saves to reports/trading_report_YYYYMMDD.txt

📋 Watchlist System

Who's on the Default Watchlist?

29 known active traders based on 2023-2024 public reporting:

Top Senate Traders:

  • Tommy Tuberville (R-AL) - Very active trader
  • Rand Paul (R-KY) - Consistent activity
  • Mark Warner (D-VA) - Tech sector focus
  • Rick Scott (R-FL) - High volume

Top House Traders:

  • Nancy Pelosi (D-CA) - Tech stocks, options
  • Dan Crenshaw (R-TX) - Energy, defense
  • Marjorie Taylor Greene (R-GA) - Various sectors
  • Josh Gottheimer (D-NJ) - Financial services
  • Brian Higgins (D-NY) - High frequency

[See full list: python scripts/fetch_congress_members.py]


🔧 Managing Your Watchlist

View Current Watchlist

python scripts/fetch_congress_members.py --list

Create/Reset Watchlist

python scripts/fetch_congress_members.py --create

Creates config/watchlist.json with default 29 active traders.

Customize Watchlist

Edit config/watchlist.json:

[
  {
    "name": "Nancy Pelosi",
    "chamber": "House",
    "party": "Democrat",
    "state": "CA"
  },
  {
    "name": "Tommy Tuberville",
    "chamber": "Senate",
    "party": "Republican",
    "state": "AL"
  }
]

Add anyone you want to track!

Get ALL Members (Not Just Active Traders)

# 1. Get free API key from ProPublica:
#    https://www.propublica.org/datastore/api/propublica-congress-api

# 2. Edit scripts/fetch_congress_members.py
#    Add your API key

# 3. Run:
python scripts/fetch_congress_members.py --propublica

This fetches all 535 members of Congress (100 Senate + 435 House).


📊 Generating Reports

Manual Report Generation

# Report from last 7 days (watchlist only)
python scripts/generate_trading_report.py --days 7 --watchlist-only

# Report from last 30 days (all officials)
python scripts/generate_trading_report.py --days 30

# Save to file
python scripts/generate_trading_report.py --output report.txt

# HTML format (for email)
python scripts/generate_trading_report.py --format html --output report.html

# JSON format (for programmatic use)
python scripts/generate_trading_report.py --format json --output report.json

Example Report Output

================================================================================
  CONGRESSIONAL TRADING REPORT
  5 New Trades
  Generated: 2024-12-15
================================================================================

────────────────────────────────────────────────────────────────────────────────
👤 Nancy Pelosi (D-CA, House)
────────────────────────────────────────────────────────────────────────────────
Side      Ticker  Company                     Sector        Value              Trade Date  Filed
--------  ------  --------------------------  ----------  -------------------  ----------  ----------
🟢 BUY     NVDA    NVIDIA Corporation          Technology  $15,001 - $50,000   2024-11-15  2024-12-01
🔴 SELL    MSFT    Microsoft Corporation       Technology  $50,001 - $100,000  2024-11-20  2024-12-01

────────────────────────────────────────────────────────────────────────────────
👤 Tommy Tuberville (R-AL, Senate)
────────────────────────────────────────────────────────────────────────────────
Side      Ticker  Company                     Sector        Value              Trade Date  Filed
--------  ------  --------------------------  ----------  -------------------  ----------  ----------
🟢 BUY     SPY     SPDR S&P 500 ETF            Financial   $100,001 - $250,000 2024-11-18  2024-12-02
🟢 BUY     AAPL    Apple Inc.                  Technology  $50,001 - $100,000  2024-11-22  2024-12-02
🔴 SELL    TSLA    Tesla, Inc.                 Automotive  $15,001 - $50,000   2024-11-25  2024-12-02

================================================================================
📊 SUMMARY
================================================================================

Total Trades:    5
  Buys:          3
  Sells:         2
Unique Officials: 2
Unique Tickers:   5

Top Tickers:
  NVDA   - 1 trades
  MSFT   - 1 trades
  SPY    - 1 trades
  AAPL   - 1 trades
  TSLA   - 1 trades

================================================================================

Automated Schedule Options

crontab -e
# Add: 0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

Why 3 PM?

  • 1 hour before market close (4 PM ET)
  • Time to review report and make decisions
  • Disclosures often appear during business hours
  • Weekdays only (no weekends)

Option 2: Pre-Market Open (8 AM ET)

crontab -e
# Add: 0 8 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

Why 8 AM?

  • 30 minutes before market opens (9:30 AM ET)
  • Catch overnight filings
  • Review before trading day

Option 3: After Market Close (5 PM ET)

crontab -e
# Add: 0 17 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

Why 5 PM?

  • After market closes
  • No trading pressure
  • Full day of potential filings captured

Option 4: Multiple Times Per Day

crontab -e
# Add: 0 8,15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

Runs at 8 AM and 3 PM daily (weekdays).


📧 Email Reports (Optional)

Setup Email Notifications

Edit scripts/pre_market_close_update.sh, add at the end:

# Send email with report
if [ -f "$REPORT_FILE" ]; then
    mail -s "POTE Trading Report $(date +%Y-%m-%d)" \
         your-email@example.com < "$REPORT_FILE"
fi

Requires:

sudo apt install mailutils
# Configure SMTP settings in /etc/postfix/main.cf

HTML Email Reports

python scripts/generate_trading_report.py \
    --format html \
    --output /tmp/report.html

# Send HTML email
python scripts/send_email.py /tmp/report.html your-email@example.com

🎯 Typical Workflow

Daily Routine (3 PM ET)

  1. Automated run at 3 PM

    • Script fetches latest disclosures
    • Generates report
  2. You receive report showing:

    • What was bought/sold
    • By whom
    • When (transaction date)
    • Value ranges
  3. You review and decide:

    • Research the stocks mentioned
    • Consider your own investment strategy
    • Remember: 30-45 day lag (old trades)
  4. Important:

    • This is for research/transparency
    • Not investment advice
    • Trades are 30-45 days old by law

🔍 Finding More Officials

Public Resources

High-Volume Traders:

Official Sources:

News Coverage:

  • "Unusual Whales" on Twitter/X
  • Financial news sites
  • ProPublica investigations

Research Specific Committees

Members of certain committees tend to trade more:

Senate:

  • Banking Committee (financial regulations)
  • Armed Services (defense contracts)
  • Energy Committee (energy stocks)

House:

  • Financial Services
  • Energy and Commerce
  • Armed Services

Add committee members to your watchlist.


📈 Example Cron Setup

# Edit crontab
crontab -e

# Add these lines:

# Pre-market report (8 AM ET weekdays)
0 8 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

# Weekly full update (Sunday night)
0 0 * * 0 /home/poteapp/pote/scripts/daily_fetch.sh

# Save and exit

This gives you:

  • Daily reports at 8 AM (weekdays)
  • Weekly full system update (prices, analytics)

🚀 Quick Start Summary

# 1. Create watchlist
python scripts/fetch_congress_members.py --create

# 2. Test report generation
python scripts/generate_trading_report.py --days 7 --watchlist-only

# 3. Setup automation (3 PM daily)
crontab -e
# Add: 0 15 * * 1-5 /home/poteapp/pote/scripts/pre_market_close_update.sh

# 4. Check logs
tail -f logs/pre_market_*.log

# 5. View reports
cat reports/trading_report_$(date +%Y%m%d).txt

FAQ

Q: Why are all the trades old (30-45 days)?
A: Federal law (STOCK Act) gives Congress 30-45 days to file. This is normal.

Q: Can I track specific senators/representatives?
A: Yes! Edit config/watchlist.json and add anyone.

Q: Where's the full list of Congress members?
A: Use --propublica option with a free API key to get all 535 members.

Q: Can I get alerts for specific stocks?
A: Yes! Modify generate_trading_report.py to filter by ticker.

Q: What if House Stock Watcher is down?
A: Reports will show existing data. Use CSV import for new data manually.

Q: Can I track past trades?
A: Yes! Adjust --days parameter: --days 365 for full year.


Ready to start tracking?

python scripts/fetch_congress_members.py --create
python scripts/generate_trading_report.py --watchlist-only