ilia/POTE
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
13 Commits 4 Branches 0 Tags
Commit Graph

13 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ilia
6b62ae96f7 Phase 2: Disclosure Timing Correlation Engine 
COMPLETE: Match congressional trades to prior market alerts

New Module:
- src/pote/monitoring/disclosure_correlator.py: Core correlation engine
  * get_alerts_before_trade(): Find alerts before trade date
  * calculate_timing_score(): Score suspicious timing (0-100 scale)
    - Factors: alert count, severity, recency, type
    - Thresholds: 60+ = suspicious, 80+ = highly suspicious
  * analyze_trade(): Complete trade analysis with timing
  * analyze_recent_disclosures(): Batch analysis of new filings
  * get_official_timing_pattern(): Historical pattern analysis
  * get_ticker_timing_analysis(): Per-stock timing patterns

Timing Score Algorithm:
- Base score: alert count × 5 + avg severity × 2
- Recency bonus: +10 per alert within 7 days
- Severity bonus: +15 per high-severity (7+) alert
- Total score: 0-100 (capped)
- Interpretation:
  * 80-100: Highly suspicious (likely timing advantage)
  * 60-79: Suspicious (possible timing advantage)
  * 40-59: Notable (some unusual activity)
  * 0-39: Normal (no significant pattern)

New Script:
- scripts/analyze_disclosure_timing.py: CLI analysis tool
  * Analyze recent disclosures (--days N)
  * Filter by timing score (--min-score)
  * Analyze specific official (--official NAME)
  * Analyze specific ticker (--ticker SYMBOL)
  * Text/JSON output formats
  * Detailed reports with prior alerts

Usage Examples:
  # Find suspicious trades filed recently
  python scripts/analyze_disclosure_timing.py --days 30 --min-score 60

  # Analyze specific official
  python scripts/analyze_disclosure_timing.py --official "Nancy Pelosi"

  # Analyze specific ticker
  python scripts/analyze_disclosure_timing.py --ticker NVDA

Report Includes:
- Timing score and suspicion level
- Prior alert details (count, severity, timing)
- Official name, ticker, trade details
- Assessment and reasoning
- Top suspicious trades ranked

Next: Phase 3 - Pattern Detection across officials/stocks
 2025-12-15 15:17:09 -05:00
ilia
db34f26cdc Add comprehensive tests for Phase 1 monitoring system 
New Tests (14 total, all passing):
- test_get_congressional_watchlist: Auto-detect most-traded tickers
- test_check_ticker_basic: Single ticker analysis
- test_scan_watchlist_with_mock: Batch scanning with controlled data
- test_save_alerts: Database persistence
- test_get_recent_alerts: Query filtering (ticker, type, severity, date)
- test_get_ticker_alert_summary: Aggregated statistics
- test_alert_manager_format_text: Text formatting
- test_alert_manager_format_html: HTML formatting
- test_alert_manager_filter_alerts: Multi-criteria filtering
- test_alert_manager_generate_summary_text: Report generation
- test_alert_manager_generate_summary_html: HTML reports
- test_alert_manager_empty_alerts: Edge case handling
- test_market_alert_model: ORM model validation
- test_alert_timestamp_filtering: Time-based queries

Test Coverage:
- Market monitoring core logic
- Alert detection algorithms
- Database operations
- Filtering and querying
- Report generation (text/HTML)
- Edge cases and error handling

Total Test Suite: 69 tests passing ✅
 2025-12-15 15:14:58 -05:00
ilia
cfaf38b0be Phase 1: Real-Time Market Monitoring System 
COMPLETE: Real-time unusual activity detection for congressional tickers

New Database Model:
- MarketAlert: Stores unusual market activity alerts
  * Tracks volume spikes, price movements, volatility
  * JSON details field for flexible data storage
  * Severity scoring (1-10 scale)
  * Indexed for efficient queries by ticker/timestamp

New Modules:
- src/pote/monitoring/market_monitor.py: Core monitoring engine
  * get_congressional_watchlist(): Top 50 most-traded tickers
  * check_ticker(): Analyze single stock for unusual activity
  * scan_watchlist(): Batch analysis of multiple tickers
  * Detection logic:
    - Unusual volume (3x average)
    - Price spikes/drops (>5%)
    - High volatility (2x normal)
  * save_alerts(): Persist to database
  * get_recent_alerts(): Query historical alerts

- src/pote/monitoring/alert_manager.py: Alert formatting & filtering
  * format_alert_text(): Human-readable output
  * format_alert_html(): HTML email format
  * filter_alerts(): By severity, ticker, type
  * generate_summary_report(): Text/HTML reports

Scripts:
- scripts/monitor_market.py: CLI monitoring tool
  * Continuous monitoring mode (--interval)
  * One-time scan (--once)
  * Custom ticker lists or auto-detect congressional watchlist
  * Severity filtering (--min-severity)
  * Report generation and saving

Migrations:
- alembic/versions/f44014715b40_add_market_alerts_table.py

Documentation:
- docs/11_live_market_monitoring.md: Complete explanation
  * Why you can't track WHO is trading
  * What IS possible (timing analysis)
  * How hybrid monitoring works
  * Data sources and APIs

Usage:
  # Monitor congressional tickers (one-time scan)
  python scripts/monitor_market.py --once

  # Continuous monitoring (every 5 minutes)
  python scripts/monitor_market.py --interval 300

  # Monitor specific tickers
  python scripts/monitor_market.py --tickers NVDA,MSFT,AAPL --once

Next Steps (Phase 2):
- Disclosure correlation engine
- Timing advantage calculator
- Suspicious trade flagging
 2025-12-15 15:10:49 -05:00
ilia
8ba9d7ffdd Add watchlist system and pre-market trading reports 
New Features:
- Watchlist system for tracking specific Congress members
- Trading report generation with multiple formats
- Pre-market-close automated updates (3 PM)

New Scripts:
- scripts/fetch_congress_members.py: Manage watchlist
  * 29 known active traders (curated list)
  * Optional ProPublica API integration (all 535 members)
  * Create/view/manage watchlist

- scripts/generate_trading_report.py: Generate trading reports
  * Filter by watchlist or show all
  * Multiple formats: text, HTML, JSON
  * Summary statistics (buys/sells, top tickers)
  * Color-coded output (🟢 BUY, 🔴 SELL)

- scripts/pre_market_close_update.sh: 3 PM automation
  * Quick fetch of latest trades
  * Enrichment of new securities
  * Generate and display report
  * Saves to reports/ directory

Documentation:
- WATCHLIST_GUIDE.md: Complete guide
  * List of 29 known active traders
  * How to create/customize watchlist
  * Schedule options (pre-market, post-market)
  * Email setup (optional)
  * FAQ and examples

Known Active Traders Include:
Senate: Tuberville, Rand Paul, Mark Warner, Rick Scott
House: Pelosi, Crenshaw, MTG, Gottheimer, Brian Higgins

Use Cases:
✅ Daily reports at 3 PM (1 hour before close)
✅ See what Congress bought/sold recently
✅ Track specific members you care about
✅ Export to HTML/JSON for further analysis
 2025-12-15 15:00:42 -05:00
ilia
3a89c1e6d2 Add comprehensive automation system 
New Scripts:
- scripts/daily_fetch.sh: Automated daily data updates
  * Fetches congressional trades (last 7 days)
  * Enriches securities (name, sector, industry)
  * Updates price data for all securities
  * Calculates returns and metrics
  * Logs everything to logs/ directory

- scripts/setup_automation.sh: Interactive automation setup
  * Makes scripts executable
  * Creates log directories
  * Configures cron jobs (multiple schedule options)
  * Guides user through setup

Documentation:
- docs/10_automation.md: Complete automation guide
  * Explains disclosure timing (30-45 day legal lag)
  * Why daily updates are optimal (not hourly/real-time)
  * Cron job setup instructions
  * Systemd timer alternative
  * Email notifications (optional)
  * Monitoring and logging
  * Failure handling
  * Performance optimization

Key Insights:
❌ No real-time data possible (STOCK Act = 30-45 day lag)
✅ Daily updates are optimal
✅ Automated via cron jobs
✅ Handles API failures gracefully
✅ Logs everything for debugging
 2025-12-15 14:55:05 -05:00
ilia
77bd69b85c Add comprehensive testing status documentation 2025-12-15 14:43:52 -05:00
ilia
b4e6a7c340 Fix analytics tests and add comprehensive testing guide 
Critical Fixes:
- Fixed Price model query to use security_id join with Security
- Added Security import to returns.py module
- Fixed all test fixtures to use test_db_session correctly
- Added AAPL price data to sample_prices fixture

New Tests:
- tests/test_analytics_integration.py: 10 comprehensive integration tests
  * Real-world scenarios with synthetic price data
  * Return calculations, benchmark comparisons, performance metrics
  * Edge cases: missing data, sell trades, disclosure timing

Documentation:
- LOCAL_TEST_GUIDE.md: Complete guide for local testing
  * How to test before deploying
  * Current data status (live vs fixtures)
  * Multiple options for getting real data
  * Common issues and fixes

Test Results:
✅ All 55 tests passing
✅ Analytics fully functional
✅ Ready for deployment

Live Data Status:
❌ House Stock Watcher API still down (external issue)
✅ Manual CSV import works
✅ yfinance for prices works
✅ Can use system NOW with manual data
 2025-12-15 14:42:20 -05:00
ilia
34aebb1c2e PR4: Phase 2 Analytics Foundation 
Complete analytics module with returns, benchmarks, and performance metrics.

New Modules:
- src/pote/analytics/returns.py: Return calculator for trades
- src/pote/analytics/benchmarks.py: Benchmark comparison & alpha
- src/pote/analytics/metrics.py: Performance aggregations

Scripts:
- scripts/analyze_official.py: Analyze specific official
- scripts/calculate_all_returns.py: System-wide analysis

Tests:
- tests/test_analytics.py: Full coverage of analytics

Features:
✅ Calculate returns over 30/60/90/180 day windows
✅ Compare to market benchmarks (SPY, QQQ, etc.)
✅ Calculate abnormal returns (alpha)
✅ Aggregate stats by official, sector
✅ Top performer rankings
✅ Disclosure timing analysis
✅ Command-line analysis tools

~1,210 lines of new code, all tested
 2025-12-15 11:33:21 -05:00
ilia
02c10c85d6 Add data update tools and Phase 2 plan 
- scripts/add_custom_trades.py: Manual trade entry
- scripts/scrape_alternative_sources.py: CSV import
- scripts/daily_update.sh: Automated daily updates
- docs/09_data_updates.md: Complete update guide
- docs/PR4_PLAN.md: Phase 2 analytics plan

Enables users to add representatives and set up auto-updates
 2025-12-15 10:39:18 -05:00
ilia
895c34e2c1 Add psycopg2-binary to dependencies for PostgreSQL support 
- Required for SQLAlchemy PostgreSQL connections
- Fixes: ModuleNotFoundError: No module named 'psycopg2'
 2025-12-14 21:12:36 -05:00
ilia
44ddd88879 Complete fix: Replace all sudo commands with su for LXC compatibility 
- Replace 'sudo -u' with 'su -' throughout the script
- Works perfectly in LXC containers running as root (no sudo installed)
- Also works on regular VMs/servers where sudo is available
- Fixes all remaining: sudo: command not found errors
 2025-12-14 21:05:35 -05:00
ilia
9bb39c9913 Fix proxmox_setup.sh to work when running as root in LXC 
- Detect if running as root and use 'su' instead of 'sudo' for postgres
- Fixes: sudo: command not found error in LXC containers
 2025-12-14 21:01:30 -05:00
ilia
204cd0e75b Initial commit: POTE Phase 1 complete 
- PR1: Project scaffold, DB models, price loader
- PR2: Congressional trade ingestion (House Stock Watcher)
- PR3: Security enrichment + deployment infrastructure
- 37 passing tests, 87%+ coverage
- Docker + Proxmox deployment ready
- Complete documentation
- Works 100% offline with fixtures
 2025-12-14 20:45:34 -05:00