Phase 1: Real-Time Market Monitoring System

COMPLETE: Real-time unusual activity detection for congressional tickers New Database Model: - MarketAlert: Stores unusual market activity alerts * Tracks volume spikes, price movements, volatility * JSON details field for flexible data storage * Severity scoring (1-10 scale) * Indexed for efficient queries by ticker/timestamp New Modules: - src/pote/monitoring/market_monitor.py: Core monitoring engine * get_congressional_watchlist(): Top 50 most-traded tickers * check_ticker(): Analyze single stock for unusual activity * scan_watchlist(): Batch analysis of multiple tickers * Detection logic: - Unusual volume (3x average) - Price spikes/drops (>5%) - High volatility (2x normal) * save_alerts(): Persist to database * get_recent_alerts(): Query historical alerts - src/pote/monitoring/alert_manager.py: Alert formatting & filtering * format_alert_text(): Human-readable output * format_alert_html(): HTML email format * filter_alerts(): By severity, ticker, type * generate_summary_report(): Text/HTML reports Scripts: - scripts/monitor_market.py: CLI monitoring tool * Continuous monitoring mode (--interval) * One-time scan (--once) * Custom ticker lists or auto-detect congressional watchlist * Severity filtering (--min-severity) * Report generation and saving Migrations: - alembic/versions/f44014715b40_add_market_alerts_table.py Documentation: - docs/11_live_market_monitoring.md: Complete explanation * Why you can't track WHO is trading * What IS possible (timing analysis) * How hybrid monitoring works * Data sources and APIs Usage: # Monitor congressional tickers (one-time scan) python scripts/monitor_market.py --once # Continuous monitoring (every 5 minutes) python scripts/monitor_market.py --interval 300 # Monitor specific tickers python scripts/monitor_market.py --tickers NVDA,MSFT,AAPL --once Next Steps (Phase 2): - Disclosure correlation engine - Timing advantage calculator - Suspicious trade flagging
2025-12-15 15:10:49 -05:00
parent 8ba9d7ffdd
commit cfaf38b0be
8 changed files with 1191 additions and 0 deletions
@@ -0,0 +1,407 @@
+# Live Market Monitoring + Congressional Trading Analysis
+
+## 🎯 What's Possible vs Impossible
+
+### ❌ **NOT Possible:**
+- Identify WHO is buying/selling in real-time
+- Match live trades to specific Congress members
+- See congressional trades before they're disclosed
+
+### ✅ **IS Possible:**
+- Track unusual market activity in real-time
+- Monitor stocks Congress members historically trade
+- Compare unusual activity to later disclosures
+- Detect patterns (timing, sectors, etc.)
+
+---
+
+## 🔄 **Two-Phase Monitoring System**
+
+### **Phase 1: Real-Time Market Monitoring**
+Monitor unusual activity in stocks Congress trades:
+- Unusual options flow
+- Large block trades
+- Volatility spikes
+- Volume anomalies
+
+### **Phase 2: Retroactive Analysis (30-45 days later)**
+When disclosures come in:
+- Match disclosed trades to earlier unusual activity
+- Identify if Congress bought BEFORE or AFTER spikes
+- Calculate timing advantage (if any)
+- Build pattern database
+
+---
+
+## 📊 **Implementation: Watchlist-Based Monitoring**
+
+### **Concept:**
+
+```
+Step 1: Congress Member Trades (Historical)
+  Nancy Pelosi often trades: NVDA, MSFT, GOOGL, AAPL
+  Dan Crenshaw often trades: XOM, CVX, LMT, BA
+  
+Step 2: Create Monitoring Watchlist
+  Monitor these tickers in real-time for:
+  - Unusual options activity
+  - Large block trades
+  - Price/volume anomalies
+  
+Step 3: When Disclosure Appears (30-45 days later)
+  Compare:
+  - Did they buy BEFORE unusual activity? (Suspicious)
+  - Did they buy AFTER? (Following market)
+  - What was the timing advantage?
+```
+
+---
+
+## 🛠️ **Data Sources for Live Market Monitoring**
+
+### **Free/Low-Cost Options:**
+
+1. **Yahoo Finance (yfinance)**
+   - ✅ Real-time quotes (15-min delay free)
+   - ✅ Historical options data
+   - ✅ Volume data
+   - ❌ Not true real-time for options flow
+
+2. **Unusual Whales API**
+   - ✅ Options flow data
+   - ✅ Unusual activity alerts
+   - 💰 Paid ($50-200/month)
+   - https://unusualwhales.com/
+
+3. **Tradier API**
+   - ✅ Real-time market data
+   - ✅ Options chains
+   - 💰 Paid but affordable ($10-50/month)
+   - https://tradier.com/
+
+4. **FlowAlgo**
+   - ✅ Options flow tracking
+   - ✅ Dark pool data
+   - 💰 Paid ($99-399/month)
+   - https://www.flowalgo.com/
+
+5. **Polygon.io**
+   - ✅ Real-time stock data
+   - ✅ Options data
+   - 💰 Free tier + paid plans
+   - https://polygon.io/
+
+### **Best Free Option: Build Your Own with yfinance**
+
+Track volume/price changes every 5 minutes for congressional watchlist tickers.
+
+---
+
+## 💡 **Practical Hybrid System**
+
+### **What We Can Build:**
+
+```python
+# Pseudo-code for hybrid monitoring
+
+# 1. Get stocks Congress trades
+congress_tickers = get_tickers_congress_trades()
+# Result: ["NVDA", "MSFT", "TSLA", "AAPL", "SPY", ...]
+
+# 2. Monitor these tickers for unusual activity
+while market_open():
+    for ticker in congress_tickers:
+        current_data = get_realtime_data(ticker)
+        
+        if is_unusual_activity(current_data):
+            log_alert({
+                "ticker": ticker,
+                "type": "unusual_volume",  # or "price_spike", "options_flow"
+                "timestamp": now(),
+                "details": current_data
+            })
+
+# 3. When disclosures appear (30-45 days later)
+new_disclosures = fetch_congressional_trades()
+
+for disclosure in new_disclosures:
+    # Check if we saw unusual activity BEFORE their trade
+    prior_alerts = get_alerts_before_date(
+        ticker=disclosure.ticker,
+        before_date=disclosure.transaction_date
+    )
+    
+    if prior_alerts:
+        # They bought BEFORE unusual activity = Potential inside info
+        flag_suspicious(disclosure, prior_alerts)
+    else:
+        # They bought AFTER unusual activity = Following market
+        flag_following(disclosure)
+```
+
+---
+
+## 📈 **Example: Nancy Pelosi NVDA Trade Analysis**
+
+### **Timeline:**
+
+```
+Nov 10, 2024:
+  🔔 ALERT: NVDA unusual call options activity
+     Volume: 10x average
+     Strike: $500 (2 weeks out)
+     
+Nov 15, 2024:
+  💰 Someone buys NVDA (unknown who at the time)
+  
+Nov 18, 2024:
+  📰 NVDA announces new AI chip
+  📈 Stock jumps 15%
+  
+Dec 15, 2024:
+  📋 Disclosure: Nancy Pelosi bought NVDA on Nov 15
+     Value: $15,001-$50,000
+     
+ANALYSIS:
+  ✅ She bought AFTER unusual options activity (Nov 10)
+  ❓ She bought BEFORE announcement (Nov 18)
+  ⏱️ Timing: 3 days before major news
+  🚩 Flag: Investigate if announcement was public knowledge
+```
+
+---
+
+## 🎯 **Recommended Approach**
+
+### **Phase 1: Build Congressional Ticker Watchlist**
+
+```python
+# scripts/build_ticker_watchlist.py
+
+from pote.db import get_session
+from pote.db.models import Trade, Security
+from sqlalchemy import func
+
+def get_most_traded_tickers(limit=50):
+    """Get tickers Congress trades most frequently."""
+    session = next(get_session())
+    
+    results = (
+        session.query(
+            Security.ticker,
+            func.count(Trade.id).label('trade_count')
+        )
+        .join(Trade)
+        .group_by(Security.ticker)
+        .order_by(func.count(Trade.id).desc())
+        .limit(limit)
+        .all()
+    )
+    
+    return [r[0] for r in results]
+
+# Result: Top 50 tickers Congress trades
+# Use these for real-time monitoring
+```
+
+### **Phase 2: Real-Time Monitoring (Simple)**
+
+```python
+# scripts/monitor_congressional_tickers.py
+
+import yfinance as yf
+from datetime import datetime, timedelta
+import time
+
+def monitor_tickers(tickers, interval_minutes=5):
+    """Monitor tickers for unusual activity."""
+    
+    baseline = {}  # Store baseline metrics
+    
+    while True:
+        for ticker in tickers:
+            try:
+                stock = yf.Ticker(ticker)
+                current = stock.history(period="1d", interval="1m")
+                
+                if len(current) > 0:
+                    latest = current.iloc[-1]
+                    
+                    # Check for unusual volume
+                    avg_volume = current['Volume'].mean()
+                    if latest['Volume'] > avg_volume * 3:
+                        alert(f"🔔 {ticker}: Unusual volume spike!")
+                    
+                    # Check for price movement
+                    price_change = (latest['Close'] - current['Open'].iloc[0]) / current['Open'].iloc[0]
+                    if abs(price_change) > 0.05:  # 5% move
+                        alert(f"📈 {ticker}: {price_change:.2%} move today!")
+                        
+            except Exception as e:
+                print(f"Error monitoring {ticker}: {e}")
+        
+        time.sleep(interval_minutes * 60)
+```
+
+### **Phase 3: Retroactive Analysis**
+
+When disclosures appear, analyze timing:
+
+```python
+# scripts/analyze_trade_timing.py
+
+def analyze_disclosure_timing(disclosure):
+    """
+    When a disclosure appears, check if there was unusual
+    activity BEFORE the trade date.
+    """
+    
+    # Get alerts from 7 days before trade
+    lookback_start = disclosure.transaction_date - timedelta(days=7)
+    lookback_end = disclosure.transaction_date
+    
+    alerts = get_alerts_in_range(
+        ticker=disclosure.ticker,
+        start=lookback_start,
+        end=lookback_end
+    )
+    
+    if alerts:
+        return {
+            "suspicious": True,
+            "reason": "Unusual activity before trade",
+            "alerts": alerts
+        }
+    
+    # Check if trade was before major price movement
+    post_trade_price = get_price_change(
+        ticker=disclosure.ticker,
+        start=disclosure.transaction_date,
+        days=30
+    )
+    
+    if post_trade_price > 0.10:  # 10% gain
+        return {
+            "notable": True,
+            "reason": f"Stock up {post_trade_price:.1%} after trade",
+            "gain": post_trade_price
+        }
+```
+
+---
+
+## 🚨 **Realistic Expectations**
+
+### **What This System Will Do:**
+✅ Monitor stocks Congress members historically trade
+✅ Alert on unusual market activity in those stocks
+✅ Retroactively correlate disclosures with earlier alerts
+✅ Identify timing patterns and potential advantages
+✅ Build database of congressional trading patterns
+
+### **What This System WON'T Do:**
+❌ Identify WHO is buying in real-time
+❌ Give you advance notice of congressional trades
+❌ Provide real-time inside information
+❌ Allow you to "front-run" Congress
+
+### **Legal & Ethical:**
+✅ All data is public
+✅ Analysis is retrospective
+✅ For research and transparency
+✅ Not market manipulation
+❌ Cannot and should not be used to replicate potentially illegal trades
+
+---
+
+## 📊 **Proposed Implementation**
+
+### **New Scripts to Create:**
+
+1. **`scripts/build_congressional_watchlist.py`**
+   - Analyzes historical trades
+   - Identifies most-traded tickers
+   - Creates monitoring watchlist
+
+2. **`scripts/monitor_market_live.py`**
+   - Monitors watchlist tickers
+   - Detects unusual activity
+   - Logs alerts to database
+
+3. **`scripts/analyze_disclosure_timing.py`**
+   - When new disclosures appear
+   - Checks for prior unusual activity
+   - Flags suspicious timing
+
+4. **`scripts/generate_timing_report.py`**
+   - Shows disclosures with unusual timing
+   - Calculates timing advantage
+   - Identifies patterns
+
+### **New Database Tables:**
+
+```sql
+-- Track unusual market activity
+CREATE TABLE market_alerts (
+    id SERIAL PRIMARY KEY,
+    ticker VARCHAR(20),
+    alert_type VARCHAR(50),  -- 'unusual_volume', 'price_spike', 'options_flow'
+    timestamp TIMESTAMP,
+    details JSONB,
+    created_at TIMESTAMP DEFAULT NOW()
+);
+
+-- Link disclosures to prior alerts
+CREATE TABLE disclosure_timing_analysis (
+    id SERIAL PRIMARY KEY,
+    trade_id INTEGER REFERENCES trades(id),
+    suspicious_flag BOOLEAN,
+    timing_score DECIMAL(5,2),  -- 0-100 score
+    prior_alerts JSONB,
+    post_trade_performance DECIMAL(10,4),
+    created_at TIMESTAMP DEFAULT NOW()
+);
+```
+
+---
+
+## 🎯 **Summary**
+
+### **Your Question:**
+> "Can we read live trades being made and compare them to a name?"
+
+### **Answer:**
+❌ **No** - Live trades are anonymous, can't identify individuals
+
+✅ **BUT** - You CAN:
+1. Monitor unusual activity in stocks Congress trades
+2. Log these alerts in real-time
+3. When disclosures appear (30-45 days later), correlate them
+4. Identify if Congress bought BEFORE or AFTER unusual activity
+5. Build patterns database of timing and performance
+
+### **This Gives You:**
+- ✅ Transparency on timing advantages
+- ✅ Pattern detection across officials
+- ✅ Research-grade analysis
+- ✅ Historical correlation data
+
+### **This Does NOT Give You:**
+- ❌ Real-time identity of traders
+- ❌ Advance notice of congressional trades
+- ❌ Ability to "front-run" disclosures
+
+---
+
+## 🚀 **Would You Like Me To Build This?**
+
+I can create:
+1. ✅ Real-time monitoring system for congressional tickers
+2. ✅ Alert logging and analysis
+3. ✅ Timing correlation when disclosures appear
+4. ✅ Pattern detection and reporting
+
+This would be **Phase 2.5** of POTE - the "timing analysis" module.
+
+**Should I proceed with implementation?**
+