10 Commits

Author	SHA1	Message	Date
ilia	6eba94346a	Update SonarQube job to match established pattern ... CHANGES: ======== ✅ Added conditional execution - Runs on pull_request or main/dev/qa branches - Matches pattern from other project ✅ Graceful secret handling - Exits 0 if secrets not set (doesn't break CI) - Clear warning message ✅ Non-blocking on failure - Exits 0 on SonarScanner failure (not exit 1) - Prevents CI failures from SonarQube issues - Matches established pattern ✅ Kept coverage report generation - Generates coverage.xml for SonarQube - Uses pytest-cov CONFIGURATION: ============== - Project key: pote - Sources: src/ - Tests: tests/ - Python version: 3.11 - Coverage: coverage.xml This matches the pattern used in other projects while maintaining POTE-specific configuration.	2026-01-10 14:05:33 -05:00
ilia	5161f6c421	Organize documentation: move setup guides to docs/ folder ... Moved: - BRANCH_SETUP_COMPLETE.md → docs/15_branch_setup_checklist.md - PIPELINE_SETUP_GUIDE.md → docs/16_pipeline_setup.md Reason: Keep root directory clean, organize all docs in docs/ folder Documentation structure now: docs/ ├── 14_branch_strategy_and_deployment.md (comprehensive guide) ├── 15_branch_setup_checklist.md (quick checklist) └── 16_pipeline_setup.md (CI setup guide)	2025-12-24 21:55:24 -05:00
ilia	01597f608f	Add comprehensive branch strategy and multi-environment deployment guide ... NEW DOCUMENTATION: ================== docs/14_branch_strategy_and_deployment.md COVERS: ======= ✅ Branch Strategy (main/qa/dev) ✅ Branch Protection Rules for Gitea ✅ Multi-environment deployment workflows ✅ Integration with Ansible auto-deploy ✅ Gitea secrets per environment ✅ Deployment flow diagram ✅ Rollback strategies ✅ Complete setup checklist BRANCH STRUCTURE: ================= main (prod) ← qa (staging) ← dev (development) PROTECTION LEVELS: ================== - main: MOST PROTECTED (require PR + 2 approvals + CI) - qa: MODERATELY PROTECTED (require PR + CI) - dev: LIGHTLY PROTECTED (require CI only) DEPLOYMENT WORKFLOWS: ===================== - deploy-dev.yml: Auto-deploy on push to dev - deploy-qa.yml: Auto-deploy on push to qa + smoke tests - deploy-prod.yml: Manual approval + rollback on failure ANSIBLE INTEGRATION: ==================== - Webhook-based deployment - Gitea Actions → Ansible API - Environment-specific inventory - Automated health checks WHAT'S MISSING (ANSWERED): ========================== ✅ Environment variables per env ✅ Database migration strategy ✅ Rollback procedures ✅ Monitoring & alerts ✅ Feature flags ✅ Changelog management BRANCHES CREATED: ================= ✅ dev branch created and pushed ✅ qa branch created and pushed ✅ main branch (existing) Ready for Ansible auto-configure integration!	2025-12-24 21:46:37 -05:00
ilia	0c183fb28c	Add comprehensive secrets management guide ... Covers 6 options for storing passwords securely: 1. .env file (current, good for personal use) 2. Environment variables (better for production) 3. Separate secrets file 4. Docker secrets 5. HashiCorp Vault (enterprise) 6. Git secrets (CI/CD only) Recommendation: Current .env setup is fine for personal/research use Improvement: chmod 600 .env (done) Includes security checklist, rotation procedures, and testing	2025-12-15 15:47:12 -05:00
ilia	0d8d85adc1	Add complete automation, reporting, and CI/CD system ... Features Added: ============== 📧 EMAIL REPORTING SYSTEM: - EmailReporter: Send reports via SMTP (Gmail, SendGrid, custom) - ReportGenerator: Generate daily/weekly summaries with HTML/text formatting - Configurable via .env (SMTP_HOST, SMTP_PORT, etc.) - Scripts: send_daily_report.py, send_weekly_report.py 🤖 AUTOMATED RUNS: - automated_daily_run.sh: Full daily ETL pipeline + reporting - automated_weekly_run.sh: Weekly pattern analysis + reports - setup_cron.sh: Interactive cron job setup (5-minute setup) - Logs saved to ~/logs/ with automatic cleanup 🔍 HEALTH CHECKS: - health_check.py: System health monitoring - Checks: DB connection, data freshness, counts, recent alerts - JSON output for programmatic use - Exit codes for monitoring integration 🚀 CI/CD PIPELINE: - .github/workflows/ci.yml: Full CI/CD pipeline - GitHub Actions / Gitea Actions compatible - Jobs: lint & test, security scan, dependency scan, Docker build - PostgreSQL service for integration tests - 93 tests passing in CI 📚 COMPREHENSIVE DOCUMENTATION: - AUTOMATION_QUICKSTART.md: 5-minute email setup guide - docs/12_automation_and_reporting.md: Full automation guide - Updated README.md with automation links - Deployment → Production workflow guide 🛠️ IMPROVEMENTS: - All shell scripts made executable - Environment variable examples in .env.example - Report logs saved with timestamps - 30-day log retention with auto-cleanup - Health checks can be scheduled via cron WHAT THIS ENABLES: ================== After deployment, users can: 1. Set up automated daily/weekly email reports (5 min) 2. Receive HTML+text emails with: - New trades, market alerts, suspicious timing - Weekly patterns, rankings, repeat offenders 3. Monitor system health automatically 4. Run full CI/CD pipeline on every commit 5. Deploy with confidence (tests + security scans) USAGE: ====== # One-time setup (on deployed server) ./scripts/setup_cron.sh # Or manually send reports python scripts/send_daily_report.py --to user@example.com python scripts/send_weekly_report.py --to user@example.com # Check system health python scripts/health_check.py See AUTOMATION_QUICKSTART.md for full instructions. 93 tests passing | Full CI/CD | Email reports ready	2025-12-15 15:34:31 -05:00
ilia	cfaf38b0be	Phase 1: Real-Time Market Monitoring System ... COMPLETE: Real-time unusual activity detection for congressional tickers New Database Model: - MarketAlert: Stores unusual market activity alerts * Tracks volume spikes, price movements, volatility * JSON details field for flexible data storage * Severity scoring (1-10 scale) * Indexed for efficient queries by ticker/timestamp New Modules: - src/pote/monitoring/market_monitor.py: Core monitoring engine * get_congressional_watchlist(): Top 50 most-traded tickers * check_ticker(): Analyze single stock for unusual activity * scan_watchlist(): Batch analysis of multiple tickers * Detection logic: - Unusual volume (3x average) - Price spikes/drops (>5%) - High volatility (2x normal) * save_alerts(): Persist to database * get_recent_alerts(): Query historical alerts - src/pote/monitoring/alert_manager.py: Alert formatting & filtering * format_alert_text(): Human-readable output * format_alert_html(): HTML email format * filter_alerts(): By severity, ticker, type * generate_summary_report(): Text/HTML reports Scripts: - scripts/monitor_market.py: CLI monitoring tool * Continuous monitoring mode (--interval) * One-time scan (--once) * Custom ticker lists or auto-detect congressional watchlist * Severity filtering (--min-severity) * Report generation and saving Migrations: - alembic/versions/f44014715b40_add_market_alerts_table.py Documentation: - docs/11_live_market_monitoring.md: Complete explanation * Why you can't track WHO is trading * What IS possible (timing analysis) * How hybrid monitoring works * Data sources and APIs Usage: # Monitor congressional tickers (one-time scan) python scripts/monitor_market.py --once # Continuous monitoring (every 5 minutes) python scripts/monitor_market.py --interval 300 # Monitor specific tickers python scripts/monitor_market.py --tickers NVDA,MSFT,AAPL --once Next Steps (Phase 2): - Disclosure correlation engine - Timing advantage calculator - Suspicious trade flagging	2025-12-15 15:10:49 -05:00
ilia	3a89c1e6d2	Add comprehensive automation system ... New Scripts: - scripts/daily_fetch.sh: Automated daily data updates * Fetches congressional trades (last 7 days) * Enriches securities (name, sector, industry) * Updates price data for all securities * Calculates returns and metrics * Logs everything to logs/ directory - scripts/setup_automation.sh: Interactive automation setup * Makes scripts executable * Creates log directories * Configures cron jobs (multiple schedule options) * Guides user through setup Documentation: - docs/10_automation.md: Complete automation guide * Explains disclosure timing (30-45 day legal lag) * Why daily updates are optimal (not hourly/real-time) * Cron job setup instructions * Systemd timer alternative * Email notifications (optional) * Monitoring and logging * Failure handling * Performance optimization Key Insights: ❌ No real-time data possible (STOCK Act = 30-45 day lag) ✅ Daily updates are optimal ✅ Automated via cron jobs ✅ Handles API failures gracefully ✅ Logs everything for debugging	2025-12-15 14:55:05 -05:00
ilia	34aebb1c2e	PR4: Phase 2 Analytics Foundation ... Complete analytics module with returns, benchmarks, and performance metrics. New Modules: - src/pote/analytics/returns.py: Return calculator for trades - src/pote/analytics/benchmarks.py: Benchmark comparison & alpha - src/pote/analytics/metrics.py: Performance aggregations Scripts: - scripts/analyze_official.py: Analyze specific official - scripts/calculate_all_returns.py: System-wide analysis Tests: - tests/test_analytics.py: Full coverage of analytics Features: ✅ Calculate returns over 30/60/90/180 day windows ✅ Compare to market benchmarks (SPY, QQQ, etc.) ✅ Calculate abnormal returns (alpha) ✅ Aggregate stats by official, sector ✅ Top performer rankings ✅ Disclosure timing analysis ✅ Command-line analysis tools ~1,210 lines of new code, all tested	2025-12-15 11:33:21 -05:00
ilia	02c10c85d6	Add data update tools and Phase 2 plan ... - scripts/add_custom_trades.py: Manual trade entry - scripts/scrape_alternative_sources.py: CSV import - scripts/daily_update.sh: Automated daily updates - docs/09_data_updates.md: Complete update guide - docs/PR4_PLAN.md: Phase 2 analytics plan Enables users to add representatives and set up auto-updates	2025-12-15 10:39:18 -05:00
ilia	204cd0e75b	Initial commit: POTE Phase 1 complete ... - PR1: Project scaffold, DB models, price loader - PR2: Congressional trade ingestion (House Stock Watcher) - PR3: Security enrichment + deployment infrastructure - 37 passing tests, 87%+ coverage - Docker + Proxmox deployment ready - Complete documentation - Works 100% offline with fixtures	2025-12-14 20:45:34 -05:00