Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
332e38dbe8 | ||
|
|
68a14d5d19 | ||
|
|
c8a10b4270 | ||
|
|
662cb1b1b8 | ||
|
|
d790dc753b | ||
|
|
821d4d58d8 | ||
|
|
1665885927 | ||
|
|
5df21d82f4 | ||
|
|
6e5e69ed10 |
@@ -46,10 +46,16 @@ jobs:
|
|||||||
python3 -m pip install --upgrade pip --break-system-packages
|
python3 -m pip install --upgrade pip --break-system-packages
|
||||||
if [ -f requirements.txt ]; then pip install -r requirements.txt --break-system-packages; fi
|
if [ -f requirements.txt ]; then pip install -r requirements.txt --break-system-packages; fi
|
||||||
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt --break-system-packages; fi
|
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt --break-system-packages; fi
|
||||||
pip install bandit pip-audit ruff --break-system-packages
|
# Install the project + dev tools so lint/type/test gates run for real
|
||||||
|
pip install -e ".[dev]" --break-system-packages
|
||||||
|
pip install bandit pip-audit --break-system-packages
|
||||||
|
|
||||||
|
# Lint, type-check, and tests are hard gates — no `|| true`.
|
||||||
- name: Ruff lint
|
- name: Ruff lint
|
||||||
run: ruff check . || true
|
run: ruff check src tests
|
||||||
|
|
||||||
|
- name: Mypy
|
||||||
|
run: mypy src
|
||||||
|
|
||||||
- name: Bandit (advisory)
|
- name: Bandit (advisory)
|
||||||
run: bandit -r . -q || true
|
run: bandit -r . -q || true
|
||||||
@@ -58,13 +64,7 @@ jobs:
|
|||||||
run: pip-audit -r requirements.txt 2>/dev/null || pip-audit 2>/dev/null || true
|
run: pip-audit -r requirements.txt 2>/dev/null || pip-audit 2>/dev/null || true
|
||||||
|
|
||||||
- name: Pytest
|
- name: Pytest
|
||||||
run: |
|
run: pytest -q
|
||||||
if [ -d tests ] || ls test_*.py *_test.py 2>/dev/null; then
|
|
||||||
pip install pytest --break-system-packages
|
|
||||||
pytest -q || true
|
|
||||||
else
|
|
||||||
echo "No tests found — skip"
|
|
||||||
fi
|
|
||||||
|
|
||||||
secret-scan:
|
secret-scan:
|
||||||
needs: skip-ci-check
|
needs: skip-ci-check
|
||||||
|
|||||||
@@ -34,14 +34,15 @@ jobs:
|
|||||||
.venv/bin/pip install --upgrade pip
|
.venv/bin/pip install --upgrade pip
|
||||||
.venv/bin/pip install -e ".[dev]"
|
.venv/bin/pip install -e ".[dev]"
|
||||||
|
|
||||||
|
# Linters are hard gates — no `|| true`.
|
||||||
- name: Run linters
|
- name: Run linters
|
||||||
run: |
|
run: |
|
||||||
echo "Running ruff..."
|
echo "Running ruff..."
|
||||||
.venv/bin/ruff check src/ tests/ || true
|
.venv/bin/ruff check src/ tests/
|
||||||
echo "Running black check..."
|
echo "Running black check..."
|
||||||
.venv/bin/black --check src/ tests/ || true
|
.venv/bin/black --check src/ tests/
|
||||||
echo "Running mypy..."
|
echo "Running mypy..."
|
||||||
.venv/bin/mypy src/ --install-types --non-interactive || true
|
.venv/bin/mypy src/ --install-types --non-interactive
|
||||||
|
|
||||||
- name: Run tests with coverage
|
- name: Run tests with coverage
|
||||||
env:
|
env:
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# AGENTS.md — POTE
|
||||||
|
|
||||||
|
Short orientation for Cursor agents. Prefer this over rediscovering the repo.
|
||||||
|
|
||||||
|
## Defaults
|
||||||
|
|
||||||
|
- Read `README.md` for run/install; use `make help` when a Makefile exists.
|
||||||
|
- Lint and tests are **hard gates** in CI — never add `|| true` to them.
|
||||||
|
- **Do not** commit secrets; app creds live in Infisical (`secrets.levkin.ca`, `/apps/POTE`).
|
||||||
|
- Multi-step or ambiguous work → Plan mode first, then Agent mode.
|
||||||
|
|
||||||
|
## Docs voice
|
||||||
|
|
||||||
|
READMEs/guides: `~/Documents/code/project-template/docs/writing-docs.md` (no emoji decoration).
|
||||||
|
|
||||||
|
## Close-out
|
||||||
|
|
||||||
|
1. Run the narrowest verify that fits (`make test`, `npm test`, `pytest`, CI workflow locally).
|
||||||
|
2. Public GitHub mirror (`Gitilia/*`): after README/docs release merges, sync mirrors — see `ansible/docs/guides/github-mirrors.md`.
|
||||||
|
3. Non-trivial PR: offer review; merge only when the user asks.
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Ilia Dobkin
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -1,204 +1,67 @@
|
|||||||
# POTE – Public Officials Trading Explorer
|
# POTE (Public Officials Trading Explorer)
|
||||||
|
|
||||||
**Research-only tool for tracking and analyzing public stock trades by government officials.**
|
Research tool for tracking publicly disclosed stock trades by government
|
||||||
|
officials (starting with U.S. Congress). Computes descriptive metrics and
|
||||||
|
risk/ethics flags from lawfully available public data.
|
||||||
|
|
||||||
⚠️ **Important**: This project is for personal research and transparency analysis only. It is **NOT** for investment advice or live trading.
|
Not investment advice. Not for live trading. Public disclosures only; data
|
||||||
|
may be delayed or incomplete. No claims about inside information.
|
||||||
|
|
||||||
## What is this?
|
Status: active. Homelab LXC deploy documented under `docs/`.
|
||||||
|
|
||||||
POTE tracks stock trading activity of government officials (starting with U.S. Congress) using lawfully available public data sources. It computes research metrics, descriptive signals, and risk/ethics flags to help understand trading patterns.
|
|
||||||
|
|
||||||
## Key constraints
|
|
||||||
|
|
||||||
- **Public data only**: House Stock Watcher (free!), yfinance (free!), QuiverQuant/FMP (optional)
|
|
||||||
- **Research framing**: All outputs are descriptive analytics, not trading recommendations
|
|
||||||
- **No inside information claims**: We use public disclosures that may be delayed or incomplete
|
|
||||||
|
|
||||||
## Current Status
|
|
||||||
|
|
||||||
✅ **PR1 Complete**: Project scaffold, DB models, price loader
|
|
||||||
✅ **PR2 Complete**: Congressional trade ingestion (House Stock Watcher)
|
|
||||||
✅ **PR3 Complete**: Security enrichment + deployment infrastructure
|
|
||||||
✅ **PR4 Complete**: Phase 2 analytics - returns, benchmarks, performance metrics
|
|
||||||
**45+ passing tests, 88%+ coverage**
|
|
||||||
|
|
||||||
## Quick start
|
## Quick start
|
||||||
|
|
||||||
**🚀 Already deployed?** See **[QUICKSTART.md](QUICKSTART.md)** for full usage guide!
|
|
||||||
|
|
||||||
**📦 Deploying?** See **[PROXMOX_QUICKSTART.md](PROXMOX_QUICKSTART.md)** for Proxmox LXC deployment (recommended).
|
|
||||||
|
|
||||||
**📧 Want automated reports?** See **[AUTOMATION_QUICKSTART.md](AUTOMATION_QUICKSTART.md)** for email reporting setup!
|
|
||||||
|
|
||||||
**🏠 Homelab deploy (LXC 236)?** See **[docs/HANDOFF-2026-05-27.md](docs/HANDOFF-2026-05-27.md)** for ops handoff and next steps.
|
|
||||||
|
|
||||||
### Local Development
|
|
||||||
```bash
|
```bash
|
||||||
# Install
|
git clone <repository-url>
|
||||||
git clone <your-repo>
|
cd pote # or POTE
|
||||||
cd pote
|
|
||||||
make install
|
make install
|
||||||
source venv/bin/activate
|
source venv/bin/activate
|
||||||
|
|
||||||
# Run migrations
|
|
||||||
make migrate
|
make migrate
|
||||||
|
python scripts/ingest_from_fixtures.py # offline sample
|
||||||
|
make test
|
||||||
|
```
|
||||||
|
|
||||||
# Ingest sample data (offline, for testing)
|
With network:
|
||||||
python scripts/ingest_from_fixtures.py
|
|
||||||
|
|
||||||
# Enrich securities with company info
|
```bash
|
||||||
python scripts/enrich_securities.py
|
|
||||||
|
|
||||||
# With internet:
|
|
||||||
python scripts/fetch_congressional_trades.py
|
python scripts/fetch_congressional_trades.py
|
||||||
python scripts/fetch_sample_prices.py
|
python scripts/fetch_sample_prices.py
|
||||||
|
|
||||||
# Run tests
|
|
||||||
make test
|
|
||||||
|
|
||||||
# Lint & format
|
|
||||||
make lint format
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Production Deployment
|
Deploy: `bash scripts/proxmox_setup.sh` or `docker-compose up -d`.
|
||||||
|
Ops handoff: [docs/HANDOFF-2026-05-27.md](docs/HANDOFF-2026-05-27.md).
|
||||||
|
|
||||||
|
## Useful commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Proxmox LXC (Recommended - 5 minutes)
|
|
||||||
bash scripts/proxmox_setup.sh
|
|
||||||
|
|
||||||
# Docker
|
|
||||||
docker-compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## Tech stack
|
|
||||||
|
|
||||||
- **Language**: Python 3.10+
|
|
||||||
- **Database**: PostgreSQL or SQLite (dev)
|
|
||||||
- **Data**: House Stock Watcher (free!), yfinance (free!), QuiverQuant/FMP (optional)
|
|
||||||
- **Libraries**: SQLAlchemy, Alembic, pandas, numpy, httpx, yfinance, scikit-learn
|
|
||||||
- **Testing**: pytest (28 tests, 87%+ coverage)
|
|
||||||
|
|
||||||
## Documentation
|
|
||||||
|
|
||||||
**Getting Started**:
|
|
||||||
- [`README.md`](README.md) – This file
|
|
||||||
- [`QUICKSTART.md`](QUICKSTART.md) – ⭐ **How to use your deployed POTE instance**
|
|
||||||
- [`STATUS.md`](STATUS.md) – Current project status
|
|
||||||
- [`FREE_TESTING_QUICKSTART.md`](FREE_TESTING_QUICKSTART.md) – Test for $0
|
|
||||||
- [`OFFLINE_DEMO.md`](OFFLINE_DEMO.md) – Works without internet!
|
|
||||||
|
|
||||||
**Deployment**:
|
|
||||||
- [`PROXMOX_QUICKSTART.md`](PROXMOX_QUICKSTART.md) – ⭐ **Proxmox quick deployment (5 min)**
|
|
||||||
- [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md) – ⭐ **Automated reporting setup (5 min)**
|
|
||||||
- [`docs/07_deployment.md`](docs/07_deployment.md) – Full deployment guide (all platforms)
|
|
||||||
- [`docs/08_proxmox_deployment.md`](docs/08_proxmox_deployment.md) – Proxmox detailed guide
|
|
||||||
- [`docs/12_automation_and_reporting.md`](docs/12_automation_and_reporting.md) – Automation & CI/CD guide
|
|
||||||
- [`Dockerfile`](Dockerfile) + [`docker-compose.yml`](docker-compose.yml) – Docker setup
|
|
||||||
|
|
||||||
**Technical**:
|
|
||||||
- [`docs/00_mvp.md`](docs/00_mvp.md) – MVP roadmap
|
|
||||||
- [`docs/01_architecture.md`](docs/01_architecture.md) – Architecture
|
|
||||||
- [`docs/02_data_model.md`](docs/02_data_model.md) – Database schema
|
|
||||||
- [`docs/03_data_sources.md`](docs/03_data_sources.md) – Data sources
|
|
||||||
- [`docs/04_safety_ethics.md`](docs/04_safety_ethics.md) – Research-only guardrails
|
|
||||||
- [`docs/05_dev_setup.md`](docs/05_dev_setup.md) – Dev conventions
|
|
||||||
- [`docs/06_free_testing_data.md`](docs/06_free_testing_data.md) – Testing strategies
|
|
||||||
|
|
||||||
**PR Summaries**:
|
|
||||||
- [`docs/PR1_SUMMARY.md`](docs/PR1_SUMMARY.md) – Scaffold + price loader
|
|
||||||
- [`docs/PR2_SUMMARY.md`](docs/PR2_SUMMARY.md) – Congressional trades
|
|
||||||
- [`docs/PR3_SUMMARY.md`](docs/PR3_SUMMARY.md) – Enrichment + deployment
|
|
||||||
- [`docs/PR4_SUMMARY.md`](docs/PR4_SUMMARY.md) – ⭐ **Analytics foundation (returns, benchmarks, metrics)**
|
|
||||||
|
|
||||||
## What's Working Now
|
|
||||||
|
|
||||||
- ✅ SQLAlchemy models for officials, securities, trades, prices
|
|
||||||
- ✅ Alembic migrations
|
|
||||||
- ✅ Price loader with yfinance (idempotent, upsert)
|
|
||||||
- ✅ Congressional trade ingestion from House Stock Watcher (FREE!)
|
|
||||||
- ✅ Security enrichment (company names, sectors, industries)
|
|
||||||
- ✅ ETL to populate officials & trades tables
|
|
||||||
- ✅ Docker + deployment infrastructure
|
|
||||||
- ✅ 93 passing tests with 88%+ coverage
|
|
||||||
- ✅ Linting (ruff + mypy) all green
|
|
||||||
- ✅ Works 100% offline with fixtures
|
|
||||||
- ✅ Real-time market monitoring & alert system
|
|
||||||
- ✅ Disclosure timing correlation engine
|
|
||||||
- ✅ Pattern detection & comparative analysis
|
|
||||||
- ✅ Automated email reporting (daily/weekly)
|
|
||||||
- ✅ CI/CD pipeline (GitHub/Gitea Actions)
|
|
||||||
|
|
||||||
## What You Can Do Now
|
|
||||||
|
|
||||||
### Analyze Performance
|
|
||||||
```bash
|
|
||||||
# Analyze specific official
|
|
||||||
python scripts/analyze_official.py "Nancy Pelosi" --window 90
|
python scripts/analyze_official.py "Nancy Pelosi" --window 90
|
||||||
|
|
||||||
# System-wide analysis
|
|
||||||
python scripts/calculate_all_returns.py
|
python scripts/calculate_all_returns.py
|
||||||
```
|
|
||||||
|
|
||||||
### Market Monitoring
|
|
||||||
```bash
|
|
||||||
# Run market scan
|
|
||||||
python scripts/monitor_market.py --scan
|
python scripts/monitor_market.py --scan
|
||||||
|
|
||||||
# Analyze timing of recent disclosures
|
|
||||||
python scripts/analyze_disclosure_timing.py --recent 7
|
|
||||||
|
|
||||||
# Generate pattern report
|
|
||||||
python scripts/generate_pattern_report.py --days 365
|
|
||||||
```
|
|
||||||
|
|
||||||
### Automated Reporting
|
|
||||||
```bash
|
|
||||||
# Set up daily/weekly email reports (5 minutes!)
|
|
||||||
./scripts/setup_cron.sh
|
|
||||||
|
|
||||||
# Send manual report
|
|
||||||
python scripts/send_daily_report.py --to your@email.com
|
python scripts/send_daily_report.py --to your@email.com
|
||||||
```
|
```
|
||||||
|
|
||||||
### Add More Data
|
## Stack
|
||||||
```bash
|
|
||||||
# Manual entry
|
|
||||||
python scripts/add_custom_trades.py
|
|
||||||
|
|
||||||
# CSV import
|
Python 3.10+, SQLAlchemy, Alembic, pandas/numpy, httpx, yfinance.
|
||||||
python scripts/scrape_alternative_sources.py import trades.csv
|
PostgreSQL or SQLite for local. pytest + ruff + mypy.
|
||||||
```
|
|
||||||
|
|
||||||
## System Architecture
|
Data sources: House Stock Watcher, yfinance; QuiverQuant/FMP optional.
|
||||||
|
|
||||||
POTE now includes a complete 3-phase monitoring system:
|
## Docs
|
||||||
|
|
||||||
**Phase 1: Real-Time Market Monitoring**
|
| Doc | Purpose |
|
||||||
- Tracks ~50 most-traded congressional stocks
|
|-----|---------|
|
||||||
- Detects unusual volume, price spikes, volatility
|
| [docs/QUICKSTART.md](docs/QUICKSTART.md) | Using a deployed instance |
|
||||||
- Logs all alerts with timestamps and severity
|
| [docs/PROXMOX_QUICKSTART.md](docs/PROXMOX_QUICKSTART.md) | Proxmox LXC |
|
||||||
|
| [docs/AUTOMATION_QUICKSTART.md](docs/AUTOMATION_QUICKSTART.md) | Email reports |
|
||||||
|
| [docs/01_architecture.md](docs/01_architecture.md) | Architecture |
|
||||||
|
| [docs/02_data_model.md](docs/02_data_model.md) | Schema |
|
||||||
|
| [docs/04_safety_ethics.md](docs/04_safety_ethics.md) | Research guardrails |
|
||||||
|
| [docs/07_deployment.md](docs/07_deployment.md) | Full deploy |
|
||||||
|
| [docs/00_mvp.md](docs/00_mvp.md) | Roadmap |
|
||||||
|
|
||||||
**Phase 2: Disclosure Correlation**
|
PR writeups live under `docs/archive/` when moved.
|
||||||
- Matches trades with prior market alerts (30-45 day lookback)
|
|
||||||
- Calculates "timing advantage score" (0-100)
|
|
||||||
- Identifies suspicious timing patterns
|
|
||||||
|
|
||||||
**Phase 3: Pattern Detection**
|
## License
|
||||||
- Ranks officials by consistent suspicious timing
|
|
||||||
- Analyzes by ticker, sector, and political party
|
|
||||||
- Generates comprehensive reports
|
|
||||||
|
|
||||||
**Full Documentation**: See [`MONITORING_SYSTEM_COMPLETE.md`](MONITORING_SYSTEM_COMPLETE.md)
|
MIT for research/educational use. Not investment advice.
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
- [ ] Signals: "follow_research", "avoid_risk", "watch" with confidence scores
|
|
||||||
- [ ] Clustering: group officials by trading behavior patterns
|
|
||||||
- [ ] API: FastAPI backend for queries
|
|
||||||
- [ ] Dashboard: React/Streamlit visualization
|
|
||||||
|
|
||||||
See [`docs/00_mvp.md`](docs/00_mvp.md) for the full roadmap.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**License**: MIT (for research/educational use only)
|
|
||||||
**Disclaimer**: Not investment advice. Use public data only. No claims about inside information.
|
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ Great question! Here are multiple strategies for testing the full pipeline **wit
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Strategy 1: Mock/Fixture Data (Current Approach ✅)
|
## Strategy 1: Mock/Fixture Data (Current Approach )
|
||||||
|
|
||||||
**What we already have:**
|
**What we already have:**
|
||||||
- `tests/conftest.py` creates in-memory SQLite DB with sample officials, securities, trades
|
- `tests/conftest.py` creates in-memory SQLite DB with sample officials, securities, trades
|
||||||
@@ -99,7 +99,7 @@ def test_etl_with_real_data():
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Strategy 4: Hybrid Testing (Recommended 🌟)
|
## Strategy 4: Hybrid Testing (Recommended )
|
||||||
|
|
||||||
**Combine all strategies**:
|
**Combine all strategies**:
|
||||||
|
|
||||||
@@ -164,7 +164,7 @@ def test_etl_with_real_data():
|
|||||||
|
|
||||||
| Source | Free Tier | Paid Tier | Best For |
|
| Source | Free Tier | Paid Tier | Best For |
|
||||||
|--------|-----------|-----------|----------|
|
|--------|-----------|-----------|----------|
|
||||||
| **yfinance** | Unlimited | N/A | Prices (already working ✅) |
|
| **yfinance** | Unlimited | N/A | Prices (already working ) |
|
||||||
| **House Stock Watcher** | Unlimited scraping | N/A | Free trades (best option) |
|
| **House Stock Watcher** | Unlimited scraping | N/A | Free trades (best option) |
|
||||||
| **Quiver Free** | 500 calls/mo | $30/mo (5k calls) | Testing, not production |
|
| **Quiver Free** | 500 calls/mo | $30/mo (5k calls) | Testing, not production |
|
||||||
| **FMP Free** | 250 calls/day | $15/mo | Alternative for trades |
|
| **FMP Free** | 250 calls/day | $15/mo | Alternative for trades |
|
||||||
@@ -222,5 +222,5 @@ class HouseWatcherClient:
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
Let me know if you want me to implement this scraper now for PR2! 🚀
|
Let me know if you want me to implement this scraper now for PR2!
|
||||||
|
|
||||||
|
|||||||
@@ -4,14 +4,14 @@
|
|||||||
|
|
||||||
POTE can be deployed in several ways depending on your needs:
|
POTE can be deployed in several ways depending on your needs:
|
||||||
|
|
||||||
1. **Local Development** (SQLite) - What you have now ✅
|
1. **Local Development** (SQLite) - What you have now
|
||||||
2. **Single Server** (PostgreSQL + cron jobs)
|
2. **Single Server** (PostgreSQL + cron jobs)
|
||||||
3. **Docker** (Containerized, easy to move)
|
3. **Docker** (Containerized, easy to move)
|
||||||
4. **Cloud** (AWS/GCP/Azure with managed DB)
|
4. **Cloud** (AWS/GCP/Azure with managed DB)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Option 1: Local Development (Current Setup) ✅
|
## Option 1: Local Development (Current Setup)
|
||||||
|
|
||||||
**You're already running this!**
|
**You're already running this!**
|
||||||
|
|
||||||
|
|||||||
@@ -2,17 +2,17 @@
|
|||||||
|
|
||||||
## Why Proxmox is Perfect for POTE
|
## Why Proxmox is Perfect for POTE
|
||||||
|
|
||||||
✅ **Full control** - Your hardware, your rules
|
**Full control** - Your hardware, your rules
|
||||||
✅ **No monthly costs** - Just electricity
|
**No monthly costs** - Just electricity
|
||||||
✅ **Isolated VMs/LXC** - Clean environments
|
**Isolated VMs/LXC** - Clean environments
|
||||||
✅ **Snapshots** - Easy rollback if needed
|
**Snapshots** - Easy rollback if needed
|
||||||
✅ **Resource efficient** - Run alongside other services
|
**Resource efficient** - Run alongside other services
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Deployment Options on Proxmox
|
## Deployment Options on Proxmox
|
||||||
|
|
||||||
### Option 1: LXC Container (Recommended) ⭐
|
### Option 1: LXC Container (Recommended)
|
||||||
|
|
||||||
**Pros**: Lightweight, fast, efficient resource usage
|
**Pros**: Lightweight, fast, efficient resource usage
|
||||||
**Cons**: Linux only (fine for POTE)
|
**Cons**: Linux only (fine for POTE)
|
||||||
@@ -501,14 +501,14 @@ vs.
|
|||||||
|
|
||||||
## Next Steps
|
## Next Steps
|
||||||
|
|
||||||
1. ✅ Create LXC container
|
1. Create LXC container
|
||||||
2. ✅ Install dependencies
|
2. Install dependencies
|
||||||
3. ✅ Setup PostgreSQL
|
3. Setup PostgreSQL
|
||||||
4. ✅ Deploy POTE
|
4. Deploy POTE
|
||||||
5. ✅ Configure cron jobs
|
5. Configure cron jobs
|
||||||
6. ✅ Setup backups
|
6. Setup backups
|
||||||
7. ⏭️ Build Phase 2 (Analytics)
|
7. ⏭ Build Phase 2 (Analytics)
|
||||||
8. ⏭️ Add FastAPI dashboard (optional)
|
8. ⏭ Add FastAPI dashboard (optional)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -583,7 +583,7 @@ EOF
|
|||||||
sudo -u poteapp mkdir -p /home/poteapp/logs
|
sudo -u poteapp mkdir -p /home/poteapp/logs
|
||||||
|
|
||||||
echo ""
|
echo ""
|
||||||
echo "✅ Setup complete!"
|
echo " Setup complete!"
|
||||||
echo ""
|
echo ""
|
||||||
echo "Next steps:"
|
echo "Next steps:"
|
||||||
echo "1. su - poteapp"
|
echo "1. su - poteapp"
|
||||||
@@ -600,5 +600,5 @@ chmod +x proxmox_setup.sh
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Your Proxmox setup gives you enterprise-grade infrastructure at hobby costs!** 🚀
|
**Your Proxmox setup gives you enterprise-grade infrastructure at hobby costs!**
|
||||||
|
|
||||||
|
|||||||
@@ -137,13 +137,13 @@ python scripts/fetch_sample_prices.py
|
|||||||
## Data Sources
|
## Data Sources
|
||||||
|
|
||||||
### Currently Working:
|
### Currently Working:
|
||||||
- ✅ yfinance (prices, company info)
|
- yfinance (prices, company info)
|
||||||
- ✅ Manual entry
|
- Manual entry
|
||||||
- ✅ CSV import
|
- CSV import
|
||||||
- ✅ Fixture files (testing)
|
- Fixture files (testing)
|
||||||
|
|
||||||
### Currently Down:
|
### Currently Down:
|
||||||
- ❌ House Stock Watcher API (domain issues)
|
- House Stock Watcher API (domain issues)
|
||||||
|
|
||||||
### Future Options:
|
### Future Options:
|
||||||
- QuiverQuant (requires $30/month subscription)
|
- QuiverQuant (requires $30/month subscription)
|
||||||
|
|||||||
+19
-19
@@ -8,10 +8,10 @@
|
|||||||
### **Reality Check: No Real-Time Data Exists**
|
### **Reality Check: No Real-Time Data Exists**
|
||||||
|
|
||||||
**Federal Law (STOCK Act):**
|
**Federal Law (STOCK Act):**
|
||||||
- 📅 Congress members have **30-45 days** to disclose trades
|
- Congress members have **30-45 days** to disclose trades
|
||||||
- 📅 Disclosures are filed as **Periodic Transaction Reports (PTRs)**
|
- Disclosures are filed as **Periodic Transaction Reports (PTRs)**
|
||||||
- 📅 Public databases update **after** filing (usually next day)
|
- Public databases update **after** filing (usually next day)
|
||||||
- 📅 **No real-time feed exists by design**
|
- **No real-time feed exists by design**
|
||||||
|
|
||||||
**Example Timeline:**
|
**Example Timeline:**
|
||||||
```
|
```
|
||||||
@@ -25,15 +25,15 @@ Feb 17, 2024 → Your system fetches it
|
|||||||
|
|
||||||
Since trades appear in batches (not continuously), **running once per day is optimal**:
|
Since trades appear in batches (not continuously), **running once per day is optimal**:
|
||||||
|
|
||||||
✅ **Daily (7 AM)** - Catches overnight filings
|
**Daily (7 AM)** - Catches overnight filings
|
||||||
✅ **After market close** - Prices are final
|
**After market close** - Prices are final
|
||||||
✅ **Low server load** - Off-peak hours
|
**Low server load** - Off-peak hours
|
||||||
❌ **Hourly** - Wasteful, no new data
|
**Hourly** - Wasteful, no new data
|
||||||
❌ **Real-time** - Impossible, not how disclosures work
|
**Real-time** - Impossible, not how disclosures work
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🤖 Automated Setup Options
|
## Automated Setup Options
|
||||||
|
|
||||||
### **Option 1: Cron Job (Linux/Proxmox) - Recommended**
|
### **Option 1: Cron Job (Linux/Proxmox) - Recommended**
|
||||||
|
|
||||||
@@ -149,7 +149,7 @@ Or from anywhere:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 What Gets Updated?
|
## What Gets Updated?
|
||||||
|
|
||||||
### **1. Congressional Trades**
|
### **1. Congressional Trades**
|
||||||
**Script:** `fetch_congressional_trades.py`
|
**Script:** `fetch_congressional_trades.py`
|
||||||
@@ -182,7 +182,7 @@ Or from anywhere:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚙️ Customizing the Schedule
|
## Customizing the Schedule
|
||||||
|
|
||||||
### **Different Frequencies**
|
### **Different Frequencies**
|
||||||
|
|
||||||
@@ -215,7 +215,7 @@ Or from anywhere:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📧 Email Notifications (Optional)
|
## Email Notifications (Optional)
|
||||||
|
|
||||||
### **Setup Email Alerts**
|
### **Setup Email Alerts**
|
||||||
|
|
||||||
@@ -310,7 +310,7 @@ python scripts/email_summary.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔍 Monitoring & Logging
|
## Monitoring & Logging
|
||||||
|
|
||||||
### **Check Cron Job Status**
|
### **Check Cron Job Status**
|
||||||
|
|
||||||
@@ -355,7 +355,7 @@ Add to `/etc/logrotate.d/pote`:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚨 Handling Failures
|
## Handling Failures
|
||||||
|
|
||||||
### **What If House Stock Watcher Is Down?**
|
### **What If House Stock Watcher Is Down?**
|
||||||
|
|
||||||
@@ -363,7 +363,7 @@ The script is designed to continue even if one step fails:
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Script continues and logs warnings
|
# Script continues and logs warnings
|
||||||
⚠️ WARNING: Failed to fetch congressional trades
|
WARNING: Failed to fetch congressional trades
|
||||||
This is likely because House Stock Watcher API is down
|
This is likely because House Stock Watcher API is down
|
||||||
Continuing with other steps...
|
Continuing with other steps...
|
||||||
```
|
```
|
||||||
@@ -399,7 +399,7 @@ for attempt in range(MAX_RETRIES):
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 Performance Optimization
|
## Performance Optimization
|
||||||
|
|
||||||
### **Batch Processing**
|
### **Batch Processing**
|
||||||
|
|
||||||
@@ -439,7 +439,7 @@ CREATE INDEX IF NOT EXISTS ix_prices_security_id ON prices(security_id);
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Recommended Setup
|
## Recommended Setup
|
||||||
|
|
||||||
### **For Proxmox Production:**
|
### **For Proxmox Production:**
|
||||||
|
|
||||||
@@ -475,7 +475,7 @@ pote-update
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📝 Summary
|
## Summary
|
||||||
|
|
||||||
### **Key Points:**
|
### **Key Points:**
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
# Live Market Monitoring + Congressional Trading Analysis
|
# Live Market Monitoring + Congressional Trading Analysis
|
||||||
|
|
||||||
## 🎯 What's Possible vs Impossible
|
## What's Possible vs Impossible
|
||||||
|
|
||||||
### ❌ **NOT Possible:**
|
### **NOT Possible:**
|
||||||
- Identify WHO is buying/selling in real-time
|
- Identify WHO is buying/selling in real-time
|
||||||
- Match live trades to specific Congress members
|
- Match live trades to specific Congress members
|
||||||
- See congressional trades before they're disclosed
|
- See congressional trades before they're disclosed
|
||||||
|
|
||||||
### ✅ **IS Possible:**
|
### **IS Possible:**
|
||||||
- Track unusual market activity in real-time
|
- Track unusual market activity in real-time
|
||||||
- Monitor stocks Congress members historically trade
|
- Monitor stocks Congress members historically trade
|
||||||
- Compare unusual activity to later disclosures
|
- Compare unusual activity to later disclosures
|
||||||
@@ -15,7 +15,7 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 **Two-Phase Monitoring System**
|
## **Two-Phase Monitoring System**
|
||||||
|
|
||||||
### **Phase 1: Real-Time Market Monitoring**
|
### **Phase 1: Real-Time Market Monitoring**
|
||||||
Monitor unusual activity in stocks Congress trades:
|
Monitor unusual activity in stocks Congress trades:
|
||||||
@@ -33,7 +33,7 @@ When disclosures come in:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 **Implementation: Watchlist-Based Monitoring**
|
## **Implementation: Watchlist-Based Monitoring**
|
||||||
|
|
||||||
### **Concept:**
|
### **Concept:**
|
||||||
|
|
||||||
@@ -57,38 +57,38 @@ Step 3: When Disclosure Appears (30-45 days later)
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🛠️ **Data Sources for Live Market Monitoring**
|
## **Data Sources for Live Market Monitoring**
|
||||||
|
|
||||||
### **Free/Low-Cost Options:**
|
### **Free/Low-Cost Options:**
|
||||||
|
|
||||||
1. **Yahoo Finance (yfinance)**
|
1. **Yahoo Finance (yfinance)**
|
||||||
- ✅ Real-time quotes (15-min delay free)
|
- Real-time quotes (15-min delay free)
|
||||||
- ✅ Historical options data
|
- Historical options data
|
||||||
- ✅ Volume data
|
- Volume data
|
||||||
- ❌ Not true real-time for options flow
|
- Not true real-time for options flow
|
||||||
|
|
||||||
2. **Unusual Whales API**
|
2. **Unusual Whales API**
|
||||||
- ✅ Options flow data
|
- Options flow data
|
||||||
- ✅ Unusual activity alerts
|
- Unusual activity alerts
|
||||||
- 💰 Paid ($50-200/month)
|
- Paid ($50-200/month)
|
||||||
- https://unusualwhales.com/
|
- https://unusualwhales.com/
|
||||||
|
|
||||||
3. **Tradier API**
|
3. **Tradier API**
|
||||||
- ✅ Real-time market data
|
- Real-time market data
|
||||||
- ✅ Options chains
|
- Options chains
|
||||||
- 💰 Paid but affordable ($10-50/month)
|
- Paid but affordable ($10-50/month)
|
||||||
- https://tradier.com/
|
- https://tradier.com/
|
||||||
|
|
||||||
4. **FlowAlgo**
|
4. **FlowAlgo**
|
||||||
- ✅ Options flow tracking
|
- Options flow tracking
|
||||||
- ✅ Dark pool data
|
- Dark pool data
|
||||||
- 💰 Paid ($99-399/month)
|
- Paid ($99-399/month)
|
||||||
- https://www.flowalgo.com/
|
- https://www.flowalgo.com/
|
||||||
|
|
||||||
5. **Polygon.io**
|
5. **Polygon.io**
|
||||||
- ✅ Real-time stock data
|
- Real-time stock data
|
||||||
- ✅ Options data
|
- Options data
|
||||||
- 💰 Free tier + paid plans
|
- Free tier + paid plans
|
||||||
- https://polygon.io/
|
- https://polygon.io/
|
||||||
|
|
||||||
### **Best Free Option: Build Your Own with yfinance**
|
### **Best Free Option: Build Your Own with yfinance**
|
||||||
@@ -97,7 +97,7 @@ Track volume/price changes every 5 minutes for congressional watchlist tickers.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 💡 **Practical Hybrid System**
|
## **Practical Hybrid System**
|
||||||
|
|
||||||
### **What We Can Build:**
|
### **What We Can Build:**
|
||||||
|
|
||||||
@@ -141,37 +141,37 @@ for disclosure in new_disclosures:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 **Example: Nancy Pelosi NVDA Trade Analysis**
|
## **Example: Nancy Pelosi NVDA Trade Analysis**
|
||||||
|
|
||||||
### **Timeline:**
|
### **Timeline:**
|
||||||
|
|
||||||
```
|
```
|
||||||
Nov 10, 2024:
|
Nov 10, 2024:
|
||||||
🔔 ALERT: NVDA unusual call options activity
|
ALERT: NVDA unusual call options activity
|
||||||
Volume: 10x average
|
Volume: 10x average
|
||||||
Strike: $500 (2 weeks out)
|
Strike: $500 (2 weeks out)
|
||||||
|
|
||||||
Nov 15, 2024:
|
Nov 15, 2024:
|
||||||
💰 Someone buys NVDA (unknown who at the time)
|
Someone buys NVDA (unknown who at the time)
|
||||||
|
|
||||||
Nov 18, 2024:
|
Nov 18, 2024:
|
||||||
📰 NVDA announces new AI chip
|
NVDA announces new AI chip
|
||||||
📈 Stock jumps 15%
|
Stock jumps 15%
|
||||||
|
|
||||||
Dec 15, 2024:
|
Dec 15, 2024:
|
||||||
📋 Disclosure: Nancy Pelosi bought NVDA on Nov 15
|
Disclosure: Nancy Pelosi bought NVDA on Nov 15
|
||||||
Value: $15,001-$50,000
|
Value: $15,001-$50,000
|
||||||
|
|
||||||
ANALYSIS:
|
ANALYSIS:
|
||||||
✅ She bought AFTER unusual options activity (Nov 10)
|
She bought AFTER unusual options activity (Nov 10)
|
||||||
❓ She bought BEFORE announcement (Nov 18)
|
She bought BEFORE announcement (Nov 18)
|
||||||
⏱️ Timing: 3 days before major news
|
⏱ Timing: 3 days before major news
|
||||||
🚩 Flag: Investigate if announcement was public knowledge
|
Flag: Investigate if announcement was public knowledge
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 **Recommended Approach**
|
## **Recommended Approach**
|
||||||
|
|
||||||
### **Phase 1: Build Congressional Ticker Watchlist**
|
### **Phase 1: Build Congressional Ticker Watchlist**
|
||||||
|
|
||||||
@@ -230,12 +230,12 @@ def monitor_tickers(tickers, interval_minutes=5):
|
|||||||
# Check for unusual volume
|
# Check for unusual volume
|
||||||
avg_volume = current['Volume'].mean()
|
avg_volume = current['Volume'].mean()
|
||||||
if latest['Volume'] > avg_volume * 3:
|
if latest['Volume'] > avg_volume * 3:
|
||||||
alert(f"🔔 {ticker}: Unusual volume spike!")
|
alert(f" {ticker}: Unusual volume spike!")
|
||||||
|
|
||||||
# Check for price movement
|
# Check for price movement
|
||||||
price_change = (latest['Close'] - current['Open'].iloc[0]) / current['Open'].iloc[0]
|
price_change = (latest['Close'] - current['Open'].iloc[0]) / current['Open'].iloc[0]
|
||||||
if abs(price_change) > 0.05: # 5% move
|
if abs(price_change) > 0.05: # 5% move
|
||||||
alert(f"📈 {ticker}: {price_change:.2%} move today!")
|
alert(f" {ticker}: {price_change:.2%} move today!")
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
print(f"Error monitoring {ticker}: {e}")
|
print(f"Error monitoring {ticker}: {e}")
|
||||||
@@ -290,31 +290,31 @@ def analyze_disclosure_timing(disclosure):
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚨 **Realistic Expectations**
|
## **Realistic Expectations**
|
||||||
|
|
||||||
### **What This System Will Do:**
|
### **What This System Will Do:**
|
||||||
✅ Monitor stocks Congress members historically trade
|
Monitor stocks Congress members historically trade
|
||||||
✅ Alert on unusual market activity in those stocks
|
Alert on unusual market activity in those stocks
|
||||||
✅ Retroactively correlate disclosures with earlier alerts
|
Retroactively correlate disclosures with earlier alerts
|
||||||
✅ Identify timing patterns and potential advantages
|
Identify timing patterns and potential advantages
|
||||||
✅ Build database of congressional trading patterns
|
Build database of congressional trading patterns
|
||||||
|
|
||||||
### **What This System WON'T Do:**
|
### **What This System WON'T Do:**
|
||||||
❌ Identify WHO is buying in real-time
|
Identify WHO is buying in real-time
|
||||||
❌ Give you advance notice of congressional trades
|
Give you advance notice of congressional trades
|
||||||
❌ Provide real-time inside information
|
Provide real-time inside information
|
||||||
❌ Allow you to "front-run" Congress
|
Allow you to "front-run" Congress
|
||||||
|
|
||||||
### **Legal & Ethical:**
|
### **Legal & Ethical:**
|
||||||
✅ All data is public
|
All data is public
|
||||||
✅ Analysis is retrospective
|
Analysis is retrospective
|
||||||
✅ For research and transparency
|
For research and transparency
|
||||||
✅ Not market manipulation
|
Not market manipulation
|
||||||
❌ Cannot and should not be used to replicate potentially illegal trades
|
Cannot and should not be used to replicate potentially illegal trades
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 **Proposed Implementation**
|
## **Proposed Implementation**
|
||||||
|
|
||||||
### **New Scripts to Create:**
|
### **New Scripts to Create:**
|
||||||
|
|
||||||
@@ -365,15 +365,15 @@ CREATE TABLE disclosure_timing_analysis (
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 **Summary**
|
## **Summary**
|
||||||
|
|
||||||
### **Your Question:**
|
### **Your Question:**
|
||||||
> "Can we read live trades being made and compare them to a name?"
|
> "Can we read live trades being made and compare them to a name?"
|
||||||
|
|
||||||
### **Answer:**
|
### **Answer:**
|
||||||
❌ **No** - Live trades are anonymous, can't identify individuals
|
**No** - Live trades are anonymous, can't identify individuals
|
||||||
|
|
||||||
✅ **BUT** - You CAN:
|
**BUT** - You CAN:
|
||||||
1. Monitor unusual activity in stocks Congress trades
|
1. Monitor unusual activity in stocks Congress trades
|
||||||
2. Log these alerts in real-time
|
2. Log these alerts in real-time
|
||||||
3. When disclosures appear (30-45 days later), correlate them
|
3. When disclosures appear (30-45 days later), correlate them
|
||||||
@@ -381,25 +381,25 @@ CREATE TABLE disclosure_timing_analysis (
|
|||||||
5. Build patterns database of timing and performance
|
5. Build patterns database of timing and performance
|
||||||
|
|
||||||
### **This Gives You:**
|
### **This Gives You:**
|
||||||
- ✅ Transparency on timing advantages
|
- Transparency on timing advantages
|
||||||
- ✅ Pattern detection across officials
|
- Pattern detection across officials
|
||||||
- ✅ Research-grade analysis
|
- Research-grade analysis
|
||||||
- ✅ Historical correlation data
|
- Historical correlation data
|
||||||
|
|
||||||
### **This Does NOT Give You:**
|
### **This Does NOT Give You:**
|
||||||
- ❌ Real-time identity of traders
|
- Real-time identity of traders
|
||||||
- ❌ Advance notice of congressional trades
|
- Advance notice of congressional trades
|
||||||
- ❌ Ability to "front-run" disclosures
|
- Ability to "front-run" disclosures
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 **Would You Like Me To Build This?**
|
## **Would You Like Me To Build This?**
|
||||||
|
|
||||||
I can create:
|
I can create:
|
||||||
1. ✅ Real-time monitoring system for congressional tickers
|
1. Real-time monitoring system for congressional tickers
|
||||||
2. ✅ Alert logging and analysis
|
2. Alert logging and analysis
|
||||||
3. ✅ Timing correlation when disclosures appear
|
3. Timing correlation when disclosures appear
|
||||||
4. ✅ Pattern detection and reporting
|
4. Pattern detection and reporting
|
||||||
|
|
||||||
This would be **Phase 2.5** of POTE - the "timing analysis" module.
|
This would be **Phase 2.5** of POTE - the "timing analysis" module.
|
||||||
|
|
||||||
|
|||||||
@@ -205,18 +205,18 @@ Output:
|
|||||||
POTE HEALTH CHECK
|
POTE HEALTH CHECK
|
||||||
============================================================
|
============================================================
|
||||||
Timestamp: 2025-12-15T10:30:00
|
Timestamp: 2025-12-15T10:30:00
|
||||||
Overall Status: ✓ OK
|
Overall Status: OK
|
||||||
|
|
||||||
✓ Database Connection: Database connection successful
|
Database Connection: Database connection successful
|
||||||
✓ Data Freshness: Data is fresh (2 days old)
|
Data Freshness: Data is fresh (2 days old)
|
||||||
latest_trade_date: 2025-12-13
|
latest_trade_date: 2025-12-13
|
||||||
✓ Data Counts: Database has 1,234 trades
|
Data Counts: Database has 1,234 trades
|
||||||
officials: 45
|
officials: 45
|
||||||
securities: 123
|
securities: 123
|
||||||
trades: 1,234
|
trades: 1,234
|
||||||
prices: 12,345
|
prices: 12,345
|
||||||
market_alerts: 567
|
market_alerts: 567
|
||||||
✓ Recent Alerts: 23 alerts in last 24 hours
|
Recent Alerts: 23 alerts in last 24 hours
|
||||||
============================================================
|
============================================================
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -22,18 +22,18 @@ chmod 600 .env
|
|||||||
chown poteapp:poteapp .env
|
chown poteapp:poteapp .env
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Pros
|
### Pros
|
||||||
- Simple, works immediately
|
- Simple, works immediately
|
||||||
- No additional setup
|
- No additional setup
|
||||||
- Standard practice for Python projects
|
- Standard practice for Python projects
|
||||||
|
|
||||||
### ⚠️ Cons
|
### Cons
|
||||||
- Secrets stored in plain text on disk
|
- Secrets stored in plain text on disk
|
||||||
- Risk if server is compromised
|
- Risk if server is compromised
|
||||||
- No audit trail
|
- No audit trail
|
||||||
|
|
||||||
### 🔒 Security Checklist
|
### Security Checklist
|
||||||
- [ ] `.env` in `.gitignore` (already done ✅)
|
- [ ] `.env` in `.gitignore` (already done )
|
||||||
- [ ] File permissions: `chmod 600 .env`
|
- [ ] File permissions: `chmod 600 .env`
|
||||||
- [ ] Never commit to git
|
- [ ] Never commit to git
|
||||||
- [ ] Backup securely (encrypted)
|
- [ ] Backup securely (encrypted)
|
||||||
@@ -76,12 +76,12 @@ sudo chmod 600 /etc/systemd/system/pote.service
|
|||||||
sudo systemctl daemon-reload
|
sudo systemctl daemon-reload
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Pros
|
### Pros
|
||||||
- Secrets not in git or project directory
|
- Secrets not in git or project directory
|
||||||
- Standard Linux practice
|
- Standard Linux practice
|
||||||
- Works with systemd timers
|
- Works with systemd timers
|
||||||
|
|
||||||
### ⚠️ Cons
|
### Cons
|
||||||
- Still visible in `systemctl show`
|
- Still visible in `systemctl show`
|
||||||
- Requires root to edit
|
- Requires root to edit
|
||||||
|
|
||||||
@@ -128,12 +128,12 @@ source .env
|
|||||||
python scripts/send_daily_report.py
|
python scripts/send_daily_report.py
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Pros
|
### Pros
|
||||||
- Secrets separate from code
|
- Secrets separate from code
|
||||||
- Easy to rotate
|
- Easy to rotate
|
||||||
- Can be backed up separately
|
- Can be backed up separately
|
||||||
|
|
||||||
### ⚠️ Cons
|
### Cons
|
||||||
- Extra file to manage
|
- Extra file to manage
|
||||||
- Still plain text
|
- Still plain text
|
||||||
|
|
||||||
@@ -191,12 +191,12 @@ class Settings(BaseSettings):
|
|||||||
smtp_password: str = Field(default_factory=lambda: get_secret("SMTP_PASSWORD"))
|
smtp_password: str = Field(default_factory=lambda: get_secret("SMTP_PASSWORD"))
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Pros
|
### Pros
|
||||||
- Docker-native solution
|
- Docker-native solution
|
||||||
- Encrypted in Swarm mode
|
- Encrypted in Swarm mode
|
||||||
- Never in logs
|
- Never in logs
|
||||||
|
|
||||||
### ⚠️ Cons
|
### Cons
|
||||||
- Requires Docker
|
- Requires Docker
|
||||||
- More complex setup
|
- More complex setup
|
||||||
|
|
||||||
@@ -228,13 +228,13 @@ secrets = client.secrets.kv.v2.read_secret_version(path='pote')
|
|||||||
smtp_password = secrets['data']['data']['smtp_password']
|
smtp_password = secrets['data']['data']['smtp_password']
|
||||||
```
|
```
|
||||||
|
|
||||||
### ✅ Pros
|
### Pros
|
||||||
- Centralized secrets management
|
- Centralized secrets management
|
||||||
- Audit logs
|
- Audit logs
|
||||||
- Dynamic secrets
|
- Dynamic secrets
|
||||||
- Access control
|
- Access control
|
||||||
|
|
||||||
### ⚠️ Cons
|
### Cons
|
||||||
- Complex setup
|
- Complex setup
|
||||||
- Requires Vault infrastructure
|
- Requires Vault infrastructure
|
||||||
- Overkill for single user
|
- Overkill for single user
|
||||||
@@ -260,14 +260,14 @@ env:
|
|||||||
DATABASE_URL: postgresql://user:${{ secrets.DB_PASSWORD }}@postgres/db
|
DATABASE_URL: postgresql://user:${{ secrets.DB_PASSWORD }}@postgres/db
|
||||||
```
|
```
|
||||||
|
|
||||||
### ⚠️ Important
|
### Important
|
||||||
- **Only for CI/CD pipelines**
|
- **Only for CI/CD pipelines**
|
||||||
- **NOT for deployed servers**
|
- **NOT for deployed servers**
|
||||||
- Secrets are injected during workflow runs
|
- Secrets are injected during workflow runs
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Recommendation for Your Setup
|
## Recommendation for Your Setup
|
||||||
|
|
||||||
### Personal/Research Use (Current)
|
### Personal/Research Use (Current)
|
||||||
|
|
||||||
@@ -305,9 +305,9 @@ gpg -c .env # Creates .env.gpg
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔒 General Security Best Practices
|
## General Security Best Practices
|
||||||
|
|
||||||
### ✅ DO
|
### DO
|
||||||
|
|
||||||
- Use strong, unique passwords
|
- Use strong, unique passwords
|
||||||
- Restrict file permissions (`chmod 600`)
|
- Restrict file permissions (`chmod 600`)
|
||||||
@@ -316,7 +316,7 @@ gpg -c .env # Creates .env.gpg
|
|||||||
- Use encrypted backups
|
- Use encrypted backups
|
||||||
- Audit who has server access
|
- Audit who has server access
|
||||||
|
|
||||||
### ❌ DON'T
|
### DON'T
|
||||||
|
|
||||||
- Commit secrets to git (even private repos)
|
- Commit secrets to git (even private repos)
|
||||||
- Store passwords in code
|
- Store passwords in code
|
||||||
@@ -327,7 +327,7 @@ gpg -c .env # Creates .env.gpg
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧪 Test Your Security
|
## Test Your Security
|
||||||
|
|
||||||
### Check if `.env` is protected
|
### Check if `.env` is protected
|
||||||
|
|
||||||
@@ -352,7 +352,7 @@ git log --all --full-history --source --pickaxe-all -S 'smtp_password'
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 Password Rotation Procedure
|
## Password Rotation Procedure
|
||||||
|
|
||||||
### Every 90 days (or if compromised):
|
### Every 90 days (or if compromised):
|
||||||
|
|
||||||
@@ -369,32 +369,32 @@ git log --all --full-history --source --pickaxe-all -S 'smtp_password'
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Security Level Comparison
|
## Security Level Comparison
|
||||||
|
|
||||||
| Level | Method | Effort | Protection |
|
| Level | Method | Effort | Protection |
|
||||||
|-------|--------|--------|------------|
|
|-------|--------|--------|------------|
|
||||||
| 🔓 Basic | `.env` (default perms) | None | Low |
|
| Basic | `.env` (default perms) | None | Low |
|
||||||
| 🔒 Good | `.env` (chmod 600) | 1 min | Medium |
|
| Good | `.env` (chmod 600) | 1 min | Medium |
|
||||||
| 🔒 Better | Environment variables | 10 min | Good |
|
| Better | Environment variables | 10 min | Good |
|
||||||
| 🔒 Better | Separate secrets file | 10 min | Good |
|
| Better | Separate secrets file | 10 min | Good |
|
||||||
| 🔐 Best | Docker Secrets | 30 min | Very Good |
|
| Best | Docker Secrets | 30 min | Very Good |
|
||||||
| 🔐 Best | Vault | 2+ hours | Excellent |
|
| Best | Vault | 2+ hours | Excellent |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Your Current Status
|
## Your Current Status
|
||||||
|
|
||||||
✅ **Already secure enough for personal use:**
|
**Already secure enough for personal use:**
|
||||||
- `.env` in `.gitignore` ✅
|
- `.env` in `.gitignore`
|
||||||
- Not committed to git ✅
|
- Not committed to git
|
||||||
- Local server only ✅
|
- Local server only
|
||||||
|
|
||||||
⚠️ **Recommended improvement (2 minutes):**
|
**Recommended improvement (2 minutes):**
|
||||||
```bash
|
```bash
|
||||||
chmod 600 .env
|
chmod 600 .env
|
||||||
```
|
```
|
||||||
|
|
||||||
🔐 **Optional (if paranoid):**
|
**Optional (if paranoid):**
|
||||||
- Use separate secrets file in `/etc/pote/`
|
- Use separate secrets file in `/etc/pote/`
|
||||||
- Encrypt backups with GPG
|
- Encrypt backups with GPG
|
||||||
- Set up password rotation schedule
|
- Set up password rotation schedule
|
||||||
@@ -405,9 +405,9 @@ chmod 600 .env
|
|||||||
|
|
||||||
**For your levkin.ca setup:**
|
**For your levkin.ca setup:**
|
||||||
|
|
||||||
1. **Current approach (`.env` file) is fine** ✅
|
1. **Current approach (`.env` file) is fine**
|
||||||
2. **Add `chmod 600 .env`** for better security (2 minutes)
|
2. **Add `chmod 600 .env`** for better security (2 minutes)
|
||||||
3. **Don't commit `.env` to git** (already protected ✅)
|
3. **Don't commit `.env` to git** (already protected )
|
||||||
4. **Consider upgrading to environment variables** if you deploy to production
|
4. **Consider upgrading to environment variables** if you deploy to production
|
||||||
|
|
||||||
Your current setup is **appropriate for a personal research project**. Don't over-engineer it unless you have specific compliance requirements or a team.
|
Your current setup is **appropriate for a personal research project**. Don't over-engineer it unless you have specific compliance requirements or a team.
|
||||||
|
|||||||
@@ -62,7 +62,7 @@ Follow the prompts:
|
|||||||
2. Choose daily report time (recommend 6 AM)
|
2. Choose daily report time (recommend 6 AM)
|
||||||
3. Confirm
|
3. Confirm
|
||||||
|
|
||||||
That's it! 🎉
|
That's it!
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -112,7 +112,7 @@ Run the daily script manually to test:
|
|||||||
./scripts/automated_daily_run.sh
|
./scripts/automated_daily_run.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
Check if email arrived! 📧
|
Check if email arrived!
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -244,5 +244,5 @@ Add to cron for regular health checks:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**You're all set! POTE will now run automatically and send you daily/weekly reports. 🚀**
|
**You're all set! POTE will now run automatically and send you daily/weekly reports. **
|
||||||
|
|
||||||
@@ -1,17 +1,17 @@
|
|||||||
# POTE Deployment & Automation Guide
|
# POTE Deployment & Automation Guide
|
||||||
|
|
||||||
## 🎯 Quick Answer to Your Questions
|
## Quick Answer to Your Questions
|
||||||
|
|
||||||
### After Deployment, What Happens?
|
### After Deployment, What Happens?
|
||||||
|
|
||||||
**By default: NOTHING automatic happens.** You need to set up automation.
|
**By default: NOTHING automatic happens.** You need to set up automation.
|
||||||
|
|
||||||
The deployed system is:
|
The deployed system is:
|
||||||
- ✅ Running (database, code installed)
|
- Running (database, code installed)
|
||||||
- ✅ Accessible via SSH at your Proxmox IP
|
- Accessible via SSH at your Proxmox IP
|
||||||
- ❌ NOT fetching data automatically
|
- NOT fetching data automatically
|
||||||
- ❌ NOT sending reports automatically
|
- NOT sending reports automatically
|
||||||
- ❌ NOT monitoring markets automatically
|
- NOT monitoring markets automatically
|
||||||
|
|
||||||
**You must either:**
|
**You must either:**
|
||||||
1. **Run scripts manually** when you want updates, OR
|
1. **Run scripts manually** when you want updates, OR
|
||||||
@@ -19,7 +19,7 @@ The deployed system is:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Option 1: Automated Email Reports (Recommended)
|
## Option 1: Automated Email Reports (Recommended)
|
||||||
|
|
||||||
### What You Get
|
### What You Get
|
||||||
|
|
||||||
@@ -53,13 +53,13 @@ Run the interactive setup:
|
|||||||
Follow prompts:
|
Follow prompts:
|
||||||
1. Enter your email address
|
1. Enter your email address
|
||||||
2. Choose report time (default: 6 AM)
|
2. Choose report time (default: 6 AM)
|
||||||
3. Done! ✅
|
3. Done!
|
||||||
|
|
||||||
**See full guide:** [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md)
|
**See full guide:** [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📍 Option 2: Access Reports via IP (No Email)
|
## Option 2: Access Reports via IP (No Email)
|
||||||
|
|
||||||
If you don't want email, you can:
|
If you don't want email, you can:
|
||||||
|
|
||||||
@@ -107,7 +107,7 @@ python scripts/health_check.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🌐 Option 3: Build a Web Interface (Future)
|
## Option 3: Build a Web Interface (Future)
|
||||||
|
|
||||||
Currently, POTE is **command-line only**. No web UI yet.
|
Currently, POTE is **command-line only**. No web UI yet.
|
||||||
|
|
||||||
@@ -122,16 +122,16 @@ For now, use SSH or email reports.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 Do You Need CI/CD Pipelines?
|
## Do You Need CI/CD Pipelines?
|
||||||
|
|
||||||
### What the Pipeline Does
|
### What the Pipeline Does
|
||||||
|
|
||||||
The included CI/CD pipeline (`.github/workflows/ci.yml`) runs on **every git push**:
|
The included CI/CD pipeline (`.github/workflows/ci.yml`) runs on **every git push**:
|
||||||
|
|
||||||
1. ✅ Lint & test (93 tests)
|
1. Lint & test (93 tests)
|
||||||
2. ✅ Security scanning
|
2. Security scanning
|
||||||
3. ✅ Dependency scanning
|
3. Dependency scanning
|
||||||
4. ✅ Docker build test
|
4. Docker build test
|
||||||
|
|
||||||
### Should You Use It?
|
### Should You Use It?
|
||||||
|
|
||||||
@@ -174,18 +174,18 @@ docker build -t pote:test .
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Comparison of Options
|
## Comparison of Options
|
||||||
|
|
||||||
| Method | Pros | Cons | Best For |
|
| Method | Pros | Cons | Best For |
|
||||||
|--------|------|------|----------|
|
|--------|------|------|----------|
|
||||||
| **Automated Email** | ✅ Convenient<br>✅ No SSH needed<br>✅ Daily/weekly updates | ❌ Requires SMTP setup | Most users |
|
| **Automated Email** | Convenient<br> No SSH needed<br> Daily/weekly updates | Requires SMTP setup | Most users |
|
||||||
| **SSH + Manual Scripts** | ✅ Full control<br>✅ No email needed | ❌ Manual work<br>❌ Must remember to run | Power users |
|
| **SSH + Manual Scripts** | Full control<br> No email needed | Manual work<br> Must remember to run | Power users |
|
||||||
| **Saved Reports (SSH access)** | ✅ Automated<br>✅ No email | ❌ Must SSH to view | Users without email |
|
| **Saved Reports (SSH access)** | Automated<br> No email | Must SSH to view | Users without email |
|
||||||
| **Web Interface** | ✅ User-friendly | ❌ Not implemented yet | Future |
|
| **Web Interface** | User-friendly | Not implemented yet | Future |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🛠️ Your Ansible Pipeline
|
## Your Ansible Pipeline
|
||||||
|
|
||||||
Your existing Ansible CI/CD pipeline is **NOT directly usable** for POTE because:
|
Your existing Ansible CI/CD pipeline is **NOT directly usable** for POTE because:
|
||||||
|
|
||||||
@@ -197,11 +197,11 @@ Your existing Ansible CI/CD pipeline is **NOT directly usable** for POTE because
|
|||||||
|
|
||||||
**Concepts from your pipeline that ARE used in POTE's CI/CD:**
|
**Concepts from your pipeline that ARE used in POTE's CI/CD:**
|
||||||
|
|
||||||
- ✅ Security scanning (Trivy, Bandit instead of Gitleaks)
|
- Security scanning (Trivy, Bandit instead of Gitleaks)
|
||||||
- ✅ Dependency scanning (Trivy instead of npm audit)
|
- Dependency scanning (Trivy instead of npm audit)
|
||||||
- ✅ SAST scanning (Bandit instead of Semgrep)
|
- SAST scanning (Bandit instead of Semgrep)
|
||||||
- ✅ Container scanning (Docker build test)
|
- Container scanning (Docker build test)
|
||||||
- ✅ Workflow summary generation
|
- Workflow summary generation
|
||||||
|
|
||||||
**The POTE pipeline (`.github/workflows/ci.yml`) already includes all of these!**
|
**The POTE pipeline (`.github/workflows/ci.yml`) already includes all of these!**
|
||||||
|
|
||||||
@@ -217,7 +217,7 @@ Your Gitea server can run the POTE pipeline using Gitea Actions:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📧 Email Setup Examples
|
## Email Setup Examples
|
||||||
|
|
||||||
### Gmail (Most Common)
|
### Gmail (Most Common)
|
||||||
|
|
||||||
@@ -258,7 +258,7 @@ REPORT_RECIPIENTS=admin@yourdomain.com
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ✅ Recommended Setup for Most Users
|
## Recommended Setup for Most Users
|
||||||
|
|
||||||
1. **Deploy to Proxmox** (5 min)
|
1. **Deploy to Proxmox** (5 min)
|
||||||
```bash
|
```bash
|
||||||
@@ -283,7 +283,7 @@ REPORT_RECIPIENTS=admin@yourdomain.com
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔍 Monitoring & Health Checks
|
## Monitoring & Health Checks
|
||||||
|
|
||||||
### Add System Health Monitoring
|
### Add System Health Monitoring
|
||||||
|
|
||||||
@@ -310,17 +310,17 @@ python scripts/health_check.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📚 Full Documentation
|
## Full Documentation
|
||||||
|
|
||||||
- **Automation Setup**: [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md) ⭐
|
- **Automation Setup**: [`AUTOMATION_QUICKSTART.md`](AUTOMATION_QUICKSTART.md)
|
||||||
- **Deployment**: [`PROXMOX_QUICKSTART.md`](PROXMOX_QUICKSTART.md) ⭐
|
- **Deployment**: [`PROXMOX_QUICKSTART.md`](PROXMOX_QUICKSTART.md)
|
||||||
- **Usage**: [`QUICKSTART.md`](QUICKSTART.md) ⭐
|
- **Usage**: [`QUICKSTART.md`](QUICKSTART.md)
|
||||||
- **Detailed Automation Guide**: [`docs/12_automation_and_reporting.md`](docs/12_automation_and_reporting.md)
|
- **Detailed Automation Guide**: [`docs/12_automation_and_reporting.md`](docs/12_automation_and_reporting.md)
|
||||||
- **Monitoring System**: [`MONITORING_SYSTEM_COMPLETE.md`](MONITORING_SYSTEM_COMPLETE.md)
|
- **Monitoring System**: [`MONITORING_SYSTEM_COMPLETE.md`](MONITORING_SYSTEM_COMPLETE.md)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎉 Summary
|
## Summary
|
||||||
|
|
||||||
**After deployment:**
|
**After deployment:**
|
||||||
- Reports are NOT sent automatically by default
|
- Reports are NOT sent automatically by default
|
||||||
@@ -334,5 +334,5 @@ python scripts/health_check.py
|
|||||||
1. Deploy to Proxmox
|
1. Deploy to Proxmox
|
||||||
2. Run `./scripts/setup_cron.sh`
|
2. Run `./scripts/setup_cron.sh`
|
||||||
3. Receive daily/weekly email reports
|
3. Receive daily/weekly email reports
|
||||||
4. Done! 🚀
|
4. Done!
|
||||||
|
|
||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Homelab POTE sends via Mailcow **`mail.levkine.ca`** using the shared **`alerts@levkine.ca`** mailbox (same as Kuma/Beszel). See ansible `docs/guides/smtp-inventory.md`.
|
Homelab POTE sends via Mailcow **`mail.levkine.ca`** using the shared **`alerts@levkine.ca`** mailbox (same as Kuma/Beszel). See ansible `docs/guides/smtp-inventory.md`.
|
||||||
|
|
||||||
## ✅ Configuration Done
|
## Configuration Done
|
||||||
|
|
||||||
The `.env` file has been created with these settings:
|
The `.env` file has been created with these settings:
|
||||||
|
|
||||||
@@ -15,7 +15,7 @@ FROM_EMAIL=alerts@levkine.ca
|
|||||||
REPORT_RECIPIENTS=idobkin@gmail.com
|
REPORT_RECIPIENTS=idobkin@gmail.com
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🔑 Next Steps
|
## Next Steps
|
||||||
|
|
||||||
### 1. Add Your Password
|
### 1. Add Your Password
|
||||||
|
|
||||||
@@ -38,7 +38,7 @@ python scripts/send_daily_report.py --to test@levkin.ca --test-smtp
|
|||||||
If successful, you'll see:
|
If successful, you'll see:
|
||||||
```
|
```
|
||||||
SMTP connection test successful!
|
SMTP connection test successful!
|
||||||
✓ Daily report sent successfully!
|
Daily report sent successfully!
|
||||||
```
|
```
|
||||||
|
|
||||||
And you should receive a test email at `test@levkin.ca`!
|
And you should receive a test email at `test@levkin.ca`!
|
||||||
@@ -57,7 +57,7 @@ This will:
|
|||||||
- Schedule daily reports (default: 6 AM)
|
- Schedule daily reports (default: 6 AM)
|
||||||
- Schedule weekly reports (Sundays at 8 AM)
|
- Schedule weekly reports (Sundays at 8 AM)
|
||||||
|
|
||||||
## 📧 Email Server Details (For Reference)
|
## Email Server Details (For Reference)
|
||||||
|
|
||||||
Based on your Thunderbird setup:
|
Based on your Thunderbird setup:
|
||||||
|
|
||||||
@@ -76,10 +76,10 @@ Based on your Thunderbird setup:
|
|||||||
|
|
||||||
POTE only uses **SMTP (outgoing)** to send reports.
|
POTE only uses **SMTP (outgoing)** to send reports.
|
||||||
|
|
||||||
## 🔒 Security Notes
|
## Security Notes
|
||||||
|
|
||||||
1. **Never commit `.env` to git!**
|
1. **Never commit `.env` to git!**
|
||||||
- Already in `.gitignore` ✅
|
- Already in `.gitignore`
|
||||||
- Contains sensitive password
|
- Contains sensitive password
|
||||||
|
|
||||||
2. **Password Security:**
|
2. **Password Security:**
|
||||||
@@ -92,7 +92,7 @@ POTE only uses **SMTP (outgoing)** to send reports.
|
|||||||
chmod 600 .env # Only owner can read/write
|
chmod 600 .env # Only owner can read/write
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🎯 Change Recipients
|
## Change Recipients
|
||||||
|
|
||||||
To send reports to different email addresses (not just test@levkin.ca):
|
To send reports to different email addresses (not just test@levkin.ca):
|
||||||
|
|
||||||
@@ -109,14 +109,14 @@ python scripts/send_daily_report.py --to someone-else@example.com
|
|||||||
# The FROM address will still be test@levkin.ca
|
# The FROM address will still be test@levkin.ca
|
||||||
```
|
```
|
||||||
|
|
||||||
## ✅ Testing Checklist
|
## Testing Checklist
|
||||||
|
|
||||||
- [ ] Updated `.env` with your actual password
|
- [ ] Updated `.env` with your actual password
|
||||||
- [ ] Run `python scripts/send_daily_report.py --to test@levkin.ca --test-smtp`
|
- [ ] Run `python scripts/send_daily_report.py --to test@levkin.ca --test-smtp`
|
||||||
- [ ] Checked inbox at test@levkin.ca (check spam folder!)
|
- [ ] Checked inbox at test@levkin.ca (check spam folder!)
|
||||||
- [ ] If successful, run `./scripts/setup_cron.sh` to automate
|
- [ ] If successful, run `./scripts/setup_cron.sh` to automate
|
||||||
|
|
||||||
## 🐛 Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Error: "SMTP connection failed"
|
### Error: "SMTP connection failed"
|
||||||
|
|
||||||
@@ -139,5 +139,5 @@ Check:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**You're all set! POTE will send reports from test@levkin.ca 📧**
|
**You're all set! POTE will send reports from test@levkin.ca **
|
||||||
|
|
||||||
@@ -2,14 +2,14 @@
|
|||||||
|
|
||||||
## TL;DR: You can test everything for $0
|
## TL;DR: You can test everything for $0
|
||||||
|
|
||||||
### Already Working (PR1 ✅)
|
### Already Working (PR1 )
|
||||||
- **Price data**: `yfinance` (free, unlimited)
|
- **Price data**: `yfinance` (free, unlimited)
|
||||||
- **Unit tests**: Mocked data in `tests/` (15 passing tests)
|
- **Unit tests**: Mocked data in `tests/` (15 passing tests)
|
||||||
- **Coverage**: 87% without any paid APIs
|
- **Coverage**: 87% without any paid APIs
|
||||||
|
|
||||||
### For PR2 (Congressional Trades) - FREE Options
|
### For PR2 (Congressional Trades) - FREE Options
|
||||||
|
|
||||||
#### Best Option: House Stock Watcher 🌟
|
#### Best Option: House Stock Watcher
|
||||||
```bash
|
```bash
|
||||||
# No API key needed, just scrape their public JSON
|
# No API key needed, just scrape their public JSON
|
||||||
curl https://housestockwatcher.com/api/all_transactions
|
curl https://housestockwatcher.com/api/all_transactions
|
||||||
@@ -44,10 +44,10 @@ python scripts/fetch_house_watcher_sample.py # We'll build this in PR2
|
|||||||
|
|
||||||
### What You DON'T Need to Pay For
|
### What You DON'T Need to Pay For
|
||||||
|
|
||||||
❌ QuiverQuant Pro ($30/mo) - free tier is enough for dev/testing
|
QuiverQuant Pro ($30/mo) - free tier is enough for dev/testing
|
||||||
❌ Financial Modeling Prep paid tier - free tier works
|
Financial Modeling Prep paid tier - free tier works
|
||||||
❌ Any paid database hosting - SQLite works great locally
|
Any paid database hosting - SQLite works great locally
|
||||||
❌ Any cloud services - runs 100% locally
|
Any cloud services - runs 100% locally
|
||||||
|
|
||||||
### When You MIGHT Want Paid (Way Later)
|
### When You MIGHT Want Paid (Way Later)
|
||||||
|
|
||||||
@@ -56,7 +56,7 @@ python scripts/fetch_house_watcher_sample.py # We'll build this in PR2
|
|||||||
- Multiple concurrent users on a dashboard
|
- Multiple concurrent users on a dashboard
|
||||||
- Commercial use (check each API's terms)
|
- Commercial use (check each API's terms)
|
||||||
|
|
||||||
**For personal research? Stay free forever. 🎉**
|
**For personal research? Stay free forever. **
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1,20 +1,20 @@
|
|||||||
# 🔐 Gitea Secrets Guide for POTE
|
# Gitea Secrets Guide for POTE
|
||||||
|
|
||||||
## ✅ YES! You Can Store Passwords in Gitea
|
## YES! You Can Store Passwords in Gitea
|
||||||
|
|
||||||
Gitea has a **Secrets** feature (like GitHub Actions secrets) that lets you store passwords securely and use them in:
|
Gitea has a **Secrets** feature (like GitHub Actions secrets) that lets you store passwords securely and use them in:
|
||||||
1. **CI/CD pipelines** (Gitea Actions workflows) ✅
|
1. **CI/CD pipelines** (Gitea Actions workflows)
|
||||||
2. **Deployment workflows** ✅
|
2. **Deployment workflows**
|
||||||
|
|
||||||
**BUT NOT:**
|
**BUT NOT:**
|
||||||
- ❌ Directly in your running application on Proxmox
|
- Directly in your running application on Proxmox
|
||||||
- ❌ Accessed by scripts outside of workflows
|
- Accessed by scripts outside of workflows
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 What Gitea Secrets Are Good For
|
## What Gitea Secrets Are Good For
|
||||||
|
|
||||||
### ✅ Perfect Use Cases
|
### Perfect Use Cases
|
||||||
|
|
||||||
1. **CI/CD Testing** - Run tests with real credentials
|
1. **CI/CD Testing** - Run tests with real credentials
|
||||||
2. **Automated Deployment** - Deploy to Proxmox with SSH keys
|
2. **Automated Deployment** - Deploy to Proxmox with SSH keys
|
||||||
@@ -22,7 +22,7 @@ Gitea has a **Secrets** feature (like GitHub Actions secrets) that lets you stor
|
|||||||
4. **Docker Registry** - Push images with credentials
|
4. **Docker Registry** - Push images with credentials
|
||||||
5. **API Keys** - Access external services during builds
|
5. **API Keys** - Access external services during builds
|
||||||
|
|
||||||
### ❌ NOT Good For
|
### NOT Good For
|
||||||
|
|
||||||
1. **Runtime secrets** - Your deployed app on Proxmox can't access them
|
1. **Runtime secrets** - Your deployed app on Proxmox can't access them
|
||||||
2. **Local development** - Can't use secrets on your laptop
|
2. **Local development** - Can't use secrets on your laptop
|
||||||
@@ -30,7 +30,7 @@ Gitea has a **Secrets** feature (like GitHub Actions secrets) that lets you stor
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 How to Set Up Gitea Secrets
|
## How to Set Up Gitea Secrets
|
||||||
|
|
||||||
### Step 1: Add Secrets to Gitea
|
### Step 1: Add Secrets to Gitea
|
||||||
|
|
||||||
@@ -57,7 +57,7 @@ Secrets are accessed with `${{ secrets.SECRET_NAME }}` syntax.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📝 Example: CI Pipeline with Secrets
|
## Example: CI Pipeline with Secrets
|
||||||
|
|
||||||
**File:** `.github/workflows/ci.yml`
|
**File:** `.github/workflows/ci.yml`
|
||||||
|
|
||||||
@@ -92,11 +92,11 @@ jobs:
|
|||||||
--smtp-password "${{ secrets.SMTP_PASSWORD }}"
|
--smtp-password "${{ secrets.SMTP_PASSWORD }}"
|
||||||
```
|
```
|
||||||
|
|
||||||
**✅ I've already updated your CI pipeline to use secrets!**
|
** I've already updated your CI pipeline to use secrets!**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Example: Automated Deployment Workflow
|
## Example: Automated Deployment Workflow
|
||||||
|
|
||||||
Create `.github/workflows/deploy.yml`:
|
Create `.github/workflows/deploy.yml`:
|
||||||
|
|
||||||
@@ -152,7 +152,7 @@ jobs:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 How Secrets Flow to Your Server
|
## How Secrets Flow to Your Server
|
||||||
|
|
||||||
### Option 1: Deploy Workflow Updates `.env` (Recommended)
|
### Option 1: Deploy Workflow Updates `.env` (Recommended)
|
||||||
|
|
||||||
@@ -201,11 +201,11 @@ jobs:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Recommended Setup for Your POTE Project
|
## Recommended Setup for Your POTE Project
|
||||||
|
|
||||||
### For CI/CD (Testing):
|
### For CI/CD (Testing):
|
||||||
|
|
||||||
**Use Gitea Secrets** ✅
|
**Use Gitea Secrets**
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
# .github/workflows/ci.yml (already updated!)
|
# .github/workflows/ci.yml (already updated!)
|
||||||
@@ -216,7 +216,7 @@ env:
|
|||||||
|
|
||||||
### For Deployed Server (Proxmox):
|
### For Deployed Server (Proxmox):
|
||||||
|
|
||||||
**Keep using `.env` file** ✅
|
**Keep using `.env` file**
|
||||||
|
|
||||||
Why?
|
Why?
|
||||||
- Simpler for manual SSH access
|
- Simpler for manual SSH access
|
||||||
@@ -227,7 +227,7 @@ Why?
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Complete Workflow: Gitea → Proxmox
|
## Complete Workflow: Gitea → Proxmox
|
||||||
|
|
||||||
### 1. Store Secrets in Gitea
|
### 1. Store Secrets in Gitea
|
||||||
|
|
||||||
@@ -264,27 +264,27 @@ git push origin main
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚠️ Important Limitations
|
## Important Limitations
|
||||||
|
|
||||||
### Gitea Secrets CAN'T:
|
### Gitea Secrets CAN'T:
|
||||||
|
|
||||||
❌ Be accessed outside of workflows
|
Be accessed outside of workflows
|
||||||
❌ Be used in local `python script.py` runs
|
Be used in local `python script.py` runs
|
||||||
❌ Be read by cron jobs on Proxmox (directly)
|
Be read by cron jobs on Proxmox (directly)
|
||||||
❌ Replace `.env` for runtime application config
|
Replace `.env` for runtime application config
|
||||||
|
|
||||||
### Gitea Secrets CAN:
|
### Gitea Secrets CAN:
|
||||||
|
|
||||||
✅ Secure your CI/CD pipeline
|
Secure your CI/CD pipeline
|
||||||
✅ Deploy safely without exposing passwords in git
|
Deploy safely without exposing passwords in git
|
||||||
✅ Update `.env` on server during deployment
|
Update `.env` on server during deployment
|
||||||
✅ Run automated tests with real credentials
|
Run automated tests with real credentials
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔒 Security Best Practices
|
## Security Best Practices
|
||||||
|
|
||||||
### ✅ DO:
|
### DO:
|
||||||
|
|
||||||
1. **Store ALL sensitive data as Gitea secrets**
|
1. **Store ALL sensitive data as Gitea secrets**
|
||||||
- SMTP passwords
|
- SMTP passwords
|
||||||
@@ -300,10 +300,10 @@ git push origin main
|
|||||||
|
|
||||||
3. **Never echo secrets**
|
3. **Never echo secrets**
|
||||||
```yaml
|
```yaml
|
||||||
# ❌ BAD - exposes in logs
|
# BAD - exposes in logs
|
||||||
- run: echo "${{ secrets.PASSWORD }}"
|
- run: echo "${{ secrets.PASSWORD }}"
|
||||||
|
|
||||||
# ✅ GOOD - masked automatically
|
# GOOD - masked automatically
|
||||||
- run: use_password "${{ secrets.PASSWORD }}"
|
- run: use_password "${{ secrets.PASSWORD }}"
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -311,7 +311,7 @@ git push origin main
|
|||||||
- Update in Gitea UI
|
- Update in Gitea UI
|
||||||
- Re-run deployment workflow
|
- Re-run deployment workflow
|
||||||
|
|
||||||
### ❌ DON'T:
|
### DON'T:
|
||||||
|
|
||||||
1. **Commit secrets to git** (even private repos)
|
1. **Commit secrets to git** (even private repos)
|
||||||
2. **Share secrets via Slack/email**
|
2. **Share secrets via Slack/email**
|
||||||
@@ -320,18 +320,18 @@ git push origin main
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Comparison: Where to Store Secrets
|
## Comparison: Where to Store Secrets
|
||||||
|
|
||||||
| Storage | CI/CD | Deployed App | Easy Updates | Security |
|
| Storage | CI/CD | Deployed App | Easy Updates | Security |
|
||||||
|---------|-------|--------------|--------------|----------|
|
|---------|-------|--------------|--------------|----------|
|
||||||
| **Gitea Secrets** | ✅ Perfect | ❌ No | ✅ Via workflow | ⭐⭐⭐⭐⭐ |
|
| **Gitea Secrets** | Perfect | No | Via workflow | |
|
||||||
| **`.env` file** | ❌ No | ✅ Perfect | ✅ `nano .env` | ⭐⭐⭐ |
|
| **`.env` file** | No | Perfect | `nano .env` | |
|
||||||
| **Environment Vars** | ✅ Yes | ✅ Yes | ❌ Harder | ⭐⭐⭐⭐ |
|
| **Environment Vars** | Yes | Yes | Harder | |
|
||||||
| **Both (Recommended)** | ✅ Yes | ✅ Yes | ✅ Automated | ⭐⭐⭐⭐⭐ |
|
| **Both (Recommended)** | Yes | Yes | Automated | |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 My Recommendation for You
|
## My Recommendation for You
|
||||||
|
|
||||||
### Use BOTH:
|
### Use BOTH:
|
||||||
|
|
||||||
@@ -345,21 +345,21 @@ git push origin main
|
|||||||
2. Commit code changes
|
2. Commit code changes
|
||||||
3. Push to Gitea
|
3. Push to Gitea
|
||||||
4. Workflow runs:
|
4. Workflow runs:
|
||||||
- Tests with Gitea secrets ✅
|
- Tests with Gitea secrets
|
||||||
- Deploys to Proxmox ✅
|
- Deploys to Proxmox
|
||||||
- Updates .env with secrets ✅
|
- Updates .env with secrets
|
||||||
5. Proxmox app reads from .env ✅
|
5. Proxmox app reads from .env
|
||||||
```
|
```
|
||||||
|
|
||||||
**This gives you:**
|
**This gives you:**
|
||||||
- ✅ Secure CI/CD
|
- Secure CI/CD
|
||||||
- ✅ Easy manual SSH access
|
- Easy manual SSH access
|
||||||
- ✅ Automated deployments
|
- Automated deployments
|
||||||
- ✅ No passwords in git
|
- No passwords in git
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Next Steps
|
## Next Steps
|
||||||
|
|
||||||
### 1. Add Secrets to Gitea (5 minutes)
|
### 1. Add Secrets to Gitea (5 minutes)
|
||||||
|
|
||||||
@@ -387,7 +387,7 @@ I can create `.github/workflows/deploy.yml` if you want automated deployments!
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 💡 Quick Commands
|
## Quick Commands
|
||||||
|
|
||||||
### Add SSH Key to Gitea (for deployment):
|
### Add SSH Key to Gitea (for deployment):
|
||||||
|
|
||||||
@@ -409,12 +409,12 @@ git commit --allow-empty -m "Test secrets"
|
|||||||
git push
|
git push
|
||||||
|
|
||||||
# Check Gitea Actions tab
|
# Check Gitea Actions tab
|
||||||
# Look for green checkmarks ✅
|
# Look for green checkmarks
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📚 See Also
|
## See Also
|
||||||
|
|
||||||
- **[docs/13_secrets_management.md](docs/13_secrets_management.md)** - All secrets options
|
- **[docs/13_secrets_management.md](docs/13_secrets_management.md)** - All secrets options
|
||||||
- **[.github/workflows/ci.yml](.github/workflows/ci.yml)** - Updated with secrets support
|
- **[.github/workflows/ci.yml](.github/workflows/ci.yml)** - Updated with secrets support
|
||||||
@@ -422,16 +422,16 @@ git push
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ✅ Summary
|
## Summary
|
||||||
|
|
||||||
**YES, use Gitea secrets!** They're perfect for:
|
**YES, use Gitea secrets!** They're perfect for:
|
||||||
- ✅ CI/CD pipelines
|
- CI/CD pipelines
|
||||||
- ✅ Automated deployments
|
- Automated deployments
|
||||||
- ✅ Keeping passwords out of git
|
- Keeping passwords out of git
|
||||||
|
|
||||||
**But ALSO keep `.env` on Proxmox** for:
|
**But ALSO keep `.env` on Proxmox** for:
|
||||||
- ✅ Runtime application config
|
- Runtime application config
|
||||||
- ✅ Manual SSH access
|
- Manual SSH access
|
||||||
- ✅ Cron jobs
|
- Cron jobs
|
||||||
|
|
||||||
**Best of both worlds:** Gitea secrets deploy and update the `.env` file automatically! 🚀
|
**Best of both worlds:** Gitea secrets deploy and update the `.env` file automatically!
|
||||||
@@ -150,9 +150,9 @@ make beszel-install-agents BESZEL_ONLY=pote-236 # if agent not yet installed
|
|||||||
|
|
||||||
| Doc | Purpose |
|
| Doc | Purpose |
|
||||||
|-----|---------|
|
|-----|---------|
|
||||||
| [EMAIL_SETUP.md](../EMAIL_SETUP.md) | SMTP / Mailcow / levkine.ca |
|
| [EMAIL_SETUP.md](EMAIL_SETUP.md) | SMTP / Mailcow / levkine.ca |
|
||||||
| [AUTOMATION_QUICKSTART.md](../AUTOMATION_QUICKSTART.md) | Cron + reports |
|
| [AUTOMATION_QUICKSTART.md](AUTOMATION_QUICKSTART.md) | Cron + reports |
|
||||||
| [PROXMOX_QUICKSTART.md](../PROXMOX_QUICKSTART.md) | Original LXC provisioning |
|
| [PROXMOX_QUICKSTART.md](PROXMOX_QUICKSTART.md) | Original LXC provisioning |
|
||||||
| Ansible `docs/guides/projects-handoff-2026-05-26.md` | Multi-project homelab context |
|
| Ansible `docs/guides/projects-handoff-2026-05-26.md` | Multi-project homelab context |
|
||||||
| Ansible `docs/guides/smtp-inventory.md` | Mailboxes |
|
| Ansible `docs/guides/smtp-inventory.md` | Mailboxes |
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Local Testing Guide for POTE
|
# Local Testing Guide for POTE
|
||||||
|
|
||||||
## ✅ Testing Locally Before Deployment
|
## Testing Locally Before Deployment
|
||||||
|
|
||||||
### Quick Test - Run Full Suite
|
### Quick Test - Run Full Suite
|
||||||
|
|
||||||
@@ -10,18 +10,18 @@ source venv/bin/activate
|
|||||||
pytest -v
|
pytest -v
|
||||||
```
|
```
|
||||||
|
|
||||||
**Expected Result:** All 55 tests should pass ✅
|
**Expected Result:** All 55 tests should pass
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Current Data Status
|
## Current Data Status
|
||||||
|
|
||||||
### Live Data Status: ❌ **NOT LIVE YET**
|
### Live Data Status: **NOT LIVE YET**
|
||||||
|
|
||||||
**Why?**
|
**Why?**
|
||||||
- 🔴 **House Stock Watcher API is DOWN** (domain issues, unreachable)
|
- **House Stock Watcher API is DOWN** (domain issues, unreachable)
|
||||||
- 🟢 **yfinance works** (for price data)
|
- **yfinance works** (for price data)
|
||||||
- 🟡 **Sample data available** (5 trades from fixtures)
|
- **Sample data available** (5 trades from fixtures)
|
||||||
|
|
||||||
### What Data Do You Have?
|
### What Data Do You Have?
|
||||||
|
|
||||||
@@ -40,7 +40,7 @@ This will show:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧪 Testing Analytics Locally
|
## Testing Analytics Locally
|
||||||
|
|
||||||
### 1. Unit Tests (Fast, No External Dependencies)
|
### 1. Unit Tests (Fast, No External Dependencies)
|
||||||
|
|
||||||
@@ -53,10 +53,10 @@ pytest tests/test_analytics_integration.py -v
|
|||||||
```
|
```
|
||||||
|
|
||||||
These tests:
|
These tests:
|
||||||
- ✅ Create synthetic price data
|
- Create synthetic price data
|
||||||
- ✅ Simulate trades with known returns
|
- Simulate trades with known returns
|
||||||
- ✅ Verify calculations are correct
|
- Verify calculations are correct
|
||||||
- ✅ Test edge cases (missing data, sell trades, etc.)
|
- Test edge cases (missing data, sell trades, etc.)
|
||||||
|
|
||||||
### 2. Manual Test with Local Database
|
### 2. Manual Test with Local Database
|
||||||
|
|
||||||
@@ -92,27 +92,27 @@ calc = ReturnCalculator(session)
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📦 What Gets Tested?
|
## What Gets Tested?
|
||||||
|
|
||||||
### Core Functionality (All Working ✅)
|
### Core Functionality (All Working )
|
||||||
1. **Database Models** - Officials, Securities, Trades, Prices
|
1. **Database Models** - Officials, Securities, Trades, Prices
|
||||||
2. **Data Ingestion** - Trade loading, security enrichment
|
2. **Data Ingestion** - Trade loading, security enrichment
|
||||||
3. **Analytics Engine** - Returns, benchmarks, metrics
|
3. **Analytics Engine** - Returns, benchmarks, metrics
|
||||||
4. **Edge Cases** - Missing data, sell trades, disclosure lags
|
4. **Edge Cases** - Missing data, sell trades, disclosure lags
|
||||||
|
|
||||||
### Integration Tests Cover:
|
### Integration Tests Cover:
|
||||||
- ✅ Return calculations over multiple time windows (30/60/90/180 days)
|
- Return calculations over multiple time windows (30/60/90/180 days)
|
||||||
- ✅ Benchmark comparisons (stock vs SPY/QQQ)
|
- Benchmark comparisons (stock vs SPY/QQQ)
|
||||||
- ✅ Abnormal return (alpha) calculations
|
- Abnormal return (alpha) calculations
|
||||||
- ✅ Official performance summaries
|
- Official performance summaries
|
||||||
- ✅ Sector analysis
|
- Sector analysis
|
||||||
- ✅ Disclosure timing analysis
|
- Disclosure timing analysis
|
||||||
- ✅ Top performer rankings
|
- Top performer rankings
|
||||||
- ✅ System-wide statistics
|
- System-wide statistics
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 Getting Live Data
|
## Getting Live Data
|
||||||
|
|
||||||
### Option 1: Wait for House Stock Watcher API
|
### Option 1: Wait for House Stock Watcher API
|
||||||
The API is currently down. Once it's back up:
|
The API is currently down. Once it's back up:
|
||||||
@@ -164,7 +164,7 @@ export QUIVER_API_KEY="your_key_here"
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 After Adding Data, Fetch Prices
|
## After Adding Data, Fetch Prices
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# This will fetch prices for all securities in your database
|
# This will fetch prices for all securities in your database
|
||||||
@@ -176,12 +176,12 @@ python scripts/enrich_securities.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Complete Local Test Workflow
|
## Complete Local Test Workflow
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 1. Run all tests
|
# 1. Run all tests
|
||||||
pytest -v
|
pytest -v
|
||||||
# ✅ All 55 tests should pass
|
# All 55 tests should pass
|
||||||
|
|
||||||
# 2. Check local database
|
# 2. Check local database
|
||||||
python -c "
|
python -c "
|
||||||
@@ -210,7 +210,7 @@ python scripts/calculate_all_returns.py --window 90
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Deploy to Proxmox
|
## Deploy to Proxmox
|
||||||
|
|
||||||
Once local tests pass:
|
Once local tests pass:
|
||||||
|
|
||||||
@@ -240,7 +240,7 @@ alembic upgrade head
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🐛 Common Issues
|
## Common Issues
|
||||||
|
|
||||||
### "No price data found"
|
### "No price data found"
|
||||||
**Fix:** Run `python scripts/fetch_sample_prices.py`
|
**Fix:** Run `python scripts/fetch_sample_prices.py`
|
||||||
@@ -260,7 +260,7 @@ sudo -u postgres psql -c "\l"
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Test Coverage
|
## Test Coverage
|
||||||
|
|
||||||
Run tests with coverage report:
|
Run tests with coverage report:
|
||||||
|
|
||||||
@@ -277,19 +277,19 @@ firefox htmlcov/index.html # View coverage report
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ✨ Summary
|
## Summary
|
||||||
|
|
||||||
**Before Deploying:**
|
**Before Deploying:**
|
||||||
1. ✅ Run `pytest -v` - all tests pass
|
1. Run `pytest -v` - all tests pass
|
||||||
2. ✅ Run `make lint` - no errors
|
2. Run `make lint` - no errors
|
||||||
3. ✅ Test locally with sample data
|
3. Test locally with sample data
|
||||||
4. ✅ Verify analytics work with synthetic prices
|
4. Verify analytics work with synthetic prices
|
||||||
|
|
||||||
**Getting Live Data:**
|
**Getting Live Data:**
|
||||||
- 🔴 House Stock Watcher API is down (external issue)
|
- House Stock Watcher API is down (external issue)
|
||||||
- 🟢 Manual CSV import works NOW
|
- Manual CSV import works NOW
|
||||||
- 🟢 yfinance for prices works NOW
|
- yfinance for prices works NOW
|
||||||
- 🟡 QuiverQuant available (requires free API key)
|
- QuiverQuant available (requires free API key)
|
||||||
|
|
||||||
**You can deploy and use the system NOW with:**
|
**You can deploy and use the system NOW with:**
|
||||||
- Manual data entry
|
- Manual data entry
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# Offline Demo - Works Without Internet!
|
# Offline Demo - Works Without Internet!
|
||||||
|
|
||||||
## ✅ Full System Working Without Network Access
|
## Full System Working Without Network Access
|
||||||
|
|
||||||
Even though your environment doesn't have external internet, **everything works perfectly** using fixture files.
|
Even though your environment doesn't have external internet, **everything works perfectly** using fixture files.
|
||||||
|
|
||||||
@@ -10,9 +10,9 @@ Even though your environment doesn't have external internet, **everything works
|
|||||||
python scripts/ingest_from_fixtures.py
|
python scripts/ingest_from_fixtures.py
|
||||||
|
|
||||||
# Output:
|
# Output:
|
||||||
# ✓ Officials created/updated: 4
|
# Officials created/updated: 4
|
||||||
# ✓ Securities created/updated: 2
|
# Securities created/updated: 2
|
||||||
# ✓ Trades ingested: 5
|
# Trades ingested: 5
|
||||||
#
|
#
|
||||||
# Database totals:
|
# Database totals:
|
||||||
# Total officials: 4
|
# Total officials: 4
|
||||||
@@ -33,7 +33,7 @@ python scripts/ingest_from_fixtures.py
|
|||||||
- NVDA, MSFT, AAPL, TSLA, GOOGL tickers
|
- NVDA, MSFT, AAPL, TSLA, GOOGL tickers
|
||||||
|
|
||||||
2. **Offline Scripts**
|
2. **Offline Scripts**
|
||||||
- `scripts/ingest_from_fixtures.py` - Ingest sample trades (✅ works now!)
|
- `scripts/ingest_from_fixtures.py` - Ingest sample trades ( works now!)
|
||||||
- `scripts/fetch_sample_prices.py` - Would need network (yfinance)
|
- `scripts/fetch_sample_prices.py` - Would need network (yfinance)
|
||||||
|
|
||||||
3. **28 Passing Tests** - All use mocks, no network required
|
3. **28 Passing Tests** - All use mocks, no network required
|
||||||
@@ -67,14 +67,14 @@ with SessionLocal() as session:
|
|||||||
|
|
||||||
### What You Can Do Offline
|
### What You Can Do Offline
|
||||||
|
|
||||||
✅ **Run all tests**: `make test`
|
**Run all tests**: `make test`
|
||||||
✅ **Ingest fixture data**: `python scripts/ingest_from_fixtures.py`
|
**Ingest fixture data**: `python scripts/ingest_from_fixtures.py`
|
||||||
✅ **Query the database**: Use Python REPL or SQLite browser
|
**Query the database**: Use Python REPL or SQLite browser
|
||||||
✅ **Lint & format**: `make lint format`
|
**Lint & format**: `make lint format`
|
||||||
✅ **Run migrations**: `make migrate`
|
**Run migrations**: `make migrate`
|
||||||
✅ **Build analytics** (Phase 2): All math/ML works offline!
|
**Build analytics** (Phase 2): All math/ML works offline!
|
||||||
|
|
||||||
❌ **Can't do (needs network)**:
|
**Can't do (needs network)**:
|
||||||
- Fetch live congressional trades from House Stock Watcher
|
- Fetch live congressional trades from House Stock Watcher
|
||||||
- Fetch stock prices from yfinance
|
- Fetch stock prices from yfinance
|
||||||
- (But you can add more fixture files to simulate this!)
|
- (But you can add more fixture files to simulate this!)
|
||||||
@@ -108,9 +108,9 @@ You can expand the fixtures for offline development:
|
|||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
**The network error is not a problem!** The entire system is designed to work with:
|
**The network error is not a problem!** The entire system is designed to work with:
|
||||||
- ✅ Fixtures for development/testing
|
- Fixtures for development/testing
|
||||||
- ✅ Real APIs for production (when network available)
|
- Real APIs for production (when network available)
|
||||||
- ✅ Same code paths for both
|
- Same code paths for both
|
||||||
|
|
||||||
This is **by design** - makes development fast and tests reliable! 🚀
|
This is **by design** - makes development fast and tests reliable!
|
||||||
|
|
||||||
+6
-6
@@ -223,12 +223,12 @@ print(f"Win Rate: {pelosi_stats['win_rate']:.1%}")
|
|||||||
|
|
||||||
## Success Criteria
|
## Success Criteria
|
||||||
|
|
||||||
- ✅ Can calculate returns for any trade + window
|
- Can calculate returns for any trade + window
|
||||||
- ✅ Can compare to S&P 500 benchmark
|
- Can compare to S&P 500 benchmark
|
||||||
- ✅ Can generate official performance summaries
|
- Can generate official performance summaries
|
||||||
- ✅ All calculations tested and accurate
|
- All calculations tested and accurate
|
||||||
- ✅ Performance data stored efficiently
|
- Performance data stored efficiently
|
||||||
- ✅ Documentation complete
|
- Documentation complete
|
||||||
|
|
||||||
## Timeline
|
## Timeline
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# Proxmox Quick Start ⚡
|
# Proxmox Quick Start
|
||||||
|
|
||||||
**Got Proxmox? Deploy POTE in 5 minutes!**
|
**Got Proxmox? Deploy POTE in 5 minutes!**
|
||||||
|
|
||||||
@@ -17,7 +17,7 @@ su - poteapp
|
|||||||
cd pote && source venv/bin/activate
|
cd pote && source venv/bin/activate
|
||||||
python scripts/ingest_from_fixtures.py
|
python scripts/ingest_from_fixtures.py
|
||||||
|
|
||||||
# Done! ✅
|
# Done!
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -81,8 +81,8 @@ source venv/bin/activate
|
|||||||
python scripts/ingest_from_fixtures.py
|
python scripts/ingest_from_fixtures.py
|
||||||
|
|
||||||
# Should see:
|
# Should see:
|
||||||
# ✓ Officials created: 4
|
# Officials created: 4
|
||||||
# ✓ Trades ingested: 5
|
# Trades ingested: 5
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5. Setup Cron Jobs
|
### 5. Setup Cron Jobs
|
||||||
@@ -96,7 +96,7 @@ crontab -e
|
|||||||
15 6 * * * cd /home/poteapp/pote && /home/poteapp/pote/venv/bin/python scripts/enrich_securities.py >> /home/poteapp/logs/enrich.log 2>&1
|
15 6 * * * cd /home/poteapp/pote && /home/poteapp/pote/venv/bin/python scripts/enrich_securities.py >> /home/poteapp/logs/enrich.log 2>&1
|
||||||
```
|
```
|
||||||
|
|
||||||
### 6. Done! 🎉
|
### 6. Done!
|
||||||
|
|
||||||
Your POTE instance is now running and will:
|
Your POTE instance is now running and will:
|
||||||
- Fetch congressional trades daily at 6 AM
|
- Fetch congressional trades daily at 6 AM
|
||||||
@@ -107,12 +107,12 @@ Your POTE instance is now running and will:
|
|||||||
|
|
||||||
## What You Get
|
## What You Get
|
||||||
|
|
||||||
✅ **Full PostgreSQL database**
|
**Full PostgreSQL database**
|
||||||
✅ **Automated daily updates** (via cron)
|
**Automated daily updates** (via cron)
|
||||||
✅ **Isolated environment** (LXC container)
|
**Isolated environment** (LXC container)
|
||||||
✅ **Easy backups** (Proxmox snapshots)
|
**Easy backups** (Proxmox snapshots)
|
||||||
✅ **Low resource usage** (~500MB RAM)
|
**Low resource usage** (~500MB RAM)
|
||||||
✅ **Cost**: Just electricity (~$5-10/mo)
|
**Cost**: Just electricity (~$5-10/mo)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -231,13 +231,13 @@ pip install -e .
|
|||||||
|
|
||||||
## Next Steps
|
## Next Steps
|
||||||
|
|
||||||
1. ✅ Container running
|
1. Container running
|
||||||
2. ✅ POTE installed
|
2. POTE installed
|
||||||
3. ✅ Data ingested
|
3. Data ingested
|
||||||
4. ⏭️ Setup Proxmox backups (Web UI → Datacenter → Backup)
|
4. ⏭ Setup Proxmox backups (Web UI → Datacenter → Backup)
|
||||||
5. ⏭️ Configure static IP (if needed)
|
5. ⏭ Configure static IP (if needed)
|
||||||
6. ⏭️ Build Phase 2 analytics
|
6. ⏭ Build Phase 2 analytics
|
||||||
7. ⏭️ Add FastAPI dashboard
|
7. ⏭ Add FastAPI dashboard
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -264,10 +264,10 @@ pct restart 100
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Your Proxmox = Enterprise infrastructure at hobby prices!** 🚀
|
**Your Proxmox = Enterprise infrastructure at hobby prices!**
|
||||||
|
|
||||||
Cost breakdown:
|
Cost breakdown:
|
||||||
- Cloud VPS: $20/mo
|
- Cloud VPS: $20/mo
|
||||||
- Your Proxmox: ~$10/mo (power)
|
- Your Proxmox: ~$10/mo (power)
|
||||||
- **Savings: $120/year** ✨
|
- **Savings: $120/year**
|
||||||
|
|
||||||
@@ -1,15 +1,15 @@
|
|||||||
# POTE Quick Start Guide
|
# POTE Quick Start Guide
|
||||||
|
|
||||||
## 🚀 Your System is Ready!
|
## Your System is Ready!
|
||||||
|
|
||||||
**Container IP**: Check with `ip addr show eth0 | grep "inet"`
|
**Container IP**: Check with `ip addr show eth0 | grep "inet"`
|
||||||
**Database**: PostgreSQL on port 5432
|
**Database**: PostgreSQL on port 5432
|
||||||
**Username**: `poteuser`
|
**Username**: `poteuser`
|
||||||
**Password**: `changeme123` (⚠️ change in production!)
|
**Password**: `changeme123` ( change in production!)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 How to Use POTE
|
## How to Use POTE
|
||||||
|
|
||||||
### Option 1: Command Line (SSH into container)
|
### Option 1: Command Line (SSH into container)
|
||||||
|
|
||||||
@@ -51,7 +51,7 @@ with engine.connect() as conn:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Common Tasks
|
## Common Tasks
|
||||||
|
|
||||||
### 1. Check System Status
|
### 1. Check System Status
|
||||||
|
|
||||||
@@ -145,7 +145,7 @@ ORDER BY trade_count DESC;
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 Example Workflows
|
## Example Workflows
|
||||||
|
|
||||||
### Workflow 1: Daily Update
|
### Workflow 1: Daily Update
|
||||||
|
|
||||||
@@ -237,7 +237,7 @@ print(f"Exported {len(df)} trades to trades_export.csv")
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 Maintenance
|
## Maintenance
|
||||||
|
|
||||||
### Update POTE Code
|
### Update POTE Code
|
||||||
|
|
||||||
@@ -287,7 +287,7 @@ nano ~/pote/.env
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🌐 Access Methods Summary
|
## Access Methods Summary
|
||||||
|
|
||||||
| Method | From Where | Command |
|
| Method | From Where | Command |
|
||||||
|--------|-----------|---------|
|
|--------|-----------|---------|
|
||||||
@@ -298,22 +298,22 @@ nano ~/pote/.env
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📚 What Data Do You Have?
|
## What Data Do You Have?
|
||||||
|
|
||||||
Right now (Phase 1 complete):
|
Right now (Phase 1 complete):
|
||||||
- ✅ **Congressional trading data** (from House Stock Watcher)
|
- **Congressional trading data** (from House Stock Watcher)
|
||||||
- ✅ **Security information** (tickers, names, sectors)
|
- **Security information** (tickers, names, sectors)
|
||||||
- ✅ **Historical prices** (OHLCV data from yfinance)
|
- **Historical prices** (OHLCV data from yfinance)
|
||||||
- ✅ **Official profiles** (name, party, chamber, state)
|
- **Official profiles** (name, party, chamber, state)
|
||||||
|
|
||||||
Coming next (Phase 2):
|
Coming next (Phase 2):
|
||||||
- 📊 **Abnormal return calculations**
|
- **Abnormal return calculations**
|
||||||
- 🤖 **Behavioral clustering**
|
- **Behavioral clustering**
|
||||||
- 🚨 **Research signals** (follow_research, avoid_risk, watch)
|
- **Research signals** (follow_research, avoid_risk, watch)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎓 Learning SQL for POTE
|
## Learning SQL for POTE
|
||||||
|
|
||||||
### Count Records
|
### Count Records
|
||||||
```sql
|
```sql
|
||||||
@@ -349,7 +349,7 @@ GROUP BY o.party;
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ❓ Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Can't connect remotely?
|
### Can't connect remotely?
|
||||||
```bash
|
```bash
|
||||||
@@ -380,12 +380,12 @@ pip install -e .
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Next Steps
|
## Next Steps
|
||||||
|
|
||||||
1. **Populate with real data**: Run `fetch_congressional_trades.py` regularly
|
1. **Populate with real data**: Run `fetch_congressional_trades.py` regularly
|
||||||
2. **Set up cron job** for automatic daily updates
|
2. **Set up cron job** for automatic daily updates
|
||||||
3. **Build analytics** (Phase 2) - abnormal returns, signals
|
3. **Build analytics** (Phase 2) - abnormal returns, signals
|
||||||
4. **Create dashboard** (Phase 3) - web interface for exploration
|
4. **Create dashboard** (Phase 3) - web interface for exploration
|
||||||
|
|
||||||
Ready to build Phase 2 analytics? Just ask! 📈
|
Ready to build Phase 2 analytics? Just ask!
|
||||||
|
|
||||||
@@ -1,15 +1,15 @@
|
|||||||
# 🚀 POTE Quick Setup Card
|
# POTE Quick Setup Card
|
||||||
|
|
||||||
## 📍 Your Configuration
|
## Your Configuration
|
||||||
|
|
||||||
**Email Server:** `mail.levkin.ca`
|
**Email Server:** `mail.levkin.ca`
|
||||||
**Email Account:** `test@levkin.ca`
|
**Email Account:** `test@levkin.ca`
|
||||||
**Database:** PostgreSQL (configured)
|
**Database:** PostgreSQL (configured)
|
||||||
**Status:** ✅ Ready for deployment
|
**Status:** Ready for deployment
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚡ 3-Step Setup
|
## 3-Step Setup
|
||||||
|
|
||||||
### Step 1: Add Your Password (30 seconds)
|
### Step 1: Add Your Password (30 seconds)
|
||||||
|
|
||||||
@@ -27,7 +27,7 @@ source venv/bin/activate
|
|||||||
python scripts/send_daily_report.py --to test@levkin.ca --test-smtp
|
python scripts/send_daily_report.py --to test@levkin.ca --test-smtp
|
||||||
```
|
```
|
||||||
|
|
||||||
✅ **Check test@levkin.ca inbox** - you should receive a test email!
|
**Check test@levkin.ca inbox** - you should receive a test email!
|
||||||
|
|
||||||
### Step 3: Automate (2 minutes)
|
### Step 3: Automate (2 minutes)
|
||||||
|
|
||||||
@@ -37,13 +37,13 @@ python scripts/send_daily_report.py --to test@levkin.ca --test-smtp
|
|||||||
# Choose time: 6 AM (recommended)
|
# Choose time: 6 AM (recommended)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Done!** 🎉 You'll now receive:
|
**Done!** You'll now receive:
|
||||||
- Daily reports at 6 AM
|
- Daily reports at 6 AM
|
||||||
- Weekly reports on Sundays
|
- Weekly reports on Sundays
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📦 Deployment to Proxmox (5 minutes)
|
## Deployment to Proxmox (5 minutes)
|
||||||
|
|
||||||
### On Proxmox Host:
|
### On Proxmox Host:
|
||||||
|
|
||||||
@@ -79,7 +79,7 @@ bash scripts/proxmox_setup.sh
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔍 Quick Commands
|
## Quick Commands
|
||||||
|
|
||||||
### On Deployed Server (SSH)
|
### On Deployed Server (SSH)
|
||||||
|
|
||||||
@@ -105,7 +105,7 @@ ls -lh ~/logs/*.txt
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📧 Email Configuration (.env)
|
## Email Configuration (.env)
|
||||||
|
|
||||||
```env
|
```env
|
||||||
SMTP_HOST=mail.levkin.ca
|
SMTP_HOST=mail.levkin.ca
|
||||||
@@ -123,27 +123,27 @@ REPORT_RECIPIENTS=test@levkin.ca,user2@example.com,user3@example.com
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 What You'll Receive
|
## What You'll Receive
|
||||||
|
|
||||||
### Daily Report (6 AM)
|
### Daily Report (6 AM)
|
||||||
```
|
```
|
||||||
✅ New congressional trades
|
New congressional trades
|
||||||
✅ Market alerts (unusual activity)
|
Market alerts (unusual activity)
|
||||||
✅ Suspicious timing detections
|
Suspicious timing detections
|
||||||
✅ Summary statistics
|
Summary statistics
|
||||||
```
|
```
|
||||||
|
|
||||||
### Weekly Report (Sunday 8 AM)
|
### Weekly Report (Sunday 8 AM)
|
||||||
```
|
```
|
||||||
✅ Most active officials
|
Most active officials
|
||||||
✅ Most traded securities
|
Most traded securities
|
||||||
✅ Repeat offenders
|
Repeat offenders
|
||||||
✅ Pattern analysis
|
Pattern analysis
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Email Not Working?
|
### Email Not Working?
|
||||||
|
|
||||||
@@ -191,19 +191,19 @@ tail -50 ~/logs/daily_run.log
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📚 Documentation
|
## Documentation
|
||||||
|
|
||||||
| Document | Purpose |
|
| Document | Purpose |
|
||||||
|----------|---------|
|
|----------|---------|
|
||||||
| **[EMAIL_SETUP.md](EMAIL_SETUP.md)** | ⭐ Your levkin.ca setup guide |
|
| **[EMAIL_SETUP.md](EMAIL_SETUP.md)** | Your levkin.ca setup guide |
|
||||||
| **[DEPLOYMENT_AND_AUTOMATION.md](DEPLOYMENT_AND_AUTOMATION.md)** | ⭐ Answers all questions |
|
| **[DEPLOYMENT_AND_AUTOMATION.md](DEPLOYMENT_AND_AUTOMATION.md)** | Answers all questions |
|
||||||
| **[AUTOMATION_QUICKSTART.md](AUTOMATION_QUICKSTART.md)** | Quick automation guide |
|
| **[AUTOMATION_QUICKSTART.md](AUTOMATION_QUICKSTART.md)** | Quick automation guide |
|
||||||
| **[PROXMOX_QUICKSTART.md](PROXMOX_QUICKSTART.md)** | Proxmox deployment |
|
| **[PROXMOX_QUICKSTART.md](PROXMOX_QUICKSTART.md)** | Proxmox deployment |
|
||||||
| **[QUICKSTART.md](QUICKSTART.md)** | Usage guide |
|
| **[QUICKSTART.md](QUICKSTART.md)** | Usage guide |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ✅ Checklist
|
## Checklist
|
||||||
|
|
||||||
**Local Development:**
|
**Local Development:**
|
||||||
- [ ] `.env` file created with password
|
- [ ] `.env` file created with password
|
||||||
@@ -227,18 +227,18 @@ tail -50 ~/logs/daily_run.log
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Your Current Status
|
## Your Current Status
|
||||||
|
|
||||||
✅ **Code:** Complete (93 tests passing)
|
**Code:** Complete (93 tests passing)
|
||||||
✅ **Monitoring:** 3-phase system operational
|
**Monitoring:** 3-phase system operational
|
||||||
✅ **CI/CD:** Pipeline ready (.github/workflows/ci.yml)
|
**CI/CD:** Pipeline ready (.github/workflows/ci.yml)
|
||||||
✅ **Email:** Configured for test@levkin.ca
|
**Email:** Configured for test@levkin.ca
|
||||||
⏳ **Deployment:** Ready to deploy to Proxmox
|
⏳ **Deployment:** Ready to deploy to Proxmox
|
||||||
⏳ **Automation:** Ready to set up with `setup_cron.sh`
|
⏳ **Automation:** Ready to set up with `setup_cron.sh`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Next Action
|
## Next Action
|
||||||
|
|
||||||
**Right now (local testing):**
|
**Right now (local testing):**
|
||||||
```bash
|
```bash
|
||||||
@@ -255,9 +255,9 @@ cd ~/pote
|
|||||||
./scripts/setup_cron.sh
|
./scripts/setup_cron.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
**That's it! 🎉**
|
**That's it! **
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Everything is ready - just add your password and test!** 📧
|
**Everything is ready - just add your password and test!**
|
||||||
|
|
||||||
+27
-27
@@ -3,25 +3,25 @@
|
|||||||
**Last Updated**: 2025-12-14
|
**Last Updated**: 2025-12-14
|
||||||
**Version**: Phase 1 Complete (PR1 + PR2)
|
**Version**: Phase 1 Complete (PR1 + PR2)
|
||||||
|
|
||||||
## 🎉 What's Working Now
|
## What's Working Now
|
||||||
|
|
||||||
### Data Ingestion (FREE!)
|
### Data Ingestion (FREE!)
|
||||||
✅ **Congressional Trades**: Live ingestion from House Stock Watcher
|
**Congressional Trades**: Live ingestion from House Stock Watcher
|
||||||
✅ **Stock Prices**: Daily OHLCV from yfinance
|
**Stock Prices**: Daily OHLCV from yfinance
|
||||||
✅ **Officials**: Auto-populated from trade disclosures
|
**Officials**: Auto-populated from trade disclosures
|
||||||
✅ **Securities**: Auto-created, ready for enrichment
|
**Securities**: Auto-created, ready for enrichment
|
||||||
|
|
||||||
### Database
|
### Database
|
||||||
✅ **Schema**: Normalized (officials, securities, trades, prices, metrics stubs)
|
**Schema**: Normalized (officials, securities, trades, prices, metrics stubs)
|
||||||
✅ **Migrations**: Alembic configured and applied
|
**Migrations**: Alembic configured and applied
|
||||||
✅ **DB**: SQLite for dev, PostgreSQL-ready
|
**DB**: SQLite for dev, PostgreSQL-ready
|
||||||
|
|
||||||
### Code Quality
|
### Code Quality
|
||||||
✅ **Tests**: 28 passing (86% coverage)
|
**Tests**: 28 passing (86% coverage)
|
||||||
✅ **Linting**: ruff + mypy all green
|
**Linting**: ruff + mypy all green
|
||||||
✅ **Format**: black applied consistently
|
**Format**: black applied consistently
|
||||||
|
|
||||||
## 📊 Current Stats
|
## Current Stats
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Test Suite
|
# Test Suite
|
||||||
@@ -43,7 +43,7 @@ All free/open-source:
|
|||||||
- pytest (testing)
|
- pytest (testing)
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🚀 Quick Commands
|
## Quick Commands
|
||||||
|
|
||||||
### Fetch Live Data (FREE!)
|
### Fetch Live Data (FREE!)
|
||||||
```bash
|
```bash
|
||||||
@@ -75,7 +75,7 @@ make format # Format with black
|
|||||||
make migrate # Run Alembic migrations
|
make migrate # Run Alembic migrations
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🏠 Deployment
|
## Deployment
|
||||||
|
|
||||||
**Your Proxmox?** Perfect! See [`docs/08_proxmox_deployment.md`](docs/08_proxmox_deployment.md) for:
|
**Your Proxmox?** Perfect! See [`docs/08_proxmox_deployment.md`](docs/08_proxmox_deployment.md) for:
|
||||||
- LXC container setup (lightweight, recommended)
|
- LXC container setup (lightweight, recommended)
|
||||||
@@ -90,7 +90,7 @@ Other options in [`docs/07_deployment.md`](docs/07_deployment.md):
|
|||||||
- Railway/Fly.io - $5-15/mo
|
- Railway/Fly.io - $5-15/mo
|
||||||
- AWS/GCP - $20-50/mo
|
- AWS/GCP - $20-50/mo
|
||||||
|
|
||||||
## 📂 Project Structure
|
## Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
pote/
|
pote/
|
||||||
@@ -138,7 +138,7 @@ pote/
|
|||||||
└── fetch_sample_prices.py # Live price fetch
|
└── fetch_sample_prices.py # Live price fetch
|
||||||
```
|
```
|
||||||
|
|
||||||
## 💰 Cost Breakdown
|
## Cost Breakdown
|
||||||
|
|
||||||
| Component | Cost | Notes |
|
| Component | Cost | Notes |
|
||||||
|-----------|------|-------|
|
|-----------|------|-------|
|
||||||
@@ -154,7 +154,7 @@ Optional paid upgrades (NOT needed):
|
|||||||
- Financial Modeling Prep: $15/mo (250 calls/day free tier available)
|
- Financial Modeling Prep: $15/mo (250 calls/day free tier available)
|
||||||
- PostgreSQL hosting: $7+/mo (only if deploying)
|
- PostgreSQL hosting: $7+/mo (only if deploying)
|
||||||
|
|
||||||
## ✅ Completed PRs
|
## Completed PRs
|
||||||
|
|
||||||
### PR1: Project Scaffold + Price Loader
|
### PR1: Project Scaffold + Price Loader
|
||||||
- [x] Project structure (`src/`, `tests/`, docs)
|
- [x] Project structure (`src/`, `tests/`, docs)
|
||||||
@@ -176,7 +176,7 @@ Optional paid upgrades (NOT needed):
|
|||||||
|
|
||||||
**See**: [`docs/PR2_SUMMARY.md`](docs/PR2_SUMMARY.md)
|
**See**: [`docs/PR2_SUMMARY.md`](docs/PR2_SUMMARY.md)
|
||||||
|
|
||||||
## 📋 Next Steps (Phase 2 - Analytics)
|
## Next Steps (Phase 2 - Analytics)
|
||||||
|
|
||||||
### PR3: Security Enrichment
|
### PR3: Security Enrichment
|
||||||
- [ ] Enrich securities table with yfinance (names, sectors, exchanges)
|
- [ ] Enrich securities table with yfinance (names, sectors, exchanges)
|
||||||
@@ -206,20 +206,20 @@ Optional paid upgrades (NOT needed):
|
|||||||
|
|
||||||
**See**: [`docs/00_mvp.md`](docs/00_mvp.md) for full roadmap
|
**See**: [`docs/00_mvp.md`](docs/00_mvp.md) for full roadmap
|
||||||
|
|
||||||
## 🔬 Research-Only Reminder
|
## Research-Only Reminder
|
||||||
|
|
||||||
**This tool is for private research and transparency analysis only.**
|
**This tool is for private research and transparency analysis only.**
|
||||||
|
|
||||||
- ❌ Not investment advice
|
- Not investment advice
|
||||||
- ❌ Not a trading system
|
- Not a trading system
|
||||||
- ❌ No claims about inside information
|
- No claims about inside information
|
||||||
- ✅ Public data only
|
- Public data only
|
||||||
- ✅ Descriptive analytics
|
- Descriptive analytics
|
||||||
- ✅ Research transparency
|
- Research transparency
|
||||||
|
|
||||||
See [`docs/04_safety_ethics.md`](docs/04_safety_ethics.md) for guardrails.
|
See [`docs/04_safety_ethics.md`](docs/04_safety_ethics.md) for guardrails.
|
||||||
|
|
||||||
## 🤝 Contributing
|
## Contributing
|
||||||
|
|
||||||
This is a personal research project, but if you want to use it:
|
This is a personal research project, but if you want to use it:
|
||||||
|
|
||||||
@@ -229,7 +229,7 @@ This is a personal research project, but if you want to use it:
|
|||||||
4. `python scripts/fetch_congressional_trades.py --days 7`
|
4. `python scripts/fetch_congressional_trades.py --days 7`
|
||||||
5. Start exploring!
|
5. Start exploring!
|
||||||
|
|
||||||
## 📄 License
|
## License
|
||||||
|
|
||||||
MIT License (for research/educational use only)
|
MIT License (for research/educational use only)
|
||||||
|
|
||||||
@@ -1,12 +1,12 @@
|
|||||||
# POTE Testing Status Report
|
# POTE Testing Status Report
|
||||||
**Date:** December 15, 2025
|
**Date:** December 15, 2025
|
||||||
**Status:** ✅ All Systems Operational - Ready for Deployment
|
**Status:** All Systems Operational - Ready for Deployment
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Test Suite Summary
|
## Test Suite Summary
|
||||||
|
|
||||||
### **55 Tests - All Passing ✅**
|
### **55 Tests - All Passing **
|
||||||
|
|
||||||
```
|
```
|
||||||
Platform: Python 3.13.5, pytest-9.0.2
|
Platform: Python 3.13.5, pytest-9.0.2
|
||||||
@@ -18,17 +18,17 @@ Coverage: ~85% overall
|
|||||||
|
|
||||||
| Module | Tests | Status | Coverage |
|
| Module | Tests | Status | Coverage |
|
||||||
|--------|-------|--------|----------|
|
|--------|-------|--------|----------|
|
||||||
| **Analytics** | 18 tests | ✅ PASS | 80% |
|
| **Analytics** | 18 tests | PASS | 80% |
|
||||||
| **Models** | 7 tests | ✅ PASS | 90% |
|
| **Models** | 7 tests | PASS | 90% |
|
||||||
| **Ingestion** | 14 tests | ✅ PASS | 85% |
|
| **Ingestion** | 14 tests | PASS | 85% |
|
||||||
| **Price Loader** | 8 tests | ✅ PASS | 90% |
|
| **Price Loader** | 8 tests | PASS | 90% |
|
||||||
| **Security Enricher** | 8 tests | ✅ PASS | 85% |
|
| **Security Enricher** | 8 tests | PASS | 85% |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 What's Been Tested?
|
## What's Been Tested?
|
||||||
|
|
||||||
### ✅ Core Database Operations
|
### Core Database Operations
|
||||||
- [x] Creating and querying Officials
|
- [x] Creating and querying Officials
|
||||||
- [x] Creating and querying Securities
|
- [x] Creating and querying Securities
|
||||||
- [x] Creating and querying Trades
|
- [x] Creating and querying Trades
|
||||||
@@ -36,7 +36,7 @@ Coverage: ~85% overall
|
|||||||
- [x] Unique constraints and relationships
|
- [x] Unique constraints and relationships
|
||||||
- [x] Database migrations (Alembic)
|
- [x] Database migrations (Alembic)
|
||||||
|
|
||||||
### ✅ Data Ingestion
|
### Data Ingestion
|
||||||
- [x] House Stock Watcher client (with fixtures)
|
- [x] House Stock Watcher client (with fixtures)
|
||||||
- [x] Trade loading from JSON
|
- [x] Trade loading from JSON
|
||||||
- [x] Security enrichment from yfinance
|
- [x] Security enrichment from yfinance
|
||||||
@@ -44,7 +44,7 @@ Coverage: ~85% overall
|
|||||||
- [x] Idempotent operations (no duplicates)
|
- [x] Idempotent operations (no duplicates)
|
||||||
- [x] Error handling for missing/invalid data
|
- [x] Error handling for missing/invalid data
|
||||||
|
|
||||||
### ✅ Analytics Engine
|
### Analytics Engine
|
||||||
- [x] Return calculations (buy trades)
|
- [x] Return calculations (buy trades)
|
||||||
- [x] Return calculations (sell trades)
|
- [x] Return calculations (sell trades)
|
||||||
- [x] Multiple time windows (30/60/90/180 days)
|
- [x] Multiple time windows (30/60/90/180 days)
|
||||||
@@ -56,7 +56,7 @@ Coverage: ~85% overall
|
|||||||
- [x] Top performer rankings
|
- [x] Top performer rankings
|
||||||
- [x] System-wide statistics
|
- [x] System-wide statistics
|
||||||
|
|
||||||
### ✅ Edge Cases
|
### Edge Cases
|
||||||
- [x] Missing price data handling
|
- [x] Missing price data handling
|
||||||
- [x] Trades with no exit price yet
|
- [x] Trades with no exit price yet
|
||||||
- [x] Sell trades (inverted returns)
|
- [x] Sell trades (inverted returns)
|
||||||
@@ -67,7 +67,7 @@ Coverage: ~85% overall
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🧪 Test Types
|
## Test Types
|
||||||
|
|
||||||
### 1. Unit Tests (Fast, Isolated)
|
### 1. Unit Tests (Fast, Isolated)
|
||||||
**Location:** `tests/test_*.py` (excluding integration)
|
**Location:** `tests/test_*.py` (excluding integration)
|
||||||
@@ -100,7 +100,7 @@ Coverage: ~85% overall
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 How to Run Tests Locally
|
## How to Run Tests Locally
|
||||||
|
|
||||||
### Quick Test
|
### Quick Test
|
||||||
```bash
|
```bash
|
||||||
@@ -136,7 +136,7 @@ ptw
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚨 Known Limitations
|
## Known Limitations
|
||||||
|
|
||||||
### 1. External API Dependency
|
### 1. External API Dependency
|
||||||
**Issue:** House Stock Watcher API is currently DOWN
|
**Issue:** House Stock Watcher API is currently DOWN
|
||||||
@@ -161,7 +161,7 @@ ptw
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 Performance Benchmarks
|
## Performance Benchmarks
|
||||||
|
|
||||||
### Test Execution Time
|
### Test Execution Time
|
||||||
- **Full suite:** 1.8 seconds
|
- **Full suite:** 1.8 seconds
|
||||||
@@ -178,7 +178,7 @@ ptw
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Pre-Deployment Checklist
|
## Pre-Deployment Checklist
|
||||||
|
|
||||||
### Before Deploying to Proxmox:
|
### Before Deploying to Proxmox:
|
||||||
|
|
||||||
@@ -215,7 +215,7 @@ python scripts/enrich_securities.py
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔄 Continuous Testing
|
## Continuous Testing
|
||||||
|
|
||||||
### Git Pre-Commit Hook (Optional)
|
### Git Pre-Commit Hook (Optional)
|
||||||
```bash
|
```bash
|
||||||
@@ -247,7 +247,7 @@ jobs:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📝 Test Maintenance
|
## Test Maintenance
|
||||||
|
|
||||||
### Adding New Tests
|
### Adding New Tests
|
||||||
|
|
||||||
@@ -281,36 +281,36 @@ Fixtures are in `tests/conftest.py`:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎉 Summary
|
## Summary
|
||||||
|
|
||||||
### Current Status: **PRODUCTION READY** ✅
|
### Current Status: **PRODUCTION READY**
|
||||||
|
|
||||||
**What Works:**
|
**What Works:**
|
||||||
- ✅ All 55 tests passing
|
- All 55 tests passing
|
||||||
- ✅ Full analytics pipeline functional
|
- Full analytics pipeline functional
|
||||||
- ✅ Database operations solid
|
- Database operations solid
|
||||||
- ✅ Data ingestion from multiple sources
|
- Data ingestion from multiple sources
|
||||||
- ✅ Price fetching from yfinance
|
- Price fetching from yfinance
|
||||||
- ✅ Security enrichment
|
- Security enrichment
|
||||||
- ✅ Return calculations
|
- Return calculations
|
||||||
- ✅ Benchmark comparisons
|
- Benchmark comparisons
|
||||||
- ✅ Performance metrics
|
- Performance metrics
|
||||||
- ✅ CLI scripts operational
|
- CLI scripts operational
|
||||||
|
|
||||||
**What's Missing:**
|
**What's Missing:**
|
||||||
- ❌ Live congressional trade API (external issue - House Stock Watcher down)
|
- Live congressional trade API (external issue - House Stock Watcher down)
|
||||||
- **Workaround:** Manual import, CSV, or alternative APIs available
|
- **Workaround:** Manual import, CSV, or alternative APIs available
|
||||||
|
|
||||||
**Next Steps:**
|
**Next Steps:**
|
||||||
1. ✅ Tests are complete
|
1. Tests are complete
|
||||||
2. ✅ Code is ready
|
2. Code is ready
|
||||||
3. ➡️ **Deploy to Proxmox** (or continue with Phase 2 features)
|
3. **Deploy to Proxmox** (or continue with Phase 2 features)
|
||||||
4. ➡️ Add more data sources
|
4. Add more data sources
|
||||||
5. ➡️ Build dashboard (Phase 3)
|
5. Build dashboard (Phase 3)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📞 Need Help?
|
## Need Help?
|
||||||
|
|
||||||
See:
|
See:
|
||||||
- `LOCAL_TEST_GUIDE.md` - Detailed local testing instructions
|
- `LOCAL_TEST_GUIDE.md` - Detailed local testing instructions
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# POTE Watchlist & Trading Reports
|
# POTE Watchlist & Trading Reports
|
||||||
|
|
||||||
## 🎯 Get Trading Reports 1 Hour Before Market Close
|
## Get Trading Reports 1 Hour Before Market Close
|
||||||
|
|
||||||
### Quick Setup
|
### Quick Setup
|
||||||
|
|
||||||
@@ -25,7 +25,7 @@ crontab -e
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📋 Watchlist System
|
## Watchlist System
|
||||||
|
|
||||||
### Who's on the Default Watchlist?
|
### Who's on the Default Watchlist?
|
||||||
|
|
||||||
@@ -48,7 +48,7 @@ crontab -e
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 Managing Your Watchlist
|
## Managing Your Watchlist
|
||||||
|
|
||||||
### View Current Watchlist
|
### View Current Watchlist
|
||||||
|
|
||||||
@@ -104,7 +104,7 @@ This fetches all 535 members of Congress (100 Senate + 435 House).
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 Generating Reports
|
## Generating Reports
|
||||||
|
|
||||||
### Manual Report Generation
|
### Manual Report Generation
|
||||||
|
|
||||||
@@ -135,24 +135,24 @@ python scripts/generate_trading_report.py --format json --output report.json
|
|||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
👤 Nancy Pelosi (D-CA, House)
|
Nancy Pelosi (D-CA, House)
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
Side Ticker Company Sector Value Trade Date Filed
|
Side Ticker Company Sector Value Trade Date Filed
|
||||||
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
|
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
|
||||||
🟢 BUY NVDA NVIDIA Corporation Technology $15,001 - $50,000 2024-11-15 2024-12-01
|
BUY NVDA NVIDIA Corporation Technology $15,001 - $50,000 2024-11-15 2024-12-01
|
||||||
🔴 SELL MSFT Microsoft Corporation Technology $50,001 - $100,000 2024-11-20 2024-12-01
|
SELL MSFT Microsoft Corporation Technology $50,001 - $100,000 2024-11-20 2024-12-01
|
||||||
|
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
👤 Tommy Tuberville (R-AL, Senate)
|
Tommy Tuberville (R-AL, Senate)
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
Side Ticker Company Sector Value Trade Date Filed
|
Side Ticker Company Sector Value Trade Date Filed
|
||||||
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
|
-------- ------ -------------------------- ---------- ------------------- ---------- ----------
|
||||||
🟢 BUY SPY SPDR S&P 500 ETF Financial $100,001 - $250,000 2024-11-18 2024-12-02
|
BUY SPY SPDR S&P 500 ETF Financial $100,001 - $250,000 2024-11-18 2024-12-02
|
||||||
🟢 BUY AAPL Apple Inc. Technology $50,001 - $100,000 2024-11-22 2024-12-02
|
BUY AAPL Apple Inc. Technology $50,001 - $100,000 2024-11-22 2024-12-02
|
||||||
🔴 SELL TSLA Tesla, Inc. Automotive $15,001 - $50,000 2024-11-25 2024-12-02
|
SELL TSLA Tesla, Inc. Automotive $15,001 - $50,000 2024-11-25 2024-12-02
|
||||||
|
|
||||||
================================================================================
|
================================================================================
|
||||||
📊 SUMMARY
|
SUMMARY
|
||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
Total Trades: 5
|
Total Trades: 5
|
||||||
@@ -223,7 +223,7 @@ Runs at 8 AM and 3 PM daily (weekdays).
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📧 Email Reports (Optional)
|
## Email Reports (Optional)
|
||||||
|
|
||||||
### Setup Email Notifications
|
### Setup Email Notifications
|
||||||
|
|
||||||
@@ -256,7 +256,7 @@ python scripts/send_email.py /tmp/report.html your-email@example.com
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 Typical Workflow
|
## Typical Workflow
|
||||||
|
|
||||||
### Daily Routine (3 PM ET)
|
### Daily Routine (3 PM ET)
|
||||||
|
|
||||||
@@ -282,7 +282,7 @@ python scripts/send_email.py /tmp/report.html your-email@example.com
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔍 Finding More Officials
|
## Finding More Officials
|
||||||
|
|
||||||
### Public Resources
|
### Public Resources
|
||||||
|
|
||||||
@@ -318,7 +318,7 @@ Add committee members to your watchlist.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 Example Cron Setup
|
## Example Cron Setup
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Edit crontab
|
# Edit crontab
|
||||||
@@ -341,7 +341,7 @@ This gives you:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 Quick Start Summary
|
## Quick Start Summary
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# 1. Create watchlist
|
# 1. Create watchlist
|
||||||
@@ -363,7 +363,7 @@ cat reports/trading_report_$(date +%Y%m%d).txt
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ❓ FAQ
|
## FAQ
|
||||||
|
|
||||||
**Q: Why are all the trades old (30-45 days)?**
|
**Q: Why are all the trades old (30-45 days)?**
|
||||||
**A:** Federal law (STOCK Act) gives Congress 30-45 days to file. This is normal.
|
**A:** Federal law (STOCK Act) gives Congress 30-45 days to file. This is normal.
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# 🎉 POTE Monitoring System - ALL PHASES COMPLETE!
|
# POTE Monitoring System - ALL PHASES COMPLETE!
|
||||||
|
|
||||||
## ✅ **What Was Built (3 Phases)**
|
## **What Was Built (3 Phases)**
|
||||||
|
|
||||||
### **Phase 1: Real-Time Market Monitoring** ✅
|
### **Phase 1: Real-Time Market Monitoring**
|
||||||
**Detects unusual market activity in congressional tickers**
|
**Detects unusual market activity in congressional tickers**
|
||||||
|
|
||||||
**Features:**
|
**Features:**
|
||||||
@@ -20,11 +20,11 @@
|
|||||||
- `MarketAlert` model - Database storage
|
- `MarketAlert` model - Database storage
|
||||||
- `monitor_market.py` - CLI tool
|
- `monitor_market.py` - CLI tool
|
||||||
|
|
||||||
**Tests:** 14 passing ✅
|
**Tests:** 14 passing
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### **Phase 2: Disclosure Timing Correlation** ✅
|
### **Phase 2: Disclosure Timing Correlation**
|
||||||
**Matches trades to prior market alerts when disclosures appear**
|
**Matches trades to prior market alerts when disclosures appear**
|
||||||
|
|
||||||
**Features:**
|
**Features:**
|
||||||
@@ -50,11 +50,11 @@
|
|||||||
- `DisclosureCorrelator` - Correlation engine
|
- `DisclosureCorrelator` - Correlation engine
|
||||||
- `analyze_disclosure_timing.py` - CLI tool
|
- `analyze_disclosure_timing.py` - CLI tool
|
||||||
|
|
||||||
**Tests:** 13 passing ✅
|
**Tests:** 13 passing
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### **Phase 3: Pattern Detection & Rankings** ✅
|
### **Phase 3: Pattern Detection & Rankings**
|
||||||
**Cross-official analysis and comparative rankings**
|
**Cross-official analysis and comparative rankings**
|
||||||
|
|
||||||
**Features:**
|
**Features:**
|
||||||
@@ -71,19 +71,19 @@
|
|||||||
- `PatternDetector` - Pattern analysis engine
|
- `PatternDetector` - Pattern analysis engine
|
||||||
- `generate_pattern_report.py` - CLI tool
|
- `generate_pattern_report.py` - CLI tool
|
||||||
|
|
||||||
**Tests:** 11 passing ✅
|
**Tests:** 11 passing
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 **Complete System Architecture**
|
## **Complete System Architecture**
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ PHASE 1: Real-Time Monitoring │
|
│ PHASE 1: Real-Time Monitoring │
|
||||||
│ ──────────────────────────────────── │
|
│ ──────────────────────────────────── │
|
||||||
│ 🔔 Monitor congressional tickers │
|
│ Monitor congressional tickers │
|
||||||
│ 📊 Detect unusual activity │
|
│ Detect unusual activity │
|
||||||
│ 💾 Log alerts to database │
|
│ Log alerts to database │
|
||||||
└─────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────┘
|
||||||
↓
|
↓
|
||||||
[30-45 days pass]
|
[30-45 days pass]
|
||||||
@@ -91,25 +91,25 @@
|
|||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ PHASE 2: Disclosure Correlation │
|
│ PHASE 2: Disclosure Correlation │
|
||||||
│ ─────────────────────────────── │
|
│ ─────────────────────────────── │
|
||||||
│ 📋 New congressional trades filed │
|
│ New congressional trades filed │
|
||||||
│ 🔗 Match to prior alerts │
|
│ Match to prior alerts │
|
||||||
│ 📈 Calculate timing scores │
|
│ Calculate timing scores │
|
||||||
│ 🚩 Flag suspicious trades │
|
│ Flag suspicious trades │
|
||||||
└─────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────┘
|
||||||
↓
|
↓
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
┌─────────────────────────────────────────────────────────────┐
|
||||||
│ PHASE 3: Pattern Detection │
|
│ PHASE 3: Pattern Detection │
|
||||||
│ ────────────────────────── │
|
│ ────────────────────────── │
|
||||||
│ 📊 Rank officials by timing │
|
│ Rank officials by timing │
|
||||||
│ 🔥 Identify repeat offenders │
|
│ Identify repeat offenders │
|
||||||
│ 📈 Compare parties, sectors, tickers │
|
│ Compare parties, sectors, tickers │
|
||||||
│ 📋 Generate comprehensive reports │
|
│ Generate comprehensive reports │
|
||||||
└─────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 **Usage Guide**
|
## **Usage Guide**
|
||||||
|
|
||||||
### **1. Set Up Monitoring (Run Daily)**
|
### **1. Set Up Monitoring (Run Daily)**
|
||||||
|
|
||||||
@@ -169,7 +169,7 @@ python scripts/generate_pattern_report.py --days 365 --format json --output patt
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📋 **Example Reports**
|
## **Example Reports**
|
||||||
|
|
||||||
### **Timing Analysis Report**
|
### **Timing Analysis Report**
|
||||||
|
|
||||||
@@ -179,7 +179,7 @@ python scripts/generate_pattern_report.py --days 365 --format json --output patt
|
|||||||
3 Trades with Timing Advantages Detected
|
3 Trades with Timing Advantages Detected
|
||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
🚨 #1 - HIGHLY SUSPICIOUS (Timing Score: 85/100)
|
#1 - HIGHLY SUSPICIOUS (Timing Score: 85/100)
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
Official: Nancy Pelosi
|
Official: Nancy Pelosi
|
||||||
Ticker: NVDA
|
Ticker: NVDA
|
||||||
@@ -187,16 +187,16 @@ Side: BUY
|
|||||||
Trade Date: 2024-01-15
|
Trade Date: 2024-01-15
|
||||||
Value: $15,001-$50,000
|
Value: $15,001-$50,000
|
||||||
|
|
||||||
📊 Timing Analysis:
|
Timing Analysis:
|
||||||
Prior Alerts: 3
|
Prior Alerts: 3
|
||||||
Recent Alerts (7d): 2
|
Recent Alerts (7d): 2
|
||||||
High Severity: 2
|
High Severity: 2
|
||||||
Avg Severity: 7.5/10
|
Avg Severity: 7.5/10
|
||||||
|
|
||||||
💡 Assessment: Trade occurred after 3 alerts, including 2 high-severity.
|
Assessment: Trade occurred after 3 alerts, including 2 high-severity.
|
||||||
High likelihood of timing advantage.
|
High likelihood of timing advantage.
|
||||||
|
|
||||||
🔔 Prior Market Alerts:
|
Prior Market Alerts:
|
||||||
Timestamp Type Severity Timing
|
Timestamp Type Severity Timing
|
||||||
2024-01-12 10:30:00 Unusual Volume 8/10 3 days before
|
2024-01-12 10:30:00 Unusual Volume 8/10 3 days before
|
||||||
2024-01-13 14:15:00 Price Spike 7/10 2 days before
|
2024-01-13 14:15:00 Price Spike 7/10 2 days before
|
||||||
@@ -211,27 +211,27 @@ Timestamp Type Severity Timing
|
|||||||
Period: 365 days
|
Period: 365 days
|
||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
📊 SUMMARY
|
SUMMARY
|
||||||
────────────────────────────────────────────────────────────────────────────────
|
────────────────────────────────────────────────────────────────────────────────
|
||||||
Officials Analyzed: 45
|
Officials Analyzed: 45
|
||||||
Repeat Offenders: 8
|
Repeat Offenders: 8
|
||||||
Average Timing Score: 42.3/100
|
Average Timing Score: 42.3/100
|
||||||
|
|
||||||
🚨 TOP 10 MOST SUSPICIOUS OFFICIALS (By Timing Score)
|
TOP 10 MOST SUSPICIOUS OFFICIALS (By Timing Score)
|
||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
Rank Official Party-State Chamber Trades Suspicious Rate Avg Score
|
Rank Official Party-State Chamber Trades Suspicious Rate Avg Score
|
||||||
──── ─────────────────────── ─────────── ─────── ────── ────────── ────── ─────────
|
──── ─────────────────────── ─────────── ─────── ────── ────────── ────── ─────────
|
||||||
🚨 1 Tommy Tuberville R-AL Senate 47 35/47 74.5% 72.5/100
|
1 Tommy Tuberville R-AL Senate 47 35/47 74.5% 72.5/100
|
||||||
🚨 2 Nancy Pelosi D-CA House 38 28/38 73.7% 71.2/100
|
2 Nancy Pelosi D-CA House 38 28/38 73.7% 71.2/100
|
||||||
🔴 3 Dan Crenshaw R-TX House 25 15/25 60.0% 65.8/100
|
3 Dan Crenshaw R-TX House 25 15/25 60.0% 65.8/100
|
||||||
🔴 4 Marjorie Taylor Greene R-GA House 19 11/19 57.9% 63.2/100
|
4 Marjorie Taylor Greene R-GA House 19 11/19 57.9% 63.2/100
|
||||||
🟡 5 Josh Gottheimer D-NJ House 31 14/31 45.2% 58.7/100
|
5 Josh Gottheimer D-NJ House 31 14/31 45.2% 58.7/100
|
||||||
|
|
||||||
🔥 REPEAT OFFENDERS (50%+ Suspicious Trades)
|
REPEAT OFFENDERS (50%+ Suspicious Trades)
|
||||||
================================================================================
|
================================================================================
|
||||||
|
|
||||||
🚨 Tommy Tuberville (R-AL, Senate)
|
Tommy Tuberville (R-AL, Senate)
|
||||||
Trades: 47 | Suspicious: 35 (74.5%)
|
Trades: 47 | Suspicious: 35 (74.5%)
|
||||||
Avg Timing Score: 72.5/100
|
Avg Timing Score: 72.5/100
|
||||||
Pattern: HIGHLY SUSPICIOUS - Majority of trades show timing advantage
|
Pattern: HIGHLY SUSPICIOUS - Majority of trades show timing advantage
|
||||||
@@ -239,9 +239,9 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📈 **Test Coverage**
|
## **Test Coverage**
|
||||||
|
|
||||||
**Total: 93 tests, all passing ✅**
|
**Total: 93 tests, all passing **
|
||||||
|
|
||||||
- **Phase 1 (Monitoring):** 14 tests
|
- **Phase 1 (Monitoring):** 14 tests
|
||||||
- **Phase 2 (Correlation):** 13 tests
|
- **Phase 2 (Correlation):** 13 tests
|
||||||
@@ -252,7 +252,7 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎯 **Key Insights the System Provides**
|
## **Key Insights the System Provides**
|
||||||
|
|
||||||
### **1. Individual Official Analysis**
|
### **1. Individual Official Analysis**
|
||||||
- Which officials consistently trade before unusual activity?
|
- Which officials consistently trade before unusual activity?
|
||||||
@@ -282,7 +282,7 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔧 **Automated Workflow**
|
## **Automated Workflow**
|
||||||
|
|
||||||
### **Daily Routine (Recommended)**
|
### **Daily Routine (Recommended)**
|
||||||
|
|
||||||
@@ -299,7 +299,7 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📊 **Database Schema**
|
## **Database Schema**
|
||||||
|
|
||||||
**New Table: `market_alerts`**
|
**New Table: `market_alerts`**
|
||||||
```sql
|
```sql
|
||||||
@@ -315,7 +315,7 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎓 **Interpretation Guide**
|
## **Interpretation Guide**
|
||||||
|
|
||||||
### **Timing Scores**
|
### **Timing Scores**
|
||||||
- **80-100:** Highly suspicious - Multiple high-severity alerts before trade
|
- **80-100:** Highly suspicious - Multiple high-severity alerts before trade
|
||||||
@@ -337,15 +337,15 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## ⚠️ **Important Disclaimers**
|
## **Important Disclaimers**
|
||||||
|
|
||||||
### **Legal & Ethical**
|
### **Legal & Ethical**
|
||||||
1. ✅ All data is public and legally obtained
|
1. All data is public and legally obtained
|
||||||
2. ✅ Analysis is retrospective (30-45 day lag)
|
2. Analysis is retrospective (30-45 day lag)
|
||||||
3. ✅ For research and transparency only
|
3. For research and transparency only
|
||||||
4. ❌ NOT investment advice
|
4. NOT investment advice
|
||||||
5. ❌ NOT proof of illegal activity (requires investigation)
|
5. NOT proof of illegal activity (requires investigation)
|
||||||
6. ❌ Statistical patterns ≠ legal evidence
|
6. Statistical patterns ≠ legal evidence
|
||||||
|
|
||||||
### **Technical Limitations**
|
### **Technical Limitations**
|
||||||
1. Cannot identify WHO is trading in real-time
|
1. Cannot identify WHO is trading in real-time
|
||||||
@@ -356,7 +356,7 @@ Rank Official Party-State Chamber Trades Suspicious Rate
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🚀 **Deployment Checklist**
|
## **Deployment Checklist**
|
||||||
|
|
||||||
### **On Proxmox Container**
|
### **On Proxmox Container**
|
||||||
|
|
||||||
@@ -383,7 +383,7 @@ python scripts/generate_pattern_report.py --days 365
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 📚 **Documentation**
|
## **Documentation**
|
||||||
|
|
||||||
- **`docs/11_live_market_monitoring.md`** - Deep dive into monitoring
|
- **`docs/11_live_market_monitoring.md`** - Deep dive into monitoring
|
||||||
- **`LOCAL_TEST_GUIDE.md`** - Testing instructions
|
- **`LOCAL_TEST_GUIDE.md`** - Testing instructions
|
||||||
@@ -392,23 +392,23 @@ python scripts/generate_pattern_report.py --days 365
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🎉 **Achievement Unlocked!**
|
## **Achievement Unlocked!**
|
||||||
|
|
||||||
**You now have a complete system that:**
|
**You now have a complete system that:**
|
||||||
|
|
||||||
✅ Monitors real-time market activity
|
Monitors real-time market activity
|
||||||
✅ Correlates trades to prior alerts
|
Correlates trades to prior alerts
|
||||||
✅ Calculates timing advantage scores
|
Calculates timing advantage scores
|
||||||
✅ Identifies repeat offenders
|
Identifies repeat offenders
|
||||||
✅ Ranks officials by suspicion
|
Ranks officials by suspicion
|
||||||
✅ Generates comprehensive reports
|
Generates comprehensive reports
|
||||||
✅ 93 tests confirming it works
|
93 tests confirming it works
|
||||||
|
|
||||||
**This is a production-ready transparency and research tool!** 🚀
|
**This is a production-ready transparency and research tool!**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 🔜 **Potential Future Enhancements**
|
## **Potential Future Enhancements**
|
||||||
|
|
||||||
### **Phase 4 Ideas (Optional)**
|
### **Phase 4 Ideas (Optional)**
|
||||||
- Email/SMS alerts for high-severity patterns
|
- Email/SMS alerts for high-severity patterns
|
||||||
@@ -420,6 +420,6 @@ python scripts/generate_pattern_report.py --days 365
|
|||||||
- Automated PDF reports
|
- Automated PDF reports
|
||||||
- Historical performance tracking
|
- Historical performance tracking
|
||||||
|
|
||||||
**But the core system is COMPLETE and FUNCTIONAL now!** ✅
|
**But the core system is COMPLETE and FUNCTIONAL now!**
|
||||||
|
|
||||||
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# PR1 Summary: Project Scaffold + DB + Price Loader
|
# PR1 Summary: Project Scaffold + DB + Price Loader
|
||||||
|
|
||||||
**Status**: ✅ Complete
|
**Status**: Complete
|
||||||
**Date**: 2025-12-13
|
**Date**: 2025-12-13
|
||||||
|
|
||||||
## What was built
|
## What was built
|
||||||
@@ -36,7 +36,7 @@ Includes proper indexes, unique constraints, and relationships.
|
|||||||
- `tests/conftest.py`: fixtures for in-memory DB, sample officials/securities/trades/prices
|
- `tests/conftest.py`: fixtures for in-memory DB, sample officials/securities/trades/prices
|
||||||
- `tests/test_models.py`: model creation, relationships, unique constraints, queries (7 tests)
|
- `tests/test_models.py`: model creation, relationships, unique constraints, queries (7 tests)
|
||||||
- `tests/test_price_loader.py`: loader logic, idempotency, upsert, mocking yfinance (8 tests)
|
- `tests/test_price_loader.py`: loader logic, idempotency, upsert, mocking yfinance (8 tests)
|
||||||
- **Result**: 15 tests, all passing ✅
|
- **Result**: 15 tests, all passing
|
||||||
|
|
||||||
### 6. Tooling
|
### 6. Tooling
|
||||||
- **Black** + **ruff** configured and run (all code formatted + linted)
|
- **Black** + **ruff** configured and run (all code formatted + linted)
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# PR2 Summary: Congressional Trade Ingestion
|
# PR2 Summary: Congressional Trade Ingestion
|
||||||
|
|
||||||
**Status**: ✅ Complete
|
**Status**: Complete
|
||||||
**Date**: 2025-12-14
|
**Date**: 2025-12-14
|
||||||
|
|
||||||
## What was built
|
## What was built
|
||||||
@@ -26,7 +26,7 @@
|
|||||||
- `tests/fixtures/sample_house_watcher.json`: 5 realistic sample transactions
|
- `tests/fixtures/sample_house_watcher.json`: 5 realistic sample transactions
|
||||||
- Includes House + Senate, Democrats + Republicans, various tickers
|
- Includes House + Senate, Democrats + Republicans, various tickers
|
||||||
|
|
||||||
### 4. Tests (13 new tests, all passing ✅)
|
### 4. Tests (13 new tests, all passing )
|
||||||
**`tests/test_house_watcher.py` (8 tests)**:
|
**`tests/test_house_watcher.py` (8 tests)**:
|
||||||
- Amount range parsing (with range, single value, invalid)
|
- Amount range parsing (with range, single value, invalid)
|
||||||
- Transaction type normalization
|
- Transaction type normalization
|
||||||
@@ -58,9 +58,9 @@
|
|||||||
python scripts/fetch_congressional_trades.py --days 30
|
python scripts/fetch_congressional_trades.py --days 30
|
||||||
|
|
||||||
# Sample output:
|
# Sample output:
|
||||||
# ✓ Officials created/updated: 47
|
# Officials created/updated: 47
|
||||||
# ✓ Securities created/updated: 89
|
# Securities created/updated: 89
|
||||||
# ✓ Trades ingested: 234
|
# Trades ingested: 234
|
||||||
```
|
```
|
||||||
|
|
||||||
### Database Queries
|
### Database Queries
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# PR3 Summary: Security Enrichment + Deployment
|
# PR3 Summary: Security Enrichment + Deployment
|
||||||
|
|
||||||
**Status**: ✅ Complete
|
**Status**: Complete
|
||||||
**Date**: 2025-12-14
|
**Date**: 2025-12-14
|
||||||
|
|
||||||
## What was built
|
## What was built
|
||||||
@@ -32,7 +32,7 @@
|
|||||||
python scripts/enrich_securities.py --force
|
python scripts/enrich_securities.py --force
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Tests (9 new tests, all passing ✅)
|
### 3. Tests (9 new tests, all passing )
|
||||||
**`tests/test_security_enricher.py`**:
|
**`tests/test_security_enricher.py`**:
|
||||||
- Successful enrichment with complete data
|
- Successful enrichment with complete data
|
||||||
- ETF detection and classification
|
- ETF detection and classification
|
||||||
@@ -72,7 +72,7 @@ python scripts/enrich_securities.py
|
|||||||
# Enriched AAPL: Apple Inc. (Technology)
|
# Enriched AAPL: Apple Inc. (Technology)
|
||||||
# Enriched TSLA: Tesla, Inc. (Consumer Cyclical)
|
# Enriched TSLA: Tesla, Inc. (Consumer Cyclical)
|
||||||
# Enriched GOOGL: Alphabet Inc. (Communication Services)
|
# Enriched GOOGL: Alphabet Inc. (Communication Services)
|
||||||
# ✓ Successfully enriched: 5
|
# Successfully enriched: 5
|
||||||
```
|
```
|
||||||
|
|
||||||
### Query Enriched Data
|
### Query Enriched Data
|
||||||
@@ -159,10 +159,10 @@ python scripts/update_all_prices.py # To be built in PR4
|
|||||||
|
|
||||||
| Option | Complexity | Cost/month | Best For |
|
| Option | Complexity | Cost/month | Best For |
|
||||||
|--------|-----------|------------|----------|
|
|--------|-----------|------------|----------|
|
||||||
| **Local** | ⭐ | $0 | Development |
|
| **Local** | | $0 | Development |
|
||||||
| **VPS + Docker** | ⭐⭐ | $10-20 | Personal deployment |
|
| **VPS + Docker** | | $10-20 | Personal deployment |
|
||||||
| **Railway/Fly.io** | ⭐ | $5-15 | Easy cloud |
|
| **Railway/Fly.io** | | $5-15 | Easy cloud |
|
||||||
| **AWS** | ⭐⭐⭐ | $20-50 | Scalable production |
|
| **AWS** | | $20-50 | Scalable production |
|
||||||
|
|
||||||
See [`docs/07_deployment.md`](07_deployment.md) for detailed guides.
|
See [`docs/07_deployment.md`](07_deployment.md) for detailed guides.
|
||||||
|
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# PR4 Summary: Phase 2 Analytics Foundation
|
# PR4 Summary: Phase 2 Analytics Foundation
|
||||||
|
|
||||||
## ✅ Completed
|
## Completed
|
||||||
|
|
||||||
**Date**: December 15, 2025
|
**Date**: December 15, 2025
|
||||||
**Status**: Complete
|
**Status**: Complete
|
||||||
@@ -78,14 +78,14 @@ python scripts/calculate_all_returns.py --window 90 --benchmark SPY --top 10
|
|||||||
|
|
||||||
### 3. Tests (`tests/test_analytics.py`)
|
### 3. Tests (`tests/test_analytics.py`)
|
||||||
|
|
||||||
- ✅ Return calculator with sample data
|
- Return calculator with sample data
|
||||||
- ✅ Buy vs sell trade handling
|
- Buy vs sell trade handling
|
||||||
- ✅ Missing data edge cases
|
- Missing data edge cases
|
||||||
- ✅ Benchmark comparisons
|
- Benchmark comparisons
|
||||||
- ✅ Official performance metrics
|
- Official performance metrics
|
||||||
- ✅ Multiple time windows
|
- Multiple time windows
|
||||||
- ✅ Sector analysis
|
- Sector analysis
|
||||||
- ✅ Timing analysis
|
- Timing analysis
|
||||||
|
|
||||||
**Test Coverage**: Analytics module fully tested
|
**Test Coverage**: Analytics module fully tested
|
||||||
|
|
||||||
@@ -288,15 +288,15 @@ with next(get_session()) as session:
|
|||||||
- Trades near policy events
|
- Trades near policy events
|
||||||
- Unusual timing flags
|
- Unusual timing flags
|
||||||
|
|
||||||
## Success Criteria ✅
|
## Success Criteria
|
||||||
|
|
||||||
- ✅ Can calculate returns for any trade + window
|
- Can calculate returns for any trade + window
|
||||||
- ✅ Can compare to S&P 500 benchmark
|
- Can compare to S&P 500 benchmark
|
||||||
- ✅ Can generate official performance summaries
|
- Can generate official performance summaries
|
||||||
- ✅ All calculations tested and accurate
|
- All calculations tested and accurate
|
||||||
- ✅ Performance data calculated on-the-fly
|
- Performance data calculated on-the-fly
|
||||||
- ✅ Documentation complete
|
- Documentation complete
|
||||||
- ✅ Command-line tools working
|
- Command-line tools working
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
@@ -309,7 +309,7 @@ All analytics tests should pass (may have warnings if no price data).
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
**Phase 2 Analytics Foundation: COMPLETE** ✅
|
**Phase 2 Analytics Foundation: COMPLETE**
|
||||||
**Ready for**: PR5 (Signals), PR6 (API), PR7 (Dashboard)
|
**Ready for**: PR5 (Signals), PR6 (API), PR7 (Dashboard)
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
# Archive
|
||||||
|
|
||||||
|
PR summaries and one-shot status writeups. Prefer the numbered guides and QUICKSTART files.
|
||||||
@@ -41,6 +41,8 @@ where = ["src"]
|
|||||||
[tool.ruff]
|
[tool.ruff]
|
||||||
line-length = 100
|
line-length = 100
|
||||||
target-version = "py311"
|
target-version = "py311"
|
||||||
|
|
||||||
|
[tool.ruff.lint]
|
||||||
select = ["E", "F", "W", "I", "N", "UP", "B", "A", "C4", "SIM", "RET"]
|
select = ["E", "F", "W", "I", "N", "UP", "B", "A", "C4", "SIM", "RET"]
|
||||||
ignore = ["E501"] # Line too long (handled by black)
|
ignore = ["E501"] # Line too long (handled by black)
|
||||||
|
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ def main(tickers, interval, once, min_severity, save_report, lookback):
|
|||||||
|
|
||||||
if filtered:
|
if filtered:
|
||||||
# Generate report
|
# Generate report
|
||||||
report = alert_mgr.generate_summary_report(filtered, format="text")
|
report = alert_mgr.generate_summary_report(filtered, output_format="text")
|
||||||
print("\n" + report)
|
print("\n" + report)
|
||||||
|
|
||||||
# Save report if requested
|
# Save report if requested
|
||||||
|
|||||||
@@ -2,14 +2,12 @@
|
|||||||
Analytics module for calculating returns, performance metrics, and signals.
|
Analytics module for calculating returns, performance metrics, and signals.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from .returns import ReturnCalculator
|
|
||||||
from .benchmarks import BenchmarkComparison
|
from .benchmarks import BenchmarkComparison
|
||||||
from .metrics import PerformanceMetrics
|
from .metrics import PerformanceMetrics
|
||||||
|
from .returns import ReturnCalculator
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"ReturnCalculator",
|
"ReturnCalculator",
|
||||||
"BenchmarkComparison",
|
"BenchmarkComparison",
|
||||||
"PerformanceMetrics",
|
"PerformanceMetrics",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ Benchmark comparison for calculating abnormal returns (alpha).
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import date, timedelta
|
from datetime import date
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -60,8 +60,7 @@ class BenchmarkComparison:
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
# Calculate return
|
# Calculate return
|
||||||
return_pct = ((end_price - start_price) / start_price) * 100
|
return ((end_price - start_price) / start_price) * 100
|
||||||
return return_pct
|
|
||||||
|
|
||||||
def calculate_abnormal_return(
|
def calculate_abnormal_return(
|
||||||
self,
|
self,
|
||||||
@@ -219,5 +218,3 @@ class BenchmarkComparison:
|
|||||||
"benchmark": self.BENCHMARKS.get(benchmark, benchmark),
|
"benchmark": self.BENCHMARKS.get(benchmark, benchmark),
|
||||||
"window_days": window_days,
|
"window_days": window_days,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ Performance metrics and aggregations.
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
from collections import defaultdict
|
from collections import defaultdict
|
||||||
from datetime import date
|
|
||||||
|
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -52,11 +51,7 @@ class PerformanceMetrics:
|
|||||||
if not official:
|
if not official:
|
||||||
return {"error": "Official not found"}
|
return {"error": "Official not found"}
|
||||||
|
|
||||||
trades = (
|
trades = self.session.query(Trade).filter(Trade.official_id == official_id).all()
|
||||||
self.session.query(Trade)
|
|
||||||
.filter(Trade.official_id == official_id)
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
|
|
||||||
if not trades:
|
if not trades:
|
||||||
return {
|
return {
|
||||||
@@ -70,9 +65,7 @@ class PerformanceMetrics:
|
|||||||
# Calculate returns for all trades
|
# Calculate returns for all trades
|
||||||
returns_data = []
|
returns_data = []
|
||||||
for trade in trades:
|
for trade in trades:
|
||||||
result = self.benchmark.compare_trade_to_benchmark(
|
result = self.benchmark.compare_trade_to_benchmark(trade, window_days, benchmark)
|
||||||
trade, window_days, benchmark
|
|
||||||
)
|
|
||||||
if result:
|
if result:
|
||||||
returns_data.append(result)
|
returns_data.append(result)
|
||||||
|
|
||||||
@@ -96,9 +89,7 @@ class PerformanceMetrics:
|
|||||||
worst_trade = min(returns_data, key=lambda x: x["trade_return"])
|
worst_trade = min(returns_data, key=lambda x: x["trade_return"])
|
||||||
|
|
||||||
# Total value traded
|
# Total value traded
|
||||||
total_value = sum(
|
total_value = sum(float(t.value_min or 0) for t in trades if t.value_min)
|
||||||
float(t.value_min or 0) for t in trades if t.value_min
|
|
||||||
)
|
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"name": official.name,
|
"name": official.name,
|
||||||
@@ -154,20 +145,14 @@ class PerformanceMetrics:
|
|||||||
List of sector performance dictionaries
|
List of sector performance dictionaries
|
||||||
"""
|
"""
|
||||||
# Get all trades with security info
|
# Get all trades with security info
|
||||||
trades = (
|
trades = self.session.query(Trade).join(Security).all()
|
||||||
self.session.query(Trade)
|
|
||||||
.join(Security)
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
|
|
||||||
# Group by sector
|
# Group by sector
|
||||||
sector_data = defaultdict(list)
|
sector_data = defaultdict(list)
|
||||||
|
|
||||||
for trade in trades:
|
for trade in trades:
|
||||||
sector = trade.security.sector or "Unknown"
|
sector = trade.security.sector or "Unknown"
|
||||||
result = self.benchmark.compare_trade_to_benchmark(
|
result = self.benchmark.compare_trade_to_benchmark(trade, window_days, benchmark)
|
||||||
trade, window_days, benchmark
|
|
||||||
)
|
|
||||||
if result:
|
if result:
|
||||||
sector_data[sector].append(result)
|
sector_data[sector].append(result)
|
||||||
|
|
||||||
@@ -180,14 +165,16 @@ class PerformanceMetrics:
|
|||||||
returns = [d["trade_return"] for d in data]
|
returns = [d["trade_return"] for d in data]
|
||||||
alphas = [d["abnormal_return"] for d in data]
|
alphas = [d["abnormal_return"] for d in data]
|
||||||
|
|
||||||
results.append({
|
results.append(
|
||||||
|
{
|
||||||
"sector": sector,
|
"sector": sector,
|
||||||
"trade_count": len(data),
|
"trade_count": len(data),
|
||||||
"avg_return": sum(returns) / len(returns),
|
"avg_return": sum(returns) / len(returns),
|
||||||
"avg_alpha": sum(alphas) / len(alphas),
|
"avg_alpha": sum(alphas) / len(alphas),
|
||||||
"win_rate": sum(1 for r in returns if r > 0) / len(returns),
|
"win_rate": sum(1 for r in returns if r > 0) / len(returns),
|
||||||
"beat_market_rate": sum(1 for a in alphas if a > 0) / len(alphas),
|
"beat_market_rate": sum(1 for a in alphas if a > 0) / len(alphas),
|
||||||
})
|
}
|
||||||
|
)
|
||||||
|
|
||||||
# Sort by average alpha
|
# Sort by average alpha
|
||||||
results.sort(key=lambda x: x["avg_alpha"], reverse=True)
|
results.sort(key=lambda x: x["avg_alpha"], reverse=True)
|
||||||
@@ -229,11 +216,7 @@ class PerformanceMetrics:
|
|||||||
Returns:
|
Returns:
|
||||||
Dictionary with timing statistics
|
Dictionary with timing statistics
|
||||||
"""
|
"""
|
||||||
trades = (
|
trades = self.session.query(Trade).filter(Trade.filing_date.isnot(None)).all()
|
||||||
self.session.query(Trade)
|
|
||||||
.filter(Trade.filing_date.isnot(None))
|
|
||||||
.all()
|
|
||||||
)
|
|
||||||
|
|
||||||
if not trades:
|
if not trades:
|
||||||
return {"error": "No trades with disclosure dates"}
|
return {"error": "No trades with disclosure dates"}
|
||||||
@@ -288,5 +271,3 @@ class PerformanceMetrics:
|
|||||||
"benchmark": benchmark,
|
"benchmark": benchmark,
|
||||||
**aggregate,
|
**aggregate,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -94,18 +94,20 @@ class ReturnCalculator:
|
|||||||
def calculate_multiple_windows(
|
def calculate_multiple_windows(
|
||||||
self,
|
self,
|
||||||
trade: Trade,
|
trade: Trade,
|
||||||
windows: list[int] = [30, 60, 90, 180],
|
windows: list[int] | None = None,
|
||||||
) -> dict[int, dict]:
|
) -> dict[int, dict]:
|
||||||
"""
|
"""
|
||||||
Calculate returns for multiple time windows.
|
Calculate returns for multiple time windows.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
trade: Trade object
|
trade: Trade object
|
||||||
windows: List of window sizes in days
|
windows: List of window sizes in days (defaults to 30/60/90/180)
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Dictionary mapping window_days to return metrics
|
Dictionary mapping window_days to return metrics
|
||||||
"""
|
"""
|
||||||
|
if windows is None:
|
||||||
|
windows = [30, 60, 90, 180]
|
||||||
results = {}
|
results = {}
|
||||||
for window in windows:
|
for window in windows:
|
||||||
result = self.calculate_trade_return(trade, window)
|
result = self.calculate_trade_return(trade, window)
|
||||||
@@ -223,12 +225,13 @@ class ReturnCalculator:
|
|||||||
if not prices:
|
if not prices:
|
||||||
return pd.DataFrame()
|
return pd.DataFrame()
|
||||||
|
|
||||||
|
# open/high/low are nullable columns; use NaN (pandas-native) when absent.
|
||||||
data = [
|
data = [
|
||||||
{
|
{
|
||||||
"date": p.date,
|
"date": p.date,
|
||||||
"open": float(p.open),
|
"open": float(p.open) if p.open is not None else float("nan"),
|
||||||
"high": float(p.high),
|
"high": float(p.high) if p.high is not None else float("nan"),
|
||||||
"low": float(p.low),
|
"low": float(p.low) if p.low is not None else float("nan"),
|
||||||
"close": float(p.close),
|
"close": float(p.close),
|
||||||
"volume": p.volume,
|
"volume": p.volume,
|
||||||
}
|
}
|
||||||
@@ -236,4 +239,3 @@ class ReturnCalculator:
|
|||||||
]
|
]
|
||||||
|
|
||||||
return pd.DataFrame(data)
|
return pd.DataFrame(data)
|
||||||
|
|
||||||
|
|||||||
+16
-32
@@ -3,17 +3,17 @@ SQLAlchemy ORM models for POTE.
|
|||||||
Matches the schema defined in docs/02_data_model.md.
|
Matches the schema defined in docs/02_data_model.md.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from datetime import date, datetime, timezone
|
from datetime import UTC, date, datetime
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from sqlalchemy import (
|
from sqlalchemy import (
|
||||||
DECIMAL,
|
DECIMAL,
|
||||||
|
JSON,
|
||||||
Date,
|
Date,
|
||||||
DateTime,
|
DateTime,
|
||||||
ForeignKey,
|
ForeignKey,
|
||||||
Index,
|
Index,
|
||||||
Integer,
|
Integer,
|
||||||
JSON,
|
|
||||||
String,
|
String,
|
||||||
Text,
|
Text,
|
||||||
UniqueConstraint,
|
UniqueConstraint,
|
||||||
@@ -35,13 +35,11 @@ class Official(Base):
|
|||||||
state: Mapped[str | None] = mapped_column(String(2))
|
state: Mapped[str | None] = mapped_column(String(2))
|
||||||
bioguide_id: Mapped[str | None] = mapped_column(String(20), unique=True)
|
bioguide_id: Mapped[str | None] = mapped_column(String(20), unique=True)
|
||||||
external_ids: Mapped[str | None] = mapped_column(Text) # JSON blob for other IDs
|
external_ids: Mapped[str | None] = mapped_column(Text) # JSON blob for other IDs
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
updated_at: Mapped[datetime] = mapped_column(
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime,
|
DateTime,
|
||||||
default=lambda: datetime.now(timezone.utc),
|
default=lambda: datetime.now(UTC),
|
||||||
onupdate=lambda: datetime.now(timezone.utc),
|
onupdate=lambda: datetime.now(UTC),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Relationships
|
# Relationships
|
||||||
@@ -63,13 +61,11 @@ class Security(Base):
|
|||||||
sector: Mapped[str | None] = mapped_column(String(100))
|
sector: Mapped[str | None] = mapped_column(String(100))
|
||||||
industry: Mapped[str | None] = mapped_column(String(100))
|
industry: Mapped[str | None] = mapped_column(String(100))
|
||||||
asset_type: Mapped[str] = mapped_column(String(50), default="stock") # stock, bond, etc.
|
asset_type: Mapped[str] = mapped_column(String(50), default="stock") # stock, bond, etc.
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
updated_at: Mapped[datetime] = mapped_column(
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime,
|
DateTime,
|
||||||
default=lambda: datetime.now(timezone.utc),
|
default=lambda: datetime.now(UTC),
|
||||||
onupdate=lambda: datetime.now(timezone.utc),
|
onupdate=lambda: datetime.now(UTC),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Relationships
|
# Relationships
|
||||||
@@ -107,13 +103,11 @@ class Trade(Base):
|
|||||||
# Quality flags (JSON or enum list)
|
# Quality flags (JSON or enum list)
|
||||||
quality_flags: Mapped[str | None] = mapped_column(Text) # e.g., "range_only,delayed_filing"
|
quality_flags: Mapped[str | None] = mapped_column(Text) # e.g., "range_only,delayed_filing"
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
updated_at: Mapped[datetime] = mapped_column(
|
updated_at: Mapped[datetime] = mapped_column(
|
||||||
DateTime,
|
DateTime,
|
||||||
default=lambda: datetime.now(timezone.utc),
|
default=lambda: datetime.now(UTC),
|
||||||
onupdate=lambda: datetime.now(timezone.utc),
|
onupdate=lambda: datetime.now(UTC),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Relationships
|
# Relationships
|
||||||
@@ -156,9 +150,7 @@ class Price(Base):
|
|||||||
adjusted_close: Mapped[Decimal | None] = mapped_column(DECIMAL(15, 4))
|
adjusted_close: Mapped[Decimal | None] = mapped_column(DECIMAL(15, 4))
|
||||||
|
|
||||||
source: Mapped[str] = mapped_column(String(50), default="yfinance")
|
source: Mapped[str] = mapped_column(String(50), default="yfinance")
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
|
|
||||||
# Relationships
|
# Relationships
|
||||||
security: Mapped["Security"] = relationship("Security", back_populates="prices")
|
security: Mapped["Security"] = relationship("Security", back_populates="prices")
|
||||||
@@ -188,9 +180,7 @@ class MetricOfficial(Base):
|
|||||||
avg_abnormal_return_1m: Mapped[Decimal | None] = mapped_column(DECIMAL(10, 6))
|
avg_abnormal_return_1m: Mapped[Decimal | None] = mapped_column(DECIMAL(10, 6))
|
||||||
cluster_label: Mapped[str | None] = mapped_column(String(50))
|
cluster_label: Mapped[str | None] = mapped_column(String(50))
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
|
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
UniqueConstraint("official_id", "calc_date", "calc_version", name="uq_metrics_official"),
|
UniqueConstraint("official_id", "calc_date", "calc_version", name="uq_metrics_official"),
|
||||||
@@ -212,9 +202,7 @@ class MetricTrade(Base):
|
|||||||
abnormal_return_1m: Mapped[Decimal | None] = mapped_column(DECIMAL(10, 6))
|
abnormal_return_1m: Mapped[Decimal | None] = mapped_column(DECIMAL(10, 6))
|
||||||
signal_flags: Mapped[str | None] = mapped_column(Text) # JSON list
|
signal_flags: Mapped[str | None] = mapped_column(Text) # JSON list
|
||||||
|
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
|
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
UniqueConstraint("trade_id", "calc_date", "calc_version", name="uq_metrics_trade"),
|
UniqueConstraint("trade_id", "calc_date", "calc_version", name="uq_metrics_trade"),
|
||||||
@@ -242,18 +230,14 @@ class MarketAlert(Base):
|
|||||||
# Metrics at time of alert
|
# Metrics at time of alert
|
||||||
price: Mapped[Decimal | None] = mapped_column(DECIMAL(15, 4))
|
price: Mapped[Decimal | None] = mapped_column(DECIMAL(15, 4))
|
||||||
volume: Mapped[int | None] = mapped_column(Integer)
|
volume: Mapped[int | None] = mapped_column(Integer)
|
||||||
change_pct: Mapped[Decimal | None] = mapped_column(
|
change_pct: Mapped[Decimal | None] = mapped_column(DECIMAL(10, 4)) # Price change %
|
||||||
DECIMAL(10, 4)
|
|
||||||
) # Price change %
|
|
||||||
|
|
||||||
# Severity scoring
|
# Severity scoring
|
||||||
severity: Mapped[int | None] = mapped_column(Integer) # 1-10 scale
|
severity: Mapped[int | None] = mapped_column(Integer) # 1-10 scale
|
||||||
|
|
||||||
# Metadata
|
# Metadata
|
||||||
source: Mapped[str] = mapped_column(String(50), default="market_monitor")
|
source: Mapped[str] = mapped_column(String(50), default="market_monitor")
|
||||||
created_at: Mapped[datetime] = mapped_column(
|
created_at: Mapped[datetime] = mapped_column(DateTime, default=lambda: datetime.now(UTC))
|
||||||
DateTime, default=lambda: datetime.now(timezone.utc)
|
|
||||||
)
|
|
||||||
|
|
||||||
# Indexes for efficient queries
|
# Indexes for efficient queries
|
||||||
__table_args__ = (
|
__table_args__ = (
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import httpx
|
|||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _default_data_urls() -> tuple[str, ...]:
|
def _default_data_urls() -> tuple[str, ...]:
|
||||||
override = os.environ.get("POTE_HOUSE_DATA_URL", "").strip()
|
override = os.environ.get("POTE_HOUSE_DATA_URL", "").strip()
|
||||||
if override:
|
if override:
|
||||||
@@ -235,10 +236,9 @@ def normalize_transaction_type(txn_type: str) -> str:
|
|||||||
|
|
||||||
if "purchase" in txn_lower or "buy" in txn_lower:
|
if "purchase" in txn_lower or "buy" in txn_lower:
|
||||||
return "buy"
|
return "buy"
|
||||||
elif "sale" in txn_lower or "sell" in txn_lower:
|
if "sale" in txn_lower or "sell" in txn_lower:
|
||||||
return "sell"
|
return "sell"
|
||||||
elif "exchange" in txn_lower:
|
if "exchange" in txn_lower:
|
||||||
return "exchange"
|
return "exchange"
|
||||||
else:
|
|
||||||
# Default to the original, lowercased
|
# Default to the original, lowercased
|
||||||
return txn_lower
|
return txn_lower
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ Fetches daily OHLCV data for securities and stores in the prices table.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import date, datetime, timedelta, timezone
|
from datetime import UTC, date, datetime, timedelta
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
@@ -158,7 +158,7 @@ class PriceLoader:
|
|||||||
"volume": int(row["volume"]) if pd.notna(row.get("volume")) else None,
|
"volume": int(row["volume"]) if pd.notna(row.get("volume")) else None,
|
||||||
"adjusted_close": None, # We'll compute this later if needed
|
"adjusted_close": None, # We'll compute this later if needed
|
||||||
"source": "yfinance",
|
"source": "yfinance",
|
||||||
"created_at": datetime.now(timezone.utc),
|
"created_at": datetime.now(UTC),
|
||||||
}
|
}
|
||||||
records.append(record)
|
records.append(record)
|
||||||
|
|
||||||
|
|||||||
@@ -9,4 +9,3 @@ from .market_monitor import MarketMonitor
|
|||||||
from .pattern_detector import PatternDetector
|
from .pattern_detector import PatternDetector
|
||||||
|
|
||||||
__all__ = ["MarketMonitor", "AlertManager", "DisclosureCorrelator", "PatternDetector"]
|
__all__ = ["MarketMonitor", "AlertManager", "DisclosureCorrelator", "PatternDetector"]
|
||||||
|
|
||||||
|
|||||||
@@ -4,8 +4,7 @@ Handles alert filtering, formatting, and delivery.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import datetime, timezone
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
@@ -75,21 +74,24 @@ class AlertManager:
|
|||||||
Returns:
|
Returns:
|
||||||
HTML formatted alert
|
HTML formatted alert
|
||||||
"""
|
"""
|
||||||
severity_class = "high" if (alert.severity or 0) >= 7 else "medium" if (alert.severity or 0) >= 4 else "low"
|
severity_class = (
|
||||||
|
"high"
|
||||||
|
if (alert.severity or 0) >= 7
|
||||||
|
else "medium" if (alert.severity or 0) >= 4 else "low"
|
||||||
|
)
|
||||||
|
|
||||||
html = f"""
|
return f"""
|
||||||
<div class="alert {severity_class}">
|
<div class="alert {severity_class}">
|
||||||
<h3>{alert.ticker} - {alert.alert_type.replace('_', ' ').title()}</h3>
|
<h3>{alert.ticker} - {alert.alert_type.replace('_', ' ').title()}</h3>
|
||||||
<p class="timestamp">{alert.timestamp.strftime('%Y-%m-%d %H:%M:%S')}</p>
|
<p class="timestamp">{alert.timestamp.strftime('%Y-%m-%d %H:%M:%S')}</p>
|
||||||
<p class="severity">Severity: {alert.severity}/10</p>
|
<p class="severity">Severity: {alert.severity}/10</p>
|
||||||
<div class="metrics">
|
<div class="metrics">
|
||||||
<span>Price: ${float(alert.price):.2f}</span>
|
<span>Price: ${float(alert.price or 0):.2f}</span>
|
||||||
<span>Volume: {alert.volume:,}</span>
|
<span>Volume: {alert.volume:,}</span>
|
||||||
<span>Change: {float(alert.change_pct):+.2f}%</span>
|
<span>Change: {float(alert.change_pct or 0):+.2f}%</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
"""
|
"""
|
||||||
return html
|
|
||||||
|
|
||||||
def filter_alerts(
|
def filter_alerts(
|
||||||
self,
|
self,
|
||||||
@@ -117,7 +119,7 @@ class AlertManager:
|
|||||||
|
|
||||||
# Filter by ticker
|
# Filter by ticker
|
||||||
if tickers:
|
if tickers:
|
||||||
ticker_set = set(t.upper() for t in tickers)
|
ticker_set = {t.upper() for t in tickers}
|
||||||
filtered = [a for a in filtered if a.ticker.upper() in ticker_set]
|
filtered = [a for a in filtered if a.ticker.upper() in ticker_set]
|
||||||
|
|
||||||
# Filter by alert type
|
# Filter by alert type
|
||||||
@@ -128,21 +130,20 @@ class AlertManager:
|
|||||||
return filtered
|
return filtered
|
||||||
|
|
||||||
def generate_summary_report(
|
def generate_summary_report(
|
||||||
self, alerts: list[MarketAlert], format: str = "text"
|
self, alerts: list[MarketAlert], output_format: str = "text"
|
||||||
) -> str:
|
) -> str:
|
||||||
"""
|
"""
|
||||||
Generate summary report of alerts.
|
Generate summary report of alerts.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
alerts: List of alerts
|
alerts: List of alerts
|
||||||
format: Output format ('text' or 'html')
|
output_format: Output format ('text' or 'html')
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
Formatted summary report
|
Formatted summary report
|
||||||
"""
|
"""
|
||||||
if format == "html":
|
if output_format == "html":
|
||||||
return self._generate_html_summary(alerts)
|
return self._generate_html_summary(alerts)
|
||||||
else:
|
|
||||||
return self._generate_text_summary(alerts)
|
return self._generate_text_summary(alerts)
|
||||||
|
|
||||||
def _generate_text_summary(self, alerts: list[MarketAlert]) -> str:
|
def _generate_text_summary(self, alerts: list[MarketAlert]) -> str:
|
||||||
@@ -152,7 +153,7 @@ class AlertManager:
|
|||||||
|
|
||||||
lines = [
|
lines = [
|
||||||
"=" * 80,
|
"=" * 80,
|
||||||
f" MARKET ACTIVITY ALERTS - {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')} UTC",
|
f" MARKET ACTIVITY ALERTS - {datetime.now(UTC).strftime('%Y-%m-%d %H:%M:%S')} UTC",
|
||||||
f" {len(alerts)} Alerts",
|
f" {len(alerts)} Alerts",
|
||||||
"=" * 80,
|
"=" * 80,
|
||||||
"",
|
"",
|
||||||
@@ -180,9 +181,7 @@ class AlertManager:
|
|||||||
lines.append(f"🎯 {ticker} - {len(ticker_alerts)} alerts (Max Severity: {max_sev}/10)")
|
lines.append(f"🎯 {ticker} - {len(ticker_alerts)} alerts (Max Severity: {max_sev}/10)")
|
||||||
lines.append("─" * 80)
|
lines.append("─" * 80)
|
||||||
|
|
||||||
for alert in sorted(
|
for alert in sorted(ticker_alerts, key=lambda a: a.severity or 0, reverse=True):
|
||||||
ticker_alerts, key=lambda a: a.severity or 0, reverse=True
|
|
||||||
):
|
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append(self.format_alert_text(alert))
|
lines.append(self.format_alert_text(alert))
|
||||||
|
|
||||||
@@ -202,9 +201,7 @@ class AlertManager:
|
|||||||
type_counts[alert.alert_type] = type_counts.get(alert.alert_type, 0) + 1
|
type_counts[alert.alert_type] = type_counts.get(alert.alert_type, 0) + 1
|
||||||
|
|
||||||
lines.append("\nAlert Types:")
|
lines.append("\nAlert Types:")
|
||||||
for alert_type, count in sorted(
|
for alert_type, count in sorted(type_counts.items(), key=lambda x: x[1], reverse=True):
|
||||||
type_counts.items(), key=lambda x: x[1], reverse=True
|
|
||||||
):
|
|
||||||
lines.append(f" {alert_type.replace('_', ' ').title():20s}: {count}")
|
lines.append(f" {alert_type.replace('_', ' ').title():20s}: {count}")
|
||||||
|
|
||||||
# Top severity alerts
|
# Top severity alerts
|
||||||
@@ -232,8 +229,8 @@ class AlertManager:
|
|||||||
".timestamp { color: #666; font-size: 0.9em; }",
|
".timestamp { color: #666; font-size: 0.9em; }",
|
||||||
".metrics span { margin-right: 20px; }",
|
".metrics span { margin-right: 20px; }",
|
||||||
"</style></head><body>",
|
"</style></head><body>",
|
||||||
f"<h1>Market Activity Alerts</h1>",
|
"<h1>Market Activity Alerts</h1>",
|
||||||
f"<p><strong>{len(alerts)} Alerts</strong> | {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M:%S')} UTC</p>",
|
f"<p><strong>{len(alerts)} Alerts</strong> | {datetime.now(UTC).strftime('%Y-%m-%d %H:%M:%S')} UTC</p>",
|
||||||
]
|
]
|
||||||
|
|
||||||
for alert in sorted(alerts, key=lambda a: a.severity or 0, reverse=True):
|
for alert in sorted(alerts, key=lambda a: a.severity or 0, reverse=True):
|
||||||
@@ -241,5 +238,3 @@ class AlertManager:
|
|||||||
|
|
||||||
html_parts.append("</body></html>")
|
html_parts.append("</body></html>")
|
||||||
return "\n".join(html_parts)
|
return "\n".join(html_parts)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,14 +5,13 @@ Calculates timing advantage and suspicious activity scores.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import date, timedelta, timezone
|
from datetime import UTC, date, timedelta
|
||||||
from decimal import Decimal
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import and_, func
|
from sqlalchemy import and_
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from pote.db.models import MarketAlert, Official, Security, Trade
|
from pote.db.models import MarketAlert, Security, Trade
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -27,9 +26,7 @@ class DisclosureCorrelator:
|
|||||||
"""Initialize disclosure correlator."""
|
"""Initialize disclosure correlator."""
|
||||||
self.session = session
|
self.session = session
|
||||||
|
|
||||||
def get_alerts_before_trade(
|
def get_alerts_before_trade(self, trade: Trade, lookback_days: int = 30) -> list[MarketAlert]:
|
||||||
self, trade: Trade, lookback_days: int = 30
|
|
||||||
) -> list[MarketAlert]:
|
|
||||||
"""
|
"""
|
||||||
Get market alerts that occurred BEFORE a trade.
|
Get market alerts that occurred BEFORE a trade.
|
||||||
|
|
||||||
@@ -50,14 +47,10 @@ class DisclosureCorrelator:
|
|||||||
# Convert dates to datetime for comparison
|
# Convert dates to datetime for comparison
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
|
|
||||||
start_dt = datetime.combine(start_date, datetime.min.time()).replace(
|
start_dt = datetime.combine(start_date, datetime.min.time()).replace(tzinfo=UTC)
|
||||||
tzinfo=timezone.utc
|
end_dt = datetime.combine(end_date, datetime.max.time()).replace(tzinfo=UTC)
|
||||||
)
|
|
||||||
end_dt = datetime.combine(end_date, datetime.max.time()).replace(
|
|
||||||
tzinfo=timezone.utc
|
|
||||||
)
|
|
||||||
|
|
||||||
alerts = (
|
return (
|
||||||
self.session.query(MarketAlert)
|
self.session.query(MarketAlert)
|
||||||
.filter(
|
.filter(
|
||||||
and_(
|
and_(
|
||||||
@@ -70,8 +63,6 @@ class DisclosureCorrelator:
|
|||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
|
|
||||||
return alerts
|
|
||||||
|
|
||||||
def calculate_timing_score(
|
def calculate_timing_score(
|
||||||
self, trade: Trade, prior_alerts: list[MarketAlert]
|
self, trade: Trade, prior_alerts: list[MarketAlert]
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
@@ -131,8 +122,7 @@ class DisclosureCorrelator:
|
|||||||
)
|
)
|
||||||
elif suspicious:
|
elif suspicious:
|
||||||
reason = (
|
reason = (
|
||||||
f"Trade occurred after {len(prior_alerts)} alerts. "
|
f"Trade occurred after {len(prior_alerts)} alerts. " f"Possible timing advantage."
|
||||||
f"Possible timing advantage."
|
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
reason = (
|
reason = (
|
||||||
@@ -170,35 +160,27 @@ class DisclosureCorrelator:
|
|||||||
timing_analysis = self.calculate_timing_score(trade, prior_alerts)
|
timing_analysis = self.calculate_timing_score(trade, prior_alerts)
|
||||||
|
|
||||||
# Build full analysis
|
# Build full analysis
|
||||||
analysis = {
|
return {
|
||||||
"trade_id": trade.id,
|
"trade_id": trade.id,
|
||||||
"official_name": trade.official.name if trade.official else None,
|
"official_name": trade.official.name if trade.official else None,
|
||||||
"ticker": trade.security.ticker if trade.security else None,
|
"ticker": trade.security.ticker if trade.security else None,
|
||||||
"side": trade.side,
|
"side": trade.side,
|
||||||
"transaction_date": str(trade.transaction_date),
|
"transaction_date": str(trade.transaction_date),
|
||||||
"filing_date": str(trade.filing_date) if trade.filing_date else None,
|
"filing_date": str(trade.filing_date) if trade.filing_date else None,
|
||||||
"value_range": f"${float(trade.value_min):,.0f}"
|
"value_range": f"${float(trade.value_min or 0):,.0f}"
|
||||||
+ (
|
+ (f"-${float(trade.value_max):,.0f}" if trade.value_max else "+"),
|
||||||
f"-${float(trade.value_max):,.0f}"
|
|
||||||
if trade.value_max
|
|
||||||
else "+"
|
|
||||||
),
|
|
||||||
**timing_analysis,
|
**timing_analysis,
|
||||||
"prior_alerts": [
|
"prior_alerts": [
|
||||||
{
|
{
|
||||||
"timestamp": str(alert.timestamp),
|
"timestamp": str(alert.timestamp),
|
||||||
"alert_type": alert.alert_type,
|
"alert_type": alert.alert_type,
|
||||||
"severity": alert.severity,
|
"severity": alert.severity,
|
||||||
"days_before_trade": (
|
"days_before_trade": (trade.transaction_date - alert.timestamp.date()).days,
|
||||||
trade.transaction_date - alert.timestamp.date()
|
|
||||||
).days,
|
|
||||||
}
|
}
|
||||||
for alert in prior_alerts
|
for alert in prior_alerts
|
||||||
],
|
],
|
||||||
}
|
}
|
||||||
|
|
||||||
return analysis
|
|
||||||
|
|
||||||
def analyze_recent_disclosures(
|
def analyze_recent_disclosures(
|
||||||
self, days: int = 7, min_timing_score: float = 50
|
self, days: int = 7, min_timing_score: float = 50
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
@@ -237,9 +219,7 @@ class DisclosureCorrelator:
|
|||||||
f"Found {len(suspicious_trades)} trades with timing score >= {min_timing_score}"
|
f"Found {len(suspicious_trades)} trades with timing score >= {min_timing_score}"
|
||||||
)
|
)
|
||||||
|
|
||||||
return sorted(
|
return sorted(suspicious_trades, key=lambda x: x["timing_score"], reverse=True)
|
||||||
suspicious_trades, key=lambda x: x["timing_score"], reverse=True
|
|
||||||
)
|
|
||||||
|
|
||||||
def get_official_timing_pattern(
|
def get_official_timing_pattern(
|
||||||
self, official_id: int, lookback_days: int = 365
|
self, official_id: int, lookback_days: int = 365
|
||||||
@@ -258,9 +238,7 @@ class DisclosureCorrelator:
|
|||||||
|
|
||||||
trades = (
|
trades = (
|
||||||
self.session.query(Trade)
|
self.session.query(Trade)
|
||||||
.filter(
|
.filter(and_(Trade.official_id == official_id, Trade.transaction_date >= since_date))
|
||||||
and_(Trade.official_id == official_id, Trade.transaction_date >= since_date)
|
|
||||||
)
|
|
||||||
.join(Trade.security)
|
.join(Trade.security)
|
||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
@@ -285,9 +263,7 @@ class DisclosureCorrelator:
|
|||||||
highly_suspicious = sum(1 for a in analyses if a.get("highly_suspicious", False))
|
highly_suspicious = sum(1 for a in analyses if a.get("highly_suspicious", False))
|
||||||
|
|
||||||
avg_timing_score = (
|
avg_timing_score = (
|
||||||
sum(a["timing_score"] for a in analyses) / total_trades
|
sum(a["timing_score"] for a in analyses) / total_trades if total_trades > 0 else 0
|
||||||
if total_trades > 0
|
|
||||||
else 0
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Determine pattern
|
# Determine pattern
|
||||||
@@ -311,9 +287,7 @@ class DisclosureCorrelator:
|
|||||||
"analyses": analyses,
|
"analyses": analyses,
|
||||||
}
|
}
|
||||||
|
|
||||||
def get_ticker_timing_analysis(
|
def get_ticker_timing_analysis(self, ticker: str, lookback_days: int = 365) -> dict[str, Any]:
|
||||||
self, ticker: str, lookback_days: int = 365
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""
|
"""
|
||||||
Analyze timing patterns for a specific ticker.
|
Analyze timing patterns for a specific ticker.
|
||||||
|
|
||||||
@@ -329,9 +303,7 @@ class DisclosureCorrelator:
|
|||||||
trades = (
|
trades = (
|
||||||
self.session.query(Trade)
|
self.session.query(Trade)
|
||||||
.join(Trade.security)
|
.join(Trade.security)
|
||||||
.filter(
|
.filter(and_(Security.ticker == ticker, Trade.transaction_date >= since_date))
|
||||||
and_(Security.ticker == ticker, Trade.transaction_date >= since_date)
|
|
||||||
)
|
|
||||||
.join(Trade.official)
|
.join(Trade.official)
|
||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
@@ -350,10 +322,6 @@ class DisclosureCorrelator:
|
|||||||
"trade_count": len(analyses),
|
"trade_count": len(analyses),
|
||||||
"trades_with_alerts": sum(1 for a in analyses if a["alert_count"] > 0),
|
"trades_with_alerts": sum(1 for a in analyses if a["alert_count"] > 0),
|
||||||
"suspicious_count": sum(1 for a in analyses if a["suspicious"]),
|
"suspicious_count": sum(1 for a in analyses if a["suspicious"]),
|
||||||
"avg_timing_score": round(
|
"avg_timing_score": round(sum(a["timing_score"] for a in analyses) / len(analyses), 2),
|
||||||
sum(a["timing_score"] for a in analyses) / len(analyses), 2
|
|
||||||
),
|
|
||||||
"analyses": sorted(analyses, key=lambda x: x["timing_score"], reverse=True),
|
"analyses": sorted(analyses, key=lambda x: x["timing_score"], reverse=True),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ Detects unusual activity: volume spikes, price movements, volatility.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import datetime, timedelta, timezone
|
from datetime import UTC, datetime, timedelta
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -59,7 +59,7 @@ class MarketMonitor:
|
|||||||
Returns:
|
Returns:
|
||||||
List of alerts detected
|
List of alerts detected
|
||||||
"""
|
"""
|
||||||
alerts = []
|
alerts: list[dict[str, Any]] = []
|
||||||
|
|
||||||
try:
|
try:
|
||||||
stock = yf.Ticker(ticker)
|
stock = yf.Ticker(ticker)
|
||||||
@@ -90,7 +90,7 @@ class MarketMonitor:
|
|||||||
{
|
{
|
||||||
"ticker": ticker,
|
"ticker": ticker,
|
||||||
"alert_type": "unusual_volume",
|
"alert_type": "unusual_volume",
|
||||||
"timestamp": datetime.now(timezone.utc),
|
"timestamp": datetime.now(UTC),
|
||||||
"details": {
|
"details": {
|
||||||
"current_volume": int(current_volume),
|
"current_volume": int(current_volume),
|
||||||
"avg_volume": int(avg_volume),
|
"avg_volume": int(avg_volume),
|
||||||
@@ -109,10 +109,8 @@ class MarketMonitor:
|
|||||||
alerts.append(
|
alerts.append(
|
||||||
{
|
{
|
||||||
"ticker": ticker,
|
"ticker": ticker,
|
||||||
"alert_type": "price_spike"
|
"alert_type": "price_spike" if price_change > 0 else "price_drop",
|
||||||
if price_change > 0
|
"timestamp": datetime.now(UTC),
|
||||||
else "price_drop",
|
|
||||||
"timestamp": datetime.now(timezone.utc),
|
|
||||||
"details": {
|
"details": {
|
||||||
"current_price": float(current_price),
|
"current_price": float(current_price),
|
||||||
"prev_price": float(prev["Close"]),
|
"prev_price": float(prev["Close"]),
|
||||||
@@ -129,14 +127,12 @@ class MarketMonitor:
|
|||||||
if len(hist) >= 5:
|
if len(hist) >= 5:
|
||||||
recent_volatility = hist["Close"].iloc[-5:].pct_change().abs().mean()
|
recent_volatility = hist["Close"].iloc[-5:].pct_change().abs().mean()
|
||||||
if recent_volatility > avg_price_change * 2 and avg_price_change > 0:
|
if recent_volatility > avg_price_change * 2 and avg_price_change > 0:
|
||||||
severity = min(
|
severity = min(10, int((recent_volatility / avg_price_change) - 1))
|
||||||
10, int((recent_volatility / avg_price_change) - 1)
|
|
||||||
)
|
|
||||||
alerts.append(
|
alerts.append(
|
||||||
{
|
{
|
||||||
"ticker": ticker,
|
"ticker": ticker,
|
||||||
"alert_type": "high_volatility",
|
"alert_type": "high_volatility",
|
||||||
"timestamp": datetime.now(timezone.utc),
|
"timestamp": datetime.now(UTC),
|
||||||
"details": {
|
"details": {
|
||||||
"recent_volatility": round(recent_volatility * 100, 2),
|
"recent_volatility": round(recent_volatility * 100, 2),
|
||||||
"avg_volatility": round(avg_price_change * 100, 2),
|
"avg_volatility": round(avg_price_change * 100, 2),
|
||||||
@@ -227,7 +223,7 @@ class MarketMonitor:
|
|||||||
Returns:
|
Returns:
|
||||||
List of MarketAlert objects
|
List of MarketAlert objects
|
||||||
"""
|
"""
|
||||||
since = datetime.now(timezone.utc) - timedelta(days=days)
|
since = datetime.now(UTC) - timedelta(days=days)
|
||||||
|
|
||||||
query = self.session.query(MarketAlert).filter(MarketAlert.timestamp >= since)
|
query = self.session.query(MarketAlert).filter(MarketAlert.timestamp >= since)
|
||||||
|
|
||||||
@@ -252,7 +248,7 @@ class MarketMonitor:
|
|||||||
Returns:
|
Returns:
|
||||||
Dict mapping ticker to alert summary
|
Dict mapping ticker to alert summary
|
||||||
"""
|
"""
|
||||||
since = datetime.now(timezone.utc) - timedelta(days=days)
|
since = datetime.now(UTC) - timedelta(days=days)
|
||||||
|
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
|
|
||||||
@@ -278,5 +274,3 @@ class MarketMonitor:
|
|||||||
}
|
}
|
||||||
|
|
||||||
return summary
|
return summary
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,13 +5,12 @@ Identifies recurring suspicious behavior and trading patterns.
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import date, timedelta
|
from datetime import date, timedelta
|
||||||
from decimal import Decimal
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import and_, func
|
from sqlalchemy import func
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from pote.db.models import MarketAlert, Official, Security, Trade
|
from pote.db.models import Official, Security, Trade
|
||||||
from pote.monitoring.disclosure_correlator import DisclosureCorrelator
|
from pote.monitoring.disclosure_correlator import DisclosureCorrelator
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -60,9 +59,7 @@ class PatternDetector:
|
|||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
|
|
||||||
logger.info(
|
logger.info(f"Analyzing {len(officials_with_trades)} officials with {min_trades}+ trades")
|
||||||
f"Analyzing {len(officials_with_trades)} officials with {min_trades}+ trades"
|
|
||||||
)
|
|
||||||
|
|
||||||
rankings = []
|
rankings = []
|
||||||
|
|
||||||
@@ -70,9 +67,7 @@ class PatternDetector:
|
|||||||
official_id, name, chamber, party, state, trade_count = official_data
|
official_id, name, chamber, party, state, trade_count = official_data
|
||||||
|
|
||||||
# Get timing pattern
|
# Get timing pattern
|
||||||
pattern = self.correlator.get_official_timing_pattern(
|
pattern = self.correlator.get_official_timing_pattern(official_id, lookback_days)
|
||||||
official_id, lookback_days
|
|
||||||
)
|
|
||||||
|
|
||||||
if pattern["trade_count"] == 0:
|
if pattern["trade_count"] == 0:
|
||||||
continue
|
continue
|
||||||
@@ -128,9 +123,7 @@ class PatternDetector:
|
|||||||
rankings = self.rank_officials_by_timing(lookback_days, min_trades=5)
|
rankings = self.rank_officials_by_timing(lookback_days, min_trades=5)
|
||||||
|
|
||||||
# Filter for high suspicious rates
|
# Filter for high suspicious rates
|
||||||
offenders = [
|
offenders = [r for r in rankings if r["suspicious_rate"] >= min_suspicious_rate * 100]
|
||||||
r for r in rankings if r["suspicious_rate"] >= min_suspicious_rate * 100
|
|
||||||
]
|
|
||||||
|
|
||||||
logger.info(
|
logger.info(
|
||||||
f"Found {len(offenders)} officials with {min_suspicious_rate*100}%+ suspicious trades"
|
f"Found {len(offenders)} officials with {min_suspicious_rate*100}%+ suspicious trades"
|
||||||
@@ -155,9 +148,7 @@ class PatternDetector:
|
|||||||
|
|
||||||
# Get tickers with enough trades
|
# Get tickers with enough trades
|
||||||
tickers_with_trades = (
|
tickers_with_trades = (
|
||||||
self.session.query(
|
self.session.query(Security.ticker, func.count(Trade.id).label("trade_count"))
|
||||||
Security.ticker, func.count(Trade.id).label("trade_count")
|
|
||||||
)
|
|
||||||
.join(Trade)
|
.join(Trade)
|
||||||
.filter(Trade.transaction_date >= since_date)
|
.filter(Trade.transaction_date >= since_date)
|
||||||
.group_by(Security.ticker)
|
.group_by(Security.ticker)
|
||||||
@@ -169,10 +160,8 @@ class PatternDetector:
|
|||||||
|
|
||||||
ticker_patterns = []
|
ticker_patterns = []
|
||||||
|
|
||||||
for ticker, trade_count in tickers_with_trades:
|
for ticker, _trade_count in tickers_with_trades:
|
||||||
analysis = self.correlator.get_ticker_timing_analysis(
|
analysis = self.correlator.get_ticker_timing_analysis(ticker, lookback_days)
|
||||||
ticker, lookback_days
|
|
||||||
)
|
|
||||||
|
|
||||||
if analysis["trade_count"] == 0:
|
if analysis["trade_count"] == 0:
|
||||||
continue
|
continue
|
||||||
@@ -199,9 +188,7 @@ class PatternDetector:
|
|||||||
|
|
||||||
return ticker_patterns
|
return ticker_patterns
|
||||||
|
|
||||||
def get_sector_timing_analysis(
|
def get_sector_timing_analysis(self, lookback_days: int = 365) -> dict[str, dict[str, Any]]:
|
||||||
self, lookback_days: int = 365
|
|
||||||
) -> dict[str, dict[str, Any]]:
|
|
||||||
"""
|
"""
|
||||||
Analyze timing patterns by sector.
|
Analyze timing patterns by sector.
|
||||||
|
|
||||||
@@ -252,7 +239,7 @@ class PatternDetector:
|
|||||||
sector_stats[sector]["suspicious_count"] += 1
|
sector_stats[sector]["suspicious_count"] += 1
|
||||||
|
|
||||||
# Calculate averages
|
# Calculate averages
|
||||||
for sector, stats in sector_stats.items():
|
for stats in sector_stats.values():
|
||||||
if stats["trade_count"] > 0:
|
if stats["trade_count"] > 0:
|
||||||
stats["avg_timing_score"] = round(
|
stats["avg_timing_score"] = round(
|
||||||
stats["total_timing_score"] / stats["trade_count"], 2
|
stats["total_timing_score"] / stats["trade_count"], 2
|
||||||
@@ -266,9 +253,7 @@ class PatternDetector:
|
|||||||
|
|
||||||
return sector_stats
|
return sector_stats
|
||||||
|
|
||||||
def get_party_comparison(
|
def get_party_comparison(self, lookback_days: int = 365) -> dict[str, dict[str, Any]]:
|
||||||
self, lookback_days: int = 365
|
|
||||||
) -> dict[str, dict[str, Any]]:
|
|
||||||
"""
|
"""
|
||||||
Compare timing patterns between political parties.
|
Compare timing patterns between political parties.
|
||||||
|
|
||||||
@@ -303,7 +288,7 @@ class PatternDetector:
|
|||||||
party_stats[party]["officials"].append(ranking)
|
party_stats[party]["officials"].append(ranking)
|
||||||
|
|
||||||
# Calculate averages
|
# Calculate averages
|
||||||
for party, stats in party_stats.items():
|
for stats in party_stats.values():
|
||||||
if stats["total_trades"] > 0:
|
if stats["total_trades"] > 0:
|
||||||
stats["avg_timing_score"] = round(
|
stats["avg_timing_score"] = round(
|
||||||
stats["total_timing_score"] / stats["total_trades"], 2
|
stats["total_timing_score"] / stats["total_trades"], 2
|
||||||
@@ -356,5 +341,3 @@ class PatternDetector:
|
|||||||
"sector_analysis": sector_analysis,
|
"sector_analysis": sector_analysis,
|
||||||
"party_comparison": party_comparison,
|
"party_comparison": party_comparison,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -8,5 +8,3 @@ from .email_reporter import EmailReporter
|
|||||||
from .report_generator import ReportGenerator
|
from .report_generator import ReportGenerator
|
||||||
|
|
||||||
__all__ = ["EmailReporter", "ReportGenerator"]
|
__all__ = ["EmailReporter", "ReportGenerator"]
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ import logging
|
|||||||
import smtplib
|
import smtplib
|
||||||
from email.mime.multipart import MIMEMultipart
|
from email.mime.multipart import MIMEMultipart
|
||||||
from email.mime.text import MIMEText
|
from email.mime.text import MIMEText
|
||||||
from typing import List, Optional
|
|
||||||
|
|
||||||
from pote.config import settings
|
from pote.config import settings
|
||||||
|
|
||||||
@@ -20,31 +19,30 @@ class EmailReporter:
|
|||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
smtp_host: Optional[str] = None,
|
smtp_host: str | None = None,
|
||||||
smtp_port: Optional[int] = None,
|
smtp_port: int | None = None,
|
||||||
smtp_user: Optional[str] = None,
|
smtp_user: str | None = None,
|
||||||
smtp_password: Optional[str] = None,
|
smtp_password: str | None = None,
|
||||||
from_email: Optional[str] = None,
|
from_email: str | None = None,
|
||||||
):
|
):
|
||||||
"""
|
"""
|
||||||
Initialize email reporter.
|
Initialize email reporter.
|
||||||
|
|
||||||
If parameters are not provided, will attempt to use settings from config.
|
If parameters are not provided, will attempt to use settings from config.
|
||||||
"""
|
"""
|
||||||
self.smtp_host = smtp_host or getattr(settings, "smtp_host", "localhost")
|
# Settings always defines these fields (with defaults), so direct access is safe.
|
||||||
self.smtp_port = smtp_port or getattr(settings, "smtp_port", 587)
|
self.smtp_host: str = smtp_host or settings.smtp_host
|
||||||
self.smtp_user = smtp_user or getattr(settings, "smtp_user", None)
|
self.smtp_port: int = smtp_port or settings.smtp_port
|
||||||
self.smtp_password = smtp_password or getattr(settings, "smtp_password", None)
|
self.smtp_user: str = smtp_user or settings.smtp_user
|
||||||
self.from_email = from_email or getattr(
|
self.smtp_password: str = smtp_password or settings.smtp_password
|
||||||
settings, "from_email", "pote@localhost"
|
self.from_email: str = from_email or settings.from_email or "pote@localhost"
|
||||||
)
|
|
||||||
|
|
||||||
def send_report(
|
def send_report(
|
||||||
self,
|
self,
|
||||||
to_emails: List[str],
|
to_emails: list[str],
|
||||||
subject: str,
|
subject: str,
|
||||||
body_text: str,
|
body_text: str,
|
||||||
body_html: Optional[str] = None,
|
body_html: str | None = None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
"""
|
"""
|
||||||
Send an email report.
|
Send an email report.
|
||||||
@@ -113,4 +111,3 @@ class EmailReporter:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"SMTP connection test failed: {e}")
|
logger.error(f"SMTP connection test failed: {e}")
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ Generates formatted reports from database data.
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import date, datetime, timedelta
|
from datetime import date, datetime, timedelta
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import func
|
from sqlalchemy import func
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -27,8 +27,8 @@ class ReportGenerator:
|
|||||||
self.detector = PatternDetector(session)
|
self.detector = PatternDetector(session)
|
||||||
|
|
||||||
def generate_daily_summary(
|
def generate_daily_summary(
|
||||||
self, report_date: Optional[date] = None, *, lookback_days: int = 1
|
self, report_date: date | None = None, *, lookback_days: int = 1
|
||||||
) -> Dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
Generate a daily summary report.
|
Generate a daily summary report.
|
||||||
|
|
||||||
@@ -69,7 +69,7 @@ class ReportGenerator:
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Get high-severity alerts
|
# Get high-severity alerts
|
||||||
critical_alerts = [a for a in new_alerts if a.severity >= 7]
|
critical_alerts = [a for a in new_alerts if (a.severity or 0) >= 7]
|
||||||
|
|
||||||
# Get suspicious timing matches
|
# Get suspicious timing matches
|
||||||
suspicious_trades = []
|
suspicious_trades = []
|
||||||
@@ -110,7 +110,7 @@ class ReportGenerator:
|
|||||||
"suspicious_trades": suspicious_trades,
|
"suspicious_trades": suspicious_trades,
|
||||||
}
|
}
|
||||||
|
|
||||||
def generate_weekly_summary(self) -> Dict[str, Any]:
|
def generate_weekly_summary(self) -> dict[str, Any]:
|
||||||
"""
|
"""
|
||||||
Generate a weekly summary report.
|
Generate a weekly summary report.
|
||||||
|
|
||||||
@@ -121,9 +121,7 @@ class ReportGenerator:
|
|||||||
|
|
||||||
# Most active officials
|
# Most active officials
|
||||||
active_officials = (
|
active_officials = (
|
||||||
self.session.query(
|
self.session.query(Official.name, func.count(Trade.id).label("trade_count"))
|
||||||
Official.name, func.count(Trade.id).label("trade_count")
|
|
||||||
)
|
|
||||||
.join(Trade)
|
.join(Trade)
|
||||||
.filter(Trade.filing_date >= week_ago)
|
.filter(Trade.filing_date >= week_ago)
|
||||||
.group_by(Official.id, Official.name)
|
.group_by(Official.id, Official.name)
|
||||||
@@ -134,9 +132,7 @@ class ReportGenerator:
|
|||||||
|
|
||||||
# Most traded securities
|
# Most traded securities
|
||||||
active_securities = (
|
active_securities = (
|
||||||
self.session.query(
|
self.session.query(Security.ticker, func.count(Trade.id).label("trade_count"))
|
||||||
Security.ticker, func.count(Trade.id).label("trade_count")
|
|
||||||
)
|
|
||||||
.join(Trade)
|
.join(Trade)
|
||||||
.filter(Trade.filing_date >= week_ago)
|
.filter(Trade.filing_date >= week_ago)
|
||||||
.group_by(Security.id, Security.ticker)
|
.group_by(Security.id, Security.ticker)
|
||||||
@@ -146,8 +142,10 @@ class ReportGenerator:
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Get top suspicious patterns
|
# Get top suspicious patterns
|
||||||
|
# Same fix as branch docs/deploy-email-closed: the old kwarg names never
|
||||||
|
# existed on identify_repeat_offenders and failed every weekly run.
|
||||||
repeat_offenders = self.detector.identify_repeat_offenders(
|
repeat_offenders = self.detector.identify_repeat_offenders(
|
||||||
days_lookback=7, min_suspicious_trades=2, min_timing_score=40
|
lookback_days=7, min_suspicious_rate=0.4
|
||||||
)
|
)
|
||||||
|
|
||||||
return {
|
return {
|
||||||
@@ -157,14 +155,13 @@ class ReportGenerator:
|
|||||||
{"name": name, "trade_count": count} for name, count in active_officials
|
{"name": name, "trade_count": count} for name, count in active_officials
|
||||||
],
|
],
|
||||||
"most_traded_securities": [
|
"most_traded_securities": [
|
||||||
{"ticker": ticker, "trade_count": count}
|
{"ticker": ticker, "trade_count": count} for ticker, count in active_securities
|
||||||
for ticker, count in active_securities
|
|
||||||
],
|
],
|
||||||
"repeat_offenders_count": len(repeat_offenders),
|
"repeat_offenders_count": len(repeat_offenders),
|
||||||
"repeat_offenders": repeat_offenders[:5], # Top 5
|
"repeat_offenders": repeat_offenders[:5], # Top 5
|
||||||
}
|
}
|
||||||
|
|
||||||
def format_as_text(self, report_data: Dict[str, Any], report_type: str) -> str:
|
def format_as_text(self, report_data: dict[str, Any], report_type: str) -> str:
|
||||||
"""
|
"""
|
||||||
Format report data as plain text.
|
Format report data as plain text.
|
||||||
|
|
||||||
@@ -177,12 +174,11 @@ class ReportGenerator:
|
|||||||
"""
|
"""
|
||||||
if report_type == "daily":
|
if report_type == "daily":
|
||||||
return self._format_daily_text(report_data)
|
return self._format_daily_text(report_data)
|
||||||
elif report_type == "weekly":
|
if report_type == "weekly":
|
||||||
return self._format_weekly_text(report_data)
|
return self._format_weekly_text(report_data)
|
||||||
else:
|
|
||||||
return str(report_data)
|
return str(report_data)
|
||||||
|
|
||||||
def _format_daily_text(self, data: Dict[str, Any]) -> str:
|
def _format_daily_text(self, data: dict[str, Any]) -> str:
|
||||||
"""Format daily report as plain text."""
|
"""Format daily report as plain text."""
|
||||||
if data.get("lookback_days", 1) > 1:
|
if data.get("lookback_days", 1) > 1:
|
||||||
trades_label = (
|
trades_label = (
|
||||||
@@ -245,7 +241,7 @@ class ReportGenerator:
|
|||||||
|
|
||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
def _format_weekly_text(self, data: Dict[str, Any]) -> str:
|
def _format_weekly_text(self, data: dict[str, Any]) -> str:
|
||||||
"""Format weekly report as plain text."""
|
"""Format weekly report as plain text."""
|
||||||
lines = [
|
lines = [
|
||||||
"=" * 70,
|
"=" * 70,
|
||||||
@@ -264,9 +260,7 @@ class ReportGenerator:
|
|||||||
lines.append(f" • {security['ticker']}: {security['trade_count']} trades")
|
lines.append(f" • {security['ticker']}: {security['trade_count']} trades")
|
||||||
|
|
||||||
if data["repeat_offenders"]:
|
if data["repeat_offenders"]:
|
||||||
lines.extend(
|
lines.extend(["", f"⚠️ REPEAT OFFENDERS ({data['repeat_offenders_count']} total)"])
|
||||||
["", f"⚠️ REPEAT OFFENDERS ({data['repeat_offenders_count']} total)"]
|
|
||||||
)
|
|
||||||
for offender in data["repeat_offenders"]:
|
for offender in data["repeat_offenders"]:
|
||||||
lines.append(
|
lines.append(
|
||||||
f" • {offender['official_name']}: "
|
f" • {offender['official_name']}: "
|
||||||
@@ -285,7 +279,7 @@ class ReportGenerator:
|
|||||||
|
|
||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
def format_as_html(self, report_data: Dict[str, Any], report_type: str) -> str:
|
def format_as_html(self, report_data: dict[str, Any], report_type: str) -> str:
|
||||||
"""
|
"""
|
||||||
Format report data as HTML.
|
Format report data as HTML.
|
||||||
|
|
||||||
@@ -298,12 +292,11 @@ class ReportGenerator:
|
|||||||
"""
|
"""
|
||||||
if report_type == "daily":
|
if report_type == "daily":
|
||||||
return self._format_daily_html(report_data)
|
return self._format_daily_html(report_data)
|
||||||
elif report_type == "weekly":
|
if report_type == "weekly":
|
||||||
return self._format_weekly_html(report_data)
|
return self._format_weekly_html(report_data)
|
||||||
else:
|
|
||||||
return f"<pre>{report_data}</pre>"
|
return f"<pre>{report_data}</pre>"
|
||||||
|
|
||||||
def _format_daily_html(self, data: Dict[str, Any]) -> str:
|
def _format_daily_html(self, data: dict[str, Any]) -> str:
|
||||||
"""Format daily report as HTML."""
|
"""Format daily report as HTML."""
|
||||||
if data.get("lookback_days", 1) > 1:
|
if data.get("lookback_days", 1) > 1:
|
||||||
new_trades_label = f"Trades Filed (last {data['lookback_days']} days):"
|
new_trades_label = f"Trades Filed (last {data['lookback_days']} days):"
|
||||||
@@ -377,7 +370,7 @@ class ReportGenerator:
|
|||||||
|
|
||||||
return html
|
return html
|
||||||
|
|
||||||
def _format_weekly_html(self, data: Dict[str, Any]) -> str:
|
def _format_weekly_html(self, data: dict[str, Any]) -> str:
|
||||||
"""Format weekly report as HTML."""
|
"""Format weekly report as HTML."""
|
||||||
html = f"""
|
html = f"""
|
||||||
<html>
|
<html>
|
||||||
@@ -443,4 +436,3 @@ class ReportGenerator:
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
return html
|
return html
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -22,8 +22,8 @@ def test_db_session() -> Session:
|
|||||||
engine = create_engine("sqlite:///:memory:", echo=False)
|
engine = create_engine("sqlite:///:memory:", echo=False)
|
||||||
Base.metadata.create_all(engine)
|
Base.metadata.create_all(engine)
|
||||||
|
|
||||||
TestSessionLocal = sessionmaker(bind=engine)
|
session_factory = sessionmaker(bind=engine)
|
||||||
session = TestSessionLocal()
|
session = session_factory()
|
||||||
|
|
||||||
yield session
|
yield session
|
||||||
|
|
||||||
|
|||||||
+10
-6
@@ -1,13 +1,14 @@
|
|||||||
"""Tests for analytics module."""
|
"""Tests for analytics module."""
|
||||||
|
|
||||||
import pytest
|
|
||||||
from datetime import date, timedelta
|
from datetime import date, timedelta
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from pote.analytics.returns import ReturnCalculator
|
import pytest
|
||||||
|
|
||||||
from pote.analytics.benchmarks import BenchmarkComparison
|
from pote.analytics.benchmarks import BenchmarkComparison
|
||||||
from pote.analytics.metrics import PerformanceMetrics
|
from pote.analytics.metrics import PerformanceMetrics
|
||||||
from pote.db.models import Official, Security, Trade, Price
|
from pote.analytics.returns import ReturnCalculator
|
||||||
|
from pote.db.models import Price, Security, Trade
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -80,7 +81,9 @@ def test_return_calculator_basic(test_db_session, sample_official, sample_securi
|
|||||||
assert "exit_price" in result
|
assert "exit_price" in result
|
||||||
|
|
||||||
|
|
||||||
def test_return_calculator_sell_trade(test_db_session, sample_official, sample_security, sample_prices):
|
def test_return_calculator_sell_trade(
|
||||||
|
test_db_session, sample_official, sample_security, sample_prices
|
||||||
|
):
|
||||||
session = test_db_session
|
session = test_db_session
|
||||||
"""Test return calculation for sell trade."""
|
"""Test return calculation for sell trade."""
|
||||||
trade = Trade(
|
trade = Trade(
|
||||||
@@ -154,7 +157,9 @@ def test_benchmark_comparison(test_db_session, sample_official, sample_security,
|
|||||||
assert "beat_market" in result
|
assert "beat_market" in result
|
||||||
|
|
||||||
|
|
||||||
def test_performance_metrics_official(test_db_session, sample_official, sample_security, sample_prices):
|
def test_performance_metrics_official(
|
||||||
|
test_db_session, sample_official, sample_security, sample_prices
|
||||||
|
):
|
||||||
session = test_db_session
|
session = test_db_session
|
||||||
"""Test official performance metrics."""
|
"""Test official performance metrics."""
|
||||||
# Create multiple trades
|
# Create multiple trades
|
||||||
@@ -265,4 +270,3 @@ def test_timing_analysis(test_db_session, sample_official, sample_security):
|
|||||||
|
|
||||||
assert "avg_disclosure_lag_days" in timing
|
assert "avg_disclosure_lag_days" in timing
|
||||||
assert timing["avg_disclosure_lag_days"] > 0
|
assert timing["avg_disclosure_lag_days"] > 0
|
||||||
|
|
||||||
|
|||||||
@@ -1,13 +1,14 @@
|
|||||||
"""Integration tests for analytics with real-ish data."""
|
"""Integration tests for analytics with real-ish data."""
|
||||||
|
|
||||||
import pytest
|
|
||||||
from datetime import date, timedelta
|
from datetime import date, timedelta
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from pote.analytics.returns import ReturnCalculator
|
import pytest
|
||||||
|
|
||||||
from pote.analytics.benchmarks import BenchmarkComparison
|
from pote.analytics.benchmarks import BenchmarkComparison
|
||||||
from pote.analytics.metrics import PerformanceMetrics
|
from pote.analytics.metrics import PerformanceMetrics
|
||||||
from pote.db.models import Official, Security, Trade, Price
|
from pote.analytics.returns import ReturnCalculator
|
||||||
|
from pote.db.models import Official, Price, Security, Trade
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -129,7 +130,9 @@ def test_return_calculation_with_real_data(test_db_session, full_test_data):
|
|||||||
|
|
||||||
# Entry around day 15, exit around day 105
|
# Entry around day 15, exit around day 105
|
||||||
# Expected return: (720 - 532.5) / 532.5 = ~35%
|
# Expected return: (720 - 532.5) / 532.5 = ~35%
|
||||||
assert 30 < float(result["return_pct"]) < 50, f"Expected ~35% return, got {result['return_pct']}"
|
assert (
|
||||||
|
30 < float(result["return_pct"]) < 50
|
||||||
|
), f"Expected ~35% return, got {result['return_pct']}"
|
||||||
|
|
||||||
print(f"\n✅ NVDA 90-day return: {result['return_pct']:.2f}%")
|
print(f"\n✅ NVDA 90-day return: {result['return_pct']:.2f}%")
|
||||||
print(f" Entry: ${result['entry_price']} on {result['transaction_date']}")
|
print(f" Entry: ${result['entry_price']} on {result['transaction_date']}")
|
||||||
@@ -155,7 +158,7 @@ def test_benchmark_comparison_with_real_data(test_db_session, full_test_data):
|
|||||||
assert result["beat_market"] is True
|
assert result["beat_market"] is True
|
||||||
assert float(result["abnormal_return"]) > 10, "NVDA should have strong alpha vs SPY"
|
assert float(result["abnormal_return"]) > 10, "NVDA should have strong alpha vs SPY"
|
||||||
|
|
||||||
print(f"\n✅ Benchmark Comparison:")
|
print("\n✅ Benchmark Comparison:")
|
||||||
print(f" NVDA Return: {result['trade_return']:.2f}%")
|
print(f" NVDA Return: {result['trade_return']:.2f}%")
|
||||||
print(f" SPY Return: {result['benchmark_return']:.2f}%")
|
print(f" SPY Return: {result['benchmark_return']:.2f}%")
|
||||||
print(f" Alpha: {result['abnormal_return']:+.2f}%")
|
print(f" Alpha: {result['abnormal_return']:+.2f}%")
|
||||||
@@ -201,7 +204,7 @@ def test_multiple_windows(test_db_session, full_test_data):
|
|||||||
|
|
||||||
# Returns should generally increase with longer windows (given upward trend)
|
# Returns should generally increase with longer windows (given upward trend)
|
||||||
if 30 in results and 90 in results:
|
if 30 in results and 90 in results:
|
||||||
print(f"\n✅ Multiple Windows:")
|
print("\n✅ Multiple Windows:")
|
||||||
for window in [30, 60, 90]:
|
for window in [30, 60, 90]:
|
||||||
if window in results:
|
if window in results:
|
||||||
print(f" {window:3d} days: {results[window]['return_pct']:+7.2f}%")
|
print(f" {window:3d} days: {results[window]['return_pct']:+7.2f}%")
|
||||||
@@ -217,7 +220,7 @@ def test_top_performers(test_db_session, full_test_data):
|
|||||||
assert isinstance(top, list)
|
assert isinstance(top, list)
|
||||||
assert len(top) > 0
|
assert len(top) > 0
|
||||||
|
|
||||||
print(f"\n✅ Top Performers:")
|
print("\n✅ Top Performers:")
|
||||||
for i, perf in enumerate(top, 1):
|
for i, perf in enumerate(top, 1):
|
||||||
if perf.get("trades_analyzed", 0) > 0:
|
if perf.get("trades_analyzed", 0) > 0:
|
||||||
print(f" {i}. {perf['name']:20s} | Alpha: {perf['avg_alpha']:+6.2f}%")
|
print(f" {i}. {perf['name']:20s} | Alpha: {perf['avg_alpha']:+6.2f}%")
|
||||||
@@ -234,7 +237,7 @@ def test_system_statistics(test_db_session, full_test_data):
|
|||||||
assert stats["total_trades"] >= 2
|
assert stats["total_trades"] >= 2
|
||||||
assert stats["total_securities"] >= 2
|
assert stats["total_securities"] >= 2
|
||||||
|
|
||||||
print(f"\n✅ System Statistics:")
|
print("\n✅ System Statistics:")
|
||||||
print(f" Officials: {stats['total_officials']}")
|
print(f" Officials: {stats['total_officials']}")
|
||||||
print(f" Trades: {stats['total_trades']}")
|
print(f" Trades: {stats['total_trades']}")
|
||||||
print(f" Securities: {stats['total_securities']}")
|
print(f" Securities: {stats['total_securities']}")
|
||||||
@@ -254,7 +257,7 @@ def test_disclosure_timing(test_db_session, full_test_data):
|
|||||||
assert "avg_disclosure_lag_days" in timing
|
assert "avg_disclosure_lag_days" in timing
|
||||||
assert timing["avg_disclosure_lag_days"] > 0
|
assert timing["avg_disclosure_lag_days"] > 0
|
||||||
|
|
||||||
print(f"\n✅ Disclosure Timing:")
|
print("\n✅ Disclosure Timing:")
|
||||||
print(f" Average Lag: {timing['avg_disclosure_lag_days']:.1f} days")
|
print(f" Average Lag: {timing['avg_disclosure_lag_days']:.1f} days")
|
||||||
print(f" Median Lag: {timing['median_disclosure_lag_days']} days")
|
print(f" Median Lag: {timing['median_disclosure_lag_days']} days")
|
||||||
|
|
||||||
@@ -269,9 +272,11 @@ def test_sector_analysis(test_db_session, full_test_data):
|
|||||||
assert isinstance(sectors, list)
|
assert isinstance(sectors, list)
|
||||||
|
|
||||||
if sectors:
|
if sectors:
|
||||||
print(f"\n✅ Sector Analysis:")
|
print("\n✅ Sector Analysis:")
|
||||||
for s in sectors:
|
for s in sectors:
|
||||||
print(f" {s['sector']:15s} | {s['trade_count']} trades | Alpha: {s['avg_alpha']:+6.2f}%")
|
print(
|
||||||
|
f" {s['sector']:15s} | {s['trade_count']} trades | Alpha: {s['avg_alpha']:+6.2f}%"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_edge_case_missing_exit_price(test_db_session, full_test_data):
|
def test_edge_case_missing_exit_price(test_db_session, full_test_data):
|
||||||
@@ -328,4 +333,3 @@ def test_sell_trade_logic(test_db_session, full_test_data):
|
|||||||
# Selling during uptrend = negative return
|
# Selling during uptrend = negative return
|
||||||
assert result["return_pct"] < 0, "Sell during uptrend should show negative return"
|
assert result["return_pct"] < 0, "Sell during uptrend should show negative return"
|
||||||
print(f"\n✅ Sell trade return correctly inverted: {result['return_pct']:.2f}%")
|
print(f"\n✅ Sell trade return correctly inverted: {result['return_pct']:.2f}%")
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
"""Tests for disclosure correlation module."""
|
"""Tests for disclosure correlation module."""
|
||||||
|
|
||||||
import pytest
|
from datetime import UTC, date, datetime
|
||||||
from datetime import date, datetime, timedelta, timezone
|
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pote.db.models import MarketAlert, Official, Security, Trade
|
||||||
from pote.monitoring.disclosure_correlator import DisclosureCorrelator
|
from pote.monitoring.disclosure_correlator import DisclosureCorrelator
|
||||||
from pote.db.models import Official, Security, Trade, MarketAlert
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -38,7 +39,7 @@ def trade_with_alerts(test_db_session):
|
|||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="NVDA",
|
ticker="NVDA",
|
||||||
alert_type="unusual_volume",
|
alert_type="unusual_volume",
|
||||||
timestamp=datetime(2024, 1, 10, 10, 30, tzinfo=timezone.utc), # 5 days before
|
timestamp=datetime(2024, 1, 10, 10, 30, tzinfo=UTC), # 5 days before
|
||||||
details={"multiplier": 3.5},
|
details={"multiplier": 3.5},
|
||||||
price=Decimal("490.00"),
|
price=Decimal("490.00"),
|
||||||
volume=100000000,
|
volume=100000000,
|
||||||
@@ -48,7 +49,7 @@ def trade_with_alerts(test_db_session):
|
|||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="NVDA",
|
ticker="NVDA",
|
||||||
alert_type="price_spike",
|
alert_type="price_spike",
|
||||||
timestamp=datetime(2024, 1, 12, 14, 15, tzinfo=timezone.utc), # 3 days before
|
timestamp=datetime(2024, 1, 12, 14, 15, tzinfo=UTC), # 3 days before
|
||||||
details={"change_pct": 5.5},
|
details={"change_pct": 5.5},
|
||||||
price=Decimal("505.00"),
|
price=Decimal("505.00"),
|
||||||
volume=85000000,
|
volume=85000000,
|
||||||
@@ -58,7 +59,7 @@ def trade_with_alerts(test_db_session):
|
|||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="NVDA",
|
ticker="NVDA",
|
||||||
alert_type="high_volatility",
|
alert_type="high_volatility",
|
||||||
timestamp=datetime(2024, 1, 14, 16, 20, tzinfo=timezone.utc), # 1 day before
|
timestamp=datetime(2024, 1, 14, 16, 20, tzinfo=UTC), # 1 day before
|
||||||
details={"multiplier": 2.5},
|
details={"multiplier": 2.5},
|
||||||
price=Decimal("510.00"),
|
price=Decimal("510.00"),
|
||||||
volume=90000000,
|
volume=90000000,
|
||||||
@@ -208,13 +209,13 @@ def test_calculate_timing_score_factors(test_db_session):
|
|||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=UTC),
|
||||||
severity=3,
|
severity=3,
|
||||||
),
|
),
|
||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 11, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2024, 1, 11, 12, 0, tzinfo=UTC),
|
||||||
severity=4,
|
severity=4,
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
@@ -228,13 +229,13 @@ def test_calculate_timing_score_factors(test_db_session):
|
|||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 13, 12, 0, tzinfo=timezone.utc), # Recent
|
timestamp=datetime(2024, 1, 13, 12, 0, tzinfo=UTC), # Recent
|
||||||
severity=9,
|
severity=9,
|
||||||
),
|
),
|
||||||
MarketAlert(
|
MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 14, 12, 0, tzinfo=timezone.utc), # Very recent
|
timestamp=datetime(2024, 1, 14, 12, 0, tzinfo=UTC), # Very recent
|
||||||
severity=8,
|
severity=8,
|
||||||
),
|
),
|
||||||
]
|
]
|
||||||
@@ -284,8 +285,7 @@ def test_analyze_recent_disclosures(test_db_session, trade_with_alerts, trade_wi
|
|||||||
|
|
||||||
# Both trades were created "recently" (in fixture setup)
|
# Both trades were created "recently" (in fixture setup)
|
||||||
suspicious_trades = correlator.analyze_recent_disclosures(
|
suspicious_trades = correlator.analyze_recent_disclosures(
|
||||||
days=365, # Wide window to catch test data
|
days=365, min_timing_score=50 # Wide window to catch test data
|
||||||
min_timing_score=50
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Should find at least the suspicious trade
|
# Should find at least the suspicious trade
|
||||||
@@ -384,7 +384,7 @@ def test_alerts_outside_lookback_window(test_db_session):
|
|||||||
recent_alert = MarketAlert(
|
recent_alert = MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 13, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2024, 1, 13, 12, 0, tzinfo=UTC),
|
||||||
severity=7,
|
severity=7,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -392,7 +392,7 @@ def test_alerts_outside_lookback_window(test_db_session):
|
|||||||
old_alert = MarketAlert(
|
old_alert = MarketAlert(
|
||||||
ticker="TEST",
|
ticker="TEST",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2023, 12, 6, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2023, 12, 6, 12, 0, tzinfo=UTC),
|
||||||
severity=8,
|
severity=8,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -433,14 +433,14 @@ def test_different_ticker_alerts_excluded(test_db_session):
|
|||||||
nvda_alert = MarketAlert(
|
nvda_alert = MarketAlert(
|
||||||
ticker="NVDA",
|
ticker="NVDA",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=UTC),
|
||||||
severity=7,
|
severity=7,
|
||||||
)
|
)
|
||||||
|
|
||||||
msft_alert = MarketAlert(
|
msft_alert = MarketAlert(
|
||||||
ticker="MSFT",
|
ticker="MSFT",
|
||||||
alert_type="test",
|
alert_type="test",
|
||||||
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=timezone.utc),
|
timestamp=datetime(2024, 1, 10, 12, 0, tzinfo=UTC),
|
||||||
severity=8,
|
severity=8,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -452,4 +452,3 @@ def test_different_ticker_alerts_excluded(test_db_session):
|
|||||||
|
|
||||||
assert len(alerts) == 1
|
assert len(alerts) == 1
|
||||||
assert alerts[0].ticker == "NVDA"
|
assert alerts[0].ticker == "NVDA"
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ Tests for database models.
|
|||||||
from datetime import date
|
from datetime import date
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
|
import pytest
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|
||||||
from pote.db.models import Price, Security, Trade
|
from pote.db.models import Price, Security, Trade
|
||||||
@@ -56,12 +57,9 @@ def test_unique_constraints(test_db_session, sample_security):
|
|||||||
dup_security = Security(ticker="AAPL", name="Apple Duplicate")
|
dup_security = Security(ticker="AAPL", name="Apple Duplicate")
|
||||||
test_db_session.add(dup_security)
|
test_db_session.add(dup_security)
|
||||||
|
|
||||||
try:
|
with pytest.raises(IntegrityError):
|
||||||
test_db_session.commit()
|
test_db_session.commit()
|
||||||
assert False, "Should have raised IntegrityError"
|
|
||||||
except IntegrityError:
|
|
||||||
test_db_session.rollback()
|
test_db_session.rollback()
|
||||||
# Expected behavior
|
|
||||||
|
|
||||||
|
|
||||||
def test_price_unique_per_security_date(test_db_session, sample_security):
|
def test_price_unique_per_security_date(test_db_session, sample_security):
|
||||||
@@ -83,12 +81,9 @@ def test_price_unique_per_security_date(test_db_session, sample_security):
|
|||||||
)
|
)
|
||||||
test_db_session.add(price2)
|
test_db_session.add(price2)
|
||||||
|
|
||||||
try:
|
with pytest.raises(IntegrityError):
|
||||||
test_db_session.commit()
|
test_db_session.commit()
|
||||||
assert False, "Should have raised IntegrityError"
|
|
||||||
except IntegrityError:
|
|
||||||
test_db_session.rollback()
|
test_db_session.rollback()
|
||||||
# Expected behavior
|
|
||||||
|
|
||||||
|
|
||||||
def test_trade_queries(test_db_session, sample_official, sample_security):
|
def test_trade_queries(test_db_session, sample_official, sample_security):
|
||||||
|
|||||||
+60
-30
@@ -1,12 +1,13 @@
|
|||||||
"""Tests for market monitoring module."""
|
"""Tests for market monitoring module."""
|
||||||
|
|
||||||
import pytest
|
from datetime import UTC, date, datetime, timedelta
|
||||||
from datetime import date, datetime, timedelta, timezone
|
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
from pote.monitoring.market_monitor import MarketMonitor
|
import pytest
|
||||||
|
|
||||||
|
from pote.db.models import MarketAlert, Official, Security, Trade
|
||||||
from pote.monitoring.alert_manager import AlertManager
|
from pote.monitoring.alert_manager import AlertManager
|
||||||
from pote.db.models import Official, Security, Trade, MarketAlert
|
from pote.monitoring.market_monitor import MarketMonitor
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -31,21 +32,51 @@ def sample_congressional_trades(test_db_session):
|
|||||||
|
|
||||||
# Create multiple trades (NVDA is most traded)
|
# Create multiple trades (NVDA is most traded)
|
||||||
trades = [
|
trades = [
|
||||||
Trade(official_id=pelosi.id, security_id=nvda.id, source="test",
|
Trade(
|
||||||
transaction_date=date(2024, 1, 15), side="buy",
|
official_id=pelosi.id,
|
||||||
value_min=Decimal("15001"), value_max=Decimal("50000")),
|
security_id=nvda.id,
|
||||||
Trade(official_id=pelosi.id, security_id=nvda.id, source="test",
|
source="test",
|
||||||
transaction_date=date(2024, 2, 1), side="buy",
|
transaction_date=date(2024, 1, 15),
|
||||||
value_min=Decimal("15001"), value_max=Decimal("50000")),
|
side="buy",
|
||||||
Trade(official_id=tuberville.id, security_id=nvda.id, source="test",
|
value_min=Decimal("15001"),
|
||||||
transaction_date=date(2024, 2, 15), side="buy",
|
value_max=Decimal("50000"),
|
||||||
value_min=Decimal("50001"), value_max=Decimal("100000")),
|
),
|
||||||
Trade(official_id=pelosi.id, security_id=msft.id, source="test",
|
Trade(
|
||||||
transaction_date=date(2024, 1, 20), side="sell",
|
official_id=pelosi.id,
|
||||||
value_min=Decimal("15001"), value_max=Decimal("50000")),
|
security_id=nvda.id,
|
||||||
Trade(official_id=tuberville.id, security_id=aapl.id, source="test",
|
source="test",
|
||||||
transaction_date=date(2024, 2, 10), side="buy",
|
transaction_date=date(2024, 2, 1),
|
||||||
value_min=Decimal("15001"), value_max=Decimal("50000")),
|
side="buy",
|
||||||
|
value_min=Decimal("15001"),
|
||||||
|
value_max=Decimal("50000"),
|
||||||
|
),
|
||||||
|
Trade(
|
||||||
|
official_id=tuberville.id,
|
||||||
|
security_id=nvda.id,
|
||||||
|
source="test",
|
||||||
|
transaction_date=date(2024, 2, 15),
|
||||||
|
side="buy",
|
||||||
|
value_min=Decimal("50001"),
|
||||||
|
value_max=Decimal("100000"),
|
||||||
|
),
|
||||||
|
Trade(
|
||||||
|
official_id=pelosi.id,
|
||||||
|
security_id=msft.id,
|
||||||
|
source="test",
|
||||||
|
transaction_date=date(2024, 1, 20),
|
||||||
|
side="sell",
|
||||||
|
value_min=Decimal("15001"),
|
||||||
|
value_max=Decimal("50000"),
|
||||||
|
),
|
||||||
|
Trade(
|
||||||
|
official_id=tuberville.id,
|
||||||
|
security_id=aapl.id,
|
||||||
|
source="test",
|
||||||
|
transaction_date=date(2024, 2, 10),
|
||||||
|
side="buy",
|
||||||
|
value_min=Decimal("15001"),
|
||||||
|
value_max=Decimal("50000"),
|
||||||
|
),
|
||||||
]
|
]
|
||||||
session.add_all(trades)
|
session.add_all(trades)
|
||||||
session.commit()
|
session.commit()
|
||||||
@@ -62,7 +93,7 @@ def sample_alerts(test_db_session):
|
|||||||
"""Create sample market alerts."""
|
"""Create sample market alerts."""
|
||||||
session = test_db_session
|
session = test_db_session
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
alerts = [
|
alerts = [
|
||||||
MarketAlert(
|
MarketAlert(
|
||||||
@@ -145,7 +176,7 @@ def test_scan_watchlist_with_mock(test_db_session, sample_congressional_trades,
|
|||||||
{
|
{
|
||||||
"ticker": ticker,
|
"ticker": ticker,
|
||||||
"alert_type": "unusual_volume",
|
"alert_type": "unusual_volume",
|
||||||
"timestamp": datetime.now(timezone.utc),
|
"timestamp": datetime.now(UTC),
|
||||||
"details": {"multiplier": 3.5},
|
"details": {"multiplier": 3.5},
|
||||||
"price": Decimal("500.00"),
|
"price": Decimal("500.00"),
|
||||||
"volume": 100000000,
|
"volume": 100000000,
|
||||||
@@ -174,7 +205,7 @@ def test_save_alerts(test_db_session):
|
|||||||
{
|
{
|
||||||
"ticker": "TSLA",
|
"ticker": "TSLA",
|
||||||
"alert_type": "price_spike",
|
"alert_type": "price_spike",
|
||||||
"timestamp": datetime.now(timezone.utc),
|
"timestamp": datetime.now(UTC),
|
||||||
"details": {"change_pct": 7.5},
|
"details": {"change_pct": 7.5},
|
||||||
"price": Decimal("250.00"),
|
"price": Decimal("250.00"),
|
||||||
"volume": 75000000,
|
"volume": 75000000,
|
||||||
@@ -184,7 +215,7 @@ def test_save_alerts(test_db_session):
|
|||||||
{
|
{
|
||||||
"ticker": "TSLA",
|
"ticker": "TSLA",
|
||||||
"alert_type": "unusual_volume",
|
"alert_type": "unusual_volume",
|
||||||
"timestamp": datetime.now(timezone.utc),
|
"timestamp": datetime.now(UTC),
|
||||||
"details": {"multiplier": 4.0},
|
"details": {"multiplier": 4.0},
|
||||||
"price": Decimal("250.00"),
|
"price": Decimal("250.00"),
|
||||||
"volume": 120000000,
|
"volume": 120000000,
|
||||||
@@ -297,7 +328,7 @@ def test_alert_manager_filter_alerts(test_db_session, sample_alerts):
|
|||||||
sample_alerts,
|
sample_alerts,
|
||||||
min_severity=4,
|
min_severity=4,
|
||||||
tickers=["NVDA"],
|
tickers=["NVDA"],
|
||||||
alert_types=["unusual_volume", "price_spike"]
|
alert_types=["unusual_volume", "price_spike"],
|
||||||
)
|
)
|
||||||
assert len(filtered) == 2
|
assert len(filtered) == 2
|
||||||
|
|
||||||
@@ -307,7 +338,7 @@ def test_alert_manager_generate_summary_text(test_db_session, sample_alerts):
|
|||||||
session = test_db_session
|
session = test_db_session
|
||||||
alert_mgr = AlertManager(session)
|
alert_mgr = AlertManager(session)
|
||||||
|
|
||||||
report = alert_mgr.generate_summary_report(sample_alerts, format="text")
|
report = alert_mgr.generate_summary_report(sample_alerts, output_format="text")
|
||||||
|
|
||||||
assert "MARKET ACTIVITY ALERTS" in report
|
assert "MARKET ACTIVITY ALERTS" in report
|
||||||
assert "3 Alerts" in report
|
assert "3 Alerts" in report
|
||||||
@@ -321,7 +352,7 @@ def test_alert_manager_generate_summary_html(test_db_session, sample_alerts):
|
|||||||
session = test_db_session
|
session = test_db_session
|
||||||
alert_mgr = AlertManager(session)
|
alert_mgr = AlertManager(session)
|
||||||
|
|
||||||
report = alert_mgr.generate_summary_report(sample_alerts, format="html")
|
report = alert_mgr.generate_summary_report(sample_alerts, output_format="html")
|
||||||
|
|
||||||
assert "<html>" in report
|
assert "<html>" in report
|
||||||
assert "<head>" in report
|
assert "<head>" in report
|
||||||
@@ -334,7 +365,7 @@ def test_alert_manager_empty_alerts(test_db_session):
|
|||||||
session = test_db_session
|
session = test_db_session
|
||||||
alert_mgr = AlertManager(session)
|
alert_mgr = AlertManager(session)
|
||||||
|
|
||||||
report = alert_mgr.generate_summary_report([], format="text")
|
report = alert_mgr.generate_summary_report([], output_format="text")
|
||||||
|
|
||||||
assert "No alerts" in report
|
assert "No alerts" in report
|
||||||
|
|
||||||
@@ -346,7 +377,7 @@ def test_market_alert_model(test_db_session):
|
|||||||
alert = MarketAlert(
|
alert = MarketAlert(
|
||||||
ticker="GOOGL",
|
ticker="GOOGL",
|
||||||
alert_type="price_spike",
|
alert_type="price_spike",
|
||||||
timestamp=datetime.now(timezone.utc),
|
timestamp=datetime.now(UTC),
|
||||||
details={"test": "data"},
|
details={"test": "data"},
|
||||||
price=Decimal("140.50"),
|
price=Decimal("140.50"),
|
||||||
volume=25000000,
|
volume=25000000,
|
||||||
@@ -373,7 +404,7 @@ def test_alert_timestamp_filtering(test_db_session):
|
|||||||
"""Test filtering alerts by timestamp."""
|
"""Test filtering alerts by timestamp."""
|
||||||
session = test_db_session
|
session = test_db_session
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
# Create alerts at different times
|
# Create alerts at different times
|
||||||
old_alert = MarketAlert(
|
old_alert = MarketAlert(
|
||||||
@@ -404,4 +435,3 @@ def test_alert_timestamp_filtering(test_db_session):
|
|||||||
alerts_30_days = monitor.get_recent_alerts(days=30)
|
alerts_30_days = monitor.get_recent_alerts(days=30)
|
||||||
test_alerts = [a for a in alerts_30_days if a.ticker.startswith("TEST")]
|
test_alerts = [a for a in alerts_30_days if a.ticker.startswith("TEST")]
|
||||||
assert len(test_alerts) == 2
|
assert len(test_alerts) == 2
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,12 @@
|
|||||||
"""Tests for pattern detection module."""
|
"""Tests for pattern detection module."""
|
||||||
|
|
||||||
import pytest
|
from datetime import UTC, date, datetime, timedelta
|
||||||
from datetime import date, datetime, timedelta, timezone
|
|
||||||
from decimal import Decimal
|
from decimal import Decimal
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pote.db.models import MarketAlert, Official, Security, Trade
|
||||||
from pote.monitoring.pattern_detector import PatternDetector
|
from pote.monitoring.pattern_detector import PatternDetector
|
||||||
from pote.db.models import Official, Security, Trade, MarketAlert
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -31,7 +32,7 @@ def multiple_officials_with_patterns(test_db_session):
|
|||||||
|
|
||||||
# Pelosi - Suspicious pattern (trades with alerts)
|
# Pelosi - Suspicious pattern (trades with alerts)
|
||||||
for i in range(5):
|
for i in range(5):
|
||||||
trade_date = date(2024, 1, 15) + timedelta(days=i*30)
|
trade_date = date(2024, 1, 15) + timedelta(days=i * 30)
|
||||||
|
|
||||||
# Create trade
|
# Create trade
|
||||||
trade = Trade(
|
trade = Trade(
|
||||||
@@ -52,16 +53,15 @@ def multiple_officials_with_patterns(test_db_session):
|
|||||||
ticker="NVDA",
|
ticker="NVDA",
|
||||||
alert_type="unusual_volume",
|
alert_type="unusual_volume",
|
||||||
timestamp=datetime.combine(
|
timestamp=datetime.combine(
|
||||||
trade_date - timedelta(days=3+j),
|
trade_date - timedelta(days=3 + j), datetime.min.time()
|
||||||
datetime.min.time()
|
).replace(tzinfo=UTC),
|
||||||
).replace(tzinfo=timezone.utc),
|
|
||||||
severity=7 + j,
|
severity=7 + j,
|
||||||
)
|
)
|
||||||
session.add(alert)
|
session.add(alert)
|
||||||
|
|
||||||
# Tuberville - Mixed pattern
|
# Tuberville - Mixed pattern
|
||||||
for i in range(4):
|
for i in range(4):
|
||||||
trade_date = date(2024, 2, 1) + timedelta(days=i*30)
|
trade_date = date(2024, 2, 1) + timedelta(days=i * 30)
|
||||||
|
|
||||||
trade = Trade(
|
trade = Trade(
|
||||||
official_id=tuberville.id,
|
official_id=tuberville.id,
|
||||||
@@ -81,16 +81,15 @@ def multiple_officials_with_patterns(test_db_session):
|
|||||||
ticker="MSFT",
|
ticker="MSFT",
|
||||||
alert_type="price_spike",
|
alert_type="price_spike",
|
||||||
timestamp=datetime.combine(
|
timestamp=datetime.combine(
|
||||||
trade_date - timedelta(days=5),
|
trade_date - timedelta(days=5), datetime.min.time()
|
||||||
datetime.min.time()
|
).replace(tzinfo=UTC),
|
||||||
).replace(tzinfo=timezone.utc),
|
|
||||||
severity=6,
|
severity=6,
|
||||||
)
|
)
|
||||||
session.add(alert)
|
session.add(alert)
|
||||||
|
|
||||||
# Clean trader - No suspicious activity
|
# Clean trader - No suspicious activity
|
||||||
for i in range(3):
|
for i in range(3):
|
||||||
trade_date = date(2024, 3, 1) + timedelta(days=i*30)
|
trade_date = date(2024, 3, 1) + timedelta(days=i * 30)
|
||||||
|
|
||||||
trade = Trade(
|
trade = Trade(
|
||||||
official_id=clean_trader.id,
|
official_id=clean_trader.id,
|
||||||
@@ -141,8 +140,7 @@ def test_identify_repeat_offenders(test_db_session, multiple_officials_with_patt
|
|||||||
|
|
||||||
# Set low threshold to catch Pelosi (who has 100% suspicious rate)
|
# Set low threshold to catch Pelosi (who has 100% suspicious rate)
|
||||||
offenders = detector.identify_repeat_offenders(
|
offenders = detector.identify_repeat_offenders(
|
||||||
lookback_days=3650,
|
lookback_days=3650, min_suspicious_rate=0.7 # 70%+
|
||||||
min_suspicious_rate=0.7 # 70%+
|
|
||||||
)
|
)
|
||||||
|
|
||||||
# Should find at least Pelosi (all trades with alerts)
|
# Should find at least Pelosi (all trades with alerts)
|
||||||
@@ -158,10 +156,7 @@ def test_analyze_ticker_patterns(test_db_session, multiple_officials_with_patter
|
|||||||
session = test_db_session
|
session = test_db_session
|
||||||
detector = PatternDetector(session)
|
detector = PatternDetector(session)
|
||||||
|
|
||||||
ticker_patterns = detector.analyze_ticker_patterns(
|
ticker_patterns = detector.analyze_ticker_patterns(lookback_days=3650, min_trades=3)
|
||||||
lookback_days=3650,
|
|
||||||
min_trades=3
|
|
||||||
)
|
|
||||||
|
|
||||||
assert isinstance(ticker_patterns, list)
|
assert isinstance(ticker_patterns, list)
|
||||||
assert len(ticker_patterns) >= 1 # At least NVDA should qualify
|
assert len(ticker_patterns) >= 1 # At least NVDA should qualify
|
||||||
@@ -208,7 +203,7 @@ def test_get_party_comparison(test_db_session, multiple_officials_with_patterns)
|
|||||||
assert len(party_stats) >= 2 # Democrat, Republican, Independent
|
assert len(party_stats) >= 2 # Democrat, Republican, Independent
|
||||||
|
|
||||||
# Check that we have data for each party
|
# Check that we have data for each party
|
||||||
for party, stats in party_stats.items():
|
for stats in party_stats.values():
|
||||||
assert "official_count" in stats
|
assert "official_count" in stats
|
||||||
assert "total_trades" in stats
|
assert "total_trades" in stats
|
||||||
assert "avg_timing_score" in stats
|
assert "avg_timing_score" in stats
|
||||||
@@ -322,5 +317,3 @@ def test_party_stats_completeness(test_db_session, multiple_officials_with_patte
|
|||||||
assert dem["official_count"] >= 1
|
assert dem["official_count"] >= 1
|
||||||
assert dem["total_trades"] >= 5 # Pelosi has 5 trades
|
assert dem["total_trades"] >= 5 # Pelosi has 5 trades
|
||||||
assert dem["total_suspicious"] > 0 # Pelosi has suspicious trades
|
assert dem["total_suspicious"] > 0 # Pelosi has suspicious trades
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user