204cd0e75b
- PR1: Project scaffold, DB models, price loader - PR2: Congressional trade ingestion (House Stock Watcher) - PR3: Security enrichment + deployment infrastructure - 37 passing tests, 87%+ coverage - Docker + Proxmox deployment ready - Complete documentation - Works 100% offline with fixtures
81 lines
2.8 KiB
Markdown
81 lines
2.8 KiB
Markdown
# PR1 Summary: Project Scaffold + DB + Price Loader
|
|
|
|
**Status**: ✅ Complete
|
|
**Date**: 2025-12-13
|
|
|
|
## What was built
|
|
|
|
### 1. Project scaffold
|
|
- `pyproject.toml` with all dependencies (SQLAlchemy, Alembic, yfinance, pandas, pytest, ruff, black, etc.)
|
|
- `src/pote/` layout with config, db, and ingestion modules
|
|
- `.gitignore`, `.env.example`, `Makefile` for dev workflow
|
|
- Docs: `README.md` + 6 `.md` files in `docs/` covering MVP, architecture, schema, sources, safety/ethics, and dev setup
|
|
|
|
### 2. Database models (SQLAlchemy 2.0)
|
|
- **Officials**: Congress members (name, chamber, party, state, bioguide_id)
|
|
- **Securities**: stocks/bonds (ticker, name, exchange, sector)
|
|
- **Trades**: disclosed transactions (official_id, security_id, transaction_date, filing_date, side, value ranges)
|
|
- **Prices**: daily OHLCV (security_id, date, open/high/low/close/volume)
|
|
- **Metrics stubs**: `metrics_official` and `metrics_trade` (Phase 2)
|
|
|
|
Includes proper indexes, unique constraints, and relationships.
|
|
|
|
### 3. Alembic migrations
|
|
- Initialized Alembic with `env.py` wired to our config
|
|
- Generated and applied initial migration (`66fd166195e8`)
|
|
- DB file: `pote.db` (SQLite for dev)
|
|
|
|
### 4. Price loader (`PriceLoader`)
|
|
- Fetches daily price data from **yfinance**
|
|
- Idempotent: skips existing dates, resumes from gaps
|
|
- Upsert logic (insert or update on conflict)
|
|
- Handles single ticker or bulk fetches
|
|
- Logging + basic error handling
|
|
|
|
### 5. Tests (pytest)
|
|
- `tests/conftest.py`: fixtures for in-memory DB, sample officials/securities/trades/prices
|
|
- `tests/test_models.py`: model creation, relationships, unique constraints, queries (7 tests)
|
|
- `tests/test_price_loader.py`: loader logic, idempotency, upsert, mocking yfinance (8 tests)
|
|
- **Result**: 15 tests, all passing ✅
|
|
|
|
### 6. Tooling
|
|
- **Black** + **ruff** configured and run (all code formatted + linted)
|
|
- `Makefile` with targets: `install`, `test`, `lint`, `format`, `migrate`, `clean`
|
|
- Smoke-test script: `scripts/fetch_sample_prices.py` (verified live with AAPL/MSFT/TSLA)
|
|
|
|
## What works now
|
|
- You can spin up the DB, run migrations, fetch price data, and query it
|
|
- All core Phase 1 foundations are in place
|
|
- Tests confirm models and ingestion work correctly
|
|
|
|
## Next steps (PR2+)
|
|
Per `docs/00_mvp.md`:
|
|
- **PR2**: QuiverQuant or FMP client for Congress trades
|
|
- **PR3**: ETL job to populate `officials` and `trades` tables
|
|
- **PR4+**: Analytics (abnormal returns, clustering, signals)
|
|
|
|
## How to run
|
|
```bash
|
|
# Install
|
|
make install
|
|
source venv/bin/activate
|
|
|
|
# Run migrations
|
|
make migrate
|
|
|
|
# Fetch sample prices
|
|
python scripts/fetch_sample_prices.py
|
|
|
|
# Run tests
|
|
make test
|
|
|
|
# Lint + format
|
|
make lint
|
|
make format
|
|
```
|
|
|
|
---
|
|
|
|
**Research-only reminder**: This tool is for transparency and descriptive analytics using public data. Not investment advice.
|
|
|