- scripts/add_custom_trades.py: Manual trade entry - scripts/scrape_alternative_sources.py: CSV import - scripts/daily_update.sh: Automated daily updates - docs/09_data_updates.md: Complete update guide - docs/PR4_PLAN.md: Phase 2 analytics plan Enables users to add representatives and set up auto-updates
7.8 KiB
7.8 KiB
POTE Quick Start Guide
🚀 Your System is Ready!
Container IP: Check with ip addr show eth0 | grep "inet"
Database: PostgreSQL on port 5432
Username: poteuser
Password: changeme123 (⚠️ change in production!)
📊 How to Use POTE
Option 1: Command Line (SSH into container)
# SSH to your container
ssh root@YOUR_CONTAINER_IP
# Switch to poteapp user
su - poteapp
# Activate Python environment
cd pote && source venv/bin/activate
# Now you can run any POTE command!
Option 2: Remote Database Access
Connect from any machine with PostgreSQL client:
psql -h YOUR_CONTAINER_IP -U poteuser -d pote
# Password: changeme123
Option 3: Python Client (From Anywhere)
from sqlalchemy import create_engine, text
# Connect remotely
engine = create_engine("postgresql://poteuser:changeme123@YOUR_CONTAINER_IP:5432/pote")
with engine.connect() as conn:
result = conn.execute(text("SELECT * FROM officials"))
for row in result:
print(row)
🎯 Common Tasks
1. Check System Status
su - poteapp
cd pote && source venv/bin/activate
~/status.sh # Shows current database stats
2. Ingest Sample Data (Offline)
su - poteapp
cd pote && source venv/bin/activate
python scripts/ingest_from_fixtures.py
Output: Ingests 5 sample congressional trades (Nancy Pelosi, etc.)
3. Fetch Live Congressional Trades
su - poteapp
cd pote && source venv/bin/activate
python scripts/fetch_congressional_trades.py
What it does:
- Fetches latest trades from House Stock Watcher API
- Deduplicates against existing trades
- Shows summary of what was added
4. Enrich Securities (Add Company Info)
su - poteapp
cd pote && source venv/bin/activate
python scripts/enrich_securities.py
What it does:
- Fetches company names, sectors, industries from yfinance
- Updates securities table with real company data
5. Fetch Historical Prices
su - poteapp
cd pote && source venv/bin/activate
python scripts/fetch_sample_prices.py
What it does:
- Fetches historical price data for securities in database
- Stores daily OHLCV data for analysis
6. Run Database Queries
# Connect to database
psql -h localhost -U poteuser -d pote
Useful queries:
-- View all officials
SELECT name, chamber, party FROM officials;
-- View all trades
SELECT o.name, s.ticker, t.side, t.amount_min, t.transaction_date
FROM trades t
JOIN officials o ON t.official_id = o.id
JOIN securities s ON t.security_id = s.id
ORDER BY t.transaction_date DESC;
-- Top traders
SELECT o.name, COUNT(t.id) as trade_count
FROM officials o
LEFT JOIN trades t ON o.id = t.official_id
GROUP BY o.id, o.name
ORDER BY trade_count DESC;
-- Trades by ticker
SELECT s.ticker, s.name, COUNT(t.id) as trade_count
FROM securities s
LEFT JOIN trades t ON s.id = t.security_id
GROUP BY s.id, s.ticker, s.name
ORDER BY trade_count DESC;
📈 Example Workflows
Workflow 1: Daily Update
su - poteapp
cd pote && source venv/bin/activate
# Fetch new trades
python scripts/fetch_congressional_trades.py
# Enrich any new securities
python scripts/enrich_securities.py
# Update prices
python scripts/fetch_sample_prices.py
# Check status
~/status.sh
Workflow 2: Research Query
# research.py - Save this in ~/pote/
from sqlalchemy import create_engine, text
from pote.config import settings
engine = create_engine(settings.DATABASE_URL)
# Find all NVDA trades
with engine.connect() as conn:
result = conn.execute(text("""
SELECT
o.name,
o.party,
t.side,
t.amount_min,
t.transaction_date,
t.disclosure_date
FROM trades t
JOIN officials o ON t.official_id = o.id
JOIN securities s ON t.security_id = s.id
WHERE s.ticker = 'NVDA'
ORDER BY t.transaction_date DESC
"""))
for row in result:
print(f"{row.name:20s} | {row.side:8s} | ${row.amount_min:,} | {row.transaction_date}")
Run it:
python research.py
Workflow 3: Export to CSV
# export_trades.py
import pandas as pd
from sqlalchemy import create_engine
from pote.config import settings
engine = create_engine(settings.DATABASE_URL)
# Export all trades to CSV
query = """
SELECT
o.name as official_name,
o.party,
o.chamber,
s.ticker,
s.name as company_name,
t.side,
t.amount_min,
t.amount_max,
t.transaction_date,
t.disclosure_date
FROM trades t
JOIN officials o ON t.official_id = o.id
JOIN securities s ON t.security_id = s.id
ORDER BY t.transaction_date DESC
"""
df = pd.read_sql(query, engine)
df.to_csv('trades_export.csv', index=False)
print(f"Exported {len(df)} trades to trades_export.csv")
🔧 Maintenance
Update POTE Code
su - poteapp
cd pote
git pull
source venv/bin/activate
pip install -e .
Backup Database
# Create backup
su - poteapp
pg_dump -h localhost -U poteuser pote > ~/backups/pote_$(date +%Y%m%d).sql
# Restore backup
psql -h localhost -U poteuser -d pote < ~/backups/pote_20250115.sql
View Logs
# PostgreSQL logs
tail -f /var/log/postgresql/postgresql-15-main.log
# Application logs (if you create them)
tail -f ~/logs/pote.log
Change Database Password
# As root
su - postgres
psql << EOF
ALTER USER poteuser WITH PASSWORD 'your_new_secure_password';
EOF
# Update .env
su - poteapp
nano ~/pote/.env
# Change DATABASE_URL password
🌐 Access Methods Summary
| Method | From Where | Command |
|---|---|---|
| SSH + CLI | Any network client | ssh root@IP, then su - poteapp |
| psql | Any network client | psql -h IP -U poteuser -d pote |
| Python | Any machine | sqlalchemy.create_engine("postgresql://...") |
| Web UI | Coming in Phase 3! | http://IP:8000 (FastAPI + dashboard) |
📚 What Data Do You Have?
Right now (Phase 1 complete):
- ✅ Congressional trading data (from House Stock Watcher)
- ✅ Security information (tickers, names, sectors)
- ✅ Historical prices (OHLCV data from yfinance)
- ✅ Official profiles (name, party, chamber, state)
Coming next (Phase 2):
- 📊 Abnormal return calculations
- 🤖 Behavioral clustering
- 🚨 Research signals (follow_research, avoid_risk, watch)
🎓 Learning SQL for POTE
Count Records
SELECT COUNT(*) FROM officials;
SELECT COUNT(*) FROM trades;
SELECT COUNT(*) FROM securities;
Filter by Party
SELECT name, party FROM officials WHERE party = 'Democrat';
Join Tables
SELECT o.name, s.ticker, t.side
FROM trades t
JOIN officials o ON t.official_id = o.id
JOIN securities s ON t.security_id = s.id
LIMIT 10;
Aggregate Stats
SELECT
o.party,
COUNT(t.id) as trade_count,
AVG(t.amount_min) as avg_amount
FROM trades t
JOIN officials o ON t.official_id = o.id
GROUP BY o.party;
❓ Troubleshooting
Can't connect remotely?
# Check PostgreSQL is listening
ss -tlnp | grep 5432
# Should show: 0.0.0.0:5432
# Check firewall (if enabled)
ufw status
Database connection fails?
# Test locally first
psql -h localhost -U poteuser -d pote
# Check credentials in .env
cat ~/pote/.env
Python import errors?
# Reinstall dependencies
su - poteapp
cd pote && source venv/bin/activate
pip install -e .
🚀 Next Steps
- Populate with real data: Run
fetch_congressional_trades.pyregularly - Set up cron job for automatic daily updates
- Build analytics (Phase 2) - abnormal returns, signals
- Create dashboard (Phase 3) - web interface for exploration
Ready to build Phase 2 analytics? Just ask! 📈