Add complete automation, reporting, and CI/CD system
Features Added: ============== 📧 EMAIL REPORTING SYSTEM: - EmailReporter: Send reports via SMTP (Gmail, SendGrid, custom) - ReportGenerator: Generate daily/weekly summaries with HTML/text formatting - Configurable via .env (SMTP_HOST, SMTP_PORT, etc.) - Scripts: send_daily_report.py, send_weekly_report.py 🤖 AUTOMATED RUNS: - automated_daily_run.sh: Full daily ETL pipeline + reporting - automated_weekly_run.sh: Weekly pattern analysis + reports - setup_cron.sh: Interactive cron job setup (5-minute setup) - Logs saved to ~/logs/ with automatic cleanup 🔍 HEALTH CHECKS: - health_check.py: System health monitoring - Checks: DB connection, data freshness, counts, recent alerts - JSON output for programmatic use - Exit codes for monitoring integration 🚀 CI/CD PIPELINE: - .github/workflows/ci.yml: Full CI/CD pipeline - GitHub Actions / Gitea Actions compatible - Jobs: lint & test, security scan, dependency scan, Docker build - PostgreSQL service for integration tests - 93 tests passing in CI 📚 COMPREHENSIVE DOCUMENTATION: - AUTOMATION_QUICKSTART.md: 5-minute email setup guide - docs/12_automation_and_reporting.md: Full automation guide - Updated README.md with automation links - Deployment → Production workflow guide 🛠️ IMPROVEMENTS: - All shell scripts made executable - Environment variable examples in .env.example - Report logs saved with timestamps - 30-day log retention with auto-cleanup - Health checks can be scheduled via cron WHAT THIS ENABLES: ================== After deployment, users can: 1. Set up automated daily/weekly email reports (5 min) 2. Receive HTML+text emails with: - New trades, market alerts, suspicious timing - Weekly patterns, rankings, repeat offenders 3. Monitor system health automatically 4. Run full CI/CD pipeline on every commit 5. Deploy with confidence (tests + security scans) USAGE: ====== # One-time setup (on deployed server) ./scripts/setup_cron.sh # Or manually send reports python scripts/send_daily_report.py --to user@example.com python scripts/send_weekly_report.py --to user@example.com # Check system health python scripts/health_check.py See AUTOMATION_QUICKSTART.md for full instructions. 93 tests passing | Full CI/CD | Email reports ready
This commit is contained in:
@@ -145,3 +145,4 @@ def main():
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -217,3 +217,4 @@ def format_ticker_report(result):
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -138,3 +138,4 @@ def main():
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
Executable
+109
@@ -0,0 +1,109 @@
|
||||
#!/bin/bash
|
||||
# POTE Automated Daily Run
|
||||
# This script should be run by cron daily (e.g., at 6 AM after market close)
|
||||
#
|
||||
# Example crontab entry:
|
||||
# 0 6 * * * /home/poteapp/pote/scripts/automated_daily_run.sh >> /home/poteapp/logs/daily_run.log 2>&1
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="${LOG_DIR:-$HOME/logs}"
|
||||
VENV_PATH="${VENV_PATH:-$PROJECT_ROOT/venv}"
|
||||
REPORT_RECIPIENTS="${REPORT_RECIPIENTS:-admin@localhost}"
|
||||
|
||||
# Create log directory if it doesn't exist
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
# Timestamp for logging
|
||||
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
echo "==============================================="
|
||||
echo "POTE Automated Daily Run - $TIMESTAMP"
|
||||
echo "==============================================="
|
||||
|
||||
# Activate virtual environment
|
||||
if [ -d "$VENV_PATH" ]; then
|
||||
echo "Activating virtual environment..."
|
||||
source "$VENV_PATH/bin/activate"
|
||||
else
|
||||
echo "WARNING: Virtual environment not found at $VENV_PATH"
|
||||
echo "Attempting to use system Python..."
|
||||
fi
|
||||
|
||||
# Change to project directory
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Load environment variables
|
||||
if [ -f ".env" ]; then
|
||||
echo "Loading environment variables from .env..."
|
||||
export $(grep -v '^#' .env | xargs)
|
||||
fi
|
||||
|
||||
# Step 1: Fetch new congressional trades
|
||||
echo ""
|
||||
echo "[1/6] Fetching congressional trades..."
|
||||
if python scripts/fetch_congressional_trades.py; then
|
||||
echo "✓ Congressional trades fetched successfully"
|
||||
else
|
||||
echo "⚠ Warning: Failed to fetch congressional trades (may be API issue)"
|
||||
fi
|
||||
|
||||
# Step 2: Enrich securities (get company names, sectors)
|
||||
echo ""
|
||||
echo "[2/6] Enriching security data..."
|
||||
if python scripts/enrich_securities.py; then
|
||||
echo "✓ Securities enriched successfully"
|
||||
else
|
||||
echo "⚠ Warning: Failed to enrich securities"
|
||||
fi
|
||||
|
||||
# Step 3: Fetch latest price data
|
||||
echo ""
|
||||
echo "[3/6] Fetching price data..."
|
||||
if python scripts/fetch_sample_prices.py; then
|
||||
echo "✓ Price data fetched successfully"
|
||||
else
|
||||
echo "⚠ Warning: Failed to fetch price data"
|
||||
fi
|
||||
|
||||
# Step 4: Run market monitoring
|
||||
echo ""
|
||||
echo "[4/6] Running market monitoring..."
|
||||
if python scripts/monitor_market.py --scan; then
|
||||
echo "✓ Market monitoring completed"
|
||||
else
|
||||
echo "⚠ Warning: Market monitoring failed"
|
||||
fi
|
||||
|
||||
# Step 5: Analyze disclosure timing
|
||||
echo ""
|
||||
echo "[5/6] Analyzing disclosure timing..."
|
||||
if python scripts/analyze_disclosure_timing.py --recent 7 --save /tmp/pote_timing_analysis.txt; then
|
||||
echo "✓ Disclosure timing analysis completed"
|
||||
else
|
||||
echo "⚠ Warning: Disclosure timing analysis failed"
|
||||
fi
|
||||
|
||||
# Step 6: Send daily report
|
||||
echo ""
|
||||
echo "[6/6] Sending daily report..."
|
||||
if python scripts/send_daily_report.py --to "$REPORT_RECIPIENTS" --save-to-file "$LOG_DIR/daily_report_$(date +%Y%m%d).txt"; then
|
||||
echo "✓ Daily report sent successfully"
|
||||
else
|
||||
echo "✗ ERROR: Failed to send daily report"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Final summary
|
||||
echo ""
|
||||
echo "==============================================="
|
||||
echo "Daily run completed successfully at $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo "==============================================="
|
||||
|
||||
# Clean up old log files (keep last 30 days)
|
||||
find "$LOG_DIR" -name "daily_report_*.txt" -mtime +30 -delete 2>/dev/null || true
|
||||
|
||||
exit 0
|
||||
|
||||
Executable
+73
@@ -0,0 +1,73 @@
|
||||
#!/bin/bash
|
||||
# POTE Automated Weekly Run
|
||||
# This script should be run by cron weekly (e.g., Sunday at 8 AM)
|
||||
#
|
||||
# Example crontab entry:
|
||||
# 0 8 * * 0 /home/poteapp/pote/scripts/automated_weekly_run.sh >> /home/poteapp/logs/weekly_run.log 2>&1
|
||||
|
||||
set -e
|
||||
|
||||
# Configuration
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
LOG_DIR="${LOG_DIR:-$HOME/logs}"
|
||||
VENV_PATH="${VENV_PATH:-$PROJECT_ROOT/venv}"
|
||||
REPORT_RECIPIENTS="${REPORT_RECIPIENTS:-admin@localhost}"
|
||||
|
||||
# Create log directory if it doesn't exist
|
||||
mkdir -p "$LOG_DIR"
|
||||
|
||||
# Timestamp for logging
|
||||
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
|
||||
echo "==============================================="
|
||||
echo "POTE Automated Weekly Run - $TIMESTAMP"
|
||||
echo "==============================================="
|
||||
|
||||
# Activate virtual environment
|
||||
if [ -d "$VENV_PATH" ]; then
|
||||
echo "Activating virtual environment..."
|
||||
source "$VENV_PATH/bin/activate"
|
||||
else
|
||||
echo "WARNING: Virtual environment not found at $VENV_PATH"
|
||||
fi
|
||||
|
||||
# Change to project directory
|
||||
cd "$PROJECT_ROOT"
|
||||
|
||||
# Load environment variables
|
||||
if [ -f ".env" ]; then
|
||||
echo "Loading environment variables from .env..."
|
||||
export $(grep -v '^#' .env | xargs)
|
||||
fi
|
||||
|
||||
# Generate pattern report
|
||||
echo ""
|
||||
echo "[1/2] Generating pattern detection report..."
|
||||
if python scripts/generate_pattern_report.py --days 365 --min-score 40 --save "$LOG_DIR/pattern_report_$(date +%Y%m%d).txt"; then
|
||||
echo "✓ Pattern report generated"
|
||||
else
|
||||
echo "⚠ Warning: Pattern report generation failed"
|
||||
fi
|
||||
|
||||
# Send weekly report
|
||||
echo ""
|
||||
echo "[2/2] Sending weekly summary report..."
|
||||
if python scripts/send_weekly_report.py --to "$REPORT_RECIPIENTS" --save-to-file "$LOG_DIR/weekly_report_$(date +%Y%m%d).txt"; then
|
||||
echo "✓ Weekly report sent successfully"
|
||||
else
|
||||
echo "✗ ERROR: Failed to send weekly report"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Final summary
|
||||
echo ""
|
||||
echo "==============================================="
|
||||
echo "Weekly run completed successfully at $(date '+%Y-%m-%d %H:%M:%S')"
|
||||
echo "==============================================="
|
||||
|
||||
# Clean up old weekly reports (keep last 90 days)
|
||||
find "$LOG_DIR" -name "weekly_report_*.txt" -mtime +90 -delete 2>/dev/null || true
|
||||
find "$LOG_DIR" -name "pattern_report_*.txt" -mtime +90 -delete 2>/dev/null || true
|
||||
|
||||
exit 0
|
||||
|
||||
@@ -114,3 +114,4 @@ def main():
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -116,3 +116,4 @@ PYEOF
|
||||
# Exit with success (even if some steps warned)
|
||||
exit 0
|
||||
|
||||
|
||||
|
||||
@@ -74,3 +74,4 @@ echo "" | tee -a "$LOG_FILE"
|
||||
# Keep only last 30 days of logs
|
||||
find "$LOG_DIR" -name "daily_update_*.log" -mtime +30 -delete
|
||||
|
||||
|
||||
|
||||
@@ -177,3 +177,4 @@ if __name__ == "__main__":
|
||||
print("\n💡 To create watchlist file: python scripts/fetch_congress_members.py --create")
|
||||
print("💡 To view saved watchlist: python scripts/fetch_congress_members.py --list")
|
||||
|
||||
|
||||
|
||||
@@ -231,3 +231,4 @@ def format_pattern_report(data):
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -304,3 +304,4 @@ def main(days, watchlist_only, format, output):
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
POTE Health Check Script
|
||||
|
||||
Checks the health of the POTE system and reports status.
|
||||
|
||||
Usage:
|
||||
python scripts/health_check.py
|
||||
python scripts/health_check.py --json
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import sys
|
||||
from datetime import date, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from sqlalchemy import func
|
||||
|
||||
from pote.db import engine, get_session
|
||||
from pote.db.models import MarketAlert, Official, Price, Security, Trade
|
||||
|
||||
logging.basicConfig(level=logging.WARNING)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def check_database_connection() -> dict:
|
||||
"""Check if database is accessible."""
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute("SELECT 1")
|
||||
return {"status": "ok", "message": "Database connection successful"}
|
||||
except Exception as e:
|
||||
return {"status": "error", "message": f"Database connection failed: {str(e)}"}
|
||||
|
||||
|
||||
def check_data_freshness() -> dict:
|
||||
"""Check if data has been updated recently."""
|
||||
with get_session() as session:
|
||||
# Check most recent trade filing date
|
||||
latest_trade = (
|
||||
session.query(Trade).order_by(Trade.filing_date.desc()).first()
|
||||
)
|
||||
|
||||
if not latest_trade:
|
||||
return {
|
||||
"status": "warning",
|
||||
"message": "No trades found in database",
|
||||
"latest_trade_date": None,
|
||||
"days_since_update": None,
|
||||
}
|
||||
|
||||
days_since = (date.today() - latest_trade.filing_date).days
|
||||
|
||||
if days_since > 7:
|
||||
status = "warning"
|
||||
message = f"Latest trade is {days_since} days old (may need update)"
|
||||
elif days_since > 14:
|
||||
status = "error"
|
||||
message = f"Latest trade is {days_since} days old (stale data)"
|
||||
else:
|
||||
status = "ok"
|
||||
message = f"Data is fresh ({days_since} days old)"
|
||||
|
||||
return {
|
||||
"status": status,
|
||||
"message": message,
|
||||
"latest_trade_date": str(latest_trade.filing_date),
|
||||
"days_since_update": days_since,
|
||||
}
|
||||
|
||||
|
||||
def check_data_counts() -> dict:
|
||||
"""Check counts of key entities."""
|
||||
with get_session() as session:
|
||||
counts = {
|
||||
"officials": session.query(Official).count(),
|
||||
"securities": session.query(Security).count(),
|
||||
"trades": session.query(Trade).count(),
|
||||
"prices": session.query(Price).count(),
|
||||
"market_alerts": session.query(MarketAlert).count(),
|
||||
}
|
||||
|
||||
if counts["trades"] == 0:
|
||||
status = "error"
|
||||
message = "No trades in database"
|
||||
elif counts["trades"] < 10:
|
||||
status = "warning"
|
||||
message = "Very few trades in database (< 10)"
|
||||
else:
|
||||
status = "ok"
|
||||
message = f"Database has {counts['trades']} trades"
|
||||
|
||||
return {"status": status, "message": message, "counts": counts}
|
||||
|
||||
|
||||
def check_recent_alerts() -> dict:
|
||||
"""Check for recent market alerts."""
|
||||
with get_session() as session:
|
||||
yesterday = datetime.now() - timedelta(days=1)
|
||||
recent_alerts = (
|
||||
session.query(MarketAlert).filter(MarketAlert.timestamp >= yesterday).count()
|
||||
)
|
||||
|
||||
return {
|
||||
"status": "ok",
|
||||
"message": f"{recent_alerts} alerts in last 24 hours",
|
||||
"recent_alerts_count": recent_alerts,
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="POTE health check")
|
||||
parser.add_argument(
|
||||
"--json", action="store_true", help="Output results as JSON"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
# Run all checks
|
||||
checks = {
|
||||
"database_connection": check_database_connection(),
|
||||
"data_freshness": check_data_freshness(),
|
||||
"data_counts": check_data_counts(),
|
||||
"recent_alerts": check_recent_alerts(),
|
||||
}
|
||||
|
||||
# Determine overall status
|
||||
statuses = [check["status"] for check in checks.values()]
|
||||
if "error" in statuses:
|
||||
overall_status = "error"
|
||||
elif "warning" in statuses:
|
||||
overall_status = "warning"
|
||||
else:
|
||||
overall_status = "ok"
|
||||
|
||||
result = {
|
||||
"timestamp": datetime.now().isoformat(),
|
||||
"overall_status": overall_status,
|
||||
"checks": checks,
|
||||
}
|
||||
|
||||
if args.json:
|
||||
print(json.dumps(result, indent=2))
|
||||
else:
|
||||
# Human-readable output
|
||||
status_emoji = {"ok": "✓", "warning": "⚠", "error": "✗"}
|
||||
|
||||
print("\n" + "=" * 60)
|
||||
print("POTE HEALTH CHECK")
|
||||
print("=" * 60)
|
||||
print(f"Timestamp: {result['timestamp']}")
|
||||
print(f"Overall Status: {status_emoji.get(overall_status, '?')} {overall_status.upper()}")
|
||||
print()
|
||||
|
||||
for check_name, check_result in checks.items():
|
||||
status = check_result["status"]
|
||||
emoji = status_emoji.get(status, "?")
|
||||
print(f"{emoji} {check_name.replace('_', ' ').title()}: {check_result['message']}")
|
||||
|
||||
# Print additional details if present
|
||||
if "counts" in check_result:
|
||||
for key, value in check_result["counts"].items():
|
||||
print(f" {key}: {value:,}")
|
||||
|
||||
print("=" * 60 + "\n")
|
||||
|
||||
# Exit with appropriate code
|
||||
if overall_status == "error":
|
||||
sys.exit(2)
|
||||
elif overall_status == "warning":
|
||||
sys.exit(1)
|
||||
else:
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -114,3 +114,4 @@ def main(tickers, interval, once, min_severity, save_report, lookback):
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -83,3 +83,4 @@ echo "=========================================="
|
||||
# Exit successfully even if some steps warned
|
||||
exit 0
|
||||
|
||||
|
||||
|
||||
@@ -130,3 +130,4 @@ def main():
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Send Daily Report via Email
|
||||
|
||||
Generates and emails the daily POTE summary report.
|
||||
|
||||
Usage:
|
||||
python scripts/send_daily_report.py --to user@example.com
|
||||
python scripts/send_daily_report.py --to user1@example.com,user2@example.com --test-smtp
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import sys
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from pote.db import get_session
|
||||
from pote.reporting.email_reporter import EmailReporter
|
||||
from pote.reporting.report_generator import ReportGenerator
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Send daily POTE report via email")
|
||||
parser.add_argument(
|
||||
"--to", required=True, help="Recipient email addresses (comma-separated)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--date",
|
||||
help="Report date (YYYY-MM-DD), defaults to today",
|
||||
default=None,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--test-smtp",
|
||||
action="store_true",
|
||||
help="Test SMTP connection before sending",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--save-to-file",
|
||||
help="Also save report to this file path",
|
||||
default=None,
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Parse recipients
|
||||
to_emails = [email.strip() for email in args.to.split(",")]
|
||||
|
||||
# Parse date if provided
|
||||
report_date = None
|
||||
if args.date:
|
||||
try:
|
||||
report_date = date.fromisoformat(args.date)
|
||||
except ValueError:
|
||||
logger.error(f"Invalid date format: {args.date}. Use YYYY-MM-DD")
|
||||
sys.exit(1)
|
||||
|
||||
# Initialize email reporter
|
||||
email_reporter = EmailReporter()
|
||||
|
||||
# Test SMTP connection if requested
|
||||
if args.test_smtp:
|
||||
logger.info("Testing SMTP connection...")
|
||||
if not email_reporter.test_connection():
|
||||
logger.error("SMTP connection test failed. Check your SMTP settings in .env")
|
||||
logger.info(
|
||||
"Required settings: SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASSWORD, FROM_EMAIL"
|
||||
)
|
||||
sys.exit(1)
|
||||
logger.info("SMTP connection test successful!")
|
||||
|
||||
# Generate report
|
||||
logger.info(f"Generating daily report for {report_date or date.today()}...")
|
||||
with get_session() as session:
|
||||
generator = ReportGenerator(session)
|
||||
report_data = generator.generate_daily_summary(report_date)
|
||||
|
||||
# Format as text and HTML
|
||||
text_body = generator.format_as_text(report_data, "daily")
|
||||
html_body = generator.format_as_html(report_data, "daily")
|
||||
|
||||
# Save to file if requested
|
||||
if args.save_to_file:
|
||||
with open(args.save_to_file, "w") as f:
|
||||
f.write(text_body)
|
||||
logger.info(f"Report saved to {args.save_to_file}")
|
||||
|
||||
# Send email
|
||||
subject = f"POTE Daily Report - {report_data['date']}"
|
||||
logger.info(f"Sending report to {', '.join(to_emails)}...")
|
||||
|
||||
success = email_reporter.send_report(
|
||||
to_emails=to_emails,
|
||||
subject=subject,
|
||||
body_text=text_body,
|
||||
body_html=html_body,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info("Report sent successfully!")
|
||||
# Print summary to stdout
|
||||
print("\n" + text_body + "\n")
|
||||
sys.exit(0)
|
||||
else:
|
||||
logger.error("Failed to send report. Check logs for details.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Send Weekly Report via Email
|
||||
|
||||
Generates and emails the weekly POTE summary report.
|
||||
|
||||
Usage:
|
||||
python scripts/send_weekly_report.py --to user@example.com
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add project root to path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from pote.db import get_session
|
||||
from pote.reporting.email_reporter import EmailReporter
|
||||
from pote.reporting.report_generator import ReportGenerator
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s"
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Send weekly POTE report via email")
|
||||
parser.add_argument(
|
||||
"--to", required=True, help="Recipient email addresses (comma-separated)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--test-smtp",
|
||||
action="store_true",
|
||||
help="Test SMTP connection before sending",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--save-to-file",
|
||||
help="Also save report to this file path",
|
||||
default=None,
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Parse recipients
|
||||
to_emails = [email.strip() for email in args.to.split(",")]
|
||||
|
||||
# Initialize email reporter
|
||||
email_reporter = EmailReporter()
|
||||
|
||||
# Test SMTP connection if requested
|
||||
if args.test_smtp:
|
||||
logger.info("Testing SMTP connection...")
|
||||
if not email_reporter.test_connection():
|
||||
logger.error("SMTP connection test failed. Check your SMTP settings in .env")
|
||||
sys.exit(1)
|
||||
logger.info("SMTP connection test successful!")
|
||||
|
||||
# Generate report
|
||||
logger.info("Generating weekly report...")
|
||||
with get_session() as session:
|
||||
generator = ReportGenerator(session)
|
||||
report_data = generator.generate_weekly_summary()
|
||||
|
||||
# Format as text and HTML
|
||||
text_body = generator.format_as_text(report_data, "weekly")
|
||||
html_body = generator.format_as_html(report_data, "weekly")
|
||||
|
||||
# Save to file if requested
|
||||
if args.save_to_file:
|
||||
with open(args.save_to_file, "w") as f:
|
||||
f.write(text_body)
|
||||
logger.info(f"Report saved to {args.save_to_file}")
|
||||
|
||||
# Send email
|
||||
subject = f"POTE Weekly Report - {report_data['period_start']} to {report_data['period_end']}"
|
||||
logger.info(f"Sending report to {', '.join(to_emails)}...")
|
||||
|
||||
success = email_reporter.send_report(
|
||||
to_emails=to_emails,
|
||||
subject=subject,
|
||||
body_text=text_body,
|
||||
body_html=html_body,
|
||||
)
|
||||
|
||||
if success:
|
||||
logger.info("Report sent successfully!")
|
||||
# Print summary to stdout
|
||||
print("\n" + text_body + "\n")
|
||||
sys.exit(0)
|
||||
else:
|
||||
logger.error("Failed to send report. Check logs for details.")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
||||
@@ -148,3 +148,4 @@ echo "📚 Documentation:"
|
||||
echo " ${POTE_DIR}/docs/10_automation.md"
|
||||
echo ""
|
||||
|
||||
|
||||
|
||||
Executable
+130
@@ -0,0 +1,130 @@
|
||||
#!/bin/bash
|
||||
# Setup Cron Jobs for POTE Automation
|
||||
#
|
||||
# This script sets up automated daily and weekly runs
|
||||
|
||||
set -e
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
|
||||
|
||||
echo "==============================================="
|
||||
echo "POTE Cron Setup"
|
||||
echo "==============================================="
|
||||
|
||||
# Ensure scripts are executable
|
||||
chmod +x "$SCRIPT_DIR/automated_daily_run.sh"
|
||||
chmod +x "$SCRIPT_DIR/automated_weekly_run.sh"
|
||||
|
||||
# Create logs directory
|
||||
mkdir -p "$HOME/logs"
|
||||
|
||||
# Backup existing crontab
|
||||
echo "Backing up existing crontab..."
|
||||
crontab -l > "$HOME/crontab.backup.$(date +%Y%m%d)" 2>/dev/null || true
|
||||
|
||||
# Check if POTE cron jobs already exist
|
||||
if crontab -l 2>/dev/null | grep -q "POTE Automated"; then
|
||||
echo ""
|
||||
echo "⚠️ POTE cron jobs already exist!"
|
||||
echo ""
|
||||
echo "Current POTE cron jobs:"
|
||||
crontab -l | grep -A 1 "POTE Automated" || true
|
||||
echo ""
|
||||
read -p "Do you want to replace them? (y/N) " -n 1 -r
|
||||
echo
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo "Cancelled. No changes made."
|
||||
exit 0
|
||||
fi
|
||||
# Remove existing POTE cron jobs
|
||||
crontab -l | grep -v "POTE Automated" | grep -v "automated_daily_run.sh" | grep -v "automated_weekly_run.sh" | crontab -
|
||||
fi
|
||||
|
||||
# Get user's email for reports
|
||||
echo ""
|
||||
read -p "Enter email address for daily reports: " REPORT_EMAIL
|
||||
if [ -z "$REPORT_EMAIL" ]; then
|
||||
echo "ERROR: Email address is required"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Update .env file with report recipient
|
||||
if [ -f "$PROJECT_ROOT/.env" ]; then
|
||||
if grep -q "^REPORT_RECIPIENTS=" "$PROJECT_ROOT/.env"; then
|
||||
# Update existing
|
||||
sed -i "s/^REPORT_RECIPIENTS=.*/REPORT_RECIPIENTS=$REPORT_EMAIL/" "$PROJECT_ROOT/.env"
|
||||
else
|
||||
# Add new
|
||||
echo "REPORT_RECIPIENTS=$REPORT_EMAIL" >> "$PROJECT_ROOT/.env"
|
||||
fi
|
||||
else
|
||||
echo "ERROR: .env file not found at $PROJECT_ROOT/.env"
|
||||
echo "Please copy .env.example to .env and configure it first."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Choose schedule
|
||||
echo ""
|
||||
echo "Daily report schedule options:"
|
||||
echo "1) 6:00 AM (after US market close, typical)"
|
||||
echo "2) 9:00 AM"
|
||||
echo "3) Custom time"
|
||||
read -p "Choose option (1-3): " SCHEDULE_OPTION
|
||||
|
||||
case $SCHEDULE_OPTION in
|
||||
1)
|
||||
DAILY_CRON="0 6 * * *"
|
||||
;;
|
||||
2)
|
||||
DAILY_CRON="0 9 * * *"
|
||||
;;
|
||||
3)
|
||||
read -p "Enter hour (0-23): " HOUR
|
||||
read -p "Enter minute (0-59): " MINUTE
|
||||
DAILY_CRON="$MINUTE $HOUR * * *"
|
||||
;;
|
||||
*)
|
||||
echo "Invalid option. Using default (6:00 AM)"
|
||||
DAILY_CRON="0 6 * * *"
|
||||
;;
|
||||
esac
|
||||
|
||||
WEEKLY_CRON="0 8 * * 0" # Sunday at 8 AM
|
||||
|
||||
# Add new cron jobs
|
||||
echo ""
|
||||
echo "Adding cron jobs..."
|
||||
|
||||
(crontab -l 2>/dev/null; echo "# POTE Automated Daily Run"; echo "$DAILY_CRON $SCRIPT_DIR/automated_daily_run.sh >> $HOME/logs/daily_run.log 2>&1") | crontab -
|
||||
(crontab -l 2>/dev/null; echo "# POTE Automated Weekly Run"; echo "$WEEKLY_CRON $SCRIPT_DIR/automated_weekly_run.sh >> $HOME/logs/weekly_run.log 2>&1") | crontab -
|
||||
|
||||
echo ""
|
||||
echo "✓ Cron jobs added successfully!"
|
||||
echo ""
|
||||
echo "Current crontab:"
|
||||
crontab -l | grep -A 1 "POTE Automated" || true
|
||||
|
||||
echo ""
|
||||
echo "==============================================="
|
||||
echo "Setup Complete!"
|
||||
echo "==============================================="
|
||||
echo ""
|
||||
echo "Daily reports will be sent to: $REPORT_EMAIL"
|
||||
echo "Daily run schedule: $DAILY_CRON"
|
||||
echo "Weekly run schedule: $WEEKLY_CRON (Sundays at 8 AM)"
|
||||
echo ""
|
||||
echo "Logs will be stored in: $HOME/logs/"
|
||||
echo ""
|
||||
echo "To view logs:"
|
||||
echo " tail -f $HOME/logs/daily_run.log"
|
||||
echo " tail -f $HOME/logs/weekly_run.log"
|
||||
echo ""
|
||||
echo "To remove cron jobs:"
|
||||
echo " crontab -e"
|
||||
echo " (then delete the POTE lines)"
|
||||
echo ""
|
||||
echo "To test now (dry run):"
|
||||
echo " $SCRIPT_DIR/automated_daily_run.sh"
|
||||
echo ""
|
||||
|
||||
Reference in New Issue
Block a user