POTE/scripts/setup_cron.sh
ilia 0d8d85adc1 Add complete automation, reporting, and CI/CD system
Features Added:
==============

📧 EMAIL REPORTING SYSTEM:
- EmailReporter: Send reports via SMTP (Gmail, SendGrid, custom)
- ReportGenerator: Generate daily/weekly summaries with HTML/text formatting
- Configurable via .env (SMTP_HOST, SMTP_PORT, etc.)
- Scripts: send_daily_report.py, send_weekly_report.py

🤖 AUTOMATED RUNS:
- automated_daily_run.sh: Full daily ETL pipeline + reporting
- automated_weekly_run.sh: Weekly pattern analysis + reports
- setup_cron.sh: Interactive cron job setup (5-minute setup)
- Logs saved to ~/logs/ with automatic cleanup

🔍 HEALTH CHECKS:
- health_check.py: System health monitoring
- Checks: DB connection, data freshness, counts, recent alerts
- JSON output for programmatic use
- Exit codes for monitoring integration

🚀 CI/CD PIPELINE:
- .github/workflows/ci.yml: Full CI/CD pipeline
- GitHub Actions / Gitea Actions compatible
- Jobs: lint & test, security scan, dependency scan, Docker build
- PostgreSQL service for integration tests
- 93 tests passing in CI

📚 COMPREHENSIVE DOCUMENTATION:
- AUTOMATION_QUICKSTART.md: 5-minute email setup guide
- docs/12_automation_and_reporting.md: Full automation guide
- Updated README.md with automation links
- Deployment → Production workflow guide

🛠️ IMPROVEMENTS:
- All shell scripts made executable
- Environment variable examples in .env.example
- Report logs saved with timestamps
- 30-day log retention with auto-cleanup
- Health checks can be scheduled via cron

WHAT THIS ENABLES:
==================
After deployment, users can:
1. Set up automated daily/weekly email reports (5 min)
2. Receive HTML+text emails with:
   - New trades, market alerts, suspicious timing
   - Weekly patterns, rankings, repeat offenders
3. Monitor system health automatically
4. Run full CI/CD pipeline on every commit
5. Deploy with confidence (tests + security scans)

USAGE:
======
# One-time setup (on deployed server)
./scripts/setup_cron.sh

# Or manually send reports
python scripts/send_daily_report.py --to user@example.com
python scripts/send_weekly_report.py --to user@example.com

# Check system health
python scripts/health_check.py

See AUTOMATION_QUICKSTART.md for full instructions.

93 tests passing | Full CI/CD | Email reports ready
2025-12-15 15:34:31 -05:00

131 lines
3.7 KiB
Bash
Executable File

#!/bin/bash
# Setup Cron Jobs for POTE Automation
#
# This script sets up automated daily and weekly runs
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
echo "==============================================="
echo "POTE Cron Setup"
echo "==============================================="
# Ensure scripts are executable
chmod +x "$SCRIPT_DIR/automated_daily_run.sh"
chmod +x "$SCRIPT_DIR/automated_weekly_run.sh"
# Create logs directory
mkdir -p "$HOME/logs"
# Backup existing crontab
echo "Backing up existing crontab..."
crontab -l > "$HOME/crontab.backup.$(date +%Y%m%d)" 2>/dev/null || true
# Check if POTE cron jobs already exist
if crontab -l 2>/dev/null | grep -q "POTE Automated"; then
echo ""
echo "⚠️ POTE cron jobs already exist!"
echo ""
echo "Current POTE cron jobs:"
crontab -l | grep -A 1 "POTE Automated" || true
echo ""
read -p "Do you want to replace them? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Cancelled. No changes made."
exit 0
fi
# Remove existing POTE cron jobs
crontab -l | grep -v "POTE Automated" | grep -v "automated_daily_run.sh" | grep -v "automated_weekly_run.sh" | crontab -
fi
# Get user's email for reports
echo ""
read -p "Enter email address for daily reports: " REPORT_EMAIL
if [ -z "$REPORT_EMAIL" ]; then
echo "ERROR: Email address is required"
exit 1
fi
# Update .env file with report recipient
if [ -f "$PROJECT_ROOT/.env" ]; then
if grep -q "^REPORT_RECIPIENTS=" "$PROJECT_ROOT/.env"; then
# Update existing
sed -i "s/^REPORT_RECIPIENTS=.*/REPORT_RECIPIENTS=$REPORT_EMAIL/" "$PROJECT_ROOT/.env"
else
# Add new
echo "REPORT_RECIPIENTS=$REPORT_EMAIL" >> "$PROJECT_ROOT/.env"
fi
else
echo "ERROR: .env file not found at $PROJECT_ROOT/.env"
echo "Please copy .env.example to .env and configure it first."
exit 1
fi
# Choose schedule
echo ""
echo "Daily report schedule options:"
echo "1) 6:00 AM (after US market close, typical)"
echo "2) 9:00 AM"
echo "3) Custom time"
read -p "Choose option (1-3): " SCHEDULE_OPTION
case $SCHEDULE_OPTION in
1)
DAILY_CRON="0 6 * * *"
;;
2)
DAILY_CRON="0 9 * * *"
;;
3)
read -p "Enter hour (0-23): " HOUR
read -p "Enter minute (0-59): " MINUTE
DAILY_CRON="$MINUTE $HOUR * * *"
;;
*)
echo "Invalid option. Using default (6:00 AM)"
DAILY_CRON="0 6 * * *"
;;
esac
WEEKLY_CRON="0 8 * * 0" # Sunday at 8 AM
# Add new cron jobs
echo ""
echo "Adding cron jobs..."
(crontab -l 2>/dev/null; echo "# POTE Automated Daily Run"; echo "$DAILY_CRON $SCRIPT_DIR/automated_daily_run.sh >> $HOME/logs/daily_run.log 2>&1") | crontab -
(crontab -l 2>/dev/null; echo "# POTE Automated Weekly Run"; echo "$WEEKLY_CRON $SCRIPT_DIR/automated_weekly_run.sh >> $HOME/logs/weekly_run.log 2>&1") | crontab -
echo ""
echo "✓ Cron jobs added successfully!"
echo ""
echo "Current crontab:"
crontab -l | grep -A 1 "POTE Automated" || true
echo ""
echo "==============================================="
echo "Setup Complete!"
echo "==============================================="
echo ""
echo "Daily reports will be sent to: $REPORT_EMAIL"
echo "Daily run schedule: $DAILY_CRON"
echo "Weekly run schedule: $WEEKLY_CRON (Sundays at 8 AM)"
echo ""
echo "Logs will be stored in: $HOME/logs/"
echo ""
echo "To view logs:"
echo " tail -f $HOME/logs/daily_run.log"
echo " tail -f $HOME/logs/weekly_run.log"
echo ""
echo "To remove cron jobs:"
echo " crontab -e"
echo " (then delete the POTE lines)"
echo ""
echo "To test now (dry run):"
echo " $SCRIPT_DIR/automated_daily_run.sh"
echo ""