508 lines
15 KiB
Markdown
508 lines
15 KiB
Markdown
# Ansible Infrastructure Management
|
|
|
|
This Ansible project provides comprehensive infrastructure automation for development environments, server management, and VM provisioning across multiple machines and platforms.
|
|
|
|
## 🏗️ Architecture
|
|
|
|
### Host Groups
|
|
- `dev`: Development machines (dev01, bottom, debianDesktopVM)
|
|
- `gitea`: Gitea server (Alpine Linux)
|
|
- `portainer`: Portainer container management (Alpine Linux)
|
|
- `homepage`: Homepage dashboard (Debian)
|
|
- `ansible`: Ansible control node
|
|
- `local`: Local machine management
|
|
|
|
### Roles
|
|
|
|
#### Core Infrastructure Roles
|
|
- **`maintenance`**: System updates, package cleanup, and automated reboots
|
|
- **`base`**: Core system packages, security tools, and system hardening
|
|
- **`ssh`**: SSH server hardening and firewall configuration
|
|
- **`user`**: User management and configuration
|
|
|
|
#### Development & Shell Roles
|
|
- **`development`**: Development tools (git, nodejs, build-essential, python3)
|
|
- **`shell`**: Shell configuration (zsh + oh-my-zsh + powerlevel10k)
|
|
- **`docker`**: Docker CE installation and user configuration
|
|
|
|
#### Application Roles
|
|
- **`applications`**: Desktop applications (Brave, LibreOffice, Redshift, Evince)
|
|
- **`snap`**: Snap daemon and snap applications (VSCode, Cursor)
|
|
|
|
#### Network & Monitoring Roles
|
|
- **`tailscale`**: VPN mesh networking across all machines
|
|
- **`monitoring`**: System monitoring tools and scripts
|
|
- **`backup`**: Automated backup solutions (✨ NEW)
|
|
|
|
#### Infrastructure Roles
|
|
- **`proxmox_vm`**: Proxmox VM creation and management (✨ NEW)
|
|
|
|
## 🚀 Usage
|
|
|
|
### Quick Start with Makefile (Recommended)
|
|
```bash
|
|
# Setup dependencies
|
|
make bootstrap
|
|
|
|
# Test everything
|
|
make test
|
|
|
|
# Dry run to see what would change
|
|
make check
|
|
|
|
# Apply to all development hosts
|
|
make apply
|
|
|
|
# Run on specific host
|
|
make dev HOST=dev01
|
|
|
|
# Run locally
|
|
make local
|
|
```
|
|
|
|
### New Infrastructure Features
|
|
|
|
#### Proxmox VM Creation (✨ NEW)
|
|
```bash
|
|
# Create new VMs on Proxmox
|
|
make create-vm
|
|
|
|
# Or manually:
|
|
ansible-playbook proxmox-create-vm.yml
|
|
```
|
|
|
|
#### Automated Backups (✨ NEW)
|
|
```bash
|
|
# Deploy backup system
|
|
make backup
|
|
|
|
# Includes:
|
|
# - Daily home directory backups (2:00 AM)
|
|
# - Daily system config backups (2:30 AM)
|
|
# - 7-day retention for home, 30-day for system
|
|
# - Automated cleanup and logging
|
|
```
|
|
|
|
#### System Monitoring (✨ NEW)
|
|
```bash
|
|
# Deploy monitoring tools
|
|
make monitoring
|
|
|
|
# Includes:
|
|
# - Advanced system monitoring (btop, iotop, nethogs)
|
|
# - Custom monitoring scripts
|
|
# - System information dashboards
|
|
# - Tailscale network status integration
|
|
```
|
|
|
|
#### Tailscale VPN Network
|
|
```bash
|
|
# Deploy Tailscale across all machines
|
|
make tailscale
|
|
|
|
# Check Tailscale status
|
|
make tailscale-status
|
|
|
|
# Deploy to development machines only
|
|
make tailscale-dev
|
|
```
|
|
|
|
### Prerequisites (Manual Setup)
|
|
```bash
|
|
# Install required collections
|
|
ansible-galaxy collection install -r collections/requirements.yml
|
|
```
|
|
|
|
### Vault Password Setup
|
|
Host variables and sensitive data are encrypted with Ansible Vault:
|
|
|
|
#### Option 1: Vault Password File (Recommended)
|
|
```bash
|
|
# Create the vault password file
|
|
echo "your_vault_password" > ~/.ansible-vault-pass
|
|
chmod 600 ~/.ansible-vault-pass
|
|
```
|
|
|
|
#### Option 2: Interactive Password Prompt
|
|
Use `--ask-vault-pass` with each command.
|
|
|
|
### Vault Configuration
|
|
Create vault files with encrypted secrets:
|
|
```bash
|
|
# Create/edit vault files
|
|
make create-vault
|
|
make edit-vault HOST=dev01
|
|
|
|
# Required vault variables:
|
|
# - vault_tailscale_auth_key: "tskey-auth-your-key"
|
|
# - vault_proxmox_host: "proxmox-server-ip"
|
|
# - vault_proxmox_user: "root@pam"
|
|
# - vault_proxmox_password: "proxmox-password"
|
|
# - vault_vm_cipassword: "vm-user-password"
|
|
# - vault_ssh_public_key: "ssh-ed25519 AAAA..."
|
|
```
|
|
|
|
### Selective Execution with Tags
|
|
|
|
#### Using Makefile (Recommended)
|
|
```bash
|
|
# Infrastructure roles
|
|
make security # Security-related roles only
|
|
make monitoring # Monitoring tools only
|
|
make backup # Backup system only
|
|
|
|
# Development tools
|
|
make docker # Docker installation only
|
|
make shell # Shell configuration only
|
|
make apps # Applications only
|
|
|
|
# Network services
|
|
make tailscale # VPN network setup
|
|
make tailscale-status # Check VPN status
|
|
|
|
# Maintenance (unified system)
|
|
make maintenance # All hosts
|
|
make maintenance GROUP=dev # Specific group
|
|
make maintenance HOST=dev01 # Specific host
|
|
make maintenance CHECK=true # Dry-run all hosts
|
|
make maintenance GROUP=dev SERIAL=1 # Serial execution
|
|
|
|
# Infrastructure management
|
|
make create-vm # Create new Proxmox VMs
|
|
make status # Check connectivity
|
|
make facts # Gather system facts
|
|
```
|
|
|
|
#### Manual Commands
|
|
```bash
|
|
# Security-related roles only
|
|
ansible-playbook dev-playbook.yml --tags security
|
|
|
|
# Development tools only
|
|
ansible-playbook dev-playbook.yml --tags development,docker
|
|
|
|
# Network services
|
|
ansible-playbook tailscale-playbook.yml
|
|
|
|
# Infrastructure provisioning
|
|
ansible-playbook proxmox-create-vm.yml
|
|
|
|
# Skip maintenance
|
|
ansible-playbook dev-playbook.yml --skip-tags maintenance
|
|
```
|
|
|
|
### Playbook Overview
|
|
- **`dev-playbook.yml`**: Complete development environment setup
|
|
- **`local-playbook.yml`**: Local machine configuration
|
|
- **`tailscale-playbook.yml`**: VPN network deployment
|
|
- **`proxmox-create-vm.yml`**: VM provisioning on Proxmox
|
|
- **`maintenance-playbook.yml`**: System maintenance operations
|
|
|
|
## 🔧 Configuration
|
|
|
|
### Global Variables (`group_vars/all.yml`)
|
|
- `timezone`: System timezone (default: UTC)
|
|
- `locale`: System locale (default: en_US.UTF-8)
|
|
- `ansible_debug_output`: Show debug information (default: false)
|
|
- `fail2ban_bantime`: Ban duration in seconds
|
|
- `fail2ban_findtime`: Time window for failures
|
|
- `fail2ban_maxretry`: Max failures before ban
|
|
|
|
### Tailscale Configuration
|
|
- `tailscale_auth_key`: Authentication key (stored in vault)
|
|
- `tailscale_accept_routes`: Accept subnet routes (default: true)
|
|
- `tailscale_accept_dns`: Accept DNS settings (default: true)
|
|
- `tailscale_ssh`: Enable SSH access through Tailscale (default: true)
|
|
|
|
### Backup Configuration (`roles/backup/defaults/main.yml`)
|
|
- `backup_enable_cron`: Enable automated backups (default: true)
|
|
- `backup_retention_days_home`: Home backup retention (default: 7)
|
|
- `backup_retention_days_system`: System backup retention (default: 30)
|
|
- `backup_users`: Users to backup (default: ['master', 'beast', 'ladmin', 'user'])
|
|
|
|
### SSH Configuration (`roles/ssh/defaults/main.yml`)
|
|
Comprehensive security hardening:
|
|
- `ssh_port`: SSH port (default: 22)
|
|
- `ssh_permit_root_login`: Root login setting (default: 'no')
|
|
- `ssh_password_authentication`: Password auth (default: 'no')
|
|
- `ssh_max_auth_tries`: Authentication attempts (default: 3)
|
|
- `ssh_allowed_users`: Restrict to specific users (default: [])
|
|
- `ssh_allowed_groups`: Restrict to specific groups (default: ['sudo', 'ssh'])
|
|
|
|
### Proxmox VM Configuration (`roles/proxmox_vm/defaults/main.yml`)
|
|
- `vm_memory`: RAM allocation (default: 8192MB)
|
|
- `vm_cores`: CPU cores (default: 2)
|
|
- `vm_disk_size`: Disk size (default: 20G)
|
|
- `vm_iso`: Ubuntu Server ISO (default: ubuntu-24.04-live-server-amd64.iso)
|
|
- `vm_ciuser`: Default user (default: master)
|
|
|
|
## 🛡️ Security Features
|
|
|
|
### Comprehensive SSH Hardening
|
|
- Modern cryptographic algorithms (ChaCha20-Poly1305, AES-256-GCM)
|
|
- Secure key exchange (Curve25519, DH Group 16)
|
|
- Disabled password authentication
|
|
- Connection rate limiting and timeouts
|
|
- User/group access restrictions
|
|
- Configuration validation and automatic backup
|
|
|
|
### Fail2ban Integration
|
|
- SSH brute force protection
|
|
- Configurable ban times and retry limits
|
|
- Email notifications
|
|
|
|
### UFW Firewall
|
|
- Deny-by-default policy
|
|
- SSH access allowed
|
|
- Automatic enablement
|
|
|
|
### Tailscale VPN Security
|
|
- Zero-trust mesh networking
|
|
- End-to-end encryption
|
|
- SSH access through secure tunnel
|
|
- Subnet routing capabilities
|
|
|
|
## 📦 Installed Packages
|
|
|
|
### Base System
|
|
- **Core utilities**: `curl`, `wget`, `unzip`, `xclip`, `tree`
|
|
- **Network/Security**: `net-tools`, `ufw`, `fail2ban`, `mailutils`
|
|
- **Monitoring**: `iotop`, `nethogs`, `logwatch`, `btop` (via snap)
|
|
- **Modern CLI**: `jq`, `yq` (via snap), `ripgrep`, `fd-find`
|
|
|
|
### Development Tools
|
|
- `git`, `nodejs`, `npm`
|
|
- `build-essential`, `python3`, `python3-pip`
|
|
|
|
### Applications
|
|
- `brave-browser`, `libreoffice`, `evince`, `redshift`
|
|
- `code` (VSCode), `cursor` (via snap)
|
|
|
|
### Docker & Containers
|
|
- Docker CE with all components
|
|
- Docker Compose
|
|
- User added to docker group
|
|
|
|
### Backup Tools (✨ NEW)
|
|
- `rsync`, `borgbackup`, `rclone`, `restic`
|
|
- Automated backup scripts and cron jobs
|
|
|
|
### Monitoring Tools (✨ NEW)
|
|
- `htop`, `iotop`, `nethogs`, `btop`
|
|
- Custom system information scripts
|
|
- Network monitoring utilities
|
|
|
|
### VPN & Network
|
|
- `tailscale` - Mesh VPN networking
|
|
- Network utilities and monitoring
|
|
|
|
## 🔧 Modern CLI Tools
|
|
|
|
The base role installs modern replacements for traditional Unix tools:
|
|
|
|
### Available Commands
|
|
```bash
|
|
# Fast searching
|
|
rg "pattern" files/ # ripgrep - faster than grep
|
|
fd "filename" # fd-find - intuitive find replacement
|
|
|
|
# Data processing
|
|
jq '.key' file.json # JSON processor and formatter
|
|
yq '.key' file.yaml # YAML processor and formatter
|
|
|
|
# System monitoring
|
|
btop # Modern system monitor (better than htop)
|
|
tree directory/ # Directory structure visualization
|
|
|
|
# File operations
|
|
tree -L 2 # Limit tree depth
|
|
rg -i "case insensitive" # Case-insensitive search
|
|
fd -e yml # Find only YAML files
|
|
jq -r '.items[].name' # Raw JSON output
|
|
```
|
|
|
|
## 🔄 Maintenance & Operations
|
|
|
|
### Unified Maintenance System
|
|
```bash
|
|
# Basic usage
|
|
make maintenance # Run on all hosts
|
|
make maintenance GROUP=dev # Run on specific group
|
|
make maintenance HOST=dev01 # Run on specific host
|
|
|
|
# Advanced options
|
|
make maintenance CHECK=true # Dry-run (safe testing)
|
|
make maintenance GROUP=dev SERIAL=1 # One host at a time
|
|
make maintenance GROUP=local # Local machine (auto-sudo)
|
|
```
|
|
|
|
### Backup Operations (✨ NEW)
|
|
```bash
|
|
# Deploy backup system
|
|
make backup
|
|
|
|
# Manual backup operations
|
|
sudo /opt/backups/scripts/backup-home.sh # Run home backup
|
|
sudo /opt/backups/scripts/backup-system.sh # Run system backup
|
|
|
|
# Check backup logs
|
|
tail -f /var/log/backups/home.log
|
|
tail -f /var/log/backups/system.log
|
|
```
|
|
|
|
### Monitoring Operations (✨ NEW)
|
|
```bash
|
|
# Deploy monitoring tools
|
|
make monitoring
|
|
|
|
# Use monitoring scripts
|
|
/usr/local/bin/monitoring/sysinfo # System information dashboard
|
|
/usr/local/bin/monitoring/netinfo # Network information
|
|
|
|
# System monitoring
|
|
btop # Interactive system monitor
|
|
```
|
|
|
|
### Tailscale Network Management
|
|
```bash
|
|
# Deploy VPN network
|
|
make tailscale
|
|
|
|
# Check status across all machines
|
|
make tailscale-status
|
|
|
|
# Manual Tailscale commands
|
|
tailscale status # Check connection status
|
|
tailscale ip # Show Tailscale IP
|
|
tailscale netcheck # Network connectivity check
|
|
```
|
|
|
|
### Infrastructure Provisioning (✨ NEW)
|
|
```bash
|
|
# Create new VMs on Proxmox
|
|
make create-vm
|
|
|
|
# Custom VM creation
|
|
ansible-playbook proxmox-create-vm.yml -e "vm_name=new-server vm_id=111"
|
|
```
|
|
|
|
## 🐛 Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
1. **SSH Connection Issues**
|
|
- Check `ansible.cfg` SSH settings
|
|
- Verify host keys and user permissions
|
|
- Test Tailscale connectivity: `tailscale ping hostname`
|
|
|
|
2. **Vault Access Issues**
|
|
- Verify vault password file: `~/.ansible-vault-pass`
|
|
- Test vault decryption: `ansible-vault view host_vars/hostname.yml`
|
|
|
|
3. **Tailscale Connection Issues**
|
|
- Check service status: `sudo systemctl status tailscaled`
|
|
- Verify auth key in vault
|
|
- Check firewall: `sudo ufw status`
|
|
|
|
4. **Proxmox VM Creation Issues**
|
|
- Verify Proxmox credentials in vault
|
|
- Check ISO availability: `pvesm list local --content iso`
|
|
- Ensure sufficient resources on Proxmox node
|
|
|
|
5. **Backup Issues**
|
|
- Check backup directories: `ls -la /opt/backups/`
|
|
- Review logs: `tail -f /var/log/backups/*.log`
|
|
- Verify cron jobs: `sudo crontab -l`
|
|
|
|
### Debug Commands
|
|
```bash
|
|
# Using Makefile
|
|
make status # Test connectivity to all hosts
|
|
make facts # Gather facts from all hosts
|
|
make debug # Run with debug output
|
|
make verbose # Run with verbose output
|
|
|
|
# Manual commands
|
|
ansible dev -m ping # Test connectivity
|
|
ansible dev -m setup # Check facts
|
|
ansible-playbook dev-playbook.yml --tags base # Run specific role
|
|
|
|
# Verify installations
|
|
ansible dev -m shell -a "tailscale status" # Check Tailscale
|
|
ansible dev -m shell -a "docker --version" # Check Docker
|
|
ansible dev -m shell -a "sudo sshd -t" # Validate SSH config
|
|
```
|
|
|
|
## 🛠️ Makefile Workflows
|
|
|
|
### Development Workflow
|
|
```bash
|
|
make bootstrap # Install collections
|
|
make test # Lint + syntax check
|
|
make check # Dry run
|
|
make apply # Deploy to all hosts
|
|
```
|
|
|
|
### Infrastructure Management
|
|
```bash
|
|
make create-vm # Provision new VMs
|
|
make tailscale # Deploy VPN network
|
|
make monitoring # Deploy monitoring
|
|
make backup # Deploy backup system
|
|
```
|
|
|
|
### Host-Specific Operations
|
|
```bash
|
|
make dev HOST=dev01 # Deploy to specific host
|
|
make edit-vault HOST=dev01 # Edit encrypted host variables
|
|
make tailscale-dev # Deploy Tailscale to dev hosts only
|
|
```
|
|
|
|
### Maintenance and Utilities
|
|
```bash
|
|
make clean # Clean up artifacts
|
|
make status # Check host connectivity
|
|
make install-tools # Install recommended CLI tools locally
|
|
```
|
|
|
|
Run `make help` for the complete list of available commands.
|
|
|
|
## 📝 File Structure
|
|
ansible/
|
|
├── ansible.cfg # Enhanced Ansible configuration
|
|
├── Makefile # Workflow automation with unified maintenance
|
|
├── hosts # Inventory file
|
|
├── dev-playbook.yml # Main development playbook
|
|
├── local-playbook.yml # Local machine setup
|
|
├── tailscale-playbook.yml # VPN network deployment
|
|
├── proxmox-create-vm.yml # VM provisioning playbook
|
|
├── maintenance-playbook.yml # Dedicated maintenance playbook
|
|
├── collections/
|
|
│ └── requirements.yml # Required Ansible collections
|
|
├── group_vars/
|
|
│ └── all.yml # Global variables and Tailscale config
|
|
├── host_vars/ # Host-specific variables (encrypted)
|
|
└── roles/
|
|
├── maintenance/ # System maintenance
|
|
├── base/ # Core system setup
|
|
├── development/ # Development tools
|
|
├── shell/ # Shell configuration (zsh + oh-my-zsh)
|
|
├── docker/ # Docker installation
|
|
├── ssh/ # SSH hardening and configuration
|
|
├── user/ # User management
|
|
├── applications/ # Desktop applications
|
|
├── snap/ # Snap applications
|
|
├── tailscale/ # VPN mesh networking
|
|
├── monitoring/ # System monitoring tools
|
|
├── backup/ # Automated backup solutions
|
|
└── proxmox_vm/ # VM provisioning on Proxmox
|
|
|
|
|
|
## 🤝 Contributing
|
|
|
|
1. Test changes with `--check` first
|
|
2. Update documentation for new roles/tasks
|
|
3. Use proper handlers for service restarts
|
|
4. Follow existing naming conventions
|
|
5. Encrypt sensitive data with ansible-vault
|
|
6. Test across different OS distributions (Ubuntu, Debian, Alpine) |