ansible/README.md

Ansible Infrastructure Management

This Ansible project provides comprehensive infrastructure automation for development environments, server management, and VM provisioning across multiple machines and platforms.

🏗️ Architecture

Host Groups

• dev: Development machines (dev01, bottom, debianDesktopVM)
• gitea: Gitea server (Alpine Linux)
• portainer: Portainer container management (Alpine Linux)
• homepage: Homepage dashboard (Debian)
• ansible: Ansible control node
• local: Local machine management

Roles

Core Infrastructure Roles

• maintenance: System updates, package cleanup, and automated reboots
• base: Core system packages, security tools, and system hardening
• ssh: SSH server hardening and firewall configuration
• user: User management and configuration

Development & Shell Roles

• development: Development tools (git, nodejs, build-essential, python3)
• shell: Shell configuration (zsh + oh-my-zsh + powerlevel10k)
• docker: Docker CE installation and user configuration

Application Roles

• applications: Desktop applications (Brave, LibreOffice, Redshift, Evince)
• snap: Snap daemon and snap applications (VSCode, Cursor)

Network & Monitoring Roles

• tailscale: VPN mesh networking across all machines
• monitoring: System monitoring tools and scripts
• backup: Automated backup solutions (✨ NEW)

Infrastructure Roles

• proxmox_vm: Proxmox VM creation and management (✨ NEW)

🚀 Usage

Quick Start with Makefile (Recommended)

# Setup dependencies
make bootstrap

# Test everything
make test

# Dry run to see what would change  
make check

# Apply to all development hosts
make apply

# Run on specific host
make dev HOST=dev01

# Run locally
make local

New Infrastructure Features

Proxmox VM Creation (✨ NEW)

# Create new VMs on Proxmox
make create-vm

# Or manually:
ansible-playbook proxmox-create-vm.yml

Automated Backups (✨ NEW)

# Deploy backup system
make backup

# Includes:
# - Daily home directory backups (2:00 AM)
# - Daily system config backups (2:30 AM)  
# - 7-day retention for home, 30-day for system
# - Automated cleanup and logging

System Monitoring (✨ NEW)

# Deploy monitoring tools
make monitoring

# Includes:
# - Advanced system monitoring (btop, iotop, nethogs)
# - Custom monitoring scripts
# - System information dashboards
# - Tailscale network status integration

Tailscale VPN Network

# Deploy Tailscale across all machines
make tailscale

# Check Tailscale status
make tailscale-status

# Deploy to development machines only
make tailscale-dev

Prerequisites (Manual Setup)

# Install required collections
ansible-galaxy collection install -r collections/requirements.yml

Vault Password Setup

Host variables and sensitive data are encrypted with Ansible Vault:

Option 1: Vault Password File (Recommended)

# Create the vault password file
echo "your_vault_password" > ~/.ansible-vault-pass
chmod 600 ~/.ansible-vault-pass

Option 2: Interactive Password Prompt

Use --ask-vault-pass with each command.

Vault Configuration

Create vault files with encrypted secrets:

# Create/edit vault files
make create-vault
make edit-vault HOST=dev01

# Required vault variables:
# - vault_tailscale_auth_key: "tskey-auth-your-key"
# - vault_proxmox_host: "proxmox-server-ip"
# - vault_proxmox_user: "root@pam"
# - vault_proxmox_password: "proxmox-password"
# - vault_vm_cipassword: "vm-user-password"
# - vault_ssh_public_key: "ssh-ed25519 AAAA..."

Selective Execution with Tags

Using Makefile (Recommended)

# Infrastructure roles
make security          # Security-related roles only
make monitoring        # Monitoring tools only
make backup           # Backup system only

# Development tools
make docker           # Docker installation only
make shell            # Shell configuration only
make apps             # Applications only

# Network services
make tailscale        # VPN network setup
make tailscale-status # Check VPN status

# Maintenance (unified system)
make maintenance                    # All hosts
make maintenance GROUP=dev         # Specific group
make maintenance HOST=dev01        # Specific host  
make maintenance CHECK=true        # Dry-run all hosts
make maintenance GROUP=dev SERIAL=1 # Serial execution

# Infrastructure management
make create-vm        # Create new Proxmox VMs
make status          # Check connectivity
make facts           # Gather system facts

Manual Commands

# Security-related roles only
ansible-playbook dev-playbook.yml --tags security

# Development tools only  
ansible-playbook dev-playbook.yml --tags development,docker

# Network services
ansible-playbook tailscale-playbook.yml

# Infrastructure provisioning
ansible-playbook proxmox-create-vm.yml

# Skip maintenance
ansible-playbook dev-playbook.yml --skip-tags maintenance

Playbook Overview

• dev-playbook.yml: Complete development environment setup
• local-playbook.yml: Local machine configuration
• tailscale-playbook.yml: VPN network deployment
• proxmox-create-vm.yml: VM provisioning on Proxmox
• maintenance-playbook.yml: System maintenance operations

🔧 Configuration

Global Variables (group_vars/all.yml)

• timezone: System timezone (default: UTC)
• locale: System locale (default: en_US.UTF-8)
• ansible_debug_output: Show debug information (default: false)
• fail2ban_bantime: Ban duration in seconds
• fail2ban_findtime: Time window for failures
• fail2ban_maxretry: Max failures before ban

Tailscale Configuration

• tailscale_auth_key: Authentication key (stored in vault)
• tailscale_accept_routes: Accept subnet routes (default: true)
• tailscale_accept_dns: Accept DNS settings (default: true)
• tailscale_ssh: Enable SSH access through Tailscale (default: true)

Backup Configuration (roles/backup/defaults/main.yml)

• backup_enable_cron: Enable automated backups (default: true)
• backup_retention_days_home: Home backup retention (default: 7)
• backup_retention_days_system: System backup retention (default: 30)
• backup_users: Users to backup (default: ['master', 'beast', 'ladmin', 'user'])

SSH Configuration (roles/ssh/defaults/main.yml)

Comprehensive security hardening:

• ssh_port: SSH port (default: 22)
• ssh_permit_root_login: Root login setting (default: 'no')
• ssh_password_authentication: Password auth (default: 'no')
• ssh_max_auth_tries: Authentication attempts (default: 3)
• ssh_allowed_users: Restrict to specific users (default: [])
• ssh_allowed_groups: Restrict to specific groups (default: ['sudo', 'ssh'])

Proxmox VM Configuration (roles/proxmox_vm/defaults/main.yml)

• vm_memory: RAM allocation (default: 8192MB)
• vm_cores: CPU cores (default: 2)
• vm_disk_size: Disk size (default: 20G)
• vm_iso: Ubuntu Server ISO (default: ubuntu-24.04-live-server-amd64.iso)
• vm_ciuser: Default user (default: master)

🛡️ Security Features

Comprehensive SSH Hardening

• Modern cryptographic algorithms (ChaCha20-Poly1305, AES-256-GCM)
• Secure key exchange (Curve25519, DH Group 16)
• Disabled password authentication
• Connection rate limiting and timeouts
• User/group access restrictions
• Configuration validation and automatic backup

Fail2ban Integration

• SSH brute force protection
• Configurable ban times and retry limits
• Email notifications

UFW Firewall

• Deny-by-default policy
• SSH access allowed
• Automatic enablement

Tailscale VPN Security

• Zero-trust mesh networking
• End-to-end encryption
• SSH access through secure tunnel
• Subnet routing capabilities

📦 Installed Packages

Base System

• Core utilities: curl, wget, unzip, xclip, tree
• Network/Security: net-tools, ufw, fail2ban, mailutils
• Monitoring: iotop, nethogs, logwatch, btop (via snap)
• Modern CLI: jq, yq (via snap), ripgrep, fd-find

Development Tools

• git, nodejs, npm
• build-essential, python3, python3-pip

Applications

• brave-browser, libreoffice, evince, redshift
• code (VSCode), cursor (via snap)

Docker & Containers

• Docker CE with all components
• Docker Compose
• User added to docker group

Backup Tools (✨ NEW)

• rsync, borgbackup, rclone, restic
• Automated backup scripts and cron jobs

Monitoring Tools (✨ NEW)

• htop, iotop, nethogs, btop
• Custom system information scripts
• Network monitoring utilities

VPN & Network

• tailscale - Mesh VPN networking
• Network utilities and monitoring

🔧 Modern CLI Tools

The base role installs modern replacements for traditional Unix tools:

Available Commands

# Fast searching
rg "pattern" files/        # ripgrep - faster than grep
fd "filename"              # fd-find - intuitive find replacement

# Data processing  
jq '.key' file.json        # JSON processor and formatter
yq '.key' file.yaml        # YAML processor and formatter

# System monitoring
btop                       # Modern system monitor (better than htop)
tree directory/            # Directory structure visualization

# File operations
tree -L 2                  # Limit tree depth
rg -i "case insensitive"   # Case-insensitive search
fd -e yml                  # Find only YAML files
jq -r '.items[].name'      # Raw JSON output

🔄 Maintenance & Operations

Unified Maintenance System

# Basic usage
make maintenance                    # Run on all hosts
make maintenance GROUP=dev         # Run on specific group  
make maintenance HOST=dev01        # Run on specific host

# Advanced options
make maintenance CHECK=true        # Dry-run (safe testing)
make maintenance GROUP=dev SERIAL=1 # One host at a time
make maintenance GROUP=local       # Local machine (auto-sudo)

Backup Operations (✨ NEW)

# Deploy backup system
make backup

# Manual backup operations
sudo /opt/backups/scripts/backup-home.sh     # Run home backup
sudo /opt/backups/scripts/backup-system.sh   # Run system backup

# Check backup logs
tail -f /var/log/backups/home.log
tail -f /var/log/backups/system.log

Monitoring Operations (✨ NEW)

# Deploy monitoring tools
make monitoring

# Use monitoring scripts
/usr/local/bin/monitoring/sysinfo    # System information dashboard
/usr/local/bin/monitoring/netinfo    # Network information

# System monitoring
btop                                 # Interactive system monitor

Tailscale Network Management

# Deploy VPN network
make tailscale

# Check status across all machines
make tailscale-status

# Manual Tailscale commands
tailscale status                     # Check connection status
tailscale ip                         # Show Tailscale IP
tailscale netcheck                   # Network connectivity check

Infrastructure Provisioning (✨ NEW)

# Create new VMs on Proxmox
make create-vm

# Custom VM creation
ansible-playbook proxmox-create-vm.yml -e "vm_name=new-server vm_id=111"

🐛 Troubleshooting

Common Issues

1. SSH Connection Issues

   • Check ansible.cfg SSH settings
   • Verify host keys and user permissions
   • Test Tailscale connectivity: tailscale ping hostname

2. Vault Access Issues

   • Verify vault password file: ~/.ansible-vault-pass
   • Test vault decryption: ansible-vault view host_vars/hostname.yml

3. Tailscale Connection Issues

   • Check service status: sudo systemctl status tailscaled
   • Verify auth key in vault
   • Check firewall: sudo ufw status

4. Proxmox VM Creation Issues

   • Verify Proxmox credentials in vault
   • Check ISO availability: pvesm list local --content iso
   • Ensure sufficient resources on Proxmox node

5. Backup Issues

   • Check backup directories: ls -la /opt/backups/
   • Review logs: tail -f /var/log/backups/*.log
   • Verify cron jobs: sudo crontab -l

Debug Commands

# Using Makefile
make status         # Test connectivity to all hosts
make facts          # Gather facts from all hosts  
make debug          # Run with debug output
make verbose        # Run with verbose output

# Manual commands
ansible dev -m ping              # Test connectivity
ansible dev -m setup             # Check facts
ansible-playbook dev-playbook.yml --tags base  # Run specific role

# Verify installations
ansible dev -m shell -a "tailscale status"     # Check Tailscale
ansible dev -m shell -a "docker --version"     # Check Docker
ansible dev -m shell -a "sudo sshd -t"         # Validate SSH config

🛠️ Makefile Workflows

Development Workflow

make bootstrap    # Install collections
make test         # Lint + syntax check
make check        # Dry run
make apply        # Deploy to all hosts

Infrastructure Management

make create-vm    # Provision new VMs
make tailscale    # Deploy VPN network
make monitoring   # Deploy monitoring
make backup       # Deploy backup system

Host-Specific Operations

make dev HOST=dev01           # Deploy to specific host
make edit-vault HOST=dev01    # Edit encrypted host variables
make tailscale-dev           # Deploy Tailscale to dev hosts only

Maintenance and Utilities

make clean        # Clean up artifacts
make status       # Check host connectivity
make install-tools # Install recommended CLI tools locally

Run make help for the complete list of available commands.

📝 File Structure

ansible/ ├── ansible.cfg # Enhanced Ansible configuration ├── Makefile # Workflow automation with unified maintenance ├── hosts # Inventory file ├── dev-playbook.yml # Main development playbook ├── local-playbook.yml # Local machine setup ├── tailscale-playbook.yml # VPN network deployment ├── proxmox-create-vm.yml # VM provisioning playbook ├── maintenance-playbook.yml # Dedicated maintenance playbook ├── collections/ │ └── requirements.yml # Required Ansible collections ├── group_vars/ │ └── all.yml # Global variables and Tailscale config ├── host_vars/ # Host-specific variables (encrypted) └── roles/ ├── maintenance/ # System maintenance ├── base/ # Core system setup ├── development/ # Development tools ├── shell/ # Shell configuration (zsh + oh-my-zsh) ├── docker/ # Docker installation ├── ssh/ # SSH hardening and configuration ├── user/ # User management ├── applications/ # Desktop applications ├── snap/ # Snap applications ├── tailscale/ # VPN mesh networking ├── monitoring/ # System monitoring tools ├── backup/ # Automated backup solutions └── proxmox_vm/ # VM provisioning on Proxmox

🤝 Contributing

1. Test changes with --check first
2. Update documentation for new roles/tasks
3. Use proper handlers for service restarts
4. Follow existing naming conventions
5. Encrypt sensitive data with ansible-vault
6. Test across different OS distributions (Ubuntu, Debian, Alpine)