66 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
ilia	980423cd61	Fix CI pip on Debian bookworm (PEP 668) ... Bootstrap pip with --break-system-packages; set PIP_BREAK_SYSTEM_PACKAGES so ansible jobs work in node:20-bookworm without python3-venv. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:55:09 -04:00
ilia	5874605467	ci: re-run after git-ci-01 disk cleanup ... Runner was 100% full; pruned Docker and pre-pulled node:20-bookworm. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:50:47 -04:00
ilia	798fef2b67	Fix CI: use node:20-bookworm for ansible jobs ... actions/checkout@v4 requires Node; python:3.11-slim broke job 2. Bootstrap pip on bookworm without apt to avoid runner GPG issues. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:48:57 -04:00
ilia	52de8740c9	Fix CI: use python:3.11-slim image; reduce pip disk use. ... Runner hit errno 28 during ansible-lint install. Use slim image with built-in pip, ansible-core only, PIP_NO_CACHE_DIR, cache cleanup. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:43:22 -04:00
ilia	dfed055e7c	Fix CI: use Python-versioned get-pip URL (node image has 3.9). ... Default get-pip.py requires Python 3.10+; bullseye node image ships 3.9.2. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:38:48 -04:00
ilia	08d5cb4073	Fix CI: bootstrap pip without apt on act runner. ... Debian bullseye apt in Gitea act containers fails GPG signature checks; use get-pip.py + python3 -m pip instead of apt-get python3-pip. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:35:56 -04:00
ilia	70af1b1355	Fix CI: install python3-pip in node jobs before pip3/ansible. ... Gitea act node:20-bullseye has no pip3; apt-install Python first. Use relative roles_path, skip vault files in YAML check, stub caddy/sites inventory groups for playbook-test; soften container/sonar failures. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 21:29:48 -04:00
ilia	7224dbfd12	Drop nextcloud export notes from PR (local reference only). ... Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 17:10:38 -04:00
ilia	8a507eddee	Fix CI: ansible-lint playbook schema and markdownlint for new guides. ... Use ansible.builtin.su, spaces in caddy blockinfile, relax MD060/MD036 and line length for homelab documentation tables. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 17:10:33 -04:00
ilia	de49b34cdc	Add homelab monitoring, portfolio site, and vault tooling. ... Document pve10 static IPs, monitoring stack, and site LXCs; add portfolio to inventory; Mailcow mailbox automation; vault import/export scripts; security audit guides and UniFi DHCP reference. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-22 16:25:07 -04:00
ilia	9281f12a65	inventory: add hermes VM (10.0.10.36, ladmin, VMID 117) ... Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-21 20:58:19 -04:00
ilia	659c6501bf	Merge pull request 'Fix production inventory IPs for listmonk and giteaVM.' (#7) from fix/inventory-host-ips into master ... Reviewed-on: #7	2026-05-20 15:11:31 -05:00
ilia	fda101c949	Merge pull request 'refactor(inventory): Update punimTag project configurations and environment variables' (#6) from punimTagProvision into master ... Reviewed-on: #6	2026-05-20 15:11:21 -05:00
ilia	4a5506d26a	Allow merge of CI workflow fix to default branch @skipci ... Gitea Actions reads workflows from master; this empty commit skips CI so PR #7 can land the fixed ci.yml on the default branch. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-20 15:45:25 -04:00
ilia	0b27e7870c	Fix Gitea CI workflow for constrained act runners. ... Use node:20-bullseye images with checkout-first steps to avoid broken Ubuntu jammy apt/GPG installs, replace Trivy dependency scanning with npm audit and pip-audit, and install tooling via pip or prebuilt binaries. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-20 15:44:12 -04:00
ilia	8ec94ebaf5	Fix production inventory IPs for listmonk and giteaVM. ... Correct listmonk and giteaVM ansible_host values to match current LAN addresses. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-05-20 15:35:37 -04:00
ilia	62a22812a3	refactor(inventory): Update punimTag project configurations and environment variables ... - Renamed punimTagFE to punimTag for consistency in project naming. - Updated environment variable configurations for punimTag, including backend and frontend settings. - Added backend support for punimTag with appropriate commands for deployment and migration. - Adjusted environment variables for dev, qa, and prod environments to reflect new project structure. - Ensured all changes maintain backward compatibility with existing configurations. This refactor aims to streamline the project setup and improve clarity in the inventory structure.	2026-01-10 15:19:07 -05:00
ilia	0a937fd1b4	feat(app_setup): Improves deployment reliability for app projects and adds support for mirrormatch deployment with Prisma/Next.js requirements. (#5) ... ## Summary Improves deployment reliability for app projects and adds support for mirrormatch deployment with Prisma/Next.js requirements. ## Changes ### Core Improvements (affects all app projects) 1. **Deploy Script (`deploy_app.sh.j2`)** - Fixed clone logic to handle non-git directories gracefully - Preserves `.env.*` files during repository clone - Uses temporary directory for initial clone to avoid permission issues - Added `sudo` to systemctl restart commands (appuser needs sudo for service management) 2. **Environment Template (`env.j2`)** - Removed comment lines to prevent `xargs` errors when sourcing env files - Cleaner, more reliable env file format 3. **App Setup Role (`app_setup/tasks/main.yml`)** - Added initial deploy task to run deploy script during first configure - Ensures app is fully deployed before systemd service starts 4. **Configure Playbook (`configure_app.yml`)** - Fixed migrate command precedence: checks `env_def.backend_migrate_cmd` first - Allows per-environment override of migrate commands (e.g., `db:push` for dev/qa) ### Mirrormatch-Specific Configuration - Added `mirrormatch` project definition with dev/qa/prod environments - Configured `backend_migrate_cmd: "npm run db:push"` for dev/qa (no shadow DB needed) - Added `backend_seed_cmd` support for dev/qa environments - Configured NextAuth v5 environment variables (`AUTH_TRUST_HOST`) ### Documentation - Updated `docs/guides/app_stack_proxmox.md` with: - Project-specific configuration examples - Environment file naming notes - Command precedence documentation ## Impact Analysis ### ✅ Backward Compatible - **pote**: No impact (uses separate `pote` role) - **punimTagFE/BE**: Will benefit from improved deploy script, no breaking changes - **mirrormatch**: Uses new features, fully supported ### Project-Specific Configs (isolated) All mirrormatch-specific settings are in `app_projects.mirrormatch` and don't affect other projects: - `backend_migrate_cmd: "npm run db:push"` (per-environment) - `backend_seed_cmd: "npm run db:seed"` (per-environment) - `AUTH_TRUST_HOST: "true"` (in env_vars) ## Testing - ✅ Mirrormatch dev environment successfully deployed - ✅ Service starts correctly after deployment - ✅ Environment variables loaded properly - ✅ Database schema pushed and seeded ## Related Fixes deployment issues encountered during mirrormatch setup: - Non-git directory handling - Env file preservation during clone - Service restart permissions - Prisma migrate vs db:push workflow Reviewed-on: #5	2026-01-04 16:59:48 -05:00
ilia	c3e6caf9e8	refactor-servers-workstations-shell-monitoring (#4) ... ### Summary This PR refactors the playbook layout to reduce duplication and make host intent clearer (servers vs workstations), splits monitoring by host type, and restores full Zsh setup for developers while keeping servers aliases-only. ### Key changes - **New playbooks** - `playbooks/servers.yml`: baseline for server-class hosts (no desktop apps) - `playbooks/workstations.yml`: baseline for dev/desktop/local + **desktop apps only on `desktop` group** - **Monitoring split** - `roles/monitoring_server`: server monitoring + intrusion prevention (includes `fail2ban`, sysstat) - `roles/monitoring_desktop`: desktop-oriented monitoring tooling - Updated playbooks to use the correct monitoring role per host type - **Shell role: server-safe + developer-friendly** - `roles/shell` now supports two modes: - `shell_mode: minimal` (default): aliases-only, does not overwrite `.zshrc` - `shell_mode: full`: installs Oh My Zsh + Powerlevel10k + plugins and deploys a managed `.zshrc` - `playbooks/development.yml` and `playbooks/workstations.yml` use `shell_mode: full` - `playbooks/servers.yml` remains **aliases-only** - **Applications** - Applications role runs only on `desktop` group (via `workstations.yml`) - Removed Brave installs/repo management - Added **CopyQ** to desktop apps (`applications_desktop_packages`) - **Docs + architecture** - Added canonical doc tree under `project-docs/` (overview/architecture/standards/workflow/decisions) - Consolidated architecture docs: `docs/reference/architecture.md` is now a pointer to `project-docs/architecture.md` - Fixed broken doc links by adding the missing referenced pages under `docs/` ### Behavior changes (important) - Desktop GUI apps install **only** on the `desktop` inventory group (not on servers, not on dev VMs unless they are in `desktop`). - Dev/workstation Zsh is now provisioned in **full mode** (managed `.zshrc` + p10k). ### How to test (local CI parity) ```bash make test npm test ``` Optional dry runs (interactive sudo may be required): ```bash make check make check-local ``` ### Rollout guidance - Apply to a single host first: - Workstations: `make workstations HOST=<devhost>` - Servers: `make servers HOST=<serverhost>` - Then expand to group runs. Reviewed-on: #4	2026-01-01 22:11:24 -05:00
ilia	69a39e5e5b	Add POTE app project support and improve IP conflict detection (#3) ... ## Summary This PR adds comprehensive support for deploying the **POTE** application project via Ansible, along with improvements to IP conflict detection and a new app stack provisioning system for Proxmox-managed LXC containers. ## Key Features ### 🆕 New Roles - **`roles/pote`**: Python/venv deployment role for POTE (PostgreSQL, cron jobs, Alembic migrations) - **`roles/app_setup`**: Generic app deployment role (Node.js/systemd) - **`roles/base_os`**: Base OS hardening role ### 🛡️ Safety Improvements - IP uniqueness validation within projects - Proxmox-side IP conflict detection - Enhanced error messages for IP conflicts ### 📦 New Playbooks - `playbooks/app/site.yml`: End-to-end app stack deployment - `playbooks/app/provision_vms.yml`: Proxmox guest provisioning - `playbooks/app/configure_app.yml`: OS + application configuration ## Security - ✅ All secrets stored in encrypted vault.yml - ✅ Deploy keys excluded via .gitignore - ✅ No plaintext secrets committed ## Testing - ✅ POTE successfully deployed to dev/qa/prod environments - ✅ All components validated (Git, PostgreSQL, cron, migrations) Co-authored-by: ilia <ilia@levkin.ca> Reviewed-on: #3	2026-01-01 11:19:54 -05:00
ilia	e897b1a027	Fix: Resolve linting errors and improve firewall configuration (#2) ... - Fix UFW firewall to allow outbound traffic (was blocking all outbound) - Add HOST parameter support to shell Makefile target - Fix all ansible-lint errors (trailing spaces, missing newlines, document starts) - Add changed_when: false to check commands - Fix variable naming (vault_devGPU -> vault_devgpu) - Update .ansible-lint config to exclude .gitea/ and allow strategy: free - Fix NodeSource repository GPG key handling in shell playbook - Add missing document starts to host_vars files - Clean up empty lines in datascience role files Reviewed-on: #2	2025-12-25 16:47:26 -05:00
ilia	95a301ae3f	Merge pull request 'Fix: Update CI workflow to use Alpine-based images, install Node.js and Trivy with improved methods, and enhance dependency scanning steps' (#1) from update-ci into master ... Reviewed-on: #1	2025-12-17 22:45:00 -05:00
ilia	c017ec6941	Fix: Update CI workflow to install a fixed version of Trivy for improved reliability and error handling during installation	2025-12-15 15:50:04 -05:00
ilia	9e7ef8159b	Fix: Update CI workflow to disable SCM in SonarScanner configuration for improved analysis accuracy	2025-12-15 15:36:15 -05:00
ilia	3828e04b13	Fix: Update CI workflow to install Git alongside Node.js and enhance SonarScanner installation process with improved error handling	2025-12-15 15:11:36 -05:00
ilia	d6655babd9	Refactor: Simplify connectivity analysis logic by breaking down into smaller helper functions for improved readability and maintainability	2025-12-15 14:55:10 -05:00
ilia	dc94395bbc	Fix: Enhance SonarScanner error handling in CI workflow with detailed failure messages and troubleshooting guidance	2025-12-14 21:35:52 -05:00
ilia	699aaefac3	Fix: Update CI workflow to improve SonarScanner installation process with enhanced error handling and version management	2025-12-14 21:21:26 -05:00
ilia	277a22d962	Fix: Clean up duplicate repository entries in application and development roles	2025-12-14 21:21:19 -05:00
ilia	83a5d988af	Fix: Update ansible-lint configuration to exclude specific paths and skip certain rules for improved linting flexibility	2025-12-14 21:04:45 -05:00
ilia	a45ee496e4	Fix: Update CI workflow to use Ubuntu 22.04 container, install Node.js and SonarScanner with improved methods, and enhance SonarQube connectivity verification	2025-12-14 20:51:36 -05:00
ilia	e54ecfefc1	Fix: Update CI workflow to enhance playbook syntax checking and improve SonarQube connectivity verification	2025-12-14 20:43:28 -05:00
ilia	f20b671e76	Fix: Update CI workflow to use Alpine-based images, install Node.js and Trivy with improved methods, and enhance dependency scanning steps	2025-12-14 20:28:06 -05:00
ilia	d0699d0b7a	Fix: Add SonarQube analysis to CI workflow and update host inventory for production environment	2025-12-14 20:10:38 -05:00
ilia	d4ce0a247d	Fix: Remove artifact upload, update Trivy flags, add workflow summary, and add git to shell role	2025-12-14 14:57:22 -05:00
ilia	0076155ef1	Fix: Improve Trivy installation with multiple fallback methods and better error handling	2025-12-14 09:06:53 -05:00
ilia	67a9b3ca2b	Fix: Check vault encryption header instead of decrypting files	2025-12-13 23:42:06 -05:00
ilia	6d14cf9253	Fix: Install git for Gitleaks and use direct Trivy binary download	2025-12-13 23:37:38 -05:00
ilia	a9ed19c9d2	Fix: Install Node.js in all Ubuntu containers for checkout action	2025-12-13 23:30:42 -05:00
ilia	1a565cc30e	Fix: Change all jobs to use ubuntu-latest label to match runner	2025-12-13 23:24:02 -05:00
ilia	8818de005f	Add comprehensive security scanning: SAST, license check, vault validation, playbook testing, and artifact uploads	2025-12-13 23:19:10 -05:00
ilia	990f886f02	Fix CI workflow: configure markdownlint, fix Node version, add Ansible validation	2025-12-13 23:13:40 -05:00
ilia	f3b34f3c95	Fix CI workflow: configure markdownlint and make link checking non-blocking	2025-12-13 23:06:26 -05:00
ilia	ba7d4eb5b3	Add CI workflow with markdown linting and self-hosted runner job	2025-12-13 23:00:58 -05:00
ilia	097fb33abc	Update inventory file to include new desktop host configuration ... - Add desktop-beast with ansible_host and ansible_user settings for improved access management. - Ensure consistent formatting and organization within the inventory file for better clarity. These changes enhance the inventory setup, facilitating smoother operations and management of desktop hosts within the infrastructure.	2025-10-15 15:52:30 -04:00
ilia	1fe27468a1	Update inventory file to standardize ansible_user settings for Gitea and other services ... - Adjust ansible_user for Gitea to 'root' for improved access control. - Ensure consistent ansible_user settings across all services, including Portainer, Jellyfin, and Listmonk, to streamline user management. These changes enhance the clarity and usability of the inventory setup, facilitating smoother operations across the infrastructure.	2025-10-10 09:23:40 -04:00
ilia	96f7c8a82a	Update inventory and shell configuration for improved host management ... - Adjust inventory file to standardize ansible_user settings for listmonk and jellyfin hosts, ensuring consistent user access across services. - Update .zshrc file to include SSH aliases for new hosts, enhancing accessibility for remote management. These changes streamline host management and improve the usability of SSH connections for infrastructure operations.	2025-10-09 21:43:29 -04:00
ilia	579f0709ce	Update Makefile and inventory configurations for improved task execution and organization ... - Refactor Makefile to enhance command structure, including clearer descriptions and usage examples for targets related to development, inventory, and monitoring tasks. - Update inventory files to ensure correct host configurations and user settings, including adjustments to ansible_user for specific hosts. - Modify group_vars to streamline Tailscale configuration and ensure proper handling of authentication keys. These changes improve the clarity and usability of the Makefile and inventory setup, facilitating smoother operations across the infrastructure.	2025-10-09 21:24:45 -04:00
ilia	e05b3aa0d5	Update ansible.cfg and auto-fallback script for improved connectivity handling ... - Modify ansible.cfg to increase SSH connection retries from 2 to 3 and add a connection timeout setting for better reliability. - Enhance auto-fallback.sh script to provide detailed feedback during IP connectivity tests, including clearer status messages for primary and fallback IP checks. - Update documentation to reflect changes in connectivity testing and fallback procedures. These updates improve the robustness of the connectivity testing process and ensure smoother operations during IP failover scenarios.	2025-09-16 23:00:32 -04:00
ilia	b424e9b55b	Add checks and conditional tasks for package management across roles ... - Introduce checks for existing GPG keys and repositories for Docker, NodeSource, and Tailscale to ensure correct configurations before installation. - Implement conditional removal of incorrect keys and repositories to maintain a clean setup. - Update Makefile to include a command for editing group vault variables. These changes enhance package management reliability and streamline the installation process across different roles.	2025-09-11 21:05:31 -04:00