Honest CI gates and homelab IP scrub #5

Merged
ilia merged 2 commits from chore/honest-ci-and-doc-scrub into main 2026-07-26 14:48:34 -05:00
20 changed files with 117 additions and 127 deletions
+4 -8
View File
@@ -48,8 +48,8 @@ jobs:
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt --break-system-packages; fi if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt --break-system-packages; fi
pip install bandit pip-audit ruff --break-system-packages pip install bandit pip-audit ruff --break-system-packages
- name: Ruff lint - name: Ruff lint (hard gate)
run: ruff check . || true run: ruff check .
- name: Bandit (advisory) - name: Bandit (advisory)
run: bandit -r . -q || true run: bandit -r . -q || true
@@ -57,14 +57,10 @@ jobs:
- name: pip-audit (advisory) - name: pip-audit (advisory)
run: pip-audit -r requirements.txt 2>/dev/null || pip-audit 2>/dev/null || true run: pip-audit -r requirements.txt 2>/dev/null || pip-audit 2>/dev/null || true
- name: Pytest - name: Pytest (hard gate)
run: | run: |
if [ -d tests ] || ls test_*.py *_test.py 2>/dev/null; then
pip install pytest --break-system-packages pip install pytest --break-system-packages
pytest -q || true pytest -q
else
echo "No tests found — skip"
fi
secret-scan: secret-scan:
needs: skip-ci-check needs: skip-ci-check
+5 -5
View File
@@ -53,13 +53,13 @@ python scripts/airbnb_login.py
Chromium opens. Log in, complete any 2FA, then press Enter in the terminal. Session cookies are saved to `state.json`. Chromium opens. Log in, complete any 2FA, then press Enter in the terminal. Session cookies are saved to `state.json`.
### Production (automationlab @ 10.0.10.45) ### Production (automationlab LXC)
Recommended: login on your Mac, then copy: Recommended: login on your Mac, then copy (set `ATANYRATE_HOST` to the automationlab host):
```bash ```bash
scp state.json root@10.0.10.45:/opt/atanyrate/state.json scp state.json "root@${ATANYRATE_HOST}:/opt/atanyrate/state.json"
ssh root@10.0.10.45 'chmod 600 /opt/atanyrate/state.json' ssh "root@${ATANYRATE_HOST}" 'chmod 600 /opt/atanyrate/state.json'
``` ```
Or use the ansible deploy script: `ATANYRATE_STATE=~/path/to/state.json make deploy-atanyrate` (see `docs/guides/atanyrate-deploy.md` in the ansible repo). Or use the ansible deploy script: `ATANYRATE_STATE=~/path/to/state.json make deploy-atanyrate` (see `docs/guides/atanyrate-deploy.md` in the ansible repo).
@@ -98,7 +98,7 @@ docker run --rm --env-file .env -v $(pwd)/state.json:/app/state.json eventrate
## Production deploy ## Production deploy
Deployed on **automationlab** (`10.0.10.45`) at `/opt/atanyrate`. Full guide: ansible repo `docs/guides/atanyrate-deploy.md`. Deployed on **automationlab** (`<homelab-host>`) at `/opt/atanyrate`. Full guide: ansible repo `docs/guides/atanyrate-deploy.md`.
```bash ```bash
# From ~/Documents/code/ansible # From ~/Documents/code/ansible
+4 -4
View File
@@ -1,7 +1,7 @@
# AtAnyRate — handoff # AtAnyRate — handoff
**Repo:** `gitea@git.levkin.ca:ilia/AtAnyRate.git` · local `~/Documents/code/AtAnyRate` **Repo:** `gitea@git.levkin.ca:ilia/AtAnyRate.git` · local `~/Documents/code/AtAnyRate`
**Deploy:** pve10 LXC **automationlab** @ `10.0.10.45``make deploy-atanyrate` (ansible) **Deploy:** pve10 LXC **automationlab** @ `<homelab-host>``make deploy-atanyrate` (ansible)
**Vikunja:** [todo.levkin.ca → Business → AtAnyRate](https://todo.levkin.ca) (`AAR`) **Vikunja:** [todo.levkin.ca → Business → AtAnyRate](https://todo.levkin.ca) (`AAR`)
**Epic backlog:** [../BACKLOG.md](../BACKLOG.md) **Epic backlog:** [../BACKLOG.md](../BACKLOG.md)
@@ -22,7 +22,7 @@
## Done (reference) ## Done (reference)
- Ticketmaster + SeatGeek providers, Telegram alerter, Playwright calendar automation - Ticketmaster + SeatGeek providers, Telegram alerter, Playwright calendar automation
- Deploy on automationlab (`10.0.10.45`); ansible vault `vault_atanyrate_*` - Deploy on automationlab (`<homelab-host>`); ansible vault `vault_atanyrate_*`
- SeatGeek client ID on server (both providers 200 OK) - SeatGeek client ID on server (both providers 200 OK)
--- ---
@@ -36,8 +36,8 @@ python -m src.main --alerts-only --dry-run
# on Mac: refresh session (display required) # on Mac: refresh session (display required)
python scripts/airbnb_login.py python scripts/airbnb_login.py
scp state.json root@10.0.10.45:/opt/atanyrate/state.json scp state.json "root@${ATANYRATE_HOST}:/opt/atanyrate/state.json"
ssh root@10.0.10.45 'chmod 600 /opt/atanyrate/state.json' ssh "root@${ATANYRATE_HOST}" 'chmod 600 /opt/atanyrate/state.json'
``` ```
--- ---
+6
View File
@@ -1,3 +1,9 @@
[tool.pytest.ini_options] [tool.pytest.ini_options]
testpaths = ["tests"] testpaths = ["tests"]
pythonpath = ["."] pythonpath = ["."]
[tool.ruff]
line-length = 120
[tool.ruff.lint]
select = ["E4", "E7", "E9", "F", "I"]
+2 -2
View File
@@ -16,11 +16,11 @@ Optional stealth Firefox (if Airbnb blocks Chromium — same mode for login + ru
Then copy to automationlab: Then copy to automationlab:
scp state.json root@10.0.10.45:/opt/atanyrate/state.json scp state.json "root@${ATANYRATE_HOST}:/opt/atanyrate/state.json"
""" """
from pathlib import Path
import sys import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+2 -1
View File
@@ -16,7 +16,8 @@ import re
import time import time
from datetime import date from datetime import date
from playwright.sync_api import Locator, Page, TimeoutError as PlaywrightTimeout from playwright.sync_api import Locator, Page
from playwright.sync_api import TimeoutError as PlaywrightTimeout
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
-1
View File
@@ -6,7 +6,6 @@ Uses the Bot API sendMessage endpoint with MarkdownV2 formatting.
from __future__ import annotations from __future__ import annotations
import logging import logging
from datetime import date
from itertools import groupby from itertools import groupby
import httpx import httpx
+1 -1
View File
@@ -1,5 +1,5 @@
from src.providers.base import EventProvider from src.providers.base import EventProvider
from src.providers.ticketmaster import TicketmasterProvider
from src.providers.seatgeek import SeatGeekProvider from src.providers.seatgeek import SeatGeekProvider
from src.providers.ticketmaster import TicketmasterProvider
__all__ = ["EventProvider", "TicketmasterProvider", "SeatGeekProvider"] __all__ = ["EventProvider", "TicketmasterProvider", "SeatGeekProvider"]
+1
View File
@@ -3,6 +3,7 @@
from __future__ import annotations from __future__ import annotations
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from src.models import NormalizedEvent from src.models import NormalizedEvent
+1
View File
@@ -11,6 +11,7 @@ layout at Scotiabank Arena).
from __future__ import annotations from __future__ import annotations
import logging import logging
from src.models import NormalizedEvent from src.models import NormalizedEvent
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
+5 -3
View File
@@ -1,11 +1,11 @@
"""Tests for Airbnb authentication / storage state management.""" """Tests for Airbnb authentication / storage state management."""
from pathlib import Path from pathlib import Path
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock
import pytest import pytest
from src.airbnb.auth import load_authenticated_context, DEFAULT_STATE_PATH from src.airbnb.auth import DEFAULT_STATE_PATH, load_authenticated_context
class TestLoadAuthenticatedContext: class TestLoadAuthenticatedContext:
@@ -27,7 +27,9 @@ class TestLoadAuthenticatedContext:
ctx = load_authenticated_context(mock_browser, state_path=state_path) ctx = load_authenticated_context(mock_browser, state_path=state_path)
mock_browser.new_context.assert_called_once_with( mock_browser.new_context.assert_called_once_with(
storage_state=str(state_path) storage_state=str(state_path),
viewport={"width": 1440, "height": 900},
locale="en-CA",
) )
assert ctx is mock_context assert ctx is mock_context
-1
View File
@@ -1,6 +1,5 @@
"""Tests for optional stealth browser launcher.""" """Tests for optional stealth browser launcher."""
import os
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
import pytest import pytest
+62 -70
View File
@@ -1,99 +1,91 @@
"""Tests for the Airbnb calendar price automation module.""" """Tests for the Airbnb calendar price automation module."""
from datetime import date from datetime import date
from unittest.mock import MagicMock, call, patch from unittest.mock import MagicMock, patch
import pytest from playwright.sync_api import TimeoutError as PlaywrightTimeout
from src.airbnb.calendar import ( from src.airbnb.calendar import (
update_price,
_navigate_to_month,
CALENDAR_URL,
SELECTORS,
_MAX_UPDATE_ATTEMPTS, _MAX_UPDATE_ATTEMPTS,
DEFAULT_HOST_ORIGIN,
parse_month_heading_text,
resolve_calendar_url,
update_price,
) )
CALENDAR_URL = f"{DEFAULT_HOST_ORIGIN}/multicalendar/12345"
class TestResolveCalendarUrl:
def test_override_wins(self):
url = resolve_calendar_url("12345", "https://www.airbnb.ca/multicalendar/999/")
assert url == "https://www.airbnb.ca/multicalendar/999"
def test_listing_id_builds_multicalendar_url(self):
assert resolve_calendar_url("12345", "") == f"{DEFAULT_HOST_ORIGIN}/multicalendar/12345"
def test_no_listing_id_falls_back_to_hosting_calendar(self):
assert resolve_calendar_url("", "") == f"{DEFAULT_HOST_ORIGIN}/hosting/calendar"
class TestParseMonthHeadingText:
def test_parses_month_and_year(self):
assert parse_month_heading_text("April 2026") == (2026, 4)
def test_parses_with_surrounding_text(self):
assert parse_month_heading_text(" December 2025 ") == (2025, 12)
def test_rejects_garbage(self):
assert parse_month_heading_text("not a month heading") is None
class TestUpdatePrice: class TestUpdatePrice:
def _make_page(self, *, fail_on_click: bool = False) -> MagicMock:
page = MagicMock()
if fail_on_click:
from playwright.sync_api import TimeoutError as PlaywrightTimeout
page.click.side_effect = PlaywrightTimeout("timed out")
return page
def test_successful_price_update(self): def test_successful_price_update(self):
page = self._make_page() page = MagicMock()
result = update_price(page, date(2026, 5, 10), 180) with patch("src.airbnb.calendar._run_price_update_attempt") as attempt:
result = update_price(page, date(2026, 5, 10), 180, CALENDAR_URL)
assert result is True assert result is True
page.goto.assert_called_once() attempt.assert_called_once_with(page, date(2026, 5, 10), 180, CALENDAR_URL)
assert "2026-05-10" in str(page.click.call_args_list[0])
page.fill.assert_called_once()
def test_navigates_to_calendar_url(self):
page = self._make_page()
update_price(page, date(2026, 5, 10), 180)
page.goto.assert_called_once_with(
CALENDAR_URL, wait_until="networkidle", timeout=30_000
)
def test_fills_correct_price(self):
page = self._make_page()
update_price(page, date(2026, 5, 10), 200)
page.fill.assert_called_once_with(
SELECTORS["price_input"], "200", timeout=5_000
)
def test_retries_on_timeout(self): def test_retries_on_timeout(self):
from playwright.sync_api import TimeoutError as PlaywrightTimeout
page = MagicMock() page = MagicMock()
page.goto.side_effect = PlaywrightTimeout("timed out") with (
patch(
with patch("src.airbnb.calendar.time.sleep"): "src.airbnb.calendar._run_price_update_attempt",
result = update_price(page, date(2026, 5, 10), 180) side_effect=PlaywrightTimeout("timed out"),
) as attempt,
patch("src.airbnb.calendar.time.sleep"),
):
result = update_price(page, date(2026, 5, 10), 180, CALENDAR_URL)
assert result is False assert result is False
assert page.goto.call_count == _MAX_UPDATE_ATTEMPTS assert attempt.call_count == _MAX_UPDATE_ATTEMPTS
def test_retries_on_generic_exception(self): def test_retries_on_generic_exception(self):
page = MagicMock() page = MagicMock()
page.goto.side_effect = RuntimeError("unexpected") with (
patch(
with patch("src.airbnb.calendar.time.sleep"): "src.airbnb.calendar._run_price_update_attempt",
result = update_price(page, date(2026, 5, 10), 180) side_effect=RuntimeError("unexpected"),
) as attempt,
patch("src.airbnb.calendar.time.sleep"),
):
result = update_price(page, date(2026, 5, 10), 180, CALENDAR_URL)
assert result is False assert result is False
assert page.goto.call_count == _MAX_UPDATE_ATTEMPTS assert attempt.call_count == _MAX_UPDATE_ATTEMPTS
def test_succeeds_on_second_attempt(self): def test_succeeds_on_second_attempt(self):
from playwright.sync_api import TimeoutError as PlaywrightTimeout
page = MagicMock() page = MagicMock()
page.goto.side_effect = [PlaywrightTimeout("first fail"), None] with (
patch(
with patch("src.airbnb.calendar.time.sleep"): "src.airbnb.calendar._run_price_update_attempt",
result = update_price(page, date(2026, 5, 10), 180) side_effect=[PlaywrightTimeout("first fail"), None],
) as attempt,
patch("src.airbnb.calendar.time.sleep"),
):
result = update_price(page, date(2026, 5, 10), 180, CALENDAR_URL)
assert result is True assert result is True
assert page.goto.call_count == 2 assert attempt.call_count == 2
class TestNavigateToMonth:
def test_stub_does_not_crash(self):
page = MagicMock()
_navigate_to_month(page, date(2026, 5, 10))
class TestSelectors:
def test_date_cell_selector_uses_date_format(self):
sel = SELECTORS["date_cell"].format(date_str="2026-05-10")
assert "2026-05-10" in sel
def test_all_selectors_defined(self):
assert "date_cell" in SELECTORS
assert "price_input" in SELECTORS
assert "save_button" in SELECTORS
+2 -4
View File
@@ -2,15 +2,13 @@
from datetime import date from datetime import date
import pytest
from src.models import NormalizedEvent from src.models import NormalizedEvent
from src.notifications.telegram import ( from src.notifications.telegram import (
send_alert,
_format_message,
_escape_md, _escape_md,
_format_message,
_score_indicator, _score_indicator,
_severity_summary, _severity_summary,
send_alert,
) )
+1 -3
View File
@@ -2,9 +2,7 @@
from datetime import date from datetime import date
import pytest from src.providers.seatgeek import SeatGeekProvider
from src.providers.seatgeek import SeatGeekProvider, MIN_SCORE_THRESHOLD
from tests.conftest import SEATGEEK_RESPONSE from tests.conftest import SEATGEEK_RESPONSE
+1 -3
View File
@@ -2,9 +2,7 @@
from datetime import date from datetime import date
import pytest from src.providers.ticketmaster import MAJOR_VENUES, TicketmasterProvider
from src.providers.ticketmaster import TicketmasterProvider, MAJOR_VENUES
from tests.conftest import TICKETMASTER_RESPONSE from tests.conftest import TICKETMASTER_RESPONSE
+1 -1
View File
@@ -3,7 +3,7 @@
from datetime import date from datetime import date
from src.models import NormalizedEvent from src.models import NormalizedEvent
from src.scoring.impact import score_event, score_events, VENUE_CAPACITY, MAX_CAPACITY, _team_boost from src.scoring.impact import MAX_CAPACITY, VENUE_CAPACITY, _team_boost, score_event, score_events
def _make_event( def _make_event(
+1 -1
View File
@@ -2,7 +2,7 @@
from datetime import date from datetime import date
from src.dedup import deduplicate, _similarity, _is_same_event from src.dedup import _is_same_event, _similarity, deduplicate
from src.models import NormalizedEvent from src.models import NormalizedEvent
+1 -1
View File
@@ -1,6 +1,6 @@
"""Tests for the date-window filter in main.py.""" """Tests for the date-window filter in main.py."""
from datetime import date, timedelta from datetime import date
from unittest.mock import patch from unittest.mock import patch
from src.main import filter_by_window from src.main import filter_by_window
+13 -14
View File
@@ -1,18 +1,18 @@
"""Tests for the main orchestration runner.""" """Tests for the main orchestration runner."""
from datetime import date from datetime import date
from unittest.mock import patch, MagicMock from unittest.mock import MagicMock, patch
import pytest import pytest
from src.main import ( from src.main import (
parse_args,
fetch_all_events, fetch_all_events,
filter_noise,
filter_by_min_score, filter_by_min_score,
filter_noise,
main,
parse_args,
print_summary, print_summary,
update_airbnb_prices, update_airbnb_prices,
main,
) )
from src.models import NormalizedEvent from src.models import NormalizedEvent
@@ -185,24 +185,23 @@ class TestUpdateAirbnbPrices:
settings.airbnb_listing_id = "12345" settings.airbnb_listing_id = "12345"
settings.airbnb_base_price = 100 settings.airbnb_base_price = 100
settings.price_increase_pct = 25 settings.price_increase_pct = 25
settings.airbnb_calendar_url = ""
settings.airbnb_headed = False
events = [_make_event("A", date(2026, 5, 10), "V")] events = [_make_event("A", date(2026, 5, 10), "V")]
with patch("playwright.sync_api.sync_playwright") as mock_pw, \
patch("src.airbnb.auth.load_authenticated_context") as mock_auth, \
patch("src.airbnb.calendar.update_price") as mock_update:
mock_page = MagicMock() mock_page = MagicMock()
mock_context = MagicMock() mock_context = MagicMock()
mock_context.new_page.return_value = mock_page mock_context.new_page.return_value = mock_page
mock_browser = MagicMock()
mock_auth.return_value = mock_context
mock_pw.return_value.__enter__ = MagicMock(
return_value=MagicMock(chromium=MagicMock(launch=MagicMock(return_value=mock_browser)))
)
mock_update.return_value = True
with patch("src.airbnb.browser.open_browser"), \
patch("src.airbnb.auth.load_authenticated_context", return_value=mock_context), \
patch("src.airbnb.calendar.update_price", return_value=True) as mock_update:
update_airbnb_prices(events, settings) update_airbnb_prices(events, settings)
mock_update.assert_called_once_with(mock_page, date(2026, 5, 10), 125)
mock_update.assert_called_once_with(
mock_page, date(2026, 5, 10), 125, "https://www.airbnb.ca/multicalendar/12345"
)
def test_handles_missing_state_file(self): def test_handles_missing_state_file(self):
settings = MagicMock() settings = MagicMock()