Initial commit: Talos edge-facing scan orchestrator.
Some checks failed
CI / skip-ci-check (push) Successful in 8s
CI / secret-scan (push) Successful in 7s
CI / python-ci (push) Failing after 19s

FastAPI + Kali tools UI for authorized scans of levkin.ca / LAN targets.
Gitignore local data/.env; add pytest + gitleaks CI.
This commit is contained in:
ilia 2026-07-12 11:46:54 -04:00
commit 12a6e2b6bc
29 changed files with 3075 additions and 0 deletions

7
.env.example Normal file
View File

@ -0,0 +1,7 @@
# Ollama — use homelab devGPU (RTX 4080) instead of bundled container:
# OLLAMA_URL=http://10.0.10.122:11434
OLLAMA_URL=http://ollama:11434
OLLAMA_MODEL=qwen2.5:14b
ALLOW_LOOPBACK=false
BLOCKED_TARGETS=
MAX_CONCURRENT_SCANS=5

64
.gitea/workflows/ci.yml Normal file
View File

@ -0,0 +1,64 @@
# Homelab CI — pytest (safety/profiles) + gitleaks
# Skip: @skipci in branch name or commit message
name: CI
on:
push:
branches: [master, main]
pull_request:
workflow_dispatch:
jobs:
skip-ci-check:
runs-on: [homelab, self-hosted, linux]
container:
image: node:20-bookworm
outputs:
should-skip: ${{ steps.check.outputs.skip }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- id: check
run: |
SKIP=0
BRANCH="${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}"
MSG="${GITHUB_EVENT_HEAD_COMMIT_MESSAGE:-$(git log -1 --pretty=%B 2>/dev/null || true)}"
echo "$BRANCH" "$MSG" | grep -qi '@skipci' && SKIP=1
echo "skip=$SKIP" >> $GITHUB_OUTPUT
python-ci:
needs: skip-ci-check
if: needs.skip-ci-check.outputs.should-skip != '1'
runs-on: [homelab, self-hosted, linux, python]
container:
image: node:20-bookworm
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4
- name: Install Python + deps
run: |
apt-get update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq python3 python3-pip python3-venv
python3 -m venv .ci-venv
.ci-venv/bin/pip install -q -U pip
.ci-venv/bin/pip install -q -e ".[dev]"
- name: Pytest
run: .ci-venv/bin/pytest -q
secret-scan:
needs: skip-ci-check
if: needs.skip-ci-check.outputs.should-skip != '1'
runs-on: [homelab, self-hosted, linux, heavy]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Gitleaks
run: |
docker run --rm -v "$PWD:/repo" ghcr.io/gitleaks/gitleaks:latest \
detect --source /repo --no-banner --redact

14
.gitignore vendored Normal file
View File

@ -0,0 +1,14 @@
.env
data/
ollama/
backend/.venv/
backend/*.egg-info/
*.egg-info/
__pycache__/
*.pyc
.pytest_cache/
.ruff_cache/
.mypy_cache/
.DS_Store
*.db
*.log

13
Caddyfile.snippet Normal file
View File

@ -0,0 +1,13 @@
talos.levkin.ca {
reverse_proxy localhost:8000
@websockets {
header Connection *Upgrade*
header Upgrade websocket
}
reverse_proxy @websockets localhost:8000
basicauth {
admin $2a$14$REPLACE_WITH_BCRYPT_HASH
}
}

104
README.md Normal file
View File

@ -0,0 +1,104 @@
# TALOS
**Legal warning:** Talos runs real offensive security tools (Nmap, Nikto, Gobuster, sqlmap, Hydra, etc.). Use **only** against systems you own or have **written authorization** to test. Unauthorized access is illegal. The UI requires an explicit authorization checkbox before any scan starts.
Self-hosted pentest orchestrator with live tool output, WebSocket streaming, and Ollama-powered finding analysis.
## What we use it for (Levkin)
Primary use: **bulk-scan our own edge-facing sites** (Caddy/`*.levkin.ca`) and LAN app ports before/after exposing them — headers, directory enum, Nikto, optional redteam profile. Scan results stay in local `data/` (gitignored). Not a general internet scanner.
## Homelab deployment (Levkin / Ansible)
From `~/Documents/code/ansible/inventories/production/hosts`:
| Host | IP | Use for Talos |
|------|-----|------------------|
| **devGPU** | `10.0.10.122:11434` | **Ollama + RTX 4080** — point `OLLAMA_URL` here (do not run bundled Ollama on a small host) |
| **edge** (LXC 222) | `10.0.10.20` | Caddy reverse proxy — add `Caddyfile.snippet` |
| **automationlab** (225) | `10.0.10.45` | Docker stacks already; disk ~85% full — possible but tight |
| **PVE.BU.SVR** (VM 200) | `10.0.10.200` | Lab VM — good candidate for heavy Kali-based container |
| **dev02** (P53) | `10.0.10.100` | QA workstation — local dev / smoke tests |
| **osiris** (228) | `10.0.10.40` | OSINT (different tool) — not Talos |
| **atlas** (237) | `10.0.10.71` | Atlas brain — already uses devGPU for LLM |
**Recommended:** Run Talos on **PVE.BU.SVR** or a new pve10 LXC in `.40.79`, with:
```env
OLLAMA_URL=http://10.0.10.122:11434
OLLAMA_MODEL=qwen2.5:14b
```
Verify Ollama from the LAN:
```bash
curl -s http://10.0.10.122:11434/api/tags
```
## Quickstart
```bash
cd talos
docker compose --profile build-only build tools
docker compose build talos
# Option A: bundled Ollama (needs GPU profile on host)
docker compose --profile bundled-ollama up -d
docker exec -it $(docker compose ps -q ollama) ollama pull qwen2.5:14b
# Option B: homelab devGPU (recommended)
echo 'OLLAMA_URL=http://10.0.10.122:11434' >> .env
docker compose up -d talos
```
Open http://localhost:8000
**First scan (loopback):** set `ALLOW_LOOPBACK=true` in `.env`, target `127.0.0.1`, Passive profile.
**Legal external target:** `scanme.nmap.org`, Standard profile, authorization checked.
## Profiles
| Profile | Tools |
|---------|--------|
| **passive** | nmap discovery, port scan, service detect, optional testssl |
| **standard** | passive + whatweb, curl headers, gobuster, nikto |
| **redteam** | standard + sqlmap, optional hydra SSH brute, searchsploit |
## Caddy
See `Caddyfile.snippet` on edge `@ 10.0.10.20` for `talos.levkin.ca`. Use basic auth — this tool is dangerous.
## Hardening
- Run on a management VLAN; no WAN exposure without auth
- Use `BLOCKED_TARGETS` for sensitive IPs
- Prefer devGPU for LLM to avoid OOM on small GPUs
- `cap_add: NET_RAW` required for `nmap -sS`
## Tests
```bash
cd backend && pip install -e ".[dev]" && pytest tests
# or
docker compose exec talos pytest
```
## Architecture
```
Browser ──WS/REST──► talos (FastAPI)
├── subprocess ──► Kali tools (same image)
└── httpx ──────► Ollama (devGPU or container)
└── SQLite (/data)
```
## Troubleshooting
| Issue | Fix |
|-------|-----|
| nmap SYN scan fails | Ensure `NET_RAW` / `NET_ADMIN` in compose |
| Ollama OOM | Use devGPU; set `OLLAMA_MAX_LOADED_MODELS=1` on host |
| sqlmap errors | Writable `/tmp` in container |
| Tools badge degraded | Rebuild `tools` image; check PATH |

16
backend/Dockerfile Normal file
View File

@ -0,0 +1,16 @@
FROM talos-tools:latest
RUN apt-get update && apt-get install -y --no-install-recommends \
python3-venv \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY pyproject.toml ./
RUN python3 -m venv /venv && /venv/bin/pip install --upgrade pip \
&& /venv/bin/pip install -e .
COPY app ./app
ENV PATH="/venv/bin:$PATH"
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

1
backend/app/__init__.py Normal file
View File

@ -0,0 +1 @@
"""Talos pentest orchestrator."""

25
backend/app/config.py Normal file
View File

@ -0,0 +1,25 @@
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
ollama_url: str = "http://ollama:11434"
ollama_model: str = "qwen2.5:14b"
allow_loopback: bool = False
blocked_targets: str = ""
db_path: str = "/data/talos.db"
audit_log_path: str = "/data/audit.log"
# Caps parallel tool runs (CPU/RAM/network), not total queued scans.
max_concurrent_scans: int = 5
tool_timeout_seconds: int = 600
llm_output_max_chars: int = 15_000
@property
def blocked_targets_list(self) -> list[str]:
if not self.blocked_targets.strip():
return []
return [t.strip() for t in self.blocked_targets.split(",") if t.strip()]
settings = Settings()

424
backend/app/db.py Normal file
View File

@ -0,0 +1,424 @@
import json
import uuid
from datetime import datetime, timezone
from typing import Any
import aiosqlite
from app.config import settings
from app.models import BulkRun, Finding, Phase, Scan, ScanOptions
_db: aiosqlite.Connection | None = None
def _utcnow() -> datetime:
return datetime.now(timezone.utc)
def _dt_iso(dt: datetime | None) -> str | None:
if dt is None:
return None
return dt.isoformat()
def _parse_dt(s: str | None) -> datetime | None:
if not s:
return None
return datetime.fromisoformat(s)
async def init_db() -> None:
global _db
_db = await aiosqlite.connect(settings.db_path)
_db.row_factory = aiosqlite.Row
await _db.executescript(
"""
CREATE TABLE IF NOT EXISTS scans (
id TEXT PRIMARY KEY,
target TEXT NOT NULL,
profile TEXT NOT NULL,
options_json TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
completed_at TEXT,
llm_unavailable INTEGER DEFAULT 0
);
CREATE TABLE IF NOT EXISTS phases (
id TEXT PRIMARY KEY,
scan_id TEXT NOT NULL,
name TEXT NOT NULL,
tool TEXT NOT NULL,
command TEXT NOT NULL,
status TEXT NOT NULL,
started_at TEXT,
completed_at TEXT,
raw_output TEXT DEFAULT '',
exit_code INTEGER,
sort_order INTEGER NOT NULL,
FOREIGN KEY (scan_id) REFERENCES scans(id)
);
CREATE TABLE IF NOT EXISTS findings (
id TEXT PRIMARY KEY,
scan_id TEXT NOT NULL,
phase_id TEXT NOT NULL,
severity TEXT NOT NULL,
title TEXT NOT NULL,
description TEXT NOT NULL,
evidence TEXT NOT NULL,
recommendation TEXT NOT NULL,
cve_refs_json TEXT DEFAULT '[]',
confidence TEXT NOT NULL,
FOREIGN KEY (scan_id) REFERENCES scans(id)
);
CREATE TABLE IF NOT EXISTS bulk_runs (
id TEXT PRIMARY KEY,
profile TEXT NOT NULL,
status TEXT NOT NULL,
created_at TEXT NOT NULL,
completed_at TEXT
);
"""
)
await _migrate_schema()
await _db.commit()
async def _migrate_schema() -> None:
"""Add columns/tables introduced after initial deploy."""
cursor = await _conn().execute("PRAGMA table_info(scans)")
cols = {row[1] for row in await cursor.fetchall()}
if "bulk_id" not in cols:
await _conn().execute("ALTER TABLE scans ADD COLUMN bulk_id TEXT")
async def close_db() -> None:
global _db
if _db:
await _db.close()
_db = None
def _conn() -> aiosqlite.Connection:
if _db is None:
raise RuntimeError("Database not initialized")
return _db
async def create_scan(
target: str,
profile: str,
options: ScanOptions,
*,
bulk_id: str | None = None,
) -> Scan:
scan_id = str(uuid.uuid4())
now = _utcnow()
scan = Scan(
id=scan_id,
target=target,
profile=profile, # type: ignore[arg-type]
options=options,
status="queued",
created_at=now,
bulk_id=bulk_id,
)
await _conn().execute(
"""INSERT INTO scans (id, target, profile, options_json, status, created_at, bulk_id)
VALUES (?,?,?,?,?,?,?)""",
(scan_id, target, profile, options.model_dump_json(), "queued", _dt_iso(now), bulk_id),
)
await _conn().commit()
return scan
async def create_bulk_run(profile: str, total: int) -> BulkRun:
bulk_id = str(uuid.uuid4())
now = _utcnow()
await _conn().execute(
"INSERT INTO bulk_runs (id, profile, status, created_at) VALUES (?,?,?,?)",
(bulk_id, profile, "queued", _dt_iso(now)),
)
await _conn().commit()
return BulkRun(
id=bulk_id,
profile=profile, # type: ignore[arg-type]
status="queued",
created_at=now,
total=total,
)
async def _count_bulk_scans(bulk_id: str) -> dict[str, int]:
cursor = await _conn().execute(
"""SELECT status, COUNT(*) AS n FROM scans WHERE bulk_id=? GROUP BY status""",
(bulk_id,),
)
rows = await cursor.fetchall()
counts: dict[str, int] = {}
for r in rows:
counts[r["status"]] = int(r["n"])
total = sum(counts.values())
terminal = {"complete", "failed", "cancelled"}
done = sum(counts.get(s, 0) for s in terminal)
return {
"total": total,
"complete": counts.get("complete", 0),
"failed": counts.get("failed", 0),
"running": counts.get("running", 0),
"queued": counts.get("queued", 0),
"done": done,
}
async def refresh_bulk_run_status(bulk_id: str) -> BulkRun | None:
bulk = await get_bulk_run(bulk_id)
if not bulk:
return None
stats = await _count_bulk_scans(bulk_id)
if stats["total"] == 0:
return bulk
if stats["running"] > 0 or (stats["queued"] > 0 and stats["done"] > 0):
new_status = "running"
elif stats["queued"] == stats["total"]:
new_status = "queued"
elif stats["done"] == stats["total"]:
new_status = "complete"
else:
new_status = "running"
completed_at = _dt_iso(_utcnow()) if new_status == "complete" else None
await _conn().execute(
"UPDATE bulk_runs SET status=?, completed_at=COALESCE(?, completed_at) WHERE id=?",
(new_status, completed_at, bulk_id),
)
await _conn().commit()
return await get_bulk_run(bulk_id)
async def get_bulk_run(bulk_id: str) -> BulkRun | None:
row = await (await _conn().execute("SELECT * FROM bulk_runs WHERE id=?", (bulk_id,))).fetchone()
if not row:
return None
stats = await _count_bulk_scans(bulk_id)
return BulkRun(
id=row["id"],
profile=row["profile"], # type: ignore[arg-type]
status=row["status"], # type: ignore[arg-type]
created_at=_parse_dt(row["created_at"]) or _utcnow(),
completed_at=_parse_dt(row["completed_at"]),
total=stats["total"],
complete=stats["complete"],
failed=stats["failed"],
running=stats["running"],
queued=stats["queued"],
)
async def list_scans_for_bulk(bulk_id: str) -> list[Scan]:
cursor = await _conn().execute(
"SELECT * FROM scans WHERE bulk_id=? ORDER BY created_at ASC",
(bulk_id,),
)
rows = await cursor.fetchall()
return [_row_to_scan(r) for r in rows]
async def has_active_scan_for_target(target: str) -> bool:
row = await (
await _conn().execute(
"SELECT 1 FROM scans WHERE target=? AND status IN ('queued','running') LIMIT 1",
(target,),
)
).fetchone()
return row is not None
def _row_to_scan(row: aiosqlite.Row) -> Scan:
bulk_id = row["bulk_id"] if "bulk_id" in row.keys() else None
return Scan(
id=row["id"],
target=row["target"],
profile=row["profile"], # type: ignore[arg-type]
options=ScanOptions.model_validate_json(row["options_json"]),
status=row["status"], # type: ignore[arg-type]
created_at=_parse_dt(row["created_at"]) or _utcnow(),
completed_at=_parse_dt(row["completed_at"]),
llm_unavailable=bool(row["llm_unavailable"]),
bulk_id=bulk_id,
)
async def update_scan_status(
scan_id: str,
status: str,
llm_unavailable: bool | None = None,
) -> None:
completed = _dt_iso(_utcnow()) if status in ("complete", "failed", "cancelled") else None
if llm_unavailable is not None:
await _conn().execute(
"UPDATE scans SET status=?, completed_at=?, llm_unavailable=? WHERE id=?",
(status, completed, int(llm_unavailable), scan_id),
)
else:
await _conn().execute(
"UPDATE scans SET status=?, completed_at=? WHERE id=?",
(status, completed, scan_id),
)
await _conn().commit()
async def get_scan(scan_id: str) -> Scan | None:
row = await (await _conn().execute("SELECT * FROM scans WHERE id=?", (scan_id,))).fetchone()
if not row:
return None
return _row_to_scan(row)
async def list_scans(limit: int = 50) -> list[Scan]:
cursor = await _conn().execute(
"SELECT * FROM scans ORDER BY created_at DESC LIMIT ?", (limit,)
)
rows = await cursor.fetchall()
return [_row_to_scan(r) for r in rows]
async def get_bulk_export_payload(bulk_id: str) -> dict[str, Any] | None:
bulk = await get_bulk_run(bulk_id)
if not bulk:
return None
scans = await list_scans_for_bulk(bulk_id)
scan_details: list[dict[str, Any]] = []
for s in scans:
phases = await get_phases(s.id)
findings = await get_findings(s.id)
scan_details.append(
{
"scan": s.model_dump(mode="json"),
"phases": [p.model_dump(mode="json") for p in phases],
"findings": [f.model_dump(mode="json") for f in findings],
}
)
return {
"bulk": bulk.model_dump(mode="json"),
"scans": scan_details,
}
async def create_phases(scan_id: str, phases: list[tuple[str, str, str, str]]) -> list[Phase]:
"""phases: (id, name, tool, command)"""
out: list[Phase] = []
for i, (pid, name, tool, command) in enumerate(phases):
ph = Phase(
id=pid,
scan_id=scan_id,
name=name,
tool=tool,
command=command,
status="pending",
)
await _conn().execute(
"""INSERT INTO phases (id, scan_id, name, tool, command, status, sort_order)
VALUES (?,?,?,?,?,?,?)""",
(pid, scan_id, name, tool, command, "pending", i),
)
out.append(ph)
await _conn().commit()
return out
async def update_phase(
phase_id: str,
status: str,
raw_output: str | None = None,
exit_code: int | None = None,
started: bool = False,
) -> None:
now = _dt_iso(_utcnow())
if started:
await _conn().execute(
"UPDATE phases SET status=?, started_at=? WHERE id=?",
(status, now, phase_id),
)
elif raw_output is not None:
await _conn().execute(
"UPDATE phases SET status=?, completed_at=?, raw_output=?, exit_code=? WHERE id=?",
(status, now, raw_output, exit_code, phase_id),
)
else:
await _conn().execute("UPDATE phases SET status=? WHERE id=?", (status, phase_id))
await _conn().commit()
async def append_phase_output(phase_id: str, line: str) -> None:
await _conn().execute(
"UPDATE phases SET raw_output = raw_output || ? || char(10) WHERE id=?",
(line, phase_id),
)
await _conn().commit()
async def get_phases(scan_id: str) -> list[Phase]:
cursor = await _conn().execute(
"SELECT * FROM phases WHERE scan_id=? ORDER BY sort_order", (scan_id,)
)
rows = await cursor.fetchall()
return [
Phase(
id=r["id"],
scan_id=r["scan_id"],
name=r["name"],
tool=r["tool"],
command=r["command"],
status=r["status"], # type: ignore[arg-type]
started_at=_parse_dt(r["started_at"]),
completed_at=_parse_dt(r["completed_at"]),
raw_output=r["raw_output"] or "",
exit_code=r["exit_code"],
)
for r in rows
]
async def save_finding(finding: Finding) -> None:
await _conn().execute(
"""INSERT INTO findings
(id, scan_id, phase_id, severity, title, description, evidence, recommendation, cve_refs_json, confidence)
VALUES (?,?,?,?,?,?,?,?,?,?)""",
(
finding.id,
finding.scan_id,
finding.phase_id,
finding.severity,
finding.title,
finding.description,
finding.evidence,
finding.recommendation,
json.dumps(finding.cve_refs),
finding.confidence,
),
)
await _conn().commit()
async def get_findings(scan_id: str) -> list[Finding]:
cursor = await _conn().execute(
"SELECT * FROM findings WHERE scan_id=? ORDER BY severity", (scan_id,)
)
rows = await cursor.fetchall()
severity_order = {"critical": 0, "high": 1, "medium": 2, "low": 3, "info": 4}
findings = [
Finding(
id=r["id"],
scan_id=r["scan_id"],
phase_id=r["phase_id"],
severity=r["severity"], # type: ignore[arg-type]
title=r["title"],
description=r["description"],
evidence=r["evidence"],
recommendation=r["recommendation"],
cve_refs=json.loads(r["cve_refs_json"] or "[]"),
confidence=r["confidence"], # type: ignore[arg-type]
)
for r in rows
]
findings.sort(key=lambda f: severity_order.get(f.severity, 99))
return findings

148
backend/app/export.py Normal file
View File

@ -0,0 +1,148 @@
"""Bulk scan report rendering for Cursor and other consumers."""
from __future__ import annotations
from typing import Any
_SEVERITY_ORDER = ("critical", "high", "medium", "low", "info")
_PHASE_OUTPUT_MAX = 4000
def _sort_findings(findings: list[dict[str, Any]]) -> list[dict[str, Any]]:
order = {s: i for i, s in enumerate(_SEVERITY_ORDER)}
return sorted(findings, key=lambda f: order.get(str(f.get("severity", "")).lower(), 99))
def _truncate(text: str, limit: int = _PHASE_OUTPUT_MAX) -> str:
text = (text or "").strip()
if len(text) <= limit:
return text
return text[: limit - 40] + "\n\n… (truncated)"
def render_bulk_markdown(payload: dict[str, Any], *, for_cursor: bool = False) -> str:
bulk = payload["bulk"]
scans = payload["scans"]
lines: list[str] = []
if for_cursor:
lines.extend(
[
"<!-- Talos bulk report — paste into Cursor for triage / remediation -->",
"",
"You are reviewing authorized security scan results from Talos.",
"Prioritize critical/high findings, then failed phases that may hide issues.",
"",
]
)
lines.extend(
[
f"# Talos bulk report",
"",
f"- **Bulk ID:** `{bulk['id']}`",
f"- **Profile:** {bulk['profile']}",
f"- **Status:** {bulk['status']}",
f"- **Progress:** {bulk['complete']}/{bulk['total']} complete"
+ (f", {bulk['failed']} failed" if bulk.get("failed") else "")
+ (f", {bulk['running']} running" if bulk.get("running") else "")
+ (f", {bulk['queued']} queued" if bulk.get("queued") else ""),
"",
]
)
# Cross-target finding rollup (severity >= low with issues)
all_findings: list[tuple[str, dict[str, Any]]] = []
for entry in scans:
target = entry["scan"]["target"]
for f in entry.get("findings") or []:
if f.get("severity") in ("critical", "high", "medium", "low"):
all_findings.append((target, f))
if all_findings:
lines.append("## Findings rollup (all targets)")
lines.append("")
for target, f in sorted(
all_findings,
key=lambda x: _SEVERITY_ORDER.index(x[1]["severity"])
if x[1]["severity"] in _SEVERITY_ORDER
else 99,
):
sev = f["severity"].upper()
lines.append(f"- **[{sev}]** `{target}` — {f['title']}")
lines.append("")
for entry in scans:
scan = entry["scan"]
phases = entry.get("phases") or []
findings = _sort_findings(entry.get("findings") or [])
lines.append(f"## {scan['target']}")
lines.append("")
lines.append(f"- Scan ID: `{scan['id']}`")
lines.append(f"- Status: **{scan['status']}**")
if scan.get("llm_unavailable"):
lines.append("- LLM analysis was unavailable for some phases")
lines.append("")
lines.append("### Phases")
lines.append("")
lines.append("| Phase | Status | Tool |")
lines.append("|-------|--------|------|")
for p in phases:
lines.append(f"| {p['name']} | {p['status']} | {p.get('tool', '')} |")
lines.append("")
failed_phases = [p for p in phases if p["status"] in ("failed", "blocked", "inconclusive")]
if failed_phases:
lines.append("### Phase notes (non-complete)")
lines.append("")
for p in failed_phases:
snippet = _truncate(p.get("raw_output") or "", 800)
lines.append(f"**{p['name']}** ({p['status']})")
if snippet:
lines.append("")
lines.append("```")
lines.append(snippet)
lines.append("```")
lines.append("")
if findings:
lines.append("### Findings")
lines.append("")
for f in findings:
sev = str(f["severity"]).upper()
lines.append(f"#### [{sev}] {f['title']}")
lines.append("")
lines.append(f["description"])
lines.append("")
if f.get("evidence"):
lines.append(f"**Evidence:** `{_truncate(str(f['evidence']), 500)}`")
lines.append("")
lines.append(f"**Recommendation:** {f['recommendation']}")
lines.append("")
else:
lines.append("_No LLM findings recorded._")
lines.append("")
lines.append("---")
lines.append("")
return "\n".join(lines).rstrip() + "\n"
def render_bulk_json(payload: dict[str, Any]) -> dict[str, Any]:
"""JSON export with truncated phase output to keep size reasonable."""
out = dict(payload)
scans_out: list[dict[str, Any]] = []
for entry in payload.get("scans") or []:
entry = dict(entry)
phases = []
for p in entry.get("phases") or []:
p = dict(p)
if p.get("raw_output"):
p["raw_output"] = _truncate(str(p["raw_output"]))
phases.append(p)
entry["phases"] = phases
scans_out.append(entry)
out["scans"] = scans_out
return out

152
backend/app/llm.py Normal file
View File

@ -0,0 +1,152 @@
import json
import uuid
from typing import Any
import httpx
import structlog
from app.config import settings
from app.models import Finding, Phase
log = structlog.get_logger(__name__)
SYSTEM_PROMPT = """You are a senior penetration tester analyzing raw security tool output.
Identify real vulnerabilities (not noise), assign accurate severity, and write concrete remediation steps.
Return ONLY valid JSON matching this schema:
{
"findings": [
{
"severity": "critical|high|medium|low|info",
"title": "Short specific title",
"description": "What the issue is and why it matters",
"evidence": "Exact snippet from the tool output that shows this",
"recommendation": "Concrete fix steps the operator can take",
"cve_refs": ["CVE-YYYY-XXXX"],
"confidence": "high|medium|low"
}
]
}
Rules:
- Only report real findings. Empty findings list is valid.
- "info" severity for benign observations (open standard ports without issues).
- Cite specific CVEs only when version info confirms them.
- Be concise. No marketing language. No filler.
"""
USER_TEMPLATE = """Target: {target}
Tool: {tool}
Command: {command}
Profile: {profile}
Raw output:
{output}
Analyze and return JSON."""
def truncate_output(text: str, max_chars: int | None = None) -> str:
max_chars = max_chars or settings.llm_output_max_chars
if len(text) <= max_chars:
return text
half = max_chars // 2
return text[:half] + "\n\n...[truncated]...\n\n" + text[-half:]
async def ollama_health() -> bool:
try:
async with httpx.AsyncClient(timeout=5.0) as client:
r = await client.get(f"{settings.ollama_url.rstrip('/')}/api/tags")
return r.status_code == 200
except httpx.HTTPError:
return False
async def _chat(messages: list[dict[str, str]], model: str) -> str:
url = f"{settings.ollama_url.rstrip('/')}/api/chat"
payload: dict[str, Any] = {
"model": model,
"messages": messages,
"stream": False,
"format": "json",
}
async with httpx.AsyncClient(timeout=120.0) as client:
r = await client.post(url, json=payload)
r.raise_for_status()
data = r.json()
return str(data.get("message", {}).get("content", ""))
def _parse_findings_json(
raw: str,
scan_id: str,
phase: Phase,
) -> list[Finding]:
parsed = json.loads(raw)
items = parsed.get("findings", []) if isinstance(parsed, dict) else []
findings: list[Finding] = []
for item in items:
try:
f = Finding(
id=str(uuid.uuid4()),
scan_id=scan_id,
phase_id=phase.id,
severity=item["severity"],
title=item["title"],
description=item["description"],
evidence=item.get("evidence", ""),
recommendation=item.get("recommendation", ""),
cve_refs=item.get("cve_refs", []),
confidence=item.get("confidence", "medium"),
)
findings.append(f)
except Exception as exc:
log.warning("finding_validation_failed", error=str(exc), item=item)
return findings
async def analyze_phase(
phase: Phase,
output: str,
*,
target: str,
profile: str,
model: str,
) -> tuple[list[Finding], bool]:
"""Returns (findings, llm_unavailable)."""
if not await ollama_health():
return [], True
truncated = truncate_output(output)
user_msg = USER_TEMPLATE.format(
target=target,
tool=phase.tool,
command=phase.command,
profile=profile,
output=truncated,
)
messages: list[dict[str, str]] = [
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": user_msg},
]
for attempt in range(2):
try:
content = await _chat(messages, model)
return _parse_findings_json(content, phase.scan_id, phase), False
except json.JSONDecodeError:
if attempt == 0:
messages.append(
{
"role": "user",
"content": "Your previous response was not valid JSON — try again.",
}
)
else:
log.warning("llm_json_parse_failed", phase_id=phase.id)
except httpx.HTTPError as exc:
log.warning("llm_request_failed", error=str(exc))
return [], True
return [], False

276
backend/app/main.py Normal file
View File

@ -0,0 +1,276 @@
import asyncio
import json
from collections import defaultdict
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Any
import httpx
import structlog
from fastapi import FastAPI, HTTPException, Request, WebSocket, WebSocketDisconnect
from fastapi.responses import FileResponse, JSONResponse, PlainTextResponse
from fastapi.staticfiles import StaticFiles
from app import db, export, llm, profiles, scanner, tools
from app.config import settings
from app.models import BulkScanCreate, ScanCreate
from app.safety import validate_scan_request
structlog.configure(processors=[structlog.processors.JSONRenderer()])
log = structlog.get_logger(__name__)
def _frontend_dir() -> Path:
here = Path(__file__).resolve()
for candidate in (here.parent.parent / "frontend", here.parent.parent.parent / "frontend"):
if (candidate / "index.html").exists():
return candidate
return here.parent.parent.parent / "frontend"
FRONTEND_DIR = _frontend_dir()
@asynccontextmanager
async def lifespan(_app: FastAPI):
Path(settings.db_path).parent.mkdir(parents=True, exist_ok=True)
await db.init_db()
yield
await db.close_db()
app = FastAPI(title="Talos", lifespan=lifespan)
if FRONTEND_DIR.exists():
app.mount("/static", StaticFiles(directory=FRONTEND_DIR), name="static")
@app.get("/")
async def index() -> FileResponse:
return FileResponse(FRONTEND_DIR / "index.html")
@app.get("/api/health")
async def health() -> dict[str, Any]:
ollama_ok = await llm.ollama_health()
tool_checks: dict[str, str] = {}
for name, cmd in [("nmap", ["nmap", "--version"]), ("curl", ["curl", "--version"])]:
try:
r = await tools.run_tool(cmd, timeout=10)
tool_checks[name] = "ok" if r.exit_code == 0 else "error"
except Exception:
tool_checks[name] = "error"
return {
"ollama": "ok" if ollama_ok else "down",
"tools": tool_checks,
"db": "ok",
"max_concurrent_scans": settings.max_concurrent_scans,
}
@app.get("/api/profiles")
async def api_profiles() -> list[dict[str, object]]:
return profiles.get_profiles_payload()
@app.post("/api/scans")
async def create_scan(body: ScanCreate, request: Request) -> dict[str, str]:
target = validate_scan_request(body.target, body.authorized)
busy, reason = await scanner.target_busy(target)
if busy:
raise HTTPException(status_code=429, detail=reason)
scan = await db.create_scan(target, body.profile, body.options)
source_ip = request.client.host if request.client else "unknown"
await scanner.queue_scan(scan, source_ip=source_ip)
return {"scan_id": scan.id}
@app.post("/api/scans/bulk")
async def create_bulk_scans(body: BulkScanCreate, request: Request) -> dict[str, Any]:
source_ip = request.client.host if request.client else "unknown"
targets = [t.strip() for t in (body.targets or []) if t and t.strip()]
if not targets:
raise HTTPException(status_code=400, detail="At least one target is required")
bulk = await db.create_bulk_run(body.profile, total=0)
queued: list[dict[str, str]] = []
rejected: list[dict[str, str]] = []
seen_targets: set[str] = set()
for raw in targets:
try:
target = validate_scan_request(raw, body.authorized)
if target in seen_targets:
rejected.append({"target": target, "reason": "Duplicate target in this bulk list"})
continue
seen_targets.add(target)
busy, reason = await scanner.target_busy(target)
if busy:
rejected.append({"target": target, "reason": reason})
continue
scan = await db.create_scan(
target, body.profile, body.options, bulk_id=bulk.id
)
await scanner.queue_scan(scan, source_ip=source_ip)
queued.append({"target": target, "scan_id": scan.id})
except HTTPException as exc:
rejected.append({"target": raw, "reason": str(exc.detail)})
await db.refresh_bulk_run_status(bulk.id)
bulk = await db.get_bulk_run(bulk.id)
return {
"bulk_id": bulk.id,
"queued": queued,
"rejected": rejected,
"bulk": bulk.model_dump(mode="json") if bulk else None,
}
@app.get("/api/bulks/{bulk_id}")
async def get_bulk_status(bulk_id: str) -> dict[str, Any]:
bulk = await db.refresh_bulk_run_status(bulk_id)
if not bulk:
raise HTTPException(status_code=404, detail="Bulk run not found")
scans = await db.list_scans_for_bulk(bulk_id)
return {
"bulk": bulk.model_dump(mode="json"),
"scans": [
{
"id": s.id,
"target": s.target,
"status": s.status,
"created_at": s.created_at.isoformat(),
}
for s in scans
],
}
@app.get("/api/bulks/{bulk_id}/export")
async def export_bulk(bulk_id: str, format: str = "cursor") -> Any:
payload = await db.get_bulk_export_payload(bulk_id)
if not payload:
raise HTTPException(status_code=404, detail="Bulk run not found")
if format in ("md", "cursor"):
text = export.render_bulk_markdown(payload, for_cursor=(format == "cursor"))
return PlainTextResponse(text, media_type="text/markdown")
if format == "json":
return JSONResponse(export.render_bulk_json(payload))
raise HTTPException(status_code=400, detail="format must be cursor, md, or json")
@app.get("/api/scans")
async def list_scans() -> list[dict[str, Any]]:
scans = await db.list_scans()
return [
{
"id": s.id,
"target": s.target,
"profile": s.profile,
"status": s.status,
"created_at": s.created_at.isoformat(),
}
for s in scans
]
@app.get("/api/scans/{scan_id}")
async def get_scan_detail(scan_id: str) -> dict[str, Any]:
scan = await db.get_scan(scan_id)
if not scan:
raise HTTPException(status_code=404, detail="Scan not found")
phases = await db.get_phases(scan_id)
findings = await db.get_findings(scan_id)
return {
"scan": scan.model_dump(mode="json"),
"phases": [p.model_dump(mode="json") for p in phases],
"findings": [f.model_dump(mode="json") for f in findings],
}
@app.get("/api/scans/{scan_id}/findings")
async def get_scan_findings(scan_id: str) -> dict[str, list[dict[str, Any]]]:
scan = await db.get_scan(scan_id)
if not scan:
raise HTTPException(status_code=404, detail="Scan not found")
findings = await db.get_findings(scan_id)
grouped: dict[str, list[dict[str, Any]]] = defaultdict(list)
for f in findings:
grouped[f.severity].append(f.model_dump(mode="json"))
return dict(grouped)
@app.post("/api/scans/{scan_id}/cancel")
async def cancel_scan(scan_id: str) -> dict[str, bool]:
scan = await db.get_scan(scan_id)
if not scan:
raise HTTPException(status_code=404, detail="Scan not found")
await scanner.cancel_scan(scan_id)
return {"cancelled": True}
@app.get("/api/scans/{scan_id}/export")
async def export_scan(scan_id: str, format: str = "json") -> Any:
detail = await get_scan_detail(scan_id)
if format == "md":
lines = [
f"# Talos Report — {detail['scan']['target']}",
f"Profile: {detail['scan']['profile']}",
f"Status: {detail['scan']['status']}",
"",
]
for f in detail["findings"]:
lines.append(f"## [{f['severity'].upper()}] {f['title']}")
lines.append(f["description"])
lines.append(f"**Recommendation:** {f['recommendation']}")
lines.append("")
return PlainTextResponse("\n".join(lines), media_type="text/markdown")
return JSONResponse(detail)
@app.websocket("/ws/scans/{scan_id}")
async def ws_scan(websocket: WebSocket, scan_id: str) -> None:
scan = await db.get_scan(scan_id)
if not scan:
await websocket.close(code=4004)
return
await websocket.accept()
bc = scanner.get_broadcaster(scan_id)
queue = bc.subscribe()
try:
# If the client connects immediately after queueing a scan, the scan task may not
# have created phase rows yet. Poll briefly so the UI can render the full phase list.
phases = await db.get_phases(scan_id)
if not phases and scan.status in ("queued", "running"):
for _ in range(20): # ~2s total
await asyncio.sleep(0.1)
phases = await db.get_phases(scan_id)
if phases:
break
for p in phases:
await websocket.send_json(
{"type": "phase", "phase_id": p.id, "phase": p.name, "status": p.status}
)
findings = await db.get_findings(scan_id)
for f in findings:
await websocket.send_json({"type": "finding", "finding": f.model_dump(mode="json")})
while True:
try:
msg = await asyncio.wait_for(queue.get(), timeout=1.0)
await websocket.send_json(msg)
if msg.get("type") == "done":
break
except TimeoutError:
if not scanner.is_scan_active(scan_id):
current = await db.get_scan(scan_id)
if current and current.status in ("complete", "failed", "cancelled"):
await websocket.send_json(
{"type": "done", "scan_id": scan_id, "status": current.status}
)
break
except WebSocketDisconnect:
pass
finally:
if queue in bc._queues:
bc._queues.remove(queue)

84
backend/app/models.py Normal file
View File

@ -0,0 +1,84 @@
from datetime import datetime
from typing import Literal
from pydantic import BaseModel, Field
class ScanOptions(BaseModel):
port_range: str = "1-1000"
include_web: bool = True
include_brute: bool = False
include_ssl: bool = True
llm_model: str = "qwen2.5:14b"
class ScanCreate(BaseModel):
target: str
profile: Literal["passive", "standard", "redteam"]
options: ScanOptions = Field(default_factory=ScanOptions)
authorized: bool = False
class BulkScanCreate(BaseModel):
targets: list[str]
profile: Literal["passive", "standard", "redteam"]
options: ScanOptions = Field(default_factory=ScanOptions)
authorized: bool = False
class Scan(BaseModel):
id: str
target: str
profile: Literal["passive", "standard", "redteam"]
options: ScanOptions
status: Literal["queued", "running", "complete", "failed", "cancelled"]
created_at: datetime
completed_at: datetime | None = None
llm_unavailable: bool = False
bulk_id: str | None = None
class BulkRun(BaseModel):
id: str
profile: Literal["passive", "standard", "redteam"]
status: Literal["queued", "running", "complete", "cancelled"]
created_at: datetime
completed_at: datetime | None = None
total: int = 0
complete: int = 0
failed: int = 0
running: int = 0
queued: int = 0
class Phase(BaseModel):
id: str
scan_id: str
name: str
tool: str
command: str
status: Literal["pending", "running", "complete", "failed", "skipped", "blocked", "inconclusive"]
started_at: datetime | None = None
completed_at: datetime | None = None
raw_output: str = ""
exit_code: int | None = None
class Finding(BaseModel):
id: str
scan_id: str
phase_id: str
severity: Literal["critical", "high", "medium", "low", "info"]
title: str
description: str
evidence: str
recommendation: str
cve_refs: list[str] = Field(default_factory=list)
confidence: Literal["high", "medium", "low"]
class ToolResult(BaseModel):
stdout: str
stderr: str
exit_code: int
duration: float

241
backend/app/profiles.py Normal file
View File

@ -0,0 +1,241 @@
from dataclasses import dataclass
from typing import Callable
import shlex
import re
from app.models import ScanOptions
ProfileName = str
@dataclass(frozen=True)
class PhaseDef:
name: str
tool: str
command_template: str
requires: tuple[str, ...] = ()
success_exit_codes: tuple[int, ...] = (0,)
timeout_seconds: int | None = None
retries: int = 0
retry_delay_seconds: float = 0.0
llm_analyze: bool = True
def should_run(self, options: ScanOptions) -> bool:
if "include_web" in self.requires and not options.include_web:
return False
if "include_brute" in self.requires and not options.include_brute:
return False
if "include_ssl" in self.requires and not options.include_ssl:
return False
return True
def _target_url(target: str) -> str:
if target.startswith("http://") or target.startswith("https://"):
return target
return f"http://{target}"
def _https_target_url(target: str) -> str:
if target.startswith("https://"):
return target
if target.startswith("http://"):
return "https://" + target.removeprefix("http://")
return f"https://{target}"
def _root_domain(target: str) -> str:
# Best-effort heuristic for homelab/public domains like levkin.ca.
# If target is an IP, just return it.
host = target
if host.startswith("http://") or host.startswith("https://"):
host = re.sub(r"^https?://", "", host)
host = host.split("/")[0]
if re.fullmatch(r"\d{1,3}(\.\d{1,3}){3}", host):
return host
parts = [p for p in host.split(".") if p]
if len(parts) >= 2:
return ".".join(parts[-2:])
return host
def _ctx(target: str, options: ScanOptions, scan_id: str, open_ports: str = "80,443") -> dict[str, str]:
t = shlex.quote(target)
return {
"target": t,
"target_url": shlex.quote(_target_url(target)),
"https_target_url": shlex.quote(_https_target_url(target)),
"root_domain": shlex.quote(_root_domain(target)),
"port_range": shlex.quote(options.port_range),
"open_ports": shlex.quote(open_ports),
"scan_id": shlex.quote(scan_id),
"users": shlex.quote("/usr/share/seclists/Usernames/top-usernames-shortlist.txt"),
"pwds": shlex.quote("/usr/share/wordlists/rockyou.txt"),
}
def render_command(template: str, ctx: dict[str, str]) -> list[str]:
rendered = template.format(**ctx)
return shlex.split(rendered)
PASSIVE: list[PhaseDef] = [
PhaseDef("host_alive", "nmap", "nmap -sn {target}"),
PhaseDef(
"port_scan",
"nmap",
"nmap -sS -p {port_range} --open -T4 -oX - {target}",
),
PhaseDef(
"service_detect",
"nmap",
"nmap -sV -sC -p {open_ports} -oX - {target}",
llm_analyze=True,
),
PhaseDef(
"ssl_check",
"testssl",
"testssl.sh --jsonfile-pretty - {final_https_url}",
requires=("include_ssl",),
success_exit_codes=(0, 1),
timeout_seconds=600,
retries=1,
retry_delay_seconds=2.0,
),
]
STANDARD_EXTRA: list[PhaseDef] = [
PhaseDef("web_fingerprint", "whatweb", "whatweb --log-json=- {final_url}", requires=("include_web",)),
PhaseDef(
"http_headers",
"curl",
"curl -sI -L {final_url}",
requires=("include_web",),
llm_analyze=False,
),
PhaseDef(
"dir_enum",
"gobuster",
"/bin/sh -lc \"set -e; "
"url={final_url}; "
"wl=''; "
"for f in "
"/usr/share/wordlists/dirb/common.txt "
"/usr/share/wordlists/dirb/big.txt "
"/usr/share/seclists/Discovery/Web-Content/common.txt "
"/usr/share/seclists/Discovery/Web-Content/directory-list-2.3-small.txt "
"; do "
"if [ -f \\\"$f\\\" ]; then wl=\\\"$f\\\"; break; fi; "
"done; "
"if [ -z \\\"$wl\\\" ]; then "
"echo 'ERROR: no wordlist found (install dirb/seclists or mount one)'; "
"exit 2; "
"fi; "
"echo 'Using wordlist:' $wl; "
"gobuster dir -u \\\"$url\\\" -w \\\"$wl\\\" -o - -q -r --random-agent --delay 100ms -t 10\"",
requires=("include_web",),
timeout_seconds=600,
retries=1,
retry_delay_seconds=2.0,
),
PhaseDef(
"web_misconfig",
"curl",
"/bin/sh -lc \"set -e; "
"base={final_https_url}; "
"echo 'Base:' $base; "
"echo; "
"for p in /.git/ /.git/config /.env /security.txt /.well-known/security.txt /robots.txt /sitemap.xml /sitemap_index.xml /backup /old /admin ; do "
"echo '== '\"'$p'\"; "
"curl -skI --max-time 10 \"$base$p\" | tr -d '\\r' || true; "
"echo; "
"done; "
"echo '== CSP header on /'; "
"curl -skI --max-time 10 \"$base/\" | tr -d '\\r' | (grep -i '^content-security-policy:' || echo '(no Content-Security-Policy header)')\"",
requires=("include_web",),
llm_analyze=False,
timeout_seconds=120,
retries=1,
retry_delay_seconds=1.0,
),
PhaseDef(
"nikto",
"nikto",
"nikto -h {final_url} -Format json -output -",
requires=("include_web",),
),
PhaseDef(
"mail_dns",
"dig",
"/bin/sh -lc \"set -e; "
"d={root_domain}; "
"echo 'Domain:' $d; "
"echo; "
"echo '== SPF (TXT)'; dig +short TXT $d || true; "
"echo; "
"echo '== DMARC (_dmarc)'; dig +short TXT _dmarc.$d || true; "
"echo; "
"echo '== DKIM (common selectors)'; "
"for s in default selector1 selector2 google mail ; do "
"echo '-- '\"'$s'\"; "
"dig +short TXT $s._domainkey.$d || true; "
"done\"",
llm_analyze=False,
timeout_seconds=60,
retries=1,
retry_delay_seconds=1.0,
),
]
REDTEAM_EXTRA: list[PhaseDef] = [
PhaseDef(
"sqlmap_probe",
"sqlmap",
"sqlmap -u {target_url} --batch --crawl=2 --level=2 --risk=1 --output-dir=/tmp/sqlmap-{scan_id}",
requires=("include_web",),
),
PhaseDef(
"ssh_brute",
"hydra",
"hydra -L {users} -P {pwds} -t 4 -f ssh://{target}",
requires=("include_brute",),
llm_analyze=False,
),
PhaseDef(
"searchsploit",
"searchsploit",
"searchsploit --json {target}",
llm_analyze=True,
),
]
_PROFILE_PHASES: dict[ProfileName, list[PhaseDef]] = {
"passive": PASSIVE,
"standard": PASSIVE + STANDARD_EXTRA,
"redteam": PASSIVE + STANDARD_EXTRA + REDTEAM_EXTRA,
}
def get_phase_defs(profile: ProfileName) -> list[PhaseDef]:
return list(_PROFILE_PHASES.get(profile, PASSIVE))
def get_profiles_payload() -> list[dict[str, object]]:
return [
{
"id": "passive",
"label": "Passive Recon",
"description": "Host discovery, port scan, service detection, optional SSL check.",
"phase_count": len(PASSIVE),
},
{
"id": "standard",
"label": "Standard Pentest",
"description": "Passive plus web fingerprinting, directory enum, and Nikto.",
"phase_count": len(PASSIVE) + len(STANDARD_EXTRA),
},
{
"id": "redteam",
"label": "Full Red Team",
"description": "Standard plus sqlmap, optional SSH brute, searchsploit.",
"phase_count": len(PASSIVE) + len(STANDARD_EXTRA) + len(REDTEAM_EXTRA),
},
]

113
backend/app/safety.py Normal file
View File

@ -0,0 +1,113 @@
import ipaddress
import re
from urllib.parse import urlparse
from fastapi import HTTPException
from app.config import settings
METADATA_IPS = frozenset({"169.254.169.254", "169.254.170.2"})
HOSTNAME_RE = re.compile(
r"^(?=.{1,253}$)(?!-)[a-zA-Z0-9-]{1,63}(?<!-)(\.(?!-)[a-zA-Z0-9-]{1,63}(?<!-))*$"
)
SHELL_METACHAR_RE = re.compile(r"[;&|`$()<>\n\r]")
def _parse_target(raw: str) -> tuple[str, str]:
"""Return (kind, normalized) where kind is ip|cidr|hostname|url."""
raw = raw.strip()
if not raw:
raise HTTPException(status_code=400, detail="Target is required")
if SHELL_METACHAR_RE.search(raw):
raise HTTPException(status_code=400, detail="Target contains invalid characters")
if "://" in raw:
parsed = urlparse(raw)
if parsed.scheme not in ("http", "https"):
raise HTTPException(status_code=400, detail="URL must be http or https")
host = parsed.hostname
if not host:
raise HTTPException(status_code=400, detail="Invalid URL hostname")
return "url", raw
if "/" in raw and not raw.startswith("/"):
try:
net = ipaddress.ip_network(raw, strict=False)
return "cidr", str(net)
except ValueError:
pass
try:
addr = ipaddress.ip_address(raw)
return "ip", str(addr)
except ValueError:
pass
host_part = raw.split(":")[0]
if HOSTNAME_RE.match(host_part) or host_part == "localhost":
return "hostname", raw
raise HTTPException(status_code=400, detail="Invalid target: must be IP, CIDR, hostname, or URL")
def _extract_ips(target: str, kind: str) -> list[ipaddress.IPv4Address | ipaddress.IPv6Address]:
if kind == "url":
host = urlparse(target).hostname or ""
try:
return [ipaddress.ip_address(host)]
except ValueError:
return []
if kind == "cidr":
net = ipaddress.ip_network(target, strict=False)
if net.num_addresses == 1:
return [ipaddress.ip_address(net.network_address)]
return []
if kind in ("ip", "hostname"):
try:
return [ipaddress.ip_address(target.split(":")[0])]
except ValueError:
return []
return []
def _is_loopback_target(target: str, kind: str) -> bool:
if kind == "hostname" and target.lower().startswith("localhost"):
return True
for ip in _extract_ips(target, kind):
if ip.is_loopback:
return True
if kind == "url":
host = (urlparse(target).hostname or "").lower()
if host in ("localhost", "127.0.0.1", "::1"):
return True
return False
def validate_scan_request(target: str, authorized: bool) -> str:
if not authorized:
raise HTTPException(
status_code=400,
detail="Authorization required: set authorized=true after confirming ownership",
)
kind, normalized = _parse_target(target)
if normalized in settings.blocked_targets_list or target in settings.blocked_targets_list:
raise HTTPException(status_code=400, detail="Target is blocklisted")
for blocked in METADATA_IPS:
if blocked in target or normalized == blocked:
raise HTTPException(status_code=400, detail="Cloud metadata endpoints are blocked")
for blocked in settings.blocked_targets_list:
if blocked in target:
raise HTTPException(status_code=400, detail="Target is blocklisted")
if _is_loopback_target(normalized, kind) and not settings.allow_loopback:
raise HTTPException(status_code=400, detail="Loopback targets are blocked (set ALLOW_LOOPBACK=true to override)")
for ip in _extract_ips(normalized, kind):
if str(ip) in METADATA_IPS:
raise HTTPException(status_code=400, detail="Cloud metadata endpoints are blocked")
return normalized

274
backend/app/scanner.py Normal file
View File

@ -0,0 +1,274 @@
import asyncio
import re
import uuid
from collections import defaultdict
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
import httpx
import structlog
from app import db, llm, profiles, tools
from app.config import settings
from app.models import Finding, Scan
log = structlog.get_logger(__name__)
_active_scans: set[str] = set()
_target_active: dict[str, set[str]] = defaultdict(set)
_cancelled: set[str] = set()
_scan_semaphore: asyncio.Semaphore | None = None
def _semaphore() -> asyncio.Semaphore:
global _scan_semaphore
if _scan_semaphore is None:
_scan_semaphore = asyncio.Semaphore(settings.max_concurrent_scans)
return _scan_semaphore
class WsBroadcaster:
def __init__(self) -> None:
self._queues: list[asyncio.Queue[dict[str, Any]]] = []
def subscribe(self) -> asyncio.Queue[dict[str, Any]]:
q: asyncio.Queue[dict[str, Any]] = asyncio.Queue()
self._queues.append(q)
return q
async def send(self, message: dict[str, Any]) -> None:
for q in list(self._queues):
await q.put(message)
_scan_broadcasters: dict[str, WsBroadcaster] = {}
def get_broadcaster(scan_id: str) -> WsBroadcaster:
if scan_id not in _scan_broadcasters:
_scan_broadcasters[scan_id] = WsBroadcaster()
return _scan_broadcasters[scan_id]
def is_scan_active(scan_id: str) -> bool:
return scan_id in _active_scans
async def target_busy(target: str) -> tuple[bool, str]:
"""One queued or running scan per target at a time."""
if await db.has_active_scan_for_target(target):
return True, "A scan is already queued or running for this target"
return False, ""
def _audit_log(target: str, profile: str, source_ip: str) -> None:
path = Path(settings.audit_log_path)
path.parent.mkdir(parents=True, exist_ok=True)
ts = datetime.now(timezone.utc).isoformat()
line = f"{ts}\ttarget={target}\tprofile={profile}\tsource_ip={source_ip}\n"
with path.open("a", encoding="utf-8") as f:
f.write(line)
def _extract_open_ports(nmap_output: str) -> str:
ports: list[str] = []
for m in re.finditer(r'portid="(\d+)"', nmap_output):
ports.append(m.group(1))
if not ports:
for m in re.finditer(r"(\d+)/tcp\s+open", nmap_output):
ports.append(m.group(1))
return ",".join(sorted(set(ports), key=int)) if ports else "80,443"
async def _resolve_final_url(start_url: str) -> str:
timeout = httpx.Timeout(connect=5.0, read=10.0, write=5.0, pool=5.0)
async with httpx.AsyncClient(follow_redirects=True, timeout=timeout, verify=True) as client:
try:
r = await client.head(start_url)
except Exception:
r = await client.get(start_url)
return str(r.url)
def _ensure_https(url: str) -> str:
if url.startswith("https://"):
return url
if url.startswith("http://"):
return "https://" + url.removeprefix("http://")
return "https://" + url
def _classify_nonzero_phase(pdef: profiles.PhaseDef, output: str) -> str | None:
if pdef.name == "dir_enum":
low = output.lower()
if "no wordlist found" in low or ("wordlist" in low and ("no such file" in low or "not found" in low)):
return "inconclusive"
if "429" in low or "too many requests" in low:
return "blocked"
if "403" in low or "forbidden" in low:
return "blocked"
if "timeout" in low:
return "inconclusive"
if pdef.name == "ssl_check":
low = output.lower()
if "timeout" in low:
return "inconclusive"
return None
async def queue_scan(scan: Scan, source_ip: str = "unknown") -> None:
"""Enqueue scan; waits on semaphore when max concurrent scans are active."""
_audit_log(scan.target, scan.profile, source_ip)
asyncio.create_task(_run_with_slot(scan.id))
async def _run_with_slot(scan_id: str) -> None:
async with _semaphore():
await _run_scan(scan_id)
async def cancel_scan(scan_id: str) -> bool:
_cancelled.add(scan_id)
await tools.cancel_scan_processes(scan_id)
await db.update_scan_status(scan_id, "cancelled")
bc = get_broadcaster(scan_id)
await bc.send({"type": "done", "scan_id": scan_id, "status": "cancelled"})
scan = await db.get_scan(scan_id)
if scan and scan.bulk_id:
await db.refresh_bulk_run_status(scan.bulk_id)
return True
async def _run_scan(scan_id: str) -> None:
scan = await db.get_scan(scan_id)
if not scan:
return
_active_scans.add(scan_id)
_target_active[scan.target].add(scan_id)
bc = get_broadcaster(scan_id)
llm_down = False
try:
await db.update_scan_status(scan_id, "running")
if scan.bulk_id:
await db.refresh_bulk_run_status(scan.bulk_id)
phase_defs = profiles.get_phase_defs(scan.profile)
open_ports = "80,443"
ctx_base = profiles._ctx(scan.target, scan.options, scan_id)
ctx_dynamic: dict[str, str] = {}
if scan.options.include_web or scan.options.include_ssl:
try:
start = profiles._target_url(scan.target)
final_url = await _resolve_final_url(start)
final_https_url = _ensure_https(final_url)
except Exception as exc:
final_url = profiles._target_url(scan.target)
final_https_url = profiles._https_target_url(scan.target)
ctx_dynamic["url_resolution_error"] = str(exc)
ctx_dynamic["final_url"] = final_url
ctx_dynamic["final_https_url"] = final_https_url
phase_rows: list[tuple[str, str, str, str]] = []
for pdef in phase_defs:
if not pdef.should_run(scan.options):
continue
cmd = profiles.render_command(
pdef.command_template,
{**ctx_base, **ctx_dynamic, "open_ports": open_ports},
)
phase_rows.append((str(uuid.uuid4()), pdef.name, pdef.tool, " ".join(cmd)))
phases = await db.create_phases(scan_id, phase_rows)
runnable_defs = [pd for pd in phase_defs if pd.should_run(scan.options)]
for p in phases:
await bc.send({"type": "phase", "phase_id": p.id, "phase": p.name, "status": p.status})
for phase, pdef in zip(phases, runnable_defs):
if scan_id in _cancelled:
break
ctx = {**ctx_base, **ctx_dynamic, "open_ports": open_ports}
command = profiles.render_command(pdef.command_template, ctx)
if pdef.name == "ssh_brute" and "22" not in (open_ports or "").split(","):
note = f"skipped: ssh port 22 not in open ports ({open_ports})"
await db.update_phase(phase.id, "skipped", raw_output=note, exit_code=0)
await bc.send({"type": "phase", "phase_id": phase.id, "phase": phase.name, "status": "skipped"})
continue
await db.update_phase(phase.id, "running", started=True)
await bc.send({"type": "phase", "phase_id": phase.id, "phase": phase.name, "status": "running"})
async def on_line(line: str, pid: str = phase.id) -> None:
await db.append_phase_output(pid, line)
await bc.send({"type": "log", "phase_id": pid, "line": line})
last_result = None
attempts = 1 + max(0, pdef.retries)
for attempt in range(attempts):
if attempt > 0 and pdef.retry_delay_seconds > 0:
await asyncio.sleep(pdef.retry_delay_seconds)
last_result = await tools.run_tool(
command,
scan_id=scan_id,
on_line=on_line,
timeout=pdef.timeout_seconds,
)
if last_result.exit_code in pdef.success_exit_codes:
break
assert last_result is not None
result = last_result
status = "complete" if result.exit_code in pdef.success_exit_codes else "failed"
full_out = result.stdout + ("\n" + result.stderr if result.stderr else "")
if status == "failed":
classified = _classify_nonzero_phase(pdef, full_out)
if classified:
status = classified
await db.update_phase(phase.id, status, raw_output=full_out, exit_code=result.exit_code)
await bc.send(
{
"type": "phase",
"phase_id": phase.id,
"phase": phase.name,
"status": status,
"duration": result.duration,
}
)
if phase.name == "port_scan" and result.stdout:
open_ports = _extract_open_ports(result.stdout)
if pdef.llm_analyze and full_out.strip():
phase.raw_output = full_out
findings, unavailable = await llm.analyze_phase(
phase,
full_out,
target=scan.target,
profile=scan.profile,
model=scan.options.llm_model,
)
if unavailable:
llm_down = True
for f in findings:
await db.save_finding(f)
await bc.send({"type": "finding", "finding": f.model_dump(mode="json")})
final = "cancelled" if scan_id in _cancelled else "complete"
await db.update_scan_status(scan_id, final, llm_unavailable=llm_down)
await bc.send({"type": "done", "scan_id": scan_id, "status": final, "llm_unavailable": llm_down})
except Exception as exc:
log.exception("scan_failed", scan_id=scan_id, error=str(exc))
await db.update_scan_status(scan_id, "failed")
await bc.send({"type": "error", "message": str(exc)})
await bc.send({"type": "done", "scan_id": scan_id, "status": "failed"})
finally:
_active_scans.discard(scan_id)
_target_active[scan.target].discard(scan_id)
_cancelled.discard(scan_id)
if scan and scan.bulk_id:
await db.refresh_bulk_run_status(scan.bulk_id)

95
backend/app/tools.py Normal file
View File

@ -0,0 +1,95 @@
import asyncio
import time
from collections.abc import Awaitable, Callable
import structlog
from app.config import settings
from app.models import ToolResult
log = structlog.get_logger(__name__)
# Track running processes for cancellation: scan_id -> list[asyncio.subprocess.Process]
_running: dict[str, list[asyncio.subprocess.Process]] = {}
def register_process(scan_id: str, proc: asyncio.subprocess.Process) -> None:
_running.setdefault(scan_id, []).append(proc)
def unregister_process(scan_id: str, proc: asyncio.subprocess.Process) -> None:
procs = _running.get(scan_id, [])
if proc in procs:
procs.remove(proc)
async def cancel_scan_processes(scan_id: str) -> None:
for proc in _running.get(scan_id, []):
try:
proc.kill()
except ProcessLookupError:
pass
_running[scan_id] = []
async def run_tool(
command: list[str],
scan_id: str | None = None,
cwd: str | None = None,
timeout: int | None = None,
on_line: Callable[[str], Awaitable[None]] | None = None,
) -> ToolResult:
"""Spawn process without shell; stream stdout line-by-line."""
timeout = timeout or settings.tool_timeout_seconds
start = time.monotonic()
stdout_parts: list[str] = []
stderr_parts: list[str] = []
proc = await asyncio.create_subprocess_exec(
*command,
cwd=cwd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
if scan_id:
register_process(scan_id, proc)
async def read_stream(stream: asyncio.StreamReader, is_stderr: bool) -> None:
while True:
line = await stream.readline()
if not line:
break
text = line.decode(errors="replace").rstrip("\n")
if is_stderr:
stderr_parts.append(text)
else:
stdout_parts.append(text)
if on_line:
await on_line(text)
try:
await asyncio.wait_for(
asyncio.gather(
read_stream(proc.stdout, False), # type: ignore[arg-type]
read_stream(proc.stderr, True), # type: ignore[arg-type]
),
timeout=timeout,
)
exit_code = await proc.wait()
except asyncio.TimeoutError:
proc.kill()
await proc.wait()
exit_code = -1
stderr_parts.append(f"timeout after {timeout}s")
log.warning("tool_timeout", command=command, timeout=timeout)
finally:
if scan_id:
unregister_process(scan_id, proc)
duration = time.monotonic() - start
return ToolResult(
stdout="\n".join(stdout_parts),
stderr="\n".join(stderr_parts),
exit_code=exit_code,
duration=duration,
)

34
backend/pyproject.toml Normal file
View File

@ -0,0 +1,34 @@
[project]
name = "talos"
version = "0.1.0"
description = "Self-hosted pentest orchestrator"
requires-python = ">=3.12"
dependencies = [
"fastapi>=0.115.0",
"uvicorn[standard]>=0.32.0",
"pydantic>=2.9.0",
"pydantic-settings>=2.6.0",
"aiosqlite>=0.20.0",
"httpx>=0.27.0",
"structlog>=24.4.0",
]
[project.optional-dependencies]
dev = ["pytest>=8.3.0", "pytest-asyncio>=0.24.0", "httpx>=0.27.0"]
[build-system]
requires = ["setuptools>=75.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["."]
include = ["app*"]
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
[tool.mypy]
python_version = "3.12"
strict = true
packages = ["app"]

View File

@ -0,0 +1,60 @@
from app.export import render_bulk_json, render_bulk_markdown
def _sample_payload() -> dict:
return {
"bulk": {
"id": "bulk-1",
"profile": "standard",
"status": "complete",
"created_at": "2026-05-27T15:00:00+00:00",
"completed_at": "2026-05-27T16:00:00+00:00",
"total": 1,
"complete": 1,
"failed": 0,
"running": 0,
"queued": 0,
},
"scans": [
{
"scan": {
"id": "scan-1",
"target": "auth.example.com",
"profile": "standard",
"status": "complete",
"llm_unavailable": False,
},
"phases": [
{
"name": "port_scan",
"status": "complete",
"tool": "nmap",
"raw_output": "80/tcp open",
}
],
"findings": [
{
"severity": "low",
"title": "Uncommon HTTP Headers",
"description": "Custom headers present.",
"evidence": "x-custom: 1",
"recommendation": "Review headers.",
}
],
}
],
}
def test_render_bulk_markdown_includes_target_and_finding():
text = render_bulk_markdown(_sample_payload(), for_cursor=True)
assert "auth.example.com" in text
assert "Uncommon HTTP Headers" in text
assert "paste into Cursor" in text
def test_render_bulk_json_truncates_long_output():
payload = _sample_payload()
payload["scans"][0]["phases"][0]["raw_output"] = "x" * 5000
out = render_bulk_json(payload)
assert len(out["scans"][0]["phases"][0]["raw_output"]) < 5000

View File

@ -0,0 +1,65 @@
import json
from unittest.mock import AsyncMock, patch
import pytest
from app.llm import analyze_phase, truncate_output, _parse_findings_json
from app.models import Phase
def test_truncate_head_tail():
text = "a" * 20000
out = truncate_output(text, max_chars=1000)
assert len(out) <= 1100
assert "truncated" in out
def test_parse_findings_valid():
raw = json.dumps(
{
"findings": [
{
"severity": "high",
"title": "Open SSH",
"description": "SSH exposed",
"evidence": "22/tcp open",
"recommendation": "Firewall SSH",
"cve_refs": [],
"confidence": "high",
}
]
}
)
phase = Phase(
id="p1",
scan_id="s1",
name="port_scan",
tool="nmap",
command="nmap",
status="complete",
)
findings = _parse_findings_json(raw, "s1", phase)
assert len(findings) == 1
assert findings[0].severity == "high"
@pytest.mark.asyncio
async def test_analyze_phase_retries_bad_json():
phase = Phase(
id="p1",
scan_id="s1",
name="port_scan",
tool="nmap",
command="nmap -sn target",
status="complete",
)
bad = "not json"
good = json.dumps({"findings": []})
with patch("app.llm.ollama_health", AsyncMock(return_value=True)):
with patch("app.llm._chat", AsyncMock(side_effect=[bad, good])):
findings, unavailable = await analyze_phase(
phase, "22/tcp open ssh", target="t", profile="passive", model="test"
)
assert findings == []
assert unavailable is False

View File

@ -0,0 +1,33 @@
from app.models import ScanOptions
from app.profiles import get_phase_defs, render_command, _ctx
def test_passive_phase_order():
phases = get_phase_defs("passive")
names = [p.name for p in phases]
assert names[0] == "host_alive"
assert "port_scan" in names
def test_web_phases_gated():
opts = ScanOptions(include_web=False)
standard = get_phase_defs("standard")
for p in standard:
if p.requires and "include_web" in p.requires:
assert not p.should_run(opts)
def test_command_uses_argv_list():
opts = ScanOptions()
ctx = _ctx("scanme.nmap.org", opts, "abc")
cmd = render_command("nmap -sn {target}", ctx)
assert cmd[0] == "nmap"
assert cmd[-1] == "scanme.nmap.org"
assert " ".join(cmd) == "nmap -sn scanme.nmap.org"
def test_redteam_brute_gated():
opts = ScanOptions(include_brute=False)
phases = get_phase_defs("redteam")
brute = next(p for p in phases if p.name == "ssh_brute")
assert not brute.should_run(opts)

View File

@ -0,0 +1,41 @@
import pytest
from app import db, scanner
from app.config import settings
from app.models import ScanOptions
@pytest.fixture
async def initialized_db(tmp_path, monkeypatch):
db_path = tmp_path / "talos.db"
monkeypatch.setattr(settings, "db_path", str(db_path))
monkeypatch.setattr(settings, "audit_log_path", str(tmp_path / "audit.log"))
monkeypatch.setattr(settings, "max_concurrent_scans", 2)
scanner._scan_semaphore = None
await db.init_db()
yield
await db.close_db()
@pytest.mark.asyncio
async def test_bulk_creates_queued_scans(initialized_db):
bulk = await db.create_bulk_run("passive", total=0)
for target in ("a.example", "b.example", "c.example"):
await db.create_scan(target, "passive", ScanOptions(), bulk_id=bulk.id)
scans = await db.list_scans_for_bulk(bulk.id)
assert len(scans) == 3
assert all(s.status == "queued" for s in scans)
bulk_run = await db.refresh_bulk_run_status(bulk.id)
assert bulk_run is not None
assert bulk_run.total == 3
assert bulk_run.queued == 3
@pytest.mark.asyncio
async def test_target_busy_blocks_duplicate(initialized_db):
scan = await db.create_scan("dup.example", "passive", ScanOptions())
await scanner.queue_scan(scan)
busy, reason = await scanner.target_busy("dup.example")
assert busy is True
assert "already" in reason.lower()

View File

@ -0,0 +1,42 @@
import pytest
from fastapi import HTTPException
from app.config import settings
from app.safety import validate_scan_request
def test_blocks_metadata_ip():
with pytest.raises(HTTPException) as exc:
validate_scan_request("169.254.169.254", authorized=True)
assert exc.value.status_code == 400
def test_blocks_without_authorization():
with pytest.raises(HTTPException):
validate_scan_request("scanme.nmap.org", authorized=False)
def test_blocks_shell_metacharacters():
with pytest.raises(HTTPException):
validate_scan_request("scanme.nmap.org; rm -rf /", authorized=True)
def test_blocks_loopback_by_default(monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr(settings, "allow_loopback", False)
with pytest.raises(HTTPException):
validate_scan_request("127.0.0.1", authorized=True)
def test_allows_loopback_when_configured(monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr(settings, "allow_loopback", True)
assert validate_scan_request("127.0.0.1", authorized=True) == "127.0.0.1"
def test_accepts_hostname():
assert validate_scan_request("scanme.nmap.org", authorized=True) == "scanme.nmap.org"
def test_blocklist(monkeypatch: pytest.MonkeyPatch):
monkeypatch.setattr(settings, "blocked_targets", "evil.example")
with pytest.raises(HTTPException):
validate_scan_request("evil.example", authorized=True)

48
docker-compose.yml Normal file
View File

@ -0,0 +1,48 @@
services:
tools:
build: ./tools
image: talos-tools:latest
command: ["true"]
profiles: ["build-only"]
talos:
build:
context: ./backend
image: talos:latest
environment:
- OLLAMA_URL=${OLLAMA_URL:-http://ollama:11434}
- OLLAMA_MODEL=${OLLAMA_MODEL:-qwen2.5:14b}
- ALLOW_LOOPBACK=${ALLOW_LOOPBACK:-false}
- BLOCKED_TARGETS=${BLOCKED_TARGETS:-}
- DB_PATH=/data/talos.db
- MAX_CONCURRENT_SCANS=${MAX_CONCURRENT_SCANS:-5}
- AUDIT_LOG_PATH=/data/audit.log
volumes:
- ./data:/data
- ./frontend:/app/frontend:ro
cap_add:
- NET_RAW
- NET_ADMIN
ports:
- "8000:8000"
restart: unless-stopped
extra_hosts:
# Reach homelab Ollama from container (optional override via OLLAMA_URL)
- "devgpu.host:10.0.10.122"
ollama:
image: ollama/ollama:latest
profiles: ["bundled-ollama"]
volumes:
- ./ollama:/root/.ollama
ports:
- "11434:11434"
restart: unless-stopped
# Uncomment for GPU on host with nvidia-container-toolkit:
# deploy:
# resources:
# reservations:
# devices:
# - driver: nvidia
# count: all
# capabilities: [gpu]

323
frontend/app.js Normal file
View File

@ -0,0 +1,323 @@
const $ = (id) => document.getElementById(id);
let currentScanId = null;
let currentBulkId = null;
let bulkPollTimer = null;
let ws = null;
let consoleLines = [];
let scanMode = "single";
const MAX_LINES = 5000;
function api(path, opts = {}) {
return fetch(path, opts).then((r) => {
if (!r.ok) return r.json().then((e) => Promise.reject(e));
const ct = r.headers.get("content-type") || "";
if (ct.includes("json")) return r.json();
return r.text();
});
}
function parseBulkTargets(raw) {
return raw
.split(/\r?\n/)
.map((line) => line.trim())
.filter((line) => line && !line.startsWith("#"));
}
function updateBulkCount() {
const count = parseBulkTargets($("targets-bulk").value || "").length;
$("bulk-count").textContent = count === 1 ? "1 target" : `${count} targets`;
updateLaunchLabel();
}
function updateLaunchLabel() {
if (!$("authorized").checked) {
$("launch").textContent = "Launch scan";
return;
}
if (scanMode === "bulk") {
const count = parseBulkTargets($("targets-bulk").value || "").length;
$("launch").textContent = count > 0 ? `Launch ${count} scan${count === 1 ? "" : "s"}` : "Launch scan";
return;
}
$("launch").textContent = "Launch scan";
}
function setScanMode(mode) {
scanMode = mode;
$("mode-single").classList.toggle("active", mode === "single");
$("mode-bulk").classList.toggle("active", mode === "bulk");
$("single-target").classList.toggle("hidden", mode !== "single");
$("bulk-target").classList.toggle("hidden", mode !== "bulk");
updateLaunchLabel();
}
function scanOptions() {
return {
include_web: $("opt-web").checked,
include_ssl: $("opt-ssl").checked,
include_brute: $("opt-brute").checked,
};
}
function setBulkProgress(visible, text) {
$("bulk-progress").classList.toggle("hidden", !visible);
if (text) $("bulk-progress-text").textContent = text;
}
function stopBulkPoll() {
if (bulkPollTimer) {
clearInterval(bulkPollTimer);
bulkPollTimer = null;
}
}
function pollBulkProgress(bulkId) {
stopBulkPoll();
const tick = () => {
api(`/api/bulks/${bulkId}`)
.then((d) => {
const b = d.bulk || {};
const line = `Bulk ${bulkId.slice(0, 8)}… · ${b.complete || 0}/${b.total || 0} complete`
+ (b.running ? ` · ${b.running} running` : "")
+ (b.queued ? ` · ${b.queued} queued` : "")
+ (b.failed ? ` · ${b.failed} failed` : "");
setBulkProgress(true, line);
if (b.status === "complete") {
stopBulkPoll();
logLine(`bulk finished: ${b.complete}/${b.total} complete`);
$("export-cursor").disabled = false;
}
})
.catch(() => {});
};
tick();
bulkPollTimer = setInterval(tick, 5000);
}
function resetScanView() {
consoleLines = [];
$("phases").innerHTML = "";
$("findings").innerHTML = "";
stopBulkPoll();
setBulkProgress(false);
currentBulkId = null;
$("export-cursor").classList.add("hidden");
$("export-cursor").disabled = true;
}
function setHealth() {
api("/api/health").then((h) => {
$("ollama-badge").textContent = `Ollama: ${h.ollama}`;
$("ollama-badge").className = `badge ${h.ollama === "ok" ? "ok" : "down"}`;
const toolsOk = Object.values(h.tools || {}).every((v) => v === "ok");
$("tools-badge").textContent = `Tools: ${toolsOk ? "ok" : "degraded"}`;
$("tools-badge").className = `badge ${toolsOk ? "ok" : "down"}`;
}).catch(() => {
$("ollama-badge").textContent = "Ollama: down";
$("tools-badge").textContent = "Tools: unknown";
});
}
function logLine(line) {
consoleLines.push(line);
if (consoleLines.length > MAX_LINES) consoleLines = consoleLines.slice(-MAX_LINES);
const el = $("console");
el.textContent = consoleLines.join("\n");
if (!$("pause-scroll").checked) el.scrollTop = el.scrollHeight;
}
function renderPhases(phases) {
const ul = $("phases");
ul.innerHTML = "";
(phases || []).forEach((p) => {
const li = document.createElement("li");
const name = p.phase || p.name;
const status = p.status || "pending";
li.innerHTML = `<span>${name}</span><span class="phase-${status}">${status}</span>`;
ul.appendChild(li);
});
}
function renderFinding(f) {
const div = document.createElement("div");
div.className = "finding-card";
div.innerHTML = `
<span class="severity-chip sev-${f.severity}">${f.severity}</span>
<strong>${f.title}</strong>
<p>${f.description}</p>
<p><em>Confidence:</em> ${f.confidence}</p>
<p class="evidence"><code>${(f.evidence || "").slice(0, 500)}</code></p>
<p><strong>Fix:</strong> ${f.recommendation}</p>
`;
div.addEventListener("click", () => div.classList.toggle("open"));
$("findings").prepend(div);
}
function loadRecent() {
api("/api/scans").then((scans) => {
const ul = $("recent-list");
ul.innerHTML = "";
scans.forEach((s) => {
const li = document.createElement("li");
li.textContent = `${s.target} · ${s.profile} · ${s.status}`;
li.dataset.id = s.id;
li.addEventListener("click", () => openScan(s.id));
ul.appendChild(li);
});
});
}
function openScan(id) {
currentScanId = id;
$("export-md").disabled = false;
$("export-json").disabled = false;
$("cancel").disabled = false;
api(`/api/scans/${id}`).then((d) => {
consoleLines = [];
logLine(`loaded scan ${id}${d.scan.target}`);
renderPhases(d.phases.map((p) => ({ name: p.name, status: p.status })));
$("findings").innerHTML = "";
d.findings.forEach(renderFinding);
connectWs(id);
});
}
function connectWs(id) {
if (ws) ws.close();
const proto = location.protocol === "https:" ? "wss" : "ws";
ws = new WebSocket(`${proto}://${location.host}/ws/scans/${id}`);
ws.onmessage = (ev) => {
const msg = JSON.parse(ev.data);
if (msg.type === "log") logLine(msg.line);
if (msg.type === "phase") {
const ul = $("phases");
let li = [...ul.children].find((c) => c.textContent.startsWith(msg.phase));
if (!li) {
li = document.createElement("li");
ul.appendChild(li);
}
li.innerHTML = `<span>${msg.phase}</span><span class="phase-${msg.status}">${msg.status}</span>`;
}
if (msg.type === "finding") renderFinding(msg.finding);
if (msg.type === "done") {
logLine(`scan ${msg.status}`);
$("cancel").disabled = true;
loadRecent();
}
if (msg.type === "error") logLine(`ERROR: ${msg.message}`);
};
}
function launchSingle() {
const body = {
target: $("target").value.trim(),
profile: $("profile").value,
authorized: $("authorized").checked,
options: scanOptions(),
};
resetScanView();
api("/api/scans", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
})
.then((r) => {
currentScanId = r.scan_id;
logLine(`scan queued: ${r.scan_id}`);
$("export-md").disabled = false;
$("export-json").disabled = false;
$("cancel").disabled = false;
connectWs(r.scan_id);
loadRecent();
})
.catch((e) => logLine(`launch failed: ${e.detail || JSON.stringify(e)}`));
}
function launchBulk() {
const targets = parseBulkTargets($("targets-bulk").value || "");
if (!targets.length) {
logLine("bulk launch failed: add at least one target");
return;
}
const body = {
targets,
profile: $("profile").value,
authorized: $("authorized").checked,
options: scanOptions(),
};
resetScanView();
logLine(`bulk launch: ${targets.length} target${targets.length === 1 ? "" : "s"}`);
api("/api/scans/bulk", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
})
.then((r) => {
const queued = r.queued || [];
const rejected = r.rejected || [];
currentBulkId = r.bulk_id || null;
if (currentBulkId) {
$("export-cursor").classList.remove("hidden");
$("export-cursor").disabled = false;
pollBulkProgress(currentBulkId);
logLine(`bulk id: ${currentBulkId} (${queued.length} targets queued)`);
}
queued.forEach((q) => logLine(`scan queued: ${q.scan_id} (${q.target})`));
rejected.forEach((rej) => logLine(`skipped: ${rej.target}${rej.reason}`));
logLine(`bulk submitted: ${queued.length} queued, ${rejected.length} skipped`);
if (queued.length) {
currentScanId = queued[0].scan_id;
$("export-md").disabled = false;
$("export-json").disabled = false;
$("cancel").disabled = false;
connectWs(queued[0].scan_id);
logLine(`watching first scan: ${queued[0].target}`);
}
loadRecent();
})
.catch((e) => logLine(`bulk launch failed: ${e.detail || JSON.stringify(e)}`));
}
$("mode-single").addEventListener("click", () => setScanMode("single"));
$("mode-bulk").addEventListener("click", () => setScanMode("bulk"));
$("targets-bulk").addEventListener("input", updateBulkCount);
$("authorized").addEventListener("change", () => {
$("launch").disabled = !$("authorized").checked;
updateLaunchLabel();
});
$("launch").addEventListener("click", () => {
if (scanMode === "bulk") launchBulk();
else launchSingle();
});
$("cancel").addEventListener("click", () => {
if (!currentScanId) return;
api(`/api/scans/${currentScanId}/cancel`, { method: "POST" }).then(() => logLine("cancel requested"));
});
$("export-md").addEventListener("click", () => {
if (!currentScanId) return;
window.open(`/api/scans/${currentScanId}/export?format=md`, "_blank");
});
$("export-json").addEventListener("click", () => {
if (!currentScanId) return;
window.open(`/api/scans/${currentScanId}/export?format=json`, "_blank");
});
$("export-cursor").addEventListener("click", () => {
if (!currentBulkId) return;
window.open(`/api/bulks/${currentBulkId}/export?format=cursor`, "_blank");
});
$("clear").addEventListener("click", () => {
consoleLines = [];
$("console").textContent = "talos@scan:~$";
$("phases").innerHTML = "";
$("findings").innerHTML = "";
});
setScanMode("single");
updateBulkCount();
setHealth();
setInterval(setHealth, 30000);
loadRecent();

5
frontend/favicon.svg Normal file
View File

@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32" role="img" aria-label="TALOS">
<rect width="32" height="32" rx="7" fill="#0d0f12"/>
<path d="M16 4L6 8v8c0 6.2 4.3 12 10 14 5.7-2 10-7.8 10-14V8L16 4z" fill="#e8500a"/>
<path d="M16 9v14M11 14h10" stroke="#0d0f12" stroke-width="2.2" stroke-linecap="round"/>
</svg>

After

Width:  |  Height:  |  Size: 331 B

110
frontend/index.html Normal file
View File

@ -0,0 +1,110 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>TALOS</title>
<link rel="icon" href="/static/favicon.svg" type="image/svg+xml" />
<link rel="apple-touch-icon" href="/static/favicon.svg" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;600&family=JetBrains+Mono:wght@400;700&display=swap" rel="stylesheet" />
<link rel="stylesheet" href="/static/styles.css" />
</head>
<body class="talos-body">
<header class="header-bar">
<div class="brand">
<img src="/static/favicon.svg" alt="" class="brand-icon" width="28" height="28" />
<div class="logo">TALOS</div>
</div>
<div class="health-badges">
<span id="ollama-badge" class="badge">Ollama: …</span>
<span id="tools-badge" class="badge">Tools: …</span>
</div>
</header>
<div class="layout">
<aside class="sidebar">
<section class="panel">
<h2>New scan</h2>
<div class="scan-mode" role="tablist" aria-label="Scan mode">
<button type="button" id="mode-single" class="mode-btn active" data-mode="single">Single</button>
<button type="button" id="mode-bulk" class="mode-btn" data-mode="bulk">Bulk</button>
</div>
<div id="single-target" class="target-panel">
<label for="target">Target</label>
<input id="target" type="text" placeholder="IP, hostname, or URL" />
</div>
<div id="bulk-target" class="target-panel hidden">
<div class="bulk-label-row">
<label for="targets-bulk">Targets</label>
<span id="bulk-count" class="bulk-count">0 targets</span>
</div>
<textarea
id="targets-bulk"
rows="7"
spellcheck="false"
placeholder="example.com&#10;https://app.example.com&#10;1.2.3.4&#10;# lines starting with # are ignored"
></textarea>
<p class="bulk-hint">One target per line. Blank lines and <code>#</code> comments are ignored.</p>
</div>
<label for="profile">Profile</label>
<select id="profile">
<option value="passive">Passive Recon</option>
<option value="standard" selected>Standard Pentest</option>
<option value="redteam">Full Red Team</option>
</select>
<label class="checkbox-row">
<input type="checkbox" id="opt-web" checked /> Web tools
</label>
<label class="checkbox-row">
<input type="checkbox" id="opt-ssl" checked /> SSL (testssl)
</label>
<label class="checkbox-row">
<input type="checkbox" id="opt-brute" /> SSH brute (red team)
</label>
<label class="auth-row">
<input type="checkbox" id="authorized" />
I confirm I own this target or have written authorization to test it. Unauthorized testing is a crime.
</label>
<button id="launch" class="btn-launch" disabled>Launch scan</button>
</section>
<section class="panel recent">
<h2>Recent scans</h2>
<ul id="recent-list"></ul>
</section>
</aside>
<main class="main">
<div class="console-wrap panel">
<div class="console-header">
<span>Live Console</span>
<label class="checkbox-row"><input type="checkbox" id="pause-scroll" /> Pause autoscroll</label>
</div>
<pre id="console" class="console"><span class="prompt">talos@scan:~$</span> waiting…</pre>
</div>
<div class="panel">
<h2>Phases</h2>
<ul id="phases" class="phases-list"></ul>
</div>
<div class="panel">
<h2>Findings</h2>
<div id="severity-tabs" class="severity-tabs"></div>
<div id="findings"></div>
</div>
<div id="bulk-progress" class="bulk-progress hidden">
<span id="bulk-progress-text">Bulk: —</span>
</div>
<div class="actions">
<button id="export-md" class="btn-secondary" disabled>Export MD</button>
<button id="export-json" class="btn-secondary" disabled>Export JSON</button>
<button id="export-cursor" class="btn-secondary hidden" disabled>Export for Cursor</button>
<button id="cancel" class="btn-danger" disabled>Cancel</button>
<button id="clear" class="btn-secondary">Clear</button>
</div>
</main>
</div>
<script src="/static/app.js"></script>
</body>
</html>

240
frontend/styles.css Normal file
View File

@ -0,0 +1,240 @@
:root {
--bg: #0d0f12;
--surface: #131619;
--accent: #e8500a;
--text: #e8eaed;
--muted: #9ca3af;
--critical: #e83a3a;
--high: #ff6420;
--medium: #e8b20a;
--low: #3db87a;
--info: #6b7280;
--code-bg: #0a0c0f;
--code-text: #a6e22e;
}
* { box-sizing: border-box; }
body.talos-body {
margin: 0;
background: var(--bg);
color: var(--text);
font-family: Inter, system-ui, sans-serif;
min-height: 100vh;
}
h1, h2, .logo, .console, code { font-family: "JetBrains Mono", monospace; }
.header-bar {
display: flex;
justify-content: space-between;
align-items: center;
padding: 0.75rem 1.5rem;
background: var(--surface);
border-bottom: 1px solid #1f2429;
}
.brand {
display: flex;
align-items: center;
gap: 0.6rem;
}
.brand-icon { display: block; }
.logo { font-weight: 700; color: var(--accent); letter-spacing: 0.08em; }
.badge { margin-left: 1rem; font-size: 0.85rem; color: var(--muted); }
.badge.ok { color: var(--low); }
.badge.down { color: var(--critical); }
.layout { display: flex; min-height: calc(100vh - 52px); }
.sidebar {
width: 340px;
flex-shrink: 0;
padding: 1rem;
border-right: 1px solid #1f2429;
overflow-y: auto;
}
.main { flex: 1; padding: 1rem; overflow-y: auto; }
.panel {
background: var(--surface);
border-radius: 8px;
padding: 1rem;
margin-bottom: 1rem;
border: 1px solid #1f2429;
}
.panel h2 { font-size: 0.9rem; margin: 0 0 0.75rem; color: var(--accent); }
label { display: block; font-size: 0.8rem; color: var(--muted); margin: 0.5rem 0 0.25rem; }
input[type="text"],
select,
textarea {
width: 100%;
background: var(--code-bg);
border: 1px solid #2a3038;
color: var(--text);
padding: 0.5rem;
border-radius: 4px;
font-family: "JetBrains Mono", monospace;
font-size: 0.8rem;
}
input[type="text"]:focus,
select:focus,
textarea:focus {
outline: none;
border-color: var(--accent);
box-shadow: 0 0 0 1px rgba(232, 80, 10, 0.35);
}
textarea {
min-height: 8.5rem;
resize: vertical;
line-height: 1.45;
}
textarea::placeholder,
input::placeholder {
color: #5f6672;
}
.checkbox-row, .auth-row { display: flex; gap: 0.5rem; align-items: flex-start; font-size: 0.75rem; color: var(--muted); margin-top: 0.5rem; }
.auth-row { color: #f0ad4e; }
.btn-launch {
width: 100%;
margin-top: 1rem;
padding: 0.75rem;
background: var(--accent);
border: none;
color: #fff;
font-weight: 700;
border-radius: 6px;
cursor: pointer;
}
.btn-launch:disabled { opacity: 0.4; cursor: not-allowed; }
.btn-secondary, .btn-danger {
padding: 0.5rem 1rem;
border-radius: 4px;
border: 1px solid #2a3038;
background: var(--surface);
color: var(--text);
cursor: pointer;
margin-right: 0.5rem;
}
.btn-danger { border-color: var(--critical); color: var(--critical); }
.actions { margin-top: 0.5rem; }
.console-wrap { min-height: 220px; }
.console-header { display: flex; justify-content: space-between; margin-bottom: 0.5rem; font-size: 0.85rem; }
.console {
background: var(--code-bg);
border-left: 3px solid var(--accent);
color: var(--code-text);
padding: 0.75rem;
max-height: 320px;
overflow-y: auto;
font-size: 0.75rem;
white-space: pre-wrap;
margin: 0;
}
.prompt { color: var(--accent); }
.phases-list { list-style: none; padding: 0; margin: 0; }
.phases-list li {
display: flex;
justify-content: space-between;
padding: 0.35rem 0;
border-bottom: 1px solid #1f2429;
font-size: 0.85rem;
}
.phase-running { color: var(--medium); }
.phase-complete { color: var(--low); }
.phase-failed { color: var(--critical); }
.phase-blocked { color: var(--info); }
.phase-inconclusive { color: var(--info); }
.scan-mode {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 0.35rem;
margin: 0.25rem 0 0.75rem;
padding: 0.25rem;
background: var(--code-bg);
border: 1px solid #2a3038;
border-radius: 6px;
}
.mode-btn {
border: none;
background: transparent;
color: var(--muted);
padding: 0.45rem 0.5rem;
border-radius: 4px;
font-size: 0.8rem;
font-weight: 600;
cursor: pointer;
transition: background 0.15s, color 0.15s;
}
.mode-btn:hover { color: var(--text); }
.mode-btn.active {
background: var(--accent);
color: #fff;
}
.target-panel.hidden { display: none; }
.bulk-label-row {
display: flex;
justify-content: space-between;
align-items: baseline;
gap: 0.5rem;
}
.bulk-label-row label { margin-top: 0; }
.bulk-count {
font-size: 0.72rem;
color: var(--accent);
font-family: "JetBrains Mono", monospace;
white-space: nowrap;
}
.bulk-hint {
margin: 0.35rem 0 0;
font-size: 0.72rem;
color: var(--muted);
line-height: 1.4;
}
.bulk-hint code {
font-family: "JetBrains Mono", monospace;
color: var(--code-text);
font-size: 0.7rem;
}
.bulk-progress {
margin: 0 0 0.75rem;
padding: 0.55rem 0.75rem;
border: 1px solid #2a3138;
border-radius: 6px;
font-family: "JetBrains Mono", monospace;
font-size: 0.78rem;
color: var(--accent);
background: #12161a;
}
.bulk-progress.hidden { display: none; }
#recent-list { list-style: none; padding: 0; margin: 0; }
#recent-list li {
padding: 0.4rem 0;
cursor: pointer;
font-size: 0.8rem;
border-bottom: 1px solid #1f2429;
}
#recent-list li:hover { color: var(--accent); }
.finding-card {
border: 1px solid #2a3038;
border-radius: 6px;
padding: 0.75rem;
margin-bottom: 0.5rem;
}
.severity-chip {
display: inline-block;
font-size: 0.7rem;
font-weight: 700;
padding: 0.15rem 0.5rem;
border-radius: 4px;
text-transform: uppercase;
margin-bottom: 0.35rem;
}
.sev-critical { background: var(--critical); color: #fff; }
.sev-high { background: var(--high); color: #fff; }
.sev-medium { background: var(--medium); color: #111; }
.sev-low { background: var(--low); color: #111; }
.sev-info { background: var(--info); color: #fff; }
.evidence { font-size: 0.75rem; color: var(--muted); margin-top: 0.5rem; display: none; }
.finding-card.open .evidence { display: block; }

23
tools/Dockerfile Normal file
View File

@ -0,0 +1,23 @@
FROM kalilinux/kali-rolling
RUN set -eux; \
printf '%s\n' \
'deb http://kali.download/kali kali-rolling main contrib non-free non-free-firmware' \
> /etc/apt/sources.list
RUN apt-get update && apt-get install -y --no-install-recommends \
nmap nikto gobuster whatweb sqlmap hydra exploitdb \
curl ca-certificates dnsutils wordlists \
dirb seclists bsdextrautils \
python3 python3-pip git \
&& rm -rf /var/lib/apt/lists/*
ARG TESTSSL_REF=3.2
RUN git clone https://github.com/drwetter/testssl.sh.git /opt/testssl \
&& cd /opt/testssl \
&& git checkout "${TESTSSL_REF}" \
&& ln -s /opt/testssl/testssl.sh /usr/local/bin/testssl.sh
RUN gunzip -k /usr/share/wordlists/rockyou.txt.gz 2>/dev/null || true
WORKDIR /work