Document Levkin Mattermost import and channel renames in prepare-export.

Record the completed migration (channel map, mmetl/mmctl steps, caveats) and teach prepare-export to apply mattermost_channel renames plus user_emails for matching existing MM accounts.

.gitignore

@@ -1,5 +1,8 @@
 manifest.json
 prepared-export/
+prepared-export.zip
+*.bulk-import.zip
+.tools/
 node_modules/
 test-results/
 playwright-report/

README.md

@@ -3,7 +3,7 @@
 Sieve your [Slack workspace export](https://slack.com/help/articles/201658943-Export-your-workspace-data) before [Mattermost](https://mattermost.com) import — triage channels, cherry-pick messages, preview attachments.
 
 ```text
-Slack export → hydrate (attachments) → review UI → manifest.json → (prepare-export) → mmetl → mmctl
+Slack export → hydrate → review UI → manifest.json → prepare-export → mmetl → mmctl
 ```
 
 ## Quick start
@@ -59,9 +59,12 @@ Edit `presets/levkin-skips.json` (or copy it) for workspace-specific channels to
 
 ## Mattermost import (after review)
 
-1. `make prepare-export` — writes `prepared-export/` from `manifest.json` (pytest-covered; not yet run through mmetl/mmctl).
+1. Set optional `mattermost_channel` renames and `user_emails` in `manifest.json`.
-2. [mmetl](https://github.com/mattermost/mmetl) `transform slack` on `prepared-export/`
+2. `make prepare-export` — writes `prepared-export/` (pytest-covered).
-3. [mmctl](https://docs.mattermost.com/administration-guide/manage/mmctl-command-line-tool.html) `import`
+3. Zip that folder; [mmetl](https://github.com/mattermost/mmetl) `transform slack --team <team> …`
+4. [mmctl](https://docs.mattermost.com/administration-guide/manage/mmctl-command-line-tool.html) `import process --bypass-upload` on the Mattermost host
+
+**Levkin:** import completed 2026-07-10 — see [docs/HANDOFF.md](docs/HANDOFF.md) for channel map and caveats.
 
 Docs: [Migrate from Slack](https://docs.mattermost.com/administration-guide/onboard/migrate-from-slack.html)
 
@@ -72,7 +75,7 @@ Docs: [Migrate from Slack](https://docs.mattermost.com/administration-guide/onbo
 
 ## Handoff
 
-Ongoing Levkin migration status, review order, and TODOs: **[docs/HANDOFF.md](docs/HANDOFF.md)**
+Levkin migration status and channel map: **[docs/HANDOFF.md](docs/HANDOFF.md)**
 
 ## Remote
 

docs/HANDOFF.md

@@ -3,15 +3,60 @@
 **Repo:** https://git.levkin.ca/ilia/slack-sieve (public)  
 **Clone:** `git clone git@git.levkin.ca:ilia/slack-sieve.git`  
 **Vikunja:** [todo.levkin.ca → Homelab → slack-sieve](https://todo.levkin.ca) (`SLK`)  
-**Last updated:** 2026-06-03
+**Last updated:** 2026-07-10
+
+---
+
+## Status: Levkin Slack → Mattermost **done** (2026-07-10)
+
+Filtered archive is live on **https://slack.levkin.ca** (team `levkin`, VM `10.0.10.237`).
+
+- **18 channels**, ~**299 posts** + attachments (from ~765 cherry-picked Slack messages)
+- Users: Slack `jedi_jew` → MM `ilia`, `ira_feeds` → `ira` (both added to imported channels)
+- Boy/girl name lists verified in **Zoey** (`kids-zoey`)
+
+### Channel map (Slack → Mattermost)
+
+| Slack | MM name | Display name |
+|-------|---------|--------------|
+| `122harmony` | `prop-122harmony` | 122 Harmony |
+| `153niagara` | `prop-153niagara` | 153 Niagara |
+| `284richmond` | `prop-284richmond` | 284 Richmond |
+| `45ritson` | `prop-45ritson` | 45 Ritson |
+| `6keefer` | `prop-6keefer` | 6 Keefer |
+| `york` | `prop-york` | York |
+| `airbnb` | `prop-airbnb` | Airbnb |
+| `realestate` | `prop-realestate` | Real estate |
+| `zoey` | `kids-zoey` | Zoey |
+| `zane` | `kids-zane` | Zane |
+| `izik` | `kids-izik` | Izik |
+| `moishe` | `kids-moishe` | Moishe |
+| `parenting` | `kids-parenting` | Parenting |
+| `after-school-activities` | `kids-after-school` | After school |
+| `banking` | `family-banking` | Banking |
+| `purchase` | `family-purchase` | Purchase |
+| `medical` | `family-medical` | Medical |
+| `levkin` | `family-levkin` | Levkin |
+
+Skipped for import: noise channels (`random`, `jam`, `meetings`, …), `29wilson`, `520steeles`, empty picks.
+
+### Import caveats
+
+- Thread replies whose **parent was not cherry-picked** were dropped by mmetl (`couldn't find rootPost`).
+- First `mmctl` job failed updating `ilia` roles (“Cannot demote last System Admin”); fixed by omitting existing users from the JSONL and re-running. Channels from the first attempt were reused.
+- Local artifacts (gitignored): `prepared-export/`, `prepared-export.zip`, `.tools/mmetl`, `/tmp/mm-import/` on the Mac; `/tmp/mattermost-bulk-import.zip` on the VM.
+
+### Share with Irina
+
+> Slack archive is on Mattermost: https://slack.levkin.ca  
+> Team **levkin** — channels like **Zoey**, **Zane**, **45 Ritson**, **Banking**.  
+> Login **ira**. Filtered archive (files/facts, not all chat). Name lists in **Zoey**.
 
 ---
 
 ## What this is
 
-Local triage UI for migrating **Levkin** Slack (`levkine.slack.com`) → homelab Mattermost (`https://slack.levkin.ca`, VM `10.0.10.237`).
+Local triage UI for migrating Slack workspace exports → Mattermost.
-
-Pipeline:
 
 ```text
 Official Slack export (JSON)
@@ -20,7 +65,7 @@ Official Slack export (JSON)
     → make review           # manifest.json (gitignored)
     → make prepare-export   # prepared-export/ (filtered Slack shape)
     → mmetl transform slack
-    → mmctl import
+    → mmctl import --bypass-upload (on MM host)
 ```
 
 ---
@@ -30,10 +75,10 @@ Official Slack export (JSON)
 | Path | Role |
 |------|------|
 | `~/Downloads/Levkin Slack export Sep 29 2016 - May 27 2026` | Raw official export (~27MB JSON only) |
-| `~/Downloads/Levkin Slack export enriched` | **Use for review** — JSON + `__uploads/` (~1109 files) |
+| `~/Downloads/Levkin Slack export enriched` | Review + prepare source — JSON + `__uploads/` |
 | `~/Documents/code/slack-sieve/manifest.json` | Triage decisions (local only, not in git) |
 
-Hydrate copied **1109 / 1122** attachments; **13** old files returned HTTP 401 (expired export token).
+Hydrate: **1109 / 1122** attachments; **13** old files HTTP 401 (expired export token).
 
 ---
 
@@ -45,99 +90,35 @@ cd ~/Documents/code/slack-sieve
 export SLACK_EXPORT_PATH="$HOME/Downloads/Levkin Slack export enriched"
 make review              # http://127.0.0.1:8765
 
-make hydrate             # re-download attachments if needed
+make hydrate
-make preset-skips        # apply presets/levkin-skips.json (safe to re-run; keeps your channel decisions)
+make preset-skips
-make cherry-pick         # bulk data/files picks on priority channels (optional)
+make cherry-pick         # bulk data/files picks (optional)
-make prepare-export      # after review — filtered export for mmetl
+make prepare-export      # filtered export for mmetl
-make test                # pytest + Playwright
+make test
 ```
 
----
+### Re-run Mattermost import (if needed)
 
-## Review progress (manifest snapshot)
+1. Set `mattermost_channel` / `user_emails` in `manifest.json`, then `make prepare-export`.
-
+2. Zip `prepared-export/` → run [mmetl](https://github.com/mattermost/mmetl) v0.4+  
-As of preset + partial manual review:
+   `mmetl transform slack --team levkin --file prepared-export.zip -o mattermost_import.jsonl --bot-owner ilia --default-email-domain levkin.local`
-
+3. Map Slack usernames → existing MM users in the JSONL; **do not** re-import users that would change System Admin roles.
-| Status | Count | Channels |
+4. Package `data/` (attachments) + `mattermost_import.jsonl` → zip; copy to VM;  
-|--------|-------|----------|
+   `mmctl --local import process --bypass-upload /tmp/mattermost-bulk-import.zip`
-| **skip** (preset Tier A/B) | 22 | `#jewbottest`, `#meetings`, `#help`, `#baby`, `#garden`, `#recipes`, `#grants`, `#integromat`, `#clickup`, `#general`, `#after-school-activities`, `#cleaner`, `#medical`, `#purchase`, `#temp`, `#viewings`, `#moishe`, `#520steeles`, `#29wilson`, `#york`, `#jam`, Reading List |
-| **import** | 1 | `#153niagara` |
-| **partial** (cherry-pick) | 3 | `#122harmony`, `#6keefer`, `#zoey` |
-| **undecided** | 14 | See review order below |
-
-Refresh counts after more review:
-
-```bash
-python3 -c "
-import json
-from pathlib import Path
-m = json.loads(Path('manifest.json').read_text())
-from collections import Counter
-c = Counter(v.get('status','?') for v in m.get('channels',{}).values())
-print(dict(c))
-"
-```
 
 ---
 
-## Suggested review order
+## Open tasks
-
-1. `#284richmond` → `#45ritson` → finish `#122harmony`, `#6keefer`, `#zoey` (partial)
-2. `#banking` → `#realestate` → `#airbnb`
-3. `#izik` → `#levkin`
-4. **Tier D (later):** `#random`, `#professional`, `#parenting`, `#adhd`, `#baking-biz`, `#standup`, `#zane`
-
-### Inside a channel
-
-- **Shift+S** — skip whole channel  
-- **Shift+I** — import whole channel  
-- **Cherry-pick mode** — only checked messages export (use **Import whole thread** on parent)  
-- **Skip:** join/leave noise, bot posts, bare “ok” / link-only  
-- **Keep:** attachments, leases, repairs, banking, decisions with dates/names  
-
----
-
-## Presets
-
-`presets/levkin-skips.json` — Tier A/B channel names for `make preset-skips`.  
-Copy and edit for another workspace.
-
----
-
-## UI / server notes
-
-- **Port:** 8765 (`SLACK_IMPORT_PORT`)
-- **Attachments:** served at `/uploads/…` and `/api/file-download?file_id=…`
-- **DOCX preview:** server-side + browser mammoth; spaces in filenames need URL decode (fixed in server)
-- Restart after code changes: `make review` (kill old process on 8765 if needed)
-
----
-
-## What not to use
-
-| Tool | Why |
-|------|-----|
-| `slack-advanced-exporter` + rotating `xoxe.xoxp` token | Token format rejected; enrich zip stayed ~4MB |
-| `slackdump hydrate` | Rejects `?token=xoxe-…` URLs; use **this repo’s** `make hydrate` instead |
-| `slackdump` browser login | Failed in headless/automated runs; not required if official export exists |
-
----
-
-## Open tasks (source of truth)
-
-Tracked in **Vikunja → Homelab → slack-sieve**. Update here when scope changes.
 
 | P | Task | Owner | Status |
 |---|------|-------|--------|
-| **P1** | **Channel review in UI** — `make review`, finish 14 undecided + 3 partial ([order](#suggested-review-order)); save manifest | @you | ⏳ **next** |
+| — | Levkin triage + Mattermost import | @you / @agent | ✅ done 2026-07-10 |
-| **P1** | Pilot import: one channel via `make prepare-export` → `mmetl` → `mmctl` on MM VM `10.0.10.237` | @agent | ⏳ blocked on review |
+| **P3** | Re-hydrate 13 files that returned HTTP 401 | @you | optional |
-| **P2** | User mapping: Slack ID → Mattermost username for `mmetl` | @agent | backlog |
+| **P3** | Optionally re-import missing thread parents | @you | optional |
-| **P3** | Re-hydrate 13 files that returned HTTP 401 (expired export token) | @you | optional |
-| — | `scripts/prepare-export.py` + `make prepare-export` | @agent | ✅ done |
 
 ### Mattermost / homelab
 
-- SSO: [authentik-apps.md#mattermost](https://git.levkin.ca/ilia/ansible/src/branch/master/docs/guides/authentik-apps.md#mattermost) (ansible)
+- SSO: [authentik-apps.md#mattermost](https://git.levkin.ca/ilia/ansible/src/branch/master/docs/guides/authentik-apps.md#mattermost)
 - Import docs: https://docs.mattermost.com/administration-guide/onboard/migrate-from-slack.html
 
 ---
@@ -146,33 +127,17 @@ Tracked in **Vikunja → Homelab → slack-sieve**. Update here when scope chang
 
 ```text
 slack-sieve/
-  server.py           # HTTP API + static UI
+  server.py / slack_export.py / static/
-  slack_export.py     # Parse export, file index, search
+  scripts/   hydrate, preset-skips, auto-cherry-pick, prepare-export, build-test-fixture
-  static/             # Review UI
+  tests/     pytest + Playwright
-  scripts/
+  presets/   levkin-skips.json
-    hydrate-export.sh
+  docs/HANDOFF.md
-    preset-skips.py
+  manifest.json   # gitignored
-    auto-cherry-pick.py
-    prepare-export.py
-    build-test-fixture.py
-  tests/                # pytest + Playwright (mini-export fixture)
-  presets/
-    levkin-skips.json
-  manifest.json       # gitignored — your work in progress
-  docs/
-    HANDOFF.md        # this file
 ```
 
 ---
 
-## Moved from ansible
-
-Slack import tooling was removed from `git@git.levkin.ca:ilia/ansible.git` (2026-05-29). Ansible guide pointer: `docs/guides/slack-mattermost-import.md`.
-
----
-
 ## Security
 
-- Do not commit `manifest.json` or Slack export folders.
+- Do not commit `manifest.json`, Slack export folders, or import zips.
 - Rotate any Slack tokens/cookies pasted in chat during setup.
-- `~/.config/hermes/gitea.env` used for Gitea API only — unrelated to Slack.

scripts/prepare-export.py

@@ -99,8 +99,10 @@ def prepare_export(
         channels_meta = []
 
     manifest_channels = manifest.get("channels") or {}
+    email_map = manifest.get("user_emails") or {}
     included_names: set[str] = set()
     stats: dict[str, int] = {}
+    rename_map: dict[str, str] = {}
 
     for ch in channels_meta:
         name = ch.get("name")
@@ -110,25 +112,52 @@ def prepare_export(
         if not channel_included(entry):
             continue
         included_names.add(name)
+        mm_name = (entry.get("mattermost_channel") or "").strip()
+        if mm_name and mm_name != name:
+            rename_map[name] = mm_name
 
-    # channels.json subset
+    # channels.json subset (optionally renamed for Mattermost)
-    filtered_meta = [c for c in channels_meta if c.get("name") in included_names]
+    filtered_meta = []
+    for c in channels_meta:
+        name = c.get("name")
+        if name not in included_names:
+            continue
+        out_ch = dict(c)
+        if name in rename_map:
+            out_ch["name"] = rename_map[name]
+        filtered_meta.append(out_ch)
     (out_root / "channels.json").write_text(
         json.dumps(filtered_meta, indent=2, ensure_ascii=False) + "\n",
         encoding="utf-8",
     )
 
-    # users.json — copy whole (mmetl needs user mapping)
+    # users.json — copy + inject emails so mmetl can match existing MM users
     users_src = export.paths.users_json
     if users_src.is_file():
-        shutil.copy2(users_src, out_root / "users.json")
+        users = load_json(users_src)
+        if isinstance(users, list) and email_map:
+            for u in users:
+                if not isinstance(u, dict):
+                    continue
+                uid = u.get("id")
+                uname = u.get("name")
+                email = email_map.get(uid) or email_map.get(uname)
+                if email:
+                    profile = dict(u.get("profile") or {})
+                    profile["email"] = email
+                    u["profile"] = profile
+        (out_root / "users.json").write_text(
+            json.dumps(users, indent=2, ensure_ascii=False) + "\n",
+            encoding="utf-8",
+        )
 
     uploads_out = out_root / "__uploads"
     uploads_src = export.paths.root / "__uploads"
 
     for name in sorted(included_names):
         entry = manifest_channels[name]
-        ch_out = out_root / name
+        out_name = rename_map.get(name, name)
+        ch_out = out_root / out_name
         ch_out.mkdir(parents=True)
         count = 0
         for date in export.list_channel_date_files(name):
@@ -164,12 +193,13 @@ def prepare_export(
                     json.dumps(kept, indent=2, ensure_ascii=False) + "\n",
                     encoding="utf-8",
                 )
-        stats[name] = count
+        stats[out_name] = count
 
     summary = {
         "channels": len(stats),
         "messages": sum(stats.values()),
         "per_channel": stats,
+        "renames": rename_map,
     }
     (out_root / "prepare-export-summary.json").write_text(
         json.dumps(summary, indent=2) + "\n",