ilia
6a78c84bcd
- capture/: frame sampling at configurable fps, ffmpeg audio extraction (WAV, whisper-ready) - vision/: changed-pixel motion scoring, stable/moving segmentation, contour + perspective document detection, Laplacian sharpness scoring, optional CLAHE enhancement - pipeline: two-pass detect (motion timeline, then best-frame crop per stable window) writing crops, debug frames, report.json, and motion_scores.csv - CLI: probe / detect / extract-audio subcommands - config.yaml with tunable thresholds; placeholder packages for events, transcribe, ocr, naming, pdf, review_cli - synthetic sample video generator + 16 unit tests
80 lines
2.7 KiB
Markdown
80 lines
2.7 KiB
Markdown
# PaperPod
|
|
|
|
Local, privacy-first tool that converts a single overhead video recording of
|
|
documents (receipts, letters, multi-page stacks) into individual, properly
|
|
named PDFs ready for a Paperless-ngx consume folder. All processing runs
|
|
offline on your machine.
|
|
|
|
## How it works
|
|
|
|
1. Record one continuous overhead video, placing documents under the camera
|
|
one at a time (optionally saying out loud what each one is).
|
|
2. PaperPod finds "stable windows" where nothing is moving, picks the sharpest
|
|
frame in each, detects the document outline, and produces a
|
|
perspective-corrected crop.
|
|
3. (Upcoming) Spoken descriptions (faster-whisper) or OCR (Tesseract) name
|
|
each document; multi-page stacks are grouped into single PDFs; a review
|
|
step lets you rename/merge/split before export.
|
|
|
|
## Status
|
|
|
|
| Module | Purpose | Status |
|
|
|---|---|---|
|
|
| `capture/` | Frame sampling + audio extraction from video files | Built |
|
|
| `vision/` | Motion detection, document contours, perspective crop, sharpness | Built |
|
|
| `events/` | State machine: placed / page-flipped / cleared, pod grouping | Planned |
|
|
| `transcribe/` | Local speech-to-text (faster-whisper) | Planned |
|
|
| `ocr/` | OCR fallback naming (Tesseract) | Planned |
|
|
| `naming/` | Final filename assembly + summary CSV | Planned |
|
|
| `pdf/` | PDF assembly + Paperless-ngx consume staging | Planned |
|
|
| `review_cli/` | Pre-export review (rename/merge/split) | Planned |
|
|
|
|
## Setup
|
|
|
|
Requires Python 3.11+ and ffmpeg (`brew install ffmpeg`).
|
|
|
|
```bash
|
|
python3 -m venv .venv
|
|
source .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Usage
|
|
|
|
```bash
|
|
# Generate a synthetic test video (no camera needed)
|
|
python sample_data/make_sample_video.py
|
|
|
|
# Inspect a video
|
|
python -m paperpod probe sample_data/videos/synthetic_sample.mp4
|
|
|
|
# Detect stable windows + document candidates
|
|
python -m paperpod detect sample_data/videos/synthetic_sample.mp4
|
|
|
|
# Extract the audio track (16 kHz mono WAV, whisper-ready)
|
|
python -m paperpod extract-audio my_recording.mp4
|
|
```
|
|
|
|
`detect` writes to `output/<video-name>/`:
|
|
|
|
- `crops/window_NNN.png` — perspective-corrected document candidates
|
|
- `frames/window_NNN_full.png` — the full best frame per window (debugging)
|
|
- `report.json` — video metadata, motion events, stable windows, detections
|
|
- `motion_scores.csv` — per-sample motion scores, for tuning `motion.threshold`
|
|
|
|
## Tuning
|
|
|
|
All thresholds live in `config.yaml` (motion sensitivity, stable window
|
|
duration, contour size floor, Canny thresholds, speech matching window,
|
|
output paths). Plot `motion_scores.csv` to pick a `motion.threshold` that
|
|
separates your camera's noise floor from real hand movement.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
python -m pytest
|
|
```
|
|
|
|
Tests are self-contained: they synthesize frames and tiny videos on the fly,
|
|
no sample assets required.
|