Architecture#

pagescan turns a phone photo into a clean PDF by running a cascade of vision models followed by classical image-processing steps. The cascade is built so each stage’s failure mode is the next stage’s domain, and so that the whole thing degrades gracefully when the ML stages aren’t available.

Pipeline overview#

        %%{init: {'theme':'base','themeVariables':{'fontSize':'18px','fontFamily':'-apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif','primaryColor':'#f8fafc','primaryBorderColor':'#334155','primaryTextColor':'#0f172a','lineColor':'#475569'}}}%%
flowchart TB
    A[Input photo] --> B[YOLO11 detector]
    B -->|bbox| C[HQ-SAM ViT-B segmenter]
    C -->|mask| D[Quad fit<br/>hull → polyDP]
    D -->|4 corners| E[Validate &<br/>repair]
    E -->|valid| F[Perspective transform]

    B -.->|no detection| G[Legacy SA24+LCNet]
    G -.->|fallback| E

    E -.->|reject| H[Contour fallback]
    H -.->|bbox crop| F

    F --> I[Orientation correction]
    I --> J[Enhancement<br/>shadow + WB + sharpen]
    J --> K[PDF output]

    classDef cascade fill:#2563eb,stroke:#1e3a8a,stroke-width:2px,color:#ffffff
    classDef fallback fill:#d97706,stroke:#7c2d12,stroke-width:2px,color:#ffffff
    class B,C,D,E cascade
    class G,H fallback

Solid arrows are the production cascade. Dashed arrows are fallback paths when the primary chain fails or its dependencies aren’t installed.

Stages#

1. Detection — YOLO11#

A single-class (document) YOLO11n detector trained on phone photos of documents. ONNX-only at inference time — no torch required, ~145 lines in pagescan.detector.

Output: one axis-aligned bounding box with a confidence score. If confidence falls below detector_conf_threshold (default 0.25), the cascade falls through to the legacy path.

2. Segmentation — HQ-SAM ViT-B#

The detection bbox is used as a box prompt to HQ-SAM ViT-B. The segmenter returns a precise binary mask of the document — sharper boundaries than the legacy heatmap models, especially on tilted documents and folded paper.

Lives in pagescan.segmenter. Torch is imported lazily — environments without [ml] extras still get a working import pagescan (the cascade just becomes unavailable and the legacy path takes over).

3. Quad fit#

Convex hull of the mask, then cv2.approxPolyDP with an adaptive epsilon to reduce the hull to four corners. When the polygon approximation fails to produce a clean 4-vertex quad, the system falls back to cv2.minAreaRect on the contour.

4. Validate & repair#

Every candidate quad is checked against three guards before being accepted:

  1. Coverage — the quad area must be at least min_doc_coverage of the frame (default 5%). Catches the common SAM failure of segmenting an inner text block instead of the full page.

  2. Dimensions — width and height must be roughly comparable; rejects degenerate slivers.

  3. Parallelism — opposing sides should be near-parallel. Documents are rectangles; severely non-rectangular quads are perspective artefacts and rejected.

If a quad fails validation, the pipeline drops to the conservative contour-based fallback.

5. Perspective transform#

Standard four-corner perspective warp via cv2.getPerspectiveTransform + cv2.warpPerspective. The output’s aspect ratio is computed from the detected corners — no forced A4 stretch. The original document shape is preserved.

6. Orientation correction#

A small CNN classifier predicts the dominant text orientation (0/90/180/270°). When confidence is low or the prediction is 180°, Tesseract OCR scores all four rotations and picks the one with the most recognised words. Without Tesseract, the CNN result is trusted directly.

7. Enhancement#

Optional steps (all toggleable on ScanConfig):

  • Shadow removal — illumination normalization to flatten uneven lighting.

  • White balance — paper background pushed toward pure white.

  • Contrast stretch + unsharp mask — scanner-like crispness.

8. PDF output#

A4 at 300 DPI by default (overridable). Output is JPEG-encoded inside a PDF wrapper via img2pdf. Quality tunable via jpeg_quality.

Fallback chain#

The cascade is the primary path, but pagescan is built to keep working when its dependencies aren’t present:

Condition

Behaviour

Cascade weights missing

Falls back to legacy SA24+LCNet ONNX chain.

[ml] extras not installed

Same as above (torch isn’t importable).

Legacy ML also fails

Falls back to contour-based detection on the edge map.

Contour also fails

Returns the original image with a flag set on result['success'].

use_ml=False

Skips all ML, goes straight to contour fallback. Useful for headless/CI.

This means a pip install pagescan without [ml] still produces good results — just slightly weaker on hard cases. The cascade is an accuracy upgrade, not a hard requirement.

Why a cascade?#

The previous architecture used heatmap regression models (SA24 + LCNet) that predict corner pixel locations directly. They are fast (~70 ms total) and self-contained, but the model’s loss is at the per-pixel level — small detection errors compound to large IoU errors on tilted documents.

The cascade decouples the two sub-problems:

  1. “Where is the document?” — a detection problem, solved at the bbox level. YOLO is purpose-built for this.

  2. “What are its exact pixel boundaries?” — a segmentation problem. HQ-SAM is purpose-built for that, and adding a box prompt drastically narrows the search space.

The architectural cost is one extra inference pass and the torch dependency for HQ-SAM. The benefit is sharper boundaries on tilted, occluded, and shadowed documents — exactly the failure cases the legacy chain struggled with.

See Benchmark for measured comparisons on a held-out 50-photo test set.