smol-doc-analyzer

Local insurance document classification, extraction, and memo generation

Local · synthetic-only · dual pipeline

smol-doc-analyzer is a small, locally deployable pipeline for insurance document intake — classification, field extraction, and adjuster-style memos — built as a cost-efficient alternative to frontier API models for high-volume workflows.

Live on Posit Connect Cloud

This portal is published at smol-doc-analyzer on Posit Connect Cloud. Re-publish after edits with ./scripts/publish_docs_site.sh.

Start here

Quick Start

Install, secrets, and a minimal DICIE one-liner — no training required.

Usage

Full CLI reference for both pipelines, corpus, training, eval, and Discord.

Architecture

Dual-pipeline design, repository map, and chronological stage contract.

Notebooks

Rendered walkthroughs with stored outputs, figures, and inline code.

Commands

Cheat sheet of the most common entry points.

Data Provenance

Synthetic-only disclosure — no real insurer data.

Two inference paths

flowchart LR
  IN[(PDF / PNG / text)] --> DICIE
  IN --> MEMO
  subgraph DICIE["DICIE · src/docie/"]
    D1[Process] --> D2[Classify] --> D3[Extract] --> D4[Respond]
  end
  subgraph MEMO["Memo chain · src/pipeline/"]
    M1[to_markdown] --> M2[classify] --> M3[extract] --> M4[vision] --> M5[summarize]
  end

Dual inference paths in smol-doc-analyzer

Path	Best for	Guide	Notebook
DICIE	Paper Fig. 1 medical / salvage image-first flow	DICIE Pipeline	Walkthrough
Memo chain	ACORD-style intake + adjuster memos	Usage · Architecture	Walkthrough

Explore both from the Pipeline hub.

Sample corpus & classical baseline

Synthetic only

All demo data is synthetic or public. See Data Provenance.

Sample Document Corpus — SQLite house for medical / salvage docs
Random Forest notebook — TF-IDF + RF with OCR/handwriting noise
Corpus SQL notebook · Train/test notebook

Preview & publish

# local preview
./scripts/preview_docs_site.sh

# public Posit Connect Cloud (no GitHub Actions)
./scripts/publish_docs_site.sh

Details: How-to: Quarto site · About

--- title: "smol-doc-analyzer" subtitle: "Local insurance document classification, extraction, and memo generation" page-layout: full --- ::: {.hero-kicker} Local · synthetic-only · dual pipeline ::: ::: {.hero-lead} **smol-doc-analyzer** is a small, locally deployable pipeline for insurance document intake — classification, field extraction, and adjuster-style memos — built as a cost-efficient alternative to frontier API models for high-volume workflows. ::: ::: {.callout-tip} ## Live on Posit Connect Cloud This portal is published at [smol-doc-analyzer on Posit Connect Cloud](https://019f8d0a-2732-fecb-b056-1f69f4451c00.share.connect.posit.cloud). Re-publish after edits with `./scripts/publish_docs_site.sh`. ::: ## Start here ::::: {.feature-grid} ::: {.feature-card} ### [Quick Start](quick-start.qmd) Install, secrets, and a minimal DICIE one-liner — no training required. ::: ::: {.feature-card} ### [Usage](usage.md) Full CLI reference for both pipelines, corpus, training, eval, and Discord. ::: ::: {.feature-card} ### [Architecture](architecture.qmd) Dual-pipeline design, repository map, and chronological stage contract. ::: ::: {.feature-card} ### [Notebooks](notebooks/index.qmd) Rendered walkthroughs with stored outputs, figures, and inline code. ::: ::: {.feature-card} ### [Commands](reference/commands.qmd) Cheat sheet of the most common entry points. ::: ::: {.feature-card} ### [Data Provenance](data_provenance.md) Synthetic-only disclosure — no real insurer data. ::: ::::: ## Two inference paths ```{mermaid} %%| fig-cap: "Dual inference paths in smol-doc-analyzer" flowchart LR IN[(PDF / PNG / text)] --> DICIE IN --> MEMO subgraph DICIE["DICIE · src/docie/"] D1[Process] --> D2[Classify] --> D3[Extract] --> D4[Respond] end subgraph MEMO["Memo chain · src/pipeline/"] M1[to_markdown] --> M2[classify] --> M3[extract] --> M4[vision] --> M5[summarize] end ``` | Path | Best for | Guide | Notebook | |------|----------|-------|----------| | **DICIE** | Paper Fig. 1 medical / salvage image-first flow | [DICIE Pipeline](docie_pipeline.qmd) | [Walkthrough](notebooks/docie_pipeline_walkthrough.ipynb) | | **Memo chain** | ACORD-style intake + adjuster memos | [Usage](usage.md) · [Architecture](architecture.qmd) | [Walkthrough](notebooks/pipeline_walkthrough.ipynb) | Explore both from the [Pipeline hub](pipelines/index.qmd). ## Sample corpus & classical baseline ::: {.callout-note} ## Synthetic only All demo data is synthetic or public. See [Data Provenance](data_provenance.md). ::: - [Sample Document Corpus](sample_document_corpus.md) — SQLite house for medical / salvage docs - [Random Forest notebook](notebooks/random_forest_text_handwriting_classification.ipynb) — TF-IDF + RF with OCR/handwriting noise - [Corpus SQL notebook](notebooks/sample_corpus_sql_integrations.ipynb) · [Train/test notebook](notebooks/sample_corpus_train_test_pipeline.ipynb) ## Preview & publish ```bash # local preview ./scripts/preview_docs_site.sh # public Posit Connect Cloud (no GitHub Actions) ./scripts/publish_docs_site.sh ``` Details: [How-to: Quarto site](how-to/launch-quarto-site.qmd) · [About](about.qmd) ::: {.see-also} ### See also [Implementation plan](plan/plan.md) · [Bugfix audit](bugfix_audit_round2.md) · [Changelog](CHANGELOG.md) · [GitHub repository](https://github.com/Exios66/smol-doc-analyzer) :::