Structured OCR Schema

doc2mark’s OCR layer does not return a single free-form markdown blob. Instead, every image becomes a structured OCRPage, carried on OCRResult.document. The schema enforces a hard boundary between two concerns:

raw — what is literally on the page: a verbatim transcription, any tables, label/value fields, and additive verbatim indexes (headings, dates, metrics). No inference, no commentary.
interpretation — the model’s reading of the page: document type, a short summary, key findings, a knowledge graph, and confidence. This is the part that requires a language model to reason about the content.

This split is the most important idea in the schema. The raw half is always present and is the trustworthy, auditable record of the page. It is the token source for the BM42 sparse index, so everything in it is verbatim. The interpretation half is None whenever the model was not asked to — or could not — reason about the page, specifically:

detail="raw" was requested (raw transcription only),
the provider is non-LLM (e.g. Tesseract, which cannot infer), or
the structured-output parse failed and the layer fell back gracefully.

Every model on this page is a Pydantic BaseModel subclass, and every field is defaulted: the LLM providers emit them through LangChain’s with_structured_output(method="json_schema"), and OpenAI strict mode requires all properties to be present, so each field must be satisfiable without input. Optional fields serialize as anyOf: [T, null].

Note

Import these from doc2mark.ocr.schema. The convenience accessor to_markdown() collapses the structured page back into a single markdown string, which is what powers the back-compatible OCRResult.text.

The authoritative, always-current field list for each model is generated below directly from the source (defaults, types, and per-field descriptions). The prose around each block explains why the field exists and how the models fit together; the field documentation itself is the rendered docstring.

The verbatim / interpretation boundary

The single rule that holds the schema together: the interpretation layer never invents a printed value, and never moves one out of raw. Anything the model claims is on the page — a figure axis label, an entity name, a relation’s subject — is an additive copy of a string that already appears in raw.text. This keeps the sparse index intact (BM42 reads raw.text) while making the same facts queryable as typed structure.

That invariant is mechanically enforced. router_invariants() returns a list of violations for a page (empty means OK) and is intended as a CI / eval assertion over recorded outputs. It checks, among other things, that every verbatim figure string is a substring of raw.text, that every Section.heading came from raw.headings, that primary_date was selected from raw.dates rather than fabricated, that entity names and relation subjects/objects are substrings of raw.text, and that withheld (“illustrative”) values appear only on a high-confidence screenshot page.

When you read a page, treat raw as ground truth and interpretation as an additive overlay that is safe to ignore. Always check page.interpretation is not None before reading interpretive fields — with detail="raw" or a Tesseract backend it will be None.

`OCRPage` — the top-level result

One image’s structured OCR result. It bundles the always-present raw extraction with the optional interpretation, and exposes one method.

Signature

class OCRPage(BaseModel):
    raw: RawExtraction = Field(default_factory=RawExtraction)
    interpretation: Optional[Interpretation] = None

to_markdown() -> str: Render a readable markdown view of the page, used as the back-compatible OCRResult.text. It prefers structured tables/fields over the flat text dump, and renders the additive overlays (real metrics, then figures, then a section outline) degraded-safe. When the interpretation carries a page_markdown whole-page rendering that verifiably covers the verbatim text, that structured rendering replaces the flat raw.text dump (with a hidden verbatim tail preserving any uncovered tokens so the index stays complete).

class doc2mark.ocr.schema.OCRPage(*, raw=<factory>, interpretation=None)[source]

Bases: BaseModel

One image’s structured OCR result, carried on OCRResult.document.

Parameters:

raw (RawExtraction)
interpretation (Interpretation | None)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

to_markdown()[source]

Render a readable markdown view of this page.

Used as the back-compat OCRResult.text and by pipelines that want a single string. Prefers structured tables/fields over the flat text dump.

Return type:: str

`RawExtraction` — verbatim page content

A verbatim transcription of the page: no commentary, no inference. This is the raw field of OCRPage and the BM42 token source. Alongside the full text and structured tables / fields, it carries three additive verbatim indexes — headings, dates, and metrics — each entry of which is a copy of tokens already present in text, surfaced so an indexer can boost or filter without re-parsing. The metrics list holds Metric objects (typed numeric assertions); headings and dates are plain verbatim strings.

class doc2mark.ocr.schema.RawExtraction(*, text='', tables=<factory>, fields=<factory>, headings=<factory>, dates=<factory>, metrics=<factory>, detected_language=None, has_handwriting=False)[source]

Bases: BaseModel

Verbatim transcription. No commentary, no inference. BM42 token source.

Parameters:

text (str)
tables (List[Table])
fields (List[KeyValue])
headings (List[str])
dates (List[str])
metrics (List[Metric])
detected_language (str | None)
has_handwriting (bool)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

`Table` — a transcribed table

A table transcribed verbatim, used inside RawExtraction.tables. The preferred representation is html: a clean <table> that can encode merged cells via colspan / rowspan (which the flat headers / rows grid and markdown cannot). The html value is sanitized at the model boundary by a field validator (see sanitize_table_html()) to a strict table-only allowlist, so the stored string is always safe to embed in rendered output. illustrative flags a demo/mockup table whose values are not real data, and row_count records how many sample rows a header-only illustrative table intentionally left untranscribed.

class doc2mark.ocr.schema.Table(*, caption='', headers=<factory>, rows=<factory>, html='', markdown='', illustrative=False, row_count=None)[source]

Bases: BaseModel

A table transcribed verbatim from the image.

html is the preferred representation: a clean <table> that can encode merged cells via colspan/rowspan (which headers/rows and markdown cannot). headers/rows remain a best-effort flat view for simple, machine-readable access.

Parameters:

caption (str)
headers (List[str])
rows (List[List[str]])
html (str)
markdown (str)
illustrative (bool)
row_count (int | None)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

`KeyValue` — a label/value pair

A single label/value pair, used both inside RawExtraction.fields (forms, receipts) and inside Interpretation.definitions (term/definition pairs). illustrative flags a demo/sample value from a screenshot or mockup region.

class doc2mark.ocr.schema.KeyValue(*, label='', value='', illustrative=False)[source]

Bases: BaseModel

A label/value pair, e.g. for forms and receipts.

Parameters:

label (str)
value (str)
illustrative (bool)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

`Metric` — a typed numeric assertion

A single number printed on the page, captured as a flat, four-field typed fact (label, value, unit, illustrative). A Metric is an additive view of a number that is also present verbatim in raw.text — never a relocation of it. value is copied exactly as printed (e.g. "$4.2B", "98.5%", "< 100 ms") and never normalized or computed. Metrics live in RawExtraction.metrics and make a number queryable as a typed fact while the sparse index keeps reading raw.text.

class doc2mark.ocr.schema.Metric(*, label='', value='', unit='', illustrative=False)[source]

Bases: BaseModel

A single typed numeric assertion printed on the page.

Additive structured view of a number that is ALSO present verbatim in raw.text — never a relocation of it. Flat by design (4 fields) to stay fillable on weaker models; the BM42 sparse index reads raw.text while this makes the number queryable as a typed fact.

Parameters:

label (str)
value (str)
unit (str)
illustrative (bool)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

The nested interpretation models

The interpretation layer carries a small family of nested models that structure a page’s meaning — figures, hierarchy, and a knowledge graph. They are deliberately shallow (max nesting depth 4: OCRPage → interpretation → figures → data_points), with no recursion, no model-unions, and every field defaulted, so they stay fillable under with_structured_output(json_schema). They are additive and BM42-safe: every verbatim string inside them mirrors text already in raw.text, an invariant enforced by router_invariants().

Metric (above) — a typed number, in raw.metrics.
Figure — one chart / diagram / infographic panel, in interpretation.figures. Its kind drives which branch fills: quantitative kinds (bar/line/pie/…) fill data_points; structural kinds (flowchart/org_chart/network) fill nodes + edges; everything else falls back to labels + meaning. meaning (and trend for charts) is the always-attempt interpretive fallback when the typed data can’t be read.
- DataPoint — one (label, value, series) chart reading, flattened to tidy-long form (the deepest leaf, depth 4). All three fields are verbatim copies of printed text; values are never pixel-estimated.
- DiagramNode — one labelled box / shape / actor in a flowchart, org chart, or network. There is no synthetic id — edges reference nodes by their verbatim label.
- DiagramEdge — a directed connection between two nodes, referenced by verbatim from_label / to_label (each must match a DiagramNode.label in the same figure).
Section — one heading-delimited region in reading order. Hierarchy is a flat list plus an integer level (never recursive children). Its heading is a verbatim raw.headings entry; summary and key_points are paraphrase.
Entity — a typed named entity (person / org / product / location / concept / other) with a salience ranking. Its name is verbatim in raw.text; dates, money, and KPIs are not entities — they live in raw.dates / raw.metrics.
Relation — a knowledge triple (subject / relation / object) for a claim explicitly stated on the page, with an evidence quote that makes it falsifiable. subject and object are substrings of raw.text; the predicate and evidence may paraphrase.

Finally, the interpretation can carry a page_markdown string: a clean, structured Markdown re-layout of a whole-page image render. It is filled only when explicitly instructed (slide / scan image-strategy pages) and, when filled, must cover every word, number, and CJK character from raw.text verbatim — it re-flows existing content into a readable document, it never adds or removes any.

class doc2mark.ocr.schema.Figure(*, kind='other', title='', x_axis='', y_axis='', data_points=<factory>, trend='', nodes=<factory>, edges=<factory>, labels=<factory>, meaning='', illustrative=False)[source]

Bases: BaseModel

Typed structured view of ONE chart/diagram/infographic panel, in interpretation.figures (flat list, never nested in each other). kind drives which branch fills: quantitative->data_points; structural->nodes+edges; else ->labels+meaning. Every verbatim string is an ADDITIVE copy of raw.text; meaning/trend are the always-attempt interpretive fallbacks.

Parameters:

kind (Literal['bar', 'line', 'pie', 'scatter', 'area', 'combo', 'table_visual', 'flowchart', 'org_chart', 'network', 'timeline_diagram', 'map', 'infographic_panel', 'other'])
title (str)
x_axis (str)
y_axis (str)
data_points (List[DataPoint])
trend (str)
nodes (List[DiagramNode])
edges (List[DiagramEdge])
labels (List[str])
meaning (str)
illustrative (bool)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.DataPoint(*, label='', value='', series='')[source]

Bases: BaseModel

One (category, value, series) reading from a chart, flattened to tidy-long form (the deepest leaf, depth 4). label/value/series are VERBATIM copies of text also in raw.text; never pixel-estimated.

Parameters:

label (str)
value (str)
series (str)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.DiagramNode(*, label='', kind='')[source]

Bases: BaseModel

One labelled box/shape/actor in a flowchart, org chart, or network. No synthetic id — edges reference nodes by verbatim label (already in raw.text).

Parameters:

label (str)
kind (str)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.DiagramEdge(*, from_label='', to_label='', label='')[source]

Bases: BaseModel

A directed connection between two DiagramNodes, referenced by verbatim label (membership in nodes is mechanically checkable, like primary_date-in-raw.dates).

Parameters:

from_label (str)
to_label (str)
label (str)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.Section(*, heading='', level=1, summary='', key_points=<factory>)[source]

Bases: BaseModel

One heading-delimited region, in reading order. Hierarchy is a FLAT list + int level (never recursive children). heading is a VERBATIM raw.headings entry; summary/key_points are paraphrase.

Parameters:

heading (str)
level (int)
summary (str)
key_points (List[str])

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.Entity(*, name='', type='other', salience='mentioned', role='')[source]

Bases: BaseModel

A typed named entity (replaces the flat entities list). Additive view of a name already VERBATIM in raw.text. Dates/money/KPIs stay in raw.dates/raw.metrics.

Parameters:

name (str)
type (Literal['person', 'org', 'product', 'location', 'concept', 'other'])
salience (Literal['primary', 'secondary', 'mentioned'])
role (str)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class doc2mark.ocr.schema.Relation(*, subject='', relation='', object='', evidence='')[source]

Bases: BaseModel

A knowledge triple for a claim EXPLICITLY stated on the page (never inferred). Flat triple of strings; evidence makes it falsifiable. Highest confabulation risk — emit only when explicit.

Parameters:

subject (str)
relation (str)
object (str)
evidence (str)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

`Interpretation` — the model’s reading

The model’s analysis of the page, never mixed into raw. This is the interpretation field of OCRPage, and is None whenever the model was not asked to (or could not) reason.

It classifies the page via document_type (one of 16 values: document, table, form, receipt, handwriting, code, chart, photo, screenshot, diagram, infographic, logo, stamp, mixed, blank, other) and carries the retrieval / comprehension anchors and overlays described above — page_title, primary_message, keywords, the nested figures / sections / typed_entities / relations, plus column_layout, page_role, primary_date, action_items, definitions, content_fidelity, self_confidence, legibility, and page_markdown. See the rendered field list for the exact defaults and meanings.

class doc2mark.ocr.schema.Interpretation(*, document_type='other', summary='', key_findings=<factory>, visual_notes='', page_title=None, primary_message=None, keywords=<factory>, figures=<factory>, sections=<factory>, typed_entities=<factory>, relations=<factory>, column_layout='single', page_role=None, primary_date=None, action_items=<factory>, definitions=<factory>, self_confidence=0.0, legibility='high', content_fidelity='verbatim', page_markdown=None)[source]

Bases: BaseModel

The model’s reading of the page. Never mixed into raw.

Parameters:

document_type (Literal['document', 'table', 'form', 'receipt', 'handwriting', 'code', 'chart', 'photo', 'screenshot', 'diagram', 'infographic', 'logo', 'stamp', 'mixed', 'blank', 'other'])
summary (str)
key_findings (List[str])
visual_notes (str)
page_title (str | None)
primary_message (str | None)
keywords (List[str])
figures (List[Figure])
sections (List[Section])
typed_entities (List[Entity])
relations (List[Relation])
column_layout (Literal['single', 'double', 'multi', 'complex'])
page_role (Literal['title', 'agenda', 'section_header', 'content', 'data', 'case_study', 'comparison', 'timeline', 'conclusion', 'appendix', 'other'] | None)
primary_date (str | None)
action_items (List[str])
definitions (List[KeyValue])
self_confidence (Annotated[float, Ge(ge=0.0), Le(le=1.0)])
legibility (Literal['high', 'medium', 'low'])
content_fidelity (Literal['verbatim', 'described', 'caption', 'skipped', 'mixed'])
page_markdown (str | None)

model_config = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

A complete filled `OCRPage` (receipt)

The example below shows a fully populated page for a grocery receipt, with both the verbatim raw half and the model’s interpretation half.

from doc2mark.ocr.schema import (
    OCRPage, RawExtraction, Interpretation, Table, KeyValue, Metric,
)

page = OCRPage(
    raw=RawExtraction(
        text=(
            "ACME STORE\n"
            "123 Market St\n"
            "Apples  2  $3.00\n"
            "Bread   1  $2.50\n"
            "Milk    1  $4.00\n"
            "Total      $9.50\n"
            "Thank you for shopping!"
        ),
        tables=[
            Table(
                caption="Items",
                headers=["Item", "Qty", "Price"],
                rows=[
                    ["Apples", "2", "$3.00"],
                    ["Bread", "1", "$2.50"],
                    ["Milk", "1", "$4.00"],
                ],
            ),
        ],
        fields=[
            KeyValue(label="Store", value="ACME STORE"),
            KeyValue(label="Total", value="$9.50"),
        ],
        headings=["ACME STORE"],
        metrics=[Metric(label="Total", value="$9.50")],
        detected_language="en",
        has_handwriting=False,
    ),
    interpretation=Interpretation(
        document_type="receipt",
        summary=(
            "Grocery receipt from ACME Store listing three items "
            "with a total of $9.50."
        ),
        key_findings=["3 line items", "Total: $9.50"],
        primary_message="Three grocery items were purchased for $9.50.",
        self_confidence=0.94,
        legibility="high",
    ),
)

# Always-present raw half:
assert page.raw.fields[1].value == "$9.50"

# Interpretation present here because an LLM provider ran with full detail:
assert page.interpretation is not None
assert page.interpretation.document_type == "receipt"

# Single-string markdown view (back-compat OCRResult.text):
print(page.to_markdown())

Structured OCR Schema

The verbatim / interpretation boundary

OCRPage — the top-level result

RawExtraction — verbatim page content

Table — a transcribed table

KeyValue — a label/value pair

Metric — a typed numeric assertion

The nested interpretation models

Interpretation — the model’s reading

A complete filled OCRPage (receipt)

`OCRPage` — the top-level result

`RawExtraction` — verbatim page content

`Table` — a transcribed table

`KeyValue` — a label/value pair

`Metric` — a typed numeric assertion

`Interpretation` — the model’s reading

A complete filled `OCRPage` (receipt)