The result: ProcessedDocument
Every conversion returns a ProcessedDocument:
Field |
What it holds |
|---|---|
|
The document as Markdown (or JSON / plain text with |
|
A |
|
The same content as a list of typed items in reading order (PDF, Office and image files; see Content items). |
|
Pictures, when |
|
Read from |
result.markdown is content; result.text is content with # heading markers,
**bold**, *italics*, [links](...) and `code` marks removed by simple patterns
(tables, list markers and comments stay; a # followed by a space is removed anywhere, so
C# is becomes Cis). result.to_dict() returns a JSON-serialisable dict of
all fields (enums as their values), the payload of output_format="json" and of the CLI’s
--format json. result.get_chunks() splits the document for retrieval (Chunking and token usage).
Output formats
load(..., output_format=...) takes "markdown" (default), "json" or "text", or an
OutputFormat member.
"markdown":contentis Markdown."json":contentisjson.dumps(result.to_dict())(itscontentfield is the Markdown). For formats that have nojson_contentthe attributeresult.json_contentis then set to onetext:normalitem holding the Markdown (the JSON string keepsnull)."text":contentisresult.text.
Content items
Each item of json_content is a dict with type and content. PDF items also carry
page (from 1) and position_y (the vertical position on the page); Word, PowerPoint and
Excel items carry page (the page, slide or sheet number); PDF and Word titles and section
headings carry level. Word items of a header or footer carry region ("header" /
"footer"), and a Word paragraph or list item with bold, italic or linked runs carries
markdown: its text as Markdown (**bold**, *italic*, [text](url)), which the
Markdown output uses, while content keeps the text as written.
|
Content |
|---|---|
|
A level-1 heading ( |
|
A heading; |
|
A paragraph. |
|
A list, with its Markdown markers. |
|
A figure or table caption. |
|
A footnote, as printed ( |
|
The OCR text of a picture or of a whole-page render, wrapped in |
|
A table as HTML (merged cells) or a Markdown pipe table (see Tables). |
|
An extracted picture as base64 ( |
|
Running headers and footers of a PDF the Markdown leaves out: the later copies of a
repeated line, and every copy of a bare page number or of a line repeating a title. They
are not in |
from collections import Counter
from doc2mark import load
result = load("report.pdf")
print(Counter(item["type"] for item in result.json_content))
headings = [(i["level"], i["content"]) for i in result.json_content if i["type"] in ("text:title", "text:section")]
print(headings[:3])
Conversion reports in metadata.extra
A key is present only when it has something to say.
Key |
Meaning |
|---|---|
|
PDF with OCR on. |
|
PDF with OCR on: what was sent to the provider. |
|
Any format, when OCR went wrong somewhere: |
|
LLM OCR token counts of this load: |
|
PDF pages whose text layer is garbled: |
|
PDF pages with invisible text over nothing the page shows, which was left out:
|
|
PDF: number of tables found. |
|
DOCX/PPTX with OCR on: |
|
Legacy files converted by LibreOffice, e.g. |
|
The optional judge was asked something: |
from doc2mark import UnifiedDocumentLoader
loader = UnifiedDocumentLoader(ocr_provider="tesseract")
result = loader.load("scan.pdf", ocr_images=True)
extra = result.metadata.extra
print(extra["ocr_routing"])
print(extra["ocr_images"]["ocr_requests"], "image(s) sent to OCR")
if extra.get("ocr_issues"):
print("check before indexing:", extra["ocr_issues"])
Errors
load() raises FileNotFoundError for a missing file,
UnsupportedFormatError for an extension it does not know and ValueError
for an unknown output_format. Any failure during conversion is raised as
ProcessingError (Processing failed: ...) with the original exception as
its __cause__: for example an OCR engine that cannot run (Tesseract without the requested
language data, cause OCREngineError), a failed LibreOffice conversion (a chain of
ProcessingError that ends in ConversionError) or a legacy file without
LibreOffice. So catch
ProcessingError (UnsupportedFormatError is a subclass); OCRError and
ConversionError do not reach you directly. A picture or page the OCR provider could not read
does not fail the document: it is reported in ocr_issues.