Images

Image files

.png, .jpg, .jpeg, .webp, .tif, .tiff, .bmp and .gif are read with Pillow; .heic and .heif need doc2mark[heif] (pillow-heif), and .avif needs a Pillow build with AVIF support. Without OCR the Markdown describes the picture; with OCR it also holds the text the provider read:

from doc2mark import UnifiedDocumentLoader

loader = UnifiedDocumentLoader(ocr_provider="tesseract")
result = loader.load("receipt.png", ocr_images=True)
print(result.content)

# Image: receipt.png

- **Format**: PNG
- **Dimensions**: 1654 x 2339 pixels
- **Mode**: RGB
- **Size**: 41,873 bytes

## OCR Extracted Text

ACME STORE
Total 9.50

When the provider finds no text, the section is ## OCR Extraction with No text detected in image; when OCR fails for this image (the request raised, for example without an API key, or the provider flagged the answer failed), the section shows [image: OCR unavailable], the failure is counted in metadata.extra["ocr_issues"] and the file still converts, but is not stored in cache_dir (an OCR engine that cannot run at all, such as Tesseract without its language data, fails the load). extract_images=True (implied by ocr_images=True with a provider) also returns the image re-encoded as PNG in result.images ({"type": "image", "content": <base64>, "format": "png", "width", "height", "original_format", "filename"}). metadata has page_count 1, image_count 1, word_count of the OCR text, and extra with width, height, mode, format and has_ocr.

Pictures inside documents

extract_images=True without OCR returns the pictures of PDF, Word, Excel and PowerPoint files as they are: each is embedded in the Markdown as a data: URI image (![Image](data:...)), is an image item of json_content and is listed in result.images (PDF: {"type": "base64", "data": ...}; Office: {"data": <bytes>, "page": n}). Leave it off when you index text. With ocr_images=True and a provider the pictures are OCR’d instead and their text is placed where the picture is (text:image_description items):

  • PDF. A picture is OCR’d by what it shows, once per distinct image, and its text is emitted at each place the page shows it. Plain backgrounds, colour blocks, frames and icons without text are not sent to OCR; small charts, stamps and lettered logos are. A picture cropped by a clip path is read as the page shows it. A picture whose OCR failed leaves [image: OCR unavailable] where it shows; a logo repeated at the same place on most pages is kept once. Scanned pages and pages whose content is only in pictures are OCR’d as a whole-page render instead (When pages and pictures are OCR’d).

  • Word, PowerPoint, Excel. Pictures are OCR’d in one batch per document (in Word, the pictures of headers, footers and text boxes too); a picture in which OCR finds no text adds nothing, one whose OCR failed leaves [image: OCR unavailable], as in PDFs. A picture in a Word or Excel table cell is marked [Image] ([Image: OCR text] with OCR) in its cell. Word and PowerPoint files made mostly of pictures can be converted to PDF and OCR’d page by page (the Office image route, Supported formats).

metadata.extra["ocr_images"] (PDF) reports how many requests were sent and which pictures were skipped, and metadata.extra["ocr_issues"] lists pictures that were refused or could not be read (The result: ProcessedDocument).

Large images and cost

Set OCR_MAX_IMAGE_DIM (pixels) to downscale images whose longest side is larger before they are sent to an LLM provider; images within the bound are left as they are. It is off by default.