UnifiedDocumentLoader

The loader: one instance holds the OCR provider, caches, table style and judges, and converts files with load() or in batches. Guide: Loading documents.

UnifiedDocumentLoader(
    ocr_provider='openai',
    api_key=None,
    ocr_config=None,
    cache_dir=None,
    ocr_cache=None,
    model='gpt-5.4-mini',
    temperature=0,
    max_tokens=8192,
    max_workers=None,
    prompt_template='default',
    timeout=30,
    max_retries=3,
    top_p=1.0,
    frequency_penalty=0.0,
    presence_penalty=0.0,
    base_url=None,
    project=None,
    location='global',
    default_prompt=None,
    task=None,
    structured=None,
    detail=None,
    table_style=None,
    legibility_judge=None,
    boilerplate_judge=None,
    judge=None,
)
ocr_provider

"openai", "vertex_ai", "gemini", "tesseract", an OCRProvider, a provider instance (BaseOCR), or None / "none" / "disabled" for no OCR. The provider object is created here; it only calls out when a document is loaded with ocr_images=True.

api_key

OpenAI key; default OPENAI_API_KEY.

ocr_config

An OCRConfig. task, structured and detail, when not None, override its fields; for "openai" and "vertex_ai" the loader’s model, temperature and max_tokens (and base_url for OpenAI) take precedence over the config’s.

cache_dir, ocr_cache

The document cache folder and the OCR answer cache (Caching).

model, temperature, max_tokens

For "openai", "vertex_ai" and "gemini". With Vertex AI the default gpt-5.4-mini means ocr_config.model, else gemini-3.1-flash-lite-preview.

timeout, max_retries

Per-request timeout (seconds) and retries of the OpenAI or Vertex AI client.

max_workers

At most this many OCR requests at once, when ocr_config.max_concurrency is not set (None: OCR_MAX_CONCURRENCY, else LangChain’s default).

top_p, frequency_penalty, presence_penalty

Sampling settings of the OpenAI request and the Vertex AI client, sent only when they differ from these defaults (the API’s own).

base_url

OpenAI-compatible endpoint (default OPENAI_BASE_URL).

project, location

Vertex AI project (default GOOGLE_CLOUD_PROJECT) and location.

prompt_template, default_prompt

Prompt of free-form OCR answers (structured=False and the retry of an empty structured answer): default, table_focused, document_focused, multilingual, form_focused, receipt_focused, handwriting_focused, code_focused, or your own prompt text (default_prompt).

table_style

"minimal_html" (default), "markdown_grid" or "styled_html", in any case, or a TableStyle; any other name is a ValueError. It applies to every format, legacy Office files included (Tables).

legibility_judge, boilerplate_judge, judge

The judge hooks, or a judge object / "typesafe" / "none"; default $DOC2MARK_JUDGE (Optional judge (TypeSafe / Jev)).

class doc2mark.UnifiedDocumentLoader(ocr_provider='openai', api_key=None, ocr_config=None, cache_dir=None, ocr_cache=None, model='gpt-5.4-mini', temperature=0, max_tokens=8192, max_workers=None, prompt_template=PromptTemplate.DEFAULT, timeout=30, max_retries=3, top_p=1.0, frequency_penalty=0.0, presence_penalty=0.0, base_url=None, project=None, location='global', default_prompt=None, task=None, structured=None, detail=None, table_style=None, legibility_judge=None, boilerplate_judge=None, judge=None)[source]

Main document loader with unified API for all formats and enhanced OCR configuration.

load() notes: ocr_images=True implies extract_images=True when an OCR provider is configured, and both flags also apply to image files; delimiter is the CSV delimiter (detected when omitted; a .tsv file is always tab separated). The batch result dictionaries are described in Loading documents.

Parameters:

  • ocr_provider (str | OCRProvider | BaseOCR | None)

  • api_key (str | None)

  • ocr_config (OCRConfig | None)

  • cache_dir (str | None)

  • ocr_cache (OCRCache | None)

  • model (str)

  • temperature (float)

  • max_tokens (int)

  • max_workers (int | None)

  • prompt_template (str | PromptTemplate)

  • timeout (int)

  • max_retries (int)

  • top_p (float)

  • frequency_penalty (float)

  • presence_penalty (float)

  • base_url (str | None)

  • project (str | None)

  • location (str)

  • default_prompt (str | None)

  • task (str | Task | None)

  • structured (bool | None)

  • detail (str | None)

  • table_style (str | TableStyle | None)

  • legibility_judge (Callable[[str], float | None] | None)

  • boilerplate_judge (Callable[[str, Dict[str, Any]], float | None] | None)

  • judge (Any)

batch_process(input_dir, output_dir=None, output_format=OutputFormat.MARKDOWN, extract_images=False, ocr_images=False, recursive=True, show_progress=True, save_files=True, encoding='utf-8', delimiter=None, max_workers=None, progress_callback=None)[source]

Batch process multiple documents in a directory with full result tracking.

Parameters:

  • input_dir (str | Path) – Directory containing documents

  • output_dir (str | Path | None) – Optional output directory (default: same as input). The input tree is mirrored under it; files that would share an output name (report.txt and report.md) are written as report.txt.md and report.md.md. Next to the inputs (the default) an output replaces the one an earlier run wrote.

  • output_format (str | OutputFormat) – Output format (MARKDOWN, JSON, TEXT)

  • extract_images (bool) – Whether to extract images from documents (Office/PDF only)

  • ocr_images (bool) – Whether to perform OCR on images (implies extract_images when an OCR provider is configured)

  • recursive (bool) – Whether to process subdirectories

  • show_progress (bool) – Whether to show progress messages

  • save_files (bool) – Whether to save output files

  • encoding (str) – Text encoding for text/markup files

  • delimiter (str | None) – CSV delimiter (auto-detect if None)

  • max_workers (int | None) – Optional number of worker threads. When set to a value > 1 (and more than one file is found), documents are processed concurrently with a thread pool. Defaults to None, which preserves the original sequential behavior.

  • progress_callback (Callable[[int, int, str], None] | None) – Optional callable invoked as (done, total, path) after each file finishes (whether it succeeded or failed). path is the string key used in the returned results dict.

Returns:

Dictionary mapping input paths to processing results

Return type:

Dict[str, Dict[str, Any]]

Examples

# Process with image extraction but no OCR
loader.batch_process("docs/", extract_images=True, ocr_images=False)

# Process with batch OCR
loader.batch_process("docs/", extract_images=True, ocr_images=True)

# Process concurrently with a progress bar
loader.batch_process("docs/", max_workers=4,
                     progress_callback=lambda d, t, p: print(f"{d}/{t}"))
batch_process_files(file_paths, output_dir=None, output_format=OutputFormat.MARKDOWN, extract_images=False, ocr_images=False, show_progress=True, save_files=True, encoding='utf-8', delimiter=None, max_workers=None, progress_callback=None)[source]

Batch process a specific list of files.

Parameters:

  • file_paths (List[str | Path]) – List of file paths to process

  • output_dir (str | Path | None) – Optional output directory (flat; files that would share an output name, such as q1.pdf of two folders, are written as q1.pdf.md and q1.pdf-2.md)

  • output_format (str | OutputFormat) – Output format (MARKDOWN, JSON, TEXT)

  • extract_images (bool) – Whether to extract images from documents (Office/PDF only)

  • ocr_images (bool) – Whether to perform OCR on images (implies extract_images when an OCR provider is configured)

  • show_progress (bool) – Whether to show progress messages

  • save_files (bool) – Whether to save output files

  • encoding (str) – Text encoding for text/markup files

  • delimiter (str | None) – CSV delimiter (auto-detect if None)

  • max_workers (int | None) – Optional number of worker threads. When set to a value > 1 (and more than one file is provided), files are processed concurrently with a thread pool. Defaults to None, which preserves the original sequential behavior.

  • progress_callback (Callable[[int, int, str], None] | None) – Optional callable invoked as (done, total, path) after each file finishes (whether it succeeded or failed). path is the string key used in the returned results dict.

Returns:

Dictionary mapping input paths to processing results

Return type:

Dict[str, Dict[str, Any]]

Examples

# Process specific files with OCR
files = ["doc1.pdf", "doc2.docx"]
loader.batch_process_files(files, extract_images=True, ocr_images=True)

# Process concurrently with a progress callback
loader.batch_process_files(files, max_workers=4,
                           progress_callback=lambda d, t, p: print(f"{d}/{t}"))
get_ocr_configuration()[source]

Get current OCR configuration summary.

Returns:

Dictionary with OCR configuration details

Return type:

Dict[str, Any]

load(file_path, output_format=OutputFormat.MARKDOWN, extract_images=False, ocr_images=False, show_progress=False, encoding='utf-8', delimiter=None)[source]

Load and process a document.

Parameters:

  • file_path (str | Path) – Path to the document

  • output_format (str | OutputFormat) – Desired output format (MARKDOWN, JSON, TEXT)

  • extract_images (bool) – Whether to extract images as base64 (Office/PDF only)

  • ocr_images (bool) – Whether to perform OCR on images (implies extract_images when an OCR provider is configured)

  • show_progress (bool) – Whether to show progress messages during processing

  • parameters (# Format-specific)

  • encoding (str) – Text encoding for text/markup files (default: ‘utf-8’)

  • delimiter (str | None) – Delimiter for CSV files (auto-detect if None)

Returns:

ProcessedDocument with content and metadata

Raises:
Return type:

ProcessedDocument

Note

  • extract_images and ocr_images only work with Office and PDF formats

  • encoding and delimiter only apply to text-based formats

For advanced OCR configuration, use the constructor parameters or update_ocr_configuration() method.

load_directory(directory, pattern='*', recursive=True, output_format=OutputFormat.MARKDOWN, **kwargs)[source]

Load all documents from a directory.

Parameters:

  • directory (str | Path) – Directory path

  • pattern (str) – Glob pattern for files

  • recursive (bool) – Whether to search recursively

  • output_format (str | OutputFormat) – Desired output format

  • **kwargs – Additional processor options

Returns:

List of processed documents

Return type:

List[ProcessedDocument]

set_ocr_provider(provider, api_key=None, config=None, ocr_cache=None)[source]

Change OCR provider.

Parameters:

  • provider (str | OCRProvider | BaseOCR | None) – New OCR provider

  • api_key (str | None) – API key for provider

  • config (OCRConfig | None) – OCR configuration

  • ocr_cache (OCRCache | None) – Optional OCR cache handler. Reuses existing handler when omitted.

property supported_formats: List[str]

Get list of supported formats.

Returns:

List of format extensions

validate_ocr_setup()[source]

Validate OCR setup and return status information.

Returns:

Dictionary with validation results

Return type:

Dict[str, Any]