DocumentParser

DocumentParser holds a fixed ParseConfig and applies it to every document you parse with it. Beyond saving you from re-passing options, it owns two resources that a bare parse_document call cannot keep alive between documents:

• One browser, reused. Launching Chromium is the dominant fixed cost of HTML extraction. A DocumentParser launches it once, lazily, and reuses it across every HTML document in the batch.
• Document-level parallelism. Pass workers=N to parse whole documents across N processes. Works for every format.

from docslicer import DocumentParser, ParseConfig
 
with DocumentParser(ParseConfig(max_chunk_size=2000), workers=4) as parser:
    for source, result in parser.parse_all(paths):
        ...

For a single document, reach for parse_document — there is nothing to amortize.

Constructor

DocumentParser(config: ParseConfig | None = None, workers: int | None = None)

workers vs config.max_workers

These are different axes and are easy to confuse. max_workers splits one document across cores; workers splits a batch across documents.

They compose, so they can also oversubscribe: N documents in flight × P workers each. DocSlicer guards the common case — when you set workers and leave config.max_workers at None, each worker's intra-document width is forced to 1 so the outer fan-out owns the concurrency. Set max_workers explicitly and that guard steps aside, on the assumption you meant it.

A batch of large PDFs is usually fastest with workers=1 and intra-document parallelism doing the work. A batch of many small documents — or of DOCX/PPTX/HTML, which cannot parallelize internally — wants workers=N.

Lifecycle

The browser that makes DocumentParser fast is a real Chromium process, and nothing shuts it down automatically — it stays resident until your program exits. Use the parser as a context manager and it's closed for you:

with DocumentParser(config) as parser:
    result = parser.parse("https://example.com/filing")
# browser closed here

That covers almost every case. The exception is a parser that outlives a single block — held on app state, or kept around in a notebook kernel — where the browser stays resident for the life of the process; call parser.close() at shutdown.

The session launches lazily, so a parser that only ever sees PDFs never starts Chromium and costs nothing to hold.

ParseConfig

ParseConfig bundles the parsing options into one reusable object. Its fields are exactly the keyword arguments parse_document accepts (minus source, source_url, and on_stage), so see that page's parameter table for what each one does.

from docslicer import ParseConfig
 
ParseConfig(
    max_chunk_size=3200,
    optimal_chunk_size=1500,
    min_chunk_size=700,
    chunking=True,
    merge_small_chunks=True,
    exact_tokens=False,
    table_representation="markdown",
    debug=False,
    extra_fields=[],
    password=None,
    max_workers=None,
    use_browser=True,
    include_speaker_notes=True,
    include_headers_footers=False,
    include_footnotes=True,
    include_comments=False,
)

It carries one field parse_document does not expose: run_id, an auto-generated UUID identifying the parse run, surfaced on result.metadata. Leave it unset unless you need to correlate a result with your own logs.

Construction validates its arguments and raises ValueError on invalid combinations — an unknown table_representation, a non-positive chunk size, or sizes that don't satisfy min <= optimal <= max. You get the error when you build the config, not midway through a batch.

Because it's a plain dataclass, its fields are readable and assignable after construction:

config = ParseConfig(exact_tokens=True)
print(config.exact_tokens)     # True
config.max_chunk_size = 2500   # adjust before building the parser
parser = DocumentParser(config)

import dataclasses
config = dataclasses.replace(config, table_representation="jsonl")  # validated

Methods

parse(source, source_url=None, on_stage=None)

Parse a single document, reusing this instance's config and browser. Returns a ParseResult.

result = parser.parse("report.pdf")

Unlike parse_all, parse raises on failure rather than returning the exception.

parse_all(sources)

Parse a list of documents, reusing this instance's config. Yields (source, result) pairs — a ParseResult on success, or the Exception on failure, so a single bad file never aborts the batch.

for source, result in parser.parse_all(["a.pdf", "b.docx", "c.html"]):
    if isinstance(result, Exception):
        print(f"Failed {source}: {result}")
    else:
        print(f"{source}: {len(result.chunks)} chunks")

Its behaviour differs meaningfully depending on workers:

The browser point is the one that bites: worker processes can't inherit a live Playwright browser, so each builds its own. workers=8 over a batch of URLs means eight Chromium processes, which is a real memory footprint. Size workers against RAM, not just cores, when the batch is HTML.

Examples

Reuse one config across a batch

from docslicer import DocumentParser, ParseConfig
 
parser = DocumentParser(ParseConfig(
    max_chunk_size=2000,
    optimal_chunk_size=800,
    table_representation="markdown",
))
 
for source, result in parser.parse_all(["q1.pdf", "q2.pdf", "q3.pdf"]):
    if not isinstance(result, Exception):
        print(source, len(result.chunks))

Scrape many URLs with one browser

The single biggest win: Chromium launches once for the whole batch instead of once per page.

urls = ["https://example.com/a", "https://example.com/b", "https://example.com/c"]
 
with DocumentParser() as parser:
    for url, result in parser.parse_all(urls):
        if not isinstance(result, Exception):
            print(url, len(result.chunks))

Fan a mixed batch across processes

with DocumentParser(ParseConfig(max_chunk_size=2000), workers=4) as parser:
    for source, result in parser.parse_all(paths):
        if isinstance(result, Exception):
            print(f"Failed {source}: {result}")

Each worker parses whole documents single-process, since max_workers was left at None.

Track progress on a long document

with DocumentParser() as parser:
    result = parser.parse("annual-report.pdf", on_stage=lambda s: print(f"→ {s}"))

Default config

parser = DocumentParser()          # same as parse_document with defaults
result = parser.parse("report.pdf")

When to use which

• One document, one-off? parse_document.
• Many documents, same settings? DocumentParser — fix the config once, call .parse() per document.
• Many HTML pages or URLs? DocumentParser, always — one browser instead of one per page.
• Many DOCX, PPTX, or HTML documents? DocumentParser(workers=N) — these formats gain nothing from max_workers, so document-level fan-out is what puts your cores to work.
• Many small PDFs? DocumentParser(workers=N) too — each is below the intra-document page thresholds, so the batch is where the parallelism has to come from.
• A few large PDFs? DocumentParser with default workers — intra-document parallelism already uses your cores.
• A whole folder? The module-level parse_all — it discovers files and parses them in one call.