parse_document

The primary entry point for parsing documents. Pass it a file path, URL, bytes, or file-like object and it returns a ParseResult — no need to branch on format.

Signature

docslicer.parse_document(
    source,
    source_url=None,
    max_chunk_size=3200,
    optimal_chunk_size=1500,
    min_chunk_size=700,
    chunking=True,
    merge_small_chunks=True,
    table_representation="markdown",
    exact_tokens=False,
    debug=False,
    extra_fields=None,
    password=None,
    max_workers=None,
    use_browser=True,
    include_headers_footers=False,
    include_footnotes=True,
    include_comments=False,
    include_speaker_notes=True,
    on_stage=None,
) -> ParseResult

Parameters

Format detection

The format is resolved in this order:

1. Extension — .pdf, .docx, .pptx, .html, .htm, .xhtml
2. Magic bytes — PDF starts with %PDF; DOCX/PPTX detected from ZIP contents
3. URL — fetched first, then the delivered content type determines the parser
4. Fallback — anything else is treated as HTML

Examples

File paths

import docslicer
 
result = docslicer.parse_document("report.pdf")
result = docslicer.parse_document("report.docx")
result = docslicer.parse_document("slides.pptx")
result = docslicer.parse_document("page.html")
 
for chunk in result.chunks:
    print(chunk.text)

Fine as-is in a REPL or notebook. If you save this as a .py script on macOS or Windows and parse large or scanned PDFs, guard the entry point with if __name__ == "__main__": — see intra-document parallelism for why.

# Install LibreOffice (macOS)
brew install --cask libreoffice
 
# Convert to docx / pptx
libreoffice --headless --convert-to docx report.doc
libreoffice --headless --convert-to pptx slides.ppt

URLs

The URL is fetched once and the delivered content type determines which parser runs — a URL ending in .pdf that actually serves a PDF will parse as PDF, not HTML:

result = docslicer.parse_document("https://example.com/annual-report")
result = docslicer.parse_document("https://example.com/filing.pdf")

Raw bytes

with open("contract.pdf", "rb") as f:
    result = docslicer.parse_document(f)

Tune chunk sizes

result = docslicer.parse_document(
    "report.pdf",
    max_chunk_size=2000,
    optimal_chunk_size=800,
    min_chunk_size=400,
)

Control intra-document parallelism (PDF)

Text extraction dominates PDF parsing time, and pages are independent, so the PDF pipeline distributes them across a process pool by default. No opt-in is required, and the default is appropriate for most workloads. DOCX, PPTX, and HTML are parsed single-process; max_workers is accepted for those formats but has no effect.

Each stage is gated by document size. A process pool has fixed costs — process spawn, an interpreter and pandas import per worker, pickling work in and results out — that outweigh the speedup on small documents. Every page-parallel stage therefore has its own page threshold, evaluated before the worker count is resolved: below the threshold the stage runs single-process regardless of max_workers.

How the default width is chosen. With max_workers=None, DocSlicer counts performance cores, not logical CPUs: P-cores via hw.perflevel0.physicalcpu on Apple Silicon, unique physical cores from /sys topology on Linux, GetLogicalProcessorInformationEx on Windows. Efficiency cores and SMT/hyper-threading siblings are deliberately excluded — they share execution resources, so counting them over-commits compute-bound work and can make parsing slower. Detection is best-effort and zero-dependency; every probe falls back to os.cpu_count().

On Linux the count is also capped by the scheduler affinity mask. That covers cpuset pinning — docker run --cpuset-cpus, taskset, Kubernetes' static CPU-manager policy — where the process genuinely sees fewer cores. It does not cover CFS quota, which is how docker run --cpus=2, ECS task CPU units, and Kubernetes limits.cpu throttle by default. A quota caps how much CPU time you get, but the container still reports every core on the host, so detection has nothing to key off. See below.

# Override the inferred width
result = docslicer.parse_document("report.pdf", max_workers=4)
 
# Force single-process
result = docslicer.parse_document("report.pdf", max_workers=1)

When you should override it

The common thread: DocSlicer sizes the pool to the cores it can see, which is not always the CPU you are entitled to, and not always CPU you want it to take.

CPU-quota'd containers — ECS, Cloud Run, Kubernetes limits.cpu, docker --cpus. As above, a quota throttles CPU time without shrinking the visible core count, so on a 32-core host with a 2-CPU quota DocSlicer will happily start 16 workers. You get all the pool's costs — spawn, per-worker pandas import, 16 copies of the PDF in RAM — while the cgroup throttles you back to 2 CPUs of actual throughput. Set it to your quota:

# Match the CPU limit the orchestrator actually granted you
result = docslicer.parse_document("report.pdf", max_workers=2)

Cpuset-pinned containers (--cpuset-cpus, k8s static CPU-manager policy) don't need this — affinity detection already handles them.

Serverless. On AWS Lambda vCPU scales with configured memory (roughly one vCPU at 1,769 MB), so a function below that gets a fraction of a core and cannot absorb pool overhead. Memory is the sharper constraint: each worker is a real process with its own interpreter, its own pandas import, and its own pickled copy of the PDF bytes, so peak RSS scales with worker count. On a memory-capped function that is a plausible OOM, not just a slowdown.

result = docslicer.parse_document(pdf_bytes, max_workers=1)

Shared web servers. Under gunicorn/uvicorn you already run several worker processes, so a per-request pool multiplies out to requests in flight × P. Even at low traffic a compute pool competes with request handling and inflates tail latency for every other caller. Keep parsing single-process and let the server own concurrency — or move parsing to a background queue, where a larger max_workers is appropriate again.

You already parallelise at the document level. Fanning documents out yourself nests pools — N documents × P workers each. Set max_workers=1 so your outer pool owns the concurrency. DocumentParser(config, workers=N) does this for you: when you set workers and haven't chosen a max_workers, each worker's own width defaults to 1.

@app.task
def parse_task(path):
    return docslicer.parse_document(path, max_workers=1)

if __name__ == "__main__":
    result = docslicer.parse_document("report.pdf")

Skip chunking to get raw blocks only

result = docslicer.parse_document("report.pdf", chunking=False)
for block in result.blocks:
    print(block.text)

Get exact token counts

result = docslicer.parse_document("contract.pdf", exact_tokens=True)
print(result.metadata.token_count)        # document total
for chunk in result.chunks:
    print(chunk.token_count)

Attach extra fields

result = docslicer.parse_document(
    "contract.pdf",
    extra_fields=["is_bold", "font_name", "font_size"],
)
for block in result.blocks:
    print(block.extra["font_name"], block.text[:60])

Returns

A ParseResult.

Format-specific parsers

If you need parsing to fail when the file doesn't match an expected format, use the format-specific functions directly. See Format parsers.

For batches, see parse_all.