parse_all

Parse every document in a folder or a list of sources. Yields (source, result) pairs and never aborts on a failed file — errors are returned alongside successes so the rest of the batch keeps running.

Signature

docslicer.parse_all(
    source,
    recursive=False,
    **kwargs,
) -> Iterator[tuple[source, ParseResult | Exception]]

Parameters

Parameter	Type	Default	Description
`source`	`str \| Path \| list`	—	A folder path to scan, or a list of individual sources (paths, URLs, bytes).
`recursive`	`bool`	`False`	When `source` is a folder, scan subdirectories too. Has no effect when `source` is a list.
`**kwargs`			Forwarded to `parse_document` for every file (`max_chunk_size`, `chunking`, `table_representation`, etc.).

Yields

(source, ParseResult) on success, (source, Exception) on failure. Always check with isinstance(result, Exception) before using the result.

Supported extensions

When scanning a folder, only files with these extensions are picked up: .pdf, .docx, .pptx, .html, .htm, .xhtml. Files are yielded in alphabetical order.

Examples

Scan a folder

for path, result in docslicer.parse_all("documents/"):
    if isinstance(result, Exception):
        print(f"failed {path.name}: {result}")
    else:
        print(f"{path.name}: {len(result.chunks)} chunks")

Scan recursively

for path, result in docslicer.parse_all("documents/", recursive=True):
    if isinstance(result, Exception):
        print(f"failed {path}: {result}")
    else:
        print(f"{path}: {len(result.chunks)} chunks")

Pass a list of sources

sources = [
    "filings/annual_report.pdf",
    "filings/presentation.pptx",
    "https://example.com/press-release",
]
 
for source, result in docslicer.parse_all(sources):
    if isinstance(result, Exception):
        print(f"failed {source}: {result}")
    else:
        print(f"{source}: {len(result.chunks)} chunks")

Collect successful results

results = {
    path: result
    for path, result in docslicer.parse_all("documents/")
    if not isinstance(result, Exception)
}

Forward chunk size options

for path, result in docslicer.parse_all(
    "documents/",
    recursive=True,
    max_chunk_size=2000,
    optimal_chunk_size=800,
):
    ...

Returns

parse_all is a lazy generator — files are parsed one at a time as you iterate. Nothing is buffered in memory upfront.

For reusable config across a batch, see DocumentParser.