Skip to content

Structural chunker silently drops documents that contain no headings #149

Description

@ascherj

StructuralChunker.chunk() returns an empty list for any document without markdown headings, so the entire document is silently excluded from the RAG index instead of being chunked as a single block or falling back to another strategy.

Steps to reproduce:

from ingestion.chunking.structural_chunker import StructuralChunker
c = StructuralChunker()
print(len(c.chunk('This is a plain document with no headings at all. ' * 20, {})))
# observed: 0  (a ~1000-char document produces no chunks)

Related failing test: test_document_with_no_headings in tests/unit/test_structural_chunker.py.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinggood first issueGood for first-time contributors (same as tier-1)ingestionIngestion pipelinetier-1Starter: good for first-time contributors

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions