Skip to content

DESIGN.md

Latest commit

 

History

History
47 lines (34 loc) · 4.46 KB

DESIGN.md

File metadata and controls

47 lines (34 loc) · 4.46 KB
Raw

pdf-go core design

Goals

Implement core PDF capabilities in Go under github.com/lightningrag/pdf-go: file structure parsing, generic objects, page traversal, and (incrementally) writing and transforms.

Package layout

Go Role
pdf Entry types: PdfReader, PdfWriter, constants, errors; page-range parsing
pdf/generic PDF syntax-layer objects and read/write
pdf/filters Stream filters
(future) pdf/annotations Annotation subtypes, etc.

Integration tests may use PDF fixtures supplied via environment or relative paths (see candidate roots and skip logic in pdf package tests); missing fixtures cause the relevant tests to Skip.

Reader pipeline

  1. Buffering: read the whole file into []byte for xref handling and SectionReader slices.
  2. startxref: scan backward from EOF for startxref and its offset.
  3. xref chain: walk from the newest segment via /Prev; supports:
    • classic xref table + trailer dictionary;
    • PDF 1.5+ xref stream (/Type /XRef) with /W, /Index, /Size;
    • xref stream decompression with /FlateDecode + PNG predictor (e.g. 12) in /DecodeParms (important for this stage).
  4. Trailer merge: keys are written only when missing (common semantics for incremental updates: existing keys win).
  5. Object parsing: generic.ReadObject; indirect refs resolved via generic.PDF GetObject callback.
  6. ObjStm: /Type /ObjStm decoded with the same decodeStreamData, then parse N/First index; prefer the object index inside the stream (third column of /W type-2 entries in xref) to locate object bodies instead of scanning every object in the stream (fallback to lookup by number when index and object numbers disagree).
  7. Page flattening: recurse /Root → /Pages → /Kids, merging inheritable keys (/Resources, /MediaBox, /CropBox, /Rotate, /ArtBox, /TrimBox, /BleedBox, /UserUnit) consistent with common reader page-tree flattening.

Writer pipeline

  • Maintain a linear objects []generic.Object (1-based numbers = slice index + 1).
  • Emit %PDF-1.7, each n 0 obj block, xref, trailer, startxref, %%EOF; trailer includes /ID (two 16-byte random IDs) by default, optional OmitTrailerDocumentID for tests/deterministic output. Bytes() validates at least one object and that the last object is /Catalog, otherwise returns a clear error.
  • xref offsets: object offsets start after %PDF-1.7\n (per spec).
  • AppendPagesFromReader / AppendPagesFromReaderPageRange: deep-clone flat page dicts from PdfReader in clone.go (seen dedupes indirects). Empty writer creates /Pages and /Catalog; non-empty writer requires the last object to be /Catalog, /Pages as indirect or inline dict (inline is promoted with AddObject and catalog /Pages updated), new pages merged into /Kids, catalog moved back to end of object table to satisfy write conventions. PageRange is turned into index lists via PageIndices(n) on the same path.

Testing strategy

  • Resource fixtures: integration tests run when present (relative paths in reader_test.go); otherwise Skip.
  • assets/example.pdf: TestAssetsExamplePDF checks the sample PDF shipped with the repo.
  • Manifest-driven samples: TestSampleFilesManifestAgainstReader aligns with external files.json; exceptions and xref policy in docs/SAMPLE_FILES_TESTING.md, quick reference docs/TESTING.md.
  • TestWriterMinimalRoundtrip: hand-written minimal page graph, read/write roundtrip checks NumPages()==1.

Suggested extension order

Not yet implemented: encryption/decryption (password verify + RC4/AES object crypto); incremental writing (write_stream / _write_increment); layout-mode text extraction (CMap-level mapping exists, full layout needs font metrics and text space); RTL text ordering; image pixel extraction (decode compressed image bytes to raster); form appearance stream generation (TextStreamAppearance: full text-field appearance with metrics and wrapping); cross-document outlines resolving remote /D page indices after opening external PDFs; xref rebuild for damaged PDFs (full-file scan for obj keywords, etc.). (Partially supported: /Link and outlines with /URI, /Launch, in-document GoTo/Dest, GoToR/Launch /F paths (Link.uri / OutlineNode.URI/RemotePath; remote page index -1 when external files are not opened).)