Skip to content

Decide how the three legacy binary document formats are read, or that they never are #1680

Description

@Krzysztof318

Context

#1550 delivered IAttachmentTextExtractor and reads four formats: PDF, .docx, .xlsx, and .pptx. The three legacy binary formats its acceptance also named — .doc, .xls, .ppt — are recognized and deliberately not parsed. They answer FormatNotExtracted, which tells a mailbox owner their file was skipped rather than leaving them to conclude it was searched and empty.

The reason is licensing rather than difficulty. All three are OLE compound files, and no permissively licensed .NET parser reads all three:

  • NPOI 2.8.0 declares a file licence (OSMFEULA.txt) with requireLicenseAcceptance: true, which is not a permissive expression and would need the owner's explicit approval under the acceptance policy in $check-docs-licenses.
  • NPOI 2.7.6 is Apache-2.0, but it pulls in about eleven transitive packages, and unpacking npoi.2.7.6.nupkg and reading its XML documentation confirms it ships no NPOI.HWPF and no HSLF — so it would have added .xls alone, at the cost of that whole closure.

So the choice is between a licence decision, a much larger dependency closure for one of the three formats, or a different approach entirely — a converter invoked out of process, or leaving the three permanently unread and saying so.

That decision is the point of this issue. It is not a defect in #1550: the behaviour is deliberate, and it is documented in docs/features/attachment-text-extraction.md, in docs/operations/configuration-ai.md, and in IAttachmentTextExtractor's own remarks.

User stories

  • As a mailbox owner who is sent contracts as .doc files by a correspondent whose office suite has not moved in fifteen years, I want those documents searchable by what they say, so that the age of a sender's software does not decide whether my own archive is searchable.
  • As the owner of a deployment, I want the licence terms of any parser that reads a stranger's bytes to be terms I can live with, so that adding attachment coverage never quietly ends the closed-source arm of the acceptance policy.

Acceptance

  • A decision is recorded on how, or whether, the three legacy binary formats are read — including "not at all, permanently", which is a valid outcome and would close this issue by making the current behaviour the settled one rather than an interim one.
  • If a parser is adopted, its licence, its transitive closure, and its .NET 10 compatibility are reviewed and recorded in THIRD_PARTY_LICENSES.md before any code uses it, and a non-permissive expression carries the owner's explicit approval.
  • If a parser is adopted, every bound AttachmentTextExtractionOptions already declares applies to it unchanged — input size, extracted characters, element depth where the format has one, and the timeout — and the OLE compound-file container gets whatever bound its own structure needs, stated the same way the archive bounds are.
  • Nothing executes a macro, an embedded object, or any other active content a legacy binary format can carry. .doc and .xls in particular are the historical macro-delivery formats, so the nothing is executed posture in docs/features/attachment-text-extraction.md is the part of the contract this issue may not weaken.
  • Whatever is decided, AttachmentDocumentFormats.Extracted, the Embeddings:AttachmentText:Formats validation, the feature page, and the configuration page agree with it.

Dependencies

Follows #1550, which is where the port, the ceilings, and the three formats' recognition already landed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendWork in the .NET stack under backend/securityNeeds a security review before it mergestype:decisionBlocked on a decision or a measurement, not on effort; may require an ADR

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions