This repository was archived by the owner on Jun 15, 2021. It is now read-only.

This repository was archived by the owner on Jun 15, 2021. It is now read-only.

DocumentTokens: incorrect limit when stopList is used #22

Open

opened

on Feb 15, 2021

Number of tokens returned will be likely less than the limit when a stopList is used.
The tool uses int maxPos = documentStart+limit to determine when to stop searching, however this needs to be updated when one of the tokens in the stopList is encountered.

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests