fix reflink, blockquote, and list

[UziTech]

feat: add original markdown to fix reflink

fixes #4

feat: add position for each line

fixes #5
fixes #6

this requires a fix to marked: markedjs/marked#3735

[gemini-code-assist]

Summary of Changes

Hello @UziTech, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant improvement to how token positions are calculated, particularly addressing issues with reflinks and blockquotes by utilizing the original markdown string. This ensures more accurate and reliable position data for all parsed tokens, enhancing the utility of the marked-token-position library.

Highlights

• Core Logic Enhancement: The addTokenPositions function now accepts an optional markdown string. This allows the function to use the original full markdown content for more accurate position calculations, especially for cases like reflinks where token.raw might not represent the full original text.
• Marked Extension Integration: The markedTokenPosition extension has been updated to leverage the preprocess hook. This hook captures the original markdown string before tokenization, ensuring that the processAllTokens hook can then pass this original markdown to addTokenPositions for precise position mapping.
• Blockquote Position Correction: A specific refinement was added to the recursive addPosition function to correctly handle blockquote tokens. For blockquotes, the leading > characters are now stripped from the raw content when calculating positions for nested tokens, preventing incorrect offset calculations within these structures.
• Improved Test Coverage: New helper functions (getLineColumn, checkPositions) were introduced in the test suite to validate the accuracy of line and column numbers in the token positions. Existing tests were updated to utilize these new checks, enhancing the robustness of the position tracking validation.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates the library to accept the original markdown string when calculating token positions, which is necessary to correctly handle reference-style links. The marked extension is updated to use a preprocess hook to capture this original markdown. There are potential issues related to a disabled test case for reference links and a possible race condition due to the use of a shared closure variable for state.