`sphinx_highlight.js` removes text fragment from URL

[haampie]

Describe the bug

All major browsers support highlighting with Text Fragments, which is the #:~:text=... bit in a URL:

https://www.sphinx-doc.org/en/master/usage/quickstart.html#:~:text=In%20Sphinx%20source

It highlights and scrolls to the first match on the page, which is extremely useful. It's used by Google Search to scroll to a particular part of the page, and by LLMs to link to a relevant bit of the page they got their answers from, which is super useful these days.

Annoyingly, Sphinx's sphinx_highlight.js always removes this text fragment from the URL, making sharing of the exact URL the user found impossible.

How to Reproduce

Go to

https://www.sphinx-doc.org/en/master/usage/quickstart.html#:~:text=In%20Sphinx%20source

Notice that the URL bar is immediately replaced with

https://www.sphinx-doc.org/en/master/usage/quickstart.html

instead of the original

https://www.sphinx-doc.org/en/master/usage/quickstart.html#:~:text=In%20Sphinx%20source

Presumably this is due to

sphinx/sphinx/themes/basic/static/sphinx_highlight.js

Line 84 in eaebbec

window.history.replaceState({}, "", url);

which is run unconditionally from

sphinx/sphinx/themes/basic/static/sphinx_highlight.js

Line 153 in eaebbec

if (typeof Search === "undefined") SphinxHighlight.highlightSearchWords();

[haampie]

@mgeier / @AA-Turner from your conversation here: #10854 (comment)

It seems like ?highlight=... support was kept only so that you could manually add it to a URL. (Confusingly though, it's also automatically removed from the URL, so it's hard to "test" whether the right terms are highlighted if the intent is to manually create a link with highlighting for sharing... sharing those links is the sole purpose, cause otherwise I can just use Ctrl-F)

Would you agree that native browser support for Text Fragments has made that redundant? You can now add #:~:text=... instead, which has the following upsides:

1. faster than the JavaScript loop over the entire DOM (highlight delay is noticeable for me)
2. scrolls the first match into view on page load
3. you can specify a range foo,bar and it'll highlight the sentence foo ... bar
4. supports prefix/suffix for context without highlighting those

Only downside I see is that it highlights the first match only.

Regarding #10201 (comment), this is now the case.

[jayaddison]

It seems like ?highlight=... support was kept only so that you could manually add it to a URL. (Confusingly though, it's also automatically removed from the URL, so it's hard to "test" whether the right terms are highlighted if the intent is to manually create a link with highlighting for sharing... sharing those links is the sole purpose, cause otherwise I can just use Ctrl-F)

Thanks for the report/suggestion. As far as I know, the ?hightlight= URL parameter functionality exists so that after the user performs a search query using Sphinx, it's possible to highlight relevant fragments of the documents that appear in the search results.

The #:~:text= text anchoring I do have some sympathy for, but -- personal opinion from here -- the mechanism seems fragile in an information architectural sense, unlikely to be well-suited to internationalisation (I admit target text may have to be translated; however, well-chosen anchor labels may not have to be), and in a human communication theoretic sense, no replacement for explaining -- with some perhaps necessary redundancy -- to your communication partner where exactly it is within a text to find the topical information that they can seek, in order that they themselves expend some of the requisite learning effort to discover the nugget of discussion.

I wasn't initially able to find a standards document to describe the text anchoring feature, but would be glad to read about it (either via a text anchor link, or a standard hyperlink target.. :)) if you are able to share one.

[jayaddison]

Hmmm.. I think I misunderstood the key part of the request; it is to allow the text anchor / fragment part of the URL to remain unmodified in the browser address bar (window.location, I think). I'll re-read the description again when I feel less tired and have more focus; my apologies @haampie.

[jayaddison]

(if possible, I think we should also scour the code and issuetracker history to examine why the URL is cleared on page load -- maybe there will be some additional relevant factors from that that we should consider; I haven't done that yet but will do soon)

[jayaddison]

Self-quoting:

I wasn't initially able to find a standards document to describe the text anchoring feature, but would be glad to read about it (either via a text anchor link, or a standard hyperlink target.. :)) if you are able to share one.

Although I admit this may be off-topic: I had been web-searching for a WHATWG spec; in fact it seems that the relevant feature, described as Scroll-to-Text Fragment, is a Web Platform Incubator Community Group spec. I don't completely understand how that group relates to the WHATWG or W3C, but I am aware that they are different groups. The spec itself can be found at: https://wicg.github.io/scroll-to-text-fragment/

In addition, I have read opinions that under some circumstances and for certain groups, text fragment navigation can be an accessibility aid. I am trying to weigh that up against various other aspects of the feature; I consider myself easily distracted, not always a complete document/paragraph reader, and perhaps with some reading comprehension quirks -- all fairly normal human behaviours I think -- and yet I personally do not like the idea of text-fragment navigation from a technical, information organisational and syntactic perspective. However I should give it more of a chance and read some of the relevant issuetracker backlog.

[haampie]

it is to allow the text anchor / fragment part of the URL to remain unmodified in the browser address bar

Yeah. Solved either by #13918 or #13921. My preference is the second solution, since ?highlight= is imo redundant now that text fragments are supported natively by all major browsers.

I feel like the rest of your comments are somewhat off topic.

[jayaddison]

Responding out-of-order:

I feel like the rest of your comments are somewhat off topic.

They may be; my personal feeling is that I would reluctantly-if-ever share a scroll-to-text hyperlink with someone, because I feel it's a potential sign of disrespect and/or inadequetely-structured reference documentation.

Instead I would link to the nearest anchored section heading -- that I would hope the documentation authors would have prepared thoughtfully -- and if I felt it was necessary I would also add some supporting context in my message to explain what I'm sharing and where/what to read about it after following the hyperlink. I feel that that would provide a better preparation and outcome for the reader than a longer hyperlink with a fragment of text embedded in it, and presents some proof-of-effort.

I don't want to amplify or infer acceptance of features that I consider could be disrespectful of people's time and effort by supporting them in Sphinx -- so my comments are to try to explain some of the factors I'm considering.

Yeah. Solved either by #13918 or #13921. My preference is the second solution, since ?highlight= is imo redundant now that text fragments are supported natively by all major browsers.

Thanks - my thoughts on #13921 in particular will depend on how I feel about scroll-to-text more generally.

I'm only one (and non-core) reviewer, so I'll step aside if I'm being obstructive.