Handle PDF anchor links, esp `#page=N` (currently false positives)

[gwern]

PDF readers support a variety of ways to link inside them similar to HTML pages: a foo.pdf#page=5 will link to page 5 of that PDF, or a defined anchor like a section (common in Arxiv PDFs), or comments etc. See "Parameters for Opening PDF Files". This dates back to sometime around or before 2005, and is supported by many (most?) PDF viewers, although it is not an official standard AFAICT. I find them extremely useful for discussing research papers (>170 unique instances on my site so far), to hyperlink people to the exact page with the relevant figure or table or section (just as linking to parts of HTML pages is useful).

Right now link-checker treats all such arguments as errors and invalid anchors to warn about. This is unfortunate and clutters up my results considerably with false positives - the pages are all correct, but it always treats them as errors.

It should handle PDF parameters more gracefully.

The full set of parameters is probably unnecessary (I have never seen many of them in the wild, and I doubt many readers beyond Adobe Acrobat support the more obscure ones like fdf for population form fields), but page, named anchors, zoom, and view are worth supporting. page could be checked simply by, on links which return a PDF MIME type, checking that the N is a whole number less than or equal to the number of pages in the PDF. (Any PDF tool/library should be able to tell you how many pages are in a PDF.) Alternately, if a link returns a PDF MIME type, one could just skip checking anchors entirely as being too much work to figure out how to analyze arbitrary PDFs to check anchor validity and out of scope.

[dontcallmedom]

thank you for the suggestion; I've made it so that the checker no longer reports fragments on PDF as broken. I'm letting the issue open since it may still be useful to do more validation on PDF fragments (e.g. detect invalid combination of named views - I'm pretty sure the checker isn't going to add a PDF parser to verify full validity)

[asmusf]

The link checker still (or again?) reports these as broken (in the layman's definition of "broken"). The advice to update the link is not germane.

warning Lines: 67, 106, 140, 165 https://www.unicode.org/versions/latest/ch12.pdf
Status: 302 -> 200 OK

This is a temporary redirect. Update the link if you believe it makes sense, or leave it as is.
Broken fragments:

    https://www.unicode.org/versions/latest/ch12.pdf#G664195 (lines 67, 106, 140, 165)

The Unicode Standard uses these fragment identifiers copiously as stable references to (numbered) items in the text, including not only chapters or sections, but definitions.

The archival version of the standard will always be PDF, but a new interactive version will be released in HTML - supporting the same fragments.

These links should not be changed to those that link to the full file, as it would destroy the attempted citation of a specific element in the standard. Perhaps Unicode is widely used enough and foundational enough that it would be good to make sure users of the link checker are not encouraged to change normative links.

[dontcallmedom]

@asmusf I think the specific issue about https://www.unicode.org/versions/latest/ch12.pdf#G664195 is that https://www.unicode.org/versions/latest/ch12.pdf redirects (as of today) to an HTML document (in which the said anchor don't seem to exist)

[asmusf]

@dontcallmedom The problem with waiting two months is that a different version of the Standard is now the "latest" compared to the one available when I reported the issue. So, you now do get an HTML document instead of the PDF document.

However, the fragment does exist (it is for Section 12.2, and hovering over the gray number sign will reveal it). For me this link

https://www.unicode.org/versions/latest/ch12.pdf#G664195

today does resolve to that fragment in the HTML document, where in an earlier version it would have pointed to a fragment in a PDF file.

In fact, here's what Firefox gives me when I select that section header:

<span id="G664195" class="main svelte-1e1gno3"><a href="[#G664195](view-source:https://www.unicode.org/versions/Unicode16.0.0/core-spec/chapter-12/#G664195)" title="Permalink" class="svelte-1e1gno3">#</a>12.2 Bengali (Bangla)</span>

Is that what you see as well?

The Unicode editors take great care to make sure these "G" fragments are absolutely stable across any version of the standard that contains corresponding text (and in some cases even propagate them to merged/replacement text, for better user experience).

For that reason, we need to make sure that link checkers don't recommend removing these fragment ids. But if there is an issue with them not working correctly, we need to report that to the editors (and I'll be happy to pass on any problems that I can verify). But at this point, it doesn't look like there's a problem with the file on the Unicode Server.