Downloading sets of references for offline access

[helpatbbf]

Description

https://authors.ietf.org/en/references-in-rfcxml says:

For RFCs, Internet-Drafts, and documents produced by the W3C, 3GPP, IANA and NIST, the new site is bib.ietf.org (at the time of writing, this site has only just gone live and some usability issues are still to be resolved). Many of these sets of references can be downloaded as a set for offline access.

Please can you indicate how to download sets of references for offline access?

Background: I used to use rsync to maintain a local copy of the xml2rfc.bibxml tree but rsync access stopped working a while ago. I have looked at the new API (although not recently) and don't recall seeing an obvious replacement for my old rsync method. I hope that I'm just missing something.

Code of Conduct

• I agree to follow the IETF's Code of Conduct

[wlupton]

Sorry, I created this using the wrong username. I should have used my wlupton username.

[rjsparks]

This is a known deficiency.
There isn't a way right now to get all the datasets (and for some, the dataset isn't enumerable).
You can get the datatracker's input into the bibxml service from rsync.ietf.org::bobxml-ids, covering Internet-Drafts.

[wlupton]

Thanks. I used to use this command:

rsync --archive --verbose --compress \
    --exclude='.*index.xml.*' --exclude='rdf/' --exclude='*.tgz' \
    --delete-excluded rsync.ietf.org::xml2rfc.bibxml xml2rfc.bibxml/

...which now just downloads this 00readme.txt:

As of 2024-09-10, the xml2rfc citation libraries are not currently
available at this location.

Due to a deficiency in the bib.ietf.org service, the libraries
that were available here had not been updated since 2022. We are
working to repair the deficiency and will restore access to those
files via this rsync location when possible.

We apologize for any inconvenience.

When you said this:

You can get the datatracker's input into the bibxml service from rsync.ietf.org::bobxml-ids, covering Internet-Drafts.

...did you mean that this is the only bibxml data that is currently available via rsync? I verified that rsync.ietf.org::bibxml-rfcs is not available but perhaps some others are?

Is there any plan to restore rsync access to bibxml data?

[rjsparks]

Yes, that is the only dataset available by rsync and note that it is the raw input to the bib service from the datatracker and will differ in minor ways from what an include from bib.ietf.org/public/rfc/bibxml3 will return.

Yes there is intent to restore rsync for those datasets that can be enumerated (we will not provide an rsync for references for all possible DOI for example), but that is not likely to happen soon.

[cabo]

Partial workaround: kramdown-rfc includes the command kramdown-rfc-cache-i-d-bibxml, which caches reference info for all active I-Ds into ~/.cache/xml2rfc (which you then can use with xml2rfc and via KRAMDOWN_REFCACHEDIR with kramdown-rfc).
This is around 2000 to 3000 reference files.
Caching reference info for all 150 000 I-Ds (including expired/replaced/superseded) is maybe a bit heavy.
(There also is "broken reference" support in kramdown-rfc, so just go forth and reference...)