[feature request] Add caching capabilities to cache successful links for some time

[boillodmanuel]

hi,
I use markdown-link-check and thanks for it 👍

This would be nice to add a caching capabilities in order to cache successful links for some time.

This will make the checking faster and also more reliable (as some links could timeout sometimes).

[NicolasMassart]

Doesn't it cancel the interest of testing links? What if the link is broken and your cache says it's not? How long would you like your cache to be retained?
If it's about links being used in the same project in multiple places, a unicity check at the site level would be probably better and would be ok as only for a run. In this case, having this done at marksdown-link-checker level seems better to me.
Let me know if I misunderstand your need.

[boillodmanuel]

I will explicit my use-case.

I have a wiki on gitlab, and I want to check that links are not broken after each page update to detect this as soon as possible.

I would say that 50 pages are updated per day.
One link-check run of the whole wiki lasts about 5 min.

Using a cache of working links for 1 day (or more) would save a lot of time.

• A working link is broken if the website is down or if the page has moved (not so often)
• A new link could be broken for lot of reason, but mainly because it's simply wrong

Using a cache will speed up detection of new link that are wrong, whereas all links could be check only once a day.

[NicolasMassart]

Hum interesting. Let me sleep on it a bit, I have to run the scenarios in my head...
@tcort any opinion on this one?

[tcort]

@tcort any opinion on this one?

I'm open to a pull request which would implement some form of RFC 7234 compliant HTTP caching.

[tcort]

I'm considering switching from needle to got which supports caching and a ton of other features.