Skip to content

feat: add resilient EZProxy-only institutional downloads - #37

Open
Grenzlinie wants to merge 3 commits into
Rimagination:masterfrom
Grenzlinie:codex/upstream-ezproxy
Open

feat: add resilient EZProxy-only institutional downloads#37
Grenzlinie wants to merge 3 commits into
Rimagination:masterfrom
Grenzlinie:codex/upstream-ezproxy

Conversation

@Grenzlinie

Copy link
Copy Markdown

This adds a configurable EZProxy-only path for users with institutional access.

Why

Existing EZProxy attempts can lose authenticated browser state or exit before users finish publisher verification, causing valid institutional access to be reported as a failed download.

What changed

  • Adds --ezproxy-only / strategy=ezproxy_only and a configurable ezproxy_login_url.
  • Preserves the visible browser session across login redirects, loading pages, and user-completed verification.
  • Resolves publisher PDF entry points within the authenticated browser context, then captures or fetches a validated PDF with that session's cookies.
  • Stores refreshed local cookies atomically; POSIX sessions receive 0600 file permissions.
  • Adds a concise generic README workflow, with HKU shown only as a configuration example.

No institution credentials, cookies, signed URLs, PDFs, or live-access tests are included.

Validation

  • pytest tests/ -v --tb=short — 63 passed
  • Covers browser-session redirects, loading and verification recovery, publisher resolvers, cookie refresh, terminal timeout behavior, and --ezproxy-only source selection.
  • CI installs PyMuPDF because the existing PDF extraction module imports it during test collection.

@Grenzlinie
Grenzlinie marked this pull request as ready for review July 20, 2026 13:03
@Grenzlinie

Copy link
Copy Markdown
Author

作者您好,我在使用的过程中发现对于EZProxy类的文献下载有以下一些问题:

  1. 一些学校,例如HKU,在输入账号密码登录proxy后,网页会自动把原始url改成proxy之后的url然后跳转,这种情况下原来的EZProxy代码会因为页面变化记录不到登录态cookies,然后一直在等待直到超时。
  2. 在修复上面的问题并用EZproxy方式登录之后,虽然进入到了全文html页面,但是学校经常会需要反爬认证(比如Processing Verifaction需要点一下),但是scansci-pdf 1.9版本会到了反爬页面直接退出,或点击pdf后,由于是在浏览器的pdf预览器打开,会导致获取不到全文pdf的下载按钮,无法下载。

因此我开发了一些代码来fix上面的问题,并在闭源期刊(例如Acta Materialia)上进行了测试,希望能对项目的功能上有一些帮助。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant