Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
30ab2c5
fix(browser): reject sync Playwright inside event loops
jung233 Jul 17, 2026
598aca6
fix(container): install CloakBrowser and Chromium runtime dependencies
jung233 Jul 16, 2026
c49176b
fix(browser): force headless mode when no display is available
jung233 Jul 16, 2026
b0f0a44
feat(browser): provide persistent contexts for login sessions
jung233 Jul 16, 2026
07f9f26
fix(browser): enforce thread ownership and release driver resources
jung233 Jul 16, 2026
c0c1313
fix(browser): close login and cookie capture sessions deterministically
jung233 Jul 16, 2026
cfce925
Add missing is_suspicious_pdf and suspicious_pdf functions
jung233 Jun 26, 2026
87669e8
fix(pdf): close PyMuPDF documents after extraction
jung233 Jul 16, 2026
f86514c
fix(pdf): publish validated downloads atomically
jung233 Jul 16, 2026
828ecfb
fix(container): reap children and bound allocator arenas
jung233 Jul 16, 2026
4e942db
fix(auth): release institutional browser sessions
jung233 Jul 16, 2026
3f4d3d7
fix(publishers): close batch workers and browser handles
jung233 Jul 16, 2026
987d3de
fix(commands): close fetchers and CARSI clients on every path
jung233 Jul 16, 2026
2bac5ab
fix(elsevier): send the institutional token header correctly
jung233 Jul 16, 2026
0152973
refactor(network): remove the unused browser response adapter
jung233 Jul 16, 2026
4e57c50
Add proxy_only_scihub config to restrict network proxy to Sci-Hub/Lib…
jung233 Jun 25, 2026
840b121
fix(diagnostics): probe the current Sci-Hub domain
jung233 Jun 26, 2026
7ef16b1
Fix Tor download retry storm and reduce grace period
jung233 Jun 26, 2026
5fbfea1
fix(network): close responses and sessions after streaming
jung233 Jul 16, 2026
002d0ca
fix(tor): serialize downloads and reap failed subprocesses
jung233 Jul 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,19 @@ WORKDIR /app
COPY pyproject.toml .
COPY src ./src

RUN pip install --no-cache-dir ".[tor,instsci]"
RUN pip install --no-cache-dir ".[tor,cloakbrowser,instsci]"

FROM python:3.12-slim

RUN apt-get update \
&& apt-get install -y --no-install-recommends \
libglib2.0-0 libnss3 libnspr4 libdbus-1-3 \
libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 \
libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 \
libxrandr2 libgbm1 libpango-1.0-0 libcairo2 libasound2 \
libatspi2.0-0 libwayland-client0 tini \
&& rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY --from=builder /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
Expand All @@ -19,6 +28,9 @@ COPY pyproject.toml .
EXPOSE 8000

ENV SCANSCI_PDF_DATA_DIR=/data/paper-fetch
ENV CLOAKBROWSER_CACHE_DIR=/data/paper-fetch/browser-cache
ENV MCP_MODE=streamable_http
ENV MALLOC_ARENA_MAX=2

ENTRYPOINT ["tini", "--"]
CMD ["python", "-m", "scansci_pdf", "run", "--mode", "streamable_http"]
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,6 +232,7 @@ scansci-pdf run --mode streamable_http --host 0.0.0.0 --port 8000
| `scihub_enabled` | `true` | 启用 Sci-Hub/LibGen 类来源 |
| `network_proxy` | 空 | HTTP/SOCKS 代理地址 |
| `proxy_pool` | 空 | 逗号分隔的代理列表;非空时批量下载按代理轮换出口 IP |
| `proxy_only_scihub` | `false` | 仅对 Sci-Hub/LibGen 使用代理,其他来源直连 |
| `batch_workers` | `10` | 批量下载并发数(被封 IP 时建议调低到 2) |
| `request_delay_min` | `2.0` | 请求间随机延迟下限(秒) |
| `request_delay_max` | `5.0` | 请求间随机延迟上限(秒) |
Expand Down Expand Up @@ -283,6 +284,12 @@ docker compose up -d

如果你的网络无法访问某些来源,可以配置代理或使用 Tor。Docker 模式会提供 Tor 服务;本地模式可按诊断提示启用。

如果你希望代理仅用于 Sci-Hub/LibGen(避免影响机构来源的 IP 授权),启用:

```bash
scansci-pdf config-cmd proxy_only_scihub true
```

### CloakBrowser

遇到 Cloudflare、CAPTCHA、SSO 或出版商浏览器下载时,安装可选浏览器依赖:
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ dependencies = [
"requests>=2.31",
"requests[socks]>=2.31",
"beautifulsoup4>=4.12",
"PyMuPDF>=1.24",
"mcp[cli]>=1.12",
"typer>=0.15",
"uvicorn>=0.34",
Expand Down
Loading