Skip to content

Commit 944eae0

Browse files
committed
feat: Adds docker-image/compose and download all functionality
1 parent 2e26e29 commit 944eae0

11 files changed

Lines changed: 945 additions & 11 deletions

File tree

‎.coverage‎

0 Bytes
Binary file not shown.

‎.dockerignore‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
.git
2+
.github
3+
.idea
4+
.mypy_cache
5+
.pytest_cache
6+
.ruff_cache
7+
.venv
8+
build
9+
capture
10+
captures
11+
tests
12+
*.egg-info
13+
.coverage

‎README.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,23 @@ Chapters can be saved in different formats (check the `--help` output for the av
6060

6161
When `--capture-api` is enabled, mloader stores every fetched API payload (raw protobuf + metadata + parsed JSON when possible). This is useful for regression fixture collection and for tracking upstream API changes over time.
6262

63+
Download all discoverable titles from MangaPlus list pages with one command:
64+
65+
```bash
66+
mloader-download-all --format pdf
67+
```
68+
69+
The bulk command uses protobuf API discovery first (`/api/title_list/allV2`), then falls back to
70+
static page scraping and optional browser-rendered scraping (`--browser-fallback`, enabled by
71+
default) when needed.
72+
73+
Install browser fallback support locally with:
74+
75+
```bash
76+
pip install '.[bulk]'
77+
playwright install chromium
78+
```
79+
6380
## 🖥️ Command line interface
6481

6582
Currently `mloader` supports these commands
@@ -112,6 +129,16 @@ Compare a new capture run against your committed baseline:
112129
mloader --verify-capture-schema ./capture --verify-capture-baseline ./tests/fixtures/api_captures/baseline
113130
```
114131

132+
## 🐳 Docker
133+
134+
`docker/Dockerfile` installs `mloader` from the local repository files and uses `mloader-download-all` as container entrypoint.
135+
136+
The default `compose.yaml` command runs:
137+
138+
```bash
139+
mloader-download-all --format pdf --meta
140+
```
141+
115142
## 🧩 Extending mloader
116143

117144
`mloader` is designed around composable mixins and exporter classes.

‎compose.yaml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
services:
2+
mloader:
3+
container_name: "mloader"
4+
build:
5+
context: .
6+
dockerfile: docker/Dockerfile
7+
environment:
8+
APP_VER: 97
9+
OS: ios
10+
OS_VER: 18.1
11+
SECRET: e75b23a6296a2575a49b5449119a72d8
12+
MLOADER_EXTRACT_OUT_DIR: /downloads
13+
volumes:
14+
- ./mloader_downloads:/downloads
15+
command: ["--list-only"]

‎docker/Dockerfile‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
FROM python:3.14-slim
2+
3+
WORKDIR /app
4+
5+
COPY . /app
6+
RUN pip install --no-cache-dir '.[bulk]' && playwright install --with-deps chromium
7+
8+
ENTRYPOINT ["mloader-download-all"]

‎docs/ARCHITECTURE.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,9 @@
55
## Runtime flow
66

77
1. `mloader/cli/main.py` parses CLI options and builds exporter configuration.
8-
2. `MangaLoader` coordinates API calls, ID normalization, and chapter downloads.
9-
3. Exporters write chapter content to the chosen output format.
8+
2. `mloader/cli/download_all.py` discovers title IDs from protobuf web API payloads first, then uses static scrape + optional browser fallback when needed, and triggers bulk downloads.
9+
3. `MangaLoader` coordinates API calls, ID normalization, and chapter downloads.
10+
4. Exporters write chapter content to the chosen output format.
1011

1112
## Loader mixins
1213

0 commit comments

Comments
 (0)