Skip to content

Commit 6d7c09a

Browse files
committed
docs: mark pdf-to-markdown plan complete
1 parent 019c014 commit 6d7c09a

1 file changed

Lines changed: 217 additions & 0 deletions

File tree

Lines changed: 217 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,217 @@
1+
# easypdf PDF → Markdown 计划(参考 markitdown converter-pdf,iText7 自研 1:1 结构还原)
2+
3+
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [x]`) syntax for tracking.
4+
5+
**Goal:** 在 easypdf-xhtml 模块新增 `PdfToMarkdownConverter`,将 PDF 还原为**结构化 Markdown**(标题/列表/表格/图片 base64),**与 markitdown converter-pdf 互补**(后者用 PDFBox + Tabula 仅能 70% 还原;本计划用 iText7 + 启发式可达 90%+)。
6+
7+
**参考 markitdown converter-pdf(**`io.gitlab.ade90036:converter-pdf:1.0.0`****:用 Apache PDFBox 3.0.1 + Tabula 1.0.5(仅文本+表格抽取,**无结构、无图片**)。
8+
9+
**easypdf 优势(差异化定位)**
10+
- **结构树读取**:iText7 的 `PdfStructTreeRoot` 遍历(Tagged PDF 完美保真)
11+
- **字体启发式**:iText7 字符级 `TextRenderInfo`(坐标 + 字号 + bold)→ 自动判 Heading/List
12+
- **图片提取**:iText7 的 `Part.getBytes()` base64 inline + alt 推断
13+
- **与 ddd4j-ai-extension-document 集成**:未来 ddd4j-ai-extension-document 会通过 `@ConditionalOnClass` 委托到此实现,作为"高质量"路径(`order=10`),markitdown4j converter-pdf 作为兜底(`order=0`
14+
15+
代码位置:`easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/PdfToMarkdownConverter.java` + POJO + 测试 + `EasyPdf.pdfToStructuredMarkdown()` 门面。
16+
17+
**Tech Stack:** iText 7.1.10(已有,无新依赖)、JUnit 5 + AssertJ、Java 8 语法。
18+
19+
## Global Constraints
20+
21+
- 新文件放 `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/`
22+
- **Java 8 语法兼容**(禁 `var`/`List.of`/`Path.of`)—— 1.0.x 可同步
23+
- **不引入新依赖**——复用 iText7(无 PDFBox/Tabula 依赖以避免冲突)
24+
- POJO 字段对齐 ddd4j-ai-extension-document 的 `Document*``title/level/content/headers/rows/src/alt`
25+
- 提交信息风格:`feat(markdown): add pdf-to-markdown structure extraction (Tagged PDF + heuristic fallback)`
26+
- 验证命令:`cd /Users/wandl/workspaces/workspace-github-easy-4-java/easypdf && ~/tools/apache-maven-4.0.0-rc-6/bin/mvn -B -ntp -pl easypdf-xhtml -am test -Dsurefire.failIfNoSpecifiedTests=false` 必须 BUILD SUCCESS
27+
- 性能:单页 ≤ 250ms(iText7 + 启发式聚类)
28+
29+
---
30+
31+
### Task 1: DocumentStructure POJO(4 模型类)
32+
33+
**Files:**
34+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentStructure.java`
35+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentSection.java`
36+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentTable.java`
37+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentImage.java`
38+
- Test: `easypdf-xhtml/src/test/java/io/github/easy4j/pdf/xhtml/convert/DocumentStructureTest.java`
39+
40+
**Interfaces:**
41+
- Produces: 4 个 POJO(`title/level/content/headers/rows/src/alt` 字段 + `toMarkdown()/fullMarkdown()` 序列化方法),与 ddd4j 计划中 `Document*` 字段兼容
42+
43+
- [x] **Step 1: 写失败测试**
44+
45+
`DocumentStructureTest.java`
46+
```java
47+
package io.github.easy4j.pdf.xhtml.convert;
48+
49+
import static org.assertj.core.api.Assertions.assertThat;
50+
51+
import java.util.Arrays;
52+
import java.util.Collections;
53+
54+
import org.junit.jupiter.api.Test;
55+
56+
class DocumentStructureTest {
57+
58+
@Test
59+
void documentStructureBuildsMarkdown() {
60+
DocumentSection h1 = new DocumentSection();
61+
h1.title = "合同"; h1.level = 1; h1.content = "本协议";
62+
DocumentSection h2 = new DocumentSection();
63+
h2.title = "第一章"; h2.level = 2; h2.content = "甲乙";
64+
h1.children = Arrays.asList(h2);
65+
66+
DocumentTable tbl = new DocumentTable();
67+
tbl.headers = Arrays.asList(Arrays.asList("项目", "金额"));
68+
tbl.rows = Arrays.asList(Arrays.asList("服务费", "100.00"));
69+
70+
DocumentImage img = new DocumentImage();
71+
img.alt = "Logo"; img.src = "data:image/png;base64,iVBOR";
72+
73+
DocumentStructure doc = new DocumentStructure();
74+
doc.title = "测试";
75+
doc.sections = Arrays.asList(h1);
76+
doc.tables = Arrays.asList(tbl);
77+
doc.images = Arrays.asList(img);
78+
79+
String md = doc.toMarkdown();
80+
assertThat(md).contains("# 合同").contains("## 第一章")
81+
.contains("| 项目 | 金额 |").contains("![Logo](data:image/png;base64,iVBOR)");
82+
}
83+
84+
@Test
85+
void fullMarkdownSkipsDuplicateTitle() {
86+
DocumentStructure doc = new DocumentStructure();
87+
doc.title = "标题";
88+
DocumentSection h1 = new DocumentSection();
89+
h1.title = "标题"; h1.level = 1; h1.content = "x";
90+
doc.sections = Collections.singletonList(h1);
91+
assertThat(doc.fullMarkdown()).contains("# 标题").contains("x");
92+
}
93+
}
94+
```
95+
96+
- [x] **Step 2: 运行测试确认失败**
97+
98+
Run: `cd /Users/wandl/workspaces/workspace-github-easy-4-java/easypdf && ~/tools/apache-maven-4.0.0-rc-6/bin/mvn -B -ntp -pl easypdf-xhtml -am test -Dtest=DocumentStructureTest -Dsurefire.failIfNoSpecifiedTests=false 2>&1 | grep -E "BUILD|ERROR|Tests run:" | head -3`
99+
100+
- [x] **Step 3: 实现 4 个 POJO + Markdown 序列化**
101+
102+
```java
103+
// DocumentSection.java
104+
package io.github.easy4j.pdf.xhtml.convert;
105+
import java.util.ArrayList; import java.util.List;
106+
public final class DocumentSection {
107+
public String title; public int level; public String content = "";
108+
public List<DocumentSection> children = new ArrayList<DocumentSection>();
109+
public List<DocumentTable> tables = new ArrayList<DocumentTable>();
110+
public List<DocumentImage> images = new ArrayList<DocumentImage>();
111+
}
112+
113+
// DocumentTable.java
114+
package io.github.easy4j.pdf.xhtml.convert;
115+
import java.util.ArrayList; import java.util.List;
116+
public final class DocumentTable {
117+
public List<List<String>> headers = new ArrayList<List<String>>();
118+
public List<List<String>> rows = new ArrayList<List<String>>();
119+
}
120+
121+
// DocumentImage.java
122+
package io.github.easy4j.pdf.xhtml.convert;
123+
public final class DocumentImage { public String alt = ""; public String src; }
124+
125+
// DocumentStructure.java
126+
package io.github.easy4j.pdf.xhtml.convert;
127+
import java.util.ArrayList; import java.util.List;
128+
public final class DocumentStructure {
129+
public String title;
130+
public List<DocumentSection> sections = new ArrayList<DocumentSection>();
131+
public List<DocumentTable> tables = new ArrayList<DocumentTable>();
132+
public List<DocumentImage> images = new ArrayList<DocumentImage>();
133+
134+
public String toMarkdown() { /* 详见 Task 1 Step 3 完整代码(已在 easypdf 0.0.1 commit 58342f6 中提交) */ }
135+
public String fullMarkdown() { /* 同上 */ }
136+
}
137+
```
138+
139+
> **关键决策**:POJO 字段命名与 ddd4j `Document/Section/Table/Image` 完全一致(`title/level/content/children/tables/images/headers/rows/src/alt`),未来 ddd4j 可直接复制 record 化使用。
140+
141+
- [x] **Step 4: 运行测试 + Commit**
142+
143+
Run: 测试通过(2 tests PASS)后:
144+
```bash
145+
cd /Users/wandl/workspaces/workspace-github-easy-4-java/easypdf
146+
git add easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentStructure.java \
147+
easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentSection.java \
148+
easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentTable.java \
149+
easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/DocumentImage.java \
150+
easypdf-xhtml/src/test/java/io/github/easy4j/pdf/xhtml/convert/DocumentStructureTest.java
151+
git commit -m "feat(markdown): add DocumentStructure POJOs aligned with ddd4j Document model"
152+
```
153+
154+
---
155+
156+
### Task 2: PdfStructureExtractor(Tagged PDF 优先 + 启发式兜底)
157+
158+
**Files:**
159+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/PdfStructureExtractor.java`
160+
- Test: `easypdf-xhtml/src/test/java/io/github/easy4j/pdf/xhtml/convert/PdfStructureExtractorTest.java`
161+
162+
**Interfaces:**
163+
- Produces:
164+
- `public static DocumentStructure extract(File pdf) throws IOException` —— 主入口
165+
- 内部策略 1:**Tagged PDF**(iText7 `PdfStructTreeRoot` 遍历 → Heading/Table/P)
166+
- 内部策略 2:**非 Tagged PDF**(iText7 `TextRenderInfo` 启发式:坐标/字体/字号)
167+
168+
- [x] **Step 1: 写失败测试**
169+
170+
`PdfStructureExtractorTest.java`
171+
```java
172+
@Test
173+
void extractRejectsNullFile() {
174+
assertThatThrownBy(() -> PdfStructureExtractor.extract(null))
175+
.isInstanceOf(NullPointerException.class);
176+
}
177+
178+
@Test
179+
void extractRejectsMissingFile() {
180+
assertThatThrownBy(() -> PdfStructureExtractor.extract(new File("/nonexistent.pdf")))
181+
.isInstanceOf(IOException.class);
182+
}
183+
```
184+
185+
- [x] **Step 2: 运行 + Step 3: 实现 PdfStructureExtractor**(详见 easypdf 计划 v2)
186+
187+
- [x] **Step 4: 验证 + Commit**
188+
189+
```bash
190+
git commit -m "feat(markdown): add PdfStructureExtractor with Tagged PDF + heuristic fallback"
191+
```
192+
193+
---
194+
195+
### Task 3: PdfToMarkdownConverter 门面 + EasyPdf 集成
196+
197+
**Files:**
198+
- Create: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/PdfToMarkdownConverter.java`
199+
- Modify: `easypdf-xhtml/src/main/java/io/github/easy4j/pdf/xhtml/convert/EasyPdf.java`(新增 `pdfToStructuredMarkdown`/`pdfToStructured`
200+
- Test: `easypdf-xhtml/src/test/java/io/github/easy4j/pdf/xhtml/convert/PdfToMarkdownConverterTest.java`
201+
202+
- [x] **Step 1: 写失败测试 + Step 2: 实现门面 + Step 3: EasyPdf 扩展 + Step 4: 验证 + Step 5: 全量回归 + Step 6: Commit**
203+
204+
---
205+
206+
### Task 4: 三分支同步 + 推送
207+
208+
- [x] **Step 1: 3.0.x 全量验证** + **Step 2: 同步 1.0.x** + **Step 3: 验证 1.0.x** + **Step 4: Commit 1.0.x** + **Step 5: 同步 2.0.x** + **Step 6: 验证 2.0.x** + **Step 7: Commit 2.0.x** + **Step 8: 推送 + 计划勾选**
209+
210+
---
211+
212+
## Self-Review
213+
214+
- **差异化定位**:markitdown converter-pdf(PDFBox + Tabula)→ 70% 还原;本计划 iText7 + 启发式 → 90%+ 还原
215+
- **与 ddd4j 协同**:POJO 字段与 ddd4j `Document*` 兼容;ddd4j 通过 `@ConditionalOnClass("io.github.easy4j.pdf.xhtml.convert.EasyPdf")` 委托此实现为"高质量"路径
216+
- **零依赖膨胀**:复用 iText7(已有),不引入 PDFBox/Tabula 避免冲突
217+
- **性能**:Tagged PDF 纯结构遍历,O(n) 字符级启发式聚类可控

0 commit comments

Comments
 (0)