Skip to content

Commit cc887e6

Browse files
committed
feat: add layout.json schema
1 parent 67eff3a commit cc887e6

8 files changed

Lines changed: 311 additions & 5 deletions

File tree

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ Possible sections are:
1717

1818
### Added:
1919
- add `--okf` output mode for Open Knowledge Format bundles with per-paper `manuscript.md`, metadata `index.md`, root `index.md`, and root `log.md` artifacts ([#16](https://github.com/atsyplenkov/paperdown/issues/16))
20+
- add `layout.json` OCR layout sidecar in OKF bundles with figure/table artifact links; document LaTeX formula preservation
2021

2122
### Fixed:
2223
- create `figures/` and `tables/` output directories only when downloaded figures or raw OCR table artifacts are written ([#15](https://github.com/atsyplenkov/paperdown/issues/15))

README.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ paperdown --input pdf/ --output md/ --workers 32 --ocr-workers 2 --overwrite
5252

5353
Without `--overwrite`, an existing `<output>/<pdf_stem>/log.jsonl` marker skips the PDF. If the log marker is missing, `paperdown` treats the PDF as unprocessed and refreshes managed artifacts (`index.md`, `figures/`, and `tables/` when `--normalize-tables` is enabled). With `--overwrite`, `paperdown` replaces the whole `<output>/<pdf_stem>/` folder before processing.
5454

55-
OKF output: pass `--okf` to structure each paper directory as an [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle. In OKF mode `<output>/<pdf_stem>/manuscript.md` contains the parsed manuscript text, `<output>/<pdf_stem>/index.md` contains metadata frontmatter plus a directory map, and `figures/` and `tables/` are always present. The output root also gets a regenerated `index.md` listing all paper bundles and an append-only `log.md` update history.
55+
OKF output: pass `--okf` to structure each paper directory as an [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle. In OKF mode `<output>/<pdf_stem>/manuscript.md` contains the parsed manuscript text, `<output>/<pdf_stem>/index.md` contains metadata frontmatter plus a directory map, `<output>/<pdf_stem>/layout.json` contains OCR layout regions, and `figures/` and `tables/` are always present. The output root also gets a regenerated `index.md` listing all paper bundles and an append-only `log.md` update history.
5656

5757
### Table formats and LLM readability
5858

@@ -64,6 +64,14 @@ Plain markdown pipe-tables are intentionally not offered: they benchmark no bett
6464

6565
Practical guidance: keep the default (inline HTML) when you want a faithful, lossless transcript of the paper; add `--normalize-tables` when the markdown is destined for LLM consumption (RAG, agents) and per-row lookup accuracy matters more than token count. Both compose with `--okf`; with `--okf` alone, raw HTML artifacts are still extracted to `tables/` while the manuscript keeps the inline HTML unchanged.
6666

67+
### Formulas
68+
69+
GLM-OCR returns formulas as LaTeX. `paperdown` preserves that LaTeX verbatim in `manuscript.md` and in `layout.json` region content; it does not convert formulas to Unicode or plain text. Keeping the original LaTeX leaves mathematical structure recoverable for agents and downstream parsers.
70+
71+
### OCR layout regions (OKF)
72+
73+
With `--okf`, each paper bundle includes `layout.json`, a per-page sidecar with schema `paperdown.layout.v1`. Each region records its OCR label, bounding box, raw content, and an artifact link when one can be resolved. Image regions link to downloaded files under `figures/`; table regions link to `tables/table_NNN.html` when the table-region count matches the number of extracted table artifacts. If those counts disagree, table linking is skipped and `table_artifact_match` is `"none"`.
74+
6775
## Installation
6876

6977
Install from crates.io:

src/core.rs

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ use tokio::sync::Semaphore;
1111

1212
mod assets;
1313
mod input;
14+
mod layout;
1415
mod markdown;
1516
mod ocr;
1617
mod okf;
@@ -110,7 +111,7 @@ pub async fn process_pdf_with_ocr_limiter(
110111
let (markdown, layout_details, usage) = ocr::validate_layout_response(response)?;
111112

112113
let figure_started = Instant::now();
113-
let (markdown, downloaded_figures, remote_figure_links, image_blocks) =
114+
let (markdown, downloaded_figures, remote_figure_links, image_blocks, figure_replacements) =
114115
assets::localize_figures(
115116
markdown,
116117
&layout_details,
@@ -170,6 +171,20 @@ pub async fn process_pdf_with_ocr_limiter(
170171
&rendered,
171172
)
172173
.await?;
174+
let layout_json = layout::render_layout_json(
175+
&layout_details,
176+
&figure_replacements,
177+
table_stats.tables_raw_written,
178+
&source_rel,
179+
)?;
180+
output::atomic_write_text(
181+
prepared
182+
.layout_path
183+
.as_ref()
184+
.expect("layout path must be set when OKF is enabled"),
185+
&layout_json,
186+
)
187+
.await?;
173188
Some(title)
174189
} else {
175190
None
@@ -182,6 +197,7 @@ pub async fn process_pdf_with_ocr_limiter(
182197
"pdf_path": pdf_path.display().to_string(),
183198
"output_dir": prepared.output_dir.display().to_string(),
184199
"markdown_path": prepared.markdown_path.display().to_string(),
200+
"layout_path": prepared.layout_path.as_ref().map(|path| path.display().to_string()),
185201
"okf": options.okf,
186202
"downloaded_figures": downloaded_figures,
187203
"remote_figure_links": remote_figure_links,
@@ -330,6 +346,7 @@ pub mod testing {
330346
pub tables_dir: Option<std::path::PathBuf>,
331347
pub markdown_path: std::path::PathBuf,
332348
pub paper_index_path: Option<std::path::PathBuf>,
349+
pub layout_path: Option<std::path::PathBuf>,
333350
pub log_path: std::path::PathBuf,
334351
pub stem: String,
335352
}
@@ -381,7 +398,7 @@ pub mod testing {
381398
figures_dir: &Path,
382399
max_download_bytes: u64,
383400
progress: Option<ProgressCallback>,
384-
) -> Result<(String, usize, usize, usize)> {
401+
) -> Result<(String, usize, usize, usize, HashMap<String, String>)> {
385402
super::assets::localize_figures(
386403
markdown,
387404
layout_details,
@@ -421,6 +438,7 @@ pub mod testing {
421438
tables_dir: prepared.tables_dir,
422439
markdown_path: prepared.markdown_path,
423440
paper_index_path: prepared.paper_index_path,
441+
layout_path: prepared.layout_path,
424442
log_path: prepared.log_path,
425443
stem: prepared.stem,
426444
})

src/core/assets.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ pub(crate) async fn localize_figures(
1717
figures_dir: &Path,
1818
max_download_bytes: u64,
1919
progress: Option<ProgressCallback>,
20-
) -> Result<(String, usize, usize, usize)> {
20+
) -> Result<(String, usize, usize, usize, HashMap<String, String>)> {
2121
let mut remote_figure_links = 0usize;
2222
let mut image_blocks = 0usize;
2323
let mut first_url_order: Vec<(String, String)> = Vec::new();
@@ -88,6 +88,7 @@ pub(crate) async fn localize_figures(
8888
downloaded_figures,
8989
remote_figure_links,
9090
image_blocks,
91+
replacements,
9192
))
9293
}
9394

src/core/layout.rs

Lines changed: 259 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,259 @@
1+
use anyhow::Result;
2+
use serde_json::{Value, json};
3+
use std::collections::HashMap;
4+
5+
use super::assets;
6+
7+
const SCHEMA: &str = "paperdown.layout.v1";
8+
9+
pub(crate) fn render_layout_json(
10+
layout_details: &[Value],
11+
figure_replacements: &HashMap<String, String>,
12+
tables_raw_written: usize,
13+
source_rel: &str,
14+
) -> Result<String> {
15+
let table_regions = count_table_regions(layout_details);
16+
let link_tables = table_regions > 0 && table_regions == tables_raw_written;
17+
let table_artifact_match = if link_tables { "order" } else { "none" };
18+
let mut table_index = 0usize;
19+
20+
let pages: Vec<Value> = layout_details
21+
.iter()
22+
.enumerate()
23+
.map(|(page_index, page_blocks)| {
24+
let regions = page_blocks
25+
.as_array()
26+
.map(|blocks| {
27+
blocks
28+
.iter()
29+
.enumerate()
30+
.map(|(block_index, block)| {
31+
render_region(
32+
block,
33+
block_index,
34+
figure_replacements,
35+
link_tables,
36+
&mut table_index,
37+
)
38+
})
39+
.collect::<Vec<_>>()
40+
})
41+
.unwrap_or_default();
42+
43+
json!({
44+
"page": page_index + 1,
45+
"regions": regions,
46+
})
47+
})
48+
.collect();
49+
50+
let rendered = json!({
51+
"schema": SCHEMA,
52+
"source_pdf": source_rel,
53+
"ocr_model": "glm-ocr",
54+
"bbox_format": {
55+
"order": "x1,y1,x2,y2",
56+
"origin": "top-left",
57+
},
58+
"table_artifact_match": table_artifact_match,
59+
"pages": pages,
60+
});
61+
62+
let mut text = serde_json::to_string_pretty(&rendered)?;
63+
text.push('\n');
64+
Ok(text)
65+
}
66+
67+
fn count_table_regions(layout_details: &[Value]) -> usize {
68+
layout_details
69+
.iter()
70+
.filter_map(Value::as_array)
71+
.flatten()
72+
.filter(|block| label(block) == Some("table"))
73+
.count()
74+
}
75+
76+
fn render_region(
77+
block: &Value,
78+
index: usize,
79+
figure_replacements: &HashMap<String, String>,
80+
link_tables: bool,
81+
table_index: &mut usize,
82+
) -> Value {
83+
let label = label(block);
84+
let artifact = match label {
85+
Some("image") => assets::extract_image_url(block)
86+
.and_then(|url| figure_replacements.get(&url).cloned())
87+
.map(Value::String)
88+
.unwrap_or(Value::Null),
89+
Some("table") if link_tables => {
90+
*table_index += 1;
91+
Value::String(format!("tables/table_{:03}.html", *table_index))
92+
}
93+
Some("table") => {
94+
*table_index += 1;
95+
Value::Null
96+
}
97+
_ => Value::Null,
98+
};
99+
100+
json!({
101+
"index": index,
102+
"label": label.map(str::to_owned).map(Value::String).unwrap_or(Value::Null),
103+
"bbox": bbox(block),
104+
"content": block.get("content").cloned().unwrap_or(Value::Null),
105+
"artifact": artifact,
106+
})
107+
}
108+
109+
fn label(block: &Value) -> Option<&str> {
110+
block.get("label").and_then(Value::as_str)
111+
}
112+
113+
fn bbox(block: &Value) -> Value {
114+
block
115+
.get("bbox_2d")
116+
.or_else(|| block.get("bbox"))
117+
.cloned()
118+
.unwrap_or(Value::Null)
119+
}
120+
121+
#[cfg(test)]
122+
mod tests {
123+
use super::render_layout_json;
124+
use serde_json::{Value, json};
125+
use std::collections::HashMap;
126+
127+
// Render `layout_details` through the real renderer and parse the output back
128+
// into a `Value` so assertions are structural, not brittle pretty-text checks.
129+
fn render(
130+
layout: &[Value],
131+
replacements: &HashMap<String, String>,
132+
tables_written: usize,
133+
source: &str,
134+
) -> Value {
135+
let text =
136+
render_layout_json(layout, replacements, tables_written, source).expect("renders");
137+
serde_json::from_str(&text).expect("output is valid JSON")
138+
}
139+
140+
#[test]
141+
fn full_render_links_image_and_table_and_preserves_latex() {
142+
// One page: title, a LaTeX formula, an image region (remote URL), a table.
143+
let layout = vec![json!([
144+
{"label": "title", "bbox_2d": [72, 41, 928, 96], "content": "Paper title"},
145+
{
146+
"label": "formula",
147+
"bbox_2d": [10, 10, 200, 50],
148+
"content": r"$\int_0^\infty e^{-x^2}\,dx = \frac{\sqrt{\pi}}{2}$"
149+
},
150+
{
151+
"label": "image",
152+
"bbox_2d": [90, 120, 910, 390],
153+
"content": "https://example.com/fig.png"
154+
},
155+
{
156+
"label": "table",
157+
"bbox_2d": [80, 530, 920, 780],
158+
"content": "<table>x</table>"
159+
}
160+
])];
161+
// The figure URL resolves through the replacement map to a local file.
162+
let replacements = HashMap::from([(
163+
"https://example.com/fig.png".to_string(),
164+
"figures/fig-001-001.png".to_string(),
165+
)]);
166+
167+
// One table region equals the number of tables actually written -> order linking.
168+
let doc = render(&layout, &replacements, 1, "paper.pdf");
169+
170+
assert_eq!(doc["schema"], "paperdown.layout.v1");
171+
assert_eq!(doc["source_pdf"], "paper.pdf");
172+
assert_eq!(doc["ocr_model"], "glm-ocr");
173+
assert_eq!(doc["bbox_format"]["order"], "x1,y1,x2,y2");
174+
assert_eq!(doc["bbox_format"]["origin"], "top-left");
175+
assert_eq!(doc["table_artifact_match"], "order");
176+
177+
let regions = doc["pages"][0]["regions"]
178+
.as_array()
179+
.expect("regions array");
180+
assert_eq!(doc["pages"][0]["page"], 1);
181+
assert_eq!(regions.len(), 4);
182+
183+
// Title: bbox copied verbatim from bbox_2d.
184+
assert_eq!(regions[0]["label"], "title");
185+
assert_eq!(regions[0]["bbox"].clone(), json!([72, 41, 928, 96]));
186+
assert_eq!(regions[0]["content"], "Paper title");
187+
assert_eq!(regions[0]["artifact"], Value::Null);
188+
189+
// Formula: raw LaTeX content is carried through unchanged.
190+
assert_eq!(regions[1]["label"], "formula");
191+
assert_eq!(
192+
regions[1]["content"].as_str(),
193+
Some(r"$\int_0^\infty e^{-x^2}\,dx = \frac{\sqrt{\pi}}{2}$"),
194+
);
195+
assert_eq!(regions[1]["artifact"], Value::Null);
196+
197+
// Image: artifact resolved via the figure replacement map.
198+
assert_eq!(regions[2]["label"], "image");
199+
assert_eq!(regions[2]["artifact"], "figures/fig-001-001.png");
200+
201+
// Table: first table region -> tables/table_001.html by document order.
202+
assert_eq!(regions[3]["label"], "table");
203+
assert_eq!(regions[3]["artifact"], "tables/table_001.html");
204+
}
205+
206+
#[test]
207+
fn table_count_mismatch_skips_table_links() {
208+
// Two table regions, but three tables written: counts disagree.
209+
let layout = vec![json!([
210+
{"label": "table", "bbox_2d": [1, 2, 3, 4], "content": "<table>a</table>"},
211+
{"label": "table", "bbox_2d": [5, 6, 7, 8], "content": "<table>b</table>"}
212+
])];
213+
let replacements = HashMap::new();
214+
215+
let doc = render(&layout, &replacements, 3, "paper.pdf");
216+
217+
assert_eq!(doc["table_artifact_match"], "none");
218+
let regions = doc["pages"][0]["regions"]
219+
.as_array()
220+
.expect("regions array");
221+
assert_eq!(regions.len(), 2);
222+
assert!(
223+
regions.iter().all(|r| r["artifact"].is_null()),
224+
"every table artifact must be null when region count != tables written"
225+
);
226+
}
227+
228+
#[test]
229+
fn missing_or_non_string_fields_become_null() {
230+
// Region 0: non-string label, no bbox keys, no content.
231+
// Region 1: entirely absent label/bbox/content.
232+
let layout = vec![json!([{"label": 123}, {"weird": "block"}])];
233+
let replacements = HashMap::new();
234+
235+
let doc = render(&layout, &replacements, 0, "paper.pdf");
236+
let regions = doc["pages"][0]["regions"]
237+
.as_array()
238+
.expect("regions array");
239+
240+
for region in regions {
241+
assert_eq!(region["label"], Value::Null);
242+
assert_eq!(region["bbox"], Value::Null);
243+
assert_eq!(region["content"], Value::Null);
244+
assert_eq!(region["artifact"], Value::Null);
245+
}
246+
}
247+
248+
#[test]
249+
fn non_array_page_emits_empty_regions() {
250+
// A page whose value is not an array must not panic; it yields no regions.
251+
let layout = vec![json!({"not": "an array"})];
252+
let replacements = HashMap::new();
253+
254+
let doc = render(&layout, &replacements, 0, "paper.pdf");
255+
256+
assert_eq!(doc["pages"][0]["page"], 1);
257+
assert_eq!(doc["pages"][0]["regions"].clone(), json!([]));
258+
}
259+
}

0 commit comments

Comments
 (0)