diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d957abf8b..9f54c760c4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,10 +10,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Training web API: list ongoing trainings (`GET /api/allTraining`) and interrupt a running training (`DELETE /api/killTraining`). Killing a training releases the per-model lock so the model can be retrained; interruption is best-effort for native (Wapiti/DeLFT) back-ends. ### Changed +- Updated pdfalto to 0.6.1. Notable for GROBID: deterministic output (an uninitialised read and a use-after-free made line/block grouping depend on heap contents, so the same PDF could yield different ``/`` structure across runs), bounded peak memory on very large or vector-heavy documents, and several text-extraction fixes. The command-line options GROBID passes are unchanged. +- pdfalto is now given GROBID's configured temp directory (`grobid.temp`, by default `grobid-home/tmp`) through `TMPDIR`. Since 0.6.1 pdfalto streams large page DOMs to a scratch file to bound peak memory and picks its location from `TMPDIR`, falling back to `/tmp` — which is tmpfs in most containers, so spilling there leaves peak memory unchanged and risks ENOSPC. - Lexicon: introduced `Lexicon.builder()` to optionally pre-load chosen gazetteers *eagerly* (`.withDefaults()`, `.withJournals()`, `.withFunders()`, `.withOrganisations()`, etc.). Loading stays **lazy by default**: any gazetteer not named in the builder loads transparently on first lookup, so a `Lexicon` from any entry point is always fully functional and never throws for a missing gazetteer — `withX()` only controls *when* a gazetteer loads, not whether a lookup succeeds. `withDefaults()` eagerly loads the original constructor's set (wordforms, people, countries). `Lexicon.getInstance()` is now `@Deprecated` (prefer the builder) but its behavior is unchanged: eager wordforms/people/countries, everything else lazy. - Lexicon: added 4 missing ISO 3166-1 country codes (BQ, CW, SS, SX) and migrated AN (Netherlands Antilles) to its ISO 3166-3 transitional form ANHH. - Documentation: expanded the End-to-end evaluation guide (dedicated multi-dataset runner-script section for `run_evaluation.sh` with an options table and more invocation examples) and the Configuration reference; recorded the Hugging Face DOI (10.57967/hf/9553) for the `grobid-evaluation` dataset. +### Fixed +- Handle the exit codes pdfalto 0.6.1 introduced. Code 5 reports that the ALTO was written correctly but page streaming was disabled mid-run, so peak memory was no longer bounded; both execution paths rejected any non-zero exit and would have discarded these successful conversions. It is now a logged warning. Codes 4 (ALTO write failed) and 98 (allocation failure) are reported as `PDFALTO_CONVERSION_FAILURE` instead of falling through to `BAD_INPUT_DATA`, which blamed the PDF for a pdfalto-side failure. + ### Security - Prevent command injection through crafted PDF file names in the non-server `pdfalto` path: the command is no longer interpolated into a `bash -c` string but passed as positional parameters and exec'd via `"$@"` (GHSA-mgxf-7mg7-qpmf). - Stop leaking a JVM thread per request on the `/api/modelTraining` endpoint by shutting down the per-request executor (GHSA-g2r5-4c8r-c84f). diff --git a/doc/benchmarks/Benchmarking-biorxiv.md b/doc/benchmarks/Benchmarking-biorxiv.md index 504f0bdf1a..de6d977f1e 100644 --- a/doc/benchmarks/Benchmarking-biorxiv.md +++ b/doc/benchmarks/Benchmarking-biorxiv.md @@ -42,15 +42,15 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 2.31 | 2.26 | 2.29 | 1989 | +| abstract | 2.36 | 2.31 | 2.34 | 1989 | | affiliation_linked | 0.86 | 0.86 | 0.86 | 1962 | -| authors | 84.85 | 84.38 | 84.62 | 1998 | -| first_author | 96.73 | 96.29 | 96.51 | 1996 | -| keywords | 57.28 | 57.28 | 57.28 | 838 | -| title | 77.26 | 76.49 | 76.87 | 1999 | +| authors | 84.9 | 84.43 | 84.67 | 1998 | +| first_author | 96.78 | 96.34 | 96.56 | 1996 | +| keywords | 57.47 | 57.4 | 57.43 | 838 | +| title | 77.31 | 76.54 | 76.92 | 1999 | | | | | | | -| **all fields (micro avg.)** | **22.78** | **22.77** | **22.78** | 10782 | -| all fields (macro avg.) | 53.21 | 52.93 | 53.07 | 10782 | +| **all fields (micro avg.)** | **22.78** | **22.79** | **22.78** | 10782 | +| all fields (macro avg.) | 53.28 | 52.98 | 53.13 | 10782 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -58,31 +58,31 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 59.52 | 58.32 | 58.91 | 1989 | -| affiliation_linked | 75.32 | 75.59 | 75.45 | 1962 | -| authors | 85.35 | 84.88 | 85.12 | 1998 | -| first_author | 96.98 | 96.54 | 96.76 | 1996 | -| keywords | 63.01 | 63.01 | 63.01 | 838 | -| title | 79.48 | 78.69 | 79.08 | 1999 | +| abstract | 59.65 | 58.42 | 59.03 | 1989 | +| affiliation_linked | 75.16 | 75.55 | 75.35 | 1962 | +| authors | 85.41 | 84.93 | 85.17 | 1998 | +| first_author | 97.03 | 96.59 | 96.81 | 1996 | +| keywords | 63.08 | 63.01 | 63.04 | 838 | +| title | 79.54 | 78.74 | 79.14 | 1999 | | | | | | | -| **all fields (micro avg.)** | **76.5** | **76.44** | **76.47** | 10782 | -| all fields (macro avg.) | 76.61 | 76.17 | 76.39 | 10782 | +| **all fields (micro avg.)** | **76.42** | **76.44** | **76.43** | 10782 | +| all fields (macro avg.) | 76.64 | 76.21 | 76.42 | 10782 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) **Field-level results** -| label | precision | recall | f1 | support | -|-----------------------------|-----------|-----------|----------|---------| -| abstract | 80.09 | 78.48 | 79.28 | 1989 | -| affiliation_linked | 77.47 | 77.75 | 77.61 | 1962 | -| authors | 92.55 | 92.04 | 92.3 | 1998 | -| first_author | 97.23 | 96.79 | 97.01 | 1996 | -| keywords | 78.16 | 78.16 | 78.16 | 838 | -| title | 91.92 | 91 | 91.45 | 1999 | -| | | | | | -| **all fields (micro avg.)** | **81.53** | **81.47** | **81.5** | 10782 | -| all fields (macro avg.) | 86.24 | 85.7 | 85.97 | 10782 | +| label | precision | recall | f1 | support | +|-----------------------------|-----------|-----------|-----------|---------| +| abstract | 80.24 | 78.58 | 79.4 | 1989 | +| affiliation_linked | 77.33 | 77.74 | 77.53 | 1962 | +| authors | 92.6 | 92.09 | 92.35 | 1998 | +| first_author | 97.28 | 96.84 | 97.06 | 1996 | +| keywords | 78.14 | 78.04 | 78.09 | 838 | +| title | 91.97 | 91.05 | 91.5 | 1999 | +| | | | | | +| **all fields (micro avg.)** | **81.46** | **81.48** | **81.47** | 10782 | +| all fields (macro avg.) | 86.26 | 85.72 | 85.99 | 10782 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -90,15 +90,15 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 77.01 | 75.47 | 76.23 | 1989 | -| affiliation_linked | 75.97 | 76.25 | 76.11 | 1962 | -| authors | 88.48 | 87.99 | 88.23 | 1998 | -| first_author | 96.73 | 96.29 | 96.51 | 1996 | -| keywords | 70.29 | 70.29 | 70.29 | 838 | -| title | 87.67 | 86.79 | 87.23 | 1999 | +| abstract | 77.1 | 75.52 | 76.3 | 1989 | +| affiliation_linked | 75.81 | 76.21 | 76.01 | 1962 | +| authors | 88.53 | 88.04 | 88.28 | 1998 | +| first_author | 96.78 | 96.34 | 96.56 | 1996 | +| keywords | 70.25 | 70.17 | 70.21 | 838 | +| title | 87.72 | 86.84 | 87.28 | 1999 | | | | | | | -| **all fields (micro avg.)** | **79.37** | **79.31** | **79.34** | 10782 | -| all fields (macro avg.) | 82.69 | 82.18 | 82.43 | 10782 | +| **all fields (micro avg.)** | **79.28** | **79.29** | **79.29** | 10782 | +| all fields (macro avg.) | 82.7 | 82.19 | 82.44 | 10782 | Note: the "affiliation_linked" field above is a linking-aware metric (each author is paired with its gold counterpart and their attached affiliations compared). Its support column reports the number of articles the metric is computed @@ -114,15 +114,15 @@ author-to-affiliation mapping only positionally, which require the PDF superscri ``` Total expected instances: 1999 -Total correct instances: 37 (strict) -Total correct instances: 723 (soft) -Total correct instances: 1222 (Levenshtein) -Total correct instances: 1052 (ObservedRatcliffObershelp) - -Instance-level recall: 1.85 (strict) -Instance-level recall: 36.17 (soft) -Instance-level recall: 61.13 (Levenshtein) -Instance-level recall: 52.63 (RatcliffObershelp) +Total correct instances: 39 (strict) +Total correct instances: 729 (soft) +Total correct instances: 1224 (Levenshtein) +Total correct instances: 1055 (ObservedRatcliffObershelp) + +Instance-level recall: 1.95 (strict) +Instance-level recall: 36.47 (soft) +Instance-level recall: 61.23 (Levenshtein) +Instance-level recall: 52.78 (RatcliffObershelp) ``` ## Citation metadata @@ -135,20 +135,20 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|----------|---------| -| authors | 88.33 | 83.1 | 85.64 | 97164 | -| date | 91.55 | 85.86 | 88.61 | 97611 | -| doi | 71.1 | 83.57 | 76.83 | 16894 | -| first_author | 95.14 | 89.43 | 92.2 | 97164 | -| inTitle | 82.72 | 79.03 | 80.84 | 96411 | -| issue | 93.93 | 90.76 | 92.32 | 30298 | -| page | 94.83 | 77.89 | 85.53 | 88578 | +| authors | 88.36 | 83.26 | 85.73 | 97164 | +| date | 91.56 | 86.01 | 88.7 | 97611 | +| doi | 71.15 | 83.86 | 76.98 | 16894 | +| first_author | 95.16 | 89.6 | 92.3 | 97164 | +| inTitle | 82.75 | 79.2 | 80.94 | 96411 | +| issue | 93.98 | 91.18 | 92.56 | 30298 | +| page | 94.83 | 78.06 | 85.63 | 88578 | | pmcid | 65.78 | 82.9 | 73.36 | 807 | | pmid | 69.95 | 80.41 | 74.82 | 2093 | -| title | 84.81 | 83.29 | 84.04 | 92444 | -| volume | 95.97 | 94.78 | 95.37 | 87691 | +| title | 84.83 | 83.45 | 84.13 | 92444 | +| volume | 95.99 | 94.97 | 95.48 | 87691 | | | | | | | -| **all fields (micro avg.)** | **89.78** | **84.95** | **87.3** | 707155 | -| all fields (macro avg.) | 84.92 | 84.64 | 84.5 | 707155 | +| **all fields (micro avg.)** | **89.8** | **85.13** | **87.4** | 707155 | +| all fields (macro avg.) | 84.94 | 84.81 | 84.6 | 707155 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -156,41 +156,41 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 89.48 | 84.18 | 86.75 | 97164 | -| date | 91.55 | 85.86 | 88.61 | 97611 | -| doi | 75.56 | 88.82 | 81.65 | 16894 | -| first_author | 95.57 | 89.83 | 92.61 | 97164 | -| inTitle | 92.14 | 88.03 | 90.04 | 96411 | -| issue | 93.93 | 90.76 | 92.32 | 30298 | -| page | 94.83 | 77.89 | 85.53 | 88578 | +| authors | 89.5 | 84.34 | 86.85 | 97164 | +| date | 91.56 | 86.01 | 88.7 | 97611 | +| doi | 75.66 | 89.18 | 81.87 | 16894 | +| first_author | 95.59 | 90 | 92.71 | 97164 | +| inTitle | 92.17 | 88.21 | 90.15 | 96411 | +| issue | 93.98 | 91.18 | 92.56 | 30298 | +| page | 94.83 | 78.06 | 85.63 | 88578 | | pmcid | 74.73 | 94.18 | 83.33 | 807 | | pmid | 73.82 | 84.85 | 78.95 | 2093 | -| title | 93.11 | 91.43 | 92.26 | 92444 | -| volume | 95.97 | 94.78 | 95.37 | 87691 | +| title | 93.14 | 91.62 | 92.37 | 92444 | +| volume | 95.99 | 94.97 | 95.48 | 87691 | | | | | | | -| **all fields (micro avg.)** | **92.58** | **87.59** | **90.02** | 707155 | -| all fields (macro avg.) | 88.24 | 88.24 | 87.95 | 707155 | +| **all fields (micro avg.)** | **92.6** | **87.78** | **90.13** | 707155 | +| all fields (macro avg.) | 88.27 | 88.42 | 88.05 | 707155 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) **Field-level results** -| label | precision | recall | f1 | support | -|-----------------------------|-----------|-----------|----------|---------| -| authors | 94.66 | 89.05 | 91.77 | 97164 | -| date | 91.55 | 85.86 | 88.61 | 97611 | -| doi | 77.59 | 91.2 | 83.85 | 16894 | -| first_author | 95.71 | 89.97 | 92.75 | 97164 | -| inTitle | 93.18 | 89.03 | 91.06 | 96411 | -| issue | 93.93 | 90.76 | 92.32 | 30298 | -| page | 94.83 | 77.89 | 85.53 | 88578 | -| pmcid | 74.73 | 94.18 | 83.33 | 807 | -| pmid | 73.82 | 84.85 | 78.95 | 2093 | -| title | 96 | 94.27 | 95.13 | 92444 | -| volume | 95.97 | 94.78 | 95.37 | 87691 | -| | | | | | -| **all fields (micro avg.)** | **93.9** | **88.85** | **91.3** | 707155 | -| all fields (macro avg.) | 89.27 | 89.26 | 88.97 | 707155 | +| label | precision | recall | f1 | support | +|-----------------------------|-----------|-----------|-----------|---------| +| authors | 94.68 | 89.22 | 91.87 | 97164 | +| date | 91.56 | 86.01 | 88.7 | 97611 | +| doi | 77.69 | 91.57 | 84.06 | 16894 | +| first_author | 95.73 | 90.14 | 92.85 | 97164 | +| inTitle | 93.21 | 89.2 | 91.16 | 96411 | +| issue | 93.98 | 91.18 | 92.56 | 30298 | +| page | 94.83 | 78.06 | 85.63 | 88578 | +| pmcid | 74.73 | 94.18 | 83.33 | 807 | +| pmid | 73.82 | 84.85 | 78.95 | 2093 | +| title | 96.02 | 94.45 | 95.23 | 92444 | +| volume | 95.99 | 94.97 | 95.48 | 87691 | +| | | | | | +| **all fields (micro avg.)** | **93.92** | **89.03** | **91.41** | 707155 | +| all fields (macro avg.) | 89.29 | 89.44 | 89.08 | 707155 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -198,55 +198,55 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 91.66 | 86.23 | 88.86 | 97164 | -| date | 91.55 | 85.86 | 88.61 | 97611 | -| doi | 76.23 | 89.61 | 82.38 | 16894 | -| first_author | 95.19 | 89.47 | 92.24 | 97164 | -| inTitle | 90.88 | 86.82 | 88.8 | 96411 | -| issue | 93.93 | 90.76 | 92.32 | 30298 | -| page | 94.83 | 77.89 | 85.53 | 88578 | +| authors | 91.68 | 86.39 | 88.96 | 97164 | +| date | 91.56 | 86.01 | 88.7 | 97611 | +| doi | 76.33 | 89.96 | 82.58 | 16894 | +| first_author | 95.21 | 89.64 | 92.34 | 97164 | +| inTitle | 90.9 | 87 | 88.91 | 96411 | +| issue | 93.98 | 91.18 | 92.56 | 30298 | +| page | 94.83 | 78.06 | 85.63 | 88578 | | pmcid | 65.78 | 82.9 | 73.36 | 807 | | pmid | 69.95 | 80.41 | 74.82 | 2093 | -| title | 95.32 | 93.6 | 94.45 | 92444 | -| volume | 95.97 | 94.78 | 95.37 | 87691 | +| title | 95.35 | 93.79 | 94.56 | 92444 | +| volume | 95.99 | 94.97 | 95.48 | 87691 | | | | | | | -| **all fields (micro avg.)** | **92.94** | **87.94** | **90.37** | 707155 | -| all fields (macro avg.) | 87.39 | 87.12 | 86.98 | 707155 | +| **all fields (micro avg.)** | **92.96** | **88.13** | **90.48** | 707155 | +| all fields (macro avg.) | 87.41 | 87.3 | 87.08 | 707155 | #### Instance-level results ``` Total expected instances: 98780 -Total extracted instances: 97937 -Total correct instances: 43446 (strict) -Total correct instances: 54331 (soft) -Total correct instances: 58493 (Levenshtein) -Total correct instances: 55273 (RatcliffObershelp) +Total extracted instances: 98136 +Total correct instances: 43542 (strict) +Total correct instances: 54450 (soft) +Total correct instances: 58627 (Levenshtein) +Total correct instances: 55401 (RatcliffObershelp) -Instance-level precision: 44.36 (strict) +Instance-level precision: 44.37 (strict) Instance-level precision: 55.48 (soft) -Instance-level precision: 59.73 (Levenshtein) -Instance-level precision: 56.44 (RatcliffObershelp) +Instance-level precision: 59.74 (Levenshtein) +Instance-level precision: 56.45 (RatcliffObershelp) -Instance-level recall: 43.98 (strict) -Instance-level recall: 55 (soft) -Instance-level recall: 59.22 (Levenshtein) -Instance-level recall: 55.96 (RatcliffObershelp) +Instance-level recall: 44.08 (strict) +Instance-level recall: 55.12 (soft) +Instance-level recall: 59.35 (Levenshtein) +Instance-level recall: 56.09 (RatcliffObershelp) -Instance-level f-score: 44.17 (strict) -Instance-level f-score: 55.24 (soft) -Instance-level f-score: 59.47 (Levenshtein) -Instance-level f-score: 56.2 (RatcliffObershelp) +Instance-level f-score: 44.22 (strict) +Instance-level f-score: 55.3 (soft) +Instance-level f-score: 59.55 (Levenshtein) +Instance-level f-score: 56.27 (RatcliffObershelp) -Matching 1 : 78835 +Matching 1 : 78995 -Matching 2 : 4478 +Matching 2 : 4471 -Matching 3 : 4339 +Matching 3 : 4350 -Matching 4 : 2218 +Matching 4 : 2221 -Total matches : 89870 +Total matches : 90037 ``` #### Citation context resolution @@ -254,17 +254,17 @@ Total matches : 89870 ``` Total expected references: 98778 - 49.41 references per article -Total predicted references: 97937 - 48.99 references per article +Total predicted references: 98136 - 49.09 references per article Total expected citation contexts: 142847 - 71.46 citation contexts per article -Total predicted citation contexts: 134757 - 67.41 citation contexts per article +Total predicted citation contexts: 135039 - 67.55 citation contexts per article -Total correct predicted citation contexts: 116233 - 58.15 citation contexts per article -Total wrong predicted citation contexts: 18524 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) +Total correct predicted citation contexts: 116481 - 58.27 citation contexts per article +Total wrong predicted citation contexts: 18558 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) -Precision citation contexts: 86.25 -Recall citation contexts: 81.37 -fscore citation contexts: 83.74 +Precision citation contexts: 86.26 +Recall citation contexts: 81.54 +fscore citation contexts: 83.83 ``` ## Fulltext structures @@ -280,21 +280,21 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). **Field-level results** -| label | precision | recall | f1 | support | -|-----------------------------|-----------|-----------|-----------|---------| -| availability_stmt | 28.34 | 27.58 | 27.95 | 446 | -| conflict_stmt | 66.42 | 59.11 | 62.55 | 609 | -| contribution_stmt | 42.7 | 43.68 | 43.18 | 609 | -| figure_title | 4.25 | 2.36 | 3.03 | 22972 | -| funding_stmt | 3.84 | 23.96 | 6.62 | 747 | -| reference_citation | 71.96 | 70.95 | 71.45 | 147455 | -| reference_figure | 70.31 | 76.98 | 73.5 | 47976 | -| reference_table | 45.11 | 84.74 | 58.88 | 5956 | -| section_title | 69.08 | 68.62 | 68.85 | 32391 | -| table_title | 6.98 | 2.55 | 3.73 | 3924 | -| | | | | | -| **all fields (micro avg.)** | **65.11** | **64.77** | **64.94** | 263085 | -| all fields (macro avg.) | 40.9 | 46.05 | 41.97 | 263085 | +| label | precision | recall | f1 | support | +|-----------------------------|-----------|----------|-----------|---------| +| availability_stmt | 28.41 | 27.58 | 27.99 | 446 | +| conflict_stmt | 66.61 | 59.28 | 62.73 | 609 | +| contribution_stmt | 42.72 | 43.84 | 43.27 | 609 | +| figure_title | 4.32 | 2.37 | 3.06 | 22972 | +| funding_stmt | 3.85 | 23.96 | 6.63 | 747 | +| reference_citation | 71.95 | 71.07 | 71.51 | 147455 | +| reference_figure | 70.34 | 77.14 | 73.58 | 47976 | +| reference_table | 45.21 | 85.02 | 59.03 | 5956 | +| section_title | 69.7 | 68.88 | 69.29 | 32391 | +| table_title | 7.05 | 2.57 | 3.77 | 3924 | +| | | | | | +| **all fields (micro avg.)** | **65.23** | **64.9** | **65.07** | 263085 | +| all fields (macro avg.) | 41.01 | 46.17 | 42.09 | 263085 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -302,30 +302,30 @@ Evaluation on 1999 random PDF files out of 1998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| availability_stmt | 49.54 | 48.21 | 48.86 | 446 | -| conflict_stmt | 81 | 72.09 | 76.28 | 609 | -| contribution_stmt | 72.71 | 74.38 | 73.54 | 609 | -| figure_title | 66.77 | 36.97 | 47.59 | 22972 | -| funding_stmt | 4.1 | 25.57 | 7.06 | 747 | -| reference_citation | 84.29 | 83.11 | 83.69 | 147455 | -| reference_figure | 70.97 | 77.7 | 74.18 | 47976 | -| reference_table | 45.5 | 85.48 | 59.39 | 5956 | -| section_title | 74.49 | 74 | 74.25 | 32391 | +| availability_stmt | 49.65 | 48.21 | 48.92 | 446 | +| conflict_stmt | 81.18 | 72.25 | 76.46 | 609 | +| contribution_stmt | 72.64 | 74.55 | 73.58 | 609 | +| figure_title | 67.44 | 37.06 | 47.83 | 22972 | +| funding_stmt | 4.1 | 25.57 | 7.07 | 747 | +| reference_citation | 84.29 | 83.25 | 83.77 | 147455 | +| reference_figure | 71 | 77.86 | 74.27 | 47976 | +| reference_table | 45.61 | 85.78 | 59.55 | 5956 | +| section_title | 75.17 | 74.28 | 74.72 | 32391 | | table_title | 81.22 | 29.64 | 43.43 | 3924 | | | | | | | -| **all fields (micro avg.)** | **76.36** | **75.96** | **76.16** | 263085 | -| all fields (macro avg.) | 63.06 | 60.71 | 58.83 | 263085 | +| **all fields (micro avg.)** | **76.5** | **76.12** | **76.31** | 263085 | +| all fields (macro avg.) | 63.23 | 60.84 | 58.96 | 263085 | **Document-level ratio results** | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| availability_stmt | 82.35 | 97.31 | 89.21 | 446 | +| availability_stmt | 82.48 | 97.09 | 89.19 | 446 | | conflict_stmt | 95.42 | 89 | 92.1 | 609 | -| contribution_stmt | 91.08 | 102.3 | 96.37 | 609 | +| contribution_stmt | 90.98 | 102.63 | 96.45 | 609 | | | | | | | -| **all fields (micro avg.)** | **89.88** | **96.09** | **92.88** | 1664 | -| all fields (macro avg.) | 89.62 | 96.2 | 92.56 | 1664 | +| **all fields (micro avg.)** | **89.89** | **96.15** | **92.92** | 1664 | +| all fields (macro avg.) | 89.62 | 96.24 | 92.58 | 1664 | -Evaluation metrics produced in 291.439 seconds +Evaluation metrics produced in 208.92 seconds diff --git a/doc/benchmarks/Benchmarking-elife.md b/doc/benchmarks/Benchmarking-elife.md index 6b9811e10b..47646234a5 100644 --- a/doc/benchmarks/Benchmarking-elife.md +++ b/doc/benchmarks/Benchmarking-elife.md @@ -41,14 +41,14 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 8.66 | 8.33 | 8.49 | 984 | +| abstract | 8.65 | 8.33 | 8.49 | 984 | | affiliation_linked | 2.47 | 2.86 | 2.65 | 981 | -| authors | 78.18 | 77.62 | 77.9 | 983 | +| authors | 78.28 | 77.72 | 78 | 983 | | first_author | 93.95 | 93.38 | 93.67 | 982 | | title | 88.8 | 87.8 | 88.3 | 984 | | | | | | | -| **all fields (micro avg.)** | **18.26** | **20.28** | **19.22** | 4914 | -| all fields (macro avg.) | 54.41 | 54 | 54.2 | 4914 | +| **all fields (micro avg.)** | **18.25** | **20.29** | **19.21** | 4914 | +| all fields (macro avg.) | 54.43 | 54.02 | 54.22 | 4914 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -56,14 +56,14 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 21.54 | 20.73 | 21.13 | 984 | -| affiliation_linked | 68.63 | 79.51 | 73.67 | 981 | -| authors | 78.59 | 78.03 | 78.31 | 983 | +| abstract | 21.52 | 20.73 | 21.12 | 984 | +| affiliation_linked | 68.56 | 79.52 | 73.63 | 981 | +| authors | 78.69 | 78.13 | 78.41 | 983 | | first_author | 93.95 | 93.38 | 93.67 | 982 | | title | 95.79 | 94.72 | 95.25 | 984 | | | | | | | -| **all fields (micro avg.)** | **69.64** | **77.38** | **73.31** | 4914 | -| all fields (macro avg.) | 71.7 | 73.27 | 72.4 | 4914 | +| **all fields (micro avg.)** | **69.59** | **77.39** | **73.29** | 4914 | +| all fields (macro avg.) | 71.7 | 73.29 | 72.41 | 4914 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) @@ -71,14 +71,14 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 46.67 | 44.92 | 45.78 | 984 | -| affiliation_linked | 71.72 | 83.09 | 76.99 | 981 | -| authors | 89.86 | 89.22 | 89.54 | 983 | +| abstract | 46.73 | 45.02 | 45.86 | 984 | +| affiliation_linked | 71.67 | 83.12 | 76.97 | 981 | +| authors | 89.96 | 89.32 | 89.64 | 983 | | first_author | 94.26 | 93.69 | 93.97 | 982 | | title | 97.23 | 96.14 | 96.68 | 984 | | | | | | | -| **all fields (micro avg.)** | **74.27** | **82.52** | **78.18** | 4914 | -| all fields (macro avg.) | 79.95 | 81.41 | 80.59 | 4914 | +| **all fields (micro avg.)** | **74.23** | **82.55** | **78.17** | 4914 | +| all fields (macro avg.) | 79.97 | 81.46 | 80.62 | 4914 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -86,14 +86,14 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| abstract | 43.4 | 41.77 | 42.57 | 984 | -| affiliation_linked | 70.09 | 81.2 | 75.24 | 981 | -| authors | 83.3 | 82.71 | 83 | 983 | +| abstract | 43.46 | 41.87 | 42.65 | 984 | +| affiliation_linked | 70.03 | 81.22 | 75.21 | 981 | +| authors | 83.4 | 82.81 | 83.1 | 983 | | first_author | 93.95 | 93.38 | 93.67 | 982 | | title | 97.23 | 96.14 | 96.68 | 984 | | | | | | | -| **all fields (micro avg.)** | **72.42** | **80.46** | **76.23** | 4914 | -| all fields (macro avg.) | 77.59 | 79.04 | 78.23 | 4914 | +| **all fields (micro avg.)** | **72.38** | **80.49** | **76.22** | 4914 | +| all fields (macro avg.) | 77.61 | 79.08 | 78.26 | 4914 | Note: the "affiliation_linked" field above is a linking-aware metric (each author is paired with its gold counterpart and their attached affiliations compared). Its support column reports the number of articles the metric is computed @@ -130,17 +130,17 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 79.72 | 78.41 | 79.06 | 63265 | -| date | 96.01 | 93.88 | 94.93 | 63662 | -| first_author | 94.86 | 93.26 | 94.05 | 63265 | -| inTitle | 95.52 | 94.27 | 94.89 | 63213 | +| authors | 79.71 | 78.25 | 78.97 | 63265 | +| date | 96.01 | 93.69 | 94.84 | 63662 | +| first_author | 94.85 | 93.08 | 93.95 | 63265 | +| inTitle | 95.52 | 94.09 | 94.8 | 63213 | | issue | 1.52 | 81.25 | 2.99 | 16 | -| page | 95.73 | 94.88 | 95.3 | 53375 | -| title | 90.27 | 90.59 | 90.43 | 62044 | -| volume | 97.83 | 98.28 | 98.06 | 61049 | +| page | 95.73 | 94.68 | 95.2 | 53375 | +| title | 90.28 | 90.41 | 90.34 | 62044 | +| volume | 97.82 | 98.09 | 97.96 | 61049 | | | | | | | -| **all fields (micro avg.)** | **92.59** | **91.84** | **92.22** | 429889 | -| all fields (macro avg.) | 81.43 | 90.6 | 81.21 | 429889 | +| **all fields (micro avg.)** | **92.59** | **91.66** | **92.12** | 429889 | +| all fields (macro avg.) | 81.43 | 90.44 | 81.13 | 429889 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -148,17 +148,17 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 79.86 | 78.55 | 79.2 | 63265 | -| date | 96.01 | 93.88 | 94.93 | 63662 | -| first_author | 94.94 | 93.35 | 94.14 | 63265 | -| inTitle | 96.01 | 94.74 | 95.37 | 63213 | +| authors | 79.85 | 78.38 | 79.11 | 63265 | +| date | 96.01 | 93.69 | 94.84 | 63662 | +| first_author | 94.93 | 93.16 | 94.04 | 63265 | +| inTitle | 96 | 94.56 | 95.28 | 63213 | | issue | 1.52 | 81.25 | 2.99 | 16 | -| page | 95.73 | 94.88 | 95.3 | 53375 | -| title | 95.93 | 96.26 | 96.09 | 62044 | -| volume | 97.83 | 98.28 | 98.06 | 61049 | +| page | 95.73 | 94.68 | 95.2 | 53375 | +| title | 95.93 | 96.08 | 96.01 | 62044 | +| volume | 97.82 | 98.09 | 97.96 | 61049 | | | | | | | -| **all fields (micro avg.)** | **93.52** | **92.76** | **93.14** | 429889 | -| all fields (macro avg.) | 82.23 | 91.4 | 82.01 | 429889 | +| **all fields (micro avg.)** | **93.52** | **92.58** | **93.05** | 429889 | +| all fields (macro avg.) | 82.22 | 91.24 | 81.93 | 429889 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) @@ -166,17 +166,17 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 93.49 | 91.95 | 92.72 | 63265 | -| date | 96.01 | 93.88 | 94.93 | 63662 | -| first_author | 95.38 | 93.78 | 94.57 | 63265 | -| inTitle | 96.62 | 95.35 | 95.98 | 63213 | +| authors | 93.49 | 91.77 | 92.62 | 63265 | +| date | 96.01 | 93.69 | 94.84 | 63662 | +| first_author | 95.38 | 93.59 | 94.48 | 63265 | +| inTitle | 96.62 | 95.16 | 95.88 | 63213 | | issue | 1.52 | 81.25 | 2.99 | 16 | -| page | 95.73 | 94.88 | 95.3 | 53375 | -| title | 97.69 | 98.03 | 97.86 | 62044 | -| volume | 97.83 | 98.28 | 98.06 | 61049 | +| page | 95.73 | 94.68 | 95.2 | 53375 | +| title | 97.69 | 97.84 | 97.77 | 62044 | +| volume | 97.82 | 98.09 | 97.96 | 61049 | | | | | | | -| **all fields (micro avg.)** | **95.92** | **95.14** | **95.53** | 429889 | -| all fields (macro avg.) | 84.28 | 93.42 | 84.05 | 429889 | +| **all fields (micro avg.)** | **95.92** | **94.96** | **95.44** | 429889 | +| all fields (macro avg.) | 84.28 | 93.26 | 83.97 | 429889 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -184,52 +184,52 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 87.02 | 85.59 | 86.3 | 63265 | -| date | 96.01 | 93.88 | 94.93 | 63662 | -| first_author | 94.87 | 93.28 | 94.07 | 63265 | -| inTitle | 96.02 | 94.76 | 95.38 | 63213 | +| authors | 87.01 | 85.42 | 86.2 | 63265 | +| date | 96.01 | 93.69 | 94.84 | 63662 | +| first_author | 94.86 | 93.09 | 93.97 | 63265 | +| inTitle | 96.02 | 94.57 | 95.29 | 63213 | | issue | 1.52 | 81.25 | 2.99 | 16 | -| page | 95.73 | 94.88 | 95.3 | 53375 | -| title | 97.53 | 97.87 | 97.7 | 62044 | -| volume | 97.83 | 98.28 | 98.06 | 61049 | +| page | 95.73 | 94.68 | 95.2 | 53375 | +| title | 97.53 | 97.69 | 97.61 | 62044 | +| volume | 97.82 | 98.09 | 97.96 | 61049 | | | | | | | -| **all fields (micro avg.)** | **94.79** | **94.02** | **94.41** | 429889 | -| all fields (macro avg.) | 83.32 | 92.47 | 83.09 | 429889 | +| **all fields (micro avg.)** | **94.79** | **93.84** | **94.31** | 429889 | +| all fields (macro avg.) | 83.31 | 92.31 | 83.01 | 429889 | #### Instance-level results ``` Total expected instances: 63664 -Total extracted instances: 65092 -Total correct instances: 41810 (strict) -Total correct instances: 44612 (soft) -Total correct instances: 52284 (Levenshtein) -Total correct instances: 48804 (RatcliffObershelp) +Total extracted instances: 64961 +Total correct instances: 41722 (strict) +Total correct instances: 44520 (soft) +Total correct instances: 52180 (Levenshtein) +Total correct instances: 48704 (RatcliffObershelp) Instance-level precision: 64.23 (strict) -Instance-level precision: 68.54 (soft) -Instance-level precision: 80.32 (Levenshtein) -Instance-level precision: 74.98 (RatcliffObershelp) +Instance-level precision: 68.53 (soft) +Instance-level precision: 80.33 (Levenshtein) +Instance-level precision: 74.97 (RatcliffObershelp) -Instance-level recall: 65.67 (strict) -Instance-level recall: 70.07 (soft) -Instance-level recall: 82.12 (Levenshtein) -Instance-level recall: 76.66 (RatcliffObershelp) +Instance-level recall: 65.53 (strict) +Instance-level recall: 69.93 (soft) +Instance-level recall: 81.96 (Levenshtein) +Instance-level recall: 76.5 (RatcliffObershelp) -Instance-level f-score: 64.94 (strict) -Instance-level f-score: 69.3 (soft) -Instance-level f-score: 81.21 (Levenshtein) -Instance-level f-score: 75.81 (RatcliffObershelp) +Instance-level f-score: 64.87 (strict) +Instance-level f-score: 69.22 (soft) +Instance-level f-score: 81.14 (Levenshtein) +Instance-level f-score: 75.73 (RatcliffObershelp) -Matching 1 : 58590 +Matching 1 : 58480 -Matching 2 : 959 +Matching 2 : 953 Matching 3 : 1235 -Matching 4 : 391 +Matching 4 : 389 -Total matches : 61175 +Total matches : 61057 ``` #### Citation context resolution @@ -237,17 +237,17 @@ Total matches : 61175 ``` Total expected references: 63664 - 64.7 references per article -Total predicted references: 65092 - 66.15 references per article +Total predicted references: 64961 - 66.02 references per article Total expected citation contexts: 109022 - 110.79 citation contexts per article -Total predicted citation contexts: 99898 - 101.52 citation contexts per article +Total predicted citation contexts: 98096 - 99.69 citation contexts per article -Total correct predicted citation contexts: 96019 - 97.58 citation contexts per article -Total wrong predicted citation contexts: 3879 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) +Total correct predicted citation contexts: 94362 - 95.9 citation contexts per article +Total wrong predicted citation contexts: 3734 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) -Precision citation contexts: 96.12 -Recall citation contexts: 88.07 -fscore citation contexts: 91.92 +Precision citation contexts: 96.19 +Recall citation contexts: 86.55 +fscore citation contexts: 91.12 ``` ## Fulltext structures @@ -265,17 +265,17 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| availability_stmt | 25.37 | 26.67 | 26 | 585 | -| figure_title | 0.07 | 0.02 | 0.03 | 31718 | -| funding_stmt | 6.04 | 26.49 | 9.84 | 921 | -| reference_citation | 57.12 | 55.98 | 56.54 | 108949 | -| reference_figure | 58.43 | 51.08 | 54.51 | 68926 | -| reference_table | 71.35 | 73.41 | 72.37 | 2381 | -| section_title | 83.33 | 77.3 | 80.2 | 21831 | +| availability_stmt | 25.28 | 26.67 | 25.96 | 585 | +| figure_title | 0.1 | 0.03 | 0.05 | 31718 | +| funding_stmt | 6.24 | 28.77 | 10.26 | 921 | +| reference_citation | 57.07 | 54.91 | 55.97 | 108949 | +| reference_figure | 59.47 | 50.95 | 54.88 | 68926 | +| reference_table | 72.22 | 72.07 | 72.15 | 2381 | +| section_title | 82.21 | 77.85 | 79.97 | 21831 | | table_title | 0 | 0 | 0 | 1925 | | | | | | | -| **all fields (micro avg.)** | **56.16** | **48.57** | **52.09** | 237236 | -| all fields (macro avg.) | 37.71 | 38.87 | 37.44 | 237236 | +| **all fields (micro avg.)** | **56.27** | **48.09** | **51.86** | 237236 | +| all fields (macro avg.) | 37.82 | 38.91 | 37.4 | 237236 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -283,26 +283,27 @@ Evaluation on 984 random PDF files out of 982 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| availability_stmt | 36.1 | 37.95 | 37 | 585 | -| figure_title | 49.82 | 16.06 | 24.29 | 31718 | -| funding_stmt | 6.04 | 26.49 | 9.84 | 921 | -| reference_citation | 93.68 | 91.81 | 92.74 | 108949 | -| reference_figure | 58.71 | 51.33 | 54.77 | 68926 | -| reference_table | 71.43 | 73.5 | 72.45 | 2381 | -| section_title | 84.38 | 78.26 | 81.21 | 21831 | -| table_title | 95.08 | 28.1 | 43.38 | 1925 | +| availability_stmt | 35.82 | 37.78 | 36.77 | 585 | +| figure_title | 49.4 | 16.15 | 24.34 | 31718 | +| funding_stmt | 6.24 | 28.77 | 10.26 | 921 | +| reference_citation | 93.85 | 90.3 | 92.04 | 108949 | +| reference_figure | 59.77 | 51.21 | 55.16 | 68926 | +| reference_table | 72.31 | 72.15 | 72.23 | 2381 | +| section_title | 83.22 | 78.81 | 80.96 | 21831 | +| table_title | 93.08 | 27.95 | 42.99 | 1925 | | | | | | | -| **all fields (micro avg.)** | **78.15** | **67.59** | **72.48** | 237236 | -| all fields (macro avg.) | 61.9 | 50.44 | 51.96 | 237236 | +| **all fields (micro avg.)** | **78.3** | **66.91** | **72.16** | 237236 | +| all fields (macro avg.) | 61.71 | 50.39 | 51.84 | 237236 | **Document-level ratio results** -| label | precision | recall | f1 | support | -|-----------------------------|-----------|---------|----------|---------| -| availability_stmt | 93.61 | 105.13 | 99.03 | 585 | -| | | | | | -| **all fields (micro avg.)** | **93.61** | **100** | **96.7** | 585 | -| all fields (macro avg.) | 93.61 | 100 | 99.03 | 585 | +| label | precision | recall | f1 | support | +|-----------------------------|-----------|---------|-----------|---------| +| availability_stmt | 93.48 | 105.47 | 99.12 | 585 | +| | | | | | +| **all fields (micro avg.)** | **93.48** | **100** | **96.63** | 585 | +| all fields (macro avg.) | 93.48 | 100 | 99.12 | 585 | + +Evaluation metrics produced in 189.013 seconds -Evaluation metrics produced in 233.867 seconds diff --git a/doc/benchmarks/Benchmarking-plos.md b/doc/benchmarks/Benchmarking-plos.md index abe327f39d..f2defa4a4e 100644 --- a/doc/benchmarks/Benchmarking-plos.md +++ b/doc/benchmarks/Benchmarking-plos.md @@ -48,7 +48,7 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | keywords | 0 | 0 | 0 | 0 | | title | 95.18 | 94.7 | 94.94 | 1000 | | | | | | | -| **all fields (micro avg.)** | **24.02** | **25.18** | **24.59** | 4861 | +| **all fields (micro avg.)** | **24.02** | **25.18** | **24.58** | 4861 | | all fields (macro avg.) | 61.27 | 61.26 | 61.26 | 4861 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -58,13 +58,13 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| | abstract | 49.34 | 50.94 | 50.13 | 960 | -| affiliation_linked | 73.43 | 78.47 | 75.87 | 963 | +| affiliation_linked | 73.43 | 78.49 | 75.87 | 963 | | authors | 98.97 | 98.97 | 98.97 | 969 | | first_author | 99.17 | 99.17 | 99.17 | 969 | | keywords | 0 | 0 | 0 | 0 | | title | 98.79 | 98.3 | 98.55 | 1000 | | | | | | | -| **all fields (micro avg.)** | **77.52** | **81.27** | **79.35** | 4861 | +| **all fields (micro avg.)** | **77.52** | **81.28** | **79.36** | 4861 | | all fields (macro avg.) | 83.94 | 85.17 | 84.54 | 4861 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) @@ -74,13 +74,13 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|--------|---------| | abstract | 75.28 | 77.71 | 76.47 | 960 | -| affiliation_linked | 78.35 | 83.74 | 80.95 | 963 | +| affiliation_linked | 78.35 | 83.75 | 80.96 | 963 | | authors | 99.38 | 99.38 | 99.38 | 969 | | first_author | 99.28 | 99.28 | 99.28 | 969 | | keywords | 0 | 0 | 0 | 0 | | title | 99.3 | 98.8 | 99.05 | 1000 | | | | | | | -| **all fields (micro avg.)** | **83.04** | **87.06** | **85** | 4861 | +| **all fields (micro avg.)** | **83.04** | **87.07** | **85** | 4861 | | all fields (macro avg.) | 90.32 | 91.78 | 91.03 | 4861 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -90,14 +90,14 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| | abstract | 64.78 | 66.88 | 65.81 | 960 | -| affiliation_linked | 76.05 | 81.27 | 78.57 | 963 | +| affiliation_linked | 76.07 | 81.31 | 78.61 | 963 | | authors | 99.28 | 99.28 | 99.28 | 969 | | first_author | 99.17 | 99.17 | 99.17 | 969 | | keywords | 0 | 0 | 0 | 0 | | title | 98.99 | 98.5 | 98.75 | 1000 | | | | | | | -| **all fields (micro avg.)** | **80.58** | **84.48** | **82.49** | 4861 | -| all fields (macro avg.) | 87.66 | 89.02 | 88.32 | 4861 | +| **all fields (micro avg.)** | **80.6** | **84.51** | **82.51** | 4861 | +| all fields (macro avg.) | 87.66 | 89.03 | 88.32 | 4861 | Note: the "affiliation_linked" field above is a linking-aware metric (each author is paired with its gold counterpart and their attached affiliations compared). Its support column reports the number of articles the metric is computed @@ -244,10 +244,10 @@ Total expected references: 48449 - 48.45 references per article Total predicted references: 47775 - 47.77 references per article Total expected citation contexts: 69755 - 69.75 citation contexts per article -Total predicted citation contexts: 73296 - 73.3 citation contexts per article +Total predicted citation contexts: 73298 - 73.3 citation contexts per article -Total correct predicted citation contexts: 57033 - 57.03 citation contexts per article -Total wrong predicted citation contexts: 16263 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) +Total correct predicted citation contexts: 57034 - 57.03 citation contexts per article +Total wrong predicted citation contexts: 16264 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) Precision citation contexts: 77.81 Recall citation contexts: 81.76 @@ -273,14 +273,14 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | conflict_stmt | 92.57 | 91.89 | 92.23 | 962 | | figure_title | 0.18 | 0.09 | 0.12 | 8943 | | funding_stmt | 5.79 | 31.12 | 9.77 | 1507 | -| reference_citation | 87.98 | 94.53 | 91.13 | 69741 | +| reference_citation | 87.97 | 94.53 | 91.13 | 69741 | | reference_figure | 74.11 | 85.79 | 79.52 | 11010 | | reference_table | 70.21 | 94.32 | 80.5 | 5159 | -| section_title | 72.98 | 66.28 | 69.47 | 17540 | +| section_title | 72.97 | 66.29 | 69.47 | 17540 | | table_title | 0 | 0 | 0 | 6092 | | | | | | | | **all fields (micro avg.)** | **74.56** | **76.95** | **75.74** | 121733 | -| all fields (macro avg.) | 51.28 | 58 | 53.4 | 121733 | +| all fields (macro avg.) | 51.28 | 58.01 | 53.4 | 121733 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -292,13 +292,13 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | conflict_stmt | 95.6 | 94.91 | 95.25 | 962 | | figure_title | 93.18 | 45.82 | 61.43 | 8943 | | funding_stmt | 7.33 | 39.35 | 12.35 | 1507 | -| reference_citation | 87.98 | 94.54 | 91.14 | 69741 | +| reference_citation | 87.97 | 94.54 | 91.14 | 69741 | | reference_figure | 74.35 | 86.07 | 79.78 | 11010 | | reference_table | 70.37 | 94.53 | 80.68 | 5159 | -| section_title | 78.78 | 71.56 | 75 | 17540 | +| section_title | 78.77 | 71.56 | 75 | 17540 | | table_title | 52.96 | 7.49 | 13.12 | 6092 | | | | | | | -| **all fields (micro avg.)** | **79.25** | **81.79** | **80.5** | 121733 | +| **all fields (micro avg.)** | **79.24** | **81.79** | **80.5** | 121733 | | all fields (macro avg.) | 71.79 | 68.92 | 66.06 | 121733 | **Document-level ratio results** @@ -311,5 +311,5 @@ Evaluation on 1000 random PDF files out of 998 PDF (ratio 1.0). | **all fields (micro avg.)** | **99.71** | **99.83** | **99.77** | 1741 | | all fields (macro avg.) | 99.69 | 99.89 | 99.79 | 1741 | -Evaluation metrics produced in 123.164 seconds +Evaluation metrics produced in 102.355 seconds diff --git a/doc/benchmarks/Benchmarking-pmc.md b/doc/benchmarks/Benchmarking-pmc.md index e75c94ad89..7e678d2b38 100644 --- a/doc/benchmarks/Benchmarking-pmc.md +++ b/doc/benchmarks/Benchmarking-pmc.md @@ -48,7 +48,7 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | keywords | 63.19 | 61.09 | 62.12 | 1380 | | title | 84.4 | 84.1 | 84.25 | 1943 | | | | | | | -| **all fields (micro avg.)** | **30.2** | **31.45** | **30.81** | 11043 | +| **all fields (micro avg.)** | **30.21** | **31.45** | **30.82** | 11043 | | all fields (macro avg.) | 59.03 | 58.57 | 58.79 | 11043 | #### Soft Matching (ignoring punctuation, case and space characters mismatches) @@ -58,13 +58,13 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| | abstract | 62.96 | 61.9 | 62.43 | 1911 | -| affiliation_linked | 61.65 | 66.62 | 64.04 | 1927 | +| affiliation_linked | 61.66 | 66.61 | 64.04 | 1927 | | authors | 94.85 | 94.8 | 94.82 | 1941 | | first_author | 97.16 | 97.11 | 97.14 | 1941 | | keywords | 71.14 | 68.77 | 69.93 | 1380 | | title | 92.05 | 91.71 | 91.88 | 1943 | | | | | | | -| **all fields (micro avg.)** | **71.22** | **74.15** | **72.66** | 11043 | +| **all fields (micro avg.)** | **71.23** | **74.14** | **72.65** | 11043 | | all fields (macro avg.) | 79.97 | 80.15 | 80.04 | 11043 | #### Levenshtein Matching (Minimum Levenshtein distance at 0.8) @@ -74,13 +74,13 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| | abstract | 89.84 | 88.33 | 89.08 | 1911 | -| affiliation_linked | 64.16 | 69.34 | 66.65 | 1927 | +| affiliation_linked | 64.17 | 69.32 | 66.64 | 1927 | | authors | 96.7 | 96.65 | 96.68 | 1941 | | first_author | 97.37 | 97.32 | 97.35 | 1941 | | keywords | 84.71 | 81.88 | 83.27 | 1380 | | title | 98.24 | 97.89 | 98.07 | 1943 | | | | | | | -| **all fields (micro avg.)** | **76.59** | **79.74** | **78.13** | 11043 | +| **all fields (micro avg.)** | **76.6** | **79.73** | **78.13** | 11043 | | all fields (macro avg.) | 88.5 | 88.57 | 88.51 | 11043 | #### Ratcliff/Obershelp Matching (Minimum Ratcliff/Obershelp similarity at 0.95) @@ -90,13 +90,13 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|----------|---------| | abstract | 85.74 | 84.3 | 85.01 | 1911 | -| affiliation_linked | 62.59 | 67.63 | 65.01 | 1927 | +| affiliation_linked | 62.59 | 67.62 | 65.01 | 1927 | | authors | 95.77 | 95.72 | 95.75 | 1941 | | first_author | 96.7 | 96.65 | 96.68 | 1941 | | keywords | 78.26 | 75.65 | 76.93 | 1380 | | title | 96.23 | 95.88 | 96.06 | 1943 | | | | | | | -| **all fields (micro avg.)** | **74.6** | **77.66** | **76.1** | 11043 | +| **all fields (micro avg.)** | **74.6** | **77.65** | **76.1** | 11043 | | all fields (macro avg.) | 85.88 | 85.97 | 85.91 | 11043 | Note: the "affiliation_linked" field above is a linking-aware metric (each author is paired with its gold counterpart @@ -188,7 +188,7 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| authors | 85.42 | 78.03 | 81.56 | 85778 | +| authors | 85.42 | 78.04 | 81.56 | 85778 | | date | 94.56 | 83.09 | 88.46 | 87067 | | first_author | 88.98 | 81.26 | 84.95 | 85778 | | inTitle | 82.67 | 80.63 | 81.63 | 81007 | @@ -204,11 +204,11 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). ``` Total expected instances: 90125 -Total extracted instances: 85511 +Total extracted instances: 85510 Total correct instances: 38667 (strict) Total correct instances: 50686 (soft) -Total correct instances: 55503 (Levenshtein) -Total correct instances: 52059 (RatcliffObershelp) +Total correct instances: 55504 (Levenshtein) +Total correct instances: 52060 (RatcliffObershelp) Instance-level precision: 45.22 (strict) Instance-level precision: 59.27 (soft) @@ -217,7 +217,7 @@ Instance-level precision: 60.88 (RatcliffObershelp) Instance-level recall: 42.9 (strict) Instance-level recall: 56.24 (soft) -Instance-level recall: 61.58 (Levenshtein) +Instance-level recall: 61.59 (Levenshtein) Instance-level recall: 57.76 (RatcliffObershelp) Instance-level f-score: 44.03 (strict) @@ -227,9 +227,9 @@ Instance-level f-score: 59.28 (RatcliffObershelp) Matching 1 : 67606 -Matching 2 : 3944 +Matching 2 : 3945 -Matching 3 : 1784 +Matching 3 : 1783 Matching 4 : 661 @@ -241,13 +241,13 @@ Total matches : 73995 ``` Total expected references: 90125 - 46.38 references per article -Total predicted references: 85511 - 44.01 references per article +Total predicted references: 85510 - 44.01 references per article Total expected citation contexts: 139835 - 71.97 citation contexts per article -Total predicted citation contexts: 114997 - 59.19 citation contexts per article +Total predicted citation contexts: 115000 - 59.19 citation contexts per article -Total correct predicted citation contexts: 97227 - 50.04 citation contexts per article -Total wrong predicted citation contexts: 17770 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) +Total correct predicted citation contexts: 97231 - 50.04 citation contexts per article +Total wrong predicted citation contexts: 17769 (wrong callout matching, callout missing in NLM, or matching with a bib. ref. not aligned with a bib.ref. in NLM) Precision citation contexts: 84.55 Recall citation contexts: 69.53 @@ -269,11 +269,11 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| figure_title | 32.03 | 26.7 | 29.12 | 7281 | +| figure_title | 31.99 | 26.67 | 29.09 | 7281 | | reference_citation | 58.11 | 58.84 | 58.47 | 134196 | -| reference_figure | 60.58 | 68.28 | 64.2 | 19330 | -| reference_table | 82.9 | 89.64 | 86.14 | 7327 | -| section_title | 72.46 | 67.69 | 69.99 | 27619 | +| reference_figure | 60.6 | 68.29 | 64.22 | 19330 | +| reference_table | 82.91 | 89.64 | 86.14 | 7327 | +| section_title | 72.48 | 67.69 | 70 | 27619 | | table_title | 67.72 | 49.66 | 57.3 | 3971 | | | | | | | | **all fields (micro avg.)** | **60.56** | **60.75** | **60.65** | 199724 | @@ -285,15 +285,15 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | label | precision | recall | f1 | support | |-----------------------------|-----------|-----------|-----------|---------| -| figure_title | 79.9 | 66.61 | 72.65 | 7281 | -| reference_citation | 62.4 | 63.18 | 62.79 | 134196 | -| reference_figure | 61.08 | 68.84 | 64.72 | 19330 | -| reference_table | 83.06 | 89.82 | 86.31 | 7327 | -| section_title | 77.85 | 72.73 | 75.2 | 27619 | +| figure_title | 79.92 | 66.63 | 72.67 | 7281 | +| reference_citation | 62.4 | 63.18 | 62.78 | 134196 | +| reference_figure | 61.09 | 68.85 | 64.74 | 19330 | +| reference_table | 83.07 | 89.82 | 86.31 | 7327 | +| section_title | 77.88 | 72.73 | 75.21 | 27619 | | table_title | 94.3 | 69.15 | 79.79 | 3971 | | | | | | | -| **all fields (micro avg.)** | **66.05** | **66.27** | **66.16** | 199724 | -| all fields (macro avg.) | 76.43 | 71.72 | 73.58 | 199724 | +| **all fields (micro avg.)** | **66.06** | **66.27** | **66.16** | 199724 | +| all fields (macro avg.) | 76.44 | 71.72 | 73.59 | 199724 | **Document-level ratio results** @@ -303,6 +303,6 @@ Evaluation on 1943 random PDF files out of 1941 PDF (ratio 1.0). | **all fields (micro avg.)** | **0** | **0** | **0** | 0 | | all fields (macro avg.) | 0 | 0 | 0 | 0 | -Evaluation metrics produced in 285.628 seconds +Evaluation metrics produced in 162.688 seconds diff --git a/grobid-core/src/main/java/org/grobid/core/document/DocumentSource.java b/grobid-core/src/main/java/org/grobid/core/document/DocumentSource.java index c0a572b9b4..3b363b34cc 100644 --- a/grobid-core/src/main/java/org/grobid/core/document/DocumentSource.java +++ b/grobid-core/src/main/java/org/grobid/core/document/DocumentSource.java @@ -42,6 +42,21 @@ public class DocumentSource { private static final int KILLED_DUE_2_TIMEOUT = 143; private static final int MISSING_LIBXML2 = 127; private static final int MISSING_PDFALTO = 126; + // Exit codes introduced by pdfalto 0.6.1. + // + // 5 means the ALTO was written correctly, but page streaming was disabled mid-run + // so peak memory was no longer bounded. It is a warning, not a failure: the output + // is complete and usable. Treating it as an error would reject perfectly good + // conversions, so it is handled as success with a logged warning. + private static final int PDFALTO_STREAMING_DISABLED = 5; + // 4 means writing/serializing the final ALTO failed - the output may be missing or + // truncated, so this is fatal. + private static final int PDFALTO_WRITE_FAILED = 4; + // 98 is an allocation failure: pdfalto now catches GMemException in main() as the + // other xpdf front-ends do, where before the exception escaped and the process died + // on SIGABRT. Without this case it falls through to the generic branch below and an + // out-of-memory in pdfalto is reported as BAD_INPUT_DATA, blaming the PDF. + private static final int PDFALTO_OUT_OF_MEMORY = 98; public static final int PDFALTO_FILES_AMOUNT_LIMIT = 5000; private File pdfFile; @@ -278,7 +293,13 @@ private File processPdfaltoThreadMode( throw new GrobidException("PDF to XML conversion timed out", GrobidExceptionStatus.TIMEOUT); } - if (worker.getExitStatus() != 0) { + if (worker.getExitStatus() == PDFALTO_STREAMING_DISABLED) { + LOGGER.warn( + "pdfalto could not buffer pages to disk while converting {}, so its peak memory was " + + "unbounded. The conversion succeeded. Point TMPDIR at a writable on-disk " + + "directory to re-enable streaming.", + pdfPath); + } else if (worker.getExitStatus() != 0) { String errorStreamContents = worker.getErrorStreamContents(); close(true, true, true); throw new GrobidException("PDF to XML conversion failed on pdf file " @@ -323,6 +344,19 @@ private File processPdfaltoServerMode(File pdfPath, File tmpPathXML, List + * Since 0.6.1, pdfalto streams large page DOMs to a scratch file to bound peak + * memory, and picks its location from {@code $TMPDIR}, falling back to + * {@code /tmp}. In containers {@code /tmp} is usually tmpfs, i.e. RAM: spilling + * there leaves peak memory unchanged and risks ENOSPC, which is exactly the case + * streaming exists to avoid. GROBID already has a configured on-disk temp + * directory ({@code grobid.temp}), so hand pdfalto that instead of letting it + * guess. + */ + static void setTempDirectory(ProcessBuilder builder) { + try { + File tempPath = GrobidProperties.getTempPath(); + if (tempPath != null && tempPath.isDirectory()) { + builder.environment().put("TMPDIR", tempPath.getAbsolutePath()); + } + } catch (Exception e) { + // Never let this stop the conversion: without TMPDIR pdfalto falls back to + // /tmp, which still works, it just may not bound memory as effectively. + LOGGER.warn("Could not set TMPDIR for the pdfalto process, falling back to its default", e); + } + } +} diff --git a/grobid-core/src/main/java/org/grobid/core/process/ProcessPdfToXml.java b/grobid-core/src/main/java/org/grobid/core/process/ProcessPdfToXml.java index 5916f40d1e..a488195036 100644 --- a/grobid-core/src/main/java/org/grobid/core/process/ProcessPdfToXml.java +++ b/grobid-core/src/main/java/org/grobid/core/process/ProcessPdfToXml.java @@ -38,6 +38,7 @@ public static Integer process(List cmd) { String message = "error message cannot be retrieved"; try { builder = new ProcessBuilder(cmd); + ProcessEnvironment.setTempDirectory(builder); builder.redirectErrorStream(true); process = builder.start(); diff --git a/grobid-core/src/main/java/org/grobid/core/process/ProcessRunner.java b/grobid-core/src/main/java/org/grobid/core/process/ProcessRunner.java index 8f674aeb30..07f277a18e 100755 --- a/grobid-core/src/main/java/org/grobid/core/process/ProcessRunner.java +++ b/grobid-core/src/main/java/org/grobid/core/process/ProcessRunner.java @@ -84,6 +84,7 @@ public void run() { process = null; try { ProcessBuilder builder = new ProcessBuilder(cmd); + ProcessEnvironment.setTempDirectory(builder); process = builder.start(); if (useStreamGobbler) { diff --git a/grobid-core/src/test/java/org/grobid/core/process/ProcessEnvironmentTest.java b/grobid-core/src/test/java/org/grobid/core/process/ProcessEnvironmentTest.java new file mode 100644 index 0000000000..7883ac6581 --- /dev/null +++ b/grobid-core/src/test/java/org/grobid/core/process/ProcessEnvironmentTest.java @@ -0,0 +1,48 @@ +/* + * Copyright 2008-2026 GROBID contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.grobid.core.process; + +import static org.hamcrest.CoreMatchers.is; +import static org.hamcrest.CoreMatchers.notNullValue; +import static org.junit.Assert.assertThat; + +import java.io.File; + +import org.junit.BeforeClass; +import org.junit.Test; + +import org.grobid.core.utilities.GrobidProperties; + +public class ProcessEnvironmentTest { + + @BeforeClass + public static void setInitialContext() throws Exception { + GrobidProperties.getInstance(); + } + + @Test + public void testSetTempDirectory_pointsPdfaltoAtGrobidTempPath() { + ProcessBuilder builder = new ProcessBuilder("true"); + ProcessEnvironment.setTempDirectory(builder); + + // pdfalto reads $TMPDIR to place its page-streaming scratch file; left unset it + // falls back to /tmp, which is tmpfs in a container and defeats the streaming. + String tmpdir = builder.environment().get("TMPDIR"); + assertThat(tmpdir, is(notNullValue())); + assertThat(new File(tmpdir), is(GrobidProperties.getTempPath().getAbsoluteFile())); + assertThat(new File(tmpdir).isDirectory(), is(true)); + } +} diff --git a/grobid-home/pdf2xml/lin-64/pdfalto b/grobid-home/pdf2xml/lin-64/pdfalto deleted file mode 100755 index ec2c5711a1..0000000000 Binary files a/grobid-home/pdf2xml/lin-64/pdfalto and /dev/null differ diff --git a/grobid-home/pdf2xml/lin-64/pdfalto_server b/grobid-home/pdf2xml/lin-64/pdfalto_server deleted file mode 100755 index ee99d24f16..0000000000 --- a/grobid-home/pdf2xml/lin-64/pdfalto_server +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/bash - -# Timeout. -timeout=20 # 20 seconds -# Interval between checks if the process is still alive. -interval=1 -# Delay between posting the SIGTERM signal and destroying the process by SIGKILL. -delay=0 - -command=${0:0:${#0}-7} - -# kill -0 pid Exit code indicates if a signal may be sent to $pid process. -( - ((t = timeout)) - - while ((t > 0)); do - sleep $interval - kill -0 $$ || exit 0 - ((t -= interval)) - done - - # Be nice, post SIGTERM first. - # The 'exit 0' below will be executed if any preceeding command fails. - kill -s SIGTERM $$ && kill -0 $$ || exit 0 - sleep $delay - kill -s SIGKILL $$ -) 2> /dev/null & - -exec $command $@ diff --git a/grobid-home/pdf2xml/mac-64/pdfalto b/grobid-home/pdf2xml/mac-64/pdfalto deleted file mode 100755 index 6414e7fa18..0000000000 Binary files a/grobid-home/pdf2xml/mac-64/pdfalto and /dev/null differ diff --git a/grobid-home/pdf2xml/mac-64/pdfalto_server b/grobid-home/pdf2xml/mac-64/pdfalto_server deleted file mode 100755 index 3d02dc6838..0000000000 --- a/grobid-home/pdf2xml/mac-64/pdfalto_server +++ /dev/null @@ -1,28 +0,0 @@ -#!/bin/bash - -# Timeout. -timeout=20 # 20 seconds -# Interval between checks if the process is still alive. -interval=1 -# Delay between posting the SIGTERM signal and destroying the process by SIGKILL. -delay=0 -command=${0:0:${#0}-7} - -# kill -0 pid Exit code indicates if a signal may be sent to $pid process. -( - ((t = timeout)) - - while ((t > 0)); do - sleep $interval - kill -0 $$ || exit 0 - ((t -= interval)) - done - - # Be nice, post SIGTERM first. - # The 'exit 0' below will be executed if any preceeding command fails. - kill -s SIGTERM $$ && kill -0 $$ || exit 0 - sleep $delay - kill -s SIGKILL $$ -) 2> /dev/null & - -exec $command $@ diff --git a/grobid-home/pdfalto/lin-64/pdfalto b/grobid-home/pdfalto/lin-64/pdfalto index 0cbe69b66e..02a939e80b 100755 Binary files a/grobid-home/pdfalto/lin-64/pdfalto and b/grobid-home/pdfalto/lin-64/pdfalto differ diff --git a/grobid-home/pdfalto/lin_arm-64/pdfalto b/grobid-home/pdfalto/lin_arm-64/pdfalto index 28a820ca5a..d10c752ce8 100755 Binary files a/grobid-home/pdfalto/lin_arm-64/pdfalto and b/grobid-home/pdfalto/lin_arm-64/pdfalto differ diff --git a/grobid-home/pdfalto/mac-64/pdfalto b/grobid-home/pdfalto/mac-64/pdfalto index 6956373065..1411ca723c 100755 Binary files a/grobid-home/pdfalto/mac-64/pdfalto and b/grobid-home/pdfalto/mac-64/pdfalto differ diff --git a/grobid-home/pdfalto/mac_arm-64/pdfalto b/grobid-home/pdfalto/mac_arm-64/pdfalto index defa8b178b..ac9ce5a831 100755 Binary files a/grobid-home/pdfalto/mac_arm-64/pdfalto and b/grobid-home/pdfalto/mac_arm-64/pdfalto differ diff --git a/grobid-home/pdfalto/win-32/cyggcc_s-1.dll b/grobid-home/pdfalto/win-32/cyggcc_s-1.dll deleted file mode 100755 index 953b32e0ba..0000000000 Binary files a/grobid-home/pdfalto/win-32/cyggcc_s-1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cygiconv-2.dll b/grobid-home/pdfalto/win-32/cygiconv-2.dll deleted file mode 100755 index 10298cf872..0000000000 Binary files a/grobid-home/pdfalto/win-32/cygiconv-2.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cyglzma-5.dll b/grobid-home/pdfalto/win-32/cyglzma-5.dll deleted file mode 100755 index 6cac90dace..0000000000 Binary files a/grobid-home/pdfalto/win-32/cyglzma-5.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cygstdc++-6.dll b/grobid-home/pdfalto/win-32/cygstdc++-6.dll deleted file mode 100755 index f4647bfc28..0000000000 Binary files a/grobid-home/pdfalto/win-32/cygstdc++-6.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cygwin1.dll b/grobid-home/pdfalto/win-32/cygwin1.dll deleted file mode 100644 index 732542272b..0000000000 Binary files a/grobid-home/pdfalto/win-32/cygwin1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cygxml2-2.dll b/grobid-home/pdfalto/win-32/cygxml2-2.dll deleted file mode 100755 index 7de5b1c99f..0000000000 Binary files a/grobid-home/pdfalto/win-32/cygxml2-2.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/cygz.dll b/grobid-home/pdfalto/win-32/cygz.dll deleted file mode 100755 index 5d1897488c..0000000000 Binary files a/grobid-home/pdfalto/win-32/cygz.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/pdfalto/cygwin1.dll b/grobid-home/pdfalto/win-32/pdfalto/cygwin1.dll deleted file mode 100644 index 966c6892f7..0000000000 Binary files a/grobid-home/pdfalto/win-32/pdfalto/cygwin1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/pdfalto/pdfalto.bat b/grobid-home/pdfalto/win-32/pdfalto/pdfalto.bat deleted file mode 100644 index 8728dac6a9..0000000000 --- a/grobid-home/pdfalto/win-32/pdfalto/pdfalto.bat +++ /dev/null @@ -1,2 +0,0 @@ -SET PATH=%PATH%;%~dp0pdfalto_dlls -%~dp0pdfalto.exe %* \ No newline at end of file diff --git a/grobid-home/pdfalto/win-32/pdfalto/pdfalto.exe b/grobid-home/pdfalto/win-32/pdfalto/pdfalto.exe deleted file mode 100644 index 6aaee10bbb..0000000000 Binary files a/grobid-home/pdfalto/win-32/pdfalto/pdfalto.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.bat b/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.bat deleted file mode 100644 index a3434c628a..0000000000 --- a/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.bat +++ /dev/null @@ -1,2 +0,0 @@ -SET PATH=%PATH%;%~dp0pdfalto_dlls -%~dp0pdfalto_server.exe %* \ No newline at end of file diff --git a/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.exe b/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.exe deleted file mode 100644 index 6aaee10bbb..0000000000 Binary files a/grobid-home/pdfalto/win-32/pdfalto/pdfalto_server.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/pdftoxml.exe b/grobid-home/pdfalto/win-32/pdftoxml.exe deleted file mode 100755 index d0370476d3..0000000000 Binary files a/grobid-home/pdfalto/win-32/pdftoxml.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/pdftoxml_server.exe b/grobid-home/pdfalto/win-32/pdftoxml_server.exe deleted file mode 100755 index d0370476d3..0000000000 Binary files a/grobid-home/pdfalto/win-32/pdftoxml_server.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-32/xpdfrc b/grobid-home/pdfalto/win-32/xpdfrc deleted file mode 100644 index a9bf2236ac..0000000000 --- a/grobid-home/pdfalto/win-32/xpdfrc +++ /dev/null @@ -1,70 +0,0 @@ -#----- begin Arabic support package (2011-aug-15) -unicodeMap ISO-8859-6 ../languages/xpdf-arabic/ISO-8859-6.unicodeMap -#----- end Arabic support package -#----- begin Chinese Simplified support package (2011-sep-02) -cidToUnicode Adobe-GB1 ../languages/xpdf-chinese-simplified/Adobe-GB1.cidToUnicode -unicodeMap ISO-2022-CN ../languages/xpdf-chinese-simplified/ISO-2022-CN.unicodeMap -unicodeMap EUC-CN ../languages/xpdf-chinese-simplified/EUC-CN.unicodeMap -unicodeMap GBK ../languages/xpdf-chinese-simplified/GBK.unicodeMap -cMapDir Adobe-GB1 ../languages/xpdf-chinese-simplified/CMap -toUnicodeDir ../languages/xpdf-chinese-simplified/CMap -#fontFileCC Adobe-GB1 /usr/..../NotoSansCJKsc-Regular.otf -#----- end Chinese Simplified support package -#----- begin Chinese Traditional support package (2011-sep-02) -cidToUnicode Adobe-CNS1 ../languages/xpdf-chinese-traditional/Adobe-CNS1.cidToUnicode -unicodeMap Big5 ../languages/xpdf-chinese-traditional/Big5.unicodeMap -unicodeMap Big5ascii ../languages/xpdf-chinese-traditional/Big5ascii.unicodeMap -cMapDir Adobe-CNS1 ../languages/xpdf-chinese-traditional/CMap -toUnicodeDir ../languages/xpdf-chinese-traditional/CMap -#fontFileCC Adobe-CNS1 /usr/..../NotoSansCJKtc-Regular.otf" -#----- end Chinese Traditional support package -#----- begin Cyrillic support package (2011-aug-15) -nameToUnicode ../languages/xpdf-cyrillic/Bulgarian.nameToUnicode -unicodeMap KOI8-R ../languages/xpdf-cyrillic/KOI8-R.unicodeMap -#----- end Cyrillic support package -#----- begin Greek support package (2011-aug-15) -nameToUnicode ../languages/xpdf-greek/Greek.nameToUnicode -unicodeMap ISO-8859-7 ../languages/xpdf-greek/ISO-8859-7.unicodeMap -#----- end Greek support package -#----- begin Hebrew support package (2011-aug-15) -unicodeMap ISO-8859-8 ../languages/xpdf-hebrew/ISO-8859-8.unicodeMap -unicodeMap Windows-1255 ../languages/xpdf-hebrew/Windows-1255.unicodeMap -#----- end Hebrew support package -#----- begin Japanese support package (2011-sep-02) -cidToUnicode Adobe-Japan1 ../languages/xpdf-japanese/Adobe-Japan1.cidToUnicode -unicodeMap ISO-2022-JP ../languages/xpdf-japanese/ISO-2022-JP.unicodeMap -unicodeMap EUC-JP ../languages/xpdf-japanese/EUC-JP.unicodeMap -unicodeMap Shift-JIS ../languages/xpdf-japanese/Shift-JIS.unicodeMap -cMapDir Adobe-Japan1 ../languages/xpdf-japanese/CMap -toUnicodeDir ../languages/xpdf-japanese/CMap -#fontFileCC Adobe-Japan1 /usr/..../NotoSansCJKjp-Regular.otf -#----- end Japanese support package -#----- begin Korean support package (2011-sep-02) -cidToUnicode Adobe-Korea1 ../languages/xpdf-korean/Adobe-Korea1.cidToUnicode -cidToUnicode Adobe-KR ../languages/xpdf-korean/Adobe-KR.cidToUnicode -unicodeMap ISO-2022-KR ../languages/xpdf-korean/ISO-2022-KR.unicodeMap -cMapDir Adobe-Korea1 ../languages/xpdf-korean/CMap -cMapDir Adobe-KR ../languages/xpdf-korean/CMap -toUnicodeDir ../languages/xpdf-korean/CMap -#fontFileCC Adobe-Korea1 /usr/..../NotoSansCJKkr-Regular.otf -#fontFileCC Adobe-KR /usr/..../NotoSansCJKkr-Regular.otf -#----- end Korean support package -#----- begin Latin2 support package (2011-aug-15) -unicodeMap Latin2 ../languages/xpdf-latin2/Latin2.unicodeMap -#----- end Latin2 support package -#----- begin Thai support package (2011-aug-15) -nameToUnicode ../languages/xpdf-thai/Thai.nameToUnicode -unicodeMap TIS-620 ../languages/xpdf-thai/TIS-620.unicodeMap -#----- end Thai support package -#----- begin Turkish support package (2011-aug-15) -unicodeMap ISO-8859-9 ../languages/xpdf-turkish/ISO-8859-9.unicodeMap -#----- end Turkish support package -#----- begin oldstyle support package (2024-dec-31) -nameToUnicode ../languages/xpdf-others/oldstyle.nameToUnicode -nameToUnicode ../languages/xpdf-others/taboldstyle.nameToUnicode -nameToUnicode ../languages/xpdf-others/ligatures.nameToUnicode -nameToUnicode ../languages/xpdf-others/fitted.nameToUnicode -nameToUnicode ../languages/xpdf-others/others.nameToUnicode -nameToUnicode ../languages/xpdf-others/sc.nameToUnicode -unicodeRemapping ../languages/xpdf-others/symbols.unicodeRemapping -#----- end oldstyle support package diff --git a/grobid-home/pdfalto/win-64/cyggcc_s-1.dll b/grobid-home/pdfalto/win-64/cyggcc_s-1.dll deleted file mode 100755 index 953b32e0ba..0000000000 Binary files a/grobid-home/pdfalto/win-64/cyggcc_s-1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cygiconv-2.dll b/grobid-home/pdfalto/win-64/cygiconv-2.dll deleted file mode 100755 index 10298cf872..0000000000 Binary files a/grobid-home/pdfalto/win-64/cygiconv-2.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cyglzma-5.dll b/grobid-home/pdfalto/win-64/cyglzma-5.dll deleted file mode 100755 index 6cac90dace..0000000000 Binary files a/grobid-home/pdfalto/win-64/cyglzma-5.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cygstdc++-6.dll b/grobid-home/pdfalto/win-64/cygstdc++-6.dll deleted file mode 100755 index f4647bfc28..0000000000 Binary files a/grobid-home/pdfalto/win-64/cygstdc++-6.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cygwin1.dll b/grobid-home/pdfalto/win-64/cygwin1.dll deleted file mode 100644 index 732542272b..0000000000 Binary files a/grobid-home/pdfalto/win-64/cygwin1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cygxml2-2.dll b/grobid-home/pdfalto/win-64/cygxml2-2.dll deleted file mode 100755 index 7de5b1c99f..0000000000 Binary files a/grobid-home/pdfalto/win-64/cygxml2-2.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/cygz.dll b/grobid-home/pdfalto/win-64/cygz.dll deleted file mode 100755 index 5d1897488c..0000000000 Binary files a/grobid-home/pdfalto/win-64/cygz.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/pdfalto/cygwin1.dll b/grobid-home/pdfalto/win-64/pdfalto/cygwin1.dll deleted file mode 100644 index 0e5a6c4be4..0000000000 Binary files a/grobid-home/pdfalto/win-64/pdfalto/cygwin1.dll and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/pdfalto/pdfalto.bat b/grobid-home/pdfalto/win-64/pdfalto/pdfalto.bat deleted file mode 100644 index 8728dac6a9..0000000000 --- a/grobid-home/pdfalto/win-64/pdfalto/pdfalto.bat +++ /dev/null @@ -1,2 +0,0 @@ -SET PATH=%PATH%;%~dp0pdfalto_dlls -%~dp0pdfalto.exe %* \ No newline at end of file diff --git a/grobid-home/pdfalto/win-64/pdfalto/pdfalto.exe b/grobid-home/pdfalto/win-64/pdfalto/pdfalto.exe deleted file mode 100755 index f350035760..0000000000 Binary files a/grobid-home/pdfalto/win-64/pdfalto/pdfalto.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.bat b/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.bat deleted file mode 100644 index a3434c628a..0000000000 --- a/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.bat +++ /dev/null @@ -1,2 +0,0 @@ -SET PATH=%PATH%;%~dp0pdfalto_dlls -%~dp0pdfalto_server.exe %* \ No newline at end of file diff --git a/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.exe b/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.exe deleted file mode 100755 index 6789b9ad7e..0000000000 Binary files a/grobid-home/pdfalto/win-64/pdfalto/pdfalto_server.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/pdftoxml.exe b/grobid-home/pdfalto/win-64/pdftoxml.exe deleted file mode 100755 index d0370476d3..0000000000 Binary files a/grobid-home/pdfalto/win-64/pdftoxml.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/pdftoxml_server.exe b/grobid-home/pdfalto/win-64/pdftoxml_server.exe deleted file mode 100755 index d0370476d3..0000000000 Binary files a/grobid-home/pdfalto/win-64/pdftoxml_server.exe and /dev/null differ diff --git a/grobid-home/pdfalto/win-64/xpdfrc b/grobid-home/pdfalto/win-64/xpdfrc deleted file mode 100644 index a9bf2236ac..0000000000 --- a/grobid-home/pdfalto/win-64/xpdfrc +++ /dev/null @@ -1,70 +0,0 @@ -#----- begin Arabic support package (2011-aug-15) -unicodeMap ISO-8859-6 ../languages/xpdf-arabic/ISO-8859-6.unicodeMap -#----- end Arabic support package -#----- begin Chinese Simplified support package (2011-sep-02) -cidToUnicode Adobe-GB1 ../languages/xpdf-chinese-simplified/Adobe-GB1.cidToUnicode -unicodeMap ISO-2022-CN ../languages/xpdf-chinese-simplified/ISO-2022-CN.unicodeMap -unicodeMap EUC-CN ../languages/xpdf-chinese-simplified/EUC-CN.unicodeMap -unicodeMap GBK ../languages/xpdf-chinese-simplified/GBK.unicodeMap -cMapDir Adobe-GB1 ../languages/xpdf-chinese-simplified/CMap -toUnicodeDir ../languages/xpdf-chinese-simplified/CMap -#fontFileCC Adobe-GB1 /usr/..../NotoSansCJKsc-Regular.otf -#----- end Chinese Simplified support package -#----- begin Chinese Traditional support package (2011-sep-02) -cidToUnicode Adobe-CNS1 ../languages/xpdf-chinese-traditional/Adobe-CNS1.cidToUnicode -unicodeMap Big5 ../languages/xpdf-chinese-traditional/Big5.unicodeMap -unicodeMap Big5ascii ../languages/xpdf-chinese-traditional/Big5ascii.unicodeMap -cMapDir Adobe-CNS1 ../languages/xpdf-chinese-traditional/CMap -toUnicodeDir ../languages/xpdf-chinese-traditional/CMap -#fontFileCC Adobe-CNS1 /usr/..../NotoSansCJKtc-Regular.otf" -#----- end Chinese Traditional support package -#----- begin Cyrillic support package (2011-aug-15) -nameToUnicode ../languages/xpdf-cyrillic/Bulgarian.nameToUnicode -unicodeMap KOI8-R ../languages/xpdf-cyrillic/KOI8-R.unicodeMap -#----- end Cyrillic support package -#----- begin Greek support package (2011-aug-15) -nameToUnicode ../languages/xpdf-greek/Greek.nameToUnicode -unicodeMap ISO-8859-7 ../languages/xpdf-greek/ISO-8859-7.unicodeMap -#----- end Greek support package -#----- begin Hebrew support package (2011-aug-15) -unicodeMap ISO-8859-8 ../languages/xpdf-hebrew/ISO-8859-8.unicodeMap -unicodeMap Windows-1255 ../languages/xpdf-hebrew/Windows-1255.unicodeMap -#----- end Hebrew support package -#----- begin Japanese support package (2011-sep-02) -cidToUnicode Adobe-Japan1 ../languages/xpdf-japanese/Adobe-Japan1.cidToUnicode -unicodeMap ISO-2022-JP ../languages/xpdf-japanese/ISO-2022-JP.unicodeMap -unicodeMap EUC-JP ../languages/xpdf-japanese/EUC-JP.unicodeMap -unicodeMap Shift-JIS ../languages/xpdf-japanese/Shift-JIS.unicodeMap -cMapDir Adobe-Japan1 ../languages/xpdf-japanese/CMap -toUnicodeDir ../languages/xpdf-japanese/CMap -#fontFileCC Adobe-Japan1 /usr/..../NotoSansCJKjp-Regular.otf -#----- end Japanese support package -#----- begin Korean support package (2011-sep-02) -cidToUnicode Adobe-Korea1 ../languages/xpdf-korean/Adobe-Korea1.cidToUnicode -cidToUnicode Adobe-KR ../languages/xpdf-korean/Adobe-KR.cidToUnicode -unicodeMap ISO-2022-KR ../languages/xpdf-korean/ISO-2022-KR.unicodeMap -cMapDir Adobe-Korea1 ../languages/xpdf-korean/CMap -cMapDir Adobe-KR ../languages/xpdf-korean/CMap -toUnicodeDir ../languages/xpdf-korean/CMap -#fontFileCC Adobe-Korea1 /usr/..../NotoSansCJKkr-Regular.otf -#fontFileCC Adobe-KR /usr/..../NotoSansCJKkr-Regular.otf -#----- end Korean support package -#----- begin Latin2 support package (2011-aug-15) -unicodeMap Latin2 ../languages/xpdf-latin2/Latin2.unicodeMap -#----- end Latin2 support package -#----- begin Thai support package (2011-aug-15) -nameToUnicode ../languages/xpdf-thai/Thai.nameToUnicode -unicodeMap TIS-620 ../languages/xpdf-thai/TIS-620.unicodeMap -#----- end Thai support package -#----- begin Turkish support package (2011-aug-15) -unicodeMap ISO-8859-9 ../languages/xpdf-turkish/ISO-8859-9.unicodeMap -#----- end Turkish support package -#----- begin oldstyle support package (2024-dec-31) -nameToUnicode ../languages/xpdf-others/oldstyle.nameToUnicode -nameToUnicode ../languages/xpdf-others/taboldstyle.nameToUnicode -nameToUnicode ../languages/xpdf-others/ligatures.nameToUnicode -nameToUnicode ../languages/xpdf-others/fitted.nameToUnicode -nameToUnicode ../languages/xpdf-others/others.nameToUnicode -nameToUnicode ../languages/xpdf-others/sc.nameToUnicode -unicodeRemapping ../languages/xpdf-others/symbols.unicodeRemapping -#----- end oldstyle support package