Merge extinct logic into main to prevent them from bifurcating - #108

Merged

davidebbo merged 119 commits into

mainfrom

wikipedia-extractor

Jul 3, 2025

Commits on Nov 14, 2024

Initial logic to extract trees from Wikipedia
davidebbo
committed
7c5aa89
Big refactor, and switch to Dendropy
davidebbo
committed
f5dde95
Added basic support for wiki taxonomies
davidebbo
committed
8a949f1
Refactoring + added more Dinosaurs
davidebbo
committed
85a4e9e
Rename input .wikiclades folder
davidebbo
committed
1b9cc8c
Add command line support and fix comments
davidebbo
committed
5e7e1f0
Improve handling of upper case tokens
davidebbo
committed
4c2964f
Make cladogram test case insensitive
davidebbo
committed
a736800
Fix handling of links to images
davidebbo
committed
7039091
Improve .wikiclades syntax + line number as edge lengths for debugging
davidebbo
committed
13b530b
Fix WikiTaxonomyNode handling of invalid nodes
davidebbo
committed
b0f7428
Add support for adding a child to a parent
davidebbo
committed
6da9681
Add taxa exclusion support
davidebbo
committed
745cef1
Support $ROOT token + cleanup
davidebbo
committed
9b6642d
Ignore taxa with 2 or more spaces
davidebbo
committed
61ca47a
Add a few Dino clades
davidebbo
committed
de67569
Add Bohaiornithidae subtree
davidebbo
committed
856803e
Create helper to set up logging more easily
davidebbo
committed
dde7da0
New add_date_species script + bunch of refactoring
davidebbo
committed
05b5a5c
Add replacement support via => syntax
davidebbo
committed
e4bc019
Remove all dupliate taxa
davidebbo
committed
82be438
Add quoted token support to format_newick + tests
davidebbo
committed
4030919
Add taxon to page mapping functionality
davidebbo
committed
9325ff9
Save extracted newicks + store mapping there
davidebbo
committed
1439466
Use suppress_internal_node_taxa
davidebbo
committed
b4ca97c
Rename wiki_extractor to wiki_clade_extractor
davidebbo
committed
1a920d7
And entry points and package dependencies
davidebbo
committed
b30beec
range & species improvements + other fixes
davidebbo
committed
65e41a6
Corrected allow_shortened_binomial implementation
davidebbo
committed
c0e13dc
Fix range string parsing + support 'taxobox'
davidebbo
committed
4aa1c2b
Added Lessemsauridae
davidebbo
committed
327cc7c
Fix case where taxobox species is already binomial
davidebbo
committed
ecc784b
Incude taxa that start with page_title
davidebbo
committed
70ed26e
Refactor / cleanup
davidebbo
committed
f5edd82
mwparserfromhell>=0.6.6
davidebbo
committed
606267b
Improve comment support in format_newick
davidebbo
committed
2ba3a3b
Use node level comments for page override
davidebbo
committed
51b4275
Ignore 'earliest' range property
davidebbo
committed
4c1cf19
Support link/text combos, e.g. [[Mosasaurus]] hoffmannii
davidebbo
committed
191547e
Support nested templates in a limited way
davidebbo
committed
8bf3efd
Ignore duplicate taxa from the same cladogram
davidebbo
committed
80f9c20
Initial Reptiliomorpha tree
davidebbo
committed
fc0a9ff
Fix handling of quoted tokens with edge length
davidebbo
committed
0fb15ef
Added Carboniferous period names
davidebbo
committed
6d27991
Fix fossil range template parsing and handle bogus species names
davidebbo
committed
c1f96fa
Add a couple period names
davidebbo
committed
661f3e7
Ignore leaf nodes with no data
davidebbo
committed
d3473b5
Support recursive processing of .wikiclades files
davidebbo
committed
08aff2a
Handle BrokenPipeError when piping output
davidebbo
committed
3c764ab
Add Synapsida and Dinosauria as subtrees
davidebbo
committed
552e2e9
Avoid unwanted polytomies when node has no known range
davidebbo
committed
99588a9
Find an image in the wiki page
davidebbo
committed
e735cdf
Add 'redirect' entries when taxon != species name
davidebbo
committed
ca421a3
Add Stahleckeriidae
davidebbo
committed
b7a06a6
Add Diadectomorpha and Pantestudines
davidebbo
committed
2646ebd
Improve image handling in edge cases
davidebbo
committed
2147eb8
Fine tune binomial special casing
davidebbo
committed
19f4223
Strip() fossil range value
davidebbo
committed
c6de4fe
Fix Rhynchocephalia parenting
davidebbo
committed
51c3e10
Fix out of range error in WikiCladeNode class
davidebbo
committed
6db1aa3
Fix formatting and add missing dependency
davidebbo
committed
0b7b5c2
Update wikiclades files
davidebbo
committed
2a66f44
Ignore rank names in taxon names
davidebbo
committed
11ecd18
Remove new root's parent before making it the seed
davidebbo
committed
dc5547b
Improve warning
davidebbo
committed
665b719
Tweak paleoart detection
davidebbo
committed
b0a237f
Handle two types of bullet lists
davidebbo
committed
a899971
Save page id + tweak paleoart logic
davidebbo
committed
be8593b
Add Eutheria tree
davidebbo
committed
bf0b2ba
Don't include the file:/image: prefix for images
davidebbo
committed
79a35b2
Adjust the logic that ignores rank names
davidebbo
committed
d6c32cc
Add support for getting image from taxa data file
davidebbo
committed
be276da
Improve support for nested clade templates
davidebbo
committed
666002a
Add handling for special characters in cache file names
davidebbo
committed
b4dd10b
Handling for dual taxa names + add more rank names
davidebbo
committed
61e1b7a
Don't use the species name when there is no page
davidebbo
committed
7f81355
Expanded placentals
davidebbo
committed
39ee77b
Add Subgenus to rank list
davidebbo
committed
b4eafbe
Get the QID from the page wikicode
davidebbo
committed
61dff44
Improve wikiclade files
davidebbo
committed
9b00592
Favor restoration images over reconstruction
davidebbo
committed
ace620c
Fix URL encoding issue
davidebbo
committed
b79f8fe
Tweak parent replacement semantics
davidebbo
committed
9579748
Add Dasypodidae
davidebbo
committed
8329f6f
Misc page parsing fixes
davidebbo
committed
4a279d3
Added some primates and Glires
davidebbo
committed
4fb7a0b
Handle multiple QIDs and multiple image templates
davidebbo
committed
9cc1ce7
Add --source_mapping_file flag
davidebbo
committed
f0bbb86
Add initial readmes
davidebbo
committed
e3d65e3
Add Plesiosauria, Centrosaurinae, Rhabdodontomorpha
davidebbo
committed
994d541
Big Sauropodomorpha update
davidebbo
committed
68a5d6c
Megaraptora update
davidebbo
committed
ddb5470
Expand Scansoriopterygidae
davidebbo
committed
bdb6376
Tweaks to get Allosaurus genus
davidebbo
committed
1aded10
Add Compsognathidae
davidebbo
committed
b891ccf
Add Unaysauridae, Plateosauridae
davidebbo
committed
33a39fc
Prevent unwanted polytomies when child/parent dates are equal
davidebbo
committed
9384bdd
Add Dilophosauridae
davidebbo
committed
dfac179
Expanded a number of clades
davidebbo
committed
ed69875
Ignore incertae sedis nodes
davidebbo
committed
abe5899
Support additional period names
davidebbo
committed
15a4d5f
Don't keep source mapping in git
davidebbo
committed
9175bc7

Commits on Nov 15, 2024

Add vernacular csv file
davidebbo
committed
6766b3a
Add support for colon after rank name
davidebbo
committed
d1a2112
Add a few more period names
davidebbo
committed
0866012
Update vernacular csv
davidebbo
committed
d3a56d8
Add many mammal sub trees
davidebbo
committed
7b273bd

Commits on Nov 20, 2024

Proterochampsia and Equidae fixups
davidebbo
committed
0e18b35
Don't commit mapping file
davidebbo
committed
c80f464

Commits on May 30, 2025

Seymouriamorpha comment
davidebbo
committed
207622c
Updated from latest wikipedia
davidebbo
committed
524b3ce
Improve error
davidebbo
committed
f70a140

Commits on May 31, 2025

Improve rank parsing
davidebbo
committed
39bf73f
Expand ancient sloths, rhynos and elephants
davidebbo
committed
381b19e
Expand Feliformia
davidebbo
committed
5281ccc

Commits on Jun 1, 2025

Expand Ophidia
davidebbo
committed
f4db93a

Commits on Jul 3, 2025

Merge branch 'main' into wikipedia-extractor
davidebbo
committed
f126d9c
Change debug back to info
davidebbo
committed
80e83b7
Relative import
davidebbo
committed
55bca02