Search NCBI GEO for series by title, description, and sample properties, and produce a TSV/Excel report to help pick cohorts.
python -m venv .venv
.venv/Scripts/activate # Windows
pip install -e ".[dev]"Set your NCBI contact email (required by NCBI's usage policy):
export GEOTOOL_NCBI_EMAIL="you@example.com"For the LLM-assisted commands below, set an Anthropic API key (e.g. in a
local .env file, which is gitignored):
echo 'ANTHROPIC_API_KEY=sk-ant-...' > .env
set -a && source .env && set +ageotool search --title "breast cancer" --organism "Homo sapiens" \
--sample-property "tissue:liver" --sample-property "treatment:tamoxifen" \
--max-results 100 --out reportWrites data/reports/report.tsv and data/reports/report.xlsx. If any
--sample-property filters are given, per-series annotation is also saved to
data/series/<GSE_ID>/series.tsv and data/series/<GSE_ID>/samples.tsv.
Add --llm-annotate to have Claude classify each candidate's samples
(species, biopsy/cell line, tissue, diagnosis, prior therapy) from their full
GEO record; --llm-escalate re-runs low-confidence fields on a stronger
model. Needs ANTHROPIC_API_KEY.
geotool query "human biopsy pancreatic cancer cohorts with sample size more than 20"Parses the request into a diagnosis (plus synonyms) and filter categories
(species, biopsy/cell line, tissue, assay type, material selection) with one
Claude call, recalls candidate series from GEO by the diagnosis and its
synonyms, then classifies each candidate's title/summary against the filters
with one lightweight Claude call each. Every candidate is written to the
report with its own columns — nothing is silently dropped, so you can
filter/sort the table yourself. Logs the parsed filters and each candidate's
classification as it runs (--quiet to suppress). Needs ANTHROPIC_API_KEY.
geotool download GSE10846 GSE339488RNA-seq series get their supplementary expression file(s) downloaded as-is.
Microarray series get reshaped from each sample's own probe values into a
probes x samples matrix, then mapped to a genes x samples matrix via each
platform's own annotation table (probe_matrix.tsv.gz / expression.tsv.gz
-- gzip-compressed, values rounded to 3 decimal places).
Every cohort also gets a cleaned, semantically-unified annotation.tsv.
Writes into data/series/<GSE_ID>/.
Gene-level expression values (expression.tsv.gz and, for two-channel
Agilent samples, channel1_expression.tsv.gz / channel2_expression.tsv.gz
below) are log2(x + 1)-transformed unless they already look log2 scale --
checked with a simple heuristic (any value over 50 means "not yet log2",
since real log2 expression values rarely reach the low teens while raw
linear-scale intensities routinely run into the hundreds or thousands). This
happens at the gene-expression level only -- probe_matrix.tsv.gz and the
channelN_probe_matrix.tsv.gz files stay exactly as submitted -- and keeps
gene-level values comparable across platforms/submitters regardless of
whether they submitted raw or already-log-transformed values, while
correctly leaving already-log2 data (like a two-channel ratio, which can be
negative) untouched.
Two-channel Agilent samples (Cy3/Cy5 reference-design arrays) whose own data
table publishes per-channel intensity columns -- not all do; most only carry
the precomputed ratio -- also get each channel's own probe/gene matrix as an
additional output: channel1_probe_matrix.tsv.gz / channel1_expression.tsv.gz
and the channel2_* equivalents, alongside the unchanged, ratio-based
expression.tsv.gz. Neither channel is assumed to be "the real sample" or
"the reference" -- that's a per-study convention this tool has no reliable
way to infer, so both are simply named by channel number (channel 1 is
always the green/Cy3 scanner channel, channel 2 always red/Cy5).
Add --rma to also RMA-renormalize Affymetrix microarray series from their
raw CEL files (probe_matrix_rma_<GPL>.tsv.gz / expression_rma.tsv.gz,
written alongside the submitter-value files above, never replacing them).
CEL files are deleted once a platform's RMA run succeeds, to avoid keeping
both the raw data and its derived matrix on disk. This is opt-in and needs
R installed with Rscript on PATH -- Bioconductor itself and every R
package RMA needs (BiocManager, affy/oligo, and the chip-specific
CDF/pd.* package) are installed automatically on first use, into a
user-writable library, so no manual R package setup is required. The
tradeoff is that the very first --rma run for a given chip can take a
while (network install time on top of the RMA computation); every run after
that is fast since the packages persist.
Only platforms listed in geotool/renormalize.py's _CHIP_PACKAGES table are
supported; an unlisted platform, or a failed install/run, just skips
RMA for that series (logged) rather than failing the download.
geotool harmonize GSE10846 GSE98588Unifies already-downloaded cohorts' annotation tables into one master table,
one row per sample across every requested cohort. Reuses each cohort's own
annotation.tsv (from download, above) plus, if present, its cached
llm_annotations.json (from search --llm-annotate) at zero extra cost --
tissue/diagnosis/sample-source classification is joined in directly from
that cache, no LLM call needed. Whatever raw characteristic columns aren't
already covered (sex, age, cell_type, ...) get renamed onto a canonical name
via a small alias file (geotool/vocab_data/annotation_aliases.json);
anything with no alias match is kept as-is rather than dropped. Survival
columns are unioned across cohorts (one cohort's OS_time/OS_event and
another's PFS_time/PFS_event both survive, NaN where a cohort doesn't
report that type).
Add --llm-annotate to backfill tissue/diagnosis classification for
cohorts that don't have a cached llm_annotations.json yet (costs a real
LLM call per such cohort, needs ANTHROPIC_API_KEY; off by default). A
cohort that hasn't been downloaded yet (no annotation.tsv) is skipped with
a warning rather than failing the whole run. Writes
data/harmonized/<name>/annotation.tsv.
Cross-cohort expression matrix harmonization (different platforms, different gene coverage, batch effects) is a separate, harder problem and out of scope here.
Phase 1 (search + report), LLM-assisted annotation/query, Phase 2 (download,
plus opt-in CEL/RMA renormalization and two-channel Agilent splitting), and
Phase 3 (cross-cohort annotation harmonization) are all implemented — see
geotool/download.py, geotool/probe_mapping.py, geotool/renormalize.py,
and geotool/harmonize.py. Cross-cohort expression matrix harmonization
remains unimplemented (see the Harmonize section above).