TaxonForge

LLM-driven, iterative taxonomy development

TaxonForge is an implementation of the taxonomy development method by Nickerson, Varshney & Muntermann (2013), "A method for taxonomy development and its application in information systems". It is extended with large language models acting as the "researcher" in the iterative Empirical-to-Conceptual / Conceptual-to-Empirical loop.

It helps you structure complex subject areas (any textual corpus such as conversations or papers) into a useful, scientifically grounded taxonomy: a set of mutually exclusive, collectively exhaustive characteristics organized into dimensions, derived from a single user-stated meta-characteristic.

Why TaxonForge?

Taxonomies organize knowledge. The Nickerson et al. (2013) method is a frequently cited methodology for taxonomy development in information systems, but applying it is labor-intensive: a researcher must iterate between empirical observation and conceptual deduction until both objective and subjective ending conditions are met.

TaxonForge automates the loop with LLMs as the analyst, while keeping the human in control: every prompt, threshold, and dimension can be inspected, overridden, and audited.

In the era of cutting-edge LLMs, the practical value of this graph-based implementation is debatable. Recent SOTA models, such as Anthropic's Fable 5, might yield "better" taxonomies in a single shot, especially when they are equipped with an advanced agent harness. While probably being more interesting from an academic perspective, TaxonForge has a few desired properties:

• it is grounded in an established, theoretical framework and allows fine-grained configuration
• it produces traceable results, including an iteration history
• it is provider-agnostic and yields reasonable results even with local LLMs (in this case, gemma4:12b)

It's worth mentioning that this approach was originally designed for a specific domain and meta-characteristic. After the results turned out to be good, I decided to build a more generic, domain-agnostic solution for it.

How it works

1. Define a meta-characteristic: "The meta-characteristic is the most comprehensive characteristic that will serve as the basis for the choice of characteristics in the taxonomy." (Nickerson et al. 2013, p. 343).
2. TaxonForge embeds your corpus and samples a subset.
3. Each iteration, an approach-selector agent chooses between two complementary strategies:
   • Empirical-to-Conceptual: extract characteristics from data, then group into dimensions.
   • Conceptual-to-Empirical: refine dimensions conceptually, then validate against data.
4. After every iteration, an ending-conditions agent scores the taxonomy on the five subjective Nickerson criteria (conciseness, robustness, comprehensiveness, extendibility, explanatory power) and checks eight objective conditions.
5. The loop stops when both criteria sets are satisfied, or --max-iterations is reached.

Demo Use Cases

• Comparing the LLM landscape on Huggingface
• Characterizing User Interactions in conversational chatbot applications

These examples are more of demo use cases. The data used, as well as the meta-characteristics and purpose description, have not been selected with particular care.

Departures from the paper

The iterative approach and the ending conditions of Nickerson et al. (2013) are all preserved. The following adaptations were made where the paper leaves room for interpretation or where LLM-driven execution requires it.

• TaxonForge scores the subjective conditions from 1 to 5, instead of the paper's qualitative "informed argument". min_subjective_score (default 3) is the pass threshold.
• Plateau-based early stopping via early_stopping_patience (default 3): terminates when the aggregated subjective score no longer improves.
• Best-scored taxonomy returned by default (return_best_taxonomy=True) since with plateau stopping the last iteration is not necessarily the best.
• Embedding-based smart sampling (farthest-point + targeted) instead of the paper's open-ended "random / systematic / convenience".
• LLM grouping instead of statistical clustering for Step 6e. TaxonForge only ships the informal route, but the paper allows both.
• Anti-churn "stability" prompting in the three revising agents: an empty changelog is framed as a valid outcome. Without this, LLMs tend to change the taxonomy every iteration and never meet the "no new dims / no merge / no split" conditions.

Technical Setup

Prerequisite:

• Ollama setup locally
• Run gemma4:12b with Ollama (other models and provider could also be used, but this might require changes in the code)
• Python 14 installed
• uv package manager installed

Install:

• uv sync --group dev
• source .venv/bin/activate

Nickerson, R. C., Varshney, U., & Muntermann, J. (2013). A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3), 336–359. https://doi.org/10.1057/ejis.2012.26