Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ type: software
authors:
- alias: nsboan
email: nasserboan@gmail.com
repository-code: https://github.com/nasserboan/proj-clv
repository-code: https://github.com/nasserboan/clvkit
abstract: >-
Buy-till-you-die models for customer-base analysis in Python, fit by maximum
likelihood and benchmarked against the estimates published with the original
Expand Down
100 changes: 64 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,18 +15,18 @@
</p>

<p align="center">
<img src="https://img.shields.io/badge/python-3.13%2B-4F46E5?style=flat" alt="Python 3.13+">
<img src="https://img.shields.io/badge/python-3.11%2B-4F46E5?style=flat" alt="Python 3.11+">
<img src="https://img.shields.io/badge/dependencies-4-4F46E5?style=flat" alt="Four dependencies">
<img src="https://img.shields.io/badge/tests-238%20passing-4F46E5?style=flat" alt="238 tests passing">
<img src="https://img.shields.io/badge/tests-254%20passing-4F46E5?style=flat" alt="254 tests passing">
<a href="https://github.com/astral-sh/ruff"><img src="https://img.shields.io/badge/code%20style-ruff-4F46E5?style=flat" alt="Code style: ruff"></a>
<img src="https://img.shields.io/github/stars/nasserboan/proj-clv?style=flat&color=4F46E5" alt="Stars">
<img src="https://img.shields.io/github/stars/nasserboan/clvkit?style=flat&color=4F46E5" alt="Stars">
<img src="https://img.shields.io/pypi/v/clvkit?style=flat&color=4F46E5" alt="PyPI version">
<img src="https://img.shields.io/github/license/nasserboan/proj-clv?style=flat&color=4F46E5" alt="License">
<img src="https://img.shields.io/github/license/nasserboan/clvkit?style=flat&color=4F46E5" alt="License">
</p>

<p align="center">
<strong>r = 0.242595 against a published 0.243 &middot; 4 dependencies &middot; one <code>scipy.optimize</code> run &middot; a DataFrame back</strong><br>
<sub>BG/NBD fit on the CDNOW 1/10 systematic sample, 2,357 customers, 39-week calibration, which is the same data Fader, Hardie &amp; Lee published <code>r = .243, α = 4.414, a = .793, b = 2.426</code> on in 2005. The right-hand numbers are not transcribed: <a href="tests/test_bgnbd_golden.py">the test suite fits them</a> on every push, and the build fails if any drifts past half a unit in the paper's last printed digit. <a href="#reproduced-estimates">The full table</a> &middot; <a href="#the-whole-thing">six lines to run it</a>.</sub>
<strong>A transaction log in, a per-customer CLV table back &middot; six lines &middot; 2.0 s wall clock &middot; 14 packages</strong><br>
<sub>BG/NBD, Gamma-Gamma and discounted lifetime value on the CDNOW 1/10 systematic sample, 2,357 customers, imports included. The fitted parameters land on the estimates Fader, Hardie &amp; Lee published on the same data in 2005, and they aren't transcribed: <a href="tests/test_bgnbd_golden.py">the test suite fits them</a> on every push, and the build fails if any drifts past half a unit in the paper's last printed digit. <a href="#the-whole-thing">The six lines</a> &middot; <a href="#reproduced-estimates">the reproduced estimates</a>.</sub>
</p>

---
Expand All @@ -45,6 +45,37 @@ answer any of them, because they need statistical inference rather than a
The BTYD / probability-model tradition (Fader, Hardie, et al.) answered these
decades ago. The Python tooling for it is what went missing.

## The whole thing

```python
import pandas as pd

from clvkit import CLV, CustomerBase

log = pd.read_csv("transactions.csv", parse_dates=["date"]) # customer_id, date, amount

cb = CustomerBase.from_transactions(log, time_unit="W", collapse="D")
result = CLV().fit(cb).predict(horizon=52, discount_rate=0.001)

# expected_purchases, discounted_expected_transactions, expected_spend, clv
result.to_pandas() # indexed by customer_id
result.plot()
```

That is the whole pipeline. On the CDNOW sample, 2,357 customers, it runs in
2.0 s of wall clock, imports included.

`CustomerBase` is the one input currency: it turns the raw log into the RFM
sufficient statistic every model here consumes, and carries its own provenance
(`time_unit`, `collapse`, `observation_period_end`, `has_monetary`,
`on_negative`), so scoring a model fit in weeks against a base measured in
days is refused rather than quietly answered, and so is scoring it against a
base that kept a different set of purchases. The verbs are short and shared:
`fit` and `predict` everywhere, plus `probability_alive` on the transaction
models, where latent
attrition is the thing being estimated. Every result object plots itself and
hands you a DataFrame with `.to_pandas()`.

## Why this exists

Two libraries occupy this space, and each is out of reach in its own way.
Expand Down Expand Up @@ -82,13 +113,11 @@ stays correct — CI checks both on every push.

## Install

Not on PyPI yet. Until it is:

```console
uv add git+https://github.com/nasserboan/proj-clv
uv add clvkit
```

Python 3.13+. Four runtime dependencies (numpy, scipy, pandas, matplotlib), and
Or `pip install clvkit`. Python 3.11+. Four runtime dependencies (numpy, scipy, pandas, matplotlib), and
one optional extra, `dask`, for logs too big to summarise in pandas. See
[Large transaction logs](#large-transaction-logs).

Expand All @@ -105,33 +134,35 @@ one optional extra, `dask`, for logs too big to summarise in pandas. See

<sub>Both are the library's own output on the CDNOW 1/10 sample.</sub>

## The whole thing
## Four questions, and the sentence to say out loud

```python
import pandas as pd
[`examples/start_here.ipynb`](examples/start_here.ipynb) is a router, not a
lesson. Four questions a business actually asks (what is a customer worth, is
this one gone or just quiet, who gets the retention budget, why do two
retention charts disagree), each answered on real data in a couple of cells,
each ending with the sentence you could say in a meeting.

from clvkit import CLV, CustomerBase
The first one runs the formula you probably use today, `average ticket ×
frequency × margin`, against 39 weeks of CDNOW history neither method saw. The
formula overshoots by 61%. The fitted model comes in 16% low, wrong by a
quarter as much and in the safe direction, and the notebook hands you the
sentence that follows: "our current LTV number assumes nobody ever churns."

log = pd.read_csv("transactions.csv", parse_dates=["date"]) # customer_id, date, amount
The library talks the same way. `print(cb)` on the CDNOW base says
`52% bought once - the models see them only through the population, not their
own history`. The variant for a base like that, where a customer can drop out
after their first purchase, is [`MBGNBD`](#whats-in-the-box), one word away.

cb = CustomerBase.from_transactions(log, time_unit="W", collapse="D")
result = CLV().fit(cb).predict(horizon=52, discount_rate=0.001)
## Small enough to leave

# expected_purchases, discounted_expected_transactions, expected_spend, clv
result.to_pandas() # indexed by customer_id
result.plot()
```
`uv add clvkit` puts 14 packages in a fresh environment, none of them a
compiler. The library itself is about 2,600 lines of BSD-3 Python on numpy,
scipy, pandas and matplotlib.

`CustomerBase` is the one input currency: it turns the raw log into the RFM
sufficient statistic every model here consumes, and carries its own provenance
(`time_unit`, `collapse`, `observation_period_end`, `has_monetary`,
`on_negative`), so scoring a model fit in weeks against a base measured in
days is refused rather than quietly answered, and so is scoring it against a
base that kept a different set of purchases. The verbs are short and shared:
`fit` and `predict` everywhere, plus `probability_alive` on the transaction
models, where latent
attrition is the thing being estimated. Every result object plots itself and
hands you a DataFrame with `.to_pandas()`.
That count is the exit plan. If I disappear, a team that depends on this can
vendor the lines into their own repo and maintain them. A promise to maintain
a library is worth what promises are worth; a codebase small enough to take
over is worth more.

## Reproduced estimates

Expand Down Expand Up @@ -396,11 +427,8 @@ abandoned. This one is finished, not stalled — stopping here is the design.

## Example notebooks

Start with [`examples/start_here.ipynb`](examples/start_here.ipynb). It's a router,
not a lesson: four questions a business actually asks, each answered on real data
in a couple of cells, each ending with the sentence you could say in a meeting.
The first one runs your own `average ticket x frequency x margin` formula against
39 weeks of held-out CDNOW history and shows it overshooting by 61%.
Start with [`examples/start_here.ipynb`](examples/start_here.ipynb), the router
described [above](#four-questions-and-the-sentence-to-say-out-loud).

The other two are the long versions. All three are executed top-to-bottom by CI
with [`nbmake`](https://github.com/treebeardtech/nbmake), so a change that breaks
Expand Down
8 changes: 7 additions & 1 deletion docs/site/examples/from_lifetimes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,13 @@ From lifetimes
If you have working ``lifetimes`` code, the same fit is shorter here, and the two
knobs that quietly change the answer are named instead of hidden. ``lifetimes`` is
archived and takes no new releases; clvkit fits the same four models on Python
3.13.
3.11+.

Archived means nobody fixes it when the ground moves, and it already has.
setuptools 83 removed ``pkg_resources``, ``lifetimes.datasets`` stopped importing
with it, and the ``load_cdnow_summary()`` line that opens every old tutorial now
dies on a fresh install. The models themselves still fit; the on-ramp is what
broke.

The whole migration, side by side
----------------------------------
Expand Down
3 changes: 2 additions & 1 deletion docs/site/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ clvkit

Buy-till-you-die models for a business where nobody cancels a subscription, so
you never observe a customer leaving — you only ever observe them not coming back
*yet*. Python 3.13+, four dependencies, no compiler.
*yet*. Six lines take a transaction log to a per-customer CLV table, in 2.0 s
of wall clock on the CDNOW sample. Python 3.11+, four dependencies, no compiler.

.. code-block:: bash

Expand Down
2 changes: 1 addition & 1 deletion docs/site/install/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Install

uv add clvkit # or pip install clvkit

Python 3.13+, four runtime dependencies — ``numpy``, ``scipy``, ``pandas`` and
Python 3.11+, four runtime dependencies — ``numpy``, ``scipy``, ``pandas`` and
``matplotlib`` — and no compiler. If ``uv`` is new to you, it's the fastest way
in; ``pip install clvkit`` gets you the same package.

Expand Down
2 changes: 1 addition & 1 deletion docs/site/user_guide/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Install

uv add clvkit # or pip install clvkit

Python 3.13+, four runtime dependencies. If ``uv`` is new to you, it's
Python 3.11+, four runtime dependencies. If ``uv`` is new to you, it's
the fastest way in; ``pip install clvkit`` gets you the same package.

The one contract
Expand Down
4 changes: 2 additions & 2 deletions examples/cdnow_clv.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1973,7 +1973,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"wrote /Users/nasserboan/projects/personal/proj-clv/examples/output/cdnow_clv.csv (192,544 bytes)\n"
"wrote examples/output/cdnow_clv.csv (192,544 bytes)\n"
]
},
{
Expand Down Expand Up @@ -2076,7 +2076,7 @@
"source": [
"csv_path = OUTPUT / \"cdnow_clv.csv\"\n",
"result.to_pandas().to_csv(csv_path)\n",
"print(f\"wrote {csv_path} ({csv_path.stat().st_size:,} bytes)\")\n",
"print(f\"wrote {csv_path.relative_to(REPO)} ({csv_path.stat().st_size:,} bytes)\")\n",
"\n",
"result.to_pandas().nlargest(5, \"clv\").round(2)"
]
Expand Down
4 changes: 2 additions & 2 deletions examples/online_retail_ii_cohort.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -417,7 +417,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"/Users/nasserboan/projects/personal/proj-clv/examples/data/online_retail_II.xlsx (46 MB)\n"
"examples/data/online_retail_II.xlsx (46 MB)\n"
]
}
],
Expand Down Expand Up @@ -448,7 +448,7 @@
" with zipfile.ZipFile(archive) as zf:\n",
" zf.extract(\"online_retail_II.xlsx\", DATA)\n",
"\n",
"print(f\"{workbook} ({workbook.stat().st_size / 1e6:.0f} MB)\")"
"print(f\"{workbook.relative_to(REPO)} ({workbook.stat().st_size / 1e6:.0f} MB)\")"
]
},
{
Expand Down
13 changes: 10 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ readme = "README.md"
authors = [
{ name = "nsboan", email = "nasserboan@gmail.com" }
]
requires-python = ">=3.13"
requires-python = ">=3.11"
# BSD-3-Clause, matching numpy, scipy and pandas. Permissive, so a company can
# embed this without a legal review, and the no-endorsement clause keeps the
# project's name out of anyone else's marketing.
license = "BSD-3-Clause"
license-files = ["LICENSE"]
dependencies = [
"matplotlib>=3.11.0",
"numpy>=2.5.1",
"numpy>=2.3",
"pandas>=3.0.3",
"scipy>=1.18.0",
"scipy>=1.16",
]

[project.optional-dependencies]
Expand Down Expand Up @@ -70,3 +70,10 @@ docs = [
"sphinx-copybutton>=0.5.2",
"sphinx-design>=0.7.0",
]

# The runtime floor is 3.11 (validated: the suite passes on 3.11 at both the
# newest compatible set and the exact pinned minimums). Docs tooling is
# maintainer-only and sphinx>=9.1 needs 3.12, so the docs group keeps a
# higher floor rather than dragging the package's down.
[tool.uv.dependency-groups]
docs = { requires-python = ">=3.12" }
4 changes: 4 additions & 0 deletions src/clvkit/__init__.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
"""clvkit — CLV engine and cohort-retention over a shared transaction-log contract."""

from importlib.metadata import version as _version

from clvkit.clv import BGNBD, CLV, MBGNBD, GammaGamma
from clvkit.cohort import CohortMatrix, CohortSurvival
from clvkit.customer_base import CustomerBase

__version__ = _version("clvkit")

__all__ = [
"BGNBD",
"CLV",
Expand Down
Loading
Loading