docs: add workflow doctest examples#803
Conversation
EKtheSage
requested review from
genedan,
henrydingliu,
jbogaardt and
kennethshsu
as code owners
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #803 +/- ##
=======================================
Coverage 88.79% 88.79%
=======================================
Files 89 89
Lines 5060 5060
Branches 646 646
=======================================
Hits 4493 4493
Misses 423 423
Partials 144 144
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| ) | ||
| param_grid = {"benk__n_iters": [1, 4]} | ||
| scoring = { | ||
| "IBNR": lambda m: float(np.nansum(m.named_steps.benk.ibnr_.values)) |
There was a problem hiding this comment.
could you please change this to sigma? we should look like we understand the basics of machine learning in the docstring.
| ) | ||
| pipe.set_params(dev__average="volume") | ||
| ib_volume = int( | ||
| round(float(np.nansum(pipe.fit_predict(tri).ibnr_.values)), 0) |
EKtheSage
force-pushed
the
docs/704-workflow-examples
branch
from
28e6914 to
b3dd40f
Compare
|
|
||
| import numpy as np | ||
|
|
||
| raa = cl.load_sample("raa") |
There was a problem hiding this comment.
a lot of duplicate code from previous example. hide under testsetup?
|
|
||
| .. testoutput:: | ||
|
|
||
| 19694.23 |
There was a problem hiding this comment.
confusing to the user to see one example with full vector of ultimates, followed by another example that only shows a couple
| 1989 20004.502125 | ||
| 1990 21605.832631 | ||
|
|
||
| ``weights`` and ``default_weighting`` change how sub-model ultimates are |
There was a problem hiding this comment.
narratively this doesn't build on the first example
| .. testoutput:: | ||
|
|
||
| 2 | ||
| 1.422 |
There was a problem hiding this comment.
@kennethshsu does this feel like a bug to you? going from simple avg to volume weighted somehow introduced such a gargantuan increase in sigma.
There was a problem hiding this comment.
I looked into this, and I don’t think this is a bug. Most of the sigma_ difference appears to be driven by the 12-24 factor.
Since the LDFs are shifting over time as we move down the origin years, the volume-weighted factors are being pulled toward the more recent origin years. That, in turn, is driving the larger sigma_ values relative to the older origin years.
That said, I’m not entirely sure what this example is intended to demonstrate. Also, summing the sigma_ values does not really seem meaningful here. I think it would make more sense to compare the underlying arrays directly.
|
|
||
| Examples | ||
| -------- | ||
| Use ``Pipeline`` when the same triangle should pass through several |
There was a problem hiding this comment.
this example doesn't motivate why pipeline is useful. said in another way, pipeline is overkill for a straightforward chainladder(development) on a single triangle.
one instructive narrative line would be to actually compare the groupby pipeline from the user guide to a pipeline without groupby
|
@EKtheSage are you interested in finishing up this PR? |
GridSearch: score the full per-age sigma_ vector instead of a summed sigma so candidates are compared on the underlying arrays, and explain why the simple and volume rows sit on different scales. Pipeline: motivate the estimator by comparing the user guide groupby pipeline against the same pipeline without groupby, showing pooled line-of-business patterns versus unstable standalone company patterns. VotingChainladder: give the first example an actuary-facing narrative, make the second example build on the first by reusing its estimators and apriori, and print the full ultimate vector instead of two scalars.
|
@henrydingliu updated per your comments and @kennethshsu's. GridSearch now scores the full per-age sigma_ vector so candidates are compared on the underlying arrays, with an explanation of why the two averaging methods sit on different scales. The Pipeline example now motivates the estimator by comparing the user guide groupby pipeline against the same pipeline without groupby. The voting examples now build on each other and print full ultimate vectors. Main is merged in as well. |
|
|
||
| .. testoutput:: | ||
|
|
||
| simple [1.163, 0.102, 0.057, 0.038, 0.026, 0.016, 0.007, 0.01, 0.003] |
There was a problem hiding this comment.
the difference in magnitude of these sigmas is not intuitive to me. i've opened a separate issue #952 to address it (or for someone to talk some sense into me). if you are interested/have time, please help me figure out whether there's an actual bug or not. otherwise, would you mind changing this example to showcase gridsearch through a different statistic?
| the same pipeline without ``groupby`` shows why that pooling matters: | ||
| standalone patterns are fit to each company's own thin data. | ||
|
|
||
| .. testsetup:: |
There was a problem hiding this comment.
this is a powerful example. but is it actually showing people how to use pipeline? i wonder if it makes more sense to be included instead under Development to showcase groupby.
For pipeline, i think it would be more instructive to demonstrate a workflow where a development, a tail, and an ibnr method are chained together
3 participants
Summary: Add Sphinx doctest examples for GridSearch, Pipeline, and VotingChainladder workflow docs. Split from the larger #792 work and intentionally excludes .github/workflows/sync-main-to-docs.yml. Refs #704
Note
Low Risk
Documentation and doctest-only changes with no runtime or API behavior modifications.
Overview
Adds Sphinx doctest
Examplessections to workflow API docs so narrative and executable snippets appear in the built reference.GridSearchgets a med-mal example that gridsdev__average(simplevsvolume) and uses a custom scoring callable to comparesigma_by development age, plus prose on scale and interpretability.Pipelinegets a CLRD example contrastingDevelopment(groupby="LOB")vs standalone development and printing IBNR by line, with a short note on unstable standaloneothliabIBNR. A small Attributes docstring formatting fix closes thenamed_stepsdescription properly.VotingChainladderextends the existing RAA example with actuarial framing for accident-year-specificweights, and adds a second doctest that omitsweightsand usesdefault_weighting=(2, 1, 1)to blend all three estimators every year.No changes to fit/predict behavior—only docstrings and doctest blocks (refs #704, split from #792).
Reviewed by Cursor Bugbot for commit b98388e. Bugbot is set up for automated code reviews on this repo. Configure here.