Hi Eric, awesome work. I was wondering if I am thinking about it wrong, or if this is something that can be done on your end:
Currently, assess_reference_loo() outputs post_mean_pi per collection that can be summed up to repunit....
If I wanted to get at the distribution of the "post_pi" results, would this data exist in the function's calculations? In other words, is post_mean_pi the mean post proportion resulting from several sampling events at the input "true" proportion? Could you then also output "post_pi" calculations for each collection? Or is my (mis)understanding of the function way off.
The purpose I had thought of for doing so was to try to evaluate the precision of the baseline across the range of input proportions in order to see if a drop in accuracy at high proportions coincided with more variance around the estimate. I postured that if this was the case, the increased variance might be able to be reported in the final estimates, and the drop in accuracy may not be too big of a deal if the true value was contained in the interval.
I apologize if this is way off base... I haven't dug into your literature behind it all yet.
Hi Eric, awesome work. I was wondering if I am thinking about it wrong, or if this is something that can be done on your end:
Currently, assess_reference_loo() outputs post_mean_pi per collection that can be summed up to repunit....
If I wanted to get at the distribution of the "post_pi" results, would this data exist in the function's calculations? In other words, is post_mean_pi the mean post proportion resulting from several sampling events at the input "true" proportion? Could you then also output "post_pi" calculations for each collection? Or is my (mis)understanding of the function way off.
The purpose I had thought of for doing so was to try to evaluate the precision of the baseline across the range of input proportions in order to see if a drop in accuracy at high proportions coincided with more variance around the estimate. I postured that if this was the case, the increased variance might be able to be reported in the final estimates, and the drop in accuracy may not be too big of a deal if the true value was contained in the interval.
I apologize if this is way off base... I haven't dug into your literature behind it all yet.