Question about the influence of prior choice on Bayes factor

[zhengchencai]

Question and context
Hello,

Thank you very much for developing this amazing tool. May I ask a question regarding Bayes factor? in this tutorial https://easystats.github.io/bayestestR/articles/bayes_factors.html, you explained very well the calculation of BF, however, do you think it will be influenced by the prior choice? Imagine I now have a much wider prior than the one shown in your figure and also imagine our data is informative. Then using a wider prior would not influence too much the posterior, but odd for prior vs null will be influenced a lot since prior is now wider. Therefore the final BF will decrease. How do we avoid this? Thanks a lot.

[mattansb]

If you have very wide priors, it seems unlikely you would want to compute a Bayes factor against the null.

BF is essentially a bet - you are betting the data will on average fit your prior compared to the null. But a very wide prior means that are betting on a wide verity of values, and so you will certainly be more wrong than right (compared to the null).

---

Example:
I (the null) say that the value is 0.
You say: No, it is not 0 - it is somewhere between -100 and +100.

We collect data and find that in the range of -100 and +100, the data is relatively (to my 0) less compatible with most of that range. So on average, your "bet" was worst than mine.

With Bayes factors: More specific and nuanced priors yield higher "rewards". Non specific priors will always lose.

---

So how to avoid this?

Avoid the bet alltoghether - don't use BFs if you don't have an informative prior.

Or... use BFs, but not against a point null. You can compare "the coef is positive" to "the coef is negative":

bayesfactor_parameters(posterior, prior, null = c(-Inf, 0))

However, note that these are priors you are comparing - so changing your prior after seeing the data makes no sense (and is overfitting).