“Bayes is guaranteed to overfit”: What does this mean? There’s a factor of 2 here.

Yuling writes, “Bayes is guaranteed to overfit, for any model, any prior, and every data point.” This statement is not literally true: it does not hold for degenerate examples of models with no unknown parameters (for which p(theta|y) is the … Continue reading → …