Estimation: Conjugate priors and concentration inequalities Decision Making under Uncertainty, IV Christos Dimitrakakis February 12, 2013 Contents 1 Sufficient statistics 1 2 Conjugate priors 3 2.1 Bernoulli-Beta conjugate family ................... 3 3 Credible intervals 6 4 Concentration inequalities 9 5 Approximate Bayesian approaches 12 6 Other conjugate families 13 6.1 Conjugates for the normal distribution ............... 13 6.1.1 The marginal predictive distribution ............ 16 6.2 Conjugates for multivariate distributions .............. 16 6.2.1 Multinomial-Dirichlet conjugates .............. 17 6.2.2 Multivariate normal conjugate families ........... 18 1 Sufficient statistics In the previous unit, we have seen how to make optimal decisions with respect to a given risk function and belief. However, one important question is how belief can be calculated. It is one thing to say that we simply calculate the posterior distribution of parameter and another thing to actually do it. In this unit, we shall look at cases when calculating the posterior distributions of parameters is easy. This occurs when the posterior distribution can be expressed by a function that belongs to the same family sa the prior distribution, no matter what the observations are. Introduction Sometimes we want to summarise a sequence of observations x t =(x 1 ,...,x t ). 1