However, if we could use the closed-form formula of the conjugate prior, the computation becomes very light. As you saw, the computations in Bayesian Inference can be heavy or sometimes even intractable. You have to update your model as more data come in (and that’s why we use Bayesian Inference). This is very convenient! (Proof in the next section.)Īs a data/ML scientist, your model is never complete. Therefore, after carrying out more experiments, you can compute the posterior simply by adding the number of acceptances and rejections to the existing parameters α, β respectively, instead of multiplying the likelihood with the prior distribution. What does this mean? It means during the modeling phase, we already know the posterior will also be a beta distribution. In the example above, the beta distribution is a conjugate prior to the binomial likelihood. Furthermore, if your prior distribution has a closed-form form expression, you already know what the maximum posterior is going to be.

When you know that your prior is a conjugate prior, you can skip the posterior = likelihood * prior computation. Second, if there is no closed-form formula of the posterior distribution, we have to find the maximum by numerical optimization, such as gradient descent or newtons method. In order to find the maximum in a vanilla way, we need to consider every candidate - the likelihood P(X|θ) for every θ. Even if you choose not to normalize the posterior, the end goal is to find the maximum of the posteriors ( Maximum a posteriori).

Why do we have to calculate the posterior for thousands of thetas? Because you are normalizing the posterior (line 21). There are two things that make the posterior calculation expensive.įirst, we are computing the posterior for every single θ.

What does conjugate mean