RUU #20: Hypothesis testing, p=1/2 vs free binomial

[jatherobot]

Great videos, really helpfull!

Can i ask how important the discretisation of a continuous parameter space is in calculating the bayes factor? specifically when comparing two models with different number of parameters. For example if i have a 2 parameter model and i use 20 values for each parameter and i compare this to models with 4 parameters but sample them each with 10 values. is there a general rule for sampling parameter space? apologies if this is a stupid question.

[trondreitan]

Thanks. It's not a stupid question at all. I know of no general rule for discretization except this: If you have enough values in the discretisation, then the model probabilities should stabilize. So a good test for checking if you have enough discretizations is to double the number of values in each model and check if you get approximately the same result. The reason you need enough discretization values is that you want to catch the details of the posterior distribution.

[trondreitan]

With much data, the posterior distribution will typically be sharper (RUU 18) than with few data. So you will typically need to increase the number of discretization points with increasing data size.

The opposite is the case when increasing the number of parameters. With a high number of parameters, the data will have less impact, leaving the parameters quite uncertain (RUU 24-25), and you can use less discretization points. 10 points for 4 parameters when using 40 for 1 parameter sounds ok.

[trondreitan]

PS: Note that in my example, discretization, or any other tricks, isn't neccessary. It's possible to deal with a continuous parameter, using calculus (integral theory, that is). But I want to keep the mathematical sophistication down, using algebra in stead of calculus. Also, discretization can be a neat trick when analytical integration fails (as it often do), as long as there's a low number of parameters. For a high number of parameters, other methods are needed, such as Monte Carlo methods.

[9macrina9]

very interesting, as usual, Trond. thanks

[sheepwshotguns]

i have gazed and been enlightened =D

[trondreitan]

Heh! Thanks.