Dance p 3 Mar09

Dance of the p values, trial of 3 Mar 09

• Category

  Science & Technology

[Oldmaneatingatwix100]

Currently reading your book "Understanding the New Statistics". Great stuff- proving very helpful for constructing my thesis. It has encouraged me to incorporate both a narrative and meta-analytic element to my literature review. I now appreciate this approach gives a much more accurate analysis of the evidence.

I also am responsible for developing e-learning materials for my undergrad stats course- the simulations and examples you give are a great inspiration.

Thank you for writing this book.

[Geoff Cumming]

Thanks for that--I wish you well with your thesis. And for the e-learning materials. I'm currently at the Annual Convention of the Association for Psychological Science, in Washington DC, giving a workshop and two posters, and doing a book signing. There is a ton of interest. The new statistics is definitely the way of the future. I hope the book and ESCI serve you well!

Geoff

[Geoff Cumming]

Thanks Jeffrey for those 2. Consider the dance of the CIs--the bouncing-around sequence of CIs with replication. Our CI is one from such a sequence. Our CI tells us about the sequence because its length gives some idea of the extent of bouncing around. Our CI indicates where, most likely (83% chance) the mean of a replication would fall. Not Bayesian, just coming straight from the simulation of dancing CIs. Our CI is much more informative than any single p value. Hug a confidence interval today!

[Geoff Cumming]

Thanks David! I disagree! The best explanation, with evidence, why p values are so terrible and damaging is the chapter by Rex Kline at tiny dott cc slashh klinechap3

Lots more in my book 'Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis' Info at thenewstatistics dott com

Evidence that confidence intervals are better than p, and indeed that it is better to use just CIs, without p, at tiny dott cc slashh cisbetter

Enjoy!

Regards, Geoff

[gracegirl04]

FANTASTIC!!! As an epidemiologist I say THANK YOU for educating people about the "sacredness" of the p-value from research

[Jeffrey Flint]

On the other hand, if you were to use a previous study's CI as a notion of what a future study's results might be (replication?), that is clearly a Bayesian notion.

[Jeffrey Flint]

I don't see it that way since there is no assertion of truth in any study, so it doesn't make sense to talk about replication with regards to CIs. The only thing known for sure is that for alpha = 5%, if the null hypothesis is true, then 95% of studies will accept the null hypothesis. In any one trial, there is no way to know whether the CI is an indication of "truth"; it is only a 95% binomial variable.

[Geoff Cumming]

Thanks Jeffrey. Informally, I'm an everyday Bayesian, as pretty much everyone is. But my statistics advocacy is for CIs not credible intervals, and I use no analysis of Bayesian priors, etc. Yes, if we have a single study, we have a challenge, but CIs are still superior to p (Coulson ref). One way to think about their advantage is that they give better info of what a replication is likely to give. Even if we have only a single study. Geoff

[Jeffrey Flint]

I think what you are saying is that the results improve (converge) the more iterations of the study you perform. That is true whether or not you use CIs or p-values. If I see many CIs of the same study, I agree that my eye can estimate the probable or mean CI in a way that averaging p-values cannot do.

But having many CIs to study is a luxury. The whole problem is what to do when you have a single trial.

I think the point is that you are a Bayesian, which I appreciate.

[Geoff Cumming]

Thanks Jeffrey. Lots of vital issues there for discussion! Most basic for me is the enormous value of shifting from dichotomous reject vs don't-reject (NHST, p) to estimation (CIs), as is most informative for building a cumulative quantitative discipline. Prediction relates to replication, which is at the core of science, and CI does much better than p in giving info about replication. There is info about book, intro articles, podcast, video, etc at thenewstatistics dottt com Geoff