great but i wonder rather than just showing histograms of the p values for various power levels would be good the show the actual distributions as you have in your excellent 2008 article, also I think you mention that when the null hypothesis is true the p distribution is uniform, just out of interest does the power level/effect size correspond to a parameter value for the p value distribution somehow?
I respect your effort, but I found it weirdly difficult to understand you - your voice sounded as if you were trying not to wake up the baby sleeping next door
To dance with the p values yourself, go to website: thenewstatistics-dot-com and download ESCI ("ESS-key" Exploratory Software for Confidence Intervals), for Windows or Mac. Use the 'Dance p' page of the 'ESCI chapters 5-6' module. All free. Runs under Excel, simple to use.
That site also has info about my book:
Cumming, G. (2012). "Understanding The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis". New York: Routledge.
@Kalid0scop3 See comment I've just posted. There seems to be a block on stating a URL, so I had to omit the triple-w at the start and use the word 'dot', but I'm sure folks can figure it out. Enjoy. Geoff
The biggest problem with this example though (in my point of view) is that the N is inadequate to the effect size. Studying effects of this size with these typical N values is just insane! And that's even before considering how many variables - in a real experiment, in practice - you won't be able to control.
Insane, yes, but typical for much published research in psychology and other disciplines. Cell biology routinely uses N=3! It is statistical power that determines the distribution of the p value. Power of only around .5 is typical of much published research. Crazy! Explore any other N, or power, using the Dance p simulation yourself--I've just posted a comment with details. Geoff
Good introduction to statistics, confidence intervals, and problems with P-values and null-hypothesis testing. Very well done -- Geoff put a lot of work into this.
Hi Charles, Thanks! Yep, my first experiment with Camtasia. One day I'll do a super slick version. In the meantime I'm writing the book 'Introduction to The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis'. (Routledge, 2011),
great but i wonder rather than just showing histograms of the p values for various power levels would be good the show the actual distributions as you have in your excellent 2008 article, also I think you mention that when the null hypothesis is true the p distribution is uniform, just out of interest does the power level/effect size correspond to a parameter value for the p value distribution somehow?
theoldorganplayer 2 weeks ago
I respect your effort, but I found it weirdly difficult to understand you - your voice sounded as if you were trying not to wake up the baby sleeping next door
somerandomguyinshort 2 months ago
To dance with the p values yourself, go to website: thenewstatistics-dot-com and download ESCI ("ESS-key" Exploratory Software for Confidence Intervals), for Windows or Mac. Use the 'Dance p' page of the 'ESCI chapters 5-6' module. All free. Runs under Excel, simple to use.
That site also has info about my book:
Cumming, G. (2012). "Understanding The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis". New York: Routledge.
Enjoy!
Geoff
geoffdcumming 2 months ago
Could you post your simulation program?
Kalid0scop3 2 months ago
@Kalid0scop3 See comment I've just posted. There seems to be a block on stating a URL, so I had to omit the triple-w at the start and use the word 'dot', but I'm sure folks can figure it out. Enjoy. Geoff
geoffdcumming 2 months ago
Great, thank you!
The biggest problem with this example though (in my point of view) is that the N is inadequate to the effect size. Studying effects of this size with these typical N values is just insane! And that's even before considering how many variables - in a real experiment, in practice - you won't be able to control.
Komelsky 3 months ago
@Komelsky
Insane, yes, but typical for much published research in psychology and other disciplines. Cell biology routinely uses N=3! It is statistical power that determines the distribution of the p value. Power of only around .5 is typical of much published research. Crazy! Explore any other N, or power, using the Dance p simulation yourself--I've just posted a comment with details. Geoff
geoffdcumming 2 months ago
FANTASTIC!!! As an epidemiologist I say THANK YOU for educating people about the "sacredness" of the p-value from research
gracegirl04 4 months ago 2
I enjoyed your Ockham's Razor and this is wonderf Geoff, thank you.
Matt Doogue
mdoogue 4 months ago
What's that tune? I'm sure I've heard it somewhere.
drflynn73 4 months ago
Good introduction to statistics, confidence intervals, and problems with P-values and null-hypothesis testing. Very well done -- Geoff put a lot of work into this.
ctwardy 1 year ago
@ctwardy
Hi Charles, Thanks! Yep, my first experiment with Camtasia. One day I'll do a super slick version. In the meantime I'm writing the book 'Introduction to The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis'. (Routledge, 2011),
Geoff
geoffdcumming 1 year ago
@geoffdcumming
Do let me know when that's out. Best wishes. -Charles
ctwardy 1 year ago