Is it time to raise the methodological and statistical standards for all empirical contributions to PA conventions?
With the collaboration of Jessica Utts, the Program Chairs of the 56th Annual Convention of the Parapsychological Association suggest the following:
METHODOLOGICAL AND STATISTICAL GUIDELINES
- Make explicit the exploratory and pilot experiments from the formal ones;
- Make explicit the primary and the secondary hypotheses to test;
- Report all experimental conditions, including failed manipulations;
- Make explicit the initial choice of the sample size(s), and provide an explanation if it was not met;
- If possible explain the rationale for the sample size(s), including a power analysis;
- Whenever possible, report confidence intervals and effect sizes along with or instead of p-values;
- If Bayesian methods are used, be explicit about all priors, including the prior distribution represented in the alternative hypothesis.
- Exact and conceptual replications are welcomed.
- When using the frequentist Null Hypothesis Significant Testing approach, adopt the APA 2010, statistical guidelines;
- Furthermore, the acceptation and the rejection of the null hypothesis should be carried out using parameters’ confidence intervals, equivalence testing or models comparison procedures (see suggested readings and resources).
Suggested Readings and Resources:
- APA (2010). Publication Manual 6th Ed. American Psychological Association.
- APA Publications and Communications Board Working Group (2008). Reporting Standards for Research in Psychology Why Do We Need Them? What Might They Be?. American Psychologist, 63, 9, 839–851.
- Cumming G. (2012). Understanding The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis: www.thenewstatistics.com
- Cumming, G., Fidler, F., Kalinowski, P. and Lai, J. (2011). The statistical recommendations of the American Psychological Association Publication Manual: Effect sizes, confidence intervals, and meta-analysis. Australian Journal of Psychology. doi:10.1111/j.1742-9536.2011.00037.x
- Mara, C.A., Cribbie, R.A. (2012). Paired-Samples Tests of Equivalence. Communications in Statistics - Simulation and Computation, 41, 10, 1928-1943.
- Simmons, J.P., Nelson, L.D., and Simonsohn, U. (2012). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11) 1359–1366
These standards were revised and approved by Jessica Utts, Professor of Statistics, Department of Statistics University of California, Irvine.
For statistical advice before submitting your paper to the 2013 PA Convention, please contact Program Chair Patrizio Tressoldi at email@example.com.