心理学实验的可重复性

摘要： 严格意义的实验可重复性指的是实验控制条件不变的情况下其结果的可重复性，置信区间是表示这种可重复性的恰当方法，可重复性的提高可通过在实验设计和数据分析中将影响因变量的随机额外变量作为协变量引入来实现；另一种意义的可重复性指的是实验结果的可迁移性，它涉及当控制条件变化时因控制变量和自变量的交互作用而导致的实验结果的变化。在这两种意义下，心理学实验较低的可重复性都源于它的额外变量的庞杂。
Abstract: Mainly due to dissatisfaction with the conventional NHST (null hypothesis significance testing) procedure, researchers have begun to consider alternatives. In his article, “An Alternative to Null-Hypothesis Significance Tests,” Killeen urged psychological research to abandon the routine of NHST and to quantify the signal-to-noise characteristics of experimental outcomes with replication probabilities. He described the coefficient that he invented, Prep (probability of replicability), as the probability of obtaining “an effect of the same sign as that found in an original experiment” (Killeen, 2005). Replying to the proposal, the journal Psychological Science quickly came to encourage researchers to employ Prep, rather than P value of NHST, in the reporting of their experimental results. But soon after, Killeen’s computational formula of Prep was found wrong, which resulted in that his proposal had been rejected at last (Maraun, & Gabriel, 2010). However, Killeen is been thought correct in the sense that the issue of replicability should have a central role in researchers’ assessments of the empirical results. So, the problems we need solve are then how to express the replicability of psychological experiment and how to enhance it. Acording to us, not only is Killeen’s computational formula of Prep incorrect, but his definition of replication is also inappropriate. He defined replication as obtaining “an effect of the same sign as that found in an original experiment”. However, in the constraint of same sign, the difference of the sample effects of the two experiments may very large (for example, considering that the two sample effects are 0.1 and 0.9 respectively). On the other hand, when the two sample effects have different signs, the difference of their values may rather small (for example, considering that the two sample effects are 0.1 and -0.1 respectively). The followings are our proposals about replicability and their corresponding reasons: 1. When the replicate experiment and the original experiment are homogeneous, it is not necessary to consider their difference of sample effects qualitatively, we need consider it just quantitatively. In the situation, the replicability can be appropriately expressed in terms of the width of CI (confidence interval): the smaller the width is, the nearer the two sample effects are statistically,