In order to improve the understanding of media violence effects, it is crucial to extend knowledge about factors that threaten the validity of such effects in empirical research. Research artifacts can be expected when participants are (a) aware of a scientist’s hypothesis, (b) motivated to confirm or disconfirm the hypothesis, and (c) capable of manipulating their responses in line with their motivation. Based on social identity theory (SIT) and self-categorization theory (SCT), we assumed that identifying with the social group of video game players would provide a motivation to disconfirm the “violent video games increase aggression” hypothesis. We further assumed that the use of nontransparent aggression measures and cover stories would prevent research artifacts. Our results showed that highly identified (compared to lowly identified) players of video games reported less aggression on a transparent aggression measure but not on a nontransparent aggression measure. However, providing participants with a cover story did not prevent hypothesis awareness nor eliminate hypothesis-disconfirming response patterns. These results provide empirical support for the ideas that (a) motivational factors may contribute to a biased estimation of media violence effects, (b) cover stories may not always be effective, and (c) the use of nontransparent aggression measures can provide a valid methodological approach for avoiding biases in media effects research.
Roethlisberger, F.J.; Dickson, W.J.; Wright, H.A. Management and the Worker: An Account of a Research Program Conducted by the Western Electric Company, Hawthorne Works, Chicago; Harvard University Press: Cambridge, MA, USA, 1939.
Strohmetz, D.B.; Rosnow, R.L. A Mediational Model of Research Artifacts. In Probability in Theory-Building: Experimental and Nonexperimental Approaches to Scientific Research in Psychology; Brzezinski, J., Ed.; Editions Rodopi: Amsterdam, The Netherlands, 1994; pp. 177–196.
Carlopio, J.; Adair, J.G.; Lindsay, R.C.L.; Spinner, B. Avoiding Artifact in the Search for Bias: The Importance of Assessing Subjects’ Perceptions of the Experiment. J. Pers. Soc. Psychol. 1983, 44, 693–701, doi:10.1037/0022-3518.104.22.1683.
Rosnow, R.L.; Goodstadt, B.E.; Suls, J.M.; Gitter, A.G. More on the social psychology of the experiment: When compliance turns to self-defense. J. Pers. Soc. Psychol. 1973, 27, 337–343, doi:10.1037/h0034943.
Ziegler, M.; Schmidt-Atzert, L.; Bühner, M.; Krumm, S. Fakability of different measurement methods for achievement motivation: Questionnaire, semi-projective, and objective. Psychol. Sci. 2007, 49, 291–307.
Carnagey, N.L.; Anderson, C.A. The Effects of Reward and Punishment in Violent Video Games on Aggressive Affect, Cognition, and Behavior. Psychol. Sci. 2005, 16, 882–889, doi:10.1111/j.1467-9280.2005.01632.x.
Golding, S.L.; Lichtenstein, E. Confession of awareness and prior knowledge of deception as a function of interview set and approval motivation. J. Pers. Soc. Psychol. 1970, 14, 213–223, doi:10.1037/h0028853.
Barlett, C.; Anderson, C. Violent Video Games and Public Policy. In Wie wir Spielen, Was Wir Werden: Computerspiele in unserer Gesellschaft. (In German); Bevc, T., Zapf, H., Eds.; UVK Verlagsgesellschaft: Konstanz, Germany, 2009; pp. 227–240.
Branscombe, N.R.; Ellemers, N.; Spears, R.; Doosje, B. The Context and Content of Social Identity Threat. In Social Identity: Context, Commitment, Content; Ellemers, N., Spears, R., Doosje, B., Eds.; Blackwell Science: Oxford, UK, 1999; pp. 35–58.
Carnagey, N.L.; Anderson, C.A.; Bushman, B.J. The effect of video game violence on physiological desensitization to real-life violence. J. Exp. Soc. Psychol. 2007, 43, 489–496, doi:10.1016/j.jesp.2006.05.003.
Anderson, C.A.; Carnagey, N.L.; Eubanks, J. Exposure to violent media: The effects of songs with violent lyrics on aggressive thoughts and feelings. J. Pers. Soc. Psychol. 2003, 84, 960–971, doi:10.1037/0022-3522.214.171.1240.
Gentile, D.A.; Lynch, P.J.; Linder, J.R.; Walsh, D.A. The effects of violent video game habits on adolescent hostility, aggressive behaviors, and school performance. J. Adolesc. 2004, 27, 5–22, doi:10.1016/j.adolescence.2003.10.002.
Fleming, M.J.; Rickwood, D.J. Effects of Violent Versus Nonviolent Video Games on Children’s Arousal, Aggressive Mood, and Positive Mood. J. Appl. Soc. Psychol. 2001, 31, 2047–2071, doi:10.1111/j.1559-1816.2001.tb00163.x.
Anderson, C.A.; Carnagey, N.L.; Flanagan, M.; Benjamin, A.J.; Eubanks, J.; Valentine, J.C. Violent Video Games: Specific Effects of Violent Content on Aggressive Thoughts and Behavior. In Advances in Experimental Social Psychology; Zanna, M.P., Ed.; Elsevier Academic Press: San Diego, CA, USA, 2004; Volume 36, pp. 199–249.
Little’s MCAR test revealed that the missing values were missing completely at random, χ2(14, N = 100) = 12.55, p = 0.56. Based on guidelines for dealing with missing data, data were imputed using the expectation-maximization (EM) algorithm, and all analyses were run with and without imputation. Analyses with imputed data produced the same results as analyses on data with listwise deletion. Consequently, all analyses are reported for data with listwise deletion.
Kelava, A.; Moosbrugger, H. Deskriptivstatistische Evaluation von Items (Itemanalyse) und Testwertverteilungen. In Testtheorie und Fragebogenkonstruktion. (in German); Moosbrugger, H., Kelava, A., Eds.; Springer: Heidelberg, Germany, 2007; pp. 73–98.
It might be argued that in the “no cover story” condition, highly identified video game players would question the ostensible purpose of the nontransparent measure. If this was the case, then both the transparent and the nontransparent measure should be negatively related to identification in the “no cover story” condition. This, however, was not the case: Whereas identification was negatively correlated with the transparent measure (r = ？0.22, p = 0.13), it was not correlated with the nontransparent measure (r = 0.05, p = 0.74).
Straits, B.C.; Wuebben, P.L.; Crowle, A.J. Confession of Prior Knowledge about Experimental Procedures as a Function of Evaluation Apprehension and Commitment. Soc. Sci. Res. 1982, 11, 227–244, doi:10.1016/0049-089X(82)90010-2.
De Houwer, J. What Are Implicit Measures and Why Are We Using Them. In The Handbook of Implicit Cognition and Addiction; Wiers, R.W., Stacy, A.W., Eds.; Sage Publishers: Thousand Oaks, CA, 2006; pp. 11–28.
Anderson, C.A.; Shibuya, A.; Ihori, N.; Swing, E.L.; Bushman, B.J.; Sakamoto, A.; Rothstein, H.R.; Saleem, M. Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review. Psychol. Bull. 2010, 136, 151–173, doi:10.1037/a0018251.