Online studies are increasingly utilized in applied research. However, lack of external diagnostic verification in many of these investigations is seen as a threat to the reliability of the data. The present study examined the robustness of internet studies on psychosis against simulation. We compared the psychometric properties of the Community Assessment of Psychic Experiences scale (CAPE), a self-report instrument measuring psychotic symptoms, across three independent samples: (1) participants with a confirmed diagnosis of schizophrenia, (2) participants with self-reported schizophrenia who were recruited over the internet, and (3) clinical experts on schizophrenia as well as students who were asked to simulate a person with schizophrenia when completing the CAPE. The CAPE was complemented by a newly developed 4-item psychosis lie scale. Results demonstrate that experts asked to simulate schizophrenia symptoms could be distinguished from real patients: simulators overreported positive symptoms and showed elevated scores on the psychosis lie scale. The present study suggests that simulated answers in online studies on psychosis can be distinguished from authentic responses. Researchers conducting clinical online studies are advised to adopt a number of methodological precautions and to compare the psychometric properties of online studies to established clinical indices to assert the validity of their results. 1. Introduction Traditionally, psychological assessments administered by an interviewer are preferred over nonpersonal assessments such as questionnaires. Face-to-face (FTF) assessment may in some cases unveil hidden symptoms and/or partly compensate for a lack of reliable information provided by a patient alone [1]. To illustrate, a patient with psychosis may deny hearing voices upon direct questioning but at the same time be observed talking to voices without overt source. This may persuade the clinician to discard the patient's response and suspect the presence of auditory hallucinations. On the other hand, research assessing the correspondence between self- and observer ratings indicates that self-report of psychotic symptoms is more reliable than commonly thought. Studies have generally found satisfactory associations between self- and observer-based ratings for overall pathology [2], negative symptoms [3], and positive symptoms [4–8]. Despite the aforementioned merits, clinical assessment is not without weaknesses. The presence of an assessor may induce important biases [9, 10], especially underreporting, that are often smaller with
References
[1]
J. C. Coyne, “Self-reported distress: analog or Ersatz depression?” Psychological Bulletin, vol. 116, no. 1, pp. 29–45, 1994.
[2]
K. K. Morlan and T. Siang-Yang, “Comparison of the brief psychiatric rating scale and the brief symptom inventory,” Journal of Clinical Psychology, vol. 54, pp. 885–894, 1998.
[3]
R. Bottlender, M. J？ger, I. Kunze, C. Groll, I. Borski, and H. J. M？ller, “Negative symptoms from the persepective of psychiatrists, patients and their relatives,” Nervenarzt, vol. 74, pp. 762–766, 2003.
[4]
F. Liraud, T. Droulout, M. Parrot, and H. Verdoux, “Agreement between self-rated and clinically assessed symptoms in subjects with psychosis,” Journal of Nervous and Mental Disease, vol. 192, no. 5, pp. 352–356, 2004.
[5]
N. J. Preston and T. J. Harrison, “The brief symptom inventory and the Positive and Negative Syndrome Scale: discriminate validity between a self-reported and observational measure of psychopathology,” Comprehensive Psychiatry, vol. 44, no. 3, pp. 220–226, 2003.
[6]
E. K. Hamera, J. K. Schneider, M. Potocky, and M. A. Casebeer, “Validity of self-administered symptom scales in clients with schizophrenia and schizoaffective disorders,” Schizophrenia Research, vol. 19, no. 2-3, pp. 213–219, 1996.
[7]
M. J. Dixon and S. King, “The concordance between symptom information gathered from remitted schizophrenic patients and their relatives,” Journal of Psychiatric Research, vol. 29, no. 6, pp. 447–456, 1995.
[8]
T. M. Lincoln, M. Ziegler, E. Lüllmann, M. J. Müller, and W. Rief, “Can delusions be self-assessed? Concordance between self- and observer-rated delusions in schizophrenia,” Psychiatry Research, vol. 178, no. 2, pp. 249–254, 2010.
[9]
S. Moritz, J. Favrod, C. Andreou et al., “Beyond the usual suspects: positive attitudes towards positive symptoms is associated with medication noncompliance in psychosis,” Schizophrenia Bulletin, vol. 39, no. 4, pp. 917–922, 2013.
[10]
M. J. Byerly, P. A. Nakonezny, and E. Lescouflair, “Antipsychotic medication adherence in schizophrenia,” Psychiatric Clinics of North America, vol. 30, no. 3, pp. 437–452, 2007.
[11]
M. M. Hartmann, A. Fritzsche, and T. M. Lincoln, “The extent and origin of discordance between self- and observer-rated depression in patients with psychosis,” Psychiatry Research, vol. 205, no. 3, pp. 247–252, 2013.
[12]
R. Kohn, S. Saxena, I. Levav, and B. Saraceno, “The treatment gap in mental health care,” Bulletin of the World Health Organization, vol. 82, no. 11, pp. 858–866, 2004.
[13]
S. Klingberg, S. Schneider, A. Wittorf, G. Buchkremer, and G. Wiedemann, “Collaboration in outpatient antipsychotic drug treatment: Analysis of potentially influencing factors,” Psychiatry Research, vol. 161, no. 2, pp. 225–234, 2008.
[14]
C. M. C. Brett, L. C. Johns, E. P. Peters, and P. K. McGuire, “The role of metacognitive beliefs in determining the impact of anomalous experiences: a comparison of help-seeking and non-help-seeking groups of people experiencing psychotic-like anomalies,” Psychological Medicine, vol. 39, no. 6, pp. 939–950, 2009.
[15]
L. Besiroglu, A. S. Cilli, and R. Askin, “The predictors of health care seeking behavior in obsessive-compulsive disorder,” Comprehensive Psychiatry, vol. 45, pp. 99–108, 2004.
[16]
J. T. Hancock, “Digital deception: why, when and how people lie online,” in The Oxdford Handbook of Internet Psychology, A. N. Joinson, K. Y. A. McKenna, T. Postmes, and U.-D. Reips, Eds., pp. 289–301, Oxford University Press, Oxford, UK, 2007.
[17]
S. Moritz, C. E. Wittekind, M. Hauschildt, and K. R. Timpano, “Do it yourself? Self-help and online therapy for people with obsessive-compulsive disorder,” Current Opinion in Psychiatry, vol. 24, no. 6, pp. 541–548, 2011.
[18]
T. Buchanan, T. Ali, T. M. Heffernan et al., “Nonequivalence of on-line and paper-and-pencil psychological tests: the case of the prospective memory questionnaire,” Behavior Research Methods, vol. 37, no. 1, pp. 148–154, 2005.
[19]
L. C. Whitehead, “Methodological and ethical issues in Internet-mediated research in the field of health: an integrated review of the literature,” Social Science and Medicine, vol. 65, no. 4, pp. 782–791, 2007.
[20]
S. Moritz, N. Van Quaquebeke, M. Hauschildt, L. Jelinek, and S. G？nner, “Good news for allegedly bad studies. Assessment of psychometric properties may help to elucidate deception in online studies on OCD,” Obsessive-Compulsive and Related Disorders, vol. 1, no. 4, pp. 331–335, 2012.
[21]
D. V. Sheehan, Y. Lecrubier, K. H. Sheehan et al., “The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10,” Journal of Clinical Psychiatry, vol. 59, supplement 20, pp. 22–33, 1998.
[22]
S. R. Kay, L. A. Opler, and J.-P. Lindenmayer, “The Positive and Negative Syndrome Scale (PANSS): Rationale and standardisation,” British Journal of Psychiatry, vol. 155, supplement 7, pp. 59–65, 1989.
[23]
N. C. Stefanis, M. Hanssen, N. K. Smirnis et al., “Evidence that three dimensions of psychosis have a distribution in the general population,” Psychological Medicine, vol. 32, no. 2, pp. 347–358, 2002.
[24]
J. Van Os, R. J. Linscott, I. Myin-Germeys, P. Delespaul, and L. Krabbendam, “A systematic review and meta-analysis of the psychosis continuum: evidence for a psychosis proneness-persistence-impairment model of psychotic disorder,” Psychological Medicine, vol. 39, no. 2, pp. 179–195, 2009.
[25]
M. Konings, M. Bak, M. Hanssen, J. Van Os, and L. Krabbendam, “Validity and reliability of the CAPE: a self-report instrument for the measurement of psychotic experiences in the general population,” Acta Psychiatrica Scandinavica, vol. 114, no. 1, pp. 55–61, 2006.
[26]
E. Fonseca-Pedrero, M. Paino, S. Lemos-Giraldez, and J. Muniz, “Validation of the Community Assessment Psychic Experiences-42 (CAPE-42) in Spanish college students and patients with psychosis,” Actas Espanolas De Psiquiatria, vol. 40, pp. 169–176, 2012.
[27]
S. Moritz and F. Lar？i, “Differences and similarities in the sensory and cognitive signatures of voice-hearing, intrusions and thoughts,” Schizophrenia Research, vol. 102, no. 1–3, pp. 96–107, 2008.
[28]
L. Voruganti and A. G. Awad, “Neuroleptic dysphoria: towards a new synthesis,” Psychopharmacology, vol. 171, no. 2, pp. 121–132, 2004.
[29]
D. Naber, B. Bandelow, S. Bender, et al., “Subjective well-being under neuroleptic treatment with olanzapine versus clozapine: first results from a double-blind clinical trial using the SWN self-rating scale,” Schizophrenia Research, vol. 49, pp. 240–240, 2001.
[30]
A. G. Awad, T. P. Hogan, L. N. P. Voruganti, and R. J. Heslegrave, “Patients' subjective experiences on antipsychotic medications: implications for outcome and quality of life,” International Clinical Psychopharmacology, vol. 10, no. 3, pp. 123–132, 1995.
[31]
S. Moritz, L. Schilling, M. Hauschildt, J. Schr？der, and A. Treszl, “A randomized controlled trial of internet-based therapy in depression,” Behaviour Research and Therapy, vol. 50, pp. 513–521, 2012.
