The use of peer
assessment through a multisource feedback process has gained recognition as a
reliable and valid method to assess the characteristics of professionals and
trainees. A total of 168 first-year medical students
completed a 15-item questionnaire to self-assess their
professional work habits and interpersonal
abilities. Each student was expected to identify 8 first-year classmates to complete a corresponding 15-item peer assessment. Although the self and peer assessment
questionnaires had strong reliability (Cronbach’s
α = 0.85 and 0.91, respectively), an exploratory factor analysis resulted in a
3- and 2- factor solution,
respectively. The third factor was associated with items related to students’ personal attributes. Significantly
lower mean score differences for the self-report assessment were found for all
15 items (Cohen’s d = 0.27 to 1.39, p < 0.001). A decision study analysis
found that 7 peer assessors were needed to achieve a generalizability
coefficient of 0.70. The findings suggest some inconsistencies in regards to the construct validity and stability of measures between self
and peer assessment measures. The need for self-awareness of students’
strengths and limitations, however, is recommended as part of their development
in a profession that emphasizes self-regulation.
References
[1]
Accreditation Council for Graduate Medical Education (ACGME) (2013). ACGME Outcome Project. URL (last checked 15 January 2013).
http://www.acgme.org/acgmeweb/tabid/159/DataCollectionSystems /AccreditationDataSystem.aspx
[2]
Allerup, P., Aspegren, K., Ejlersen, E., et al. (2007). Use of 360-degree assessment of residents in internal medicine in a Danish setting: A feasibility study. Medical Teacher, 29, 166-170.
doi:10.1080/01421590701299256
[3]
Arnold, L. (2002). Assessing professional behavior: Yesterday, today, and tomorrow. Academic Medicine, 77, 502-515.
doi:10.1097/00001888-200206000-00006
[4]
Bandiera, G., Sherbino, J., & Frank, J. R. (2006). The CanMEDS assessment tools handbook: An introductory guide to assessment methods for the CanMEDS competencies. Ottawa, ON: The Royal College of Physicians and Surgeons of Canada.
[5]
Brennan, R. L. (2001). Generalizability theory. New York: Springer Verlag.
[6]
Brinkman, W. B., Geraghty, S. R., Lanpher, B. P., et al. (2007). Effect of multisource feedback on resident communication skills and prof essionalism. Pediatric Adolescence Medicine, 61, 44-49.
doi:10.1001/archpedi.161.1.44
[7]
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
[8]
Colthart, I., Bagnall, G., Evans, A., et al. (2008). The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide No. 10. Medical Teacher, 30, 124-145. doi:10.1080/01421590701881699
[9]
Dannefer, E. F., Henson, L. C., Bierer, S. B., et al. (2005). Peer as sessment of professional competence. Medical Education, 39, 713 722. doi:10.1111/j.1365-2929.2005.02193.x
[10]
Epstein, R. M., Dannefer, E. F., Nofziger, A., et al. (2004). Com prehensive assessment of professional competencie: The Rochester experiment. Teaching & Learning in Medicine, 16, 186-196.
doi:10.1207/s15328015tlm1602_12
[11]
Frank, J. R. (2005). The CanMEDS 2005 physician competency frame work. Better standards. Better physicians. Better care. Ottawa, ON: The Royal College of Physicians and Surgeons of Canada.
[12]
Lockyer, J. M., & Clyman, S. G. (2008). Multisource feedback (360 degree evaluation). In: E. S. Holmboe, & R. E. Hawkins (Eds.), Practical guide to the evaluation of clinical competence (pp. 75-83). Philadelphia, PA: Mosby Elsevier.
[13]
Lurie, S. J., Nofziger, A. C., Meldrum, S., Mooney, C., & Epstein, R. M. (2006a). Effects of rater selection on peer assessment among medical students. Medical Education, 40, 1088-1097.
doi:10.1111/j.1365-2929.2006.02613.x
[14]
Lurie, S. J., Nofziger, A. C., Meldrum, S., Mooney, C., & Epstein, R. M. (2006b). Temporal and group-related trends in peer assessment amongst medical students. Medical Education, 40, 840-847.
doi:10.1111/j.1365-2929.2006.02540.x
[15]
Lurie, S. J., Meldrum, S., Nofziger, A. C., Sillin, L. F., Mooney, C. J., & Epstein, R. M. (2007a). Changes in self-perceived abilities among male and female medical students after the first year of clinical training. Medical Teacher, 29, 921-926.
doi:10.1080/01421590701753559
[16]
Lurie, S. J., Lambert, D. R., Nofziger, A. C., Epstein, R. M., & Grady Weliky, T. A. (2007b). Relationship between peer assessment during medical school, Dean’s letter rankings, and ratings by internship directors. Journal of General Internal Medicine, 22, 13-16.
doi:10.1007/s11606-007-0117-4
[17]
Nofziger, A. C., Naumburg, E. H., Davis, B. J., Mooney, C. J., & Epstein, R. M. (2010). Impact of peer assessment on the professional development of medical students: A qualitative study. Academic Medicine, 85, 140-147. doi:10.1097/ACM.0b013e3181c47a5b
[18]
Papadakis, M. A., Teherani, A., Banach, M. A., et al. (2005). Disciplinary action by medical boards and prior behavior in medical schools. New England Journal of Medicine, 353, 2673-2682.
doi:10.1056/NEJMsa052596
[19]
Violato, C., & Lockyer, J. (2006). Self and peer assessment of pedia tricians, psychiatrists and medicine specialists: Implications for self directed learning. Advances in Health Science Education, 11, 235-244. doi:10.1007/s10459-005-5639-0
