We put forward a model based on item response theory that highlights the role of latent features called “proficiency” and “propensity”. The model is adjusted to data from the decisions made in a high-stakes exam taken by 10,822 Brazilian high school students. Our model aims to recover infor-mation regarding the role the latent features (proficiency and propensity) play in a decision. We find that the decision of responding or not and also the decision of responding correctly or not in a group of items can be described by a two-dimensional logistic model, even if there are imperfections from an item-by-item adjustment. Not only proficiency, but also refraining from responding is found to depend on both the characteristics of the items and the latent features of the students. In particular, the least proficient students prefer to leave an item blank, rather than respond it incorrectly. There is a negative linear correlation between scoring in the exam and propensity, and scoring and proficiency are positively correlated although nonlinear.
Cite this paper
Gomes, H. , Matsushita, R. and Silva, S. D. (2019). Item Response Theory Modeling of High School Students’ Behavior in a High-Stakes Exam. Open Access Library Journal, 6, e5242. doi: http://dx.doi.org/10.4236/oalib.1105242.
Klein, S.P. and Hamilton, L. (1999) Large-Scale Testing: Current Practices and New Directions. Rand Education. https://www.rand.org/content/dam/rand/pubs/issue_papers/2006/IP182.pdf
Hamilton, L.S., Stecher, B.M. and Klein, S.P. (2002) Making Sense of Test-Based Accountability in Education. Rand Education. https://www.rand.org/content/dam/rand/pubs/monograph_reports/2002/MR1554.pdf
Abdelfattah, F.A. (2007) Response Latency Effects on Classical and Item Response Theory Parameters Using Different Scoring Procedures. PhD Thesis, Ohio University, Athens, OH.
Lievens, F., Sackett, P.R. and Buyse, T (2009) The Effects of Response Instructions on Situational Judgment Test Performance and Validity in a High-Stakes Context. Journal of Applied Psychology, 94, 1095-1101. https://doi.org/10.1037/a0014628
Albanese, M. and Knott, M. (1992) TWOMISS: A Computer Program for Fitting a One- or Two-Factor Logit-Probit Latent Variable Model to Binary Data When Observations May Be Missing. LSE Technical Report, London.
Knott, M. and Tzamourani, P. (1997) Fitting a Latent Trait Model for Missing Observations to Racial Prejudice Data. In: Rost, J. and Langeheine, R., Eds., Applications of Latent Trait and Latent Class Models in the Social Sciences, Waxmann, Munster, 244-252.
Bartholomew, D.J., de Menezes, L.M. and Tzamourani, P. (1997) Latent Trait Class of Models Applied to Survey Data. In: In: Rost, J. and Langeheine, R., Eds., Applications of Latent Trait and Latent Class Models in the Social Sciences, Waxmann, Munster, 219-232.
O’Muircheartaigh, C. and Moustaki, I. (1996) Item Non-Response in Attitude Scales: A Latent Variable Approach. Proceedings of the American Statistical Association, Section of Survey Research Methods, 938-943.
O’Muircheartaigh, C. and Moustaki, I. (1999) Symmetric Pattern Models: A Latent Variable Approach to Item Non-Response in Attitude Scales. Journal of the Royal Statistical Society A, 162, 177-194. https://doi.org/10.1111/1467-985X.00129
Moustaki, I. and Knott, M. (2000) Weighting for Item Non-Response in Attitude Scales by Using Latent Variable Models with Covariates. Journal of the Royal Statistical Society A, 163, 445-459. https://doi.org/10.1111/1467-985X.00177
Moustaki, I. and O’Muircheartaigh, C. (2000) A One Dimension Latent Trait Model to Infer Attitude from Nonresponse for Nominal Data, Statistica, 60, 259-276.
Moustaki, I. and O’Muircheartaigh, C. (2002) Locating “Don’t Know”, “No Answer” and Middle Alternatives on an Attitude Scale: A Latent Variable Approach. In: Marcoulides, G.A. and Moustaki, I., Eds., Latent Variable and Latent Structure Models, Lawrence Erlbaum Associates, London, 15-40.
Andrade, D.F. and Tavares, H.R. (2005) Item Response Theory for Longitudinal Data: Population Parameter Estimation. Journal of Multivariate Analysis, 95, 1-22. https://doi.org/10.1016/j.jmva.2004.07.005