Can a multidimensional hierarchy of skills generate data conforming to the Rasch model? A comparison of methods

The question of the dimensionality of intelligent performance has kept researchers occupied for decades. We investigate this question in the context of learning elementary arithmetic. Our assumption of a polyhierarchy of skills in arithmetic (HiSkA) predicts a multidimensional structure of test data. This seems to contradict findings that data collected to validate the HiSkA conformed to the Rasch model. To resolve this seeming contradiction, we analysed test data from two samples of third graders with a number of methods ranging from factor analysis and Rasch analysis to multidimensional item response theory (MIRT). Additionally we simulated data sets based on different unidimensional and multidimensional models and compared the results of some of the analyses that were also applied to the empirical data. Results show that a multidimensional generating structure can produce data conforming to the Rasch model under certain conditions, that a general factor explains a substantial amount of variance in the empirical data, but that the HiSkA is capable of explaining much of the residual variance.