层级一致性指标的多级评分拓展

摘要： 在认知诊断评估实践中，属性层级合理性的验证非常重要，而现有指标仅停留在0-1计分测验，无法适应考试形式和评分方式多样化的实践需求。研究将0-1计分层级一致性指标（MHCI）拓展至多级计分的层级一致性指标（GHCI），模拟和实证研究结果表明：（1）GHCI具有和MHCI相同的本质含义，考虑了父项目和子项目得分的多种可能性，从而将MHCI纳入GHCI体系；（2）在多级或混合计分情境，MHCI会有信息损失，容易发生低估，且易受转换比例的影响；（3）GHCI在模拟和实践情境均具较好的适宜性，拟合截断值的设置可依属性层级而定。
Abstract: In educational practice of cognitive diagnostic assessment (CDA), it is crucial to validate the reasonability of the hierarchical structure of attributes, because it can affect the quality of CDA and the accuracy of classification of examinees directly. Several indices that validate the reasonability of the hierarchical structure of attributes have been developed by researchers. Based on attribute hierarchy method (AHM), Cui, Leighton, Gierl and Hunka (2006）developed hierarchy consistency index (HCI) to detect the misfit of examinees; also Ding, Mao, Wang and Luo (2011) modified the HCI, then developed a new index——Modified HCI（MHCI). In addition, Guo (2012) proposed the hierarchy misfit index (HMI) to detect the misfit of examinees’ misfit responses. Although these indexes can be used to detect misfit, all of these indexes are suitable for dichotomous. For polytomous items, in order to validate the reasonability of attribute hierarchy (AH), researchers, in general, transform these polytomous items into dichotomous items according to some prespecified rules, then calculate the HCI (Kang, Wu, Chen, & Zeng, 2015; Kang, Xin, & Tian, 2013) or MHCI (Ding et al., 2012). However, it will lose some details when transform polytomous items into dichotomous, therefore produce larger errors (Ding, Wang, & Luo, 2014) and underestimate the consistence of AH. The purpose of this study is extending the Modified HCI (MHCI) to a new HCI that suitable for both dichotomous and polytomous items, and we name the new HCI as generalized hierarchy consistency index (GHCI). To evaluate the suitability of GHCI, a simulation study and an empirical study are employed. For the simulation study, two independent variables are manipulated: type of attribute hierarchy (AH) and proportion of transformation. The AH has 4 levels (linear, convergent, divergent, and unstructured) and the proportion of transformation has 5 levels (0%，60%, 67%, 75%, 100%). The control variables are the number of attributes, K = 5, and the number of examinees, N = 2000. Results show that: (1) Both GHCI and MHCI are affected by type of AH, the linear AH has the best GHCI and MHCI, then is convergent AH, after that is divergent AH and the unstructured AH is worst. (2) GHCI has larger means than MHCI regardless of AH and proportion