自适应分组认知诊断测验设计及其选题策略

摘要： 应用OMST在线装配模式，提出自适应分组认知诊断测验（CD-AMGT）。由于知识状态的先决关系是偏序关系，而且构成格（lattice)，利用知识状态当前估计值在格中的上下确界对被试真实知识状态的可能范围进行界定，由此装配下一分组，分组中结合PWKL策略或SHE策略进行选题以兼顾诊断精度、效率和安全性。模拟实验表明，CD-AMGT与PWKL、SHE对比，当题目类型丰富时，以分类准确率略微降低为代价，其题库使用均匀性和计算用时均表现出较大优势。
Abstract: It took incredible investment of time and effort to construct item bank. Selection strategy, which is the most significant component of Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT), should react quickly and pay attention to the utilization of item bank. The two widely used item selection methods in CD-CAT are Shannon Entropy (SHE) and Posterior Weighted Kullback-Leibler (PWKL). The characteristic of SHE method and PWKL method is higher classification accuracy, but the utilization rate of item bank is uneven. Learning from the idea of on-the-fly multistage adaptive testing, a new test named adaptive multi-group testing for cognitive diagnosis (AMST) was proposed. AMST is composed of several groups and each group has multiple items which were assembled by the interim knowledge state and its upper and lower bounds. The combination of AMST and SHE named AMST-SHE and AMST coupled with PWKL named AMST-PWKL, AMST-SHE and AMST-PWKL are the optimal design for AMST. Based on the Deterministic Inputs, Noisy-and-gate, a simulation study was operated to investigate the efficiency of the AMGT-SHE method and the AMGT-PWKL method compared with the SHE method, the PWKL method and the Random selection method for four item pools with different structures. Pattern correct rate, test length, average exposure rate and time consuming per person were calculated. Suppose that the attributes are mutually independent and the number of attributes was 5. Test length was fixed to 25, and the size of the item pool was fixed to 300. Variable length test stops when the largest posterior probability of knowledge state was not smaller than 0.9 and the second largest was larger than 0.1. The Monte Carlo simulation results showed that (1) the pattern correct rate and time consuming of the AGMT-SHE method were better than those of the AGMT-PWKL method, but the average exposure rate was opposite. For the AGMT-SHE method and the AGMT-PWKL method, the simpler the item types in item bank, the higher the pattern correct rate; (2) when the item types are rich in item bank, the average exposure rate and time consuming of the AMGT-PWKL method and the AMGT-SHE method are far better than those of the PWKL method and SHE method, especially on time consuming, the former is one-ninth of the latter, but test length would be increased; (3) the