oalib
Search Results: 1 - 10 of 100 matches for " "
All listed articles are free for downloading (OA Articles)
Page 1 /100
Display every page Item
A multiple objective test assembly approach for exposure control problems in Computerized Adaptive Testing  [cached]
Bernard P. Veldkamp,Angela J. Verschoor,Theo J.H.M. Eggen
Psicológica , 2010,
Abstract: Overexposure and underexposure of items in the bank are serious problems in operational computerized adaptive testing (CAT) systems. These exposure problems might result in item compromise, or point at a waste of investments. The exposure control problem can be viewed as a test assembly problem with multiple objectives. Information in the test has to be maximized, item compromise has to be minimized, and pool usage has to be optimized. In this paper, a multiple objectives method is developed to deal with both types of exposure problems. In this method, exposure control parameters based on observed exposure rates are implemented as weights for the information in the item selection procedure. The method does not need time consuming simulation studies, and it can be implemented conditional on ability level. The method is compared with Sympson Hetter method for exposure control, with the Progressive method and with alphastratified testing. The results show that the method is successful in dealing with both kinds of exposure problems.
New Item Selection Criteria of Computerized Adaptive Testing with Exposure-Control Factor
引入曝光因子的计算机化自适应测验选题策略

CHENG Xiao-Yang,DING Shu-Liang,YAN Shen-Hai,ZHU Long-Yin,
程小扬
,丁树良,严深海,朱隆尹

心理学报 , 2011,
Abstract: As far as Computerized Adaptive Testing (CAT) is concerned, the issue of item selection strategy has received more attention because of its vital role. It is well known that there are two typical selection strategies called Maximum Information Criterion (MIC) and a-Stratification (a-STR). However, both of the two strategies have their advantages together with their downsides. On the one hand, MIC method can obtain high efficiency and accurate estimation of ability; on the other hand, its uneven item selecti...
Computerized Adaptive Testing in Reading Comprehension
Lim Tock Keng, Ho Wah Kam
TEFLIN Journal , 1997,
Abstract: : A Computerized Adaptive Testing (CAT) project in reading comprehension was established to develop multiple choice tests across four grade levels, Primary 3 and 5, and Secondary I and 3. CAT is interactive and allows participants to select their own entry points to the test and gives feedback on their performance. Building a CAT system required the development of an item bank, selection of item and items order to be presented in a test, and evaluating the test for difficulty. The creation of the item bank involved the specifying reading comprehension skills, writing items, field testing, item analysis and calibration. The software, MICROCAT, was used to develop an item bank, to select items and item order to be presented in a test, and to evaluate the test for difficulty. The project is currently at the stage of field testing.
Computerized adaptive testing: implementation issues  [PDF]
Margit Antal,Levente Er\Hos,Attila Imre
Computer Science , 2010,
Abstract: One of the fastest evolving field among teaching and learning research is students' performance evaluation. Computer based testing systems are increasingly adopted by universities. However, the implementation and maintenance of such a system and its underlying item bank is a challenge for an inexperienced tutor. Therefore, this paper discusses the advantages and disadvantages of Computer Adaptive Test (CAT) systems compared to Computer Based Test systems. Furthermore, a few item selection strategies are compared in order to overcome the item exposure drawback of such systems. The paper also presents our CAT system along its development steps. Besides, an item difficulty estimation technique is presented based on data taken from our self-assessment system.
Modern Sequential Analysis and its Applications to Computerized Adaptive Testing  [PDF]
Jay Bartroff,Matthew Finkelman,Tze Leung Lai
Statistics , 2011,
Abstract: After a brief review of recent advances in sequential analysis involving sequential generalized likelihood ratio tests, we discuss their use in psychometric testing and extend the asymptotic optimality theory of these sequential tests to the case of sequentially generated experiments, of particular interest in computerized adaptive testing. We then show how these methods can be used to design adaptive mastery tests, which are asymptotically optimal and are also shown to provide substantial improvements over currently used sequential and fixed length tests.
Sequential Design for Computerized Adaptive Testing that Allows for Response Revision  [PDF]
Shiyu Wang,Georgios Fellouris,Hua-Hua Chang
Statistics , 2015,
Abstract: In computerized adaptive testing (CAT), items (questions) are selected in real time based on the already observed responses, so that the ability of the examinee can be estimated as accurately as possible. This is typically formulated as a non-linear, sequential, experimental design problem with binary observations that correspond to the true or false responses. However, most items in practice are multiple-choice and dichotomous models do not make full use of the available data. Moreover, CAT has been heavily criticized for not allowing test-takers to review and revise their answers. In this work, we propose a novel CAT design that is based on the polytomous nominal response model and in which test-takers are allowed to revise their responses at any time during the test. We show that as the number of administered items goes to infinity, the proposed estimator is (i) strongly consistent for any item selection and revision strategy and (ii) asymptotically normal when the items are selected to maximize the Fisher information at the current ability estimate and the number of revisions is smaller than the number of items. We also present the findings of a simulation study that supports our asymptotic results.
Better Data From Better Measurements Using Computerized Adaptive Testing  [cached]
David J. Weiss
Journal of Methods and Measurement in the Social Sciences , 2011,
Abstract: The process of constructing a fixed-length conventional test frequently focuses on maximizing internal consistency reliability by selecting test items that are of average difficulty and high discrimination (a "peaked" test). The effect of constructing such a test, when viewed from the perspective of item response theory, is test scores that are precise for examinees whose trait levels are near the point at which the test is peaked; as examinee trait levels deviate from the mean, the precision of their scores decreases substantially. Results of a small simulation study demonstrate that when peaked tests are "off target" for an examinee, their scores are biased and have spuriously high standard deviations, reflecting substantial amounts of error. These errors can reduce the correlations of these kinds of scores with other variables and adversely affect the results of standard statistical tests. By contrast, scores from adaptive tests are essentially unbiased and have standard deviations that are much closer to true values. Basic concepts of adaptive testing are introduced and fully adaptive computerized tests (CATs) based on IRT are described. Several examples of response records from CATs are discussed to illustrate how CATs function. Some operational issues, including item exposure, content balancing, and enemy items are also briefly discussed. It is concluded that because CAT constructs a unique test for examinee, scores from CATs will be more precise and should provide better data for social science research and applications.
The effect of success probability on test economy and self-confidence in computerized adaptive tests  [PDF]
JOACHIM H?USLER,MARKUS SOMMER
Psychology Science Quarterly , 2008,
Abstract: Recent research on the psychological effects of different design decisions in computerized adaptive tests indicates that the maximum-information item selection rule fails to optimize respondents’ test-taking motivation. While several recent studies have investigated psychological reactions to computerized adaptive tests using a consistently higher base success rate, little research has so far been conducted on the psychometric (primarily test reliability and bias) and psychological effects (e.g. test-taking motivation, self-confidence) of using mixtures of highly informative (p = .50) and easier items (p = .80) in the item selection process. The present paper thus compares these modifications to item selection with a classical maximum-information algorithm. In a simulation study the effect of the different item selection algorithms on measurement precision and bias in the person parameter estimates is evaluated. To do so, the item pool of the Lexical Knowledge Test, measuring crystallized intelligence and self-confidence, is used. The study indicated that modifications using base success probabilities over p = .70 lead to reduced measurement accuracy and - more seriously - a bias in the person parameter estimates for higher ability respondents. However, this was not the case for the motivator item algorithm, occasionally administering easier items as well. The second study (n = 191) thus compared the unmodified maximum-information algorithm with two motivator item algorithms, which differed with regard to the percentage of motivator items presented. The results indicate that respondents yield higher self-confidence estimates under the motivator item conditions. Furthermore, the three conditions did not differ from each other with regard to the total test duration. It can be concluded that a small number of easier motivator items is sufficient to preserve test-taking motivation throughout the test without a loss of test economy.
An accurate and efficient identification of children with psychosocial problems by means of computerized adaptive testing
Antonius GC Vogels, Gert W Jacobusse, Symen A Reijneveld
BMC Medical Research Methodology , 2011, DOI: 10.1186/1471-2288-11-111
Abstract: We used a Dutch national data set obtained from parents of children invited for a routine health examination by Preventive Child Healthcare with 205 items on behavioral and emotional problems (n = 2,041, response 84%). In a random subsample we determined which items met the requirements of an Item Response Theory (IRT) model to a sufficient degree. Using those items, item parameters necessary for a CAT were calculated and a cut-off point was defined. In the remaining subsample we determined the validity and efficiency of a Computerized Adaptive Test using simulation techniques, with current treatment status and a clinical score on the Total Problem Scale (TPS) of the Child Behavior Checklist as criteria.Out of 205 items available 190 sufficiently met the criteria of the underlying IRT model. For 90% of the children a score above or below cut-off point could be determined with 95% accuracy. The mean number of items needed to achieve this was 12. Sensitivity and specificity with the TPS as a criterion were 0.89 and 0.91, respectively.An IRT-based CAT is a very promising option for the identification of psychosocial problems in children, as it can lead to an efficient, yet high-quality identification. The results of our simulation study need to be replicated in a real-life administration of this CAT.Many children suffer from behavioural and emotional problems [1-3] and these problems may seriously interfere with their daily functioning, now and later in life [4,5]. Yet many of these children remain untreated [5]. Early identification and treatment improves the prognosis of the children involved considerably [2,6].Community-based preventive child healthcare (PCH) services, especially outreaching services, are in a unique position to identify such problems as early as possible. In the Netherlands, PCH professionals offer routine well-child care to the entire Dutch population to the age of about 14, free of charge. The early detection of children with psychosocial problem
Random Generation of Response Patterns under Computerized Adaptive Testing with the R Package catR  [PDF]
David Magis,Gilles Ra?che
Journal of Statistical Software , 2012,
Abstract: This paper outlines a computerized adaptive testing (CAT) framework and presents an R package for the simulation of response patterns under CAT procedures. This package, called catR, requires a bank of items, previously calibrated according to the four-parameter logistic (4PL) model or any simpler logistic model. The package proposes several methods to select the early test items, several methods for next item selection, different estimators of ability (maximum likelihood, Bayes modal, expected a posteriori, weighted likelihood), and three stopping rules (based on the test length, the precision of ability estimates or the classification of the examinee). After a short description of the different steps of a CAT process, the commands and options of the catR package are presented and practically illustrated.
Page 1 /100
Display every page Item


Home
Copyright © 2008-2017 Open Access Library. All rights reserved.