Sensitivity of IRT Equating to the Behavior of Test Equating Items

Abstract

The delta-plot method is used to identify which common items in a common item nonequivalent groups design for test equating show large changes in their p-values across administrations. Outliers in that plot denote differential item behavior and are candidates for exclusion from the common item pool. This study investigated whether keeping or discarding those outliers has an effect on equating transformations and equated aggregates of the score distributions. Two consecutive assignments from four statewide programs were analyzed, with the item response theory (IRT) mean/sigma method used for equating the year 2 to the Year 1 tests. Samples ranged from 7,128 to 17,737 high school students. Effects are more pronounced on the average gains from one year to the next than on individual scores and slightly more so when a three-parameter logistic versus a one-parameter logistic model is used for test calibration. (Contains 1 figure, 7 tables, and 11 references.) (Author/SLD)