Selection of Common Items as an Unrecognized Source of Variability in Test Equating: A Bootstrap Approximation Assuming Random Sampling of Common Items

Abstract

The standard error of equating quantifies the variability in the estimation of an equating function. Because common items for deriving equated scores are treated as fixed, the only source of variability typically considered arises from the estimation of common-item parameters from responses of samples of examinees. Use of alternative, equally appropriate anchor sets results in different equating transformations, even when standard errors of common-item parameter estimates are negligible. A bootstrap approximation for quantifying the variability due to common-item selection is derived for a statewide assessment assuming random selection of common items from a large hypothetical item pool. The standard error of equating and the error due to common-item selection constitute only a small fraction of the variability for individual examinee scores. For group means, other sources of error shrink as sample size increases; unaffected by sample size, error due to common-item selection may become the dominant source of variability. © 2014 Copyright Taylor and Francis Group, LLC.