This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Jiang, H. Articles by Stout, W. Search for Related Content Social Bookmarking                         What's this?

Improved Type I Error Control and Reduced Estimation Bias for DIF Detection Using SIBTEST

Educational Testing Service

William Stout

University of Illinois

One emphasis in the development and evaluation of SIBTEST has been the control of Type I error (false flagging of non-differential item functioning [DIF] items) inflation and estimation bias. SIBTEST has performed well in comparative simulation studies of Type I error and estimation bias relative to other procedures such as the Mantel-Haenszel and Logistic Regression. Nevertheless it has for a minority of cases that might occur in applications displayed sizable Type I error inflation and estimation bias.

A vital part of SIBTEST is the regression correction, which adjusts for the Type I error-inflating and estimation-biasing influence of group target ability differences by using the linear regression of true on observed matching subtest scores from Classical Test Theory. In this paper, we propose a new regression correction, using essentially a two-segment piecewise linear regression of the true on observed matching subtest scores. A realistic simulation study of the new approach shows that when there is a clear group ability distributional difference, the new approach displays improved SIBTEST Type I error performance; when there is no group ability distributional difference, its Type I error rate is comparable to the current SIBTEST. We have also conducted a power study which indicates that the new approach has on average similar power as the current SIBTEST. We concluded that the new version of SIBTEST, although not perfectly robust, seems appropriately robust against sizable Type I error inflation, while retaining other desirable features of the current version.

Key Words: Keywords: differential item functioning • DIF • item response function • IRF • Mantel-Haenszel • regression correction • SIBTEST

Journal of Educational and Behavioral Statistics, Vol. 23, No. 4, 291-322 (1998)
DOI: 10.3102/10769986023004291


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				C. E. DeMars Modification of the Mantel-Haenszel and Logistic Regression DIF Procedures to Incorporate the SIBTEST Regression Correction Journal of Educational and Behavioral Statistics, June 1, 2009; 34(2): 149 - 170. [Abstract] [Full Text] [PDF]

				C. E. DeMars Polytomous Differential Item Functioning and Violations of Ordering of the Expected Latent Trait by the Raw Score Educational and Psychological Measurement, June 1, 2008; 68(3): 379 - 396. [Abstract] [PDF]

				R. Nandakumar and L. Roussos Evaluation of the CATSIB DIF Procedure in a Pretest Setting Journal of Educational and Behavioral Statistics, January 1, 2004; 29(2): 177 - 199. [Abstract] [PDF]