This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Sinharay, S. Articles by Williamson, D. M. Search for Related Content Social Bookmarking                         What's this?

Calibrating Item Families and Summarizing the Results Using Family Expected Response Functions

Educational Testing Service

Matthew S. Johnson

Baruch College

David M. Williamson

Educational Testing Service

Item families, which are groups of related items, are becoming increasingly popular in complex educational assessments. For example, in automatic item generation (AIG) systems, a test may consist of multiple items generated from each of a number of item models. Item calibration or scoring for such an assessment requires fitting models that can take into account the dependence structure inherent among the items that belong to the same item family. Glas and van der Linden (2001) suggest a Bayesian hierarchical model to analyze data involving item families with multiple-choice items. We fit the model using the Markov Chain Monte Carlo (MCMC) algorithm, introduce the family expected response function (FERF) as a way to summarize the probability of a correct response to an item randomly generated from an item family, and suggest a way to estimate the FERFs. This work is thus a step towards creating a tool that can save significant amount of resources in educational testing, by allowing proper analysis and summarization of data from tests involving item families.

Key Words: automatic item generation • Bayesian methods • family expected response function • item model • Markov Chain Monte Carlo

Journal of Educational and Behavioral Statistics, Vol. 28, No. 4, 295-313 (2003)
DOI: 10.3102/10769986028004295


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?