Guessing and the Rasch Model

There is a misunderstanding in the psychometric community that the Rasch model "does not account for guessing" while the three-parameter model does. The truth is the opposite.

Every one agrees that random responses provide no information about ability. Correct responses due to random guessing are quite different from correct responses where some choices have been eliminated before guessing. "Informed guessing" is not "random guessing" because these responses contain information about item difficulty and person ability.

The measurement task is to differentiate between random guessing, which contains no information, and informed guessing, which contains some information. This cannot be accomplished by modelling guessing as an item or even as a person parameter. The pseudo-guessing item parameter in the three-parameter model is useless. It mistakes guessing as a function solely of the item, when, in fact, guessing is an interaction between item propensity to provoke guessing and person proclivity to guess. In addition, the parameterization of guessing obliterates differentiation between random and informed guessing.

A unique strength of the Rasch model is its requirement that the outcome of any interaction between person and item be solely determined by just two parameters, the ability of the person and the difficulty of the item. This requirement establishes a strong framework against which to test data for the presence of anomalous behavior that may influence the estimation of item and person parameters. This identification of anomalies is not restricted to guessing, but addresses any potential measurement disturbance.

Here are three examples that show how Rasch measurement detects guessing in person response records. These analyses were done by IPARM (Smith, 1991) and are used in large scale testing programs like the Dental Admission Test to differentiate between those with low ability who use random responses to complete their test and those with high ability who do not capitalize on chance to improve their scores.

Person 1540 illustrates random guessing. On 6 of the 8 items answered correctly she had less than 0.1 chance of success. On the other 2, less than 0.2. This improbable pattern produced an UnWeighted Total Fit (OUTFIT) of 2.03 and a Weighted Total Fit (INFIT) of 1.63. When items were grouped by difficulty, she failed all 15 easiest items (Subgroup 1), 17 of the 21 medium items (Subgroup 2), but only 10 of the 14 hardest items (Subgroup 3). This produced an Item Difficulty Between Fit of 4.19, a strong indication that her pattern of successes is meaningless.

Person 217 produced fit statistics similar to Person 1540. But his lucky guesses are at the end of his test. This implies that he did not start wholesale guessing until he was out of time at about item 31, at which point he filled out the rest of his answer sheet with "1"s whenever a "right" choice was not immediately apparent to him. His response pattern suggests that a useful estimate of his ability might be obtained from his performance on the first 30 items.

Person 2233 brings out the difference between filling out an answer sheet to complete the test and plodding. Persons 217 and 2233 have similar raw scores (14 and 13) and total test ability estimates (-1.11 and -1.23). But when omits are identified and left unscored a difference emerges.

Person 2233 left her last 19 responses blank, but Person 217's reliance on "1" to fill out his answer sheet beyond item 30 results in 7 additional improbable correct responses. This produces an unweighted total fit of 3.62 and a difficulty between fit of 3.57.

The total test abilities for these persons may imply equality. But their response patterns show that Person 217's random guessing produced a substantial over-estimate of his ability, while Person 2233, when scored only on the 31 items she tried, earns an ability of -0.76, a standard error higher than her whole test -1.23.

The fit analyses which brought out the randomness of Person 1540's responses, the potential utility of Person 217's responses to the first 30 items and the clear differentiation of Persons 217 and 2233 depends on NOT spoiling the frame of reference by forcing an item guessing parameter into the model. That is how the Rasch model "accounts for guessing".

Smith, R.M. (1991), IPARM: Item and person analysis with the Rasch Model. Chicago: MESA Press.

Guessing and the Rasch Model, R Smith … Rasch Measurement Transactions, 1993, 6:4 p. 262-3

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn. George Engelhard, Jr. & Jue Wang	Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt64l.htm

Website: www.rasch.org/rmt/contents.htm