Some Comments About Guessing

When low scoring persons are found to succeed more often on a multiple choice item than the performance of high scoring persons would imply, a suspicion of guessing on that item is often aroused. In a plot of percent scores on that item versus scores on the whole test, this situation appears as an ogive the left lower tail of which does not asymptote toward zero but rather to some greater value such as 20%.


It is tempting to accept this empirical asymptote as an estimate of the item's "guessing parameter". The trouble is that the credibility of that estimate requires the assumption that the tendency to guess on this item is entirely a quality of the item which has exactly the same fixed effect on all persons. But we know from personal experience as well as research that persons vary in their tendency to guess. Some persons guess a lot, some a little, and some hardly ever. Research into who guesses usually shows that only a small proportion of persons do much guessing. This means that the asymptote observed depends on who has been sampled at this score level and that the value of this asymptote must be expected to vary from sample to sample. Attempts to extract an item characteristic that will be invariant with sampling are doomed, as studies of item "guessing" parameters show.

What can be done about this problem? We can make an effort to avoid the occurrence of guessing by taking care to keep the items we ask persons at their ability levels. This is what tailored testing attempts, and, when it works, guessing disappears.

There is another approach which can help when we have asked persons an item so difficult for them that we have provoked them into guessing. The particular low-scoring persons, whose lucky guesses on items too hard for them have prevented the left lower asymptote of the item characteristic curve from approaching zero, can be identified and dealt with. We can use their low scores as indications of their low abilities and we can use the item's low score as an indication of its high difficulty. From these data we can calculate the probability that persons of this ability would succeed on items of this difficulty.

Since, by definition, a lucky guess is an unexpected, i.e., improbable, right answer, we have found exactly the persons who have done the lucky guessing and thus interfered with the expected item characteristic curve.

This not only allows us to remove lucky guesses from the data and thus obtain an estimate of item difficulty unspoiled by lucky guessing. It also allows us to find and correct person scores that have become exaggerated by extra right answers due to lucky guessing rather than ability.

Some comments about guessing. Wright BD. … Rasch Measurement Transactions 1:2 p.9


Some comments about guessing. Wright BD. … Rasch Measurement Transactions, 1988, 1:2 p.9



Rasch Publications
Rasch Measurement Transactions (free, online) Rasch Measurement research papers (free, online) Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch Applying the Rasch Model 3rd. Ed., Bond & Fox Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters Introduction to Rasch Measurement, E. Smith & R. Smith Introduction to Many-Facet Rasch Measurement, Thomas Eckes Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr. Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar Journal of Applied Measurement Rasch models for measurement, David Andrich Constructing Measures, Mark Wilson Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish: Análisis de Rasch para todos, Agustín Tristán Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

To be emailed about new material on www.rasch.org
please enter your email address here:

I want to Subscribe: & click below
I want to Unsubscribe: & click below

Please set your SPAM filter to accept emails from Rasch.org

www.rasch.org welcomes your comments:

Your email address (if you want us to reply):

 

ForumRasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
May 17 - June 21, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 12 - 14, 2024, Wed.-Fri. 1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
June 21 - July 19, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 5 - Aug. 6, 2024, Fri.-Fri. 2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals
Aug. 9 - Sept. 6, 2024, Fri.-Fri. On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri. On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com

 

The URL of this page is www.rasch.org/rmt/rmt12a.htm

Website: www.rasch.org/rmt/contents.htm