Objective Measurement: Theory into Practice, Vol. 2
Edited By Mark Wilson. Norwood, NJ: Ablex. 1994.
The long-awaited proceedings of IOMW6 (Chicago, 1991) focus precisely on Rasch measurement. Editor Mark Wilson has positioned the 18 papers within three parts: Historical and Philosophical Perspectives, Practice, and Theory. Each part contains early indications of promising new lines of inquiry, reports on pilot work, and assessments of mature methodology. There is widespread use of graphs and tables to speed the readers comprehension of sometimes complex material.
In Part I, Wim van der Linden assesses the place of Rasch measurement within the dominant traditions of physical measurement (Campbell) and social science measurement (Stevens/Luce & Tukey). Specific objectivity is identified as Rasch's major contribution. Joel Michell rediscovers that, from the measurement theory implicit in Aristotle and Euclid, the quantitative status of a variable must be established experimentally, and not rest solely on the assertions and statistical procedures of the researcher. William P. Fisher Jr. points to deep philosophical differences between Rasch and other approaches to measurement. Rasch prescribes what is required of data for valid measures to be derived. Classical raw-score theory (CTT) and IRT describe whatever data is at hand, presuming that an accurate description implies valid measurement. Fisher proposes that only Rasch measurement is compatible with emerging Post-Modern approaches. George Engelhard Jr. contrasts the approaches to sample- and item-invariance of Thorndike, Thurstone and Rasch. The reproductions of original graphs give this chapter relevance even to the most concrete-thinking practitioner. How is it that Thurstone in 1927 and Rasch in 1960 so clearly pictured what invariance means, and yet most test publishers still think a reliability coefficient is all that is required?
Part II addresses two topics: computer-adaptive testing (CAT) and many-facet Rasch measurement. In three papers, Mary Lunz, Betty Bergstrom and Ben Wright report various findings from their joint work implementing CAT for high-stakes certification examinations. They report that CAT does not produce the very short, highly precise tests that early CAT proponents fantasized, but CAT does permit valid, flexible, examinee-friendly testing. A particular finding that has escaped the notice of most CAT test constructors is that allowing examinees to review and change earlier responses does not affect examination validity (see further Lunz & Bergstrom, 1994). Insist that your CAT software allows examinees to review and change earlier responses!
The many-facet Rasch model, though algebraically similar to models proposed by Gerhard Fischer, Susan Embretson and others, was specifically formulated in 1986 to construct measures from judge-awarded ratings. Four chapters reflect the early and aggressive application of this methodology. Michael Linacre gives a quick tour through the features of the model, emphasizing its tolerance of missing data and flexibility towards judging plans. Anne Fisher conducts a case study, validating the Assessment of Motor Process and Skills instrument. Three Figures illustrate the additive nature of the combination of person ability, task simplicity, skill easiness and rater leniency. These Figures also facilitate communication of measurement results to her non-technical audience of therapists. Tom Rehfeldt, picking up an idea from Vol. 1, constructs measures of chemical properties (!) from rank-order data. In his analysis, not only the hierarchy of measures, but also local deviations from it, are key to understanding and improving stain formulation. John Stahl and Mary Lunz address the problem of the crooked measures that so easily result when different sources of information are combined into single measures. Combining written and practical work into a single pass/fail decision is a frequent practice, but always a compromise. The challenge is to make the compromise fair, reasonable and reproducible.
Part III has the nature of a lucky dip. One of these 7 papers may enable you to demolish the obstacle that is holding up an entire measurement endeavor. Robert Jannarone extends the idea of objective measurement to include other models as well as Rasch models, simultaneously illuminating the concept. Henk Kelderman returns to Rasch's multidimensional interpretation of rating scale categories, but elaborates it through the use of a multi-parameter log-linear model. Raymond Adams and Ben Wright consider the effects upon measure estimates of different types of unmodelled variation in stochasticity. In general, the more excessive the noise, the more the estimates collapse towards the center of a test and vice versa. Mark Wilson mounts the sturdy steed of Rasch analysis, seizes a "Quality of Student Life" data set, and tilts at that delusional windmill of statistical sophistication, LISREL. The result is indeterminate. LISREL is too big and complex for its misapplication to ordinal data to be dispatched by a single charge, but, because of the problematic nature of the data, Rasch analysis was not viewed as successful either. Patrick Lee and Hoi Suen consider the measurement implications of dropping subjects with perfect and zero scores from item measure estimation. Their general conclusion is that this linearly transforms the measurement scale, but has minimal practical implications. Barbara Dodd and Ralph De Ayala address the benefit of polytomous items. The more categories there are the more informative (discriminating) the item. Where an item has its categories close together on the variable, the item is very discriminating, but such an item is less discriminating elsewhere than items with widely spaced categories. This paper consolidates the statistical base, so that the analyst can concentrate on extracting the most consistent meaning from the subjects' actual use of the categories. Finally Richard Smith and Chang Miao compare factor analytic and Rasch methods for detecting multidimensionality. They summarize their results in an invaluable table. Rasch analysis is as good or better than factor analysis for detecting the existence of two dimensions in data, except when the two dimensions are about equally prevalent. Then the Rasch "dimension" is an average of the two.
Vol. 1 covers a wider range of topics, but Vol. 2 is more useful as a reference source and aid to the practitioner. Buy it for yourself and recommend it to your library!
John Michael Linacre
Lunz M.E., Bergstrom B.A. (1994) An empirical study of computerized adaptive test administration conditions. Journal of Educational Measurement 31(3) p. 251-263.
Review of Objective measurement: Theory into practice, Vol. 2. Editor: Wilson M, Linacre JM. Rasch Measurement Transactions, 1994, 8:3 p.380
|Rasch Measurement Transactions (free, online)
|Rasch Measurement research papers (free, online)
|Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch
|Applying the Rasch Model 3rd. Ed., Bond & Fox
|Best Test Design, Wright & Stone
|Rating Scale Analysis, Wright & Masters
|Introduction to Rasch Measurement, E. Smith & R. Smith
|Introduction to Many-Facet Rasch Measurement, Thomas Eckes
|Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr.
|Statistical Analyses for Language Testers, Rita Green
|Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar
|Journal of Applied Measurement
|Rasch models for measurement, David Andrich
|Constructing Measures, Mark Wilson
|Rasch Analysis in the Human Sciences, Boone, Stave, Yale
|Análisis de Rasch para todos, Agustín Tristán
|Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez
|Rasch Measurement Forum to discuss any Rasch-related topic
Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement
Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.
|Coming Rasch-related Events
|Oct. 6 - Nov. 3, 2023, Fri.-Fri.
|On-line workshop: Rasch Measurement - Core Topics (E. Smith, Facets), www.statistics.com
|Oct. 12, 2023, Thursday 5 to 7 pm Colombian time
|On-line workshop: Deconstruyendo el concepto de validez y Discusiones sobre estimaciones de confiabilidad SICAPSI (J. Escobar, C.Pardo) www.colpsic.org.co
|June 12 - 14, 2024, Wed.-Fri.
|1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
|Aug. 9 - Sept. 6, 2024, Fri.-Fri.
|On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
The URL of this page is www.rasch.org/rmt/rmt83j.htm