Review of Objective measurement: Theory into practice, Vol. 2.

The long-awaited proceedings of IOMW6 (Chicago, 1991) focus precisely on Rasch measurement. Editor Mark Wilson has positioned the 18 papers within three parts: Historical and Philosophical Perspectives, Practice, and Theory. Each part contains early indications of promising new lines of inquiry, reports on pilot work, and assessments of mature methodology. There is widespread use of graphs and tables to speed the readers comprehension of sometimes complex material.

In Part I, Wim van der Linden assesses the place of Rasch measurement within the dominant traditions of physical measurement (Campbell) and social science measurement (Stevens/Luce & Tukey). Specific objectivity is identified as Rasch's major contribution. Joel Michell rediscovers that, from the measurement theory implicit in Aristotle and Euclid, the quantitative status of a variable must be established experimentally, and not rest solely on the assertions and statistical procedures of the researcher. William P. Fisher Jr. points to deep philosophical differences between Rasch and other approaches to measurement. Rasch prescribes what is required of data for valid measures to be derived. Classical raw-score theory (CTT) and IRT describe whatever data is at hand, presuming that an accurate description implies valid measurement. Fisher proposes that only Rasch measurement is compatible with emerging Post-Modern approaches. George Engelhard Jr. contrasts the approaches to sample- and item-invariance of Thorndike, Thurstone and Rasch. The reproductions of original graphs give this chapter relevance even to the most concrete-thinking practitioner. How is it that Thurstone in 1927 and Rasch in 1960 so clearly pictured what invariance means, and yet most test publishers still think a reliability coefficient is all that is required?

Part II addresses two topics: computer-adaptive testing (CAT) and many-facet Rasch measurement. In three papers, Mary Lunz, Betty Bergstrom and Ben Wright report various findings from their joint work implementing CAT for high-stakes certification examinations. They report that CAT does not produce the very short, highly precise tests that early CAT proponents fantasized, but CAT does permit valid, flexible, examinee-friendly testing. A particular finding that has escaped the notice of most CAT test constructors is that allowing examinees to review and change earlier responses does not affect examination validity (see further Lunz & Bergstrom, 1994). Insist that your CAT software allows examinees to review and change earlier responses!

The many-facet Rasch model, though algebraically similar to models proposed by Gerhard Fischer, Susan Embretson and others, was specifically formulated in 1986 to construct measures from judge-awarded ratings. Four chapters reflect the early and aggressive application of this methodology. Michael Linacre gives a quick tour through the features of the model, emphasizing its tolerance of missing data and flexibility towards judging plans. Anne Fisher conducts a case study, validating the Assessment of Motor Process and Skills instrument. Three Figures illustrate the additive nature of the combination of person ability, task simplicity, skill easiness and rater leniency. These Figures also facilitate communication of measurement results to her non-technical audience of therapists. Tom Rehfeldt, picking up an idea from Vol. 1, constructs measures of chemical properties (!) from rank-order data. In his analysis, not only the hierarchy of measures, but also local deviations from it, are key to understanding and improving stain formulation. John Stahl and Mary Lunz address the problem of the crooked measures that so easily result when different sources of information are combined into single measures. Combining written and practical work into a single pass/fail decision is a frequent practice, but always a compromise. The challenge is to make the compromise fair, reasonable and reproducible.

Part III has the nature of a lucky dip. One of these 7 papers may enable you to demolish the obstacle that is holding up an entire measurement endeavor. Robert Jannarone extends the idea of objective measurement to include other models as well as Rasch models, simultaneously illuminating the concept. Henk Kelderman returns to Rasch's multidimensional interpretation of rating scale categories, but elaborates it through the use of a multi-parameter log-linear model. Raymond Adams and Ben Wright consider the effects upon measure estimates of different types of unmodelled variation in stochasticity. In general, the more excessive the noise, the more the estimates collapse towards the center of a test and vice versa. Mark Wilson mounts the sturdy steed of Rasch analysis, seizes a "Quality of Student Life" data set, and tilts at that delusional windmill of statistical sophistication, LISREL. The result is indeterminate. LISREL is too big and complex for its misapplication to ordinal data to be dispatched by a single charge, but, because of the problematic nature of the data, Rasch analysis was not viewed as successful either. Patrick Lee and Hoi Suen consider the measurement implications of dropping subjects with perfect and zero scores from item measure estimation. Their general conclusion is that this linearly transforms the measurement scale, but has minimal practical implications. Barbara Dodd and Ralph De Ayala address the benefit of polytomous items. The more categories there are the more informative (discriminating) the item. Where an item has its categories close together on the variable, the item is very discriminating, but such an item is less discriminating elsewhere than items with widely spaced categories. This paper consolidates the statistical base, so that the analyst can concentrate on extracting the most consistent meaning from the subjects' actual use of the categories. Finally Richard Smith and Chang Miao compare factor analytic and Rasch methods for detecting multidimensionality. They summarize their results in an invaluable table. Rasch analysis is as good or better than factor analysis for detecting the existence of two dimensions in data, except when the two dimensions are about equally prevalent. Then the Rasch "dimension" is an average of the two.

Vol. 1 covers a wider range of topics, but Vol. 2 is more useful as a reference source and aid to the practitioner. Buy it for yourself and recommend it to your library!

John Michael Linacre

Lunz M.E., Bergstrom B.A. (1994) An empirical study of computerized adaptive test administration conditions. Journal of Educational Measurement 31(3) p. 251-263.

Review of Objective measurement: Theory into practice, Vol. 2. Editor: Wilson M, Linacre JM. … Rasch Measurement Transactions, 1994, 8:3 p.380

Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt83j.htm

Website: www.rasch.org/rmt/contents.htm