MESA Psychometric Laboratory

The mission of the MESA Psychometric Laboratory (University of Chicago) is to extend the theoretical basis and practical application of objective measurement.

One way the Lab does this is by hosting Midwest Objective Measurement Seminars (MOMS) the first Friday of every December and the last Friday of every May (which sometimes falls in June!). The usual MOMS program is 15 to 16 measurement research presentations by midwest social scientists and MESA students. The presentations and discussions are always interesting and often significant and provocative. RM SIG members have a standing invitation to participate.

A second way is to provide a congenial base and technical support for the measurement research of visiting scholars and post-docs. RM SIG members have a standing invitation to visit the MESA Lab whenever they can. The Lab is always amenable to helping a prospective visitor work out a proposal for external funding. For visitors who can room and board themselves, the Lab can always provide a comfortable place to work, computing support, an informed audience and enthusiastic encouragement.

Graham Douglas (MESA '75), Senior Lecturer, University of Western Australia visited the MESA Lab for six weeks this fall to study the exact probabilities associated with each of the possible patterns of response to short tests. Graham used these probabilities to identify the level at which the Rasch whole response pattern fit statistics were sensitive to improbable behaviors. His results show that standardized fit statistics of absolute value greater than two are always sufficiently improbable to be considered statistically significant. This is just the theoretical conclusion, so often criticized, that is almost always used in practice.

A third way the Lab pursues its mission is through student research. Here are some notes on the 1989 MESA dissertations.

In educational administration, Wan-Rani Abdullah, Senior Research Officer, Educational Planning and Research, Malaysian Ministry of Education, developed objective measures of Malaysian teaching styles, the definition of which clarified the working hierarchy of teacher responsibilities and brought out significant differences between urban and rural teachers. Now Wan Rani is applying "the knowledge and ideas which I gathered at U. of C., especially Rasch measurement and HLM" [hierarchical linear modelling], to his UNESCO project on educational wastage and his study of performance differences in languages, mathematics and science.

In medical career maturation, Lih Mei Yang, constructed specialty-specific measures of professional experience preferences for nine medical specialties. She used the operational definitions of these measures to show how the specialties differ substantially in the experiences they afford and how practitioner-specialty misfit predicts disenchantment and often a change of specialty.

In professional education, Wendy Rheault, Chair and Professor of Physical Therapy, Chicago Medical School, constructed quantitative comparisons of three graduate programs in Physical Therapy which brought out basic similarities in institutional effects on learning styles and also the substantial dominance of individual growth curves over the institutional effects.

In medical certification, Dorthea Juul, Assistant Professor, School of Medicine, University of Illinois, used Geoff Master's partial credit analysis to show that response patterns on patient management problems could be used to identify response sequences which defined sensitive and richly valid measurements of medical student competence.

When educational measurement depends on test data, the questions are: How can the analyst be confident that the measurement process succeeded? How can bias which threatens the measuring process be detected, identified and counteracted? What is the statistical precision of the estimates? In his methodological dissertation, "Estimating Measurement Error and its Effects on Statistical Analysis", Ray Adams, Senior Research Officer, Australian Council for Educational Research, takes on the whole range of statistical and empirical biases that might impinge on the measuring process, consolidates them into a single generic formulation, demonstrates the directions and magnitudes of their effects and develops and explains the statistical methods necessary to recognize and adjust for them.

A new and challenging measurement frontier in social science research involves the longstanding and widespread use of "expert" judges to obtain ratings of professional performance. Judging is required in the assessment of all artistic and skill-oriented competencies. The measurement analysis and management of these kinds of data is especially urgent in health care certification. In his dissertation "Many-Facet Rasch Measurement" (available through MESA Press) John Michael Linacre has extended Rasch measurement theory to data structures in which the basic observations arise from a situation with more than two facets, e.g., judges to be calibrated as well as items, so that persons can be measured judge-free as well as item-free. The theory of many-facet measurement analysis has been implemented in Mike's computer program FACETS and employed by the American Society of Clinical Pathology and the University of Illinois, Department of Occupational Therapy, for the measurement of medical specialty skill and patient rehabilitation. Not only is Mike's program FACETS now available from MESA Press, but he has also produced, under the name BIGSCALE [later BIGSTEPS] an enhanced PC-based successor to our previous programs, BICAL, MSCALE and CREDIT.

Finally in the arts, Carol Myford, by calibrating the four facets - judges, tasks, items, actors - involved in the measurement of theatrical performance, was able to document fundamental communalities among novice, buff and expert judges with respect to agreement on dramatic quality and also to identify persuasive quantitative differences in the judging behaviors of these three types of judges.

Other MESA News

Geoff Masters (MESA '80), Assistant Director (Measurement), Australian Council for Educational Research, has just distributed 112,000 Rasch KIDMAPS to New South Wales teachers and parents. A KIDMAP is an individually tailored graphic report which not only pictures a student's standing on the relevant educational variable but also marks out exactly which questions the student did surprisingly well or surprisingly poorly. The result provides visually explicit and easy to interpret general information about the student's standing among the objectives of the curriculum as well as simultaneous individual information about his particular strengths and weaknesses.

Mark Wilson (MESA '84), Assistant Professor of Education, University of California at Berkeley, organized, hosted in Berkeley during March '89 and will soon publish the impressive proceedings of the Fifth International Objective Measurement Workshop. Don't miss it!

George Engelhard (MESA '85), Assistant Professor of Education, Emory, is using his Spencer Fellowship to advance his work on "The History and Philosophy of Educational and Psychological Measurement".

William Fisher (MESA '88), Research Associate, Research and Evaluation, Marianjoy Rehabilitation Center, is developing Rasch measurement of the physical functioning of stroke disabled persons. William continues his work on "Measurement and the History of Science", a book which complements the one George Engelhard is writing. William also works with Adeline Mesquelier on the quantification of metaphor.

Benjamin D. Wright
University of Chicago

MESA Psychometric Laboratory. Wright BD. … Rasch Measurement Transactions, 1990, 3:4 p.87

Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt34g.htm

Website: www.rasch.org/rmt/contents.htm