The Necessity of Construct Theory

Measurement is the process of converting observations (e.g., counts) into measures (quantities) via a construct theory. The Rasch Model states a requirement for the way observations and construct theory combine in a probability model to make measures. There is no other combination of observation and theory that produces sufficiency invariance and objectivity in the resultant measures.

As scientists, we cherish parsimony (do much with little) and as statisticians, we cherish uniqueness. This is why we speak of data that fit the Rasch Model - not of fitting some model, Rasch or otherwise, to data. The Rasch Model combines the three components in the definition of measurement (observation, theory and measure) into a simple, elegant and, in more important respects, unique representation.

It is unfortunately too easy to apply the Rasch Model without a construct theory. Rasch programs enable practitioners to estimate item difficulties and person measures on a logit or transformed logit scale. Item maps can be built and variables named, all without any a priori of our intention. In my own case, this practice reminds me of how quickly I could name a factor structure coming out of an "exploratory" factor analysis. Never once did I specify, prior to data collection, what I expected to find in the data. To compound the heresy, I wrote a suite of Educational and Psychological Measurement articles on factor replicability and factor invariance.

If we are to have human science(s), we must know enough about the constructs we intend to measure to be able to specify item calibrations prior to data collection. These item calibrations come from a construct theory. Theory and item engineering improve as we bring observed item difficulties and theory-based item calibrations into closer and closer coincidence. The just announced "temporal thermometers" are a recent example of this kind of improvement in the measurement of human temperature.

"Exploratory" Rasch analysis is no substitute, because, without a construct theory and associated specification equation, the data does not "bite". Just like in exploratory factor analysis, exploratory Rasch analysis produces too many successful studies. We need to increase our failure rate and the shortest route is to make our intentions explicit and then see if the data confirm to the theoretical expectations.

What price do we pay for extensive reliance on exploratory Rasch analysis? We make it almost impossible to demonstrate how a common theory and specification equation can unify the diverse instruments purporting to measure a single construct.

Two hundred and fifty instruments (tests) for measuring "reading comprehension" each with proprietary non-exchangeable scales can be unified only if there is
(1) a compelling demonstration that most of these instruments measure a single unidimensional construct called "reading comprehension",
(2) a common supplemental metric is proposed that is based on a construct theory and supported by a specification equation capable of explaining high proportions of item difficulty variance within and across the candidate tests,
(3) there is some compelling advantage beyond parsimony to encourage the "community" to adopt the standard (the possibility of linking test scores to books is such an application for reading comprehension), and
(4) a business model is proposed that makes it advantageous for the power elite (e.g., publishers, academicians, key users, policy makers) to support unification.

Adoption of the metric system in the US in the 1970s failed criteria 3 and 4 and, ergo, no metric system in our everyday lives despite a huge expenditure.

There is a simple thought experiment that can inform us regarding how well we understand the construct under study. If presented with an instrument purportedly measuring the construct, can we use our knowledge about the construct-associated (construct theory) specification/calibration equation(s) together with item engineering rules to produce a clone or copy of the instrument - such that the score-to-measure table for the clone is identical to that of the original instrument?

Moving now from theory to experiment, if the clone produces measures that are statistically equivalent to those produced by the original, then we have demonstrated that we understand what we are measuring. We can reproduce the phenomena experimentally and we are in control of the major contaminants.

The first time we attempted the above protocol with "reading comprehension", we failed. After some four years and 30 attempts, we are getting pretty good at building reading comprehension tests based on theory alone that produce raw score-to-measure correspondences that conform to design requirements. With this kind of control, it is practically feasible to contemplate unifying the measurement of reading.

Finally, William P. Fisher, Jr. states: "I think that the need for these metrological networks will eventually be seen simply because of the way independent experiments addressing the same variable are converging on common constructs!" I want this to happen because human science is impossible without it. But, the unification process that precedes consensus on a unit of measurement for a construct will be built on improved construct theories and resultant improved instrumentation, not more exploratory Rasch analysis.

Exploratory studies don't converge on anything. As always, the great unifier is substantive theory. See www.lexile.com for a report card on the unification of the measurement of reading comprehension in English and Spanish.

Jack Stenner

Trevor Bond & Christine Fox (2001, 2001, pp.xxi-xxii) comment:
"The Rasch approach to investigating measures illustrates an appreciation of the dialectical nature of the measurement process: Theory informs practice, and practice informs theory. ... From our viewpoint, every item you write puts either the theory, your understanding of it, or your ability to translate it into an empirical indicator on the line. Tossing out items might be fine if you have trawled up a huge bunch of possible items and are looking for the probables to keep in your test. However, if you have developed items that encapsulate for you the very essence of the theory or construct to which you are attached, then all the items are probables. In fact, you might consider all of them as somehow indispensable! Thus, whereas your theory tells you how to write the item, the item's performance should tell you about your theory, your understanding of it, or your item-writing skills."

The Necessity of Construct Theory. Stenner J. … Rasch Measurement Transactions, 2001, 15:1 p.804-5

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn. George Engelhard, Jr. & Jue Wang	Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt151q.htm

Website: www.rasch.org/rmt/contents.htm