The Rationale for Precision Measurements

Why measure more precisely? Here are the reasons given by a physicist (G. Feinberg, 1992):

"Why should one wish to make measurements with ever increasing precision? Because the whole history of physics proves that a new discovery is likely to be found lurking in the next decimal place. (Floyd K. Richtmyer, 1931)"

Social Science does not yet capitalize on precise measurement in the building of theory. Indeed "qualitative" methodologists reject quantification.

"Great precision in measurement contributes to knowing both the great unifying principles, such as the law of conservation of energy, and to know the detailed particulars about individual phenomena. Most basically, it helps to shrink the range of our ignorance."

The unifying principles of social science, such as "People are scared of losing", are equivalent to pre-quantitative statements such as "What goes up must come down". Without an encompassing quantitative framework, more precise details increase confusion, not knowledge.

"The criterion of what constitutes a precision measurement changes over time ... Michelson's measurements of the speed of light, which represented that best that could be done in the early part of the twentieth century, are now rivaled in demonstrations done for first year physics students."

The general precision of social science measurement has changed little in 75 years (since the introduction of MCQs), though technological advances suggest that it should have.

"The contrasts between measurements which can be compared to a precise theoretical prediction, and measurements for which no accurate theory is available, is sometimes expressed through the distinction between precision tests of theory and precision measurements from data."

Nearly all social science measurement is in the category of precision measurement. An example of a precision test is the Lexile theoretical determination of reading difficulty and its prediction of empirical results.

"When an accurate theoretical prediction is available for comparisons with some precision measurement it is relatively easy to know when a problem exists, either with the measurement or with the theory."

Evaluation of construct validity, though usually far short of accurate prediction, often detects this type of problem.

"There is in general no simple relation between the precision with which measurements can, have, and should be done, and the accuracy of the theory used to understand those measurements."

In the Social Sciences, the precision of measurements is generally far higher than the accuracy of theory. In Physics, vice-versa.

"I think it can be argued that many of the profoundest experimental discoveries are made when there is no theoretical guidance available about what to expect."

There is hope here for Social Science!

"There are probably many other, yet unrecognized instances in which precisely measurable quantities involving macroscopic [large-scale] systems are related to simple properties of their components, and it seems likely that the search for and measurement of such quantities will be at the frontiers of future research."

Physics looks for simplicity in the midst of every-increasing complexity. Social Science in contrast is preoccupied with rejecting simplicity and looking for complexity, perhaps to disguise its lack of useful theory.

"Another point to what is worth measuring precisely involves the contrast between the permanent properties of systems [cf. test items], and the measurement of variable qualities [cf. examinees]. Physicists are usually interested in accurate [i.e., very precise] measurement of quantities whose value does not change with time."

In Social Science, a high precision measure of examinee ability is demanded for high-stakes pass-fail decisions, though it is evident that the examinee will not remain at that ability for long, perhaps only instantaneously.

"There has over time been a systematic increase in the precision of the estimates and a systematic tendency to underestimate errors. Of course the situation has been much worse for some results that do not qualify as precision measurements."

Social Science relies heavily on reliability coefficients and standard-error-based effect-sizes,

yet it is doubtful if the underlying measures attain to the status of "precision measurements" as viewed by physicists. Consequently, Social Science findings are more insecure even than "Hubble constant" for the expansion of the Universe. In 1929, Edwin Hubble estimated this to be 500±60, but it is now 75±25. Social Science is just starting on the road to "constants" with the rule-of-thumb "one logit per grade-level".

"Perhaps the main lesson to be learned is that we should be somewhat skeptical of the precise values and their associated errors, but that we can rely on the prospect that future experiments, by physicists who are dedicated to pushing the envelope of what can be measured, will surely produce values of considerably greater precision."

Social Science! Here we are!

Feinberg, G. (1992) On knowing things better and better. In M.E. Zeller (Ed.) A Festschrift in honor of Vernon W. Hughes. Singapore: World Scientific.

Feinberg, G. The Rationale for Precision Measurements. … Rasch Measurement Transactions, 2000, 14:3 p.764-5

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn. George Engelhard, Jr. & Jue Wang	Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt143h.htm

Website: www.rasch.org/rmt/contents.htm