Quality by Design: Taguchi and Rasch

"High quality at low cost" proclaims the product design revolution brought about by innovative Japanese statistician Genichi Taguchi. Taguchi methods achieve in industry what Rasch measurement offers in social science: a fundamental reconceptualization of what needs to be done to solve a problem and of how to do it.

Taguchi methods slashed costs and increased consumer satisfaction, and so were eagerly adopted by Japanese industry during the 1950's. Now they are well-established worldwide. Rasch's methods clarify and expedite solution of complex social science issues. But so far they have been largely ignored by social science researchers, practitioners and politicians.

Here are parallels between Taguchi and Rasch that show how Rasch measurement can lead to social science improvements comparable to those Taguchi methods have yielded in industry:

1) Capitalizing on expertise:

"At the heart of Taguchi's techniques is the strong reliance on the knowledge of a product or process that an engineer brings with him to an experiment" (Ealey 1988 p.120, emphasis his).

Rasch unidimensionality - one thing at a time, as in physical measurement - requires the test constructor to conceive and express clearly, as test items, the one construct to be investigated. Neither Rasch nor Taguchi expect perfection from the expert, but both invite expert input because the firmer the foundation, the better the outcome. Users of either method increase their capability to use that method effectively and also their knowledge and understanding of the variable (or product) to which the method is applied. In fact, only when improvements in knowledge occur has the method been used successfully.

In contrast, conventional techniques merely report the probability of a null hypothesis. Researchers obtain little insight into either statistical technique or substantive issues.

2) Noise minimization not a primary goal:

Conventional approaches, such as G-Theory, aim at parameterizing and minimizing noise (Marcoulides 1993). These yield designs that are optimum only when noise is exactly as hypothesized, i.e., well-behaved. In industrial production, well-behaved noise can sometimes be minimized by expensive components. The inevitability of ill-behaved noise, however, requires detailed quality inspection of each product and risks dissatisfied customers. This flaw is manifest in G-Theory. All noise patterns yet to be encountered by the Decision Study must be identical to those in the pilot Generalizability Study. But there is no way to force any future data collection to conform to such an ideal, or often even of knowing to what extent it happens to do so.

Taguchi develops product designs that are robust against noise of any type and level. He devises a bread recipe that produces good, cheap bread regardless of variations in baking, delivery and home storage. Compared to conventional approaches, this is analogous to a driver looking for a fast way to get to work under all weather and traffic conditions, rather than for the fastest way to get to work under a limited pattern of particular conditions, e.g., bad weather, but light traffic (Ealey p. 118).

Rasch measurement goes beyond even Taguchi. Conventional techniques try to eliminate noise from product quality; Taguchi makes noise irrelevant to product quality; Rasch recognizes noise as an essential component of product quality. It is noise that permits signals of different strengths to be located along a continuum. Without noise, Rasch measurement, indeed any kind of useful measurement, is impossible.

Both Taguchi and Rasch detect and reject noise spikes - wrong ingredients used accidentally in a bread recipe, lucky guesses by a poor performer. But detecting such clearly aberrant outcomes requires less effort and expense than conventional quality control.

3) Parameter separability and generality:

Conventional research favors fully-crossed factorial designs in which every level of every design parameter is combined with every level of every other parameter. These designs enable a best combination of parameters to be inferred. When a full design is impractical, then one parameter is changed at a time. This leads to parameter values that are optimal under limited conditions only. Alternatively, a randomized design may be used so that parameters thought to be incidental can be assumed to cancel.

Taguchi takes a different approach. He parameterizes the effect of each level of each aspect of product quality at just one value. This value is modeled to hold good across all levels of all other parameters. Thus, he constructs a model for each aspect of product quality in which the effects of the levels of one parameter can be separated from the effects of the levels of the other parameters. Taguchi obtains this with minimal balanced experimental designs, "orthogonal arrays", that require 10% of the effort of a full factorial design to obtain equivalent results. Findings from his orthogonal designs infer product quality for all combinations of all levels of all parameters, and hence identify an optimal product design.

Rasch also seeks parameter separation and generality, but employs probability theory and statistical sufficiency to achieve this end. This approach allows the practitioner great freedom to invent efficient research designs. Rasch measures also enable any specific outcome to be inferred - Mary's success probability on Question 3.

4) Interactions and misfit:

Conventional techniques model and quantify parameter interactions. Large misfit dictates a new statistical model. Taguchi and Rasch abjure such interactions, but check for their uninvited presence because they disturb parameter separability, generality and ultimately the entire conceptual structure. Misfit requires reexamination and sometimes replacement of data, but never a change in the statistical model which formulates the researcher's intentions.

In Taguchi's case, once an optimal combination of parameter levels is identified, a "confirmation experiment" is performed to verify that product quality is as predicted. If any aspect of product quality is not as predicted, then an interaction is suspected and the data is reexamined, and perhaps the research problem reconceptualized.

Rasch is more thorough. Every observation is inspected for signs of interaction (misfit). Unexpected observations are identified and diagnosed. Idiosyncratic anomalies - lucky guesses, careless errors - may be sidelined for measurement purposes. Patterns of misfit, however, usually raise doubt about particular parameters. Is Question 4 really a Math question? Was George's test form mis-scanned? Large patterns of misfit may cause reconsideration of the entire test and the variable it purports to measure.

5) Encouraging graphical techniques:

Conventional statistics tend to summarize findings numerically in single significance tests and as tables of numbers. Taguchi and Rasch encourage graphical techniques in order to see, understand and evaluate results, and also in order to communicate them to others quickly and unambiguously. "The most routine and basic analysis for Taguchi methodology is graphical" (Bendell 1989 p.11).

6) "Just-in-time" engineering:

Conventional techniques seldom begin data analysis until all data is collected. Unplanned revisions or surprise events during an experiment usually invalidate the experiment and necessitate a new one.

Taguchi methods encourage design changes during manufacture. The last product produced can appear quite different from the initial product.

Rasch measurement encourages continuous data analysis - starting as soon as data begins to be collected. Missing data robustness permits numerous and continuing changes in the test and testing conditions without corrupting the analysis. The final test instrument can appear quite different from the initial one.

7) Reaction of statistical community:

"Many statisticians were nonplussed by the cookbook nature of Taguchi's techniques. They concentrated on attacking isolated segments of his methodology... The statisticians' objections seemed to center on the fact that Taguchi wasn't playing by the established rules, that he was disregarding aspects of the full-factorial experimental-design techniques... Even as the statistical community locked ideological horns with Taguchi's system, U.S. manufacturers began reporting incredible results through its use" (Ealey p.152-3).

Even so with Rasch measurement!

John M. Linacre

Bendell T. (1989) Taguchi Methods: Proceedings of 1988 European Conference. London: Elsevier.

Ealey L. A. (1988) Quality by design: Taguchi methods and U.S. industry. Dearborn MI: ASI Press.

Marcoulides G.A (1993) Maximizing power in generalizability studies under budget constraints. Journal of Educational Statistics 18(2) p.197-206.

Quality by design: Taguchi and Rasch. Linacre JM. … Rasch Measurement Transactions, 1993, 1993, 7:2 p.292-3

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn. George Engelhard, Jr. & Jue Wang	Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt72j.htm

Website: www.rasch.org/rmt/contents.htm