Too Many Factors in Factor Analysis?

Guilford (April 1941) reports that "a factor analysis of the ten sub-tests of the Seashore test of pitch discrimination [of 0.5 to 30 cycles per second differences] revealed that more than one ability is involved. One factor, which accounted for the greater share of the variances, had loadings that decreased systematically with increasing difficulty [from 30 to 0.5 cps differences]. A second factor had strongest loadings among the more difficult items with frequency differences of 2 to 5 cps. A third had strongest loadings at differences of 5 to 12 cps. No explanation for the three factors is apparent, but the hypothesis is accepted that they represent distinct abilities. In tests so homogeneous as to content and form, where a single common factor might well have been expected, the appearance of additional common factors emphasizes the importance of considering the difficulty level of test items, both in the attempt to interpret new factors and in the practice of testing. The same kind of item may measure different abilities according as it is easy or difficult for the individuals to whom it is applied" (p.67).

Can there really be three distinct hearing discrimination abilities within such a narrow range of frequency differences? Guilford, an expert on psychophysics, cannot explain how that could be. He allows his data analysis to control his thinking. Yet he gives us a clue: "the importance of considering the difficulty level of test items.. in the attempt to interpret new factors."

Let us follow this clue to Ferguson (October 1941), a paper with no direct reference to Guilford: "In general, the greater the number of degrees of difficulty among the items in a test or among the tests in a battery, the higher the rank of the matrix of inter-correlations; that is differences in difficulty are represented in the factorial configuration as additional factors" (p. 323).

Is this the reality underlying Guilford's three factors? Not three abilities, but three item difficulty levels pertaining to one ability? In attempting to discover the meaning of the three factors of pitch discrimination, appeal to Ferguson's explanation of general statistical effects is more convincing than to Guilford's hypothesis of an inexplicable local psychophysical manifestation.

Ferguson reports the predicament that factor analysis identifies difficulty levels with different factors. Rasch analysis avoids this predicament by constructing one latent variable that spans all difficulty levels. Since the analyst's motivation is to span the data set with one meaning, rather than to stratify it, Rasch analysis is the method of choice.

Trevor G Bond 1994 RMT 8:1 p. 347
School of Education
James Cook University of North Queensland
Australia

Ferguson GA. 1941. The factorial interpretation of test difficulty. Psychometrika 6:5 323-329.

Guilford JP. 1941. The difficulty of a test and its factor composition. Psychometrika 6:2 67-77.

Note: A further confusing aspect of Factor Analysis is the analyst's choice of diagonalization, maximization, rotation and obliqueness. These can make highly-correlated content strands, which from the Rasch perspective are part of the dimension, appear to the casual reader to be orthogonal dimensions. Also small perturbations in the data can be made to appear to be large factors. If the factor loadings are plotted against item difficulty, it can be seen which factors may be due to nodes in the item distributions. To identify which factors are distribution-related, simulate data with the estimated Rasch measures as generators, and then apply EFA in order to obtain a baseline against which to interpret the empirically-based EFA. Principal Components Analysis of Residuals avoids many of the problems associated with Common Factor Analysis.

Too Many Factors?

"Therefore, one might expect the emergence of only one factor when a factor analysis would be performed on all newly defined subsets [of unidimensional items]. However, factor analysis of the newly defined subsets yielded two factors. Further inspection of the factor plot showed that the emergence of a second factor could be considered as an artefact due to the skewness of the subset scores."
Van der Ven, A.H.G.S., & Ellis, J.L. (2000). A Rasch Analysis of Raven's Standard Progressive Matrices. Personality and Individual Differences, 29 (1), 45-64.

reported in RMT 16:1

Duncan in his 1984 book "Notes on Social Measurement" has a good discussion on the limitations of factor analysis. In particular how inter-item correlations and item loadings are affected by item difficulty. Thus if a pool of items containing both easy and difficult items is factor analyzed, it will produce 2 factors, even if all the items are on exactly the same topic.
Scott Bayley

Duncan O.D. (1984). Notes on social measurement: Historical and critical. New York: Russell Sage Foundation.

For more information,
The Impact of Rasch Item Difficulty on Confirmatory Factor Analysis , S.V. Aryadoust … Rasch Measurement Transactions, 2009, 23:2 p. 1207
Confirmatory factor analysis vs. Rasch approaches: Differences and Measurement Implications, M.T. Ewing, T. Salzberger, R.R. Sinkovics … Rasch Measurement Transactions, 2009, 23:1 p. 1194-5
Conventional factor analysis vs. Rasch residual factor analysis, Wright, B.D. … 2000, 14:2 p. 753.
Rasch Analysis First or Factor Analysis First? Linacre J.M. … 1998, 11:4 p. 603.
Factor analysis and Rasch analysis, Schumacker RE, Linacre JM. … 1996, 9:4 p.470
Too many factors in Factor Analysis? Bond TG. … 1994, 8:1 p.347
Comparing factor analysis and Rasch measurement, Wright BD. … 1994, 8:1 p.350
Factor analysis vs. Rasch analysis of items, Wright BD. … 5:1 p.134

Too many factors in Factor Analysis? Bond TG. … Rasch Measurement Transactions, 1994, 8:1 p.347

Rasch Books and Publications
Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, 2nd Edn. George Engelhard, Jr. & Jue Wang	Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan
Other Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt81p.htm

Website: www.rasch.org/rmt/contents.htm