Factor Analysis versus Rasch Analysis of Items

Factor Analysis versus Rasch Item Analysis of Items

Theory:
	Factor Analysis	Rasch Analysis
T1	Motivation	Hopes to describe data by covariance	Intends to use data for measurement
T2	Model	Ordinal scores mistaken for interval measures which have been observed without error	Ordinal responses modelled as stochastic manifestations of linear parameters, estimated with measurement error
T3	Statistical basis	Covariance matrix of items over examinee	Probability of responses calculated from item and examinee parameters
T4	Outcome of analysis	Factors summarizing the covariance matrix	Operational definition of variable by item calibration and person measure
T5	Interpretation	Factors named to represent correlated items, but with easily disputed meanings	Variable defined by item text manifests the underlying concept. Unexpected outcomes signal misconceptions
T6	Principal statistics	Factor loadings (covariances of item scores with factor). Factor scores (from regressing item scores on loadings).	A linear measure, error and fit statistics, for each item, examinee, and any other element modelled
T7	Largest variance component	The factor with the most variance (largest eigenvalue)	The empirical manifestation of the underlying variable
T8	Other variance components	As many factors as diagonal elements in correlation matrix	Modelled measurement error and unmodelled misfit
T9	Other variance criterion	Factor eigenvalues greater than, usually, 1.4	Standardized misfit statistics greater than, usually, 2.0
T10	Measurement and sampling error	Factors with small eigenvalues confound these sources of error	Measurement error as standard errors, sampling error as standard deviations of examinee and item distributions
T11	Missing data	List-wise deletion loses data. Pair-wise deletion biases factor structure	Measures, standard errors and fit statistics based on all observed data

Diagnosis:
	Factor Analysis	Rasch Analysis
D1	Multi-modal distribution of examinees or items	A factor for each mode, some with large eigenvalue	Documented in item and person measure distributions. No effect on misfit
D2	Irregular examinee response	Slight variance increase. Not determinable from factors	Large misfit for examinee, and somewhat increased misfit for items to which examinee responded unexpectedly
D3	Identification of item bias (DIF, Differential Item Functioning) on many test items	By discovering factor scores correlated with group membership, and then items with loadings on those factors	Exploratory: By discovering items with significant differences between their group measures Confirmatory: By partitioning residuals of suspect items between groups to estimate bias size, significance and homogeneity
D4	Identification of solitary biased item	Undetectable; bias eigenvalues insignificant	As above, plus item misfit (particularly information-weighted (INFIT) statistics)
D5	Major multi-dimensionality (items: 50% math, 50% reading)	After rotation, one math factor and one reading factor	Variable combines math and reading items with low person separation, reliability, and patterns of poor person fit [PCA of Rasch residuals]
D6	Minor multi-dimensionality (items: 95% math, 5% reading)	One math factor; insignificant reading factor	Variable defined by math items, significant misfit in reading items
D7	Miskeyed multiple-choice item	Undetectable; eigenvalues insignificant.	Large item misfit statistic or item calibration contradicts construct

Theory:

Factor Analysis

Rasch Analysis

Motivation

Hopes to describe data by covariance

Intends to use data for measurement

Model

Ordinal scores mistaken for interval measures which have been observed without error

Ordinal responses modelled as stochastic manifestations of linear parameters, estimated with measurement error

Statistical basis

Covariance matrix of items over examinee

Probability of responses calculated from item and examinee parameters

Outcome of analysis

Factors summarizing the covariance matrix

Operational definition of variable by item calibration and person measure

Interpretation

Factors named to represent correlated items, but with easily disputed meanings

Variable defined by item text manifests the underlying concept. Unexpected outcomes signal misconceptions

Principal statistics

Factor loadings (covariances of item scores with factor). Factor scores (from regressing item scores on loadings).

A linear measure, error and fit statistics, for each item, examinee, and any other element modelled

Largest variance component

The factor with the most variance (largest eigenvalue)

The empirical manifestation of the underlying variable

Other variance components

As many factors as diagonal elements in correlation matrix

Modelled measurement error and unmodelled misfit

Other variance criterion

Factor eigenvalues greater than, usually, 1.4

Standardized misfit statistics greater than, usually, 2.0

T10

Measurement and sampling error

Factors with small eigenvalues confound these sources of error

Measurement error as standard errors, sampling error as standard deviations of examinee and item distributions

T11

Missing data

List-wise deletion loses data. Pair-wise deletion biases factor structure

Measures, standard errors and fit statistics based on all observed data

Diagnosis:

Factor Analysis

Rasch Analysis

Multi-modal distribution of examinees or items

A factor for each mode, some with large eigenvalue

Documented in item and person measure distributions. No effect on misfit

Irregular examinee response

Slight variance increase. Not determinable from factors

Large misfit for examinee, and somewhat increased misfit for items to which examinee responded unexpectedly

Identification of item bias (DIF, Differential Item Functioning) on many test items

By discovering factor scores correlated with group membership, and then items with loadings on those factors

Exploratory: By discovering items with significant differences between their group measures
Confirmatory: By partitioning residuals of suspect items between groups to estimate bias size, significance and homogeneity

Identification of solitary biased item

Undetectable; bias eigenvalues insignificant

As above, plus item misfit (particularly information-weighted (INFIT) statistics)

Major multi-dimensionality (items: 50% math, 50% reading)

After rotation, one math factor and one reading factor

Variable combines math and reading items with low person separation, reliability, and patterns of poor person fit
[PCA of Rasch residuals]

Minor multi-dimensionality (items: 95% math, 5% reading)

One math factor; insignificant reading factor

Variable defined by math items, significant misfit in reading items

Miskeyed multiple-choice item

Undetectable; eigenvalues insignificant.

Large item misfit statistic or item calibration contradicts construct

Factor Analysis versus Rasch Analysis of Items, B Wright … Rasch Measurement Transactions, 1991, 5:1 p. 134-135

Rasch-Related Resources: Rasch Measurement YouTube Channel
Rasch Measurement Transactions & Rasch Measurement research papers - free	An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse	Rasch Measurement Theory Analysis in R, Wind, Hua	Applying the Rasch Model in Social Sciences Using R, Lamprianou	El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Rasch Models for Measurement, David Andrich	Constructing Measures, Mark Wilson	Best Test Design - free, Wright & Stone Rating Scale Analysis - free, Wright & Masters
Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias	Diseño de Mejores Pruebas - free, Spanish Best Test Design	A Course in Rasch Measurement Theory, Andrich, Marais	Rasch Models in Health, Christensen, Kreiner, Mesba	Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen
Rasch Books and Publications: Winsteps and Facets
Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene	Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver	Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone	Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale	Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland
Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes	Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang	Statistical Analyses for Language Testers (Facets), Rita Green	Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind	Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M
Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind	Rasch Measurement: Applications, Khine	Winsteps Tutorials - free Facets Tutorials - free	Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre	Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

Rasch-Related Resources: Rasch Measurement YouTube Channel

Rasch Measurement Transactions & Rasch Measurement research papers - free

An Introduction to the Rasch Model with Examples in R (eRm, etc.), Debelak, Strobl, Zeigenfuse

Rasch Measurement Theory Analysis in R, Wind, Hua

Applying the Rasch Model in Social Sciences Using R, Lamprianou

El modelo métrico de Rasch: Fundamentación, implementación e interpretación de la medida en ciencias sociales (Spanish Edition), Manuel González-Montesinos M.

Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar

Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch

Rasch Models for Measurement, David Andrich

Constructing Measures, Mark Wilson

Best Test Design - free, Wright & Stone
Rating Scale Analysis - free, Wright & Masters

Virtual Standard Setting: Setting Cut Scores, Charalambos Kollias

Diseño de Mejores Pruebas - free, Spanish Best Test Design

A Course in Rasch Measurement Theory, Andrich, Marais

Rasch Models in Health, Christensen, Kreiner, Mesba

Multivariate and Mixture Distribution Rasch Models, von Davier, Carstensen

Rasch Books and Publications: Winsteps and Facets

Applying the Rasch Model (Winsteps, Facets) 4th Ed., Bond, Yan, Heene

Advances in Rasch Analyses in the Human Sciences (Winsteps, Facets) 1st Ed., Boone, Staver

Advances in Applications of Rasch Measurement in Science Education, X. Liu & W. J. Boone

Rasch Analysis in the Human Sciences (Winsteps) Boone, Staver, Yale

Appliquer le modèle de Rasch: Défis et pistes de solution (Winsteps) E. Dionne, S. Béland

Introduction to Many-Facet Rasch Measurement (Facets), Thomas Eckes

Rasch Models for Solving Measurement Problems (Facets), George Engelhard, Jr. & Jue Wang

Statistical Analyses for Language Testers (Facets), Rita Green

Invariant Measurement with Raters and Rating Scales: Rasch Models for Rater-Mediated Assessments (Facets), George Engelhard, Jr. & Stefanie Wind

Aplicação do Modelo de Rasch (Português), de Bond, Trevor G., Fox, Christine M

Exploring Rating Scale Functioning for Survey Research (R, Facets), Stefanie Wind

Rasch Measurement: Applications, Khine

Winsteps Tutorials - free
Facets Tutorials - free

Many-Facet Rasch Measurement (Facets) - free, J.M. Linacre

Fairness, Justice and Language Assessment (Winsteps, Facets), McNamara, Knoch, Fan

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

Coming Rasch-related Events
May 17 - June 21, 2024, Fri.-Fri.	On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 12 - 14, 2024, Wed.-Fri.	1st Scandinavian Applied Measurement Conference, Kristianstad University, Kristianstad, Sweden http://www.hkr.se/samc2024
June 21 - July 19, 2024, Fri.-Fri.	On-line workshop: Rasch Measurement - Further Topics (E. Smith, Winsteps), www.statistics.com
Aug. 5 - Aug. 6, 2024, Fri.-Fri.	2024 Inaugural Conference of the Society for the Study of Measurement (Berkeley, CA), Call for Proposals
Aug. 9 - Sept. 6, 2024, Fri.-Fri.	On-line workshop: Many-Facet Rasch Measurement (E. Smith, Facets), www.statistics.com
Oct. 4 - Nov. 8, 2024, Fri.-Fri.	On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
Jan. 17 - Feb. 21, 2025, Fri.-Fri.	On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
May 16 - June 20, 2025, Fri.-Fri.	On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com
June 20 - July 18, 2025, Fri.-Fri.	On-line workshop: Rasch Measurement - Further Topics (E. Smith, Facets), www.statistics.com
Oct. 3 - Nov. 7, 2025, Fri.-Fri.	On-line workshop: Rasch Measurement - Core Topics (E. Smith, Winsteps), www.statistics.com