Rasch Benchmarking

When a global 500 company was asked to assess whether they were a "Great Place to Work," they went through a lengthy vendor selection process to find a research organization that could conduct a benchmark study. The winning bid would take six months to complete at a cost of just north of $100,000. Then the economy went south and the project was canceled, but the need for the project persisted so we conducted a stealth benchmark study. Here's what we did.

A top-down approach: We put Business Week, Vault, and Consulting Magazine's ranking of company prestige into a matrix, converted the rankings into a five-point scale, ran them through Rasched them, and converted them to percentiles.

A bottom-up approach: We went to Vault.com, where employees give the inside scoop on the companies they work for and coded who is good to work for and why. For example, "[Company X's] promotion policy is among the best, if not the best in the industry. This is a major draw to X for prospective hires, and is truly one of the best aspects of the firm." This received a 1 under "Career Path," while "The company is certainly the opposite of a meritocracy, with advancement more dependent on who you know rather than what you know" received a 0. We ended up with half a dozen categories that were consistent across all companies: Compensation, Career Path, Culture, Diversity, Work-life Balance, Necessity of Face Time, and Training. As before, we put the data in a matrix, Rasched, and converted into percentiles.

A both approach: We then graphed the results (see Figure below) and were surprised to find A) that we stank and B) that a factor that distinguished whether a company was a great place to work was whether they offered tuition reimbursement. The latter (at least) was surprising, surprising because economists consistently argue that companies should not offer tuition reimbursement, because it increases employees' portable skills. "Thanks for the degree. Bye." The great companies, however, are not thinking economics. They are thinking game theory. You win the war for talent by outflanking your opponent. "Sure, you can steal my highly-educated people, but I can steal yours and we can all take from those who don't educate their people."

Three interesting things came from this study. First, it became part of a business case for us to create our own legal and accredited graduate program, which puts our competitors in a pickle, because it is too expensive to offer tuition reimbursement and their own degree and once you've offered reimbursement, employees will howl if you take it away. Second, it raises a philosophical issue; all of the data is subjective-prestige ratings and employee comments-but since anybody can replicate the study to get the same result, the study is objective. Third, we completed the study with no non-payroll cost in a week.

Tad Waddington
www.lastingcontribution.com

---

Waddington T. (2009) Rasch Benchmarking, Rasch Measurement Transactions, 2009, 23:1, 1200

Rasch Publications
Rasch Measurement Transactions (free, online)	Rasch Measurement research papers (free, online)	Probabilistic Models for Some Intelligence and Attainment Tests, Georg Rasch	Applying the Rasch Model 3rd. Ed., Bond & Fox	Best Test Design, Wright & Stone
Rating Scale Analysis, Wright & Masters	Introduction to Rasch Measurement, E. Smith & R. Smith	Introduction to Many-Facet Rasch Measurement, Thomas Eckes	Invariant Measurement: Using Rasch Models in the Social, Behavioral, and Health Sciences, George Engelhard, Jr.	Statistical Analyses for Language Testers, Rita Green
Rasch Models: Foundations, Recent Developments, and Applications, Fischer & Molenaar	Journal of Applied Measurement	Rasch models for measurement, David Andrich	Constructing Measures, Mark Wilson	Rasch Analysis in the Human Sciences, Boone, Stave, Yale
in Spanish:	Análisis de Rasch para todos, Agustín Tristán		Mediciones, Posicionamientos y Diagnósticos Competitivos, Juan Ramón Oreja Rodríguez

Forum

Rasch Measurement Forum to discuss any Rasch-related topic

Go to Top of Page
Go to index of all Rasch Measurement Transactions
AERA members: Join the Rasch Measurement SIG and receive the printed version of RMT
Some back issues of RMT are available as bound volumes
Subscribe to Journal of Applied Measurement

Go to Institute for Objective Measurement Home Page. The Rasch Measurement SIG (AERA) thanks the Institute for Objective Measurement for inviting the publication of Rasch Measurement Transactions on the Institute's website, www.rasch.org.

The URL of this page is www.rasch.org/rmt/rmt231k.htm

Website: www.rasch.org/rmt/contents.htm