I would endorse Art's suggestions with a few notes. The psychometric literature would suggest performing either an exploratory or confirmatory factor analysis to explore the dimensionality of your responses. Since you don't have an a priori notion of the structure, EFA would be appropriate. Methodologists involved with measuring human variables (attitudes, aptitude, personality traits) generally recommend most of the following choices for the "big decisions" in EFA.
Big decision #1: what method of estimation to use, for which either Principle Axis Factoring (PAF) or Maximum Likelihood (MLFA) should be preferred over PCA for exactly the reasons Art described. In addition, either of the first two are more likely to give you reproducible factor loading estimates, because they exclude random measurement error before estimation.
Big decision #2: How to determine the number of factors. This is a research interest of mine. My simulation studies have suggested that the best initial guess comes from parallel analysis if possible (again spot on by Art). If you have access to SAS and are interested, I can send you a macro to do this. I also think a scree plot can be very effective. Note that the traditional eigenvalues-over-1 rule has been repeatedly shown to not work well. If you or anyone else is interested, I can send a copy of an in-press manuscript.
Big descision #3: Factor rotation. You want to do this obtain a more interpretable solution. My one deviation from Art's recommendations is to start with an oblique rotation, which allows the factors to be correlated. After investigating the estimated factor correlations, you can then decide whether to go to the simpler orthogonal factor model given by a Varimax rotation. That way the data are helping you decide. Furthermore, human traits such as attitudes are difficult to conceive of as being completely independent of one another.
Here are a couple good references on using EFA for item analysis.
Gorsuch, R. L. (1997). Exploratory factor analysis: its role in item analysis. Journal of Personality Assessment, 68(3), 532-560.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological methods, 4(3), 272.
Conway & Huffcutt (2003). Organizational Research Methods, Vol. 6 No. 2, April 2003 147-168
Some final comments. It is not usual to first transform the data as EFA doesn't really involve probabilitistic inferences like hypothesis tests but is rather used descriptively. Analysis is usually performed on the correlation matrix, so you could say you're analyzing standardized data.
The correspondence analysis approach you refer to has not seen much attention in the psychometric literature. This would not be for lack of effort as this literature is very extensive, probably because there is so much art/finesse/subjectivity in its use. I think that approach would most closely be associated with Q factor analysis or Q-mode FA, in which the cluster analysis technique is applied to variables rather than observations. Q factor analysis has not seen a lot of attention in the last 20-30 years or so, a period in which probably many hundreds,if not thousands, of articles have been published on methods for this topic.
-------------------------------------------
Robert Pearson
Assistant Professor
University of Northern Colorado
-------------------------------------------
Original Message:
Sent: 03-12-2013 08:22
From: Eugene Gallagher
Subject: PCA of transformed Likert-Scale data
Thanks so much to the group. I had a long discussion with my student yesterday and showed her the various options available to her. I went through my collection of books and articles on multivariate analyses and did find several articles on correspondence analysis and multiple correspondence analysis of Likert scale questionnaire data. The following book was especially interesting:
Greenacre, M. and J. Blasius, eds., 2006. Multiple correspondence analysis and related methods. Chapman & Hall/CRC. 581 pp.
Articles in that book by Blasius and Greenacre (Chapter 1), Greenacre (Ch 2), Gower (Ch 3), and Nishisato (Ch 6) all deal with CA and CA-related methods for analyzing Likert-scale items. An appendix by Nenandic and Greenacre provides R code for CA and multiple correspondence analysis with Likert Scale data. I haven't tried their examples but intend to shortly. I have my own Matlab code for CA and I have used the ecological CA programs in the Vegan package.
Most of these papers in the Blasius & Greenacre symposium volume are exploratory in nature, examining how the Likert-scale questions relate to each other and to variables such as nationality of the respondent. A few papers, like Nishisato's compare the CA approach with PCA. All of the papers involving MCA recode the data from standard subject x question form to an indicator matrix form (5 7-pt Likert item questions would be coded for each subject with a 0,1 row vector with 35 columns with a row sum of 5) or the Burt matrix form (all questions cross classified by response with 5 7-pt Likert scale questions resulting in a 35 x 35 Burt matrix). The goal of the papers is not so much hypothesis testing but the exploration of the structure of a complex questionnaire and set of responses. A PCA should exhibit much of the same pattern among questions and respondents, but the MCA might allow nonlinear structures to be revealed more readily (according to Nishisato's paper his dual scaling reveals migraines are associated with both high and low blood pressure).
There were just too few classes surveyed with my student's questionnaires to do much with the data at the class level, even though the class is the experimental unit and the students are the measurement units. She'll take a look at what is there. There were other variables in the questionnaire that might be amenable for analysis, such as examining years of experience in chemistry with Likert-scale responses to questions assessing perception of chemical risk.
I'm not familiar with the literature on how to deal with the cluster effect of classes. I'll read up on that. I suspect with so few classes (5), there is not much to be done in fitting a mixed model with SAS Proc mixed or similar methods in R to assess the class effect.
I finally got a look at the questionnaire data and saw that a big part of my student's project is dealing with the coding of the responses and getting the responses in a form that can be analyzed with any method. There are many non-responses that will have to be assessed prior to doing any sort of analysis. Ecological data which result in nice sample by species matrices are much easier I think.
Generating PCA or Factor Scores, CA scores or MCA scores and analyzing the association with variables such as years of experience in chemistry might be very informative. That will depend on there being a simple structure that is interpretable once the responses are graphically displayed.
Thanks much for everyone's helpful comments.
-------------------------------------------
Eugene Gallagher
Associate Professor
Dept. of Environmental, Earth & Ocean Sciences
Univ of Massachusetts Boston
-------------------------------------------
Original Message:
Sent: 03-11-2013 14:01
From: Arthur Kendall
Subject: PCA of transformed Likert-Scale data
There are no transformation done with Likert items, a priori, but the correlations implicitly standardize the items across cases. Ipsatizing the cases seems like overkill for students. Ipsatizing might be of interest in very advanced psychometric studies.
Conventionally, Likert items are treated as not severely discrepant from interval. Their sum is treated as not severely discrepant from interval level.
There are two major approaches to factor analysis. Both major approaches use the iter-item Pearson correlation matrix. The difference is largely in what is used on the principal diagonal. The kind used in attitude scale construction, principal axes (PAF), is interested in the common variance among the items and treats the variances that are unique to the items as noise so it uses estimated of relaibility (usually the squared multiple correlation of each item with the other items). The other approach, principal components (PCA) assumes that all of item variance is of interest. [PCA is more frequently used in contexts where the variables are measured at the ratio level, e.g., wit physical constructs.
Each item is considered a rough measurement of a construct. The sum of the items is considered a more valid and reliable measure of the construct. There is no substantive meaning to zero in attitude measures.
There are refinements and variation in approach that differ according to specifics of the situation.
If the attitude items are parts of pre-existing validated Likert scales, designed to measure specific constructs, a different approach would be used.
Assuming that the are two sets of items that are designed to explore what structure there might be within each set, and further to see whether the two groups differ in where they are located on the derived dimensions, I would stick with a very conventional attitude scale analysis.
Use principal axis factor analysis with varimax rotation to maximize differential validity.
Determine the number of factors to retain in each set . This is an area that has an art aspect. For advanced students parallel analysis would be part of the decision making. For a solution that retains a number of factors. Items that do not load cleanly are not used. Items that do not load at all are not used. Typically scales to represent a factor would be created only when there were at least 3 and preferably 4 or 5 items that go together and make sense as measures of some construct.
If the set of items is well designed some items will need to be "reflected".
Scale score are created as means of sets of items that go together. An item is only used in one scale. Each item has the same weight - one. Using weighted sums of items frequently failed when scales were used across subpops and studies.
If the instrument administration was not done carefully, there may be missing data. If a factor analysis done with listwise deletion of missing cases show that the items that did not load or did not load cleanly there may be fewer cases lost when the factor analysis is re-run without those items in the variable list.
Likert items are used in creating Likert scales because people are fairly consistent in using the response scale.
Some form of GLM with the scale scores as DVs can be used to look at the difference between the groups. The simplest would be t-tests.
More complex models of the error should also be tried. The write-up should look at whether the more complex models make a substantive difference in the conclusions.
HTH
-------------------------------------------
Arthur Kendall
Social Research Consultants
-------------------------------------------
Original Message:
Sent: 03-09-2013 09:01
From: Eugene Gallagher
Subject: PCA of transformed Likert-Scale data
I teach a graduate statistics class, and students are required to do data analysis projects. I have one student who has 7-point Likert scale questionnaires (Disagree Strongly 1 ... Agree Strongly 7) about chemistry students' attitudes about different aspects of chemical safety and the personal assessment of risk of different chemicals. Her group has given the questionaire to many classes, some trained in principles of green chemistry and others with the standard curriculum. She wants to test for differences between students that have been trained in green chemistry techniques and others that have gone through the standard chemistry curriculum. She also wants to analyze the questionnaires to summarize the responses, and to note how student perception of chemicals and chemical risk factors are affected by training in green chemistry.
I'm an ecologist who uses PCA a lot. I can suggest to her that she do a PCA on the data, examining the correlation biplot to see which questions have strongly correlated responses (positive and negative). She can analyze the Euclidean distance biplot to see how the classes differ and whether there is a distinct difference between green and non-green classes. An exploratory factor analysis could indicate whether there are discrete factors, underlying the student responses on the many questions. Differences between the two groups of students might be assessed after the factor analysis by assessing group differences on the factor scores. Or the group differences in Likert Scale responses can be modeled explicitly by using redundancy analysis in which the questions that differ between groups are clearly identified.
In ecology, we'd never analyze the untransformed data. My question is, "Are there standard transformations of Likert Scale variables that are carried out prior to performing multivariate analysis?" My student can do the analysis untransformed, but perhaps the student x question data should be row normalized (sum of squared scores for each student equal to 1) prior to analysis. Students may differ in terms of the variability assigned to questions (one students 1 to 7 may be another's 3 to 5). Similarly, prior to PCA, the student x question data should be centered by subtracting the mean or the 1st axis will simply reflect the mean scores and not the student-to-student variance. A simultaneous row and column standardization could lead to a correspondence analysis of the Likert-scale data.
There might be some definitive reference or set of references on the transformation of Likert-scale data for multivariate analysis. If so, could you post it for me and my student. A google scholar search didn't come up with much useful.
-------------------------------------------
Eugene Gallagher
Associate Professor
Univ of Massachusetts
-------------------------------------------