Transcript Factor Analysis - Zayed University

Factor Analysis
1
SPSS for Windows® Intermediate & Advanced Applied Statistics
Zayed University Office of Research SPSS for Windows® Workshop Series
Presented by
Dr. Maher Khelifa
Associate Professor
Department of Humanities and Social Sciences
College of Arts and Sciences
© Dr. Maher Khelifa
Understanding Factor Analysis
2
 This workshop discusses factor analysis as an
exploratory and confirmatory multivariate
technique.
© Dr. Maher Khelifa
Understanding Factor Analysis
3
 Factor analysis is commonly used in:
 Data reduction
 Scale development
 The evaluation of the psychometric quality of a measure, and
 The assessment of the dimensionality of a set of variables.
© Dr. Maher Khelifa
Understanding Factor Analysis
4
 Regardless of purpose, factor analysis is used in:
 the determination of a small number of factors based on a
particular number of inter-related quantitative variables.
 Unlike variables directly measured such as speed,
height, weight, etc., some variables such as egoism,
creativity, happiness, religiosity, comfort are not a
single measurable entity.
 They are constructs that are derived from the
measurement of other, directly observable variables .
© Dr. Maher Khelifa
Understanding Factor Analysis
5
 Constructs are usually defined as unobservable latent variables. E.g.:
 motivation/love/hate/care/altruism/anxiety/worry/stress/product
quality/physical aptitude/democracy /reliability/power.
 Example: the construct of teaching effectiveness. Several variables
are used to allow the measurement of such construct (usually several
scale items are used) because the construct may include several
dimensions.
 Factor analysis measures not directly observable constructs by
measuring several of its underlying dimensions.
 The identification of such underlying dimensions (factors) simplifies
the understanding and description of complex constructs.
© Dr. Maher Khelifa
Understanding Factor Analysis
6
 Generally, the number of factors is much smaller than the
number of measures.
 Therefore, the expectation is that a factor represents a set of
measures.
 From this angle, factor analysis is viewed as a data-
reduction technique as it reduces a large number of
overlapping variables to a smaller set of factors that reflect
construct(s) or different dimensions of contruct(s).
© Dr. Maher Khelifa
Understanding Factor Analysis
7
 The assumption of factor analysis is that underlying
dimensions (factors) can be used to explain complex
phenomena.
 Observed correlations between variables result from
their sharing of factors.
 Example: Correlations between a person’s test scores
might be linked to shared factors such as general
intelligence, critical thinking and reasoning skills,
reading comprehension etc.
© Dr. Maher Khelifa
Ingredients of a Good Factor Analysis Solution
8
 A major goal of factor analysis is to represent
relationships among sets of variables parsimoniously
yet keeping factors meaningful.
 A good factor solution is both simple and
interpretable.
 When factors can be interpreted, new insights are
possible.
© Dr. Maher Khelifa
Application of Factor Analysis
9
 This workshop will examine three common
applications of factor analysis:



Defining indicators of constructs
Defining dimensions for an existing measure
Selecting items or scales to be included in a measure.
© Dr. Maher Khelifa
Application of Factor Analysis
10
 Defining indicators of constructs:
 Ideally 4 or more measures should be chosen to represent each
construct of interest.
 The choice of measures should, as much as possible, be guided by
theory, previous research, and logic.
© Dr. Maher Khelifa
Application of Factor Analysis
11
 Defining dimensions for an existing measure:
 In this case the variables to be analyzed are chosen by the
initial researcher and not the person conducting the analysis.
 Factor analysis is performed on a predetermined set of
items/scales.
 Results of factor analysis may not always be satisfactory:
The items or scales may be poor indicators of the construct or
constructs.
 There may be too few items or scales to represent each underlying
dimension.

© Dr. Maher Khelifa
Application of Factor Analysis
12
 Selecting items or scales to be included in a measure.


Factor analysis may be conducted to determine what items or
scales should be included and excluded from a measure.
Results of the analysis should not be used alone in making
decisions of inclusions or exclusions. Decisions should be
taken in conjunction with the theory and what is known about
the construct(s) that the items or scales assess.
© Dr. Maher Khelifa
Steps in Factor Analysis
13
 Factor analysis usually proceeds in four steps:
 1st Step: the correlation matrix for all variables is computed
 2nd Step: Factor extraction
 3rd Step: Factor rotation
 4th Step: Make final decisions about the number of underlying
factors
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
14
 1st Step: the correlation matrix
 Generate a correlation matrix for all variables
 Identify variables not related to other variables
 If the correlation between variables are small, it is unlikely that
they share common factors (variables must be related to each
other for the factor model to be appropriate).
 Think of correlations in absolute value.
 Correlation coefficients greater than 0.3 in absolute value are
indicative of acceptable correlations.
 Examine visually the appropriateness of the factor model.
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
15


Bartlett Test of Sphericity:

used to test the hypothesis the correlation matrix is an identity matrix
(all diagonal terms are 1 and all off-diagonal terms are 0).

If the value of the test statistic for sphericity is large and the
associated significance level is small, it is unlikely that the
population correlation matrix is an identity.
If the hypothesis that the population correlation matrix is an identity
cannot be rejected because the observed significance level is large,
the use of the factor model should be reconsidered.
© Dr. Maher Khelifa
Steps in Factor Analysis:
The Correlation Matrix
16

The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy:
 is an index for comparing the magnitude of the observed correlation
coefficients to the magnitude of the partial correlation coefficients.

The closer the KMO measure to 1 indicate a sizeable sampling adequacy
(.8 and higher are great, .7 is acceptable, .6 is mediocre, less than .5 is
unaccaptable ).

Reasonably large values are needed for a good factor analysis. Small KMO
values indicate that a factor analysis of the variables may not be a good
idea.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
17
 2nd Step: Factor extraction
 The primary objective of this stage is to determine the factors.
 Initial decisions can be made here about the number of factors
underlying a set of measured variables.
 Estimates of initial factors are obtained using Principal components
analysis.
 The principal components analysis is the most commonly used
extraction method . Other factor extraction methods include:
 Maximum likelihood method
 Principal axis factoring
 Alpha method
 Unweighted lease squares method
 Generalized least square method
 Image factoring.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
18
 In principal components analysis, linear combinations of
the observed variables are formed.
 The 1st principal component is the combination that accounts
for the largest amount of variance in the sample (1st
extracted factor).
 The 2nd principle component accounts for the next largest
amount of variance and is uncorrelated with the first (2nd
extracted factor).
 Successive components explain progressively smaller portions
of the total sample variance, and all are uncorrelated with
each other.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
19
 To decide on how many factors we
need to represent the data, we use
2 statistical criteria:


Total Variance Explained
Eigen Values, and
The Scree Plot.
Extraction Sums of Squared
Initial Eigenvalues
Loadings
% of
Cumulativ
% of
Cumulativ
Variance
e%
Variance
e%
Comp
 The determination of the number
of factors is usually done by
considering only factors with
Eigen values greater than 1.
 Factors with a variance less than 1
are no better than a single
variable, since each variable is
expected to have a variance of 1.
onent
Total
Total
1
3.046
30.465
30.465
3.046
30.465
30.465
2
1.801
18.011
48.476
1.801
18.011
48.476
3
1.009
10.091
58.566
1.009
10.091
58.566
4
.934
9.336
67.902
5
.840
8.404
76.307
6
.711
7.107
83.414
7
.574
5.737
89.151
8
.440
4.396
93.547
9
.337
3.368
96.915
10
.308
3.085
100.000
Extraction Method: Principal Component Analysis.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
20

The examination of the Scree plot provides a
visual of the total variance associated with
each factor.

The steep slope shows the large factors.

The gradual trailing off (scree) shows the rest
of the factors usually lower than an Eigen
value of 1.

In choosing the number of factors, in addition
to the statistical criteria, one should make
initial decisions based on conceptual and
theoretical grounds.

At this stage, the decision about the number of
factors is not final.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Extraction
21
Component Matrix using Principle Component Analysis
Component Matrixa
Component
1
2
3
I discussed my frustrations and feelings with person(s) in school
.771
-.271
.121
I tried to develop a step-by-step plan of action to remedy the problems
.545
.530
.264
I expressed my emotions to my family and close friends
.580
-.311
.265
I read, attended workshops, or sought someother educational approach to correct the
.398
.356
-.374
I tried to be emotionally honest with my self about the problems
.436
.441
-.368
I sought advice from others on how I should solve the problems
.705
-.362
.117
I explored the emotions caused by the problems
.594
.184
-.537
I took direct action to try to correct the problems
.074
.640
.443
I told someone I could trust about how I felt about the problems
.752
-.351
.081
I put aside other activities so that I could work to solve the problems
.225
.576
.272
problem
Extraction Method: Principal Component Analysis.
a. 3 components extracted.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
22
 3rd Step: Factor rotation.
 In this step, factors are rotated.
 Un-rotated factors are typically not very interpretable
(most factors are correlated with may variables).
 Factors are rotated to make them more meaningful and
easier to interpret (each variable is associated with a
minimal number of factors).
 Different rotation methods may result in the
identification of somewhat different factors.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
23
 The most popular rotational method is Varimax rotations.
 Varimax use orthogonal rotations yielding uncorrelated
factors/components.
 Varimax attempts to minimize the number of variables that have high
loadings on a factor. This enhances the interpretability of the factors.
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
24
 Other common rotational method used include Oblique rotations
which yield correlated factors.
 Oblique rotations are less frequently used because their results are
more difficult to summarize.
 Other rotational methods include:



Quartimax (Orthogonal)
Equamax (Orthogonal)
Promax (oblique)
© Dr. Maher Khelifa
Steps in Factor Analysis:
Factor Rotation
25
 A factor is interpreted or named by examining the largest values linking the
factor to the measured variables in the rotated factor matrix.
Rotated Component Matrixa
Component
1
2
3
I discussed my frustrations and feelings with person(s) in school
.803
.186
.050
I tried to develop a step-by-step plan of action to remedy the problems
.270
.304
.694
I expressed my emotions to my family and close friends
.706
-.036
.059
I read, attended workshops, or sought someother educational approach to
.050
.633
.145
I tried to be emotionally honest with my self about the problems
.042
.685
.222
I sought advice from others on how I should solve the problems
.792
.117
-.038
I explored the emotions caused by the problems
.248
.782
-.037
I took direct action to try to correct the problems
-.120
-.023
.772
.815
.172
-.040
-.014
.155
.657
correct the problem
I told someone I could trust about how I felt about the problems
I put aside other activities so that I could work to solve the problems
Extraction Method: Principal Component Analysis.
Rotation Method: Varimax with Kaiser Normalization.
© Dr. Maher Khelifa
a. Rotation converged in 5 iterations.
Steps in Factor Analysis:
Making Final Decisions
26
 4th Step: Making final decisions




The final decision about the number of factors to choose is the number of
factors for the rotated solution that is most interpretable.
To identify factors, group variables that have large loadings for the same
factor.
Plots of loadings provide a visual for variable clusters.
Interpret factors according to the meaning of the variables
 This decision should be guided by:



A priori conceptual beliefs about the number of factors from past research or
theory
Eigen values computed in step 2.
The relative interpretability of rotated solutions computed in step 3.
© Dr. Maher Khelifa
Assumptions Underlying Factor Analysis
27
 Assumption underlying factor analysis include.
 The measured variables are linearly related to the factors + errors.




This assumption is likely to be violated if items limited response scales
(two-point response scale like True/False, Right/Wrong items).
The data should have a bi-variate normal distribution for each pair of
variables.
Observations are independent.
The factor analysis model assumes that variables are determined by
common factors and unique factors. All unique factors are assumed
to be uncorrelated with each other and with the common factors.
© Dr. Maher Khelifa
Obtaining a Factor Analysis
28

Click:

Analyze and
select



© Dr. Maher Khelifa
Dimension
Reduction
Factor
A factor
Analysis Box
will appear
Obtaining a Factor Analysis
29

© Dr. Maher Khelifa
Move
variables/scale
items to
Variable box
Obtaining a Factor Analysis
30
 Factor
extraction
 When
variables
are in
variable
box,
select:

© Dr. Maher Khelifa
Extractio
n
Obtaining a Factor Analysis
31
 When the factor
extraction Box
appears, select:
 Scree Plot
 keep all default
selections
including:



© Dr. Maher Khelifa
Principle component
Analysis
Based on Eigen Value
of 1, and
Un-rotated factor
solution
Obtaining a Factor Analysis
32
 During
factor
extraction
keep
factor
rotation
default of:


© Dr. Maher Khelifa
None
Press
continue
Obtaining a Factor Analysis
33






© Dr. Maher Khelifa
During Factor
Rotation:
Decide on the
number of factors
based on actor
extraction phase
and enter the
desired number of
factors by choosing:
Fixed number of
factors and
entering the
desired number of
factors to extract.
Under Rotation
Choose Varimax
Press continue
Then OK
Bibliographical References
34













Almar, E.C. (2000). Statistical Tricks and traps. Los Angeles, CA: Pyrczak Publishing.
Bluman, A.G. (2008). Elemtary Statistics (6th Ed.). New York, NY: McGraw Hill.
Chatterjee, S., Hadi, A., & Price, B. (2000) Regression analysis by example. New York: Wiley.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral
sciences (2nd Ed.). Hillsdale, NJ.: Lawrence Erlbaum.
Darlington, R.B. (1990). Regression and linear models. New York: McGraw-Hill.
Einspruch, E.L. (2005). An introductory Guide to SPSS for Windows (2nd Ed.). Thousand Oak, CA:
Sage Publications.
Fox, J. (1997) Applied regression analysis, linear models, and related methods. Thousand Oaks, CA:
Sage Publications.
Glassnapp, D. R. (1984). Change scores and regression suppressor conditions. Educational and
Psychological Measurement (44), 851-867.
Glassnapp. D. R., & Poggio, J. (1985). Essentials of Statistical Analysis for the Behavioral Sciences.
Columbus, OH: Charles E. Merril Publishing.
Grimm, L.G., & Yarnold, P.R. (2000). Reading and understanding Multivariate statistics. Washington
DC: American Psychological Association.
Hamilton, L.C. (1992) Regression with graphics. Belmont, CA: Wadsworth.
Hochberg, Y., & Tamhane, A.C. (1987). Multiple Comparisons Procedures. New York: John Wiley.
Jaeger, R. M. Statistics: A spectator sport (2nd Ed.). Newbury Park, London: Sage Publications.
© Dr. Maher Khelifa
Bibliographical References
35










Keppel, G. (1991). Design and Analysis: A researcher’s handbook (3rd Ed.). Englwood Cliffs, NJ:
Prentice Hall.
Maracuilo, L.A., & Serlin, R.C. (1988). Statistical methods for the social and behavioral sciences. New
York: Freeman and Company.
Maxwell, S.E., & Delaney, H.D. (2000). Designing experiments and analyzing data: Amodel
comparison perspective. Mahwah, NJ. : Lawrence Erlbaum.
Norusis, J. M. (1993). SPSS for Windows Base System User’s Guide. Release 6.0. Chicago, IL: SPSS
Inc.
Norusis, J. M. (1993). SPSS for Windows Advanced Statistics. Release 6.0. Chicago, IL: SPSS Inc.
Norusis, J. M. (2006). SPSS Statistics 15.0 Guide to Data Analysis. Upper Saddle River, NJ.: Prentice
Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Guide to Data Analysis. Upper Saddle River, NJ.: Prentice
Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Statistical Procedures Companion. Upper Saddle River,
NJ.: Prentice Hall.
Norusis, J. M. (2008). SPSS Statistics 17.0 Advanced Statistical Procedures Companion. Upper
Saddle River, NJ.: Prentice Hall.
Pedhazur, E.J. (1997). Multiple regression in behavioral research, third edition. New York: Harcourt
Brace College Publishers.
© Dr. Maher Khelifa
Bibliographical References
36


















SPSS Base 7.0 Application Guide (1996). Chicago, IL: SPSS Inc.
SPSS Base 7.5 For Windows User’s Guide (1996). Chicago, IL: SPSS Inc.
SPSS Base 8.0 Application Guide (1998). Chicago, IL: SPSS Inc.
SPSS Base 8.0 Syntax Reference Guide (1998). Chicago, IL: SPSS Inc.
SPSS Base 9.0 User’s Guide (1999). Chicago, IL: SPSS Inc.
SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc.
SPSS Base 10.0 Application Guide (1999). Chicago, IL: SPSS Inc.
SPSS Interactive graphics (1999). Chicago, IL: SPSS Inc.
SPSS Regression Models 11.0 (2001). Chicago, IL: SPSS Inc.
SPSS Advanced Models 11.5 (2002) Chicago, IL: SPSS Inc.
SPSS Base 11.5 User’s Guide (2002). Chicago, IL: SPSS Inc.
SPSS Base 12.0 User’s Guide (2003). Chicago, IL: SPSS Inc.
SPSS 13.0 Base User’s Guide (2004). Chicago, IL: SPSS Inc.
SPSS Base 14.0 User’s Guide (2005). Chicago, IL: SPSS Inc..
SPSS Base 15.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
SPSS Base 16.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
SPSS Statistics Base 17.0 User’s Guide (2007). Chicago, IL: SPSS Inc.
Tabachnik, B.G., & Fidell, L.S. (2001). Using multivariate statistics (4th Ed). Boston, MA: Allyn and
Bacon.
© Dr. Maher Khelifa