Overview A canonical correlation is the correlation of two canonical (latent) variables, one representing a set of independent variables, the other a set of dependent variables. Each set may be considered a latent variable based on measured indicator variables in its set. The canonical correlation is optimized such that the linear correlation between the two latent variables is maximized. Whereas multiple regression is used for many-to-one relationships, canonical correlation is used for many-to-many relationships. There may be more than one such linear correlation relating the two sets of variables, with each such correlation representing a different dimension by which the independent set of variables is related to the dependent set. The purpose of canonical correlation is to explain the relation of the two sets of variables, not to model the individual variables. For each canonical variate we can also assess how strongly it is related to measured variables in its own set, or the set for the other canonical variate. Wilks's lambda is commonly used to test the significance of canonical correlation. Analogous with ordinary correlation, canonical correlation squared is the percent of variance in the dependent set explained by the independent set of variables along a given dimension (there may be more than one). In addition to asking how strong the relationship is between two latent variables, canonical correlation is useful in determining how many dimensions are needed to account for that relationship. Canonical correlation finds the linear combination of variables that produces the largest correlation with the second set of variables. This linear combination, or "root," is extracted and the process is repeated for the residual data, with the constraint that the second linear combination of variables must not correlate with the first one. The process is repeated until a successive linear combination is no longer significant. Canonical correlation is a member of the multiple general linear hypothesis (MLGH) family and shares many of the assumptions of multiple regression such as linearity of relationships, homoscedasticity (same level of relationship for the full range of the data), interval or near-interval data, untruncated variables, proper specification of the model, lack of high multicollinearity, and multivariate normality for purposes of hypothesis testing. It also shares with factor analysis the need to impute labels for the canonical variables based on structure correlations, which function as a form of canonical factor loading; researchers may well impute different labels based on the same data. See also the nonlinear canonical correlation procedure (OVERALS). See also partial least squares regression, which is sometimes used to predict one set of response variables from a set of independent variables.

Contents Key concepts and terms Assumptions SPSS output Frequently asked questions Bibliography




Key Concepts and Terms

• Canonical variable or variate: A canonical variable, also called a variate, is a linear combination of a set of original variables in which the within-set correlation has been controlled (that is, the variance of each variable accounted for by other variables in the set has been removed). It is a form of latent variable. There are two canonical variables per canonical correlation (function). One is the dependent canonical variable, while the one for the independents may be called the covariate canonical variable.

• Canonical correlation, also called a characteristic root, is a form of correlation relating two sets of variables. As with factor analysis, there may be more than one significant dimension (more than one canonical correlation), each representing an orthogonally separate pattern of relationships between the two latent variables. The maximum number of canonical correlations between two sets of variables is the number of variables in the smaller set.

  The first canonical correlation is always the one which explains most of the relationship. The canonical correlations are interpreted the same as Pearson's r: their square is the percent of variance in the canonical variate of one set of variables explained by the canonical variate for the other set along the dimension represented by the given canonical correlation (usually the first). Another way to put it is to say that Rc-squared is the percent of variance shared by the canonical variates along this dimension. As an arbitrary rule of thumb, some researchers state that a dimension will be of interest if its canonical correlation is .30 or higher, corresponding to about 10% of variance explained. Some researchers, when reporting canonical correlation, report just the first canonical correlation, but it is recommended that all meaningful and interpretable canonical correlations be reported.

  • Pooled R_c² (pooled canonical correlation) is the sum of the squares of all the canonical correlation coefficients, representing all the orthogonal dimensions in the solution by which the two sets of variables are related. Pooled R_c² is used to assess the extent to which one set of variables can be predicted or explained by the other set.

• Eigenvalues. The eigenvalues as computed by SPSS are approximately equal to the canonical correlations squared. They reflect the proportion of variance in the canonical variate explained by the canonical correlation relating two sets of variables. In practice, this is only useful when there is more than one extracted canonical correlation and more than one eigenvalue, one for each canonical correlation. There will be as many eigenvalues as there are canonical correlations (roots), and each successive eigenvalue will be smaller than the last since each successive root will explain less and less of the data. As in factor analysis, roots with low eigenvalues may be discounted by the researcher, though this is less of an issue in canonical correlation, where usually only the first canonical correlation is of interest. The ratio of the eigenvalues is the ratio of explanatory importance of the canonical correlations (labeled "roots" in SPSS) which are extracted for the given data. Note that while the eigenvalues tell the researcher the proportion of variance explained in the canonical variates, they do not differentiate how much variance is explained in each of the two sets of variables themselves (hence the importance of redundancy coefficients, discussed below).

• Canonical weight, also called the canonical function coefficient or the canonical coefficient: The standardized canonical weights are used to assess the relative importance of individual variables' contributions to a given canonical correlation. (The default reporting of canonical weights is in standardized form, so the term "standardized" is usually assumed when "canonical weights" are discussed). The canonical coefficients are the standardized weights in the linear equation of variables which creates the canonical variables. As such they are analogous to beta weights in regression analysis. The ratio of canonical weights is the ratio of the contribution of the variable to the given canonical correlation, controlling for other variables in the equation. There will be one canonical coefficient for each original variable in each of the two sets of variables, for each canonical correlation. Thus for the dependent set, if there are five variables and there are three canonical correlations (functions), there will be 15 canonical coefficients in three sets of five coefficients. See below for further discussion of the use of canonical coefficients.

• Canonical scores are the values on a canonical variable for a given case, based on the canonical coefficients for that variable. Canonical coefficients are multiplied by the standardized scores of the cases and summed to yield the canonical scores for each case in the analysis.

• Structure correlation coefficients, also called canonical factor loadings: A structure correlation is the correlation of a canonical variable with an original variable in its set. That is, it is the correlation of canonical variable scores for a given canonical variable with the standardized scores of an original input variable. The table of structure correlations is sometimes called the factor structure. The squared structure correlation indicates the contribution made by a given variable to the explanatory power of the canonical variate based on the set of variables to which it belongs. Alpert and Peterson (1972: 192) note, "[canonical] wieghts appear more suitable for prediction, while correlations [structure coefficients] may better explain underlying (although interrelated) constructs."

  Structure correlations are used for three purposes.

  1. Interpreting the Canonical Variables: The magnitudes of the structure correlations help in interpreting the meaning of the canonical variables with which they are associated, much like how factor loadings help interpret the meaning of factors extracted in factor analysis. Larger canonical factor loadings should be weighted more when assigning an interpretive label to the given canonical correlation. A rule of thumb is for variables with correlations of 0.3 or above to be interpreted as being part of the canonical variable, and those below not to be considered part of the canonical variable.

  2. Calculating Variance Explained in a Given Original Variable: The square of the structure correlation is the percent of the variance in a given original variable accounted for by a given canonical variable on a given (usually the first) canonical correlation. Another way of putting it is that the structure correlation squared is the percent of variance linearly shared by an original variable with one of the canonical variates.

  3. Calculating Variance Explained in a Set of Original Variable: The sum of the squared structure correlations divided by the number or original variables in a set is the average percent of variance which the covariate accounts for in the raw variables in its set.

• The canonical communality coefficient is the sum of the squared structure coefficients for a given variable. The canonical communality coefficient measures how much of a given original variable's variance is reproducible from the canonical variables. As in factor analysis, original variables with low canonical communality are those for which the model is "not working" and the researcher may use this information to drop such variables from the analysis.

• The canonical variate adequacy coefficient is the average of all the squared structure coefficients for one set of variables (the dependent or the independent set) with respect to a given canonical variable. It is a measure of how well a given canonical variable represents the original variance in that set of original variables. It is also the percent of varince in the set of variables (ex., the independent set) extracted by the canonical variate for that set.

• The redundancy coefficient, d, also called R_d, measures the percent of the variance of the original variables of one set may be predicted from a (usually the first) canonical variable from the other set. High redundancy means high ability to predict. In particular, the researcher is apt to be interested in how well the independent canonical variate predicts values of the original dependent variables. The redundancy coefficient, which is interpreted as the percent of total variance of the original variables in the set explained by the canonical variate of the other set, will almost always be different for the two sets of variables since the number of variables and their variance in each set will differ. That is, there will be two redundancy coeffcieints for each canonical correlation. The researcher will be interested primaily in the redundancy of the independent canonical variate in predicting the variance in the set of variables in the dependent set. However, there will also be a redundancy coefficient for the dependent variate predicting the variance in the independent variables. (Often, the latter redundancy coefficient is not reported).
  • Computation. Redundancy is the canonical correlation squared (R_c²) times the mean squared structure coefficient (which is the adequacy coefficient). That is, one takes the structure coefficients (the canonical factor loadings), squares them, sums the squares, and divides by the number of variables in the set to get the mean squared structure coefficient. This coefficient is multiplied by R_c² to get the redundancy coefficient for the set.

  • Interpretation. The canonical correlation is the correlation of the independent and dependent canonical variates. It is possible for the canonical variates to correlate highly, yet each variate may not extract significant proportions of variance from thier respective batteries of original variables. Alpert and Peterson (1972: 189) note therefore that the researcher may "use canonical correlation coefficients to test for the existence of ovarall relationships between sets of variables, but for a measure of the magnitude of relationships, redundancy may be more appropriate."

    Thus canonical correlation reflects the percent of variance in the dependent canonical variable explained by the independent canonical variable and is used when exploring relationships between the independent and the dependent set of variables. In contrast, redundancy has to do with the percent of variance in the set of original individual dependent variables explained by the independent canonical variable and is used when assessing the effectiveness of the canonical analysis in capturing the variance of the original variables. These are different, and one may think of redundancy analysis as a check on the meaning of the canonical correlation. Redundancy analysis is used to measure whether the dependent canonical variable predicts the values of the original independent variables, how well the independent canonical variable predicts the values of the original independent variables, and how well the dependent canonical variable predicts the values of the original dependent variables. See the redundancy analysis question in the FAQ section below.

  • The pooled redundancy coefficient is the sum of all the redundancy coefficients for all the variables in a set (the independent set or the dependent set). Where a simple redundancy coefficient assesses the effectiveness of the a given canonical variate in capturing the variance of the original variables, the pooled redunancy coefficient assesses the effectiveness of all canonical variates in the solution in capturing the variance of the original variables.

  
  

• A canonical correlation plot
  

  The canonical correlation plot shows one (usually the first) canonical correlation. The X axis is the covariate canonical variable (the latent variable for the set of independents). The Y axis is the canonical variable representing the dependent variables. The tick marks on the axes are in standardized units. Note the origin is about -4, -4. The 0,0 location is in the center of the plot. The points are the canonical scores of each case based on the case's scores on each of the two canonical variables. A regression line shows the scatter of points. The canonical correlation (.95 in the example) is printed on the regression line. When canonical correlation is high, the points will form two clusters at different points on the regression line.

  Outliers. Cases outside the two clusters in a canonical correlation plot are outliers and may merit scrutiny or separate modeling. In the example above, outliers are circled. That is, canonical correlation plots are a useful method of identifying outliers or exceptional cases which differ from other cases in not sharing the same pattern of correlation among the two sets of variables in the study.

• A helio plot
  

  The helio plot is of a single canonical correlation (usually the first one). The plot is formed of two semi-circles, with original variables arrayed around the perimeter. The left semicircle lists the dependent variables and the right semicircle lists the independents. Bars proportionate in length to the structure correlations of each variable are arrayed around an inner circle. Bars reaching outward represent positive correlations and bars reaching inward represent negative correlations. Helio plots are not output by SPSS or SAS through 2007 versions.

• Significance Tests
  • Wilks's lambda: Wilks's lambda is used in conjunction with Bartlett's V, much as in MANOVA, to test the significance of the first canonical correlation. If p < .05, the two sets of variables are significantly associated by canonical correlation. Degrees of freedom equal P*Q, where P = # variables in variable set 1 and Q = # variables in variable set 2. This test establishes the significance of the first canonical correlation but not necessarily the second (Bartlett's V may be partitioned to test the second separately, but this is not directly supported in SPSS). This is also called the greatest characteristic root test.

  • Likelihood ratio test: This is a significance test of all sources (not just the first canonical correlation) of linear relationship between the two canonical variables. It is sometimes wrongly used as a test of the significance of the first or another single canonical correlation in a set of such functions.




Assumptions

• Interval level data are assumed.

• Linearity of relationships is assumed, though there is a nonlinear canonical correlation procedure available -- OVERALS. This is found in the SPSS Categories module. Gifi (1990) discusses non-linear canonical correlation analysis in his Chapter 6. In the usual form of canonical correlation, however, analysis is performed on the correlation or variance-covariance matrices, which reflect linear relationships. Of course, one can insert exponentiated or otherwise nonlinearly transformed variables into either measured variable set in canonical correlation.

• Low multicollinearity: To the extent that the variables within the independent sets of variables are highly intercorrelated, the canonical coefficients will be unstable. The coefficients for some variables may be misleadingly low or even negative because variance has already been explained by other variables.

• Homoscedasticity and other assumptions of correlation are assumed. The covariates are created based on the correlation matrix, with regression-like assumptions that the degree of correlation is constant along the full range of the variables being correlated.

• Minimal measurement error is assumed since low reliability attenuates the correlation coefficient. Canonical correlation also can be quite sensitive to missing data.

• Unrestricted variance If variance is truncated or restricted due, for instance, to poor sampling, this can also lead to attenuation of the correlation coefficient.

• Similar underlying distributions are assumed: if two variables come from unlike distributions, their correlation may be well below +1 even when data pairs are matched as perfectly as they can be while still conforming to the underlying distributions. That is, the larger the difference in the shape of the distribution of the two variables, the more the attenuation of the correlation coefficient. This assumption may well be violated when correlating an interval variable with a dichotomy or even an ordinal variable.

• Multivariate normality is required for significance testing in canonical correlation. This assumption is violated when dichotomous, dummy, and other discrete variables are used. In such situations, where significance testing is appropriate, researchers may use a resampling method. The central limit theorem demonstrates, however, that for large samples, indices used in significance testing will be normally distributed even when the variables themselves are not normally distributed, and therefore significance testing may be employed.

• Non-singularity in the correlation matrix of original variables. This is the problem of perfect multicollinearity : a unique solution cannot be computed if some variables are redundant, thereby approaching perfect correlation with others in the model. A correlation matrix with redundancy is said to be singular or ill-conditioned. Datasets based on survey data, in which there are a large number of questions, are more likely to have redundant items.

• Adequate sample size must exist to reduce the chances of Type II error (thinking you don't have something when you do). Stevens (1986) recommends at least 20 times as many cases as variables in the analysis in order to interpret the first canonical correlation only. For two canonical correlations, Barcikowski and Stevens (1975) recommend 40 to 60 times as many cases as variables.

• No or few outliers. Outliers can substantially affect canonical correlation coefficients, particularly if sample size is not very large.




SPSS Output Example

• Canonical Correlation in SPSS MANOVA Output




Frequently Asked Questions

• How is canonical correlation obtained from SPSS?
  • Canonical correlation is part of MANOVA in SPSS, in which one has to refer to one set of variables as "dependent" and the other as "covariates." It is available only in syntax. The command syntax method is as follows, where set1 and set2 are variable lists:
    MANOVA set1 WITH SET2 /DISCRIM ALL ALPHA(1) /PRINT SIGNIF(MULTIV UNIV EIGEN DIMENR).

    Note one cannot save canonical scores in this method.

  • Canonical correlation has to be run in syntax, not from the SPSS menus. If you just want to create a dataset with canonical variables, as part of the Advanced Statistics module SPSS supplies the CANCORR macro located in the file canonic correlation.sps, usually in the same directory as the SPSS main program. Open the syntax window with File, New, Syntax. Enter this:

    INCLUDE 'c:\Program Files\SPSS\Canonical correlation.sps'.
    CANCORR SET1=varlist/
    SET2=varlist/.
    

    where "varlist" is one of two lists of numeric variables. Output will be saved to a file called "cc_tmp2.sav," which will contain the canonical scores as new variables along with the original data file. These scores will be labeled s1_cv1 and s2_cv1, s2_cv1 and s2_cv2, and the like, standing for the scores on the two canonical variables associated with each canonical correlation. The macro will create two canonical variables for a number of canonical correlations equal to the smaller number of variables in SET1 or SET2.

  • OVERALS, which is part of the SPSS Categories module, computes nonlinear canonical correlation analysis on two or more sets of variables.

• How is it that some of my variables have high structure correlations even though their canonical weights are near zero?
  Because the weights are partial coefficients whereas the structure correlations (canonical factor loadings) are not: if a given variable shares variance with other independent variables entered in the linear combination of variables used to create a canonical variable, its canonical coefficient (weight) is computed based on the residual variance it can explain after controlling for these variables. If an independent variable is totally redundant with another independent variable, its partial coefficient (canonical weight) will be zero. Nonetheless, such a variable might have a high correlation with the canonical variable (that is, a high structure coefficient). In summary, the canonical weights have to do with the unique contributions of an original variable to the canonical variable, whereas the structure correlations have to do with the simple, overall correlation of the original variable with the canonical variable.

• Is the canonical correlation a measure of the percent of variance explained in the original variables?
  No. The square of the structure correlation is the percent of the variance in a given original variable accounted for by a given canonical variable on a given (usually the first) canonical correlation. Note that the average percent of variance explained in the original variables by a canonical variable (the mean of the squared structure correlations for the canonical variable) is not at all the same as the canonical correlation, which has to do with the correlation between the weighted sums of the two sets of variables. Put another way, the canonical correlation does not tell us how much of the variance in the original variables is explained by the canonical variables. Instead, that is determined on the basis of the squares of the structure correlations.

• Can the canonical coefficients be used to explain with which original variables a canonical correlation is predominantly associated?
  The canonical coefficients are standardized coefficients and (like beta weights in regression) their magnitudes can be compared. Looking at the columns in SPSS output which list the canonical coefficients as columns and the variables in a set of variables as rows, some researchers simply note variables with the highest coefficients to determine which variables are associated with which canonical correlations and use this as the basis for inducing the meaning of the dimension represented by the canonical correlation.

  However, Levine (1977: 18-19) argues against the procedure above on the ground that the canonical coefficients may be subject to multicollinearity, leading to incorrect judgments. Also, because of suppression, a canonical coefficient may even have a different sign compared to the correlation of the original variable with the canonical variable. Therefore, instead, Levine recommends intepreting the relations of the original variables to a canonical variable in terms of the correlations of the original variables with the canonical variables - that is, by structure coefficients. This is now the standard approach.

• How does redundancy analysis work?
  Redundancy is the percent of variance in one set of variables accounted for by the variate of the other set. The researcher wants high redundancy, indicating that independent variate accounts for a high percent of the variance in the dependent set of original variables. Note this is not the canonical correlation squared, which is the percent of variance in the dependent variate accounted for by the independent variate. See the discussion of the redundancy coefficient. The redundancy analysis section of SAS output looks like that below, where rows 1 and 2 refer to the the first and second canonical correlations extracted for these data. Italicized comments are not part of SAS output.

                    Canonical Redundancy Analysis
  

  Raw variance tables are reported by SAS but are omitted here because redundancy is normally interpreted using the standardized tables.

            Standardized Variance of the dependent variables
                              Explained by
               Their Own                               The Opposite
          Canonical Variables                       Canonical Variables
  
                     Cumulative     Canonical                  Cumulative
       Proportion    Proportion     R-Squared    Proportion    Proportion
  1        0.2394        0.2394        0.4715        0.1129        0.1129
  2        0.3518        0.5912        0.0052        0.0018        0.1147
  
  

  The table above shows that, for the first canonical correlation, although the independent canonical variable explains 47.15% of the variance in the dependent canonical variable, the independent canonical variable is able to predict only 11.29% of the variance in the individual original dependent variables. Also, the dependent canonical variable predicts only 23.94% of the variance in the individual original dependent variables. Similar statements could be made about the second canonical correlation (row 2).

                      Canonical Redundancy Analysis
  
           Standardized Variance of the independent variables
                              Explained by
               Their Own                               The Opposite
          Canonical Variables                       Canonical Variables
  
                     Cumulative     Canonical                  Cumulative
       Proportion    Proportion     R-Squared    Proportion    Proportion
  1        0.5000        0.5000        0.4715        0.2357        0.2357
  2        0.5000        1.0000        0.0052        0.0026        0.2383
  

  The table above repeats the first, except for comparisons involving the independent canonical variable.

                      Canonical Redundancy Analysis
  
    Squared Multiple Correlations Between the dependent variables and
      the First 'M' Canonical Variables of the independent variables
  
                     M              1             2
                     Y1        0.1510        0.1526
                     Y2        0.0280        0.0305
                     Y3        0.1596        0.1610
  

  In the table above, the columns represent the canonical correlations and the rows represent the original dependent variables, three in this case. The R-squareds are the percent of variance in each original dependent variable explained by the independent canonical variables. A similar table for the independent variables and the dependent canonical variables is also output by SAS but is not reproduced here.

• Explain nonlinear canonical correlation (OVERALS)
  Nonlinear canonical correlation analysis corresponds to categorical canonical correlation analysis with optimal scaling. That is, it is to canonical correlation as categorical regression is to regression (see this link for further explanation of optimal scaling of nominal and ordinal variables). The OVERALS procedure in SPSS (part of SPSS Categories) implements nonlinear canonical correlation. Independent variables can be nominal, ordinal, or interval, and there can be more than two sets of variables (more than one independent set and one dependent set). Whereas ordinary canonical correlation maximizes correlations between the variable sets, in OVERALS the sets are compared to an unknown compromise set defined by the object scores

  OVERALS uses optimal scaling, which quantifies categorical variables and then treats as numerical variables, including applying nonlinear transformations to find the best-fitting model. For nominal variables, the order of the categories is not retained but values are created for each category such that goodness of fit is maximized. For ordinal variables, order is retained and values maximizing fit are created. For interval variables, order is retained as are equal distances between values.

  Obtain OVERALS from the SPSS menu by selecting Analyze, Data Reduction, Optimal Scaling; Select Multiple sets; Select either Some variable(s) not multiple nominal or All variables multiple nominal; click Define; define at least two sets of variables; define the value range and measurement scale (optimal scaling level) for each selected variable. SPSS output includes frequencies, centroids, iteration history, object scores, category quantifications, weights, component loadings, single and multiple fit, object scores plots, category coordinates plots, component loadings plots, category centroids plots, and transformation plots.

  Tip: To minimize output, use the Automatic Recode facility on the Transform menu to create consecutive categories beginning with 1 for variables treated as nominal or ordinal. To minimize output, for each variable scaled at the numerical (integer) level, subtract the smallest observed value from every value and add 1.

  Warning: Optimal scaling recodes values on the fly to maximize goodness of fit for the given data. As with any atheoretical, post-hoc data mining procedure, there is a danger of overfitting the model to the given data. Therefore, it is particularly appropriate to employ cross-validation, developing the model for a training dataset and then assessing its generalizability by running the model on a separate validation dataset.

  The SPSS manual notes, "If each set contains one variable, nonlinear canonical correlation analysis is equivalent to principal components analysis with optimal scaling. If each of these variables is multiple nominal, the analysis corresponds to homogeneity analysis. If two sets of variables are involved and one of the sets contains only one variable, the analysis is identical to categorical regression with optimal scaling."

  .

• What's the story about "backward" or "stepwise" canonical correlation?
  Backward canonical correlation is computed initially the same as ordinary canonical correlation, but then in additional iterations, one variable at a time is eliminated from consideration in an effort to see if the same magnitude of canonical correlation can be achieved using fewer variables. The reasoning is that fewer variables involve a more parsimonious model facilitating clearer insights into the dynamics of the phenomenon being studied. There are several alternative algorithms for elimination of variables. Thompson recommends an algorithm which assesses variables based on the product of their squared structure coefficient times the squared canonical correlation coefficient for each function (Thompson, 1984: 50). Thompson likewise recommends against algorithms focused on canonical coefficients rather than structure coefficients. For further discussion, see Thompson (1984: 47-51). It is to be emphasized, however, that variable elimination by any algorithm should be tempered by theory. That is, if the researcher's theory regarding the phenomenon at hand supposes the importance of a variable, it should not be eliminated from the canonical analysis.

• Should the canonical solution be rotated, as in factor analysis, for greater interpretability?
  Some authors have argued for varimax rotation of the matrix of structure coefficients or (less commonly) the canonical coefficients, similar to what is done routinely in factor analysis. This is possible whenever there are two or more canonical correlations in the solution. Rotation will not change the sums of the squared canonical correlation coefficients and it will lead to a simpler structure. SPSS supports rotation. However, Thompson (1984: 38) notes, "Although varimax rotation is appealing, the application seriously violates the fundamental logic of canonical analysis." The purposes of canonical analysis assume the importance of keeping separate the independent and dependent sets of variables, whereas factor analysis is used to explore the structure underlying all variables in the analysis. Thompson (1984:31-41) discusses possible strategies for using rotation but in general, most researchers do not use rotation as part of canonical analysis.

• What is the comparison of canonical correlation with factor analysis?
  Both create latent variables (variates) based on a linear combination of measured variables, but factor analysis usually is not focused on the correlation of these variates. In fact, in most forms of factor analysis, the factors are orthogonal (uncorrelated). Factor analysis is a non-dependent procedure, whereas canonical correlation can be conceptualized in terms of an independent and a dependent set of variables. Variates are rarely rotated in canonical correlation, whereas rotation of factors is the norm in factor analysis.

• How is factor analysis used in conjunction with canonical correlation?
  One can conduct two separate principal components factor analyses, one on the independent set of variables and one on the dependent. Then one can substitute factor scores for the original variables and conduct canonical correlation. If there are relatively few principal component factors which explain most of the variance in the original sets of variables, and if the factorized canonical correlation analysis shows the independent set accounts for most of the variation in the dependent set, then the researcher has demonstrated that "the same set of underlying factors which account for most of the variation within variable sets are also important in determining relationships across variable sets" (Dunteman, 1989: 86).

• How does SPSS MANOVA output differ from SPSS canonical correlation output?
  MANOVA produces output which is extraneous to canonical correlation, and it will not save canonical scores.

• One of my variables is a multi-response item. What do I do?
  One method is to compute a new nominal variable in which every combination of responses is a separate value (use the IF statements in SPSS), then you may use chi-square or nominal measures of association. Alternatively, you might consider using the SPSS CATEGORIES module after recoding your multi-response item as a set of separate variables. CATEGORIES then gives you the canonical correlation coefficient between your multiple response set as independents and another variable such as income level as dependent.




Bibliography

• Alpert, Mark I. & Peterson, Robert A. (1972). On the interpretation of canonical analysis. Journal of Marketing Research 9(2), 187-192.
• Barcikowski, R., and J. P. Stevens(1975). A Monte Carlo study of the stability of canonical correlations, canonical weights, and canonical variate-variable correlations. Multivariate Behavioral Research, Vol. 10: 353-364. The authors recommend 40 - 60 cases per variable included in canonical analysis, based on Monte Carlo simulation experiments.
• Dunteman, George H. (1989). Principal components analysis. Thousand Oaks, CA: Sage Publications, Quantitative Applications in the Social Sciences Series, No. 69.Pp. 80-87 deal with the use of factor analysis in conjunction with canonical correlation.
• Gifi, A. (1990), Non-linear Multivariate Analysis. Chichester, UK: Wiley & Sons. Chapter 6 discusses a nonlinear version of canonical correlation.
• Green, P. E., M. H. Halbert, & P. J. Robinson (1966). Canonical analysis: An exposition and illustrative application. Journal of Marketing Research, Vol III (Feb.): 32-39. A seminal postwar popularization of canonical correlation methodology.
• Hotelling, H. (1935). The most predictable criterion. Journal of Educational Psychology, Vol. 26: 139-142. The article in which canonical correlation was originally set forth.
• Levine, Mark S. (1977). Canonical analysis and factor comparison. Thousand Oaks, CA: Sage Publications, Quantitative Applications in the Social Sciences Series, No. 6.
• Stevens, J. (1986). Applied multivariate statistics for the social sciences. Hillsdale, NJ: Erlbaum. Covers canonical correlation, recommends 20 cases per included variable for interpretation of the first canonical correlation.
• Tabachnick, Barbara G. & Fidell, Linda S. (1996) Using multivariate statistics, Third edition. NY: Harpercollins College Publishers. Chapter 6 is a contemporary overview.
• Thompson, Bruce (1984). Canonical correlation analysis: Uses and interpretation. Thousand Oaks, CA: Sage Publications, Quantitative Applications in the Social Sciences Series, No. 47.

Copyright 1998, 2008 by G. David Garson.
last update 3/24/2008.