Multivariate GLM, MANOVA, and MANCOVA

Overview Multivariate GLM is the version of the general linear model now often used to implement two long-established statistical procedures - MANOVA and MANCOVA. Multivariate GLM, MANOVA, and MANCOVA all deal with the situation where there is more than one dependent variable and one or more independents. MANCOVA also supports use of continuous control variables as covariates. Multiple analysis of variance (MANOVA) is used to see the main and interaction effects of categorical variables on multiple dependent interval variables. MANOVA uses one or more categorical independents as predictors, like ANOVA, but unlike ANOVA, there is more than one dependent variable. Where ANOVA tests the differences in means of the interval dependent for various categories of the independent(s), MANOVA tests the differences in the centroid (vector) of means of the multiple interval dependents, for various categories of the independent(s). One may also perform planned comparison or post hoc comparisons to see which values of a factor contribute most to the explanation of the dependents. There are multiple potential purposes for MANOVA. To compare groups formed by categorical independent variables on group differences in a set of interval dependent variables. To use lack of difference for a set of dependent variables as a criterion for reducing a set of independent variables to a smaller, more easily modeled number of variables. To identify the independent variables which differentiate a set of dependent variables the most. Multiple analysis of covariance (MANCOVA) is similar to MANOVA, but interval independents may be added as "covariates." These covariates serve as control variables for the independent factors, serving to reduce the error term in the model. Like other control procedures, MANCOVA can be seen as a form of "what if" analysis, asking what would happen if all cases scored equally on the covariates, so that the effect of the factors over and beyond the covariates can be isolated. The discussion of concepts in the ANOVA section also applies, including the discussion of assumptions. See also: Univariate GLM, ANOVA, and ANCOVA Discriminant function analysis, which yields results equivalent to one-way MANOVA.

Contents Key concepts and terms GLM Significance tests Post hoc tests Contrast tests Canonical correlation Plots Profile analysis Repeated measures Assumptions SPSS output Frequently asked questions Bibliography

Key Concepts

• General Linear Model (GLM). In more recent versions of SPSS, MANOVA and MANCOVA are found under "GLM" (General Linear Model). Output is still similar, but with GLM, parameters (coefficients) are created for every category of every factor and this "full parameterization" approach handles the problem of empty cells better than traditional MANOVA. GLM accepts categorical variables which, in SPSS regression, must be manipulated manually as dummy variables. That is, GLM automatically transforms declared categorical variables into sets of indicator variables. GLM calculates parameters using IWLS (iterative weighted least squares). The seminal article on GLM is Nelder and Wedderburn (1972) and an overview is Gill (2001).
  • GLM vs. MANOVA procedures. Starting with SPSS 7.0, SPSS replaced the MANOVA procedure with the GLM procedure in the menu system. Note, however, that the MANOVA procedure remains available in syntax and has some features not available in GLM, such as correction for violation of sphericity. GLM and MANOVA syntax are listed in the FAQ section below.

  • Models are entered in the same manner as in univariate GLM, in SPSS using the Model button dialog illustrated below. One may specify a full factorial model or custom main and interaction effects.
    

    Though not illustrated here, output for multivariate GLM contains lack of fit tests as in univariate GLM, used when the researcher has employed a custom (non-full factorial) model and wishes to test if terms present in the full factorial model but absent in the custom model should have been included. See the lack of fit test output in the section on univariate GLM.

  • Estimated marginal means, as in univariate GLM, show if groups formed by the independent factor (ex., region) vary with respect to the dependent variables, controlling for any covariates in the model, as illustrated below.
    

    The more the means of the dependents vary by factor level, the stronger the relation of the factor to the dependent. In the example above, means vary only a little, indicating a weak relationship. By examining the overlap of upper and lower bounds one finds that most regional comparisons are not significant. An exception is Southeast vs. West for "Having trouble with one's boss."

  • Pairwise comparison table presents the same information as the estimated marginal means, but in a different format, illustrated below.
    

  
  

• Significance Tests
  • F-test. The omnibus or overall F test is the first of the two-step MANOVA process of analysis. The F test appears, illustrated below, in the "Corrected Model" top row of the "Tests of Between-Subjects Effects" table of GLM MANOVA output in SPSS and answers the question, "Is the model significant for each dependent?" There will be an F significance level for each dependent. That is, the F test tests the null hypothesis that there is no difference in the means of each dependent variable for the different groups formed by categories of the independent variables.

  • Multivariate tests in contrast, answer the question, "Is each effect significant?" That is, where the F test focuses on the dependents, the multivariate tests focus on the independents and their interactions. These tests appear in the "Multivariate Tests" table of SPSS output. The multivariate formula for F is based not only on the sum of squares between and within groups, as in ANOVA, but also on the sum of crossproducts - that is, it takes covariance into account as well as group means. In the figure below, both the covariate effect age and the effect of the fixed factor region are significant by any of the four leading multivariate tests of group differences.
    
    • Hotelling's T-Square is the most common, traditional test where there are two groups formed by the independent variables. SPSS prints the related statistic, Hotelling's Trace (a.k.a. Lawley-Hotelling or Hotelling-Lawley Trace). To convert from the Trace coefficient to the T-Square coefficient, multiply the Trace coefficient by (N-g), where N is the sample size across all groups and g is the number of groups. The T-Square result will still have the same F value, degrees of freedom, and significance level as the Trace coefficient. The larger the Hotelling's trace, the more the given effect contributes to the model.

    • Wilks' lambda, U. This is the most common, traditional test where there are more than two groups formed by the independent variables. Wilks' lambda is a multivariate F test, akin to the F test in univariate ANOVA. It is a measure of the difference between groups of the centroid (vector) of means on the independent variables. The smaller the lambda, the greater the differences. The Bartlett's V transformation of lambda is then used to compute the significance of lambda. Wilks's lambda is used, in conjunction with Bartlett's V, as a multivariate significance test of mean differences in MANOVA, for the case of multiple interval dependents and multiple (>2) groups formed by the independent(s). The t-test, Hotelling's T, and the F test are special cases of Wilks's lambda. Wilks' lambda ranges from 0 to 1, and the lower the Wilks' lambda, the more the given effect contributes to the model.

    • Pillai's trace, also called Pillai-Bartlett trace, V. Multiple discriminant analysis (MDA) is the part of MANOVA where canonical roots are calculated. Each significant root is a dimension on which the vector of group means is differentiated. The Pillai-Bartlett trace is the sum of explained variances on the discriminant variates, which are the variables which are computed based on the canonical coefficients for a given root. Olson (1976) found V to be the most robust of the four tests and is sometimes preferred for this reason. The larger the Pillai's trace,.the more the given effect contributes to the model. Pillai's trace is always smaller than Hotelling's trace.

    • Roy's greatest characteristic root (GCR), called "Roy's largest root" in SPSS, is similar to the Pillai-Bartlett trace but is based only on the first (and hence most important) root.Specifically, let lambda be the largest eigenvalue, then GCR = lambda/(1 + lambda). Note that Roy's largest root is sometimes also equated with the largest eigenvalue, as in SPSS's GLM procedure (however, SPSS reports GCR for MANOVA). GCR is less robust than the other tests in the face of violations of the assumption of multivariate normality. The larger the root, the more that effect contributes to the model. Note, however, that Roy's largest root sets a lower bound for the significance value, tending to make this test more prone to Type 1 error (false positives) than the nominal significance level might suggest.

    
    

  • Tests of Between Subjects Effects. As in univariate GLM, the "Tests of Between Subjects Effects," shown in the figure below, applies an F test of significance to the relation of each covariate (age in the example below) and each fixed factor (region below) in relation to each of the dependent variables (there are three work variables listed below).

    

    In the Tests of Between Subjects Effects table, partial eta-square serves as an effect size measure. The noncentrality index is used to compute the power level, which by rule of thumb should be equal or greater than .80 to accept with confidence that the chance of Type II error is low enough for a finding of non-significance by the F test (that is, to be confident that the relationship does not exist). Thus by the F test, we fail to reject the null hypothesis that region is unrelated to being unemployed. However, because the power of this test is well below .80, we acknowledge that the chance of Type II error is insufficiently low to be confident in this judgment.

  • Tests of Parameters
    • Parameters (b coefficients) in multivariate GLM are not interpreted as in OLS regression and are not usually a focus of analysis in multivariate GLM. A unit change in an independent variable k does not correspond to a b_k change in the dependent variable as in OLS regression. This is because multivariate GLM uses a nonlinear link function. If b coefficients are examined, it is in terms of first differences: the researcher determines two levels of interest for a given independent variable, then calculates the value of the dependent variables under these two conditions, holding all other variables constant at their mean value.
      

      The t-test significance for levels of a factor (ex., region in the figure above) refers to the significance of the contrast between the given level and the reference level (the last category if the default is accepted), in this case Region 3 = West. For covariates (ex., age above), the t-test significance level will be the same in level as in the Tests of Between Subjects Effects table above and refers to whether or not the covariate contributes significantly to the model.

  
  

• Post-Hoc Tests. The second step in MANOVA is that if the overall F-test shows the centroid (vector) of means of the dependent variables is not the same for all the groups formed by the categories of the independent variables, post-hoc univariate F tests of group differences are used to determine just which group means differ significantly from others. This helps specify the exact nature of the overall effect determined by the F test. Pairwise multiple comparison tests test each pair of groups to identify similarities and differences. Multiple comparison procedures and post hoc tests are discussed more extensively in the corresponding section under ANOVA.
  Bonferroni adjustment. When there are many dependents, some univariate tests might be significant due to chance alone. That is the nominal .05 level is not the actual alpha level. Researchers may adjust the nominal alpha level. Actual alpha = 1 - (1-alpha1)(1-alpha2)...(1-alphan), where alpha1 to alpha-n are the nominal levels of alpha for a series of post hoc tests. For instance, for a series of 4 tests at the nominal alpha level of .01, the actual alpha would be estimated to be 1-.99⁴ = .039. One wants an actual adjusted alpha level of at least .05.
  • Bonferroni test: If the Bonferroni test is requested, SPSS will print out a table of "Multiple Comparisons" giving the mean difference in the dependent variable between any two groups (ex., differences in test scores for any two educational groups). The significance of this difference is also printed, and an asterisk is printed next to differences significant at the .05 level or better. The Bonferroni method is preferred when the number of groups is small.

  • Tukey test: If the Tukey test is requested, SPSS will produce a similar table which is interpreted in the same way. The Tukey method is preferred when the number of groups is large.

  • Other tests: Methods when the assumption of homogeneity of variances is not met: SPSS provides these alternate methods not shown here: Games-Howell, Tamhane's T2, Dunnett's T3, and Dunnett's C.

  
  

• Contrast Tests are the same as described in the contrasts tests section for univariate GLM, except there will be multiple columns for the multiple dependent variables.

  

  
  

• Canonical Correlation
  • In general: When using MANOVA, social scientists are often concerned with the overall F-test of significance, which is a test of the null hypothesis of no group differences, as shown in the SPSS output section titled "Analysis of Variance." However, the second optional step in MANOVA is to examine the output block for each effect, titled "Eigenvalues and Canonical Correlations." This can be used to better understand the nature of group differences established by the overall F test. However, canonical correlation and related coefficients are only available in SPSS in syntax using MANOVA and is not available in GLM, either through the menu system or in GLM syntax. The meaning of elements of this MANOVA output section are discussed below.

  • Canonical roots or linear discriminant functions, LDFs. In order to test the hypothesis that groups differ significantly on weighted combinations of the observed independent variables, MANOVA conducts a multiple discriminant analysis (MDA). MDA partitions the variance of dependent variables into components also called canonical roots or LDFs. The canonical roots are analogous to principal components in factor analysis, except they seek to maximize the between-groups variance. In canonical correlation, the canonical roots are also called canonical variables.

    Each canonical root represents a dimension of meaning, but what? What meaningful label do we give to each canonical root (which SPSS labels merely, 1, 2, etc.)? In factor analysis one ascribes a label to each factor based on the factor loadings of each measured variable on the factor. In MANOVA, this is done on multiple bases, using the raw weights, standardized weights, and structure correlations. The structure correlations are often the most useful for this purpose when there is more than one significant canonical root. Structure correlations are the correlations between the measured variables and the canonical roots. In MANOVA, there will be one set of MDA output for each main and interaction effect.

  • Eigenvalue: The eigenvalue has to do with the proportion of the total variance of the group of variables included in the analysis which is accounted for by a specific canonical root. That is, the larger the eigenvalue, the larger the group differences on the variate (the variable computed by the linear combination of canonical coefficients) for that canonical root. SPSS will print out the associated percent of variance explained and cumulative percent next to each canonical root in the section "Eigenvalues and Canonical Correlations."
    • Condition indices. A "singular value" is the square root of an eigenvalue, and "condition indices" are the ratio of the largest singular values to each other singular value. Condition indices are used to flag excessive collinearity in the data. A condition index over 30 suggests serious collinearity problems and an index over 15 indicates possible collinearity problems. See further discussion in the section on regression diagnostics.

  • Canonical correlation: Every main and interaction effect has a set of canonical roots, and every canonical root has both a corresponding eigenvalue and canonical correlation coefficient. The canonical coefficients are the weights in the linear combination of canonical roots relating the independent and dependent sets of variables being canonically correlated, which maximize the correlation relating the two sets of variables. In MANOVA there will be a set of canonical roots for each main and interaction effect of the independents on the set of dependent variables, and there will be one canonical coefficient for each canonical root. Since canonical correlation coefficients are standardized, their weights may be compared. The ratio of canonical correlation weights for a set of canonical roots is their relative importance for the given effect.

  • F-to-remove index. Many computer programs output this F test, which gauges the effect of removing a given variable from the analysis. The larger the F-to-remove index, the more that variable contributes to group separation.

  
  

• Plots
  • Spread-versus-level plots depict standard deviations vs. means, or variances vs. means, for each dependent variable. Each point shows the value of a factor design matrix group cell on the mean and on the standard deviation or variance. This is useful in testing the homogeneity of variances assumption, and in identifying cells which deviate substantially from the assumption.
    

  • Observed*Predicted*Standardized Residual Plots. For each dependent variable, a plot is produced which shows the 6 comparisons among observed, predicted, and standardized residuals. For observed by predicted, one would like to see a clear pattern approaching linearity, but for the plots involving standardized residuals, one would like to see a cloud-like random pattern. The figure below illustrates two such plots, with the one on the left for a dichotomous dependent variable and the one on the right for a continuous dependent.
    

    For the credit card case on the right, for observed by predicted there is a mostly random pattern, indicative of a weak or nonexistent relationship between the predictors and the dependent. The upward linear pattern of residuals by observed shows that as the observed value of credit card debt goes up, model error increases. For the example on the right for a dichotomous dependent, as the observed value goes from 0 to 1, predicted values are a bit higher but the overlap is very large, again indicating a weak or possibly nonsignificant relationship. With a dichotomy, the residual plots cannot assume a random cloud pattern, but it is again seen that as observed increases in value, so does residual (error) value.

    
    

  • Profile plots are line plots of the predicted means of each dependent variable across levels of each factor. When two or three factors are involved, these are called interaction plots. Where a plot of observed means would show the effect being studied and the error, the profile plots of predicted means show the effect without the error. Each point in a profile plot indicates the estimated marginal mean of the dependent variable (adjusted for covariates in MANCOVA) at one level of a given factor. The profile plot shows if the estimated marginal means are increasing across levels.
    

    In profile plots, additional factors can be represented by additional lines, as shown below, where parallel lines indicate no interaction and crossing lines (as in this example) indicate interaction among the factors (here, interaction between education and defaulting on a loan, when dependent variables are types of debt).

    

  
  

• Profile Analyis
  Profile analysis is equivalent to repeated measures MANOVA. There is a within-subjects factor, which is either time (the same item administered at sequential time periods) or tests (repeated measures of the same underlying construct). Then there is a between-subjects grouping factor. For instance, tests t1, t2, t3, and t4 could be grouped by gender (m, f). For categories of the grouping factor (here, gender), one could plot the mean response on each of the multiple tests, connecting the means with lines, one line per category of the factor (thus two lines for gender). The lines are termed "profiles." One asks if the profiles are parallel, if the profiles are equal or separated, and if the means of each factor category are the same for each of the dependent variables. In SPSS, profile tests can be accomplished in MANOVA by clicking on the Paste button and adding LMATRIX and MMATRIX commands in syntax to specify the type of contrast wanted, as described below.

  Note that profile analysis in MANOVA has been superceded to some extent by multidimensional scaling, mixed model ANOVA, and/or random effects regression models.

  • Parallelism tests test if profiles are parallel. This is equivalent to testing that there is no interaction of the within-subjects factor with the between-subjects factor (ex., between tests and gender). A finding of non-significance means the profiles are not significantly different in shaper and the researcher concludes the profiles are parallel. In SPSS, the syntax is of the form:

    /LMATRIX = GENDER 1  -1
    /MMATRIX
    t1   1   t2   -1;
    t2   1   t3   -1;
    t3   1   t4   -1. 
    

    The LMATRIX command specifies a contrast between the two values of gender. The MMATRIX command asks for contrasts between t1 and t2, between t2 and t3, and between t3 and t4. Output will be in a section labeled "Custom Hypothesis Tests".

  • Equality of profiles tests, also called separation of group profiles or testing the groups hypotheses. test if parallel profiles are equal (coincident) or separated (dispersed). This is equivalent to testing that there is no main effect of the between-subjects factor (ex., gender). A finding of non-significance means the grouping variable (ex., gender) has no effect and the profiles of gender are equal. In SPSS, the syntax is of the form:

    /LMATRIX = gender   1   -1
    /MMATRIX = t1   .25   t2   .25   t3   .25   t4   .25.
    

    The LMATRIX command again specifies a contrast between the two values of gender. The MMATRIX command asks for a single equality test comparing the four levels of "test" on an equally weighted basis. Output will be in a section labeled "Custom Hypothesis Tests."

  • Equality of means tests, also called tests of flatness, test if the means are equal between factor categories for each dependent (es., between male and female for each of the t1. t2. t3. and t4 tests). This is equivalent to testing if there are significant differences across levels of the within-subjects factor (tests). That is, this is testing if there are significant differences across levels of the within-subjects factor (ex., tests t1, t2, t3, and t4), ignoring the between-groups factor (ex., ignoring gender by looking at the whole sample). No differences would mean a flat profile, hence "tests of flatness." In these tests, the contrast is with the intercept (constant), which represents the equally weighted average of the within-groups factor (the dependent measures, ex. t1, t2, t3, and t4) when the grouping factor is unknown (treated as 0). A finding of non-significance means the gwo genders are not significantly different on the average on any of the levels of the within-groups factor (ex., on t1, t2, t3, and t4). In SPSS, the syntax is of the form:

    /LMATRIX = INTERCEPT   1   gender   .5   .5
    /MMATRIX
    t1   1   t2   -1;
    t2   1   t3   -1;
    t3   1   t4   -1. 
    

    The LMATRIX command specifies a contrast between the two eqaully weighted values of gender and the intercept. The MMATRIX command asks for contrasts between t1 and t2, between t2 and t3, and between t3 and t4. SPSS output will be in a section labeled "Custom Hypothesis Tests."

  
  

• Repeated Measures MANOVA.
  • SPSS. Select Analyze, General Linear Model, Repeated. This section is discussed separately.

Assumptions

• Observations are independent of one another. The usual MANOVA (in SPSS, Analyze, General Linear Model, Multivariate) is not robust when the selection of one observation depends on selection of one or more earlier ones, as in the case of before-after and other repeated measures designs. However, there does exist a variant of MANOVA for repeated measures designs (select Analyze, .General Linear Model, Repeated).

• The independent variable is or variables are categorical.

• The dependent variables are continuous and interval level.

• Low measurement error of the covariates. The covariate variables are continuous and interval level, and are assumed to be measured without error. Imperfect measurement reduces the statistical power of the F test for MANCOVA and for experimental data, there is a conservative bias (increased likelihood of Type II errors: thinking there is no relationship when in fact there is a relationship) . As a rule of thumb, covariates should have a reliability coefficient of .80 or higher.

• Equal group sizes. To the extent that group sizes are very unequal, statistical power diminishes. SPSS adjusts automatically for unequal group sizes. In SPSS, METHOD=UNIQUE is the usual method.

• Appropriate sums of squares. Normally there are data for every cell in the design. For instance, 2-way ANOVA with a 3-level factor and a 4-level factor will have 12 cells (gropups). But if there are no data for some of the cells, the ordinary computation of sums of squares ("Type III" is the ordinary, default type) will result in bias. When there are empty cells, one must ask for "Type IV" sums of squares, which compare a given cell with averages of other cells. In SPSS, Analyze, General Linear Model, Univariate; click Model, then set "Sum of Squares" to "Type IV" or other appropriate type depending on one's design:
  1. Type I. Used in hierarchical balanced designs where main effects are specified before first-order interaction effects, and first-order interaction effects are specified before second-order interaction errects, etc. Also used for purely nested models where a first effect is nested within a second effect, the second within a third, etc. And used in polynomial regression models where simple terms are specified before higher-order terms (ex., squared terms).

  2. Type II. Used with purely nested designs which have main factors and no interaction effects, or with any regression model, or for balanced models common in experimental research.

  3. Type III. The default type and by far the most common, for any models mentioned above and any balanced or unbalanced model as long as there are no empty cells in the design.

  4. Type IV. Required if any cells are empty in a balanced or unbalanced design. This would include all nested designs, such as Latin square design.

• Adequate sample size. Small samples may have lower power. At a minimum, every cell must have more cases than there are dependent variables. With multiple factors and multiple dependents, group sizes fall below minimum levels more easily than in ANOVA/ANCOVA.

• Residuals are randomly distributed.

• Homoscedasticity (homogeneity of variances and covariances): within each group formed by the categorical independents, the error variance of each interval dependent should be similar, as tested by Levene's test, below. Also, for each of the k groups formed by the independent variables, the covariance between any two dependent variables must be the same. When sample sizes are unequal, tests of group differences (Wilks, Hotelling, Pillai-Bartlett, GCR) are not robust when this assumption is violated. Pillai-Bartlett trace was found to be more robust than the alternatives when this assumption was violated but sample sizes of the groups were equal (Olson, 1976).
  • Box's M: Box's M tests MANOVA's assumption of homoscedasticity using the F distribution. If p(M)<.05, then the covariances are significantly different. Thus we want M not to be significant, rejecting the null hypothesis that the covariances are not homogeneous. That is, the probability value of this F should be greater than .05 to demonstrate that the assumption of homoscedasticity is upheld. Note, however, that Box's M is extremely sensitive to violations of the assumption of normality, making the Box's M test less useful than might otherwise appear. For this reason, some researchers test at the p=.001 level, especially when sample sizes are unequal. In the example of SPSS output below, Box's M shows the assumption of equality of covariances among the set of dependent variables is violated with respect to the groups formed by the categorical independent factor "region".
    

  • Levene's Test. SPSS also outputs Levene's test as part of Manova. This is discussed in the section on ANOVA. If Levene's test is significant, then the data fail the assumption of equal group error variances. In the figure below, Levene's test shows that the assumption of homogeneity of error variances among the groups of "region" is violated for two of the three dependent variables listed.
    

• Homogeneity of regression. The covariate coefficients (the slopes of the regression lines) are the same for each group formed by the categorical variables and measured on the dependents. The more this assumption is violated, the more conservative MANCOVA becomes (the more likely it is to make Type I errors - accepting a false null hypothesis). When running a MANCOVA model in SPSS, include in the model options the interactions between the covariate(s) and each independent variable -- any significant interaction effects indicate that the assumption of homogeneity of regression coefficients has been violated. See the discussion in the section on testing assumptions.

• Sphericity. In a repeated measures design, the univariate ANOVA tables will not be interpreted properly unless the variance/covariance matrix of the dependent variables is circular in form (see Huynh and Mandeville, 1979). A spherical model implies that the assumptions of multiple univariate ANOVA is met, that the repeated contrasts are uncorrelated. When there is a violation of this assumption in a repeated measures design, the researcher should use MANOVA rather than multiple univariate ANOVA tests.
  
  • Bartlett's test of sphericity

    • Bartlett's test prior to MANOVA. Bartlett's test can be used to determine if the researcher is warranted in going ahead with a multiple analysis of variance rather than a series of univariate analyses. Such analysis would be inappropriate if on the whole, dependent variables were not significantly intercorrelated - which is what Bartlett's test tests. Thus it tests the null hypothesis that the covariance matrix is proportional to an identity matrix (an identity matrix would indicate all dependent variables were uncorrelated). A finding of significance means the null hypothesis is rejected, meaning that at least one of the inter-correlations is non-zero. Thus, if Bartlett's test is significant, the researcher concludes that it is appopriate to proceed with multiple analysis of variance or with factor analysis. To obtain Bartlett's test in GLM univariate, check "Descriptive Statistics" under the Options button.

    • Bartlett's test within MANOVA. When applied to the residual covariance matrix in the context of MANOVA, Bartlett's test is testing if there are still significant correlations among the dependent variables after controlling for variables in the MANOVA model. In a good model, the residual correlations approach zero because the residuals are random. Thus in the context of MANOVA, the researcher wants a finding of non-significance on Bartlett's test in order to uphold the MANOVA model. To obtain Bartlett's test, check "Residual SSCP" under the Options button in the SPSS Multivariate GLM dialog.

    • Bartlett's test in repeated measures MANOVA tests for the assumption of sphericity, required by repeated measures designs, to assure that the variance of effect differences is equal across time groups. Bartlett's test in repeated measures is discussed in the separate section on repeated measures GLM.

    Note that Bartlett's test is most useful for small samples: in large samples Bartlett's test may be too sensitive (too likely to find significant correlations).

  • Mauchly's test of sphericity. This alternative sphericity test is derogated because it lacks power for small samples (it fails too often to identify violations of sphericity) and it is too conservative for large samples (too small violations of sphericity are found significant).

  • Corrections for violation of sphericity. Two main procedures have been used to correct for lack of sphericity in repeated measures designs. Both are available in SPSS only in syntax, using the MANOVA rather than GLM procedure.

    • Huynh-Feldt correction. The epsilon adjustment factor is computed which is used to adjust the degrees of freedom in the F test of sphericity. The adjustment, which is based on the degree of heterogeneity of variances, makes the critical value of F larger, thus making a finding of significance more difficult.

    • Geisser-Greenhouse (lower bound) correction.. In this correction, epsilon is computed differently and then used to correct the degrees of freedom more than the Huynh-Feldt correction. That is, this is a more severe correction which is more conservative (a finding of significance is even more difficult) While GG correction has lower Type I error, its power is low, leading to too many Type II errors. .

• Multivariate normal distribution. For purposes of significance testing, response (dependent) variables are assumed to have multivariate normal distributions. In practice, it is common to assume multivariate normality if each variable considered separately follows a normal distribution. MANOVA is robust in the face of most violations of this assumption if sample size is not small (ex., <20).

• No outliers. MANCOVA is highly sensitive to outliers in the covariates.

• Covariates are linearly related or in a known relationship to the dependents. The form of the relationship between the covariate and the dependent must be known and most computer programs assume this relationship is linear, adjusting the dependent mean based on linear regression. Scatterplots of the covariate and the dependent for each of the k groups formed by the independents is one way to assess violations of this assumption. Covariates may be transformed (ex., log transform) to establish a linear relationship.

• Does not assume linear relationships; can handle interaction effects.

• See also: assumptions sections for ANOVA and ANCOVA.

  
  

SPSS Output Examples, with Commentary

• MANOVA
• MANCOVA

Frequently Asked Questions

• What is the GLM syntax in SPSS?

  GLM dependent varlist [BY factor list [WITH covariate list]]
  [/WSFACTOR=name levels [{DEVIATION [(refcat)]       }] name...
                          {SIMPLE [(refcat)]          }
                          {DIFFERENCE                 }
                          {HELMERT                    }
                          {REPEATED                   }
                          {POLYNOMIAL [({1,2,3...})]**}
                          {             {metric  }    }
                          {SPECIAL (matrix)           }
  [/MEASURE=newname newname...]
  [/WSDESIGN=effect effect...]†
  [/RANDOM=factor factor...]
  [/REGWGT=varname]
  [/METHOD=SSTYPE({1  })]
                  {2  }
                  {3**}
                  {4  }
  [/INTERCEPT=[INCLUDE**] [EXCLUDE]]
  [/MISSING=[INCLUDE] [EXCLUDE**]]
  [/CRITERIA=[EPS({1E-8**})][ALPHA({0.05**})]
                  {a     }         {a     }
  [/PRINT  = [DESCRIPTIVE] [HOMOGENEITY] [PARAMETER][ETASQ] [RSSCP]
             [GEF] [LOF] [OPOWER] [TEST [([SSCP] [LMATRIX] [MMATRIX])]]
  [/PLOT=[SPREADLEVEL] [RESIDUALS]
         [PROFILE (factor factor*factor factor*factor*factor ...)]
  [/TEST=effect VS {linear combination [DF(df)]}]
                   {value DF (df)              }
  [/LMATRIX={["label"] effect list effect list ...;...}]
            {["label"] effect list effect list ...    }
            {["label"] ALL list; ALL...               }
            {["label"] ALL list                       }
  [/CONTRAST (factor name)={DEVIATION[(refcat)]** ‡    }]
                           {SIMPLE [(refcat)]         }
                           {DIFFERENCE                }
                           {HELMERT                   }
                           {REPEATED                  }
                           {POLYNOMIAL  [({1,2,3...})]}
                           {              {metric  }  }
                           {SPECIAL (matrix)          }
  [/MMATRIX= {["label"] depvar value depvar value ...;["label"]...}]
             {["label"] depvar value depvar value ...             }
             {["label"] ALL list; ["label"] ...                   }
             {["label"] ALL list                                  }
  [/KMATRIX= {list of numbers    }]
             {list of numbers;...}
  [/POSTHOC = effect [effect...]
            ([SNK] [TUKEY] [BTUKEY][DUNCAN]
             [SCHEFFE] [DUNNETT(refcat)] [DUNNETTL(refcat)]
             [DUNNETTR(refcat)] [BONFERRONI] [LSD] [SIDAK]
             [GT2] [GABRIEL] [FREGW] [QREGW]  [T2] [T3] [GH][C]
             [WALLER ({100** })]]
                      {kratio}
             [VS effect]
  [/EMMEANS=TABLES({OVERALL             })] [COMPARE ADJ(LSD) (BONFERRONI) (SIDAK)]
                   {factor                }
                   {factor*factor...      }
                   {wsfactor              }
                   {wsfactor*wsfactor ... }
                   {factor*...wsfactor*...}   
  [/SAVE=[tempvar [(list of names)]] [tempvar [(list of names)]]...]
         [DESIGN]
  [/OUTFILE=[{COVB('savfile'|'dataset')}] 
             {CORB('savfile'|'dataset')}
            [EFFECT('savfile'|'dataset')] [DESIGN('savfile'|'dataset')]
  [/DESIGN={[INTERCEPT...]    }]
           {[effect effect...]}
  
  
  † WSDESIGN uses the same specification as DESIGN, with only within-subjects factors.
  ‡ DEVIATION is the default for between-subjects factors, while POLYNOMIAL is the default for within-subjects factors.
  ** Default if the subcommand or keyword is omitted.
  

• What is the MANOVA syntax in SPSS?
  The MANOVA procedure is only available in syntax. While largely superceded by GLM, MANOVA does have some functionality not present in GLM, such as correction for violation of sphericity.

  MANOVA dependent varlist [BY factor list (min,max)[factor list...]
                           [WITH covariate list]]
   [/WSFACTORS=varname (levels) [varname...] ]
   [/WSDESIGN]*
   [/TRANSFORM [(dependent varlist [/dependent varlist])]=
                       [ORTHONORM] [{CONTRAST}]  {DEVIATION (refcat)   } ]
                                    {BASIS   }   {DIFFERENCE           }
                                                 {HELMERT              } 
                                                 {SIMPLE (refcat)      }
                                                 {REPEATED             }
                                                 {POLYNOMIAL [({1,2,3...})]} 
                                                 {             {metric  }  }
                                                 {SPECIAL (matrix)     }
   [/MEASURE=newname newname...]
   [/RENAME={newname} {newname}...]
            {*      } {*      }
   [/ERROR={WITHIN           } ]
           {RESIDUAL         }
           {WITHIN + RESIDUAL}
           {n                }
   [/CONTRAST (factorname)={DEVIATION** [(refcat)]    }] †
                           {POLYNOMIAL**[({1,2,3...})]} 
                           {              {metric  }  }
                           {SIMPLE [(refcat)]         }
                           {DIFFERENCE                }
                           {HELMERT                   }
                           {REPEATED                  }
                           {SPECIAL (matrix)          }
   [/PARTITION (factorname)[=({1,1...  })]] 
                              {n1,n2...}
   [/METHOD=[{UNIQUE**  }]  [{CONSTANT**}] [{QR**    }]]
             {SEQUENTIAL}    {NOCONSTANT}   {CHOLESKY}	
   [/{PRINT  }= [CELLINFO [([MEANS] [SSCP] [COV] [COR] [ALL])]]    
     {NOPRINT}  [HOMOGENEITY [([ALL] [BARTLETT] [COCHRAN] [BOXM])]] 
                [DESIGN [([OVERALL] [ONEWAY] [DECOMP] [BIAS] [SOLUTION]
                         [REDUNDANCY] [COLLINEARITY] [ALL])]]
                [PARAMETERS [([ESTIM] [ORTHO][COR][NEGSUM][EFSIZE][OPTIMAL][ALL])]]
                [SIGNIF [[(SINGLEDF)]
                         [(MULTIV**)] [(EIGEN)] [(DIMENR)]
                         [(UNIV**)] [(HYPOTH)][(STEPDOWN)] [(BRIEF)]
                         [{(AVERF**)}] [(HF)] [(GG)] [(EFSIZE)]]
                          {(AVONLY) }
               [ERROR[(STDDEV)][(COR)][(COV)][(SSCP)]]
   [/OMEANS =[VARIABLES(varlist)] [TABLES ({factor name     }] ]
                                           {factor BY factor}
                                           {CONSTANT        }
   [/PMEANS =[VARIABLES(varlist)] [TABLES ({factor name     })] [PLOT]] ]
                                           {factor BY factor}
                                           {CONSTANT        }
   [/RESIDUALS=[CASEWISE] [PLOT] ]
   [/POWER=[T({.05**})] [F({.05**})] [{APPROXIMATE}]] 
              {a    }      {a    }    {EXACT      }
   [/CINTERVAL=[{INDIVIDUAL}][({.95}) ] 
                {JOINT     }   {a  }    
               [UNIVARIATE ({SCHEFFE})]
                            {BONFER }  
               [MULTIVARIATE  ({ROY      })]  ] 
                               {PILLAI   }
                               {BONFER   }
                               {HOTELLING}
                               {WILKS    }
   [/PCOMPS [COR] [COV] [ROTATE(rottype)]
            [NCOMP(n)] [MINEIGEN(eigencut)] [ALL] ]
   [/PLOT=[BOXPLOTS] [CELLPLOTS] [NORMAL]  [ALL] ] 
   [/DISCRIM [RAW] [STAN] [ESTIM] [COR] [ALL] 
             [ROTATE(rottype)] [ALPHA({.25**})]] 
                                      {a    }
   [/MISSING=[LISTWISE**] [{EXCLUDE**}] ]
                           {INCLUDE  } 
   [/MATRIX=[IN({file})]  [OUT({file})]]
                {[*] }         {[*] }
   [/ANALYSIS [({UNCONDITIONAL**})]=[(]dependent varlist
                {CONDITIONAL    }      [WITH covariate varlist]
                                        [/dependent varlist...][)][WITH varlist] ]
   [/DESIGN={factor [(n)]  }[BY factor[(n)]] [WITHIN factor[(n)]][WITHIN...]
            {[POOL(varlist)}
            [+ {factor [(n)] }...]
               {POOL(varlist)}
            [[= n] {AGAINST} {WITHIN  }
                   {VS     } {RESIDUAL}
                             {WR      }
                             {n       }
            [{factor [(n)] } ... ] 
             {POOL(varlist)}  
            [MWITHIN factor(n)]
            [MUPLUS]
            [CONSTANT [=n] ]
  
  
  * WSDESIGN uses the same specification as DESIGN, with only within-subjects factors.
  † DEVIATION is the default for between-subjects factors, while POLYNOMIAL is the default for within-subjects factors.
  ** Default if the subcommand or keyword is omitted. 
  This command reads the active dataset and causes execution of any pending commands. See Command Order for more information. 
  
  Example 1
  * Analysis of Variance 
  MANOVA RESULT BY TREATMNT(1,4) GROUP(1,2).
  
  Example 2 
  * Analysis of Covariance 
  MANOVA RESULT BY TREATMNT(1,4) GROUP(1,2) WITH RAINFALL.
  
  Example 3 
  * Repeated Measures Analysis 
  MANOVA SCORE1 TO SCORE4 BY CLASS(1,2)
    /WSFACTORS=MONTH(4).
  
  Example 4 
  * Parallelism Test with Crossed Factors 
  MANOVA YIELD BY PLOT(1,4) TYPEFERT(1,3) WITH FERT
    /ANALYSIS YIELD
    /DESIGN FERT, PLOT, TYPEFERT, PLOT BY TYPEFERT,
     FERT BY PLOT + FERT BY TYPEFERT
     + FERT BY PLOT BY TYPEFERT.
  

• Why can't I just use multiple univariate ANOVA tests rather than MANOVA, in my repeated measures design?
  Multiple univariate ANOVA runs are not equivalent to the multivariate analysis done by a single MANOVA and will have less power (more chance of type II errors, which are false negatives, thinking there is no relationship when there is). In repeated measures designs, when contrasts of group means are correlated with each other, multiple univariate ANOVA tests may fail to reject the null hypotheses of no group differences. This is because ANOVA tests differences in means under assumptions of independence, whereas MANOVA is sensitive not only to mean differences but also does not require the assumption of orthogonality (lack of correlations) among the contrasts. Correlation of contrasts is tested by Mauchly's sphericity test, which, if significant, indicates a violation of the sphericity model (a model which implies uncorrelated contrasts) and thus requires the researcher to employ MANOVA rather than a series of univariate ANOVAs. In general, MANOVA is used to test group differences when more than two levels are involved in a repeated measures design because, unlike multiple univariate ANOVA, MANOVA does not require the assumption that contrasts be uncorrelated. When multiple univariate ANOVA test results differ from MANOVA results on the same data, this indicates that the contrasts between the levels of the repeated measures factors are correlated across subjects.

• How do I write up the results of my MANOVA analysis?
  The answer, of course, varies with the data. Consider a simple MANOVA in which a treatment group is exposed to a stimulus and there is a partial treatment group and a control group. Treatment could be exposure to a movie about recycling. Partial treatment could be exposure to a pamphet with the same information about recycling. The control group would receive no information. Thus there is a factor labeled Group which is the independent variable. Let the dependent variables be measures post-treatment perceptions of how likely the subject is on a scale from 1 to 10 (treated as an interval measure, or objective data might be obtained from actual behavior) to recycle Paper, Cans, and Plastics.

  The write-up would first state the results of the overall test of inter-group differences: "The Wilks Lambda multivariate test of overall differences among groups was statistically significant (p=0.032)." Depending on the publication, the parenthetical term might contain the F parameters and value (ex., "F(4,90) = 6.28"). Optionally, one might also state "The F statistic for Wilks' Lambda was exact.", which is reported under the relevant SPSS table. If there had been more than one factor, then the overall test is reported similarly for each factor. Also, one should always report effect size as well as significance. Before running the analysis, under the Options button, check "Estimates of effect size," which will cause partial eta-squared to be reported. State, "Although significant, the effect size of this relationship was weak as indicated by partial eta-squared = ..11).

  The "Tests of Between-Subjects Effects" table following the overall multivariate tests gives significance levels and partial eta-squared for each dependent. The write-up would state something like, "Univariate between-subjects tests showed that media exposure was significantly and moderately related to recycling paper (p=.0001; partial eta-squared = .21) and cans (p=.041; partial eta-squared = .10), but not to plastics (p=.421; partial eta-squared = .01)."

  Since the overall test when significant only shows at least one group is significantly different from another, the next research statement in the write-up must report post-hoc contrast tests among groups. For instance, in the table of "Multiple Comparisons" SPSS prints the mean difference on the dependent variable between any two groups and its corresponding significance. After using this test, the researcher might state, "Post-hoc comparisons between groups using F statistics and Bonferroni-type simultaneous confidence intervals based on Student's t distribution also showed that the movie treatment group was significantly related to the recycling of paper (p=0.002) and cans (p=0.01) but not plastics (p=.055)." A similar statement would be made about the pamphlet group. The control group, which is the omitted category, would not have a corresponding statement. There are no partial eta-squareds to report. Optionally, one might list F parameters and values.

• How many dependents can I have in MANOVA?
  There is no theoretical limit, but keep in mind that as one increases the number of dependents there is a decline in interpretability, the likelihood of error based interactions increases, and there is a loss of power (that is, increased likelihood of Type II errors - thinking you don't have something when you do).

• Explain the syntax for MANOVA in SPSS.
  Note: This has largely been replaced by GLM syntax.
  MANOVA opinion1 opinion2 opinion3 BY EducLevel (1, 2) SESlevel (1, 3)
  / WSFACTORS = Ideology (3)
  / WSDESIGN = Ideology
  / PRINT = CELLINFO (MEANS)
  / DESIGN.

  Line 1: The MANOVA command word is followed by the three variables opinion1, opinion2, and opinion3. These represent the three levels of the within-subjects factor Ideology. The BY keyword tells SPSS that what follows are the groups or between-subjects factors; in this case, EducLevel and SESlevel. Following each of the two between-subjects factors are two numbers between parentheses. SESlevel (1,3) simply means that the variable SESlevel has three levels coded as 1, 2, and 3. One may have no grouping variable and thus no BY clause.
  Line 2: The slash mark indicates a subcommand. The WSFACTORS subcommand, tells SPSS that there is one repeated factor called Ideology and that it has three levels (matching the three opinion measurements listed after the MANOVA keyword). This is needed by SPSS to interpret the list of dependent variables in line 1. The WSFACTORS subcommand follows the MANOVA command when there is a within-subjects factor, which is to say when there is a repeated measures design.
  Line 3: The WSDESIGN subcommand tells SPSS to test the within-subjects hypotheses for repeated measures designs.
  Line 4: The PRINT subcommand specifies the output. CELLINFO (MEANS) prints cell means and standard deviations used to evaluate patterns in the data. Many additional statistics could be requested.
  Line 5: The DESIGN subcommand causes SPSS to test the between-subjects hypotheses.

  The general MANOVA syntax, from the SPSS manual, is:

  MANOVA
    depvarlist  [BY indvarlist (min,max) [indvarlist (min, max) ...]
    [WITH covarlist]]
    [/WSFACTORS = name (levels) name...]
  
    [/{PRINT | NOPRINT} = [CELLINFO [(MEANS SSCP COV COR ALL)]]
      [HOMOGENEITY [(BARTLETT COCHRAN BOXM ALL)]]
      [SIGNIF (MULTIV UNIV AVERF AVONLY EFSIZE ALL)]]
  
    [/OMEANS [VARIABLES(varlist)] [TABLES ([CONSTANT] [factor BY factor])]]
  
    [/CONTRAST(factor) = {POLYNOMIAL[(#)] | SPECIAL(k1s + contrasts)}]
    [/CONTR ...}
  
    [/WSDESIGN = effect ...]    [/DESIGN = effect ...]
  
  Notes on Effects
    Keywords: BY, W or WITHIN, MWITHIN
    Varname(#): # = one of k-1 contrasts or one of k levels
  
  

  Click here for syntax for other MANOVA commands for various designs.

• What is analysis of residuals for in MANOVA?
  A plot of standardized residual values against values expected by the MANOVA model tests the assumption of MANOVA that residuals are randomly distributed. If there are any observable systematic patterns, the model is questionable even if upheld by significance testing.

• Is there a limit on the number of covariates which can be included in an multiple analysis of variance?
  Not really. Whereas SPSS limits you to 10 in ANOVA, the limit is 200 in MANOVA -- more than appropriate for nearly all research situations. As one adds covariates, the likelihood of collinearity of the variables increases, adding little to the percent of variance explained (R-squared) and making interpretation of the standard errors of the individual covariates difficult.

• What is step-down MANOVA?
  Step-down MANOVA, also called the Roy-Bargman Stepdown F test, is a conservative approach to significance testing of the main effects, designed to prevent the inflation of Type I errors (thinking you have something when you do not). Stepdown tests perform a univariate analysis of the significance of the first dependent, then test the second dependent controlling for the first as a covariate, and so on, sequentially rotating the dependents to the status of covariates. This process continues until one encounters a significant F for a variable, leading the researcher to conclude it is significantly related to the independent (classification) variables over and above the "covaried out" previously-tested dependent variables. Step-down MANOVA is recommended only when there is an à priori theoretical basis for ordering the dependent (criterion) variables. The order of the variables is critical to the results obtained by this method. In SPSS syntax mode, specify PRINT SIG(STEP) to obtain the stepdown F test. As of SPSS Version 10, it was not available in menu mode but using GLM one could obtain identical results by conducting multiple runs, each time with a different dependent as covariate.

  In SPSS, make sure that the dependents are entered in the desired order, then in the MANOVA syntax, enter PRINT SIGNIF(STEPDOWN) or simply PRINT SIG(STEP). For example:

  manova
     var1 var2 var3 BY gender(1,2)
     /print signif(stepdown)
    ...
  

  See James Stevens, Applied Multivariate Analysis for the Social Sciences, 2nd Edition.

• What is the "protected F" or least significant difference (LSD) test in MANOVA? How does it relate to the use of discriminant analysis in MANCOVA?
  In the second step on MANOVA, when one tests for specific group differences, having established the overall difference among groups using the F test, today the most common method relies on multiple discriminant analysis (MDA) and associated significance tests (ex., Wilks, Hotelling, Pillai-Bartlett, etc.) as discussed above. However, the earlier method, of following a significant MANOVA with a series of ANOVAs on each of the dependent variables, is still used. This traditional method is called the protected F test, the protected t test, or the least significant difference test. Using multiple univariate ANOVA tests at a nominal alpha significance level (ex., .05) is misleading -- the actual significance level will be much higher (that is, > .05), affording less protection against Type I errors (thinking you have something when you don't) than the researcher may be assuming. For this reason, the protected F or LSD method is no longer recommended.

  Using discriminant analysis, the MANCOVA dependents are used as predictor variables to classify a factor (treatment) variable, and the discriminant beta weights are used to assess the relative strength of relation of the MANOVA dependents to the factor. The beta weights indicate the strength of relation of a given dependent controlling for all other dependents

Bibliography

• Bray, James H. and Scott E. Maxwell (1985). Multivariate analysis of variance. Quantitative applications in the social sciences series #54. Thousand Oaks, CA: Sage Publications.
• Gill, Jeff (2001). Generalized Linear Models: A Unified Approach. Thousand Oaks, CA: Sage Publications. Seires: Quantitative Applications in the Social Sciences, No. 134. A mathematical overview of GLM.
• Hand, D.J. and C. C. Taylor (1987). Multivariate analysis of variance and repeated measures. London: Chapman and Hall.
• Huyn, H. and G. K. Mandeville (1979). Validity conditions in a repeated measures design. Psychological Bulletin, Vol. 86: 964-973.
• Nelder, J. A. and R. W. M. Wedderburn (1972). Generalized liner models. Journal of the royal Statistical Society, A, 135: 370-384. This is the seminal article on GLM.
• Olson, C. L. (1976). On choosing a test statistic in multivariate analyses of variance. Psychological Bulletin, Vol. 83: 579-586. Olson's tests showed Pillai-Bartlett trace to be more robust than Wilks's U.