Linear Mixed Models

Overview Linear mixed models (LMM) handle data where observations are not independent. That is, LMM correctly models correlated errors, whereas procedures in the general linear model family (GLM) usually do not. (GLM includes such procedures as t-tests, analysis of variance, correlation, regression, and factor analysis, to name a few.) LMM is a further generalization of GLM to better support analysis of a continuous dependent for: random effects: where the set of values of a categorical predictor variable are seen not as the complete set but rather as a random sample of all values (ex., the variable "product" has values representing only 5 of a possible 42 brands). Through random effects models, the researcher can make inferences over a wider population in LMM than possible with GLM. hierarchical effects: where predictor variables are measured at more than one level (ex., reading achievement scores at the student level and teacher-student ratios at the school level). repeated measures: where observations are correlated rather than independent (ex., before-after studies, time series data, matched-pairs designs) It is true that GLM has been adapted to handle these models also, but problematically so and therefore LMM is preferred. For instance, GLM in SPSS does support random effects but estimates their parameters as if they were fixed, calculating variance components based on expected mean squares; LMM, in contrast, uses maximum likelihood estimation to estimate these parameters. GLM in SPSS supports repeated measures but the LMM module supports more variations and data options. Hierarchical models in SPSS require LMM implementation. Linear mixed models include a variety of multi-level modeling (MLM) approaches, including hierarchical linear models, random coefficients models (RC), and covariance components models. Differences between LMM and GLM are discussed further in the FAQ section. Note that multi-level mixed models are based on a multi-level theory which specifies expected direct effects of variables on each other within any one level, and which specifies cross-level interaction effects between variables located at different levels. That is, the researcher must postulate mediating mechanisms which cause variables at one level to influence variables at another level (ex., school-level funding may positively affect individual-level student performance by way of recruiting superior teachers, made possible by superior financial incentives). Multi-level modeling tests multi-level theories statistically, simultaneously modeling variables at different levels without necessary recourse to aggregation or disaggregation. It should be noted, though, that in practice some variables may represent aggregated scores. In SPSS, select Analyze, Mixed Models, Linear; if there are repeated measures, enter the repeated variables and the subject variable; click the Fixed Effects button and fill out that dialog; click the Random Effects button and do likewise; click the Statistics button to select output; click OK. (There are also other options). See also variance components analysis (VARCOMP). Note that the linear mixed models procedure includes all VARCOMP models, but output options differ somewhat.

Contents Key concepts and terms Random effects models Basic mixed models Hierarchical linear and multilevel models Random coefficients models Repeated measures models Assumptions FAQ Bibliography

Key Concepts and Terms

• Data
  • Hierarchical data involve measurement at multiple levels such as individual and group as, for example, a study of certain variables studied in terms of individual students' opinions, their classes, and their schools. In fact, much early work on multi-level modeling focused on educational settings. In general, hierarchical data are obtained by measurement of units grouped at different levels, such as a study of children nested within families; employees nested within agencies; soldiers nested within platoons, divisions, and armies; or subjects nested within studies.
    • Groups. Individuals are clustered within groups. In two-level mixed models, the base layer (level 1) is individuals (ex., students) who are clustered within the groups formed by the upper level layer (ex., level 2 = schools).

    • Multistage sampling. Hierarchical data are normally obtained by multistage sampling. For instance, one might sample schools within school districts, then sample students within sampled schools.

  • Centering. It is customary to center data prior to running LMM or HLM. Centering means subtracting the mean, so means become zero. Two main types of centering are group mean centering and grand mean centering. For instance, in a study of PerformanceScore, there might be a PerformanceIndividualScore at level 1 and a PerformanceAgencyScore at level 2, where the latter was a mean score for all employees in an agency. The researcher might center PerformanceIndividualScore for individuals by centering on their group means, where groups were agencies. on the theory that group performance influenced individual performance. As noted by Kreft, de Leeuw, and Aiken (1995), the choice of centering must be made on a theoretical rather than statistical basis, and "centering around the group mean amounts to fitting a different model from that obtained by centering around the grand mean or by using raw scores" (p. 1). Most LMM/HLM software packages support various types of automatic centering. Centering considerations are further discussed in Burton (1993) and Hoffman & Gavin (1998).

  • Data formats. Usually data are in 'multiple variable' (MV) format, with an individual ID variable, a nesting group ID variables, and a series of other variables, with one row per record (case). A 'multiple record' (MR) format has one row per variable. Each row has the individual ID variabl;e, the nesting group (level) variables, and one other variable of interest. Any given individual may be represented by as many rows as there are variables of interest. Repeated measures data often are in MR format.

• Variables
  • Subject variables. The initial "Linear Mixed Models: Specify Subjects and Repeated" dialog screen allows the researcher specify one or more subject variables. If there are repeated measures or random effects a subject variable is usually entered as it is used when defining subjects for the random-effects covariance structure. In a repeated measures study, SubjectID might be the subject variable. One, some, or all the factor variables may be used as subject variables as long as the groups formed by the subject factor(s) are independent - that is, measurements within each group do not help predict measurements in another group. For instance, if subject variables are Gender and JobStatus, then the groups might be Male-Working, Male-NotWorking, Female-Working, and Female-NotWorking. The researcher would be hypothesizing that these two subject variables explained variance in whatever dependent variable was measured. Observations would be similiar within groups but each group is assumed independent of the others.

    In summary, a subject variable is one which is used to define groups such that each is independent of the others. Each subject has a different set of random parameters for the random effect variable(s). Note that if an observation has a missing value on any of the subject variables, it will be dropped from analysis.

  • Repeated measures variables. This variable or variables constitute the id for the repeated observations, if there are to be any. For instance, one might have a single repeated measure "Year" to indicate each of four years in which a measurement was taken. Alternatively, there could be two repeated measures variables, Month and Day, which together would constitute the id for the repeated measurements. This, too, is specified on the initial "Linear Mixed Models: Specify Subjects and Repeated" dialog screen in SPSS.

  • Fixed factors are categorical variables where all possible category values (levels) are measured, even if one of the values is "other." (Ex., religion = Protestant, Catholic, Jewish, Other). Fixed factors may be the primary variables of interest in a research study. Fixed factors ave different, varying intercepts for each group, but the regression slope is the same for each group

  • Random factors are categorical variables where only a random sample of possible category values are measured. (ex., city = Philadelphia, Miami, Denver, Detroit, Los Angeles). Random factors often reflect the sampling and data collection design of a research study. In a simple one-level individual sample of soldiers, soldiers are a random factor representing a soldier effect. In a multilevel study random factors may be grouping factors (ex., individual-level voting data grouped by city, so that voter attributes are level 1 fixed factors and city is a level 2 random factor). It is important to note, however, that a given effect may be modeled as a random or as a fixed factor. For instance, SES (socioeconomic status) in a study of employee performance scores grouped by agency might be modeled as a random covariate if it is thought its regression coefficient varied randomly by agency, but if the regression coefficient is assumed to be constant across agencies, SES might be modeled as a fixed factor. Designating a level 1 (ex., employee) variable as a random factor means the researcher assumes its coefficient varies randomly across level 2 (ex., agency) groups.

  • Covariates are continuous predictor variables. (ex., income). SPSS syntax is dependentvariable BY factors WITH covariates. Factors and covariates are entered in the main "Linear Mixed Models" dialog box in SPSS, then either the Fixed or Random buttons or both are clicked to specify how factors and covariates are to be modeled.

  • Residual weight variable (REGWGT). Optionally the researcher can specify the name of a variable containing the regression weights. These are applied (and only applied) to the residual covariance matrix. Residual weights are used for models with unequal variances between groups formed by the subject variable(s) and are akin to weighted least squares analysis in regression models.

  • Moderator variables in a multi-level model are the level 2 or higher level independent variables. Levels are modeled as random factors. For instance, school budget at level 2 may moderate the effect of socio-economic status (SES) on test performance at the base (student) level. The theory, presumably, would be that school budget compensates for some of the educational resources implicit in SES at the individual level. The regression coefficients connecting the moderator variables at level 2 to the regression slopes and intercepts at the individual level are assumed not to vary across groups (schools) and hence these are fixed coefficients, in contrast to the random coefficients at the base level.

  • Cross-level interactions. Multi-level modeling will identify significant cross-level interactions (the joint effect of a variable at a base level in conjunction with a variable at an upper level). When such an interaction is present, the estimated coefficient of the direct variable estimates the effect of that variable when the other variable in the interaction is controlled (is zero). Sometimes, of course, a zero value is impossible for that variable because it is outside the permissible range. Interpretation may be improved in such cases by centering the variable: subtract the mean value from all cases for that variable. Interactions can also be visualized by plotting separate scatterplots of one of the interacting variables with the dependent, getting a separate plot for each value of the other interacting variable.

  
  

• Covariance Structure Type

  • What covariance structure type is. By specifying the covariance structure type, the researcher is telling the MIXED algorithm the shape (form) of the variance covariance matrix formed by the random parameters within one subject. Each subject has a different set of random parameters for the random effect(s). For instance, a diagonal structure means that in a matrix in which the rows and columns are the elements (the different categories of the subject variable), the diagonal entries are the different variances for the different subject groups and the off-diagonal entries are zeros, to reflect an assumption of zero correlation between elements.

  • When to specify the covariance structure type? If there are random effects or repeated measures, the researcher specifies the covariance structure type.

  • Why specify the covariance structure type?. The covariance structure type specified by the researcher is used as a starting point in the iterative REML or ML algorithms for estimating parameters. Either or both of two matrices may be specified. The structure of the matrix for the covariance of error for repeated effects can be specified in the initial LMM "Specify Subjects and Repeated" dialog (this specifies what is called the repeated covariance type "R" matrix). The structure of the matrix for covariance of random effects may be specified under the Random button (this specifies what is called the random effects covariance type "G" matrix). The covariance type specifications specify the researcher's assumptions about the repeated measures error covariance and the random effects covariance structures respectively. Different assumptions will affect the calculation of estimates. Assuming a too-simple covariance structure will increase Type I errors, while assuming a too-complex structure will increase Type II errors.

  • Repeated measures covariance structure types. On the initial "Linear Mixed Models: Specify Subjects and Repeated" dialog screen, if there are repeated measures, one specifies the "Repeated Covariance type," which is the researcher's estimate of the type of covariance structure for the residuals.

  • Random effects covariance structure types. Whether or not there are repeated measures, if there are random effects, after clicking the "Random" button and specifying the random effects model, the same dialog screen is where the researcher enters the "Covariance type" for the random effects model. The groups formed by the subject variable(s) are assumed to be independent and are assumed to each have the same covariance structure, so only one covariance type is specified by the researcher and only one estimated by LMM algorithms, regardless of how many groups there are. The available covariance types are the same as for repeated measures covariance types, except there is an additional type - Variance Components - which is the default for random effects models. "Variance Components" structure means that the variances of the random effects are assumed to be independent and sum to the variance of the dependent variable. The covariance structure specified for random effects need not be the same as specified for repeated measures.
    • Multiple random effects. Ordinarily there will be just one random effects model and just one covariance type specified for it. However, under the Random button in SPSS, it is possible to have multiple random effects models, each with its own specified covariance structure, which may be the same or different from the others. If there are multiple random effects models, they are assumed to be independent of one another (terms within the model for any random effect may be correlated, however). If the same random effects variable is listed in more than one random effects model, it must refer to a different combination of Subject variables. When there are multiple random effects models, a separate covariance structure is estimated for each.

  • Selecting the best covariance structure assumption. Goodness of fit statistics (-2LL, AIC, AICC, BIC - discussed in the section on structural equation modeling, for instance) can be used to select the best covariance structure type to assume. Simulation studies by Ehlers (2004) suggest BIC is preferred generally, but AICC is better for sample sizes < 20 and when there is only one group. Usually, however, this is a moot question because the criteria will agree. The researcher runs the model under the desired covariance structure assumptions (ex., under variance components, unstructured, and AR(1), or others listed below), then looks in the "Information Criteria" table in the output to take note of the BIC, AICC, or other fit statistic for each run of the model. The model with the lowest value on BIC (or other chosen fit criterion) is the one with the best-fitting covariance structure assumption. Note that if one or more fixed effects is dropped from the model, BIC (or other fit criteria) must be reestimated and reevaluated.

  • Examples of types of covariance structure. For the following types of covariance structure, the figures reflect the example, discussed below in the section on basic mixed models, of city as a random effect in a city tax study of the relation of appraisal value to selling price. Figures are the "G matrix," reflecting the City random effect covariance structure.

  • Variance Components, also called "simple structure" or "the independence model" (because residual variance is independent of effect variances) . This is the default for random effects covariance in both SPSS and SAS linear mixed models, and is only available for random effects and not repeated measures. Under this assumption, variances of the random effects are assumed to be independent and sum to the variance of the dependent variable. Thus, variance components structure assumes no within-subject correlation of error terms (ex., no subject learning or maturation effects from measurement time 1 to time 2). That is, the correlation of any pair of repeated measurements is assumed to be equal regardless of how far apart they are in time. A measure at time 1 is apt to be as similar to the measure at time 4 as it is to the measure at time 2. The variance components assumption is usual in random effects models, where an independent error variance is one of the variance components along with effect components, summing to the variance of the dependent. Variance components associates a scaled identity matrix (see below) with each random effect.
    

  • Diagonal. This is the default for repeated measures covariance but may be used in other contexts. This covariance structure has heterogenous variances (unlike variance components structure) and zero correlation between elements. That is, the researcher is assuming that different categories of the random effect variable have different variances on the dependent variable. Alternative common structures that might be assumed for repeated measures are AD(1) and AR(1)
    

  • Others. Many other types of covariance structure are possible, discussed below in the FAQ section.

  
  

• Models
  • Null model. Also called an intercept-only model or unconditional model, this model predicts the dependent based on an intercept term and error terms, with no predictor factors or covariates. Null models are mainly used to obtain baseline -2LL, used in measuring significant improvements with fixed or random models. This appears in the "Information Criteria" table of SPSS output when the researcher requests descriptive statistics under the Statistics button.
    • SPSS. For the example where the level 2 nesting factor is Agency in a study of causes of PerformanceScore, then in SPSS select Analyze, Mixed Models, Linear; in the "Linear Mixed Models: Specify Subjects and Repeated" dialog box, assign Agency to the Subjects: listbox; the Repeated: box is left blank; Continue; in the "Linear Mixed Models" dialog, assign the dependent variable (ex., PerformanceScore) to the Dependent Variable: listbox; the Factor(s):, Covariate(s):, and Residual Weights: listboxes are left blank;click Random; in the Linear Mixed Models: Random Effects dialog box, set Covariance Type to Unstructured, check Include intercept, leave Factors and Covariates: blank, and in the Subject Groups area, set Subjects: to Agency and set Combinations: to Agency; Continue; click Statistics in the Linear Mixed Models dialog; check the output statistics you wish, such as Parameter estimates and Tests for covariance parameters.; Continue; OK.

    • Interpretation. In the "Estimates of Fixed Effects" output table, The Estimate column for the Intercept row (the only row) gives the overall Agency mean PerformanceScore. Also listed is a t-test of the significance of the mean, and 95% confidence intervals. For a significant mean, zero will not be within the lower and upper confidence bounds. In the "Estimates of Covariance Parameters" table, for the row "Intercept [subject=AGENCY], the Estimate column gives the estimated variance of the intercepts between Agencies. The Estimate column for the Residual row gives the estimated variance between individuals (in this case, agency employees). Comparing the individual-level variance with the group-level variance informs the researcher about whether the variation in PerformanceScore is primarily within or between agencies. Note Singer (1998) and Singer & Willett (2003) state that the t-test of significance for the estimated variance may not be reliable.

  • Fixed effects models are models with only fixed factors and optional covariates as predictors. Most models in analysis of variance, regression, and GLM are fixed effects models, which are by far the most common type in social science. An example would be a study of job satisfaction by gender, controlling for salary level. Job satisfaction would be the dependent variable, gender the fixed factor, and salary the covariate, treated as fixed. Both GLM and LMM can model fixed effects models with very similar if not identical estimates and similar but not identical output tables. However, where LMM is clearly superior to GLM lies in handling other types of models discussed below.

  • Random effects models are models with only one or more random factors and optional covariates as predictors. If there are covariates, they are treated as fixed effect variables, so the random effects model becomes a mixed model though some authors may still call it a "random effects model." An example would be a study of job satisfaction by city, controlling for salary level. Job satisfaction would be the dependent variable, city the random factor (assuming only a random sample of cities was studied), and salary the covariate.

  • Mixed models have both fixed and random factors as well as optional covariates as predictors. An example would be a study of job satisfaction by gender by city, controlling for salary level. Job satisfaction would be the dependent variable, gender the fixed factor, city the random factor, and salary the covariate.
    • Hierarchical linear models (HLM) are a type of mixed model with hierarchical data - that is, where data exist at more than one level (ex., student-level data and school-level data). HLM models focus on differences between groups (ex., schools) in explaining a dependent variable. Put another way, the focus is on the group effect on a dependent in relation to predictor covariates.
      • Unconditional models are often used as a comparison baseline. They focus on the variability in the dependent when there are no fixed effects which condition (control, moderate) the variability.

      • Conditional models include one or more fixed factors. Covariance parameter estimates are conditional on the model's fixed effects. The likelihood ratio test (model chi-square difference test) can be used to assess the difference in fit between a conditional model and the corresponding unconditional model.

  • Random coefficients models (RC), also called multi-level regression models, are a type of mixed model with hierarchical data and each group at the higher level (ex., school level) is assumed to have different regression slopes as well as different intercepts for purposes of predicting an individual-level dependent variable. RC models focus on differences within groups (ex., schools) in the slopes and intercepts for calculating a dependent variable, as shown by the covariance estimates discussed below in the section on RC models. Note that there can be differences within groups but not between groups and vice versa, so RC and HLM models can be complementary. Note many authors consider RC models a type of HLM model since RC models have the same logic but also model one or more predictor variables in terms of random slopes and intercepts.

  • Steps in multi-level modeling. Hox (1995) suggests the following procedure for multi-level analysis:
    1. Compute deviance for the baseline (null) model which includes only the intercept.
    2. Compute deviance for the model with the base level independent variables included and the variance components of the slopes constrained to zero (that is, for the fixed model).
    3. Use a chi-square difference test to see if the fixed model has a significantly better fit than the baseline model. If it does, then the researcher proceeds to investigate higher level modifier variables using random effects and/or RC models. Also, at this second step the researcher can assess the relative contribution of the base level independent variables. Drop non-significant base level independents and covariances from the model.
    4. Identify which base-level regression slopes have significant variance across upper level groups. Compute -2LL for the model with the variance components of the base-level slopes constrained to zero only for the slopes which do not have signficant variance across upper level groups.
    5. Add upper level modifier variables, determining which improve model fit. Drop modifier variables which do not improve model fit.
    6. Add cross-level interactions between explanatory modifier variables and base level independent variables that had slope variance (in step 3). Drop interactions which do not improve model fit.

  
  

• Estimation
  • Estimation algorithm. The default algorithm used to compute coefficients for the predictor variables in SPSS is "restricted maximum likelihood" (REML), but by clicking the Estimation button in SPSS the researcher may choose "maximum likelihood" (ML) instead. REML estimation has better bias characteristics (Diggle, 1988). handles high correlations more effectively, and is less sensitive to outliers than ML, but cannot be used for model comparison of fixed effects, as noted below in the section on likelihood ratio tests. ML estimations ignores the degrees of freedom used up by fixed effects in mixed models, leading to underestimation of variance components. However, ML may nonethless be preferred when comparing two models with different parameterizations of the same effect (ex., simple variable vs. quadratically transformed version of the variable), because ML is invariant to different parameterizations of a fixed effect but REML will treat different parameterizations as different models and compute different likelihood ratios. If parameterization is not an issue, however, one may use -2RLL in likelihood ratio tests of difference between models. The Estimation button also allows the researcher to set the number of iterations and various convergence options though normally the defaults suffice. LMM estimation, whether by REML or ML, contrasts with the ANOVA methods utilized in GLM.

  • Covariance type. If the model includes random factors, the covariance type must be specified. The default in SPSS is "variance components" but under the "Random" button of the Linear Mixed Models dialog, there are over a dozen alternatives. Variance components models treat the variance of the dependent variable as the sum of variances for each level of the random factor(s), giving the advantage that the percent of variance in the dependent variable attributable to a variance component (a random effect or the interaction of a random effect with a fixed effect) is computed as the parameter estimate for that component as a percentage of the total of parameter estimates for all components (including the residual component).

  • Convergence . Occasionally the maximum likelihood estimation algorithm, which is iterative, fails to converge on a solution. SPSS will print an error message, "Iteration was terminated but convergence has not been achieved." Lack of convergence generally indicates the model is badly misspecified and must be thrown out, but it may also be due to too small a sample. Misspecification is often associated with trying to estimate random (base-level) coefficients which are close to or equal to zero, which in turn leads to lack of convergence. Several remedial actions may be possible:
    1. Check for and remove any redundant variables, where the correlation approaches 1.0.
    2. Under the Estimation button, increase the maximum iterations above the default (100).
    3. Under the Estimation button, increase step-halvings above the default (5).
    4. Under the Estimation button, increase the singularity tolerance value (a drop-down list of choices is provided).
    5. Under the Estimation button, increase the number of scoring steps above the default (1).
    6. Under the Estimation button, increase the parameter convergence value to be larger than the default (.000001).

  
  

• Random Effects Models with LMM
  • Example. A random effects model might be a study of job performance evaluation scores collected for a random sample of federal agencies. Agency would be the random effects factor.

  • Why LMM?. One could approach such an example with a GLM procedure such as fixed-effects analysis of variance, testing differences in mean scores across agencies. However, fixed-effects anova assumes independent observations, whereas in fact it is possible, even likely, that performance scores are biased up or down depending on the culture of the particular agency. This can be handled by declaring agency to be a random factor in LMM.

  • SPSS. In SPSS, select Analyze, Mixed Models, Linear; Continue past the initial dialog since there are no repeated measures; in the next "Linear Mixed Models" dialog, enter score as the Dependent Variable, agency as Factor; click the Random button and set Model to agency; click the Statistics button to select output options, such as Parameter Estimates and Tests for Covariance Parameters; click OK.

  • Output.
    • Goodness of fit. Goodness of fit measures for the model appear in the "Information Criteria" table of SPSS LMM output. This table contains five goodness of fit measures: -2 Restricted Log Likelihood (assuming REML rather than ML estimation was chosen), Akaike Information Criterion (AIC), Hurvich and Tsai's Criterion (AICC, meaning AIC Corrected, for finite sample corrected AIC), Bozdogan's Criterion (CAIC, consistent AIC), and Schwarz's Bayesian Criterion (BIC). For all five measures, the lower the value the better the model. When comparing models, the model with the lower values is the best-fitting. These measures are discussed further in the section on structural equation modeling.

    • Tests of and parameters for fixed effects. The only fixed effect in a random effects model is the constant (intercept). SPSS will generate a table labeled "Tests of Fixed Effects" with one row for the intercept as the only fixed effect. The last column of this row will be the significance level. If the intercept is significant (almost always is) then we can be 95% confident the intercept is different from 0. Usually this is of no interest. If under the Statistics button we checked we wanted "Parameter estimates," then a second table of "Estimates for Fixed Effects" will appear, where the "Estimate" column will contain the actual estimated value of the intercept along with its confidence interval. If the intercept is significant, the value 0 will not be within the confidence interval.

      • Interpreting fixed effect parameter estimates. Fixed effect parameter estimates are regression slopes. The fixed effects parameter is interpreted as a regression coefficient: let the dependent be performance score and the fixed effect be years of education with a parameter estimate of 2.5, then for each year education increases performance score will increase 2.5 units. If the fixed effect is a 0, 1 dichotomy such as men=0, women=1, then a coefficient of 2.5 would mean the mean value for women is 2.5 units higher than for men on the dependent variable. The intercept is interpreted as the overall mean of the dependent variable.

    • Covariance estimates for random effects. In this example, there is one random effect factor: agency. Whereas for fixed effects there are tests of differences of means, for random effects there are tests of hypotheses related to variance of the dependent variable within categories of the random factor. The table which is output by SPSS is labeled "Estimates of Covariance Parameters" and for each random factor and for "Residual" it shows the parameter estimates, their standard errors, their significance by the Wald test, and their confidence intervals. In our example, if agency is significant we conclude that performance scores do vary by agency. This further means that a fixed-effects analysis of scores ignoring the agency effect would violate the assumption of independence of observations since observations are biased up or down depending on agency. If the value 0 appeared within the confidence limits for agency, the random factor (agency in this case) would not be significant and an analysis with just fixed effects factors (ex., seniority and/or education) might be possible.
      • Wald test. With MLE estimation, the significance of coefficients is assessed using the Wald statistic. The Wald statistic is the ratio of a coefficient to its standard error, resulting in a Z-value which is looked up in a table of the standard normal distribution to generate a corresponding probability (p) value, the normal cut-off for which is .05. The Wald test is discussed further in the section on logistic regression, including a warning about its unreliability for small samples.
      • Interpreting random effects variance components. Let the variance component estimate for the random factor Agency = 5.0. Let the variance component estimate for Residual = 15.0. Assuming there are no other variance components, and since the agency variance component then is 25% of the total of all variance components in this simple example, we would say that the agency effect accounts for 25% of the variance in performance scores. We can also say that scores cluster by agency, meaning that two individuals randomly selected from the same agency are more likely to have similar scores than a pair of randomly selected individuals representing different agencies.

    • Likelihood ratio tests for model covariance parameters for random effects. When considering whether to drop a term from the model, the likelihood ratio test is preferred over the Wald test. The model is run with and without the term in question and the difference taken between the two -2LL coefficients (from the "Information Criteria" table discussed above). This is the model chi-square difference. The degrees of freedom are 1 (assuming one term is dropped, otherwise df = number of terms dropped, which is the difference in df between the two models). The probability of a model chi-square difference this large or larger with 1 degree of freedom can be looked up in a chi-square table, or can be obtained in SPSS under Transform, Compute, then entering the fomula sig.chisq2(d, df), where d is model chi-square difference and df is the degrees of freedom. If the computer probability > .05, then the covariance parameter for the dropped term cannot be assumed to be different from 0 and the researcher proceeds with dropping the term in question.

    • Likelihood ratio tests for model fixed effects. The likelihood ratio test may also be used when considering dropping a fixed effect from the model. A computed probability > .05 means the population coefficient for the fixed factor cannot be assumed to be different from 0 and the researcher proceeds with dropping the fixed factor. Note: Whereas the likelihood ratio test for model covariance parameters may be used under either ML or REML estimation, the likelihood ratio test for fixed effects assumes ML estimation.

  • Saving new variables. Click the "Save" button in the Linear Mixed Models dialog to have SPSS save to your dataset new variables representing predicted values, standard errors, degrees of freedom, and/or residuals for each case.

  
  

• Basic Mixed Models with LMM
  • Example. A basic random effects model might be a study of city real property appraisal values ("Appraise") to see how well they predict actual selling price of homes ("Price"), for each of three cities ("City"). The dependent variable is thus Price. City is a factor, conceptualized as a random effect. Appraise is a fixed covariate.

  • SPSS. In SPSS, select Analyze, Mixed Models, Linear; Continue past the initial dialog since there are no repeated measures; in the next "Linear Mixed Models" dialog, enter Price as the Dependent Variable, City as Factor, Appraise as covariate; click the Fixed button and set Model to Appraise (so we are modeling one main fixed effect); click the Random button and set Model to City (again modeling one main random effect); click the Statistics button to select output options such as Parameter Estimates and Tests for Covariance Parameters; click OK.

  • Output.
    • Goodness of fit. Goodness of fit measures for the model appear in the "Information Criteria" table the same as discussed above in the section on random effects models. For these fit measures, lower is better. They are used when comparing models.
      

    • Tests and estimates of fixed effects.
      
      

      SPSS will generate a table labeled "Tests of Fixed Effects" with one row for the intercept and one row for Appraise, the only fixed effect in the example. The last column of this row will be the significance level. If the intercept for Appraise is significant then we can be 95% confident that Appraise has an effect on Price different from 0. If under the Statistics button we checked "Parameter estimates," then a second table of "Estimates of Fixed Effects" will appear, where the "Estimate" column will contain the estimated values for the intercept and for the fixed effect, Appraise in this example. If there had been a fixed factor then there would be estimates for the coefficients for each level of that factor except the last category, which would be redundant and set to 0). Confidence intervals are also displayed.

      For significant fixed effects, the value 0 will not be within the confidence interval. The intercept is interpreted as the overall mean of the dependent variable. Since the coefficient for Appraise is positive and significant in this example, we can conclude that appraisal value is positively related to sales price. If the model had a fixed factor, Gender, then if gender=0 corresponded to female and the coefficient was significant and positive, then females have higher performance scores than males. Since City is conceptualized as a random variable, these conclusions may be generalized to all cities, not just those in the sample.

    • Covariance estimates for random effects. In this example, City is the random effect/factor: The SPSS table "Estimates of Covariance Parameters" shows the conditional estimates for the Residual and each random effects term specified in the model under the Random button. In this example, this was only City, but in other models there might be additional main and interaction terms. In addition to the estimate, the standard error, Wald Z value, and Wald test probability level are shown.
      

      If the City effect is significant, then City affects Price (the dependent). The "City Variance" estimate is the estimate of variance in Price attributable to the random sampling of cities. Since it is not significant in this example, we conclude that City does not affect the relationship of Appraise to Price. When a random effects parameter is found to be non-significant, that effect would normally be dropped from the model and the analysis re-run. The Residual parameter is the estimate of the unexplained variance in Price after controlling for Appraisal and sampling of the random factor City.

    • Likelihood ratio tests, as described previously in the section on random effects models, using -2 Log Likelihood from the "Information Criteria" table above, may be preferred over the Wald test when considering dropping terms from the model. The computation and interpretation is the same for mixed models as for random effects models.

    • Comparison with Variance Components. Linear mixed models in SPSS has the functionality of the variance components procedure described separately. That is, VARCOMP is mostly a subset of MIXED. For the same property appraisal example discussed above for linear mixed models, one may enter Price as dependent, Appraise as a covariate, and City as a random factor, specifying REML as the method so as to be comparable, to obtain the "Variance Estimates" below, which are the same as in the "Estimates of Variance Components" table above for linear mixed modeling. However, variance components will not generate most of the additional output tables available via linear mixed models. While mostly a subset of MIXED, VARCOMP does have a few options not present in MIXED: it offers ANOVA and MINQUE estimation, not just ML and REML; and with ANOVA estimation comes sums of squares, mean squares, and expected mean squares not produced by the ML and REML estimation methods used by MIXED.
      

    • Comparison with GLM. While one may obtain similar estimates using the general linear model (GLM), they will not be identical to the variance components and linear mixed model implementations illustrated above. When the design is unbalanced, as it is in this example (there are unequal numbers of observations in Cities 1, 2, and 3), the difference may be substantial. In GLM, one may enter Price as Dependent, Appraisal as a covariate, and City as a random factor, with the model set to main effects for Appraise and City, and selecting "Parameter estimates" under the Option button. GLM will generate these tables:
      

      GLM produces Type III sums of squares for fixed effects only. Evem though City is entered as a random factor, the table above treats City as if it were a fixed effect for purposes of computing the sums of squares used to compute the F statistic. GLM estimates variance parameters for City (or any random effect) indirectly as described below, using expected mean squares. Linear mixed models and variance components analysis, in contrast, estimate variance parameters directly, using maximum likelihood (ML) or restricted maximum likelihood (REML) methods. For unbalanced designs (unequal n's in the groups formed by the random effect), as in this example, the GLM method will return estimates different from the methods used by linear mixed models or variance components analysis. Thus where in the linear mixed model run, the F value for the main effect Appraisal was 3.010E3, for the GLM run it is 2.994E3 in the table above.

      
      

      Above it is seen that the GLM method generates coefficient estimates for the fixed effect Appraise similar to that for linear mixed model. It also generates coefficients for the random effect City, which is not part of linear mixed model output due to the LMM not being based on sums of squares methods of estimation. However, the variance estimate for City, which was 38,180,000 in linear mixed modeling and in variance components analysis, is only 21,376,633 in GLM. The GLM variance estimate is computed as Var(City)=[MS(City)-MS(Error)]/EMS(City), where MS(City) = 4.215E9 and MS(Error)=6.519E8 (both from the "Between Subjects Effects" table in GLM) and EMS(City)=166.682 (from the "Expected Mean Squares" table in GLM). Even when the LMM and GLM variance estimates are the same as they will be in balanced designs, GLM has the drawback that the standard error of estimate for the variance of random factor(s) (ex., City) that appear in the "Estimates of Covariance Parameters" table in LMM, cannot be computed in GLM.

  • Mixed modeling of an interaction effect. Basic mixed models may be adapted to model the interaction effect of a random variable with a fixed variable. Take the same example of predicting actual real estate Price from appraisal values (Appraise), with City considered a random sample of all cities, but now add Agent, representing a fixed number of appraisal firms which do appraisals for all cities. Let the researcher be interested in seeing if there is a City*Agent interaction.
    • SPSS: Analyze, Mixed Models, Linear; Continue past the initial dialog; enter Price as "Dependent Variable", City and Agent as "Factors", and Appraise as "Covariate"; click Fixed and enter Appraise and Agent as main effects in the model, Continue; click Random and enter City*Agent as an interaction in the model and set "Covariance Type" to "Scaled Identity" (which is a common assumption for the shape of the covariance matrix for such interactions), Continue; click Statistics, then check "Parameter estimates", "Tests for covariance parameters", and any other output wanted, Continue; OK.

    • SPSS alternative with Subject option. Identical output can be obtained using the Subject option in SPSS: Analyze, Mixed Models, Linear; in the initial dialog, enter City as the Subject variable, Continue; enter Price as "Dependent Variable", Agent as "Factors", and Appraise as "Covariate"; click Fixed and enter Appraise and Agent as main effects in the model, Continue; click Random and enter Agent as the main effect in the model, set "Covariance Type" to "Scaled Identity", and in the "Subject Groupings" panel, set City as the "Combination", Continue; click Statistics, then check "Parameter estimates", "Tests for covariance parameters", and any other output wanted, Continue; OK.

    • Tests and estimates of fixed effects. The fixed effects tests and estimates tables below show that Price is related to the agency doing the appraisal as well as the appraisal itself. That is, the mean Price for different agencies varies significantly. In particular, agency "C" is associated with different (and higher, since the coefficient is positive) Price. That is, it appears to be assigned to appraise homes that sell for more, compared to the reference category, agency "D".
      
      

    • Estimates of covariance parameters. The table below shows that even though agency "C" is significantly related to Price, there is no significant City-by-Agent interaction. Cities are not favoring agency "C" with homes that sell at higher prices.
      

  • Saving new variables. Click the "Save" button in the Linear Mixed Models dialog to have SPSS save to your dataset new variables representing predicted values, standard errors, degrees of freedom, and/or residuals for each case.

  
  

• Hierarchical Linear Modeling (HLM) with LMM
  • Definition. Hierarchical linear modeling (HLM) deals with observations at more than one level in terms of unit of analysis. To take the classic example, levels could be student, school, school district, etc. By convention, in this example student would be level 1, school level 2, district level 3, etc. Higher-level variables may be either attributes of the higher levels (ex., school teacher-student ratio) or aggregated lower level attributes (ex., mean reading level for a school, based on level 1 student data on reading scores).

  • Example 1: A two-level model of a level 1 dependent with a level 2 covariate. A hierarchical linear model might be a study of job performance evaluation scores collected for a random sample of federal agencies, looking at femalepercent (the proportion of an agency's staff which is female) as a control variable. Score is the level 1 (individual level) dependent variable. Femalepercent is a level 2 covariate. Agency is the random effects factor which also defines the subjects which constitute level 2 (that is, the agencies).
    • SPSS for two-level models. In SPSS, select Analyze, Mixed Models, Linear; in the Specify Subjects and Repeated dialog, enter agency as the Subjects variable; in the next "Linear Mixed Models" dialog, enter score as the Dependent Variable and enter femalepercent as the Covariate; click the Fixed button and set Model to femalepercent, making sure the Include Intercept checkbox is checked; click the Random button and make agency the Subject Grouping in the Combinations area, making sure the Covariance Type is Variance Components and Include Intercept is checked; click the Statistics button to select output options such as Parameter Estimates and Tests for Covariance Parameters; click OK.

      Example notes: The level 2 grouping variable, agency, is entered as the Subjects variable (signifying individuals are independent observations within each agency) and as the Subject Grouping variable (signifying it is level 2). Note agency is not entered as a random effect factor as that is already assumed by making it a Subject Grouping variable. Score, by virtue of being Dependent, is known to be level 1. Femalepercent is identified as a covariate in the "Linear Mixed Models" dialog and is specified as a fixed factor to be modeled.

    • Information Criteria table. Goodness of fit measures for the model appear here, the same as discussed above in the section on random effects models.

    • Tests of Fixed Effects table. For the intercept and fixed effects (here, femalepercent), this table will show the significance level. If Sig.<=.05 for the effect, it is retained in the model and means performance score and femalepercent are significantly related within agency, the grouping variable.

    • Estimates of Fixed Effects table. For the intercept and fixed effects (only femalepercent in this example), this table shows the parameter estimate, its standard error, its significance level, and its confidence intervals. Interpretation of fixed effects parameters was discussed above in the section on random effects models. The estimate of the intercept is the estimate for the dependent variable, Performance score, controlling for the group level variable, Femalepercent. The estimate for Femalepercent reflects the relationship between Performance score and Femalepercent itself.

    • Estimates of Covariance Parameters table. This table will give the covariance parameter estimates for Residual and for Intercept with Subject=agency, along with a Wald test of significance. In a random effects model with agency as a random effects variable, the covariance component is unconditional: the percent this component is of the total of all component parameter estimates is interpreted as the percent of variance in performance score attributable to differences between agency. Now, however, in an HLM model, the covariance parameter estimate for the Intercept with Subject=agency is conditional, controlling for Femalepercent as a covariate. That is, its share of the total of parameter estimates in the HLM model is the percent of variance in performance scores attributable to differences between agencies after Femalepercent is controlled. Three estimates are given, with (1,1) being the estimate for variance in the intercepts, (2,2) being the estimate for variation in the slopes, and (2,1) being the estimate for the covariance between slopes and intercepts. Each has a Wald test significance level. The (1,1) intercepts variance estimate has to do with how much Performance score varies between agencies after controlling for Femalepercent. The (2,2) variation in slopes estimate has to do with how much the relationship between Performance scores and Femalepercent varies between agencies. The (2,1) estimate of covariance has to do with whether mean Performance scores after controlling for Femalepercent are correlated with the strength of the relationship of Performance scores and Femalepercent - that is, with whether agencies with high Performance score vs. those with low Performance score differ in the relationship of Performance scores and Femalepercent.

  • Example 2: A three-level model of a level 1 dependent with fixed factor and a covariate. Let level 1 = employee, level 2 = agency, level 3 = department, such that employees are nested within agencies which are nested within departments. Let performance score be the level 1 dependent, and let femalepercent be the covariate. Let prescore be an earlier performance score used as a control variable so as to focus on change in score. Let gender be a level 1 factor.

    • SPSS for three-level models. In SPSS, select Analyze, Mixed Models, Linear; in the Specify Subjects and Repeated dialog, enter department as the Subjects variable; in the next "Linear Mixed Models" dialog, enter gender and agency as a Factors, and also enter score as the dependent variable and prescore as the Covariate (if there were any, any other factors and any covariates would be entered also); click the Fixed button and set Model to gender prescore, making sure the Include Intercept checkbox is checked; click the Random button and set Model to agency and then in the Subject Grouping in the Combinations area let department be the grouping variable, also making sure the Covariance Type is Variance Components and Include Intercept is checked; click the Statistics button to select output options such as Parameter Estimates and Tests for Covariance Parameters; in the EM Means button set "Display Means for" to gender and check "Compare main effects"; click OK. Note the level 2 variable, agency, is thus listed as a Factor in the Linear Mixed Models dialog box, and as the Model variable in the Random Effects dialog; the level 3 variable, department, is listed as the Subjects variable in the initial Specify Subjects and Repeated dialog and it is also listed as the grouping variable in the Combinations area of the Random Effects dialog. For this example, this specification models the fixed effects gender and prescore has for the dependent variable (score), for agencies within departments and for departments. That is, both agencies within department and departments are modeled as random factors.

    • Information Criteria table. Goodness of fit measures for the model appear here, the same as discussed above in the section on random effects models.

    • Tests of Fixed Effects table. For the intercept and fixed effects (here, gender and prescore), this table will show the significance level. If Sig.<=.05 for the effect, it is retained in the model and means performance score and either or both of gender and prescore are significantly related to score (the dependent).

    • Estimates of Fixed Effects table. For the intercept and fixed effects (gender and prescoret in this example), this table shows the parameter estimate, its standard error, its significance level, and its confidence intervals. Because prescore is continuous, it will have a single estimate. For a categorical variable there will be one estimate per category, except the last category is redundant and is set to 0. Thus for gender, there will be an estimate for gender=0 but not for gender=1. Interpretation of fixed effects parameters was discussed above in the section on random effects models. A significant estimate means it can be assumed to be different from 0 and that effect is contributing to the model.

    • Estimates of Covariance Parameters table. This table will give the covariance parameter estimates for Residual, for Intercept with Subject=department, and agency with Subject=department, along with a Wald test of significance for each. The significance of the intercept tests the level 3 effect: if it is significant, then in this example the variablility associated with department cannot be assumed to be 0, by the Wald test. The significance of the agency estimate tests the level 2 effect (recall agency was entered as the variable to model under the Random random effects button): if it is significant, then in this example the variability associated with agencies within deparmtnets cannot be assumed to be 0, by the Wald test. Note, however, that the preferred likelihood ratio test will not necessarily come to the same conclusions..

    • EMM Estimates table. If selected, the EMM Means button will generate this and the following two tables. Recall in the example above, we clicked the EMM Means button and set "Display Means for" to gender and checked "Compare main effects." The Estimates table gives the mean, standard error, degrees of freedom, and 95% confidence level for each level of gender (male=0, female=1).

    • Pairwise Comparisons table. Also generated under the EMM Means button, this table is of greater interest. It gives the mean difference between categories, the standard error of the difference, its significance level, and the 95% confidence interval for the difference. For gender as an example, if the difference is significant, then the mean score (the dependent) difference between men and women cannot be assumed to be 0. Of course, if the comparison was done on a variable with more than two categories (ex., ethnicity), then there will be more than one pairwise comparison to assess.

    • Univariate Tests table. Also generated under the EMM Means button, this table gives and F test significance for the effect of the selected variable (here, gender) on the dependent variable, based on the linearly independent pairwise comparisons among the estimated marginal means. For gender, which has only two categories and thus only one pairwise comparison, this significance level will be the same as that reported in the Pairwise Comparisons table. However, for a variable with more than two categories, the Univariate Tests significance level tests the significance of the predictor variable overall (ex., ethnicity), whereas the Pairwise Comparisons table will report a significance level for each pair of categories (ex., Irish vs. Italian ethnicity) but no overall significance test of ethnicity.

  
  

• Random Coefficients (RC) Models
  • Definition. A random coefficients model is a type of hierarchical linear model, but one in which each different group at the higher level (ex., the agency level in our example) is assumed to have its own different slope and intercept with respect to predicting the dependent variable. It is assumed that each agency's slope and intercept is a random one from a population of all possible slopes and intercepts for the given dependent variable and predictor variable(s). RC models are among the most common types of hierarchical linear models.

  • Example. A random coefficients model might be a study of job performance evaluation scores collected for a random sample of federal agencies, looking at seniority as a predictor variable. Score is the level 1 (individual level) dependent variable. Seniority is a level 1 covariate and also a level 1 fixed effect and also a level 1 random effect. The seniority fixed effect will estimate the average relation of seniority to score. The seniority random effect will estimate the variability of slopes, intercepts, and slope-intercept interactions across he grouping variable, agencies, each of which has its own estimated slope and intercept. Agency is the random effects factor which also defines the subjects which constitute level 2 (that is, the agencies).
    • Centering. In RC models, parameter estimates are easier to interpret when the covariates are centered (made to have means of zero by subtracting the raw mean value from every observed value).

  • SPSS. In SPSS, select Analyze, Mixed Models, Linear; in the Specify Subjects and Repeated dialog, enter agency as the Subjects variable; in the next "Linear Mixed Models" dialog, enter score as the Dependent Variable and enter seniority (and if there were any, any other individual or group level covariates) as the Covariate; click the Fixed button and set Model to seniority (and if there were any, any other individual or group level effects or interactions of fixed factors), making sure the Include Intercept checkbox is checked; click the Random button. setting Model to seniority (or other group-level covariate), and make agency the Subject Grouping in the Combinations area, making sure the Covariance Type is Unstructured and Include Intercept is checked; click the Statistics button to select output options such as Parameter Estimates and Tests for Covariance Parameters and Covariances of Random Effects; click OK.

    Example notes: The level 2 grouping variable, agency, is entered as the Subjects variable (signifying individuals are independent observations within each agency) and as the Subject Grouping variable (signifying it is level 2). Note agency is not entered as a random effect factor as that is already assumed by making it a Subject Grouping variable. Score, by virtue of being Dependent, is known to be level 1. Seniority is identified as a covariate in the "Linear Mixed Models" dialog and is specified as a fixed factor to be modeled and also specified as a random factor to be modeled. Modeling the slope and intercept of seniority as a random effect is what makes this an RC model. RC models require that the covariance structure be specified, but if there is no information to do this, setting it to Unstructured is necessary. Random effects are not assumed to be independent as when Variance Components is specified as the covariance type.

  • Information Criteria table gives the -2 restricted log likelihood (a.k.a. model chi-square or deviance), plus the AIC, AICC, CAIC, and BIC goodness of fit measures as before. Model chi-square can be used to test if the given model is significant overall (that is, significantly different from the model chi-square reported for the null model).

  • Tests of Fixed Effects table gives the significance level for the intercept and for seniority, the only fixed effect in the model in this example.

  • Estimates of Fixed Effects table gives the coefficients for the intercept and for seniority, the only fixed effect, along with the standard errors, significance, and confidence intervals. The coefficient for seniority is the average slope across all agencies and is interpreted as in regression: it is the number of units the dependent (score) increases for each unit increase in seniority, on the average. Since agency is the grouping variable, if the estimate/coefficient/slope for seniority is significant, then performance scores and seniority are related within agencies. The coefficient for the intercept is the average agency mean on performance score. In a model where additional individual or group-level fixed factors and their interactions had been entered in the model unter the Fixed button, this table would show the significance of each fixed effect and each interaction term, and the researcher would drop non-significant terms from the model, re-running the RC model.

  • Estimates of Covariance Parameters table gives the covariance parameter estimates for the random effects, which in this example are the Residual and "Intercept + seniority [subject = agency]", plus the Wald statistic and its significance level. The Residual is interpreted as the variance in performance scores in an agency controlling for seniority. The magnitude of the seniority effect could be estimated by how much the residual covariance estimate was reduced compared to a simple random effects model without seniority as a covariate: since the residual is unexplained variance, if seniority has a large effect on the variability of performance scores within agencies, the residual variance should drop appreciably in the RC model compared to the random effects model without seniority. The "Intercept + seniority [subject = agency]" term has three values: (1) the estimated variability of agency intercepts, labeled UN(1,1); (2) the estimated variability of agency slopes, labeled UN(2,2); and (3), the estimated covariance of the agency slopes and intercepts, labeled UN(2,1). If the Wald test for any of these three values is significant, then the researcher concludes that compontent (intercept, slope, covariance of intercept and slope) is not 0 and that agencies indeed differ. The larger the coefficient estimate, the more the agencies differ. The more the intercept variance (the higher UN(1,1)), the more the effect of group-level variables on the dependent.

  • Random Effect Covariance Structure (G) table shows the same information as the Estimates of Covariance Parameters table but in a different format. For this example it is a 2x2 table for which the rows and columens are "Intercept|agency" and "seniority|agency". The "Intercept|agency" row by "Intercept|agency" column cell displays the variability of agency intercepts (what above was labeled UN(1,1)). The "seniority|agency" row by "seniority|agency" column displays the variability of agency slopes (what above was labeled UN(2,2)). The other two cells (the off-diagonal) both display the slope-intercept covariances, labeled UN(2,1) in the Estimates of Covariance Parameters table.
    • Is an RC model really needed?. If the variability of the intercept (labeled UN(1,1)) is low, there is little variability between schools in mean value on the dependent. Further, if the variability of the slopes and of the interaction of slopes and intercepts is non-significant (that is, UN(2,2) and UN(2,1) respectively are non-significant), this suggests that it may not be necessary to model the dependent variable for within-group effects using an RC model. This could be confirmed by using a likelihood ratio (model chi-square difference) test. Take the -2LL from the "Information Criteria" table for the RC model, then re-run the model removing the random effects model (specified in the RC model under the Random button; in this example, remove seniority from the model, leaving no random effects to model), The new -2LL value will be for a hierarchical linear model (you are still grouping by agency) of whatever fixed factors and their interactions that were specified in the fixed factors model under the Model button. To be comparable one would leave the covariance structure type as Unstructured in this example, even though the usual HLM model sets Variance Components as type. The degrees of freedom for the chi-square difference between the two -2LL's is 2 in this example (the removed slope for seniority as a random effect, and the removed covariance of this slope with the intercept). As abiove, the probability of a model chi-square difference this large or larger with 2 degrees of freedom can be looked up in a chi-square table, or can be obtained in SPSS under Transform, Compute, then entering the fomula sig.chisq2(d, df), where d is model chi-square difference and df is the degrees of freedom. If the computed probability > .05, then the HLM model is not significantly different from the RC model and on the basis of parsimony, one need not model seniority as random coefficients (but one would still use agency as the grouping variable and have whatever individual level and agency level covariates were in the analysis).

  
  

• Repeated Measures Models Using LMM
  • Repeated measures as mixed models:. The Repeated option in LMM models within-subject variance (ex., variance in the same subjects over time, as opposed to between-subject variance modeled by the Random option discussed above). Example. In LMM, one may create a model in which level 1 is within-individuals (for the variance among repeated measures for given individuals, on the average) and level 2 is between-individuals with the individuals considered a random effect. For example, employees might be measured at three times, obtaining three performance scores rather than one. There might also be covariates, such as seniority, a baseline performance score, or a time variable - all as discussed below. The time variable baseline is coded 0 and subsequent measurement times are 1, 2, 3, etc. If the time codes represent equal metric intervals, the time variable may be used as a covariate as well as fixed effect.

  • Repeated measures data setup. The data format for repeated measures LMM is one row per person per measurement time. Thus if each person is measured at three times, there are three data rows for person 1, then three rows for person 2, etc. Each data row contains an id variable for the person, a time variable (0, 1, 2 in this example), and whatever effects are being modeled. If the researcher's raw data is of the one row per person type, with repeated measures being different columns for each person, then the data must be converted to rows per person per time format. This conversion can be done with the SPSS Data Wizard, under Data, Restructure in the menu. Restructuring is illustrated in SPSS, Inc. (2005: 1-4).

  • SPSS. Select Analyze, Mixed Models, Linear and under "Specify Subjects and Repeated" enter the id variable as "Subjects"; in the "Linear Mixed Models" dialog, enter performance score as the "Dependent Variable" and make the time variable the "Factor(s)"; click the Fixed button and Model the time variable, making sure to check "Include Intercept"; click the Random button and set the "Covariance Type" to Variance Components, make the id variable the "Subject Groupings" in the Combinations area, and make sure to check "Include Intercept"; click the Statistics button and select the desired output; click OK.

    Note that employee id is thus both the Subjects variable and the grouping variable (Combinations). Time is modeled as a fixed effect. In this example employee is level 1 as Subjects variable and is level 2 as Combinations variable.

  • Covariance structure assumptions. The initial LMM (Mixed) dialog in SPSS provides a pulldown menu of choices of assumptions to make about the covariance structure of residuals in repeated measures models. The default is "Diagonal". Options are discussed above in the "Key Concepts and Terms" section.

  • Output

    • Tests for Fixed Effects table. If the F test for the time variable is significant, then performance score varies by time of measurement within the same individuals.

    • Tests of Covariance Parameters table. If "Intercept[Subject=id]" is significant by the Wald test, then performance varies between individuals. For simplicity, if the estimate for this intercept were 750 and the estimate for the Residual parameter was 250, with no other parameters, then 75% of the variance in performance score would be attributable to variability between subjects.
      • Time as a covariate. In some research studies, the time variable codes may reflect real measurements. That is, the time codes 0, 1, 2 might not be arbitrary but reflect a real scale, where 1=elapse of one month, 2=elapse of two months, etc. If this is the case, then time can be modeled as a covariate, not just a fixed effect. In the SPSS procedure above, one would enter the time variable as a Model variable under the Random button (as well as remaining a Model variable under Fixed). In this case, the estimate in the Fixed Effects table is a regression coefficient indicating the number of performance points employees change on the average for each unit increase in time; and in the Covariance Parameters table there will be a parameter for "Intercept+time[subject=id]" labeled UN(1,1) testing if the intercept of performance score (the mean value for employees) varies signficantly between individuals, another parameter labeled UN(2,2) testing if the regression slope varies signficantly between individuals, and a third parameter labeled UN(2,1) testing for significant differences in intercept-slope interaction. The Residual covariance estimate will reflect the estimated variability in performance score between times for the average employee.

      • Additional covariates. It is, of course, possible to add other covariates such as seniority to the model. In SPSS this is done simply by adding seniority to the "Covariate(s)" list in the "Linear Mixed Models" dialog; and adding seniority to the Model list under the Fixed button. The fixed effects Model list optionally could also include interactions of covariates (ex., time*seniority, assuming time is a covariate also). Additional covariates serve as control variables. For instance, if a baseline performance score, prescore, were added as a covariate fixed effect, then one would be modeling change in performance score (score controlling for prescore).

  • Example: A three-level model with repeated measures. Consider a situation in which Verbal is a test of verbal ability given in each of three years (Schlyear is the repeated measure), to students (Student is the student id variable) who are grouped in classes (Class is the class id variable) which in turn are grouped in schools (School is the school id variable). Let the researcher be interested in the effects of gender (Sex is the variable) and socioeconomic status (SES is the variable).
    • Subject and repeated variables. In this example, three variables define the subject (Student, Class, School). Student is not enough since student id's are assumed unique to each class, and class id's assumed unique to each school. Therefore all three variables are entered in the Subjects area of the initial Linear Mixed Models: Specify Subjects and Repeated dialog in SPSS, as illustrated below. Schlyear is the repeated measure (not Verbal, which is the dependent). The covariance structure type for Schlyear is set to AR(1), which is often chosen when there is thought to be a common trend, such as increasing scores over time. The effect of this initial dialog page is to model Schlyear by student (Student, Class, and School are all needed to specify the student subject), testing if residual errors on estimates of the Verbal score (the dependent, to be named on the next dialog page) are correlated within each student. That is, if the repeated measures effect for Schlyear is significant, as shown later in the Estimates of Covariance Parameters table in SPSS output, then for a typical student, knowing error of estimate of Verbal in one year helps in predicting error or estimate in another year.
      

    • Dependent and factor variables. In the main Linear Mixed Models dialog, illustrated below, Verbal is entered as the dependent variable and the factors entered are the three subject variables (Student, Class, School), the repeated measure (Schlyear), and the causal factors of interest (SES, Sex). Including Schlyear as a fixed factor controls for trends in Verbal over the three years measured (that is, in effect the parameters for other fixed effects will be for detrended data).
      

    • Modeling the fixed factors. Clicking the Fixed button is necessary to specify exactly how the fixed factors are to be modeled (main effects, two-way effects, factorial, etc.). As illustrated below, in this example only main effects are modeled. Type III tests are specfied, which is usual in this and most statistical procedures.
      

    • Modeling random effects. Similarly, clicking the Random button allows the researcher to specify random effects to be modeled. The researcher can specify multiple random effects (in syntax, multiple RANDOM statements) by clicking the Next button and filling in the dialog. Each random effect created this way is assumed to be independent from the others. Here there are two independent random effects, with the second one illustrated below. Note, however, that multiple nonindependent random effects could be specified by entering additional variables in the Model box for a single random effect statement (not the case here, which for each of the two random effects specified only SES, once for class and once for student).
      

      The random effect above models SES as a random effect within students. That is, Estimates of Fixed Effects table in SPSS will later show if SES seems to be related to Verbal scores. In contrast, the random effect of SES, shown in the Estimates of Covariance Parameters table, assesses if a student effect due to sampling of students conceived to be at random from a larger population of students, significantly adjusts the variation in SES. A second random effect, not illustrated, does the same thing for Class: it tests if the variation in SES is significantly due to sampling of classes from a random sample of classes. Because the two random effects are modeled separately, the researcher is assuming the sampling of students is uncorrelated with sampling of classes.

    • Output. Most statistical output is specified under the Statistics button from the Linear Mixed Models dialog, as illustrated below.
      
      • Tests and estimates of fixed effects tables. In the example output below, the Type III Tests of Fixed Effects table shows that SES, School, and Schlyear all had a significant effect on Verbal score, but not Sex. That is, at least one category of the significant variables was related to Verbal. The Estimates of Fixed Effects table spells this out by category. In this example, School, Schlyear, and Sex all had three levels (categories). School level 1 and SES level 1 were significantly more related to Verbal than other levels. Schlyear 1 compared to 3 and 2 compared to 3 were both significantly related to Verbal score. The last level in each variable serves as the reference category.
        

      • Estimates of covariance parameters table. This table shows the significance of the repeated and random effects in the model. The nonsignificant AR1 rho means that residual error is not correlated by Schlyear and therefore a scaled-identity (ID) covariance structure may be assumed. One random effect is nonsignificant and the other is redundant, so these may be dropped from the model.
        

      • Information criteria table. Using AICC, BIC, or one of the other information theory measures of goodness of fit is useful in comparing models. The lower the measure, the better the model fit. For instance, one might use these measures to assess improvement in fit due to dropping a non-significant random effect from the model. In the illustration below, Model 1 is the model described above and Model 2 is the model with the covariance structure for repeated measures specified as scaled identity rather than AR(1), and with no random effects specified. CAIC and BIC indicate Model 2 to be the better fitting model but AIC and AICC do not. The researcher might conclude that the two models were largely equivalent, but might note that between the two models, Model 2 is the more parsimonious.
        

    
    

Assumptions

• Random grouping. For levels above the bottom level of individuals as units of analysis, the groups (ex., schools, where students are the bottom level) are assumed to be a random sample of all such groups (ex., all schools) in random coefficients models. This is a critical assumption in multilevel modeling. I

• Independent observations are not assumed, which is why multi-level modeling is recommended when intraclass correlation exists. That is, OLS regression and GLM in general assume error terms are independent and have equal error variances, whereas when one has hierarchical data, individual-level observations from the same upper level group will not be independent but rather will be more similar due to such factors as shared group history and group selection processes. While random effects (such as upper-level effects, which LMM models as random effects) do not affect lower-level population means they do affect the covariance structure of the data and, indeed, adjusting for this is a central point of LMM models and why they are used instead of GLM, which assumes independence.
  • Intraclass correlation is a measure of the extent to which observations are not independent of a grouping variable (ex., schools). The presence of a significant intraclass correlation is an indicator of the need to employ multi-level modeling rather than conventional regression. To pursue OLS regression modeling anyway in the face of lack of independence and lack of homoscedastic error variance will mean that significance tests will not be accurate. Put another way, OLS significance tests (and standard errors and confidence limits) are not at all robust when the assumption of independence is violated.

• Independent blocks. While observations are not assumed to be independent, the groups (blocks) formed by the subject variable(s) are assumed to be independent and to have the same covariance structures, for models that involve random effects or repeated measures.

• Properly specified block structures. For random effects and repeated measures, the researcher must specify the assumed covariance structure. Changing the specified covariance structure will change the covariance parameter estimates and tests of fixed effects.

• Adequate sample size. Hox (1995) suggests that for MLM regression models, the higher level sample size be at least 20, preferably 50, and if variance components are important, preferably 100. For structural equation modeling approaches, Hox recommended sample size should be at least 50, preferably 100. More recently Maas & Hox (2005: 90), based on simulation studies, concluded, "The standard errors of the second-level variances are estimated too small when the number of groups is substantially lower than 100. With 30 groups, the standard errors are estimated about 15% too small..." In view of this, Maas & Hox (2005: 91-92) suggest that with small samples (as small as 10), that bootstrapping be used to estimate standard errors. However, bootstrapping was only available for multilevel modeling in MLwIN software, at least as of the time of writing.

  The efficiency and power of multi-level tests rests on pooled data across the units comprising two or more levels, which implies large datasets. The REML and ML estimation methods used by LMM give asymptotically efficient estimates, meaning efficiency depends on large samples.

  For instance, simulation studies by Kreft (1996) found there was adequate statistical power with 30 groups of 30 observations each; 60 groups with 25 observations each; 150 groups with 5 observations each. The number of groups has more effect on statistical power than the number of observations, though both are important. There is a rapid fall-off in statistical power as the number of groups/observations falls below the threshhold needed. With less than adequate power there is an unacceptable risk of not detecting cross-level interactions (ex., between schools and students). However, both adequate number of individual observations and adequate number of groups are needed. Power for individual-level estimates depends on number of individuals observed, and power for second level estimates depends on number of groups.

  Specifically with regard to MSEM, Hox & Maas (2001) used simulation studies to show for small group-level sample sizes, coefficient estimates were not stable. They recommended group-level samples of at least 100. However, Cheung & Au (2005: 612) used resampling to test sample size effects and found sample size "can be as small as 50, yet the results are still comparable with other larger sample size conditions." Unbalanced individual-level samples within groups may require larger group samples. Cheung & Au's experiments also disconfirmed the assertion of some that larger individual-level samples could compensate for small group-level samples.

• Similar group sizes. The REML and ML estimation methods used by LMM give asymptotically efficient estimates for unbalanced as well as balanced designs (whereas the ANOVA methods in GLM are optimum only for balanced designs). Nonetheless, when sample sizes within groups are unbalanced, tests of parameter estimates and of overall fit will have inflated Type I error (Hox & Maas, 2001). However, FIML estimators are more robust for unbalanced designs (du Toit and du Toit, 2005).

• Normal distribution. RC models assume a normal distribution for purposes of empirical Bayes maximum likelihood estimation. However, REML and ML estimates may be assumed to display asymptotic normality for large samples. Also, extensions have been developed for non-normal data (Wong and Mason, 1985; Goldstein, 1991; Morris, 1995).

Frequently Asked Questions

• What is the SPSS syntax for linear mixed modeling (MIXED)?

  MIXED dependent varname [BY factor list] [WITH covariate list] 
  [/CRITERIA = [CIN({95** })] [HCONVERGE({0**  } {ABSOLUTE**})
                    {value}              {value} {RELATIVE  }
               [LCONVERGE({0**  } {ABSOLUTE**})] [MXITER({100**})]
                          {value} {RELATIVE  }           {n    }
               [MXSTEP({5**})] [PCONVERGE({1E-6**},{ABSOLUTE**})] [SCORING({1**})]
                       {n  }              {value } {RELATIVE  }            {n  }
               [SINGULAR({1E-12**})] ]
                         {value  } 
  [/EMMEANS = TABLES ({OVERALL          })]
                      {factor           }
                      {factor*factor ...}
              [WITH (covariate=value [covariate = value ...])
              [COMPARE [({factor})] [REFCAT({value})] [ADJ({LSD**     })] ]
                                            {FIRST}        {BONFERRONI}
                                            {LAST }        {SIDAK     }
  [/FIXED = [effect [effect ...]] [| [NOINT] [SSTYPE({1  })] ] ]
                                                     {3**}
  [/METHOD = {ML    }]
             {REML**}
  [/MISSING = {EXCLUDE**}]
              {INCLUDE  }
  [/PRINT = [CORB] [COVB] [CPS] [DESCRIPTIVES] [G] [HISTORY(1**)] [LMATRIX] [R]
                                                        (n  )
            [SOLUTION] [TESTCOV]]
  [/RANDOM = effect [effect ...]
             [| [SUBJECT(varname[*varname[*...]])] [COVTYPE({VC**      })]]]
                                                            {covstruct+}
  [/REGWGT = varname]
  [/REPEATED = varname[*varname[*...]] | SUBJECT(varname[*varname[*...]])
               [COVTYPE({DIAG**    })]]
                        {covstruct†}
  [/SAVE = [tempvar [(name)] [tempvar [(name)]] ...]
  [/TEST[(valuelist)] = 
      ['label']  effect valuelist ... [| effect valuelist ...] [divisor=value]]
      [; effect valuelist ... [| effect valuelist ...] [divisor=value]]
  [/TEST[(valuelist)] = ['label'] ALL list [| list] [divisor=value]]
                        [; ALL list [| list] [divisor=value]]
  
  ** Default if the subcommand is omitted.
  
  † covstruct can take the following values: AD1, AR1, ARH1, ARMA11, CS, CSH, CSR, DIAG, FA1, FAH1, HF, ID, TP, TPH, UN, UNR, VC.
  

• Why use LMM instead of regression?
  Data with multiple levels involve group effects on individuals which may be assessed invalidly by traditional statistical techniques. When grouping is present (ex., students in schools), observations within a group are often more similar than would be predicted on a pooled-data basis.That is, simple regression ignores grouping effects and violates the assumption of independence of observations. Mixed (multi-level) modeling handles this by using variables at upper levels (ex., school-level budgets at level 2) to adjust the regression of base level dependent variables on base level independent variables (ex., predicting student-level performance from student-level socioeconomic status scores).

  Multi-level modeling in LMM is particularly helpful in the analysis of covariance when data are sparse. For instance, in a study of a Social Security agency office, there may be too few minority employees to enable valid statistical inferences on performance evaluations, using traditional regression models. However, if multi-level data are available on employees and multiple SSA offices, then multi-level models can use not only the individual data in the SSA office but also information in the pooled data for all offices. The resulting prediction equation applied to the given SSA office will use coefficients reflecting both their own and also pooled data. For agencies with a large number of minorities, the multi-level and ordinary regression models will be similar. For agencies with sparse data -- few minorities -- it is true their estimate will rely considerably on the pooled data, but the advantage is that the pooling involved in multi-level models affords a "borrowing of strength" that supports statistical inference in a situation where no inference would be possible using traditional methods.

  Traditional regression models vs. LMM analysis. There were three traditional approaches to regression modeling of multilevel data:

  1. Simple regression, also called naive regression, simply ignored higher level effects (ex., ignored class or school effects in a study of students). This is appropriate, of course, only when the researcher can be sure there are no higher level effects. More often, there are such effects and simple regression leads to too low estimates of standard error, a higher rate of Type I errors and too-narrow confidence limits compared to multilevel modeling of the same data.

  2. Fixed effects regression. A popular traditional approach was to disaggregate data to the base level (ex., each student is assigned various school-level variables such as funding level per student, and all students in a given school have the same value on these contextual variables, and students are used as the unit of analysis). In fixed effects regression, sampling error is taken into account only for level 1 (the base) level, and sampling error at level 2 (or higher) is ignored. That is, information from fewer units at the upper level is wrongly treated as if it were independent data for the many units at the base level, and this error in treating sample size led to over-optimistic estimates of significance. Also, there was the danger of the ecological fallacy: there is no necessary correspondence between individual-level and group-level variable relationships (ex., race and literacy correlate little at the individual level but correlate well at the state level, since Southern states have many African-Americans and many illiterates of all races). Finally, under fixed effects regression, the number of dummy variables increases as the number of clusters increases, making estimation inefficient. While adding crosslevel interaction terms is possible and does represent an improvement, it is inferior to multilevel modeling in LMM, which will model separate intercepts and slopes for individuals in each level 2 (or higher level) group, where the grouping variable is treated as a random effect.

  3. Summary measures regression. Another traditional approach to multi-level problems was to aggregate data to a higher level (ex., student performance scores are averaged to the school level and schools are used at the unit of analysis). Aggregated data was often centered (the mean was subtracted so the average value was zero). Ordinary OLS regression or another traditional technique was then performed on the unit of analysis chosen. A problem with summary measures regression is that under aggregation, fewer units of analysis at the upper level replace many units at the base level, resulting in loss of statistical power. As with simple regression, summary measures regression regression leads to too low estimates of standard error, a higher rate of Type I errors and too-narrow confidence limits compared to multilevel modeling of the same data. Summary measures regression also suffers from the possibility of ecological fallacy. Finally, summary measures regression prevents the valid analysis of covariate interactions due to loss of individual-level information.

  Based on a review of the literature and on simulation studies, Ita G. G. Kreft (1996) concluded, "for researchers specifically interested in variance components, and posterior means, RC modeling provides them with separate estimates for separate contexts, and the iteration procedure improves the estimates of the variance components." That is, although effect size as revealed through regression is apt to be similar to effect size in multi-level modeling (see discussion below), multi-level modeling is more helpful in revealing differences in variance among units of analysis in different groups which comprise the levels. An empirical comparison of OLS regression with multilevel modeling by Moerbeek, van Breukelen, & Berger (2003) found that "The treatment effect and especially its standard error, are generally incorrectly estimated by traditional methods, which should, therefore, not in general be used as an alternative to multilevel regression" (p. 341). Also, multi-level modeling may be a preferred method when data are sparse, including studies (ex., twin studies) where groups are sparse.

  • In defense of OLS. Based on a review of the literature and on simulation studies, Ita G. G. Kreft (1996) concluded, "if researchers in the social sciences are interested in the estimates of the regression parameters, the results of multi-level analysis will be close to the results obtained with more traditional regression techniques. In both cases the fixed effects estimates are unbiased. The main difference is in the standard errors of these parameters, which are estimated too small if intra-class correlation is present in traditional regression analyses. This fact makes the random coefficient model more conservative than the traditional regression." Kreft also raised questions about whether multi-level models are as generalizable as ordinary regression models. Because multi-level models rely on the complex, particular distribution of relationships across and within levels, Kreft concluded, "Outcomes are less general, since each best fitting model may be very specific for that dataset collected at that time and place." The smaller or less random the sample of higher-level units, the more true this would be. See also Kreft and de Leeuw (1991); Kreft, de Leeuw, and van der Leeden. (1994).

• What is the multi-level equivalent to R-squared in OLS regression?
  There is no equivalent to R-squared. Goodness of fit measures (ex., AICC, BIC) are used instead in multi-level modeling. Analogues to R-square have been proposed but are not widely used.

• Why use LMM instead of GLM?
  Many research problems might be modeled as general linear models (GLM option in SPSS) rather than as linear mixed models (Mixed option in SPSS). LMM provides a number of advantages, however:
  1. GLM assumes independence but LMM does not, as discussed in the "Assumptions" section.
  2. Handling missing data: GLM will apply listwise deletion to drop cases with missing values, whereas Mixed (LMM) will include incomplete cases in the analysis.
  3. GLM repeated measures assumes all subjects are measured at the same points in time, whereas LMM allows subjects to be measured at different points in time.
  4. Repeated measures GLM requires subjects to have equal numbers of repeated measurements, but LMM allows unequal repetititons. That is, LMM is asymptotically efficient for both balanced and unbalanced designs, but GLM is optimally efficient only for balanced designs.
  5. GLM requires all interactions of within-subjects (repeated measures) and between-subjects factors be included in the model, whereas LMM allows the researcher to just include the interactions of interest.
  6. GLM makes certain assumptions about the covariance matrix and thus data must meet the sphericity test, whereas LMM allows for a wide variety of assumptions about the covariance matrix.
  7. LMM supports hierarchical data (data at one level nested within a higher level), but GLM does not.
  8. LMM estimates are based on maximum likelihood (ML) or restricted maximum likelihood (REML) methods, whereas GLM is based on ANOVA methods.

• Why does my GLM model give the same parameter estimates and corresponding significance levels as LMM for the same data, but LMM does not print out sums of squares?
  For some models, such as fixed effects with uncorrelated residuals, GLM and LMM will give the same parameter estimates and significance levels (LMM in the "Tests of Fixed Effects" and "Estimates of Fixed Effects" tables; GLM in the "Tests of Between-Subjects Effects" and "Parameter Estimates" tables). However, LMM can fit a wider variety of models than GLM and in some LMM models, test effects cannot be expressed in terms of ratios of sums of squares and therefore there is no "Sum of Squares" column.

• Besides variance components and diagonal covariance structure assumptions, what other assumptions might be made?
  SPSS supports the following:
  • Ante-Dependence: First Order, AD(1). This covariance structure has heterogenous variances and heterogenous correlations between adjacent elements. The correlation between two nonadjacent elements is the product of the correlations between the elements that lie between the elements of interest.
    

  • AR(1). This is a first-order autoregressive structure with homogenous variances. AR(1) is a common assumption for data where there is a common trend, such as where the correlation of any pair of repeated measurements is assumed to decrease according to how far apart they are in time. Thus a measure at time 1 is apt to be less similar to the same subject's measure at time 4 as it is to the measure at time 2. This model also assumes the time variable is metric and equally spaced. In the residual covariance matrix, the diagonal variances will be roughly equal, and the off-diagonal covariances will show a pattern, normally decreasing over time. If the correlation between any two elements is equal to r for time-adjacent elements, then it is r*2 for elements that are separated by one other element, etc., within the constraint that the bounds of +/- 1 are not exceeded.
    

  • AR(1): Heterogeneous. This is a first-order autoregressive structure with heterogenous variances. The correlation between any two elements is equal to r for adjacent elements, r2 for two elements separated by a third, and so on.
    

  • ARMA(1,1). This is a first-order autoregressive moving average structure. It has homogenous variances. The correlation between two elements is equal to f*r for adjacent elements, f*(r2) for elements separated by a third, and so on. r and f are the autoregressive and moving average parameters, respectively, and their values are constrained to lie between –1 and 1, inclusive.
    

  • Compound Symmetry. This is a common repeated covariance structure assumption. Within-subject correlation of error terms is assumed to be equal. That is, the correlation of measurements nearby in time should be the same as for measurements with a large time-distance. This covariance structure has homogenous variances and homogenous correlations between elements. Thus, in the residual covariance matrix, the diagonal variances will be roughly equal, and the off-diagonal variances will also be roughly equal to each other. This is the type assumed in univariate Anova models and is the classical approach to repeated measures. The assumption of compound symmetry is more likely to be met with classical experimental data than with longitudinal data where autoregressive or Toeplitz assumptions may be more appropriate.
    

  • Compound Symmetry: Correlation Metric. This covariance structure has homogenous variances and homogenous correlations between elements.
    

  • Compound Symmetry: Heterogeneous. This covariance structure has heterogenous variances and constant correlation between elements.
    

  • Factor Analytic: First Order. This covariance structure has heterogenous variances that are composed of a term that is heterogenous across elements and a term that is homogenous across elements. The covariance between any two elements is the square root of the product of their heterogenous variance terms.
    

  • Factor Analytic: First Order, Heterogeneous. his covariance structure has heterogenous variances that are composed of two terms that are heterogenous across elements. The covariance between any two elements is the square root of the product of the first of their heterogenous variance terms.
    

  • Huynh-Feldt. This is a "circular" matrix in which the covariance between any two elements is equal to the average of their variances minus a constant. Neither the variances nor the covariances are constant.
    

  • Scaled Identity. This structure has constant variance. There is assumed to be no correlation between any elements. This is a common assumption when modeling the interaction of a random factor (ex., City) with a fixed grouping factor (ex., Agency), where it is assumed that the City*Agency interaction effect is normally distributed around a mean of zero, with unknown variance to be estimated.
    

  • Toeplitz. This structure is a generalization of the AR(1) type. Like AR(1), the Toeplitz model assumes metric, equal time intervals and assumes pairs of within-subject correlations are equal for the same measurement time-distance apart. Unlike the special case of AR(1), however, in a Toeplitz model the pattern sequence for off-diagonal covariances does not have to step by some common multiple but may step by some unique multiple associated with that time step. That is, there is a trend as in AR(1) models but there is no common function describing that trend for all time intervals. This covariance structure has homogenous variances and heterogenous correlations between elements. The correlation between adjacent elements is homogenous across pairs of adjacent elements. The correlation between elements separated by a third is again homogenous, and so on.
    

  • Toeplitz: Heterogeneous. This covariance structure has heterogenous variances and heterogenous correlations between elements. The correlation between adjacent elements is homogenous across pairs of adjacent elements. The correlation between elements separated by a third is again homogenous, and so on.
    

  • Unstructured. This is a completely general covariance matrix. This choice is a common assumption for random effects models. The correlation of pairs of measurements for the same subject are unpredictable. Within the same subject, there is no pattern (up, down, or equal) for how measures change over time. This is the assumption in Manova models. It requires the measurement time variable to be metric, with equal time intervals between measures. This type is complex, requiring the computation of many parameters.
    

  • Unstructured: Correlations. This covariance structure can have heterogenous variances and heterogenous correlations.
    

• What about multilevel modeling in structural equation models rather than linear mixed models?
  Multilevel structural equation models (MSEM) are generalizations of path analysis and as such are part of the structural equation modeling (SEM) family. MSEM models can have multiple dependent variables or latent variables (constructs measured by indicator variables) as well as multiple levels of measurement. See McDonald (1994) and Muthén (1994). While MLM regression models (MRM) and MSEM will perform similarly, compared to MRM, the SEM approach can more easily incorporate complex path models and multiple group models. Multilevel regression modeling (MRM) is compared with multilevel structural equation modeling (MSEM) by Tomarken & Waller (2005: 38). They note the advantages of MSEM to be more and more interpretable measures of goodness of fit, better modeling of residuals, and better capacity to model latent variables. They note the advantages of MRM to be easier model specification, fewer estimation problems, and ability to handle certain types of analysis difficult to handle within MSEM. For instance, in MSEM you cannot model between-group variablility in factor loadings or path coefficients (Hox, 2002). They note, however, that the two approaches are more similar than different, and as techniques and software evolve, functional similarity is increasing. On the relation of multilevel modeling to SEM, see Curran (2003).

• By modeling a grouping factor (ex., cities) as a random effect, may the researcher generalize conclusions to all cities, not just those in the sample?
  While one sometimes encounters this statement in literature on random effects models, it is incorrect if stated in an unqualified manner. There is no way for, say, results from a convenience sample of five cities in Utah to be generalized to all U. S. cities. The statement would be true only if the cities were randomly selected from all U. S. cities, and the number of these level 2 groups (cities) was sufficient (>20 is a common rule of thumb).

• What are split plot experiments in LMM?
  Split plot experiments are a common type of linear mixed model. Arising out of agricultural applications, there would be one factor such as levels of irrigation applied randomly to whole plots of land within farms, then another factor such as seed types applied randomly to subplots (the "split plots") nested within the plots. The whole plots reflect diffent randomly-selected levels of irrigation and contain a set of subplots with randomly selected seed types. There may also be control variables, called "blocking variables," such as the farm on which the plots are located. There is, of course, also a dependent variable such as plant height, called the "response variable." In this example, irrigation level, seed type, and farm would all be "classification variables" for plant height as dependent. A variety of mixed models could be constructed using these factors as fixed and random effects. For instance, one could specify irrigation, seed type, and irrigation*seed type as fixed effects and farm and irrigation*farm as random effects. Output of LMM would include covariance parameter estimates for Farm, Farm*Irrigation interaction, and Residual, along with AIC and other fit statistics for the model, interpreted as discussed above.

• What are latent growth models?
  • Latent growth models (LGM) are a type of multilevel model suitable for clustered data with repeated observations of variables at multiple levels. While multiple group SEM is a conventional approach discussed in the SEM section of Statnotes, multilevel SEM (MSEM) is a newer methodological development which handles large numbers of groups (ex, 100 - 200). Strictly speaking, an LGM model adapts SEM to analyze change over time when both individual and group-level variables are in the model. However, LGM has been extended and generalized to other types of multi-level models and may refer simply to multilevel models based on SEM software as opposed to regression-based multilevel models.. As such, LGM is a more versatile alternative to repeated measures ANOVA (see Tomarken & Waller, 2005: 36 for a list of nine ways LGM is more versatile). For an introduction to LGM, see Duncan, Duncan, Strycker, et al. (1999) and Hox (2002). See also du Toit & du Toit (2005).

• What software is available for multilevel modeling?
  Several software packages for multi-level modeling have emerged in the last decade. Leading packages are listed below.
  • SPSS's "Linear Mixed Models" module, part of its SPSS Advanced Models extension, handles hierarchical linear models (HLM) as well as related models for random or mixed ANOVA and ANCOVA, repeated measures ANOVA and MANOVA, and variance component estimation (VARCOMP).

  • AMOS and LISREL. It is possible to implement multi-level models in structural equation modeling programs like AMOS and LISREL. LISREL's MLM module is called MULTILEV.

  • SAS's PROC MIXED procedure can implement several models: simple random-effect only, simple mixed with a single fixed and random effect, split-plot, multilocation, repeated measures, analysis of covariance, random coefficients, and spatial correlation See Littell et al. (1999).

  • HLM, authored Steve Raudenbush and Tony Bryk. Raudenbush heads the longitudinal and multi-level methods project at Michigan State University. HLM can read data from a variety of statistical packages, including SPSS, SAS, SYSTAT, and STATA, and it covers nonlinear as well as linear models. A free student version is available. This was perhaps the leading package during the development of multi-level modeling in the 1990s. HLM does not have a bulit-in data editor: data preparation must be done in SPSS (which HLM imports) or another program. HLM does not read ordinal variables, which must be converted to a series of dummy variables in the data preparation stage. Cross-level interaction terms are created automatically by HLM, and there is an option for automatic centering of variables (group mean centered or grand mean centered).

  • MLWin, a Windows program produced by the UK/Canada Multilevel Models Project, for models with any number of levels. It is the Windows version of the earlier MLn multi-level modeling software package.

  • MPlus supports multi-level modeling with latent variables.

Bibliography

• Booth, J.G. & Hobert, J.P. (1998). Standard errors of prediction in generalized linear mixed models. Journal of the American Statistical Association 93, 262 -272.
• Breslow, N. E., & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88, 9-25.
• Bryk, A.S., and Raudenbush,S.W. (1992). Hierarchical linear models. Newbury Park, Sage Publications.
• Bryk, A. S., S. W. Raudenbush, and R. Congdon (1996). Hierarchical Linear and nonlinear modeling with the HLM/2L and HLM/3L Programs. Chicago: Scientific Software International.
• Burton, B. (1993). Some observations on the effect of centering on the results obtained from hierarchical linear modeling. Washington, DC: National Center for Education Statistics, U. S. Department of Education.
• Cheung, M. W.-L. & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling 12(4), 598-619.
• Curran, P. J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research 38(4): 529–569.
• de Leeuw, J. and I. G. G. Kreft (1986). Random coefficient models for multi-level analysis. Journal of Educational Statistics 11: 57-86.
• Diggle, P. J. (1988). An approach to the analysis of repeated measurements. Biometrics 44, 959-971.
• Duncan, T. E., Duncan, S. C., Strycker, L. A., Li. F, & Alpert, A. (1999). An introduction to latent variable growth curve modeling: Concepts, issues, and applications. Mahwah, NJ: Erlbaum
• du Toit, S. H. C. & du Toit, M. (2005). Multilevel structural equation modeling. In de Jeeuw, J. & Kreft, I. G. G., eds., .Handbook of quantitative multilevel analysis. Boston: Kluwer Acad.
• Ehlers, Margaret (2004). Assessment of covariance selection strategies with repeated measures data. 34th Annual SCASA Meeting, American Statistical Association. Retrieved 4/223/07 from http://www.stat.sc.edu/scasa/Apr04t.html.
• Goldstein, H. (1991). Nonlinear multi-level models with an application to discrete response data. Biometrika 78: 45-51.
• Goldstein, H. (1995). Multilevel statistical models. London, Edward Arnold; New York, Halstead Press.
• Hoffman, D. A. & Gavin, M. B. (1998). Centering decisions in hierarchical linear models: Implications for research organizations. Journal of Management 24(5), 623-641.
• Hox, J. J. (1995). Applied multi-level analysis, 2nd Ed.. Amsterdam: TT-Publikaties. A basic, non-technical introductory text. Available at http://www.fss.uu.nl/ms/jh/publist/amaboek.pdf.
• Hox J. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum.
• Hox, J. J. & Maas, C. J. M. (2001). The accuracy of mulitlevel structural equation modeling with pseudobalanced groups and small samples. Structural Equation Modeling 8, 157-174.
• Jiang, Jiming (2007). Linear and generalized linear mixed models and their applications Springer Series in Statistics. New York: Springer.
• Julian, M. W. (2001). The consequences of ignoring multilevel data structures in nonhierarchical covariance modeling. Structural Equation Modeling 8, 325-352.
• Kreft, Ita G. G. (1996). Are multi-level techniques necessary? An overview, including simulation studies. Obtained online on Nov. 22, 1999, at http://www.calstatela.edu/faculty/ikreft/quarterly/quarterly.html.
• Kreft, Ita G. G., Jan de Leeuw, J., and Aiken, Leona S. (1995). The effect of different rorms of centering in hierarchical linear models. Multivariate Behavioral Research 30(1), 1-21.
• Kreft, Ita G. G., Jan de Leeuw, J., and R. van der Leeden (1994). Review of five multi-level analysis programs: BMDP-5V, GENMOD, HLM, ML3, VARCL. American Statistician 48(4): 324-335.
• Kreft, Ita G. G. and Leeuw, Jan de (1991). Model-based ranking of schools. International Journal of Education 15: 45-59.
• Kreft, Ita G. G. and Leeuw, Jan de (1998). Introducing multi-level modeling. Thousand Oaks, CA: Sage Publications. Introducing Statistical Methods Series. A user-oriented introductory text with a minimum of formal mathematics. Includes worked examples using real datasets using MLn, the predecessor of MLWin.
• Littell, Ramon C., George A. Milliken, Walter W. Stroup, and Russell D. Wolfinger (1999). SAS System for mixed models. Cary, NC: Sas Institute.
• Longford, N.T. and Muthen,B.O. (1992). Factor analysis for clustered populations. Psychometrika, 57, 581-97.
• Longford, N.T. (1993). Random coefficient models. Oxford, Clarendon Press.
• Luke, Douglas (2004). Multilevel modeling. Thousand Oaks, CA: Sage Publications. Vol. 142 in Quantitative Applications in the Social Sciences Series.
• Maas, CoraJ. M. & Hox, Joop J. (2005) Sufficient sample sizes for multilevel modeling. Methodology 1(3), 86-92.
• McCulloch, Charles E. (1997). Maximum likelihood algorithms for generalized linear mixed models. Journal of the American Statistical Association 92, 162-170.
• McCulloch, Charles E. & and Searle, Shayle R.(2001). Generalized, linear, and mixed models. NY: Wiley-Interscience.
• McCulloch, Charles E. (2003). Generalized linear mixed models. Regional Conference Series. Bethesda, MD: Institute of Mathematical Statistics
• McDonald, R.P. (1994). The bilevel reticular action model for path analysis with latent variables. Sociological Methods and Research, 22, 399-413.
• McDonald, R.P. and Goldstein, H. (1988). Balanced versus unbalanced designs for linear structural relations in two level data. British Journal of Mathematical and Statistical Psychology, 42, 215-32.
• Moerbeek, Mirjam; van Breukelen, Gerard J. P.; & Berger, Martijn P. F. (2003). A comparison between traditional methods and multilevel regression for the analysis of multicenter intervention studies. Journal of Clinical Epidemiology 56, 341-350.
• Morris, C. (1995). Hierarchical models for educational data - an overview. Journal of Educational and Behavioral Statistics 20(2).
• Muthén, B.O. (1989). Latent variable modelling in heterogeneous populations. Psychometrika, 54, 557-85.
• Muthén, B.O. (1994). Multilevel covariance structure analysis. Sociological Methods and Research 22: 376-399.
• Peugh, James L. & Enders. Craig K. (2005). Using the SPSS Mixed procedure to fit cross-sectional and longitudinal multilevel models. Educational and Pcyhological Measurement 65(5), 717-741. Replicates Singer (1998) using SPSS Mixed instead of SAS. Enders has made supplementary instructional material available at http://www.asu.edu/clas/psych/people/documents/FittingmultilevelmodelsusingSPSSpull-downs.pdf.
• Raudenbush, Stephen W. and Anthony S. Bryk (2002). Hierarchical linear models: Applications and data analysis methods, Second edition. Thousand Oaks, CA: Sage Publications. Advanced Quantative Techniques in the Social Sciences Series No. 1.
• Singer, J. (1998). Using SAS Proc Mixed to fit multilevel models, hierarchical models, and individual growth curves. Journal of Educational and Behavioral Statistics 24(4), 323-355.
• Singer, J. & Willett, J. (2003). Applied logitudinal data analysis. Oxford, UK: Oxford University Press.
• Snijders, T. & Bosker, R. (2000). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
• SPSS, Inc. (2005). Linear mixed-effects modeling in SPSS: An introduction to the MIXED procedure. SPSS Technical Report. Chicago, IL: SPSS, Inc.
• Sullivan, Lisa M.; Dukes, Kimberly A.; & Losina, Elena (1999). Tutorial in biostatistics: An introduction to hierarchical linear modelling. Statistics in Medicine/i> 18: 855-888. Illustrated using HLM and SAS.
• Thompson, R. (1985). A note on restricted maximum likeliho