Barbara G. Tabachnick and Linda S. Fidell (2001). Using Multivariate Statistics, Fourth Edition. Needham Heights, MA: Allyn & Bacon. ISBN 0-321-05677-9. Hardcover.
CHAPTER 1: INTRODUCTION
Define:
independent variable
dependent variable
experimental research
continuous vs. discrete data
sample vs. population
inferential statistics: making inferences about population parameters based on sample data
orthogonal: uncorrelated
sequential analysis: first variables entered are assigned both the unique variance they account for in the dependent, and the overlapping variance accounted for shared with variables entered later in the equation. In standard analysis, each variable is assigned only its unique variance accounted for.
Statistical power. Significance of .05 means there is a 5% chance you think there is a relationship but in fact it is due to the chance of the sample. Power of .95 means there is a 95% chance that a relationship which really exists in the population will be shown to be significant. These are not reciprocals!! Social scientists typically expect .05 significance and .80 power. Software exists to estimate power for a given test for a given set of data.
Correlation matrix: standardized strength of association, regardless of the magnitude of the scale of the variables. Standardized = controlling for mean and standard deviations
Covariance matrix: unstandardized correlations, used as the basis for SEM
residuals: estimated (predicted) minus actual (obtained) values
which version of SPSS is in Tabachnick, 4th ed.? : SPSS 10, SAS 7
Barbara G. Tabachnick and Linda S. Fidell (2001). Using Multivariate Statistics, Fourth Edition. Needham Heights, MA: Allyn & Bacon. ISBN 0-321-05677-9. Hardcover.
CHAPTER 2: A GUIDE TO STATISTICAL TECHNIQUES
Table 2.1, pp. 27-29, is a decision tree for selecting a procedure. Go over it in class.
Barbara G. Tabachnick and Linda S. Fidell (2001). Using Multivariate Statistics, Fourth Edition. Needham Heights, MA: Allyn & Bacon. ISBN 0-321-05677-9. Hardcover.
CHAPTER 3: REVIEW OF UNIVARIATE AND BIVARIATE STATISTICS
1. Hypothesis testing: 1-sample z tests (of difference of means) - pp. 31-34
* 95% of cases w/in 1.96 standard units
* if sample mean is further than +/- 1.96 s.d. of the hypothesized real mean, the nul hypothesis is rejected (that the sample mean is not different from the real mean).
* That is for a 2-tailed hypotheses
* If 1 tailed (only greater than...) the sample mean must be more than 1.645
* same logic tests difference between two means; or between proportions
2. One-Way Between Subjects ANOVA - pp. 35-40
* One-way means one IV
* Between subjects means regular ANOVA in which subjects are measured at one point in time
* If F is more than critical (table) F then IV has a non-random effect on the DV
* F for df=k-1 and N-k
Where k is the number of groups (values) of the DV
And N is sample size
* Must assume homogeneity of covariances across groups.
3. Factorial Between-Subjects ANOVA - pp. 40-41
* The treatment IV is factored by a second categorical IV (ex., male and female groups for each of three types of treatment - six groups in all)
* Three F tests
* One for gender: are men and women different for the pooled treatments?
* One for treatments: are treatments different for the pooled subjects?
* One for gender aand treatment: do the male/female differences vary by treatment?
4. Within-Subjects ANOVA - pp. 41-44
* Within subjects means repeated measures, test-retest, panel studies
* F test computed differently, interpreted the same
* There is a factorial within-subjects variant design
5. Mixed Between-Within Subjects ANOVA - p. 44
* Some IVs measured at one point in time between subjects, but there are other IVs which are measured at two or more time points within subjects.
* There are many other complex ANOVA designs (pp. 45-47), including
* Nested designs: Levels of one IV are confined to only one level of another IV. Ex. Classrooms nested within teaching methods, children nested within classroooms.
* Latin Square designs: control for order of exposure to the IV levels.
* Unequal n: groups differ in size; the more different in size, the more critical the assumption of homogeneity of variances
*Random effects models: only a sample of levels of each IV are tested.
6. Comparison tests - pp. 47-51
* If the overall F shows significance, we can still wonder which IV/DV value combinations contributed the most to F.
* Problem: if we do every possible comparison test, some will be significant by chance. Need to adopt a stringent approach:
* Planned comparisons: test just a few comparisons based on theory
* Post hoc tests: adjust alpha upward to be more stringent, as in Scheffe test
7. Eta - p. 52
* Eta-square is a strength of assn measure computed with ANOVA
* It is the ratio of SS-effect to SS-total
* Problem: the more the IVs, the lower the eta for the same strength of assn
* Partial eta is the ratio of SS-effect to (SS-effect +_ SS-error). It tries to deal with the problem of inflation of eta with more IVs but sum of etas does not add to total system variance of the DV and may add to more than 1.0.
* Problem: eta is a purely sample measure not attemption to estimate association in the population
* Omega-square tries to remedy this but can only be used in between-subjects designs with equal size groups.
8. Pearsonian Correlation - pp. 53-54
* r-square is the percent of variance explained
* it is the average cross-product of standardized x and y scores
9. Bivariate Regression - pp. 54-55
*. B is the amount y will change given a unit change in x
* b is the ratio of the covariance of the variables to the variance of x (the predictor)
10. Chi-square analysis - p. 55
* Chi-square formula is based on ratio of (O - E)2 to E, summed for all cases.
* The chi-square value is compared to a table to compute probabilities\
* Chi-square is a measure of significance, reflecting both assn and sample size.
Barbara G. Tabachnick and Linda S. Fidell (2001). Using Multivariate Statistics, Fourth Edition. Needham Heights, MA: Allyn & Bacon. ISBN 0-321-05677-9. Hardcover.
CHAPTER 4: CLEANING UP YOUR ACT: SCREENING DATA PRIOR TO ANALYSIS
What is the main cause of inflated correlations? P. 57
Definitional overlap. The first variable contains one or more components which also appears in similar meaning in the second variable. This is most likely to happen when the variables are scales or indexes composed of multiple items.
Scales should meet Cronbach’s alpha test of being > .7 (or at least .6); factoring is another way of testing distinctness of concepts.
What is the main cause of attenuation of correlation? P. 57
Attenuation occurs when....
* the range of the variable is truncated
* a continuous variable is made into a categorical variable with few values
* a dichotomy has a very uneven split (90:10 or worse)
You can compute r adjusted for attenuation, but this is an estimate.
Explain Table 4.1. Pp. 59-60: Testing for MAR (missing at random)
This is output from SPSS MVA (Missing Values Analysis)
Income has 26 missing out of N=439. This is over 5% so is worth analyzing.
The p-values for the t-test (you have to ask for it, it’s not a default) show if missing values on a given variable are correlated with other variables. Table 4.1 shows 1 variable, Income, has > 5% misssing (top portion) and it is not significantly correlated with any of the other variables (bottom portion). You want not significant 2-tail p values. If you fail this test, data are not missing at random.
Hands-on MVA Part 1: Testing for Missing at Random
Note: this will be similar to but not identical to Table 4.1
Load in HLTHSEM.SAVE
Analyze, MVA
load in INCODE (income), ATTHOUSE, and ATTMED
click the Descriptives button and ask for t-tests with probabilities
click OK
print out the output and be prepared to explain it
be prepared to turn in a copy of the output with your name on it
Explain the main data imputation methods. Pp. 59-66
* Deleting cases: This option is ok if there are not many and the t-test under MVA shows they are missing at random. Warning: SPSS uses this by default...look at your N and see how many were dropped without your asking!
* Imputing (estimating) cases:
- Prior knowledge: sometimes the research has the basis to make an educated guess
- Grand mean substitution: was once popular but reduces variance and correlation with other variables.
- Group mean substitution: better - computer means for groups where you DO ave non-missing data, such as state of residence or political party id, then substitute these group means for the missing data. Best to pick a grouping variable thought to correlate highly with the variable with missing cases. SPSS does not have this option.
Not in Tabachnick: Transform, Replace Missing Cases in SPSS 11 has these options:
-Series mean. Replaces missing values with the mean for the entire series.
-Mean of nearby points. Replaces missing values with the mean of valid surrounding values. The span of nearby points is the number of valid values above and below the missing value used to compute the mean.
-Median of nearby points. Replaces missing values with the median of valid surrounding values. The span of nearby points is the number of valid values above and below the missing value used to compute the median.
-Linear interpolation. Replaces missing values using a linear interpolation. The last valid value before the missing value and the first valid value after the missing value are used for the interpolation. If the first or last case in the series has a missing value, the missing value is not replaced.
-Linear trend at point. Replaces missing values with the linear trend for that point. The existing series is regressed on an index variable scaled 1 to n. Missing values are replaced with their predicted values.
- Regression imputation. Use other variables to get a regression prediction of the missing values of one variable. This only works if there are other IVs which do in fact predict the variable with missing cases. Also, regression imputation works too well: the real values might show more noise or even be outside the existing range.
Hands-on MVA Part 2: Regression Imputation.
Still in MVA with HLTHSEM.SAV
load in INCODE (income), ATTHOUSE, and ATTMED
click the Regression button and check to “Save completed results” to save the estimates
check Regression in the Estimation panel
click OK
print out the output and be prepared to explain it
be prepared to turn in a copy of the output with your name on it
- Expected maximization (EM) imputation. P. 63. Maximum likelihood estimation (MLE) rather than regression estimates of the missing values. Identical to MVA regression imputation, except you check the EM box instead of the regression box. This is now the standard method for dealing with missing data! It is preferred to regression because it handles nonlinearities and makes fewer data assumptions, as will be discussed later in the course.
- Multiple imputation P. 63. Use logistic regression in an iterative process to estimate m new data sets. Run your analysis on all m datasets. Report averaged coefficients over the m runs. This is not implemented by SPSS or SAS.
What other methods are discussed by Tabachnick for handling missing data (pp. 64 - 66)?
- Using a missing data correlation matrix. Pp.65. In SPSS correlation module, ask of pairwise deletion rather than the usual listwise. Coefficients will be calculated for all available bivariate data even if a case has missing data for variables not part of the pair being correlated at the moment. This means r’s are based on different n’s. This involves mathematical problems and is not recommended unless you have a large sample with only a few missing cases, then you can use this pairwise correlation matrix as input for factor analysis or other procedures.
- Making “Missing” into an IV. P. 65. You can create a dummy variable representing missing/not missing data on a given variable. It may be this dummy will itself have explanatory power for your DV. If a variable is thought relevant to a DV but has a lot of missing cases, this procedure is better than just dropping the variable.
- Analysis with and without missing values. Run separate analyses and see if your outcomes are the same for data with imputed values vs. the subset of data with complete information. If there is a difference, you need to examine why. If you cannot explain it, report both sets of results.
What are the four main reasons for outliers? P. 67.
1. Wrongly coded data: scan your dataset for extreme and out-of-range values
2. In SPSS you forgot to define missing values, so missing, don’t know, and other values are entered as real data. Don’t assume a .SAV file has already defined missing values.
3. The outlier case is not part of the intended population (ex., a Japanese visitor in a study intended to sample citizen-residents of a neighborhood). Remove the case with replacement.
4. The variable simply has a non-normal distribution with extreme values. There are transforms to normalize data, or the case may simply be recoded to a more moderate value.Sometimes extreme cases need to be modeled separately.
Differentiate a univariate from a multivariate outlier. What criteria are used for each?
P. 67
Univariate: more than 3 sd’s away from mean (=standardized score > 3.29); dichotomies with more than a 90:10 split. SPSS If you are using a grouped procedure like ANOVA, logistic regression, discriminant function analysis, or others, look for outliers within each group. Boxplots and normal probability plots are a graphical way to spot univariate outliers.
Multivariate: an extreme combination, like juvenile with a high income. Mahalanobis distance is the most common measure used for multivariate outliers.
How do you screen for outliers in SPSS? Pp. 67-70, 93
1. Analyze, Descriptive Statistics, Explore; click the Statistics button and check Outliers; OK. You will get the five most extreme high and low cases. You can also click on Plots and get boxplots or normal probability plots if you prefer a graphical method.
2. Analyze, Descriptive Statistics, Frequencies, Check for dichotomies with more than 90:10 split. You can also ask for histograms if you prefer a graphical method.
3. Analyze, Regression, Linear. Enter the DV and IV’s. Click the Save button and check Mahalanobis, Cook’s, and/or Leverage in the Distances panel. Note there are a variety of Influence measures you could also check in the Influence panel. SPSS will add columns at the end of your dataset showing these coefficients.
The larger the coefficient, the more it is an outlier with respect to the set of IVs (not the DV). Therefore you may wish the DV not to be a variable of interest for the substantive analysis.
Is outlier if leverage (extent case affects the prediction) is ....
Mahalanobis > critical value of Chi-Square (Table C.4, p. 933, where df = number of IVs; using the .001 column).
Cook’s D > 4/(n - k - 1), where n is the number of cases and k is the number of IVs.
Leverage > .5
What are discrepancy and influence of a case? P. 69
Discrepancy: extent case is along regression line even if too far out
Influence: combines leverage and discrepancy. DfFit is amount regression beta will change if case is dropped. Rule of thumb: if > 1, then is outlier. DfFit is in Influence panel of Analyze, Regression, Linear; Save button.
Other than eyeballing, what is one way to analyze outliers? P. 70
Create a dummy variable for outlier/not outlier and use logistic regression to analyze it.
What do you do about outliers besides analyzing them? P. 71
Check your data for correct entry and coding; make sure missing values are defined
You can apply transforms to normalize (won’t necessarily solve multivariate outliers)
You can recode to make outliers less deviant by score
You can drop resistant cases
For any of these, you must state what you did and why
How do you check for univariate normality? For multivariate normality? Pp. 72-77.
Needed for parametric procedures in the regression family.
Central Limit Theorem says not to worry if you have grouped data (ex., Anova) and you have large samples (df> 20 in Anova).
Skew and kurtosis should be within +/- 2, Tabachnick says compute significance of skew and kurtosis, but then ignore this and just look at the distribution if sample size is large (because will be significant even for small skew or kurtosis if sample size is very large).In SPSS, Analyze, Descriptive Statistics, Descriptives or Explore print skew and kurtosis.
Frequency distributions. Just eyeball graphically, looking for bell-shaped curve..
Normal probability plots: Another graphical method: if normal, will form 45-degree plot line (horizontal line for detrended normal P-P plots).
Not discussed in text: Shapiro-Wilks's W test is a formal test of univariate normality offered in the SPSS EXAMINE module Mardia's statistic is a test for multivariate normality. Based on functions of skewness and kurtosis, Mardia's PK should be less than 3 to assume the assumption of multivariate normality is met. LISREL but not SPSS computes this.
How do you test for linearity? Pp. 77 - 79
- Look at a scatterplot or scatterplot matrix
- Not discussed: difference of eta and r
- Not discussed: contrasts option in Anova, (A polynomial contrast partitions the between-groups sums of squares into trend components, which can be used to test for a trend (ex., a linear trend) of the dependent variable across the ordered levels of the categorical independent variable. SPSS supports 1st, 2nd, 3rd, 4th, and 5th degree polynomials. )
How do you test for homogeneity of variances? Pp. 79-80
A crucial assumption for the Anova family.
Homogeneity of variances is homoscedasticity for grouped data. Levene’s F test is the standard test. If the Levene statistic is significant at the .05 level or better, the researcher rejects the null hypothesis that the groups have equal variances. The Levene test is robust in the face of departures from normality. Levene's test appears in the SPSS procedures ONEWAY and T-TEST as well as EXAMINE.
Tabachnick discusses F-max test for homogeneity of variances: if the ratio of the largest to smallest group in Anova is 4:1 or less, then if the ratio of the largest to smallest group variance (F-max) is 10 or less, homogeneity of variances is not a problem. When group sizes are unequal, F-max may need to be as little as 3.
Homogeneity of variance-covariance matrices is multivariate homogeneity of variances. Box’s M is the test, but it is acknowledged to be too strict. The researcher wants this test not to be significant, so as to accept the null hypothesis that the groups do not differ. This test is very sensitive to meeting also the assumption of multivariate normality.
What is the purpose of data transformations? Pp. 80 - 82.
To normalize data.
See Figure 4.7, p. 82:
(Note: reflect = add 1 to highest score; subtract all values from this constant)
Square root for positive skew
Reflect and square root for negative skew
Logarithm for positive skew and kurtosis
Reflect and logarithm for negative skew and kurtosis
Inverse for declining hyperbolic curve
Reflect and inverse for ascending hyperbolic curve.
Garson Assumptions section: Various transformations are used to correct skew: square roots, logarithmic, and inverse (1/x) transforms "pull in" outliers and normalize right (positive) skew. Inverse (reciprocal) transforms are stronger than logarithmic, which are stronger than roots. To correct left (negative) skew, first subtract all values from the highest value plus 1, then apply square root, inverse, or logarithmic transforms. For power transforms, finer adjustments can be used by adding a constant, C, in the in the transform of X: (X + C)P. Values of P less than one (roots) correct right skew, which is the common situation (using a power of 2/3 is common when attempting to normalize). Values of P greater than 1 (powers) correct left skew. For right skew, decreasing P decreases right skew. Too great reduction of P will overcorrect and cause left skew. When the best P is found, further refinements can be made by adjusting C. For right skew, for instance, subtracting C will decrease skew. Logs vs. roots: logarithmic transformations are appropriate to achieve symmetry in the central distribution when symmetry of the tails is not important; square root transformations are used when symmetry in the tails is important; when both are important, a fourth root transform may work.
What is multicollinearity? What is singularity? Pp. 82-85
Multicollinearity is redundancy among the IVs. If the redundancy is total, one has singularity, which may prevent algorithms from computing any answer. High but not complete redundancy will still mean that the standard errors of the coefficients of the IVs are unreliable for purposes of comparing which IV is more important than another. Multicollinearity is less important in factor analysis, where there is no DV, but even there it can cause sub-optimization.
Adding crossproduct (interaction) terms and power terms sometimes introduces multicollinearity.
Rule of thumb: problem when IV’s correlate > .90.
What is tolerance in the context of multicollinearity? P. 84
Tolerance is is 1 - R2 for the regression of that independent variable on all the other independents, ignoring the dependent. There will be as many tolerance coefficients as there are independents. The higher the intercorrelation of the independents, the more the tolerance will approach zero. As a rule of thumb, if tolerance for an IV is less than .20, a problem with multicollinearity is indicated.
Many statistical packages default screen for tolerance, not including the variable if tolerance is less than .01 or .001. However, this is very stringent, a lot more so than the .20 warning level, so statpack defaults may allow multicollinearity. Tabachnick’s wording leads you to think there are no worries because statpacks screen automatically.
If you are doing grouped analysis (as in the Anova family) you should be concerned with doing separate tests for multicollinearity within each group. This is not a statpack default.
Not a question but memorize Table 4.4. Checklist for Screening Data. P. 85.
Hands-On Data Screening of Ungrouped Data I. Pp. 86-90
In SPSS load in SCREEN.SAV
Analyze, Descriptive Statistics, Frequencies; move all variables to the variable list;
Click the Statistics button and check all outputs for the Dispersion and Distribution panels, and check Means under the Central Tendency Panel, then Continue;
Click the Charts button and select Histograms with normal curve, then click Continue;
Click the Format button and note you could choose to suppress all tables with more than 10 categories, but don't because we want to look for outliers..
Print out the output and be prepared to explain it
Be prepared to turn in a copy of the output with your name on it
Hands-On Data Screening of Ungrouped Data II. Pp. 90-92
Delete the two outliers in ATTHOUSE: Data, Select Cases; click IF for If condition is satisfied, enter formula “atthouse ~= 2" (not ne as in book). OK.
Re-run frequencies and note the mean for atthouse is now 23.634. (This output is not collected)
For missing values of ATTHOUSE, perform mean substitution: Transform, Recode, Into Same Variable; Variables: ATTHOUSE; Old and New Values; Old Value: System-missing; New Value: 23.634.; Add; Continue; OK.
Produce plots to test linearity and homoscedasticity: Graphs, Scatter, Simple; Define, X-Axis: TIMEDRS; Y axis: ATTDRUG. (TIMEDRS is picked because it flunked skew/kurtosis screening; ATTDRUG was picked because it illustrates a nicely distributed variable).
Print out the output and be prepared to explain it
Be prepared to turn in a copy of the output with your name on it
Hands-On Data Screening of Ungrouped Data III. Pp. 92-98
Transform TIMEDRS to normalize it (it failed the skewness/kurtosis screening for normality).
Transform, Compute, Target Variable: LTIMEDRS (a name for the logarithmic transform of TIMEDRS); Functions: LG10(numexpr); Numeric Expression: LG10(TIMEDRS+1). Note the +1 is added because the smallest value of TIMEDRS was 1 and we don’t want a logarithm of 0. OK. SPSS adds LTIMEDRS as the last column.
Analyze, Descriptive Statistics, Frequencies for LTIMEDRS to confirm skew and kurtosis are now within acceptable bounds. (This output is not collected).
Now go on to screening for multivariate outliers using SPSS regression.
Analyze, Regression, Linear; Dependent: SUBNO (recall we want a DV which is not used in the main analysis, so we use subject number); Independents: ATTDRUG, ATTHOUSE, MSTATUS, RACE, LTIMEDRS; Save button, Distances, Mhalanobis; Statistics button: check Collinearity diagnostics.
Look at Collinearity Diagnostics table. (P. 95 in text). A condition index > 30 indicates multicollinearity problem and none are. (But >15 is a warning and two are).
SPSS adds mah_1 as the last column, showing Mahalanobis distances. There are 5 IVs in equation so by Table C.4, p. 932, the critical chi-square value is 20.515. Cases with mah_1 above this are outliers. These are cases 117 and 193.
Create a new variable called DUMMY: Transform, Compute, Target Variable: OUTLIERS; Numeric Expression: 0. OK. This creates a variable filled with 0's. Change case 117 to 1.
We now run a regression with DUMMY as the dependent, to see if we can explain this outlier.
Analyze, Regression, Linear: Dependent: DUMMY; Independents: ATTDRUG, ATTHOUSE, MSTATUS, RACE, LTIMEDRS. Method=Stepwise. Statistics Button: uncheck Collinearity Diagnostics and instead check Estimates and Model Fit. Continue. Save button: uncheck Mahalanobis. Continue. OK.
Print out the output and be prepared to explain it
Be prepared to turn in a copy of the output with your name on it
Barbara G. Tabachnick and Linda S. Fidell (2001). Using Multivariate Statistics, Fourth Edition. Needham Heights, MA: Allyn & Bacon. ISBN 0-321-05677-9. Hardcover.
CHAPTER 5: MULTIPLE REGRESSION
What is the difference between multiple regression and multiple correlation? P. 111
None. Multiple regression generates both a prediction equation and R-square, which is the percent of variance explained. A focus on the prediction equation is reported in terms of “multiple regression” while a focus on percent explained is reported in terms of “multiple correlation,” but it is the same procedure.
What is the general form of a multiple regression equation? P. 112
y = b1x1 +b2x2 ....+ bnxn+ c
where y is the DV
the x’s are the IVs
the b’s are the unstandardized regression coefficients
c is the constant = the y-intercept
What is dummy variable coding and why is it used? P. 112
It is used to include a categorical IV in the analysis when ordinarily regression requires continuous variables or dichotomies.
Each category of a categorical variable is entered as a dichotomous 0/1 variable. One category (the reference category) must be left out to prevent singularity in the data.
If ANOVA is a special case of regression (p. 113), why is it not used interchangeably with it?
1) ANOVA cannot handle continuous variables as it is a grouped procedure. While continuous variables can be coded into categories, this loses information and attenuates correlation.
2) ANOVA normally requires approximately equal n’s in each group formed by the intersection of the IVs. This is equivalent to orthogonality among the IVs. Regression allows correlation among the IVs (up to a point, lower than multicollinearity) and thus is more suitable to non-experimental data. Methods exist in ANOVA to adjust for unequal n’s, but all are problematic.
What are seven things you can do with multiple regression? Pp. 113-114
1) Show the set of IVs does significantly explain variance in the DV. (R-square is significantly different from 0).
2) Figure out the relative importance of the IVs. (Comparing beta coefficients).
3) See if adding an IV to the model helps. (Significance of difference of two R-squares, one with, one without the added IV).
4) Explore possible curvilinear or interaction effects. (Explore curvilinearity by adding a power term explore interaction by adding a crossproduct term).
5) See what effects on a DV can be explained by one or a set of IVs, over and beyond what is explained by an earlier IV or set of IVs. (Hierarchical aka sequential regression).
6) Compare the explanatory power of one set of IVs compared to a correlate alternative set of IVs. (Testing difference of correlated correlations; Tabachnick section 5.6.2.5)
7) Obtaining prediction scores (DV estimates) for a new dataset where only the IVs are known. (Using the regression aka prediction equation).
What is a regression parameter estimate and how is it interpreted? P. 115
It is the b coefficient. When the IV increases 1 unit, the DV increase b units.
As Tabachnick notes (p. 116), regression assumes the independent variables are measured without error. As this is impossible in most cases, what is the point of doing regression?
Error attributable to omitting causally important variables means that, to the extent that these unmeasured variables are correlated with the measured variables which are in the model, the b coefficients will be off. If the correlation is positive, the be coefficients will be too high; if negative, too low. That is, when a causally important variables is added to the model, the b coefficients will all change, assuming that variable is correlated with existing measured variables in the model (usually the case).
It is sometimes said regression assumes proper model specification (inclusion of all important causal variables, exclusion of extraneous non-causal variables) if the b coefficients are to be properly interpreted. This is probably clearer than stating the same thing in terms of IVs measured without error.
How big a sample do you need to do multiple regression? P. 117
Rule of thumb for testing b coefficients: N >= 104 + m, where m = number of IVs)
Rule of thumb for testing R-square: N >= 50 + 8m
If m >= N, regression gives a meaningless solution with R=square = 1.0
Factors requiring a larger N:
skewed DV
small effect sizes to be proved (rule of thumb: N>= (8/f2) + (m-1), where f2 = .01,
.15, and .35 for small medium and large effect sizes)
more measurement error in the IVs
if you are cross-validating training data to test data
if you are using stepwise regression (N >= 40m is a rule of thumb for stepwise,
which can train to noise too easily and not generalize in a smaller dataset).
Is regression robust against outliers? Pp. 117-118
No. Outliers can dramatically affect regression results.
Is regression robust against correlation among the independent variables? Pp. 118-119
No, not if the correlation is high enough to constitute multicollinearity, which will mean the standard errors of the b coefficients will be high, meaning the ratio of the beta weights will not be a reliable guide to the relative importance of the independent variables. The prediction of the regression equation, will not be less reliable.
Note that a corollary is that very high standard errors of b coefficients is an indicator of multicollinearity in the data
What regression plot tests normality, linearity, and homoscedasticity? What exactly do you look for? Pp. 119-121, 154
Plot of the predicted DV on the X axis vs. residual (estimate minus actual values) on the Y axis. In SPSS this is ZRESID vs. ZPRED.
Non-normality: points not equally above and below the Y axis 0 line
Non-linearity: points form a curve
Non-homoscedasticity: points form a funnel or other shape showing variance differs as one moves along theY axis.
What is the impact of violations of normality, linearity, and homoscedasticity? P. 121
Non-normality: Regression consistently underestimates or overestimates the DV.
Non-linearity: By not taking nonlinear correlation into account, R-square underestimates the variance explained overall and the betas underestimate the importance of the variables involved in the non-linear relationship.
Non-homoscedasticity: Two possibilities: (1) there is an interaction effect between a measured IV and an IV not in the model; or (2) some IVs are skewed and others are not.
What is the Durbin-Watson statistic in the context of residual plots? P. 121
It is a measure of auto-correlation. A plot of residuals on the Y axis against the sequence of cases (caseid) on the X axis should show no pattern, indicating independence of errors.
A significant Durbin-Watson statistic indicates non-independence of errors. Positive autocorrelation means standard errors of the b coefficients are too small. Negative autocorrelation means standard errors are too large.
Statnotes: The Durbin-Watson coefficient, d, tests for autocorrelation. The value of d ranges from 0 to 4. A value of 2 indicates no autocorrelation; 0 indicates positive autocorrelation; and 4 indicates negative autocorrelation. As a rule of thumb, d should be between 1.5 and 2.5 to indicate independence of observations.
How can outliers be spotted graphically? P. 122
Note: many prefer the Mahalanobis statistic to graphical methods.
See Figure 5.6, p. 158
As before, plot the predicted DV on the X axis vs. residual (estimate minus actual values) on the Y axis. In SPSS this is ZRESID vs. ZPRED.
Rule of thumb: outliers are those +/- 3.3 standardized residuals (corresponds to .001 level of significance)
Explain the components of the R-squared formula. P. 124. Warning! Formula 5.3 is in error. R-squared is 1 minus the ratio printed in the book, not the ratio itself.
R2 = 1 - SSreg/SSY
where SSreg = regression sum of squares = an estimate of regression error as measured by, for each case, subtracting the predicted from the actual Y, squaring this to get rid of negatives, and summing across all cases
and where SSY = total sum of squares = an estimate of maximum possible error as measured by, for each case, subtracting the default estimate which is the mean of Y from the actual Y, squaring this to get rid of negatives, and summing across all cases
Thus R2 is 1 minus regression error as a percent of total error and will be 0 when regression error is as large as it would be if you simply guessed the mean for all cases of Y.
You may skip the section on matrix compututation of regression parameters. Pp. 124-127.
Be able to read and explain the SPSS output in Table 5.4. pp. 128-129
Model summary table
R-square means 70.2% of variance in compr is explained by motiv, qual, and grade.
Adjusted R-square, however, is only 25.6% of variance explained. Normally there is not that much difference, but Adjusted R-squared penalizes as the number of independents approaches n, and for this dataset n is only 6! (See Systat output in Table 5.6).
Standard error of estimate is 3.891, which means 95% of the time the true value will be within 2*3.891= 7.782 of the estimate. Note we would want to compare 7.782 with the range of compr. If it is the entire range, this is bad news!
ANOVA Table
The overall regression equation is significant at the .411 level. That is, it is not even close to being significant. This partly reflects an n of 6!
Coefficients Table
b’s: compr = .658*motiv + .272*qual + .416*grade - 4,772
std, error: subtracting two standard errors from any of the b’s includes 0 in the range, so we cannot be 95% confident any of the b’s are different from 0
beta: Error on table. Should be no beta for constant. Whole column needs to be pushed down one. Ratio of importance of grade to motiv to qual is .402:.319:.291.
Sig(t): none of the coefficients are significant because none are .05 or less
What is the meaning of the betas in standard multiple regression? P. 131
The betas reflect the unique contribution of each IV. Joint contributions contribute to R-square but are not attributed to any particular IV. The result is that the betas may underestimate the importance of a variable which makes strong joint contributions to explaining the DV. Thus when reporting relative betas, one must also report the r of the IV with the DV as well, to acknowledge if it has a strong correlation with the DV.
What is sequential multiple regression (a.k.a. hierarchical regression)? Pp.131-132.
R-square is attributed to the first IV (or set). R-square difference is attributed to the next IV (or set). Further R-square differences are attributed to further IVs or sets. Compare b and c in Figure 5.2, p. 132.
From StatNotes:
Hierarchical multiple regression is similar to stepwise regression, but the researcher, not the computer, determines the order of entry of the variables. F-tests are used to compute the significance of each added variable (or set of variables) to the explanation reflected in R-square. This hierarchical procedure is an alternative to comparing betas for purposes of assessing the importance of the independents.
For hierarchical multiple regression, in SPSS first specify the dependent variable; then enter the first independent variable or set of variables in the independent variable box; click on "Next" to clear the IV box and enter a second variable or set of variables; etc. One also clicks on the Statistics button and selects "R-squared change."
Compare stepwise multiple regression (a.k.a statistical regression) with ordinary and sequential in terms of assessing the relative importance of IVs. Pp. 133-134.
If the IVs are compared by using the betas on the last step, it is like ordinary regression except the variables have been included for statistical rather than theoretical reasons and thus there is more danger of being arbitrary.
If the IVs are compared using R-Square difference, then it is like sequential regression but the sequence of differences is determined statistically by often arbitrary small differences in partial correlations.
What are special problems with stepwise regression, besides being atheoretical? P. 135
- can be fitting to noise: thus need for cross-validation with a test set
- danger that interactions will be ignored (neither variable alone adds sufficiently to R-squared and stepwise does not consider what happens if they are added as a set.
Which regression method does Tabachnick favor? P. 138
Sequential. She sees this as the most theoretically based as the researcher controls the order of entry of IVs or sets of IVs.
Differentiate squared correlation, squared semipartial correlation, and squared partial correlation in standard and sequential multiple regression. Pp. 139-141
See Figure 5.3, p. 140
Squared (multiple) correlation: proportion of total variance in a DV which the IV explains uniquely or jointly with other measured IVs.
Squared semipartial (part) correlation: proportion of total variance in a DV explained uniquely by the given IV (not counting jointly). When the given IV is removed from the model, R-square will be reduced by this amount. Likewise, it may be thought of as the amount R-square will increase when an IV is added to an existing model. R-square minus the sum of all squared semi-partial correlations is variance explained jointly by two or more IVs (“shared variance”). .
Squared partial correlation: proportion of variance in a DV not explained by other IVs, which is explained uniquely by the given IV (not counting jointly).
Where are part and partial correlation found in SPSS? P. 141
Analyze, Regression, Linear Regression; Statistics button; check “Part and partial correlations”
What is the significance test for the regression model as a whole? Pp. 142-144
It is the probability of F in the Anova table, which is part of the regression output. (Technically, this value is too lenient for stepwise regression and an adjustment to F is needed: see p. 143 and Table C.5 in the appendix).
What is the significance test for individual b coefficients? Pp. 143-144
The probability of the corresponding t-test is used. Warning! The t-test is a test only of the unique variance an IV accounts for, not shared variance.
What is the significance test for semi-partial (part) correlation in SPSS? P. 144
Sig F Change (the F ratio for the change in R-squareds when the IV is added to the model).
What is the significance test for the effect of adding a subset of IVs to the model
(that is, the significance of an R-square difference)? Pp. 144-145.
F-incremental =
[(R-square with - R-square w/o)/m]
divided by
[(1 - R-square with)/df]
where
m = number of IVs in new block which is added
df = N - k - 1 (where N is sample size; k is number of IVs total)
F is read with m and df degrees of freedom
How do you test if one set of predictors is significantly better than another set, for the same DV? pp. 145-147
This is not the R-squared difference test, which deals with nested models.
Testing requires a large sample to assure independence of observations
The formula for this test is given on p. 146.
What is adjusted R-square? Pp. 147-148
It is a conservative downward adjustment in R-square which is needed the more IVs you have, because each IV’s power of explanation may be partly due to its explaining noise in the data. That is, the more IVs, the more over-optimistic R-square becomes. (Tabachnick’s explanation is not good, ignore).
What is a suppressor variable in regression? Pp. 148-149
It is a variable which has a positive correlation with an IV and a negative with the DV (or vice versa) (this is called cooperative or reciprocal suppression).There can also be a suppressor effect if a variable simply serves to control for (suppresses) variance in the IV which is irrelevant to the DV (classic suppression). This might even mean that the regression coefficient might be of opposite sign from the corresponding correlation (net or negative suppression). Including a suppressor in the model will increase the regression weight of the IV with which it is correlated.
Signs a suppressor is at work: the beta is high but the r is low; or the beta and r are of different signs.
Why is centering used when adding interaction or power terms to a regression model? Pp. 151-153
Interactions between IVs are tested by adding crossproduct terms.
Nonlinearity is tested by adding power terms.
The problem is, crossproduct and power terms may introduce multicollinearity.
Multicollinearity can be minimized by centering the IV: subtract the mean from each score. Correlation and unstandardized b coefficients will not change as the result of centering an IV.
If you do find an interaction effect, it is common to run separate regressions for each level of the interacting variable.
Hands-on Regression 1: Evaluating Regression Assumptions
Load REGRESS.SAV into SPSS
Analyze, Regression, Linear; Dependent: timedrs; Independents: menheal, phyheal, stress; click the Plots button: Y: *zresid; X: *zpred; Continue;. OK
Print out the output and be prepared to explain it (this is Regression 1a output)
Be prepared to turn in a copy of the output with your name on it
Analyze, Descriptive Statistics, Frequencies; enter all variables except subject id into the variables list; click the Statistics button and check skewness and kurtosis; Continue; Click the Charts button and click for histograms; check you want normal curves Continue; OK
Print out the output and be prepared to explain it (this is Regression 1b output)
Be prepared to turn in a copy of the output with your name on it
Use Transform, Compute to create three new variables: ltimedrs as LG10(timedrs); lphyheal as LG10(phyheal); and sstress as SQRT(stress).
Repeat with the transformed variables: Analyze, Regression, Linear; Dependent: timedrs; Independents: menheal, phyheal, stress; click the Plots button: Y: *zresid; X: *zpred; Continue;. OK
Print out the output and be prepared to explain it (this is Regression 1c output)
Be prepared to turn in a copy of the output with your name on it
Repeat again, but under the Statistics button ask for collinearity diagnostics and under the Save button ask for Mahalanobis distance.
Print out the output and be prepared to explain it (this is Regression 1d output)
Be prepared to turn in a copy of the output with your name on it
Hands-on Regression 2: Standard Multiple Regression. Pp. 159-164
Analyze, Regression, Linear Regression again, but on the Statistics button ask (only) for confidence intervals, descriptives, part and partial correlations, collinearity diagnostics, Durbin-Watson, and casewise diagnostics. Continue. On the Plots button, de-select all plots. Continue. On the Save button, deselect Mahalanobis. Continue. OK.
Print out the output and be prepared to explain it (this is Regression 1d output)
Be prepared to turn in a copy of the output with your name on it
Regression 3: Sequential Regression. Pp. 165-170.
No output is to be turned in, but be prepared to explain the output in the book.
If you want to do it anyway: Analyze, Regression, Linear; Dependent: ltimedrs; Block 1 of 1, Independents: lphyheal; Next; block 2 of 2, sstress; Next; Block 3 of 3, menheal’ Plots button, Y: *zresid; X: *zpred.
Chapter 6: Canonical Correlation
1. What is canonical correlation? Pp. 177-178.
It is the correlation between two sets of variables, as sets. A linear combination of one set of variables is correlated with a linear combination of another set of variables. These combinations are called canonical variates. Thus a canonical correlation is the correlation between a pair of canonical variates and as such represents a dimension on which the two sets can be related. There can be second, third, or more dimensions, each with its own different linear combination of variables (variates).
2. What are the four research questions canonical correlation can answer? Pp. 78-179
1. How many dimensions are needed to account for the correlation between two sets of variables? That is, in how many different ways are they related?
2. What are these dimensions? What should we label them?
3. For any dimension, how strongly is the pair of variates correlated? How strongly is each variate related to the variables in its own set? In the other set?
4. For each person or case in the dataset, what is their score on each canonical variate? Having canonical scores can help us spot outlier cases. We could also use scores as variables in other types of statistical procedures.
3. Are canonical variates like factors in factor analysis? Pp. 179-180
Yes and no. In both canonical correlation and factor analysis, linear combinations of raw variables are constructed, and these are latent variables. However, factor analysis is a procedure without dependents and there are no pairs of correlated latent variables. Also, in canonical correlation, factors usually are not rotated.
4. Enumerate key assumptions of canonical correlation. Pp. 180-181.
* Linearity: To the extent variables are related curvilinearly, canonical correlation miss the relationship unless variables are exponentiated or otherwise transformed in a curvilinear fashion first.
* Homoscedasticity and other assumptions of correlation are required, as the covariates are created from the correlation matrix.
* Missing data: canonical correlation can be quite sensitive to this.
* Absence of outliers: canonical correlation can be quite sensitive to this.
* Multivariate normality is needed for significance testing, but for large samples, the central limit theorem states that normality may be assumed for the canonical variates.
* Absence of multicollinearity: To the extent that the variables within the independent sets of variables are highly intercorrelated, the canonical coefficients will be unstable. The coefficients for some variables may be misleadingly low or even negative because variance has already been explained by other variables.
5. What is an eigenvalue in canonical correlation? Pp. 183-184
It is the squared canonical correlation between variates in a pair, and thus is the percent of variance that one set explains in the other set for that dimension. There will be as many eigenvalues as there are significant dimensions relating the two sets of variables.
6. What are the two ways the significance of a canonical correlation can be evaluated? Pp. 184-185
There are both chi-square and F test alternatives.
The chi-square test has (k1-1)(k2-1) d.f., where k1 and k2 are the number of variables in sets 1 and 2
Canonical correlation can be computed within SPSS MANOVA, which uses an F test.
7. What are canonical loadings? What are they used for? Pp. 188-189
Loadings are the correlations of the raw variables with the covariate for their set. A given raw variable will have a loading on each dimension’s covariate for their set. See Figure 6.1, p. 188.
You can use the loadings to give a label to the dimensions.
For a given dimension, if you add the squared loadings for all raw variables in the set and divide by the number of raw variables, you have the (average) percent of variable which the covariate accounts for in the raw variables in its set. You would like this percentage to be high in a good model.
8. Interpret Table 6.6, pp. 196-197
* Correlations for Set-1: the r’s for one side of the canonical correlation (TS and TC)
* Correlations Between Set-1 and Set-2: r’s between raw variables on one side and those on the other.
* Tests that remaining correlations are zero: The rows are the dimensions (the canonical correlations). Wilk’s lambda test of significance is applied to each and shown in the last Sig. column. As you go down the rows, the dimensions will be less and less significant. This is referred to as “peel down” significance testing.
* Canonical loadings for Set-1: loadings on dimensions 1 and 2 for the S-side variables (TS and TC) . Here squared loadings add to 100%, but this only happens when you have two variables in the set.
* Cross loadings for Set-2: the raw variables in one set should be substantally less crossloaded on the other variate than they are on their own variate.
9. Explain redundancy analysis. Pp. 198-199
Redundancy is the percent of variance in one set of variables accounted for by the variate of other set. You want redundancy to be high, showing the independent set accounts for a high percent of the variance in the dependent set.
Note this is NOT the canonical correlation squared, is which how much the variate of one set accounts for in the variate of the other set.
10. How high should the canonical correlation be for a dimension to be of interest? P. 199
Tabachnick suggests at least .30, since that means
Hands-on example for Chapter 6:
Timedrs, phyheal, druguse, and attmar require transformation to make them more normally distributed. Use Transform, Compute, to create log10 transforms of these variables. Note that you must add +1 to the variables before taking the log if that column contains zeros (this applies to timedrs and druguse). Using Compute in this manner, create ltimedrs, lphyheal, ldruguse, and lattmar.
The object is to correlate two sets of variables. Set 1 is esteem, control, lattmar, and attrole. Set 2 is ltimedrs, attdrug,lphyheal, menheal, and ldruguse.
Check for collinearity using factor analysis. This is done twice, once for each of the two sets. Analyze, Data Reduction, Factor; Variables: esteem, control,lattmar, attrole; Extraction button; Method: Principal axis factoring. Continue; OK. This is outputs 6a and 6b.
Be prepared to interpret this output.
Get the actual canonical correlation output. Canonical correlation has to be run in syntax, not from the SPSS menus. Open the syntax window with File, New, Syntax. Enter this:
INCLUDE 'c:\Program Files\SPSS\Canonical correlation.sps'.
CANCORR SET1=esteem,control,lattmar,attrole/
SET2=ltimedrs,attdrug,lphyheal,menheal,ldruguse/.
Note the period at the end of lines 1 and 3. This gives output 6c.
Be prepared to interpret this output.
Chapter 7. Multiway Frequency Analysis
1. What is another name for multiway frequency analysis? P. 219
log-linear analysis (of tables)
2. Explain zero association for 1-way, 2-way, and 3-way relationships. P. 219
1-way: same frequency in each frequency bin (ex., histogram) would be a zero 1-way association for that variable
2-way: expected frequency in each cell of a table would be zero 2-way association (where expected = row times column marginal divided by n)
3-way: same as 2-way, for all the subtables for two variables formed by the classes of a third variable.
3. What are the six research purposes of MFA according to Tabachnick? Pp. 220-222.
1. Are variables associated?
2. What are the odds of a DV class given levels of the IVs (effect on a DV)?
3. What are the expected frequencies (table cell values) of the DV (parameter estimates)?
4. How can the IVs be ranked by relative importance (importance of effects)?
5. What is model goodness of fit for the data (strength of association)?
6. Which parts of the model contribute the most to the association (specific comparisons and trend analysis).
4. What are the four assumptions of MFA discussed by Tabachnick? Pp. 222-224
1. Independence: no repeated measures, each observation sampled independently.
2. Large enough n: Tabachnick recommends 5 times the number of cells in the tables.
3. Sampling adequacy: All expected frequencies for cells > 1; no more than 20% of cells < 5. Sampling inadequacy reduces power, but so does collapsing categories. Note this test is for expected frequencies, not observed frequencies.
4. No outliers: Very large |O - E| , where E’s are based on the best-fitting log-linear model, may mean that no model fits the data adequately. Adding new variables and/or collapsing categories my improve model fit. In any event, residuals should be analyzed to understand where the model is working best or least well.
5. Where does chi-square fit into MFA? P. 225.
To test association in a two-way table.
6. What is used instead of chi-square for multi-way tables? P. 225.
The likelihood ratio, G-square.
7. Explain Table 7.5. P. 233.
* Overall (total row) the log-linear model fits the data.
* The 3-way association of Reading Type, Sex, and Profession is not significant.
* Of the 2-way associations, only Sex by Profession is significant (which is bad, because reading type was the DV variable of interest)
* Of the 1-way associations, only Reading Type is significant. This means that Reading Type was significantly different from a flat distribution (55 SCIFI, 100 SPY), whereas Sex (80 M, 75 F) and Profession (55 Politicians, 55 Administrators, 45 Belly Dancers) were not.
8. What are effect parameters? Pp. 233-234.
The natural logarithm of the expected frequency in any cell is equal to the sum of the effect parameters plus constant. Let n be the number of variables. There will be a 1-way effect for each of n variables; n!/(n - 2)!2 two-way effects; n!/(n - 3)!3! three way effects; and if n>3, then n!/(n - r)!r! r-way effects. Thus for n=3, there are 3 one-way effects; 3 two-way effects; and 1 three-way effect. Overall, there will be (2n - 1) terms plus a constant in the full (saturated) equation for the log of the expected frequency, where n=number of variables). (Note: 0! = 1).
9. What is loglinear modeling? Pp. 234-235.
A full (saturated) equation will always fit the data perfectly and thus trivially. The purpose of log-linear modeling is to drop terms from the equation to find the most parsimonious (incomplete) model which best predicts the expected values.
10. For a loglinear model, do we want G-square to be significant or insignificant? P. 234
Insignificant, showing there is no significant difference between the observed and expected cell frequencies.
11. What are nested (hierarchical) models? Why are they the rule in loglinear analysis? P. 234.
In a nested model, one model is a subset of the other being compared. If models are hierarchical, then the difference in G-squareds can be tested for significance (using the difference in df’s) to see if the subset model is not significantly worse than the less parsimonious superset model. Again, we want G-square to be non-significant to conclude that the smaller model would be preferred on the basis of parsimony.
12. What are residuals used for? Pp. 235-237.
Once you pick the most parsimonious model, SPSS can compute the expected frequencies on that basis (for instance, in the textbook, for the model where it is assumed that only the two-way Sex-Professions and the one-way Reading Type associations are predictive). Then, on a cell by cell basis, it can subtract the Observed from the Expected frequencies, giving the residuals. Cells with large residuals are where the model is not working well; small residuals mean it is.
Residuals are shown in Table 7.7, p. 237. In this case, the largest residual is for male politicians preferring science fiction (4.4 fewer cases than would be expected on the basis of the Sex-Professions relationship and the one-way Reading Type relationship). This residual is a discrepant case which may require a different explanation.
13. What are parameter estimates used for? Pp. 236-241.
Computation of parameter estimates is discussed on pp. 236-239.
The parameter estimates are used to compute expected values.
Standardized parameter estimates (raw parameter estimates divided by their standard errors) are interpreted like other standard units: if above 1.96, then p<.05; if above 2.575 then p<.01. The most important influences on any given cell are the ratios of absolute magnitude of the standardized parameter estimates for the rows and columns for that cell.
See last paragraph of p. 240, comparing it to table 7.9 on p. 239.
14. How is testing the significance of loglinear models different from testing the significance of loglinear coefficients (effects)? P. 251.
* Models: We want G-square (the likelihood ratio) not to be significant for a good fit.
* Effects: We want the chi-square tests for partial effects or z tests for parameter estimates to be significant to retain that parameter.
Hands-on Exercise for Chapter 7 (MFA)
Run SPSS. File, Open, and navigate to the Tabachnick data located at Y:\PC\CLASS\PA765d\Datasets\Tabachnick. Then Load MFA.sav. Note the key variables:
AWARE: if psychologist thought clients were aware they were attracted to them
BENEFIT: if psychologist thought attraction was beneficial to therapy
HARM: if psychologist thought attraction was harmful to therapy
CONSULT: if psychologist sought consultation after becoming attracted to client
DISCOMF: if psychologist felt uncomfortable as result of attraction
Output 7a: Use SPSS Crosstabs to evaluate the adequacy of the expected frequencies (no cells < 0; no more than 20% <5). First make sure only in-range cases are used: Data, Select Cases; select the “If condition is satisfied” radio button; click the If button. In the Select Cases: If dialog box, enter this formula: aware < 3 & benefit < 3 & harm < 3 & consult < 3 & discomf < 3. Continue. Unselected Cases Are: Deleted. OK. (Nothing seems to happen, but subsequent analyses will apply only to the in-range cases).
Ask for the set of crosstabs from table 7.14 in the textbook (note this is different from the workbook, whose specifications should be ignored). . Analyze, Descriptive Statistics, Crosstabs; Rows: AWARE, BENEFIT, HARM; Columns: CONSULT, DISCOMF. Click the Cells button: Counts: check Observed and Expected. Percentages: Check Row and Column. Continue. OK. The repeat three more times to get the full set of all possible 2x2 tables: (1) AWARE by BENEFIT, HARM; (2) BENEFIT by HARM; (3) CONSULT by DISCOMF.
These crosstabs are output 7A. Bring to class and be prepared to discuss.
Output 7B. Conduct a hierarchical log-linear analysis of the data. Analyze, Loglinear, Model Selection; Factors: AWARE (1,2), BENEFIT (1,2), HARM (1,2), CONSULT (1,2), DISCOMF (1,2). Note you must click the “Define Range” button after each factor to enter the lower and upper bounds (1,2). In the Model Building area, check the “Enter in single step” radio button. Options: check Frequencies, Residuals, Association table. Delta=0. Continue. OK.
[What is Delta? For saturated models, by default SPSS adds .500 to the count for the observed value for all cells. By setting Delta as above to 0, this is suppressed.]
This is output 7B. Bring to class and be prepared to discuss.
Output 7C. Create a model using all 10 two-way effects, with backward elimination. Do Analyze, Loglinear, Model Selection again, but in the Model Building area, click the “Use Backward Elimination” radio button. Click the Model button. In the Model dialog, select the Custom radio button. In Build Terms set the pull-down menu to “All 2-Way,” then move all the factors to the Generating Class box. (If you have done this correctly, you will see there a list of the 10 possible 2-way relationships, like aware*discomf). Continue. OK.
This is output 7C. Bring to class and be prepared to discuss.
Output 7D. Produce the parameter estimates. Unfortunately, this has to be done in syntax in SPSS, not from the menu mode. File, New, Syntax. The cut and paste this code (taken from Table 7.18, p. 264, of Tabachnick):
LOGLINEAR
aware(1 2) benefit(1 2) harm(1 2) consult(1 2) discomf(1 2)
/PRINT=ESTIM
/DESIGN aware*benefit aware*consult aware*harm benefit*consult benefit*harm consult*discomf consult*harm discomf*harm aware benefit consult harm discomf.
Note this models the eight significant two-way effects and the five one-way effects. Run, All, in syntax to get the output.
This is output 7D. Bring to class and be prepared to discuss.
Chapter 8: Analysis of Covariance
1.What is ANCOVA, comparing it to ANOVA? P. 275
Where ANOVA tests if one or more independents cause the group means of a dependent to differ significantly, ANCOVA does the same but first factors out the variance in the dependent attributable to one or more covariates (control variables).
2. What three purposes does Tabachnick ascribe to ANCOVA? P. 275
(A) To reduce the error term
(B) To perform “what if” analysis for the situation where all respondents scored equally on the covariates. (DG: thhat ais is really another way of saying (A))
(C) ANCOVA is also part of MANCOVA, where the dependent is controlled for both covariates and additional dependents as controls.
3. Can ANCOVA be used in all ANOVA designs? P. 277.
Yes
4. What are the six types of research questions that can be answered with ANCOVA? Pp. 278-279
(1) Are the main effects of the independent(s) significant, controlling for the covariates?
(2) Are there significant interaction effects among the independents, controlling for the covariates?
(3) When there is a single independent with more than two levels (values), which values contribute the most to any relationship which may be found? (This question may be a planned comparison or a post hoc comparison; the statistics differ depending on this).
(4) What is the effect of the covariate(s)?
(5) What are the effect sizes of the main and interaction effects?
(6) What are the values of the parameter estimates? (How do the dependent group means differ after adjusting for the covariates?
5. Why does Tabachnick recommend only a very small number of covariates be used? P. 279
ANOVA assumes that the covariates are independent of each other and correlated with the dependent. When there are many covariates, the residual correlation of any one covariate with the dependent is apt to be small. Since you lose one degree of freedom for each covariate added, the gain in explanatory power may well be offset by reduction in the power of the test when there are many covariates.
6. Does an experimental use of ANCOVA differ from its use with nonexperimental data? P. 280
Yes
Experimental: Randomization of subjects into groups already controls for all possible covariates. If you introduce via ANCOVA a covariate which is correlated with the treatment (the independent), you will underestimate the effect of the treatment. And if the covariate is uncorrelated with the treatment, it is controlling for the same things as randomization and is difficult to interpret.
Non-experimental: Since there is no randomization of subjects, using covariates is the next best thing and is more or less essential.
7. What are between-subjects and within-subjects designs? P. 280. Hint: Recall this from Chapter 3.
Between-subjects: The ordinary form of analysis, where different subjects have different values on the independent variables and you are comparing values between different subjects.
Within-subjects: Also called repeated measures designs, this is where the same subjects have different values on the independent variables at different times of measurement, and you are comparing these before-after scores within subjects (within=for the same subjects) to see if the subject changes as a result of some treatment or other independent effect.
8. Is ANCOVA sensitive to outliers? P. 281
Yes. There are tests for both univariate and multivariate outliers.
9. Is multicollinearity a problem in ANCOVA? P. 281
Yes. ANCOVA has built-in regression procedures, and regression is sensitive to multicollinearity. Tabachnick recommends dropping from the analysis any covariates which have a squared correlation of .5 or higher with another covarate.
10. Do you need to have normally distributed data for ANCOVA? P. 281
Mostly no. The variables themselves do not need to be normally distributed, but ANCOVA does assume normally distributed group means. (Groups refers to groups of the dependent as partitioned by the values of the independents). By the central limit theorem,. group means will be normally distributed if sample sizes are large enough. Tabachnick says df=20 is enough to satisfy this requirement if group sizes are relatively equal, there are no outliers, and two-tailed tests are involved. If any of the three latter conditions do not apply, df must be higher.
11. What is the homogeneity of variance assumption? P. 281
ANOVA assumes each group has a similar variance on the dependent. ANCOVA also assumes each group has similar variance on the covarates. If not, the covariate must be dropped or the significant cut-off raised (ex., from .05 to .025).
12. What is the linearity assumption? P. 282
Each covariate must have a linear relationship with other covariates and with the dependent. Of course, variables can be transformed to deal with some forms of curvilinearity. If there is remaining curvilinearity, the power of significance tests is reduced (more chance of thinking you don’t have a relationship when you do..you err in a conservative direction)./
13. What is the homogeneity of regression assumption? P. 282
The slope of the regression line linking the covariates to the group means of the dependent should be similar. When the slopes differ, this means the relationship of the covariates to the dependent differs according to groups formed by the values of the independent(s). That is, there is an interaction effect between the covariate(s) and the independent(s). ANCOVA is not appropriate when there is interaction between the covariates and the independents. Therefore, homogeneity of regression should be tested.
14. ANCOVA assumes perfectly reliable covariates (measured without error). What happens when measurement is imperfect? P. 283.
Power of the significance tests is reduced and in experimental research you may make errors in the conservative direction (more chance of thinking you do not have a relationship when you do). With nonexperimental data, errors may be either Type I or Type II. While there are adjustments for lack of reliability, they are guess-timates on which statisticians differ. Tabachnick recommends not using covariates whose coefficient of reliability is not .8 or higher.
15. What is the F test in ANCOVA? P. 288
It is the overall test of significance for the ANCOVA model. It is the ratio of the mean square between groups (that is, the mean square of the treatment or independent variable) to the mean square within groups (that is, the mean square error). You look up the F ratio in an F table, using df=k-1 and df=N-k-1, where k=number of levels of the independent, N=sample size. If the corresponding significance level is .05 or less, the ANCOVA model is significant and the independent variable significantly affects the dependent variable.
16. What is eta-square? P. 288, 301
It is a measure of effect size (association). Eta-square equals the ratio of the sum of squares between groups (the treatment or independent variable sum of squares) divided by the sum of squares for all main, interaction, and error effects (but not the covariate or mean sum of squares reported by some computer programs) Eta-square is the percent of variance in the dependent variable explained by the independent variable.
17. Read and explain the output in Table 8.6. p. 289.
(1) TREATMNT is the independent variable, which forms the groups
(2) POST is the dependent variable
(3) PRE is the covariate
(4) TREATMNT has three levels (values) so there are three groups
(5) The main effect (TREATMNT) is significant at the .045 level
(6) The covariate effect (PRE) is not significant, at the .084 level.
(7) Eta-square, not printed is 137.895/(137.895+149.439) = .71
18. How is the homogeneity of regression assumption tested in SPSS? P. 292
This can be done in SPSS MANOVA. In Table 8.8, the effect of covariate by independent (here, PRE by TREATMNT) should not be significant, to show there are no significant interaction effects between the covariate and the independent. Here it is not significant, p=.967.
19. What difference does having repeated measures (within-subjects design) make? Pp. 203-294.
The interpretation is largely the same, but the calculation differs.
Also, you have to test for the additional assumption of sphericity.
20. What is the difference between traditional and GLM approaches to within-subjects ANCOVA? P. 293
The GLM adjusts (controls) the values of the independent variable(s) for interactions with the covariates. The traditional approach merely assumes the covariates are uncorrelated with the independents. If there is no interaction effect of the covariates with the independents, the GLM approach will incur a loss of degrees of freedom and thus be a less powerful test for the same data.
21. What are paired comparisons in ANCOVA, and what are planned vs. post hoc comparisons? P. 298
After the omnibus F test establishes an overall relationship, the researcher can test differences between pairs of means, assuming the independent has more than two levels. Ideally these comparisons are based on a priori theory and there are just a few of them. But if the researcher wants to investigate all possible paired comparisons on a post hoc basis, some will be found significant just be chance, so there are various adjustments (Bonferroni, Tukey, Scheffe) which make it harder to find significance. If the independent is nominal, the coefficients being tested are contrast coefficients; if the independent is ordinal or interval, the coefficients are trend coefficients.
22. What is blocking in ANCOVA? P. 304
The continuous covariate is classified (ex., high, medium, low) and used as an additional independent variable in an ANOVA. Its main effect is the effect of the covariate. If there is an interaction effect, this shows the homogeneity of regressions assumption would have been violated in ANCOVA.
This has the advantage that one need not assume the relationship between the covariate and the dependent variable is linear.
However, classification involves loss of information and attenuation of correlation. If the covariate is related to the dependent in a linear manner, ANCOVA will be more powerful than ANOVA with blocking. Also, blocking after data are collected may involve unequal group sample sizes, which also makes ANCOVA less robust.
Chapter 8 Hands-On Exercise
Load in ancova.sav.
Normalize two of the variables: Create lphyheal as a log of phyheal; create lpsydrug as a log of (psydrug+1). You have to add 1 to psydrug because its raw values contain 0's, and you cannot take the log of 0.
Check homogeneity of regression. This must be done through syntax, not the menu system in SPSS. File, New, Syntax will open the syntax window. Enter:
MANOVA
attdrug BY religion (1 4) emplmnt (1 2) WITH lphyheal lpsydrug menheal
/PRINT DESIGN (COLLINEARITY)
/NOPRINT PARAM (ESTIM)
/METHOD=unique
/ERROR WITHIN+RESIDUAL
/analysis attdrug
/DESIGN lphyheal lpsydrug menheal religion emplmnt,
religion by emplmnt, pool (lphyheal lpsydrug menheal) by religion
+pool (lphyheal lpsydrug menheal) by emplmnt
+pool (lphyheal lpsydrug menheal) by religion by emplmnt.
Be prepared to interpret the resulting output.
Proceed with analysis of covariance of drug use using GLM Univariate: Analyze, General Linear Model, Univariate, Dependent Variable: attdrug; Fixed Factors: religion, emplmnt; Covariates: lphyheal, lpsydrug, menheal; click the Model button, set Sum of Squares to Type I; sheck Include intercept in model; Specify Model: full factorial; Continue; click Options button, Display Means for religion, emplmnt, religion*emplmnt; Display: Descriptive statistics; Significance level: .05; Continue; OK button.
Be prepared to interpret the resulting output.
Proceed with evaluation of covariates: Analyze, General Liberal Model, GLM-Multivariate; Dependent Variable: attdrug, lphyheal,menheal,lpsydrug; Fixed Factor(s): emplmnt, religion; Options; Display: Residual SSCP matrix; Significance level: .05; Continue; OK.
Be prepared to interpret the resulting output.
Chapter 9: Multivariate Analysis of Variance and Covariance
1. What is the key way in which MANOVA is different from ANOVA? Pp. 322-323
In MANOVA there are multiple dependents.
2. . What are the eight types of research questions that can be answered with MANOVA and MANCOVA? Pp. 325-327
(1) Are the main effects of the independent(s) significant, controlling for the covariates?
(2) Are there significant interaction effects among the independents, controlling for the covariates?
(3) When there is a single independent with more than two levels (values), which values contribute the most to any relationship which may be found? (This question may be a planned comparison or a post hoc comparison; the statistics differ depending on this).
(4) What is the effect of the covariate(s)?
(5) What are the effect sizes of the main and interaction effects?
(6) What are the values of the parameter estimates? (How do the dependent group means differ after adjusting for the covariates?
These last two are different from ANCOVA:
(7) Which of the dependents is changed by the independents?
(8) For repeated measures designs, each response set (each time period of measurement) can be viewed as one of multiple dependent variables and analyzed with MANOVA.
3. Does MANCOVA have the same assumptions (limitations) as ANCOVA? P. 328-331
Yes
* Unequal sample sizes reduce the power of the analysis. With multiple dependents and multiple independents, there can be many groups and the chance that there are empty cells or unequal cells is greater than with ANCOVA. Every cell must have more cases than there are dependents.
* Multivariate normality is assumed for the distribution of means of the multiple dependents. By the central limit theorem, normality can be assumed if the smallest cell has a large enough number of cases (Tabachnick gives 20 as a suggested cut-off).
* No outliers: MANCOVA is highly sensitive to outliers; the researcher must test for outliers, which can cause either Type I or Type II errors.
*Homogeneity of variance-covariance structures: Box’s M tests this. You want M not to be significant. However, M is very conservative and may well show you have violated this assumption. Tabachnick recommends ignoring M is sample sizes are relatively equal (p. 330) and if they are not, test at the p=.001 level. If for unequal sample sizes, M is significant at the .001 level, conclude that significance tests in MANCOVA may not be robust. In general, if cells with large sample size have larger variances and covariances, then significance tests are too conservative (you may be making Type 2 errors: thinking you do not have something when you do). If cells with small n have larger variances and covariances, significance tests are too liberal (you may be making Type 1 errors: thinking you have a relationship when you do not).
* Linearity. Relationships among dependents, among covariates, and between covariates and dependents are assumed to be linear. Deviations from linearity reduce the power of significance tests (you are more likely to make Type 2 errors, thinking you do not have something when you do). Tabachnick recommends dropping covariates and transforming dependents which exhibit curvilinearity.
* Homogeneity of regression. Roy-Bargmann stepdown analysis assumes the regression between the covariates and the dependent are the same across groups.You set a priority order to the dependents, then add one at a time as covariates in an ANCOVA analysis. The slopes should be similar at each step. If they are not at a given step, this means there is an interaction effect of the added covariate=dependent, and that variable is dropped from subsequent steps.
* Reliability of covariates. The less reliable the covariates, the less powerful the overall F test of the MANCOVA model.
* Absence of multicollinearity and singularity. Redundant dependents should be dropped (if you have to have all dependents, factor analyze them and use the component scores as the dependent).
4. Interpret Table 9.3. p. 340.
* The dependent variables are wratr and wrata
* The independent factors are treatment (2 levels) and disablty (3 levels)
* There are no covariates, so this is MANOVA, not MANCOVA