The McNemar test assesses the significance of the difference between two dependent samples when the variable of interest is a dichotomy. It is used primarily in before-after studies to test for an experimental effect. Also for two dependent samples, the marginal homogeneity test is an extension of the McNemar test for multinomial variables (more than two categorical values). The sign test and the more powerful Wilcoxon signed-ranks test are for two dependent samples when the variable of interest is continuous.

Key Concepts and Terms

• Dependent samples, also called related samples or correlated samples, occur when the response of the nth person in the second sample is partly a function of the response of the nth person in the first sample. There are two common forms of sample dependency: (1) before-after and other studies in which the same people are surveyed at different points in time, including panel studies; and (2) matched-pairs studies in which similar people are surveyed at different points in time.

• McNemar test
  • Symmetry. McNemar's test is sometimes called McNemar's test of symmetry or McNemar symmetry chi-square because it, and the marginal homogeneity test which extends it beyond dichotomous data, apply to square tables in which the diagonal reflects subjects who did not change between the before and after samples (or matched pair samples). The test of symmetry tests whether the counts in cells above the diagonal differ from counts below the diagonal. If the two counts differ significantly, this reflects change between the samples, such as change due to an experimental effect between the before and after samples.

  • Computation of the McNemar Test. For the McNemar test, data are arranged as below:

    		Time 1	Time 1
    		Exposed	Not Exposed
    Time 2	Exposed	a	b
    Time 2	Not Exposed	c	d

    Note that you cannot have a simple table where the columns are "Time 1" and "Time 2", and the rows are "Exposed" and "Not Exposed", because then observations would appear twice in the table!

    The McNemar test uses the chi-square distribution, based on this formula:

    a	b	r1
    c	d	r2
    c1	c2	n

    Chi-square = (|a - d| - 1)²)/(a + d) degrees-of-freedom = (rows - 1)(columns - 1) = 1

    Example:

    6	20
    10	14	24
    16	34	50

    Chi-square = (|6 - 14| - 1)²)/(6 + 14) = 49/20 = 2.45 d.f. = (2 - 1)(2 - 1) = 1

    
    

  • McNemar test in SPSS. Select Analyze, Descriptive Statistics, Crosstabs; select Exposure 1 (coded 0=No; 1=Yes) as the column variable and select Exposure 2 as the row variable; click Statistics; select McNemar

  • Interpreting McNemar test chi-square. Using a table of the distribution of chi-square, with 1 degree of freedom, if the computed chi-square is less than the critical value found in the table for the desired significance level (usually .05), then the difference between samples (ex., the difference between the before and after samples) is not significant. Thus, at the .05 level of significance, the critical value of chi-square is 3.841; since 2.45 computed for the example above is less than this, the difference between samples is not significant.

• The marginal homogeneity test is similar to the McNemar test, extending it to the case where the variable of interest assumes more than two nominal values. While it may be used with ordinal data, the sign test is preferred.

• The sign test is for two dependent samples, where the variable of interest is ordinal or higher. It examines the sign (plus, minus, tied) of the difference between data pairs (ex., between the 1st subject in the 1st sample and the 1st subject in the 2nd sample, etc., on up to the nth pair). When there is no significant difference between the two samples, the number of positive and negative differences will be the same or similar in the two samples. While the sign test may be used with interval data, the Wilcoxon test is preferred.
  • Computation of the sign test. Ties are be assigned to the plus or minus category by use of a random number table. The computation of the sign test is as for to the binomial test, once data have been recoded to plus/minus (1, 0) format, with p = q = .5, n = the number of paired observations, and r = the number of pluses or minuses, whichever are fewer.

• The Wilcoxon signed-rank test, T, sometimes called the Wilcoxon matched-pairs test, is also for two dependent samples where the variable of interest is interval. However, it is more powerful than the sign test because it takes more information into account. Specifically, the Wilcoxon test factors in the size as well as the sign of the paired differences.
  • Computation of Wilcoxon's T
    1. List the paired scores in two columns corresponding to the two dependent samples.
    2. Subtract each score in the second column from its pair in the first column, creating a third column of differences.
    3. Delete pairs with zero differences (ties).
    4. Rank the differences from smallest (including negative) to largest absolute difference, from 1 to n, the sample size. Assign the average of ranks which would otherwise have been assigned in the cases of tied pairs.
    5. Let T be the smaller of the sum of ranks for positive differences, or the sum of ranks for negative differences.
    6. If computed T is equal to or less than the critical value of T in a table of the distribution of Wilcoxon's T, then the two samples can be said to differ significantly. In actual practice, computer programs like SPSS compute the probability of T for the researcher, without need for recourse to a table of critical values.

Assumptions

• Value distribution. The McNemar test is used when variables are assumed to be dichotomies which are mutually exclusive and exhaustive. For two dependent samples and multinomial categorical data, the marginal homogeneity test is used. The sign test and the Wilcoxon signed-ranks test assume a continuous value distribution, with the latter requiring interval or near-interval data.

• Data distribution. The tests in this section are all nonparametric (do not assume the normal distribution, or any other particular distribution other )

• Two samples. Data are from two samples, not necessarily from the same population. There is another set of tests for more than two dependent samples.

• Dependent samples. The two samples may be before-after samples or panel studies of the same subjects, or matched-pairs samples of similar subjects, or may be otherwise dependent (correlated). It is not required that they be dependent. For independent samples, for instance, the Wilcoxon test is directly related to the Mann-Whitney U test for two independent samples: U plus W sum to the constant m(m + 2n + 1)/2, where m is the number of observations in the smaller group and n is the number in the larger group. For this reason one would not use both tests for independent samples, and U is much more prevalent for this purpose.

• Adequate sample size. For the McNemar test, the number of cases on the a-d diagonal (see the illustration above) should be at least 10. If it is not, the binomial test is used with n = a + d and with r = a or d, whichever is smaller, as defined in the section on the binomial test.

Frequently Asked Questions

• Where is these tests (McNemar, marginal homogeneity, sign, Wilcoxon) found in SPSS?
  From the SPSS menu, select Statistics, Nonparametric Tests, 2 Related Samples. In the "Two Related Samples Tests" dialog box, check the desired test under "Test Type."

Bibliography

• Siegel, Sidney (1956). Nonparametric statistics for the behavioral sciences. NY: McGraw-Hill. A standard reference work.