Overview This set of significance coefficients tests whether an ordinal or interval variable measured in each of two independent samples can be assumed to come from the same underlying population. A finding of significant difference indicates that the two samples differ on the variable of interest. The Mann-Whitney U test is the most popular for this purpose, with alternatives being the Wald-Wolfowitz test and the Kolmogorov-Smirnov test. The Moses extreme reactions test is also for two independent samples but has a specialized use dealing with detecting a treatment effect in an experimental setting, where the treatment is assumed to have a polarizing rather than unidirectional effect. It should be used instead or in addition to one of the other three tests whenever the researcher suspects that two samples may differ in the tails rather than in the center of their distributions. In SPSS, these tests require that the Exact Tests add-on module be installed.

Contents Key concepts and terms Mann-Whitney U Test Wald-Wolfowitz Runs Test Kolmogorov-Smirnov Z Test Moses Test of Extreme Reactions Assumptions Frequently asked questions Bibliography

Key Concepts and Terms

• Independent samples. Samples are independent if the response of the nth person in the second sample is not a function of the response of the nth person in the first sample. Independent samples are also called uncorrelated samples and unrelated samples. Samples which are not independent include before-after and panel studies of the same people, or matched-pairs studies of similar people.

• SPSS user interface. The SPSS user interface, accessed by selecting Analyze, Nonparametric Tests, 2 Independent Samples.

  

• Type of significance estimate. The Exact button in the SPSS dialog above allows the researcher to select among asymptotic, exact, or Monte-Carlo estimates of the significance of the particular test value, as illustrated below. These three types of estimates are discussed separately in the section on significance testing. This requires that the SPSS Exact Tests add-on module be installed.
  

• Mann-Whitney U Test. The most widely-used significance test for comparing two independent samples, the Mann-Whitney U test may be used to test differences in ordinal data or higher. That is, for rank data, U tests to see if the ranks are randomly mixed between two samples.
  • In SPSS, select Analyze, Nonparametric Tests, 2 Independent Samples; select your variables; click Define Groups and select the grouping variable; under Test Type, select Mann-Whitney U.

  • Computation of Mann-Whitney U. The general proces is to combine observations from each of two independent groups, listing them in rank order (an average rank is assigned for ties). This process requires that the number of ties be a small percentage of all observations. If the two groups are drawn from the same underlying distribution, the ranks should be randomly arranged between the two groups. The Mann-Whitney calculation for the rank sum for group 1 computes how many times an observational rank from the first group precedes an observational rank from the second group. There is a similar calculation for the rank sum for group 2. The rank sums are used to calculate Mann-Whitney U. For sample sizes up to 20, one looks for the critical value of U in a table of critical values of U; if the computed U is less than or equal to the critical value, the researcher concludes the two samples are drawn from the same distribution. For larger sample sizes, the distribution of U approaches the normal curve and U is interpreted using the Z statistic. Z scores of less than 1.96 indicate that the two samples come from the same underlying distribution, at the p=.05 significance level. For the SPSS output below, the absolute value of Z is .159 (the p level is .874), so the researcher concludes that "General happiness" comes from the same underlying distribution for both the male and female groups.
    
    As a second example, let n be the size of each of two independent samples A and B, and let the ordinal variable of interest rank subjects from a to k (ex., a measure of "authoritarianism" which ranks subjects from a=1 to k=20). The two samples are pooled into one dataset, listing subjects in order by their rank level. In the same order, list whether the subject was from larger sample A or smaller sample B, resulting in a series of A's and B's, as in the table below:

    Rank Score	1	1	2	3	4	5	5	6	7	8	9	9	10	11	12	14
    Sample	A	B	B	A	B	A	A	A	B	A	A	B	A	B	B	B

    Let U^a equal the sum of the number of A's preceding the first B plus the number of A's (counting from the very beginning) preceding the second B, and so on for all the B's, where the B sample is the larger sample. For these data, U^a = 1 + 1 + 2 + 5 + 7 + 8 + 8 + 8 = 40. Let U^b equal the product of the sample sizes minus U^a. For this sample, U^b = (8*8)-40 = 24. Note: U^b equates to the number of B's preceding the first A, plus the number of B's preceding the second A, etc., for all the A's. Let U equal the smaller of U^a or U^b, which for this example would be 24. By looking in a table of probabilities of U for n₁ = 8, n₂ = 8, and U=24, we find that our computation corresponds to a significance level of .221. Since this is much higher than the normal social science cutoff of .05, we fail to conclude that the two samples are significantly different on the ordinal variable. In practice, of course, SPSS and other computer packages print the probability of U directly, without need to refer to a table. In SPSS the p value is listed as "[2*(1-tailed Sig)]" and reflects the two-tailed probability that the two samples differ.

  • Wilcoxon rank sum W. The Mann-Whitney U test is an alternative to and generally equivalent to the Wilcoxon rank sum test and the Kruskal-Wallis test for two groups. In SPSS output, the Wilcoxon rank sum W statistic can be output, reflecting the rank sum of the smaller of the two groups (or the first group if equal in size).

  
  

• Wald-Wolfowitz Runs Test. This is an alternative test for the significance of the difference between two independent samples of an ordinal variable, also testing if in two groups the observational rankings seem to be randomly arranged when all observations are pooled and ordered. It also tests differences in the shapes of the distributions of the two groups, not just the locations of the ranks as in the Mann-Whitney U test..
  • In SPSS, select Analyze, Nonparametric Tests, 2 Independent Samples; select your variables; click Define Groups and select the grouping variable; under Test Type, select Wald-Wolfowitz runs.

  • Computation of the Wald-Wolfowitz Test. Pool and rank the two samples as in the Mann-Whitney test, described above, resulting in a series of A's and B's, reflecting subjects from sample A and from sample B. This series of A's and B's can be tested for significance (deviation from randomness) by the runs test of randomness. Handling ties: The Wald-Wolfowitz statistic is not affected by ties between subjects within the same sample. For ties between subjects in two different samples, the order may be assigned by use of a random number table.
    
    • Example. The output above shows that when ties are broken in all possible ways, the minimum number of runs is 4 and the maximum is 2,427. Although significance levels are reported for each possibility, it is usual (in the interests of being conservative) to report the probability level for the maximum. For these data, the significance level approaches 1.0, meaning that the researcher cannot reject the null hypothesis that the male and female groups are drawn from the same underlying distribution on "general happiness."

  • Wald-Wolfowitz Median Test. Since a difference in median values between the two samples would lead to a clustering of A runs and B runs at different portions of the rank order series, the Wald-Wolfowitz test also may be used as a test of differences in median values. However, Levin (1999: 45) warns that the median test "should not be used unless the t-test is manifestly inappropriate" because the median test has lower power than the t-test.

  
  

• Kolmogorov-Smirnov Z Test. The two-sample Komogorov-Smirnov test is a third alternative test used to test whether two independent samples of an ordinal variable come from the same sample, or can be considered to be significantly different.It tests if the maximum absolute difference in cumulative distributions of the two groups are large enough to be significant, in which case the two groups are found not to be from the same distribution. That is, the Kolmogorov-Smirnov test tests differences in the shapes of the distributions of the two groups, not just the locations of the ranks as in the Mann-Whitney U test. Note this test also has one-sample variants, discussed in another section.
  • Computation of the Kolmogorov-Smirnov Test. Let A and B be two independent samples. Given an ordinal variable which ranks subjects from 1 to k, for each sample separately compute the cumulative percentage of subjects by rank, such that the cumulative percentage of the kth rank will be 100% for each sample. For each rank, the cumulative percentage in sample B is subtracted from the cumulative percentage in sample A. Let D equal the largest absolute difference in cumulative percentages for any given ordinal rank. The researcher looks in a table of critical values of Kolmogorov-Smirnov D to get the corresponding probability level. In practice, SPSS and other statistical packages compute and print this probability level without need for the researcher to consult a table.
    
    • Example 2. In the SPSS output above, the Kolmogorov-Smirnof test returns a finding of non-significance (p = .917), meaning that the researcher fails to conclude that the male and female groups differ on "general happiness" and must therefore be assumed to come from the same distribution.

  
  

• Moses Test of Extreme Reactions. This test is used in experimental studies where it is assumed that the treatment variable will affect subjects in either a positive or a negative way, creating a polarizing effect. That is, the Moses test tests for extreme responses in one sample, compared to another sample representing a control group. More generally, where the three two-sample tests above test for differences in central tendency in the middle of the distribution, the Moses test tests for differences in extreme tendencies in the tails of the distribution. If the probability associated with the Moses test is less than the desired significance level (typically .05), then the researcher concludes that the two samples do not differ.

  Imagine two samples, one normally distributed and the other polarized at the two extremes. In such a situation it is possible both samples would have the same "central tendency" and might not be found to be significantly different by some tests which emphasize differences in central tendency. However, when a distribution is polarized, central tendency is a misleading average and poor criterion for decision. The Moses test should be used not only in the case of experimental treatments noted above, but also whenever the researcher has cause to believe that it is possible that two samples differ in the tails rather than in the middle of their distributions. One may wish to use the Moses test to supplement one of the three tests above, all of which focus on central tendency.

• Treatment and control groups. In the SPSS dialog box, the control group should be "Group 1" and the treatment group "Group 2". This test is sensitive to chance outliers and it is customary to trim the most extreme 5% of cases in the control group prior to computation. This percentage can be adjusted by the researcher in SPSS command language but not through the menu dialog (see the MOSES command in the SPSS syntax manual). Some researchers recommend ruling out a small fixed number of cases at either extreme, such as the most extreme 2 or 3 on each end.
  • Computation of the Moses test. Let A be the sample thought not to be influenced by polarization (ex., the control group) and let B be the possibly polarized group (the treatment group). So as not to base inference on chance outliers, 5% (or some other small percentage or absolute number) of sample A cases are ignored at either extreme. Arrange the pooled data by rank as described for the Mann-Whitney test above. These cases are assigned ranks from 1 to N, where N equals the sum of the two sample sizes. Ties are assigned their average rank. Span is calculated as the rank of the largest control group value minus the rank of the smallest control group value, plus 1, rounded to the nearest integer. The minimum span is also calculated, equal to the sample size of A minus the number of omitted cases. The exact probability of getting an observed span greater than or equal to the minimum span can be calculated from the laws of probability, as described in Garson (1976: 266-267). However, in practice this probability is calculated and printed directly by SPSS and other statistical packages. The formula for calculating this probability of the span is found in SPSS documentation.
    
    • Example. In the SPSS output above, the Moses test returns a finding of significance (p = .000), meaning that the researcher concludes that the male and female groups come from the same distribution of "general happiness," and this is also true when the most extreme 60 cases are trimmed. (The trimmed version reduces the probability of Type II errors).

  
  

Assumptions

• Random sampling is assumed, as in all significance tests.

• Independent samples are assumed. The two samples should not be correlated (ex., not before-after studies, panel studies, or matched-pairs studies).

• Data distribution. The tests in this section are non-parametric, not assuming the normal distribution. The Mann-Whitney U test, but not the Wald-Wolfowitz or Kolmogorov-Smirnov tests, also assumes that the distribution in each sample is similar in shape. If the researcher can assume a normal distribution, t-tests are preferable since they can detect true differences between groups using a lower sample size than nonparametric tests in this section. Put another way, t-tests have greater power. Use independent, random samples. The Mann-Whitney U test requires that the two tested samples be similar in shape.

• Data level. All the tests in this section assume ordinal data or higher.

• Data pairs. When the proportion of pairs which are tied is high, none of the tests in this section should not be used.

• Sample size. For the Mann-Whitney, Wald-Wolfowitz, and Kolmogorov-Smirnov tests, sample size must be the same in the two samples so that each has the same range of rank values, from 1 to n. Small deviations from this requirement usually do not affect substantive conclusions. Populations, of course, need not be of equal size.

Frequently Asked Questions

• Where are these tests found in SPSS?
  Assuming the SPSS Exact Tests add-on module has been installed, from the SPSS menu, select Statistics, Nonparametric Tests, 2 Independent Samples. In the "Two Independent Samples Tests" dialog box, select the test you wish under "Test Type." From the variable list, select the "Grouping Variable" and click "Define Groups" to split the data file into the two samples.

Bibliography

• Conover, W. J. (1980). Practical Nonparametric Statistics, 2nd ed. NY: John Wiley and Sons.
• Garson, G. David (1976). Political science methods. Boston: Holbrook Press.
• Levin, Irwin P. (1999). Relating statistics and experimental design. Thousand Oaks, CA: Sage Publications. Quantitative Applications in the Social Sciences series #125. Elementary introduction covers t-tests and various simple ANOVA designs. Some additional discussion of chi-square, significance tests for correlation and regression. and non-parametric tests such as the runs test, median test, and Mann-Whitney U test.
• Moses, L. E. (1952). A two-sample test. Psychometrika, 17, 239-247.
• Siegel, Sidney (1956). Nonparametric statistics for the behavioral sciences. NY: McGraw-Hill. A standard reference work.
• Siegel, Sidney & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences, 2nd ed.NY: McGraw-Hill, Inc.