Tests for More Than Two Independent Samples
Kruskal-Wallis H, Median, and Jonckheere-Terpstra Tests

Overview The tests in this section test whether one can reject the null hypothesis that two or more independent samples come from the same underlying population distribution. In SPSS, these tests require that the Exact Tests add-on module be installed.

Contents Key concepts and terms Kruskal-Wallis H test Median test Jonckheere-Terpstra test Assumptions Frequently asked questions Bibliography

Key Concepts and Terms

• Independent samples. Samples are independent if the response of the nth person in the second sample is not a function of the response of the nth person in the first sample. Independent samples are also called uncorrelated samples and unrelated samples. Samples which are not independent include before-after and panel studies of the same people, or matched-pairs studies of similar people.

• Type of significance estimate. The Exact button in the SPSS dialog above allows the researcher to select among asymptotic, exact, or Monte-Carlo estimates of the significance of the particular test value. These three types of estimates are discussed separately in the section on significance testing. This requires that the SPSS Exact Tests add-on module be installed.
  

• Kruskal-Wallis H test. This extension of the Mann-Whitney U test to multiple samples is a nonparametric alternative to one-way analysis of variance. It tests the null hypothesis that the samples do not differ in mean rank for the criterion variable. Because it takes rank size into account rather than just the above-below dichotomy of the median test, discussed below, it is more powerful and preferable when its assumptions are met. The H test will reveal if there are rank differences among multiple groups. For pairwise comparisons, the Mann-Whitney test is appropriate.
  • In SPSS, select Analyze, Nonparametric Tests, K Independent Samples; select your variables; select the Grouping Variable; Check Define Range and set the min and max; Continue; in the Test Type group, select Kruskal Wallis H. Note that the test variable must be ordinal.
    

    For the example in this illustration, 30 voters receive information about a referendum issue and are asked to rank it from 1="Most important" to 5="Least important." The voters are divided into three groups of 10, receiving the information by each of three different media (TV, newspaper, Internet). The media constitute the independent samples. The rankings ("rank) is the test variable.

  • Computation. Kruskal-Wallis H is calculated on the basis of sums of ranks for combined groups. Data from all samples are ordered as in the Wald-Wolfowitz or Mann-Whitney U tests. Let n_i be the number of observations in any particular sample and let N be the number of observations in all samples combined. The data are pooled and renumbered from 1 to N, with 1 corresponding to the lowest score. Using these rank scores, data for each sample are listed by rank. These rank scores are added up for each sample, and the sums are the T_i scores in the formula below. Also, let k be the number of samples. Then Kruskal-Wallis H is computed as:
    H = 12/(N(N + 1))* SUM(T²_i/n_i) - 3(N + 1)

    degrees of freedom = k - 1

    The Kruskal-Wallis H test should not be used when the number of ties is large. For modest numbers of ties, H may be adjusted for a penalty factor. This is done by dividing H by this penalty factor, as below, where t = the number of ties in any sample and the numerator is the sum for all samples:

    H_adjusted = H /[1-(SUM(t³ - t))/(N³ - N)]

  • Output. In SPSS, descriptive statistics and quartiles are the only additional output available, apart from the Kruskal-Wallis test itself.
    

    The output for the example looks like this:

    

  • Interpretation. The H statistic, which reflects how much group ranks differ from the pooled all-groups rank, is distributed approximately as chi-square. The researcher calculates H as above and then consults a chi-square table with (k - 1) degrees of freedom, where k is the number of groups. If the critical value of chi-square for the desired significance level (typically .05) is equal to or less than the computed H value, then the researcher rejects the null hypothesis that the samples do not differ on the criterion variable. SPSS prints the corresponding significance value directly. That is, a finding of H significance means there is a rank difference between groups.

    For the example above, the significance level of .049 means that there is only a 4.9% chance of obtaining a rank-difference chi-square equal to or greater than that observed (6.041) by chance. This means that the ratings of the referendum issue do differ significantly by media. even for this small sample of 30.

  
  

• Median test. Also called the Westenberg-Mood median test, this is a more general but less powerful alternative to the Kruskal-Wallis H test for testing if several independent samples come from the same population. It tests whether two or more independent samples differ in their median values for a criterion variable.
  • In SPSS, select Analyze, Nonparametric Tests, K Independent Samples; select your variables; select the Grouping Variable; Check Define Range and set the min and max; Continue; in the Test Type group, select Median.

  • Computation. The samples are combined temporarily to determine their pooled median value. A table can then be constructed in which the columns are the samples and the two rows reflect the sampe counts above or below the pooled median value. The significance of the table is then calculated using chi-square.

  • Interpretation. For the same data example, the median test is not significant (p = .387), leading to the conclusion that the media groups do not differ in voter rankings in terms of number of rankings above and below the median (even though the Kruskal-Wallis H test showed the groups did differ, barely, in terms of rank-differences). That is, the median test tests if the groups have the same median, but does not test ordering above and below the median, nor dispersion from the median. Of course, the median test assumes that the median is an appropriate measure of the center of the distribution studied.
    

  
  

• Jonckheere-Terpstra test. The Jonckheere-Terpstra test tests for ordered differences among groups assumed to be arranged ordinally. As such it is a test for differences among several independent samples which is more powerful than the Kruskal-Wallis H or median tests. It requires that the independent samples (the grouping variable) be ordinally arranged as well as that the test variable be ordinal. The J-T test tests the hypothesis that as one moves from samples low on the ordering criterion to samples high on the criterion, the within-sample magnitude of the test variable increases or decreases systematically. The null hypothesis in the Jonckheere-Terpstra test is that the test variable does not differ between groups.

  What is being tested is the effect of the variable used to order the grouping (row) variable (amount of information by media, in this example). When the standardized J-T statistic is more than 0, increasing order is indicated. When the standardized J-T statistic is less than 0, decreasing order is indicated. The significance level indicates if the increasing or decreasing ordering effect may be assumed to be different from no order. That is, if there is a finding of significance, the researcher concludes that the grouping variable is ordered.If there is a finding of non-significance, the researcher concludes that there is no pattern of constant increast or decrease. Note that J-T tests may be one-sided or two-sided. SPSS prints the two-sided version, but if the researcher can assume that one direction (increasing or decreasing) need not be tested, perhaps because it is impossible, then a one-sided test is more appropriate.

  When only two groups are being compared, the Jonkheere-Terpstra test is equivalent to the Mann-Whitney U test.

  • Data setup. Normally, the column variable is the ordinal test variable ("rank" in our example) and the row variable is the grouping variable, assumed to be ordinally arranged ("media" in our example would require an assumption that as one moves from TV to Newspaper to Internet, one is increasing in amount of information, for example).

  • Example.A typical use of the J-T test would be in medicine, where the groups would represent dosage levels and the response (test) variable would be levels of symptom relief, for instance. However, for purposes here we continue with the same example, where the groups are media types (here assumed to represent ordered information levels) and the response variable is "rank" ranking of importance of some policy issue. The output below indicates that there is significant positive ordering of the groups formed by the media variable, based on differences in within-group ordering of the "rank" variable.
    

  
  

Assumptions

• Random sampling is assumed, as with all significance tests.

• Independent samples are also assumed.

• Independent observations. Within each sample, the response of the nth subject is not dependent on the response of previous subjects, for all cases.

• Data distribution. All tests in this section are nonparametric, not assuming the normal distribution or equal group variances. It is assumed that the distribution in each sample is similar in shape. If the researcher can assume a normal distribution, t-tests are preferable since they can detect true differences between groups using a lower sample size than nonparametric tests in this section.

• Data level. These tests assume the data are ordinal or higher. The Jonckheere-Terpstra test also assumes the samples are ordinally arranged (ascending or descending) on the criterion variable. The Kruskal-Wallis H test assumes data have a continuous distribution in the population from which they were sampled.

• Adequate cell size: For the median test, some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count.

Frequently Asked Questions

• Where are these tests located in SPSS?
  From the SPSS menu, select Statistics, Nonparametric Tests, K Independent Samples. In the "Tests for Several Independent Samples" dialog box, select the "Test Type" you want: Kruskall Wallis H, Median, or Jonckheere-Terpstra. From the variable picklist, enter the criterion variable in the "Test Variable List:" box. For continuous criterion variables, enter a grouping variable in the "Grouping Variable:" box and click on "Define Range" to enter the minimum and maximum values. Note that the J-T test is available only when the SPSS Exact Tests module is installed.

• Can I do the Jonckheere-Terpstra test in Stata?
  Jonter is a Stata module to implement this test, available at http://ideas.repec.org/c/boc/bocode/s423601.html.

Bibliography

• Conover, W. J. (1980). Practical nonparametric statistics, 2nd ed. NY: John Wiley & Sons.
• Daniel, Wayne W. (1978). Applied nonparametric statistics. Boston: Houghton Mifflin.
• Garson, G. David (1976). Political science methods. Boston, Holbrook Press. Or consult any statistics text which covers nonparametric tests.
• Hollander, B. M. & Wolfe, D. (1999). Nonparametric statistical methods. 2nd edition. NY: John Wiley & Sons.
• Jonckheere, A. R. (1954) A distribution-free. k-sample test against. ordered alternatives. Biometrika 41: 133-145.
• Lehmann, E. L. (1975_. Nonparametrics: Statistical methods based on ranks. San Francisco: Holden-Day.
• Levin, Irwin P. (1999). Relating statistics and experimental design. Thousand Oaks, CA: Sage Publications. Quantitative Applications in the Social Sciences series #125. Elementary introduction covers t-tests and various simple ANOVA designs. Some additional discussion of chi-square, significance tests for correlation and regression. and non-parametric tests such as the median test and the Mann-Whitney U test.
• Siegel, S. & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill, Inc.