Probit and Logit Response Models

Overview Response models are for grouped data, where the groups are formed by levels of some stimulus (the covariate, which is the independent variable) which causes a response (a binary response/no response variable) in some proportion of each group (this proportion is the dependent variable). One of the main objects of a response model is to determine what level of stimulus is needed to elicit a given proportion (ex., 50%) of response. Optionally there may be a grouping factor, such as gender, type of workplace, brand, etc., so one can compare the relative potency of the stimulus across groups (that is, answers the question, for which group is the lowest stimulus needed to achieve a given response rate?). Response models are primarily used in experimental designs, where it is easier to form the needed groups. Probit models are similar to logistic models but use a log-normal transformation (the probit transformation) of the dependent variable. Where logit and logistic regression are appropriate when the categories of the dependent are equal or well dispersed, probit may be recommended when the middle categories have greater frequencies than the high and low tail categories, or with binomial dependents when an underlying normal distribution is assumed. As a practical matter, probit and logistic models yield the same substantive conclusions for the same data the great majority of the time. The probit module in SPSS (in the menus, Analyze, Regression, Probit) is for a particular kind of probit model - the "probit response model.". In SPSS, in addition to the probit module itself (Analyze, Regression, Probit), other types of probit models may be implemented in the following modules, discussed separately: Logit regression Ordinal regression Generalized linear models Also, the SPSS manual notes, "the Binary Logistic Regression procedure uses a logistic link model for predicting the event probability for a categorical response variable with two outcomes, and is generally more useful than Probit Analysis when your study is not a dose-response experiment." See the logistic regression section of Statnotes. The SPSS Probit module discussed in this section is designed for a binary dependent using grouped data in a response model. The classic application is dose-response studies in medicine, where experimental groups are given varying doses of a medication under test, where the dependent is response/no response to the medication. Probit relates strength of dose to proportion responding, optionally controlling for other covariates which may also cause a response, such as age. To generalize, this procedure relates levels of an independent covariate to proportion in one category of a binary dependent variable, optionally controlling for one or more continuous covariates and optionally grouping by levels of a categorical factor. It is particularly appropriate for experimental designs where the purpose is to gauge what level of the independent variable (ex., dose) is needed to obtain a specified proportion of responses. An example would be experiments with varying levels of payments to determine the median effective payment rate needed to obtain a 50% volunteer rate for army reenlistment, as applied to randomly selected groups of soldiers nearing the end of their service obligation. Note the SPSS Probit module can also implement logit models for grouped data, assuming a binary dependent. Logit response models have nearly identical inputs and outputs but give slightly different estimates.

Contents Key concepts and terms Models Variables Statistical output Assumptions Frequently asked questions Bibliography

Key Terms and Concepts

• Models
  • Response models. Both probit and logit response models are available. Selection is done simply be checking the Logit radio button (Probit is the default) on the main Probit dialog, but otherwise entering variables and options as in probit. In SPSS syntax but not through the menus, the researcher may request both probit and logit models in the same run. As probit response models are the common variety, this section discusses response models in probit terms.

  • Probit models are often chosen in conjunction with experimental designs and logit models often chosen in conjunction with observational studies. Also, probit models discriminate better near median potency (near .5 probability of response) and are therefore more appropriate when the binary dependent is seen as representing an underlying normal distribution. The probit is the value of the left-hand side of the equation and is the inverse of the cumulative standard normal distribution. The probit transformation returns the value below which that proportion of standard normal deviates is found. For instance, for the proportion .025 (which corresponds the the .05 significance level in a two-tail test), the probit is -1.96. That is, probit(.025)=-1.96. Likewise, probit(.5)=0, and probit(.975)=+1.96.

  • Logit models discriminate better than probit models for high and low potencies and are therefore more appropriate when the binary dependent is seen as representing an underlying equal distribution (large tails). The logit model is equivalent to binary logistic regression for grouped data. The logit is the value of the left-hand side of the equation and is the natural log of the odds ratio, p/(1-p), where p is the probability of response. Thus, if the probability is .025, the logit = ln(.025/.975) = -.366; if the probability is .5, the logit=ln(.5/.5) = 0; etc.

  • Conditional vs. unconditional models. Estimates and model tests will differ depending on whether the researcher posits an unconditional model or a conditional model. An unconditional model assumes the natural response rate, discussed below, is zero and irrelevant, so it is not incorporated in the model. The response count variable, discussed below, need not be adjusted for a natural response rate in order to assess the effect of the stimulus covariate. A conditional model assumes the response count (and rate) is conditional on the underlying non-zero natural response rate as well as the magnitude of the stimulus, and the natural response rate is incorporated in the model. That is, the data on the response variable are adjusted by the natural response rate in order to then assess response due just to the stimulus covariate. The researcher must decide which model is appropriate as results will differ.

• Variables
  • Unit of analysis. Normally, data are aggregated by group. Groups are differentiated by magnitude of the covariate (stimulus) variable. There may or may not be control groups, which are groups where the covariate(s) equal zero.

  • Response count variable. The response variable is the dependent variable, and is the count of subjects in a group who have the response of interest. In a medical study, this would be the number of people in the group who respond to the dosage of the medication. This implies that the dependent is binary, with the two values being "response of interest" and "not response of interest." Note that although probit models proportions (percentage responding), percentages are calculated internally and the response variable is the dataset is a count variable, not a proportion.

  • Total observations variable. This variable is simply the number of people in each group, whether they respond or not.

  • Factor. This is a optional grouping variable which determines the intercepts (one per level of the factor) in the probit regression equations. In a medical study, brands of medication might be a factor. Levels of the factor may be labeled "agents" (in this example, brands would be agents). There can be only one factor. The factor can be nominal or ordered, but must be coded as a consecutive sequence. It is assumed that the probit regression slopes are the same for each factor level, an assumption which must be checked with the test of parallelism.

  • Covariate(s). In a medical study there might be a single covariate, which would be the dosage level variable. In an experimental design only the stimulus (ex., dosage level) covariate is needed since other variables are controlled by virtue of randomization of subjects into treatment and control groups. In an experimental design, an additional covariate is an additional stimulus. In quasi-experimental (observational) designs, additional covariates represent control variables, where the probit regression coefficient is a partial coefficient, controlling for other covariates in the model (ex., dosage level controlling for age).

  • Weighting. Note that use of a weighting variable is not appropriate for the aggregate data used in the SPSS Probit procedure.

• Input Options may be specified in SPSS under the Options button.
  • Transformation of the covariate. SPSS supports dialog options for transforming covariates by their log base 10 or by their natural log. Although no transformation is the default, this is often not the best choice. Probit response models assume linearity in the probit, which means that the relationship between the probits to the stimulus variable (the covariate, such as dosage level) should be linear. Transformed response plots are part of the default output and show graphically if there is linearity in the probit. Frequently taking a log transform will significantly improve linearity as shown in the transformed response plot. It is also possible to transform the covariate outside the probit procedure to better achieve linearity, then select "None" to decline a further log transformation. Note that since one cannot take the log of 0 and since the value of covariates for control groups is 0, asking for a log transform has the effect of excluding control groups from the analysis. Selecting "None" for transformation includes control groups in the analysis.

  • Natural response rate. This refers to the response rate to be assumed when the stimulus (ex., the dosage covariate) is zero. Note that the default value, "None" = zero, implies an unconditional model in which no response is expected in the absence of the stimulus (the covariate). Other choices, discussed below, imply a conditional model in which probit estimates reflect potency (amount of increase in response rate per magnitude of the stimulus covariate), controlling for a non-zero underlying propensity to respond even without stimulus.

    If the default "None" is selected, the probit procedure will not incorporate a natural response rate in the model. If "Calculate from data" or "Value" (the researcher enters a specific value) are selected, a natural response rate will be incorporated in the model and probit estimates and model chi-square tests will differ. If "Calculate from Data" is selected, the input data should contain a control group for which the value of the covariate(s) is zero. If there is no control group, the probit procedure estimates the natural response rate from the entire dataset and prints a notification that no control group was available. If "Value" is selected, the value entered must be non-negative and less than 1.0.

  • Estimation criteria. In SPSS one may set:
    1. Maximum iterations. The maximum number of iterations (20 is default).

    2. Step limit ,The step limit (.1 is default). the step limit prevents early steps in the iterative process from diverging too much from good initial estimates.

    3. Optimality tolerance. The optimality tolerance (.1 is default). The optimality tolerance is a convergence criterion which stops iterations when the next step will fail to produce a large enough change in the parameter estimate or the log-likelihood function.

    4. Significance level of use of heterogeneity correction. A well-fitting model is not significant by the chi square goodness of fit test discussed below. Therefore a low significance value indicates a problem with model fit. One possible cause of ill fit is heterogenous data, similar to the problem of heteroscedasticity in regression. The SPSS probit algorithm automatically incorporates a correction for heterogeneity if there is a low significance value. The researcher may set what "low" means if the default value, which is <.15, is not wanted.

• Outputs
  • Pearson goodness-of-fit chi-square: The Pearson goodness-of-fit chi-square statistic tests the fit of the data to the model. A well-fitting model is not significant by this test.
    

    In the fictional example above, the size of reenlistment bonus is tested for Army and Navy groups in a conditional response model of reenlistment response rates. Reenlistment count was transformed by the natural log. Since chi-square is non-significant, model fit is accepted as adequate.

  • Parallelism test. If the model has a single covariate (ex., dosage level) and has a factor specified (ex., brand of medication), then the Options button in SPSS enables the test of the assumption of parallelism. This tests that the slope of the probit regression is the same for each of the levels of the factor (grouping) variable. A finding of non-significance, as in the example above, upholds the parallelism assumption (fails to reject the hypothesis that the slopes are parallel). A finding of significance would mean the researcher would need to conduct separate analyses for each level of the factor.

  • Transformed response plots: This output plots the probit on the y axis against the transformed (ex., log base 10) single covariate (ex., dosage level). This is used as a graphical test of the assumption of linearity in the probit, discussed in the assumptions section.

  

  • Parameter estimates are the probit estimates of the common slope and of the intercept(s). The slope is the same for all factor levels, by virtue of the assumption of parallelism, tested above. Increasing the stimulus (covariate) has the same effect for each level of the factor (each group). If there is a factor variable, there will be one intercept for each level of the factor. Although the stimulus has the same effect for each group, the difference in intercepts shows the ratio of response levels between groups for any given stimulus (covariance magnitude) and thus also shows the relative ordering of the groups (factor levels) in terms of response.

  

  Although there are separate intercepts for Army and Navy, there is a single probit slope estimate for the covariate, which is size of reenlistment bonus. All are significant.

  • Natural response rate. A conditional model was stipulated in this example, by asking for probit to calculate a natural response rate from the data, as discussed above in the section on input options.
    

    A probit estimate of .093 means that 9.3% of the population would respond even when the magnitude of the stimulus (the covariate, which is reenlistment bonus) was zero. Army and Navy control groups were provided in the data.

  • Cell counts and residuals. This output provides observed and expected cell frequencies and is used for residual analysis, to tell where the model is working most and least well. Note this is apt to be a very lengthy table if data were individual rather than the usual aggregated type.

  

  Note the stimulus covariate (bonus size for reenlistment) is not actual dollars but is the requested natural log of the bonus. The log was requested to achieve linearity in the probit, discussed in the assumptions section. The model works least well for the Army (Branch = 1) for the larger bonuses.

  • Confidence limits. If there is only one covariate (the one representing the stimulus, such as reenlistment bonus level), "fiducial confidence intervals" are available under the Options button. This table, labeled "Confidence Limits" in SPSS output, displays estimates for the effective levels of independent variable for obtaining a response from various proportions of the population. The Confidence Limits table gives an estimate for the magnitude of the covariate (ex., dosage level) needed to produce a given probability of response. Rows in the table are these probability levels (from .01 to .99). The median potency is the estimate for the .5 probability of response level. If there is a factor, probability levels are presented for each level of the factor variable. Confidence intervals around the estimates are also presented.

  

  If the default ("None") is accepted for the natural response rate, note that estimates will then be unconditional potencies, gauging the effect of the stimulus on increasing the response, assuming the natural response rate is zero. In contrast, if a natural response rate is incorporated in a conditional model, potency estimates and relative median potencies below are interpreted in relative terms (that is, in terms which focus on the relative response of different groups to the stimulus, but which do not focus on absolute response differences since these are due to both the natural response rate and to differences in response to the stimulus among groups formed by the factor variable).

  If a logit rather than probit model is requested, logit rather than probit estimates are reported in the Estimates column of the Confidence Limits table. Both logit and probit estimates are estimates of the effective levels of the stimulus needed to achieve the row response rate.

  The abridged table above shows some of the probability rows for the armed forces reenlistment bonus example, which is a conditional model (conditional on a computed natural response rate). For any row, the estimate is the magnitude of the stimulus covariate (ex., reenlistment bonus) needed to achieve that percentage of response (ex., proportion reenlisting). For instance, for the Army, a reenlistment bonus of $75,400 was estimated to be needed to obtain a 50% reenlistment rate. Note that a lower estimate corresponds to greater potency, since less stimulus is needed to achieve the response rate.

  • Relative median potency (RMP). "Potency" refers to the probability of response. Potencies for various probability levels are shown in the table of Confidence Limits, discussed above. If there is a factor (grouping) variable and if there is only one covariate, relative median potencies are a statistics option in SPSS under the Options button. For each possible pair of factor levels, the ratio of median potencies (RMP) is printed. "Stimulus tolerance" is the covariate magnitude needed to elicit a 50% response rate. RMP is the ratio of stimulus tolerances between two factor groups.That is, for 50% probability of response, the relative median potency is the ratio of the potencies for the .5 level between two groups formed by the factor variable. A relative potency of 1.0 means the two groups are equivalent in terms of response.

    The 95% confidence interval for each relative median potency is also presented. If the value 1.0 is not within the confidence interval, then the researcher concludes that there is a significant difference in potencies. The factor level with the strongest potency is the factor level which requires the smallest magnitude of the stimulus (the covariate) to obtain median potency (to obtain a 50% response rate).

    If a natural response rate is incorporated in the model, note that estimates or relative potencies will then be conditional and the strongest factor level by relative potency is the one obtaining the greatest increase in response based on a given magnitude of the stimulus (covariate) variable. It will not necessarily be the factor level obtaining the largest absolute response since some other factor level may have a very high natural response rate to begin with.

  

  In the example a natural response rate was incorporated in the model by asking that the rate be computed from the data (under the Options button). Recall the groups were Army=1 and Navy=2. To achieve linearity in the probit, response was transformed by taking a natural log, but estimates and confidence limits are given with and without log transform. Since 1.0 is not within the confidence limits, we can assume that Army and Navy groups differ in potency (responsiveness to any given level of the stimulus, which is reenlistment bonus size). We would normally interpret RMP in untransformed terms. Thus in the Confidence Limits table, the Army potency at the median .50 response level was 75,400; for the Navy it was 31,810. That is, the reenlistment bonus stimulus had greater potency for the Navy than the Army since less was required to achieve a 50% reenlistment rate. The ratio of these two potencies is the RMP = 75400/31810 = 2.37 for Army compared to Navy. Similarly, for Navy compared to Army, the RMP computes to 0.42. The former (2.37) is greater than 1.0, meaning that compared to the Navy, in the Army achieving a 50% reenlistment rate requires a greater bonus. The latter (.42) is less than 1.0, meaning that compared to the Army, in the Navy achieving a 50% reenlistment rate can be achieved with a smaller bonus. Note interpretation is relative in a conditional model, not absolute: the RMP of .42 does not mean that Navy bonuses can be 42% of those for the Army since the RMP estimates are estimates controlling for differences between the Army and Navy in natural response (reenlistment) rates in this example.

Assumptions

• Parallelism. For models with a single covariate and a factor, there will be one intercept for each level of the grouping factor. It is assumed that the slope of the probit regression will be the same for each level. The test of parallelism, discussed above, is required to validate this assumption when a factor is in the model.

• Linearity in the probit. It is assumed that the propensity to respond is linearly related to the stimulus (covariate). This is checked in default output using transformed response plots. Log transformation of the covariate is usual.

• Normal distribution. It is assumed that the propensity to respond is normally distributed.

• Stimulus-response. If the default ("None") is accepted for the natural response rate, it is assumed that in the absence of the stimulus (covariate) there will be no responses (ex., no improvements in the medical example, no reenlistments in the recruitment example).

• Conditional potencies. If the default ("None") is accepted for the natural response rate, the potencies (the probability estimates shown in the Confidence Limits table in SPSS) and the relative median potencies do not include the natural response rate. This means that potencies and relative median potencies are unconditional: they assume that the subject enters the risk pool without the intention to respond. If a natural response rate is included in the model, estimates and relative median potencies may be used for relative comparison among groups but not for absolute comparisons, since the estimates are for the effect of the stimulus covariate controlling for the natural response rate.

• Independent observations. It is assumed data are not repeated measures on the same individuals, nor matched pairs. That is, a between-subjects design is assumed.

• Adequate sample size. As in other uses of chi-square tests, the larger the number of values for independent factors and covariates in the model, more likely cell sizes will be below 5 or even empty. This in turn renders chi-square and goodness-of-fit statistics unreliable. Also, there is greater chance of failure to converge on a solution for the probit coefficients, though SPSS may report those from the last iteration.

• Adequate number of groups. There must be more groups than the number of variables, where the number of variables includes the response variable, the total observed variable, the factor variable (if any), and any covariates.

• No negative counts. The response count variable cannot be a negative number.

• Total >= response. The total observed variable must be as great or greater than the response count variable.

Frequently Asked Questions

• What is the data set-up for a probit response model?
  The SPSS Probit module is for probit response models, where data are grouped. If data are individual rather than grouped, the researcher should use The SPSS menu choice, Data, Aggregate, to put data into grouped format.

  Rows are the groups. Each group receives a given level of the independent(s). The independents are continuous covariates (thus, in a medical study, dose could be a covariate). There can be one grouping factor, which is a categorical variable. Each group must have the same value on the factor and/or covariates (ex., if age is a covariate, then each group must have all individuals of the same age). There is a categorical factor variable (ex., brand of medication) with categories coded as integers. The dependent (response) variable is count of the number of observations in the desired binary category (ex., typically response to dose) for that group (row). There also must be a group size variable (ex., "Total") with the count of the total number of observations in that group (row).

  The SPSS manual gives this example of how to use the Aggregate command to obtain grouped data from individual data:

   Aggregating case-by-case data
  DATA LIST FREE/PREPARTN DOSE RESPONSE.
  BEGIN DATA
       1.00     1.50      .00
       ...
       4.00    20.00     1.00
  END DATA.
  AGGREGATE OUTFILE=*
    /BREAK=PREPARTN DOSE
    /SUBJECTS=N(RESPONSE)
    /NRESP=SUM(RESPONSE).
  PROBIT NRESP OF SUBJECTS BY PREPARTN(1,4) WITH DOSE.
  

• What happens if I enter individual rather than grouped data into the Probit procedure in SPSS?
  This is not recommended, but you will get output. With individual data, the chi-square goodness-of-fit statistic and its degrees of freedom are based on the individual cases. The case-based chi-square goodness-of-fit statistic generally differs from that calculated for the same data in aggregated form. Also, the Probit procedure will skip the plot of transformed response proportions and predictor values.

• What is the SPSS syntax for the probit response model?

  PROBIT response-count varname OF observation-count varname
          WITH varlist [BY varname(min,max)]
   [/MODEL={PROBIT**}]
           {LOGIT   } 
           {BOTH    } 
   [/LOG=[{10** }]
          {2.718}
          {value}
          {NONE }
   [/CRITERIA=[{OPTOL   }({epsilon**0.8})][P({0.15**})][STEPLIMIT({0.1**})]
               {CONVERGE} {n           }     {p     }             {n    }
              [ITERATE({max(50,3(p+1)**})]]
                       {n              }
   [/NATRES[=value]]
   [/PRINT={[CI**] [FREQ**] [RMP**]} [PARALL] [NONE] [ALL]] 
           {DEFAULT**              }
   [/MISSING=[{EXCLUDE**}]  ] 
              {INCLUDE  }   
  **Default if the subcommand or keyword is omitted. 
  
  Example 
  PROBIT  R OF N BY ROOT(1,2) WITH X
    /MODEL = BOTH.
  

• Couldn't we use OLS regression to create a response model?
  No. By definition, the response rate varies from 0 to 1.0. OLS regression would give estimates outside the valid range of responses.

Bibliography

• Finney, D. J. (1971). Probit analysis. Cambridge: Cambridge University Press.