15. Binary response: logit, probit and interpretation

01. How should we model a mortgage decision probability?

Our outcome is mortgage denial, coded one for denial and zero otherwise. The earlier linear probability model treated its conditional mean as a linear function of payment burden and the chosen controls. We now use the same application to introduce logit and probit probability models.

Their fitted indexes pass through smooth curves that keep probabilities between zero and one. We will define the curves and the scales of their coefficients, explain maximum likelihood, and translate estimates into probability comparisons. Keeping the data and the prediction task fixed makes those differences easier to see.

02. Build a probability from an unrestricted index

First compute the index; then convert it to a bounded prediction.

A linear index is the intercept plus the coefficient-weighted predictors. With one predictor, call it z equal to a plus b times X. The index can be any real number. To obtain a probability, pass it through an increasing S-shaped function that returns a value between zero and one.

For the logistic function, an index of zero gives probability one half, an index of minus two gives about 0.119 and an index of two gives about 0.881. The same index change does not produce a constant probability change along the curve. This is the key difference from the LPM. The next slide introduces the logistic and normal cumulative functions explicitly, so that their coefficients and probability contrasts have a defined meaning.

03. Logit and probit keep fitted probabilities in range

Increasing S-shaped functions convert the index into a probability.

A cumulative distribution function gives the probability that a reference variable is at or below a given value. Logit uses the logistic function, written capital Lambda. Plug the index z into the displayed formula to get a probability. Probit uses capital Phi. Find z on a standard normal curve and take the area to its left.

Both models give a probability of one half when the index is zero. At an index of one, logit gives about 0.731 and probit about 0.841. The same index can produce different probabilities. When you compare fitted models, use the same applicant inputs, then compare the resulting probabilities. The coefficients describe the index on its own scale.

04. A logit coefficient changes log odds

Translate it before discussing probabilities.

Start at a denial probability of 0.20. Its odds are the probability of denial divided by the probability of no denial. Divide 0.20 by 0.80 to get 0.25, or one to four.

Logit says the log of those odds equals the index. A one-unit increase in X adds b to the log odds. On the odds scale, that multiplies the starting odds by the exponential of b. To convert odds back to a probability, divide the odds by one plus the odds.

Use b equal to 0.5. At a starting probability of 0.20, the new probability is about 0.292. At a starting probability of 0.50, it is about 0.622. The same change in log odds produces different changes in probability. State the starting and ending input values when you report a fitted comparison.

05. Probit also uses an index coefficient

Its probability effect depends on the reference density.

In probit, a plus b times X determines a location on the standard-normal curve. Capital Phi is the cumulative area to that location. Lowercase phi is the density, the height of the bell curve there. A small index movement adds approximately its width times that local height to the cumulative probability.

The probability slope is therefore b times phi evaluated at the index. The partial-derivative symbol means a local change in probability per small X change, holding other predictors fixed. At an index of zero the density is about 0.399; farther in either tail it is smaller. A coefficient times an X change moves the index, while the probability change also depends on the starting location. For a finite policy-sized contrast, compute both probabilities directly.

06. Compare two candidate models on observed events

Likelihood evaluates coefficient choices with the observed outcomes held fixed.

Suppose two independent applications have outcomes one and zero. A candidate model assigns denial probabilities 0.8 and 0.2. The probability of the two observed events is 0.8 times one minus 0.2, or 0.64. Another candidate assigns probabilities 0.4 and 0.6, giving 0.4 times one minus 0.6, or 0.16.

With the outcomes held fixed, this product is the likelihood as a function of the model's coefficient choices. The first candidate gives more support to the observed event pattern. Maximum likelihood chooses coefficients that maximize this support within the specified model. The product argument uses conditional independence of applications; clustered or dependent data require corresponding modeling and inference care. We now write the same calculation for all applications.

07. Maximum likelihood chooses probabilities that support the observed events

Define the objective before interpreting the estimator.

Let theta denote the list of coefficients and p i of theta the predicted denial probability for applicant i. A Bernoulli outcome is a single zero-or-one event. A denial contributes its predicted probability p i, and an approval contributes one minus p i. The exponents in the formula select one of these contributions.

Given the observed inputs, assume the applications are independent. The capital product symbol multiplies their probabilities, as in the two-applicant example. This product is the likelihood as a function of the coefficients. Taking logs changes the product to a sum. The logarithm is increasing, so the coefficient values that maximize the sum also maximize the product. With the fit in hand, we return to the probability scale for the comparisons we want to report.

08. Compare the actual fitted probit and logit scales

Their coefficients are not directly comparable magnitudes.

Find the payment-burden coefficient in each fit. It is 2.968 in probit and 5.884 in logit. These numbers multiply the input in different indexes. The functions then turn the indexes into probabilities at different rates.

To compare the fits, hold the input values and sample fixed. Predict denial probabilities at the same payment burdens under both models. If you want a change, subtract the two probabilities within each model first. The next slide reports predictions at stated input values so we can compare on one scale.

. probit deny pir, vce(robust)

------------------------------------------------------------------------------
             |               Robust
        deny | Coefficient  std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         pir |   2.967907   .4653217     6.38   0.000     2.055893    3.879921
       _cons |  -2.194159   .1649761   -13.30   0.000    -2.517506   -1.870812
------------------------------------------------------------------------------

09. Predict probabilities at meaningful ratio values

State which covariates are changed or held fixed.

The margins output evaluates the probit model at payment-to-income ratios of 0.30, 0.40 and 0.50. With only this regressor and an intercept, these are direct fitted probabilities at the specified values. In richer models, margins can average predictions over the observed values of other covariates.

Read the probability difference between two rows as a finite contrast, not the derivative at either endpoint. The SE of that difference requires the covariance between the predicted quantities. A dedicated contrast or margins comparison should use the fitted covariance rather than subtract two SEs or assume predictions are independent.

. margins, at(pir=(.3 .4 .5))

Adjusted predictions                                     Number of obs = 2,380
Model VCE: Robust

Expression: Pr(deny), predict()
1._at: pir = .3
2._at: pir = .4
3._at: pir = .5

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         _at |
          1  |   .0961532   .0069017    13.93   0.000      .082626    .1096803
          2  |   .1569683   .0101166    15.52   0.000     .1371402    .1767965
          3  |   .2387884    .024485     9.75   0.000     .1907987    .2867781
------------------------------------------------------------------------------

10. A marginal effect is a local probability slope

For an indicator, use a discrete comparison instead.

For a continuous regressor, a marginal effect is the derivative of the fitted probability with respect to that regressor at stated covariates. In logit it is b times p times one minus p; in probit it is b times the reference density at the index.

For a zero-or-one regressor, the meaningful change is usually the probability at one minus the probability at zero, with a specified treatment of other covariates. A derivative treating the indicator as continuous would answer a different mathematical question. Define the change before asking software for an effect.

11. An average marginal effect averages individual slopes

The effect at the average covariates is another calculation.

A marginal effect for a continuous predictor is the local probability slope at specified covariates. Because logit and probit are nonlinear, this slope can differ across applicants. An average marginal effect, abbreviated AME, evaluates the slope at each observed applicant's covariates and averages the results. It is not generally the slope at the average covariate values.

For a constructed logit with index equal to X, take two applicants at X equal to minus two and plus two. Both have probability slope p times one minus p, about 0.105, so their average slope is about 0.105. Their average X is zero, where the slope is 0.25. The difference comes from the curved probability function. In the formula, j selects the continuous predictor and the evaluation bar means use applicant i's actual covariates before averaging.

. margins, dydx(pir)

Average marginal effects                                 Number of obs = 2,380
Model VCE: Robust

Expression: Pr(deny), predict()
dy/dx wrt:  pir

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         pir |   .5664828   .0866687     6.54   0.000     .3966153    .7363504
------------------------------------------------------------------------------

12. A partial effect at average inputs is a different object

A nonlinear transformation does not commute with averaging.

Call applicant i's fitted index z i. G turns that index into a probability and g gives the slope of that curve. If a continuous input enters linearly with coefficient b j and no interaction, its local probability slope is g at the applicant's index times b j.

The average partial effect calculates that slope for each applicant and then averages the slopes. It is also called the average marginal effect. The partial effect at the average reverses the order. Average each input first. Use those average inputs to form one index and evaluate the slope there. The curve is nonlinear, so the two calculations can give different values.

For an indicator, compute the difference between the two fitted probabilities at one and at zero with the other inputs fixed. Then average those differences if the question concerns the average contrast for the observed applicants.

13. A finite probability change can differ from a local approximation

Evaluate both endpoints when the proposed change is substantial.

For a continuous ratio increase of 0.10, multiplying a derivative by 0.10 is a local approximation. In an S-shaped model the slope changes along that movement. Calculating fitted probabilities at both endpoints gives the exact contrast within the fitted model.

This is the same distinction used for logs and quadratic regressions earlier in the course. A policy-sized contrast should follow the change decision-makers actually contemplate. The exactness concerns the fitted function, not a guarantee that the contrast is causal or that all its parameters are known without uncertainty.

14. Prediction performance needs more than a significance test

Check the probability model against its task.

Prediction asks how well the fitted probabilities describe new outcomes. Calibration means that events occur at roughly the predicted frequency among comparable cases assigned similar probabilities. For example, among 100 applications assigned a probability near 0.20, we would expect roughly 20 denials over repeated comparable sets, allowing sampling variation.

Classification instead converts probabilities into yes-or-no predictions using a threshold. If only five percent of applications are denied, predicting no denial for everyone gives 95-percent accuracy while finding no denials at all. Evaluate probability quality and decision costs, preferably on data not used to fit the model, rather than relying only on coefficient significance or raw accuracy. None of these prediction checks supplies the assumptions needed for a causal intervention claim.

15. A fitted denial gap is not automatically a treatment effect

Selection and omitted applicant characteristics can remain.

An adjusted probability difference can still reflect unmeasured characteristics, selection into applications or aspects of the decision process outside the model. Choosing logit instead of OLS addresses the mean function's shape, not the exogeneity of a regressor.

If the research question is causal, specify the intervention and identify a credible comparison. If the aim is descriptive or predictive, state that purpose and assess the model accordingly. This lets the same data support useful work without presenting one statistical functional form as a general solution to identification.

The selection node S means applying for a mortgage; every observed application has S equal to one. In this possible graph, payment burden and unmeasured financial readiness both affect applying, while readiness also affects denial. Selecting applicants can open the shared-consequence route. Predicting denial within the observed applicant population is still a meaningful task; a causal burden effect or a claim about all potential borrowers needs additional selection and identification arguments.

16. Write a compact applied research memo

Connect the question, data, model, assumptions and result.

A useful memo identifies the decision or economic question, defines the observation and variables, explains the comparison and reports a meaningful magnitude with uncertainty. It then states the key assumptions and one limitation that could materially change the conclusion.

For the mortgage example, use a payment-burden contrast in percentage points and explain how it was obtained from the fitted model. For a demand, traffic or panel application, use the corresponding economic units. The course's methods differ, but the quality standard is consistent: readers should understand what the number measures and why the comparison could answer the question.

17. Compare probabilities on one common scale

Reproduce the evidence, then explain its economic meaning.

Run lab_15.do using mortgages.csv. Read the fitted probit denial probabilities at payment-to-income ratios 0.30 and 0.40, and calculate their difference in percentage points. Explain why multiplying the probit coefficient by 0.10 does not generally give this change. State whether the result measures a causal lending response.

Add squared payment burden to the probit index. Recalculate the 0.30-to-0.40 fitted probability difference on the same observations and compare it with the original fit. Write and run your own commands, then save the code, execution log and an explanation in your own words. The independent practice file is a worked reference.

18. Compare probabilities on one common scale: explanation

A reproducible result needs both a calculation and an interpretation.

Read the fitted probability at each stated payment burden. At 0.30, the first probit gives about 0.0962. At 0.40 it gives about 0.1570. Subtract, then multiply by one hundred. The probability rises by about 6.08 percentage points for a ten-percentage-point increase in payment burden. We used the full predictions before rounding.

Use the same two input values in the quadratic-index probit. Its fitted probability contrast is about 5.98 percentage points. Both models use the same 2,380 applications. Changing the fitted curve changes the prediction slightly. To interpret either contrast as the effect of changing a borrower's burden, we would need evidence about borrower selection and other determinants of lending.

19. Choose the method that addresses the empirical problem

A technical tool is useful when its role is explicit.

Multiple regression accounts for stated observed differences. Fixed effects remove stable unit components. IV uses instrument-induced variation under additional assumptions. Dynamic models represent dated adjustment, and robust, HAC or clustered covariance methods address specified uncertainty structures. Logit and probit provide bounded probability functions.

These roles can be combined, but none is a universal stamp of credibility. Begin each new application with the comparison and the threat it faces. Then choose and justify the smallest set of methods that makes the result meaningful, reproducible and honest about what remains unresolved.

20. Logit coefficient

Try the comparison before opening the explanation.

Further study

Why is a logit coefficient of 0.5 not a 50-percentage-point increase in probability?

Name the coefficient’s index scale. Use the two starting-probability examples to explain why no single probability increase follows from the coefficient alone.

21. Logit coefficient: explanation

Follow the calculation and interpret its units.

Further study

It is a change in the log odds per unit of the regressor, conditional on the specification. The probability change depends on the starting index and other covariates. Report an explicit prediction contrast or marginal effect, meaning a local probability slope, to discuss probabilities.

22. A latent-variable representation fixes an otherwise arbitrary scale

Only the sign of the unobserved index is observed.

Further study

A latent variable is an unobserved continuous quantity used to describe the binary decision. Write Y star as x prime beta plus e, and record Y equal to one when Y star is positive. Multiplying the entire latent equation by a positive constant leaves that observed decision unchanged, so its absolute scale cannot be recovered from the binary outcome alone.

Probit fixes the latent error to a standard normal distribution with variance one. Standard logit fixes a logistic distribution, with variance pi squared divided by three. Here pi is the mathematical constant, about 3.14159, rather than the inflation variable used in the earlier application. Their coefficients therefore use different index scales. A mechanical coefficient conversion is only a rough comparison, not an identity or a test that the models agree. Compare fitted probabilities and stated probability contrasts on the same observed sample.

23. The Bernoulli log-likelihood makes the fitting criterion explicit

The observed outcome selects which probability enters each contribution.

Technical appendix

Let p i of beta be the probability of denial for applicant i. The response function applies the index built from the applicant's inputs and the coefficient list beta. The outcome is either one for denial or zero for approval. This is a Bernoulli outcome.

The exponents select the observed event. With Y equal to one, the first term remains and the other term is raised to zero, so this applicant contributes p i. With Y equal to zero, the contribution is one minus p i. Under conditional independence, multiply the contributions across applicants.

Taking logs turns that product into a sum. The log increases with the product, so either criterion selects the same coefficients. An optimisation routine can use the gradient, the list of slopes of the objective with respect to the coefficients, to find a maximum. The familiar large-sample inference needs a correctly specified probability model, suitable sampling and enough information about the coefficients near the target.

24. Distinguish an average slope from a slope at average inputs

Try the comparison before opening the explanation.

Further study

For the mortgage probit model, explain what margins, dydx(pir) averages and what adding atmeans changes. Which reported quantity uses the observed covariate distribution? How should an indicator regressor be handled?

State the assumptions and units alongside the calculation.

25. Distinguish an average slope from a slope at average inputs: explanation

Follow the calculation and interpret its units.

Further study

The average marginal effect averages the model’s input-specific local slopes over the observed applications. An atmeans calculation evaluates the slope at a single mean-input vector. Nonlinearity makes them generally different. Use a finite zero-to-one contrast for an indicator, with a clear reference population. Neither fitted probability comparison is automatically causal.

26. Average marginal effect

Try the comparison before opening the explanation.

Further study

Distinguish an average marginal effect from the marginal effect evaluated at the mean covariate vector.

Evaluate each applicant’s local slope before averaging. Compare this order with evaluating one slope at averaged covariates.

27. Average marginal effect: explanation

Follow the calculation and interpret its units.

Further study

The AME computes an effect at each observation's covariates and averages those effects. The effect at means computes one effect at the vector of average covariates. In nonlinear models the two generally differ, and the average vector can describe no actual person.

28. DAG practice: selected applicants and prediction

Trace the routes before choosing a control.

Further study

All observed records are mortgage applicants. In the diagram, payment burden and unmeasured readiness affect applying, while readiness also affects denial. Trace the route opened by selecting applicants. Does this make within-applicant prediction meaningless, or does it change the causal claim that needs defending?

Name the target comparison first. Trace each route through the relevant variables and say whether it is directed, blocked or open under the stated conditioning. Use the diagram controls to test your proposed explanation before opening the worked reasoning.

29. DAG practice: selected applicants and prediction: explanation

Explain what conditioning changes in this stated graph.

Further study

Selecting on application opens \(X\rightarrow S\leftarrow U\rightarrow Y\), because application is a common consequence of burden and readiness. A causal comparison of burden can then mix its relationship with the outcome and selection through readiness.

Predicting denial conditional on burden within the observed applicant population remains a legitimate descriptive task. A causal change in burden, or a conclusion about all potential borrowers, requires a clear target and additional identification and selection assumptions. A bounded logit or probit function does not itself close the path.