| Cohort | Number of banks |
|---|---|
| Never treated | 6 |
| Treated in 2013 | 1 |
| Treated in 2014 | 5 |
| Treated in 2015 | 6 |
| Treated in 2016 | 2 |
| Treated in 2017 | 4 |
| Treated in 2018 | 2 |
| Treated in 2019 | 3 |
| Treated in 2020 | 5 |
| Treated in 2021 | 2 |
| Treated in 2022 | 4 |
| Treated in 2023 | 3 |
TA4: Difference-in-Differences and Staggered DiD
Expected outcomes
By the end of this TA, you should be able to explain:
- what difference-in-differences (DiD) is doing in the simplest 2x2 case
- what the parallel-trends assumption means and why it is the key identifying assumption
- what changes when treatment is staggered across units
- why the standard two-way fixed-effects (TWFE) regression can be biased with staggered timing and heterogeneous effects
- what Sun & Abraham (2021) does differently and why it fixes the problem
- how to read an event-study plot and decide whether to trust the design
You can also follow the full source script at code/05_ta4_staggered_did.R. Everything on this page is generated by that script and saved into output/tables/ and output/figures/.
The main empirical question
This TA is built around one concrete question:
What is the effect of crossing the $10bn regulatory threshold on bank loan growth?
The economic story is simple. Once a bank crosses the $10bn asset threshold, it faces stricter regulation, more reporting, and tighter capital and liquidity supervision. We expect the additional regulatory burden to slow lending, at least on average, and especially for cohorts that cross the threshold during years of high regulatory intensity.
Banks cross the threshold in different years. Some banks never cross. That is exactly the textbook setup for staggered DiD.
Concretely, the variables we use are:
bank_id,year: panel identifiersassets_bn: bank assets, the running variable that triggers the regulatory thresholdloan_growth: outcome of interestfirst_treat: first year in which the bank crossed $10bn (0if never)treated: 1 once the bank has crossed, 0 otherwise
We drop banks that were already above $10bn in 2012 (no pre-treatment observations) and recode never-treated banks to first_treat_sa = 9999 so that fixest::sunab() recognises them as the never-treated reference group.
DiD: the basic logic
Start with the cleanest case. Two groups (treated and control), two periods (pre and post). The DiD estimator is simply the difference between two changes:
\[ \widehat{\beta}_{DiD} = \big( \bar{Y}_{T,\text{post}} - \bar{Y}_{T,\text{pre}} \big) - \big( \bar{Y}_{C,\text{post}} - \bar{Y}_{C,\text{pre}} \big). \]
In words: take the change in the outcome for the treated group, subtract the change in the outcome for the control group, and report what is left. The same idea, written as a regression, is the textbook fixed-effects DiD:
\[ Y_{i,t} = \alpha_i + \gamma_t + \beta \cdot D_{i,t} + \varepsilon_{i,t}, \]
with unit fixed effects \(\alpha_i\), time fixed effects \(\gamma_t\), and a treatment dummy \(D_{i,t}\) that switches on after treatment. The coefficient \(\beta\) is the DiD estimate.
The identifying assumption: parallel trends
DiD does not assume that treated and control units are identical. It assumes that, in the absence of treatment, their outcomes would have moved in parallel. Formally, with \(Y(0)\) denoting the potential outcome under no treatment:
\[ \mathbb{E}\big[Y(0)_{T,\text{post}} - Y(0)_{T,\text{pre}}\big] = \mathbb{E}\big[Y(0)_{C,\text{post}} - Y(0)_{C,\text{pre}}\big]. \]
This statement is untestable, since the left-hand side is a counterfactual we never observe. The only evidence we can collect for it is indirect: pre-treatment trends should look parallel, the economic story should make divergent counterfactuals implausible, and placebo or anticipation tests should not flag anything obvious. A common confusion is to read parallel trends as “treated and controls have the same level”. It is not about the level. It is about the slope of the counterfactual path.
What DiD identifies: the ATT
If parallel trends holds, DiD identifies the average treatment effect on the treated (ATT):
\[ ATT = \mathbb{E}\big[Y(1) - Y(0) \,\big|\, \text{treated}\big]. \]
This is a smaller and more honest object than the ATE. DiD speaks only about the treated group: it asks how much their outcome changed, compared with the trend the control group suggests would have happened to them in the absence of treatment.
Staggered DiD: when timing varies
In our application, banks cross the $10bn threshold in different years. Some never cross. So instead of one “treated group” and one “control group”, we have a control group of never-treated banks plus many treatment cohorts, indexed by the year they first crossed the threshold.
The natural extension of the regression DiD is the standard two-way fixed-effects (TWFE) specification, with \(D_{i,t}\) now switching on the year a bank first crosses the threshold:
\[ Y_{i,t} = \alpha_i + \gamma_t + \beta \cdot D_{i,t} + \varepsilon_{i,t}. \]
It is tempting to read \(\beta\) as “the average effect of crossing the threshold”. The literature has shown that with staggered timing this reading is wrong in general.
Why TWFE can be biased
The Goodman-Bacon (2021) decomposition shows that the TWFE coefficient is a weighted average of every possible 2x2 DiD that you can build from the panel. There are three kinds of 2x2 building blocks:
- Treated vs never-treated. Clean comparison, gives the ATT we want.
- Earlier-treated vs later-treated (during the period when the later cohort is still untreated). The later-treated act as controls before they are treated. Still clean.
- Later-treated vs earlier-treated (during the period when the earlier cohort is already treated). The earlier-treated act as controls for the later-treated, even though they are themselves already treated. This is the “forbidden comparison”.
The third building block is the problem. If the earlier-treated cohort is still adjusting to its own treatment (dynamic effects) or if its treatment effect is different from that of later cohorts (heterogeneity), then using them as controls contaminates the comparison. TWFE can also put negative weights on some 2x2 building blocks. The headline coefficient is then not interpretable as the ATT, and in extreme cases it can have the wrong sign.
A practical way to detect the problem is to estimate TWFE and a heterogeneity-robust estimator (Sun & Abraham, Callaway & Sant’Anna, did2s, …), then compare them. If the two disagree noticeably, TWFE is contaminated. If they agree, the bias is empirically small in your sample.
What Sun & Abraham (2021) does differently
Sun & Abraham define a separate ATT for each combination of cohort \(g\) (the year a bank is first treated) and event time \(\ell\) (years since first treatment):
\[ ATT(g, \ell) = \mathbb{E}\big[Y_{i, g+\ell}(1) - Y_{i, g+\ell}(0) \,\big|\, G_i = g\big]. \]
They estimate each \(ATT(g, \ell)\) using only the corresponding cohort and never-treated (or not-yet-treated) units as controls. Already-treated units are never used as controls. The cohort-specific effects are then aggregated with non-negative weights, which gives a clean event-study profile \(\widehat{ATT}(\ell)\) and a clean aggregate ATT.
The fixest function sunab(cohort, period) implements this directly inside feols(). The never-treated reference cohort must take a value larger than the maximum period of the panel, which is why we recode never-treated banks to 9999. If you forget this step, sunab() will silently treat the wrong cohort as the reference and the event-study plot will look strange.
Where DiD fits, and common mistakes
DiD is one of several identification strategies you have seen in this course. It is the right tool when treatment timing varies across units and you have a credible never-treated (or later-treated) control group.
| Strategy | Source of variation | Best when |
|---|---|---|
| OLS / fixed effects | Within-unit variation | Treatment is plausibly exogenous conditional on observables and FEs |
| Event study (TA2) | Event-window variation | A discrete event with a clean window |
| Instrumental variables (TA3) | Variation from an external instrument | The regressor is endogenous and you have a valid instrument |
| Difference-in-differences | Treatment-control × pre-post variation | A policy / event affects some units (or affects them earlier) |
| Regression discontinuity | Variation around a threshold | Treatment is a deterministic function of a running variable near a cutoff |
Even when DiD is the right tool, it is easy to misuse. The most common mistakes are:
- Treating TWFE on staggered data as a clean ATT. With staggered timing and heterogeneous effects this is biased. Always check with a heterogeneity-robust estimator.
- Reading parallel-trends as “treated and controls have the same level”. Parallel trends is about the slopes of the counterfactual, not the levels.
- Including always-treated units in an event study. Without pre-treatment observations they cannot identify a clean event-study coefficient. Drop them.
- Forgetting to cluster. Standard errors should be clustered at the level of treatment assignment (here, the bank).
- Looking only at significance. A statistically significant TWFE coefficient that disappears under Sun & Abraham is a warning sign, not a confirmation.
- Selecting the post-treatment window after seeing the data. Decide the event window in advance.
Time to code
source("code/00_utils.R")
load_finance4_packages()
bank_panel <- readRDS("data/master_bank_panel.rds")
ta4_panel <- bank_panel |>
dplyr::filter(first_treat == 0L | first_treat >= 2013L) |>
dplyr::mutate(
first_treat_sa = dplyr::if_else(
first_treat == 0L, 9999L, as.integer(first_treat)
)
)The full script is code/05_ta4_staggered_did.R. The remainder of this section walks through every result the script produces, in the same order it produces them.
Step 1: Treatment cohorts in the panel
The first thing we do after loading the panel is count how many banks fall into each cohort. This is the table the script writes to output/tables/ta4_cohort_summary.csv.
The “Never treated” group is our cleanest control group. Each “Treated in YYYY” row is its own treatment cohort with its own pre and post periods. The fact that no single year contains the entire treated population is what makes this a staggered design and why a single TWFE coefficient is suspicious by default.
Step 2: Treatment timing across banks
Next, we visualise the staggered structure directly. Each row is a bank, each column is a year, and the colour indicates whether the bank is treated in that year.
Cohorts switch on at different years and never-treated banks remain grey throughout. This is also the picture the Goodman-Bacon decomposition is operating on: every horizontal pair of rows produces a 2x2 building block, and TWFE averages all of them with weights that depend on cohort sizes and on how long each cohort spends treated.
Step 3: Pre-treatment trends across cohorts
Before any regression, we look at the raw mean of loan_growth by cohort and year. This is a descriptive plausibility check for parallel trends, not a formal test.
We are looking at two things here:
- whether the lines are roughly parallel before each cohort’s treatment year, and
- whether the lines diverge after treatment.
If the pre-treatment lines drift systematically apart, the parallel-trends story becomes harder to defend and any DiD interpretation is on shaky ground. In our sample the pre-treatment lines move together reasonably well, which gives us licence to take the post-treatment divergence at face value.
Step 4: Naive TWFE estimate
Now the workhorse regression. The TWFE specification we estimate is:
\[ loan\_growth_{i,t} = \alpha_i + \gamma_t + \beta \cdot treated_{i,t} + \varepsilon_{i,t}. \]
This is the “default” DiD spec everyone writes down, and it produces a single number for \(\beta\). Because of the staggered structure, we should treat that number as a diagnostic and not as the ATT.
| Estimator | Term | Estimate | Std. error | t-statistic | p-value |
|---|---|---|---|---|---|
| TWFE | treated | -0.0058 | 0.0036 | -1.596 | 0.118 |
The point estimate is small and negative, and the p-value sits a little above conventional thresholds. The honest reading is: “TWFE points in the direction the regulatory story predicts, but it is not by itself decisive, and we have not yet addressed the staggered-timing bias.” The next step does both.
Step 5: Sun & Abraham event-study coefficients
We re-estimate the same panel using fixest::sunab(first_treat_sa, year). This produces one coefficient per event time, weighted across cohorts. The negative event times are pre-treatment placebos; the non-negative event times are the dynamic ATTs.
| Event time | Estimate | Std. error | 95% CI low | 95% CI high |
|---|---|---|---|---|
| -11 | 0.0430 | 0.0174 | 0.0090 | 0.0770 |
| -10 | 0.0083 | 0.0203 | -0.0315 | 0.0481 |
| -9 | 0.0246 | 0.0157 | -0.0061 | 0.0553 |
| -8 | 0.0257 | 0.0108 | 0.0045 | 0.0470 |
| -7 | -0.0096 | 0.0088 | -0.0269 | 0.0077 |
| -6 | 0.0259 | 0.0082 | 0.0098 | 0.0421 |
| -5 | 0.0098 | 0.0087 | -0.0072 | 0.0269 |
| -4 | 0.0042 | 0.0082 | -0.0118 | 0.0203 |
| -3 | 0.0085 | 0.0071 | -0.0055 | 0.0224 |
| -2 | 0.0011 | 0.0059 | -0.0104 | 0.0127 |
| 0 | 0.0006 | 0.0074 | -0.0139 | 0.0151 |
| 1 | 0.0004 | 0.0072 | -0.0138 | 0.0145 |
| 2 | -0.0022 | 0.0088 | -0.0195 | 0.0151 |
| 3 | -0.0040 | 0.0083 | -0.0202 | 0.0123 |
| 4 | -0.0099 | 0.0100 | -0.0295 | 0.0098 |
| 5 | 0.0009 | 0.0117 | -0.0221 | 0.0240 |
| 6 | -0.0038 | 0.0151 | -0.0333 | 0.0257 |
| 7 | -0.0022 | 0.0118 | -0.0253 | 0.0208 |
| 8 | 0.0056 | 0.0183 | -0.0304 | 0.0415 |
| 9 | 0.0046 | 0.0136 | -0.0220 | 0.0312 |
A few features to notice:
- The coefficients before event time
0are mostly small and statistically indistinguishable from zero. That is reassuring evidence in favour of parallel trends. - The post-treatment coefficients are also small and noisy, which is what produced the small TWFE point estimate above.
- The standard errors are clustered at the bank level. This is the same level at which treatment is assigned, which is exactly where clustering should happen.
Step 6: Event-study plot
The same coefficients become much easier to read once we plot them with their 95% confidence intervals.
How to read it:
- Event time = 0 is the year the bank first crosses $10bn.
- Coefficients to the left of the dotted line are pre-treatment placebos. They should hover around zero if parallel trends is plausible.
- Coefficients to the right of the dotted line are the dynamic ATTs. A flat negative profile would suggest a constant negative effect; a downward-sloping profile would suggest an effect that grows over time.
In our sample, the pre-treatment placebos are flat around zero and the post-treatment coefficients are slightly negative but noisy. That is exactly the picture we would expect from a regulatory friction that quietly slows lending without producing a dramatic break.
Step 7: TWFE versus Sun & Abraham
This is the headline diagnostic. We put the naive TWFE coefficient and the Sun & Abraham aggregate ATT side by side, with their 95% confidence intervals.
| Estimator | Estimate | Std. error | 95% CI low | 95% CI high |
|---|---|---|---|---|
| Naive TWFE | -0.0058 | 0.0036 | -0.0129 | 0.0013 |
| Sun & Abraham (aggregate ATT) | -0.0016 | 0.0071 | -0.0155 | 0.0122 |
Two things to check whenever you make this comparison in your own work:
- Sign agreement. If the two estimators disagreed on the direction of the effect, that would be a strong warning that the staggered-timing weighting is doing real damage to the TWFE coefficient.
- Magnitude agreement. The gap between the two coefficients is a measure of the weighting issue from Goodman-Bacon. In this sample, TWFE looks more negative than the Sun & Abraham ATT, which is exactly the pattern Goodman-Bacon predicts when earlier-treated cohorts are dragged into the comparison as controls for later-treated cohorts.
The number we report as our headline causal estimate is the Sun & Abraham aggregate ATT, with TWFE kept as a cross-check.
Step 8: Full regression output
For completeness, the script also writes a single combined table that stacks every model coefficient — TWFE, the Sun & Abraham event-study terms, and the Sun & Abraham aggregate ATT — into one object. This is the artefact you would lift directly into a paper or a memo.
| Model | Term | Estimate | Std. error | t-statistic | p-value |
|---|---|---|---|---|---|
| TWFE | treated | -0.0058 | 0.0036 | -1.596 | 0.118 |
| Sun & Abraham (event-study) | year::-11 | 0.0430 | 0.0174 | 2.477 | 0.017 |
| Sun & Abraham (event-study) | year::-10 | 0.0083 | 0.0203 | 0.410 | 0.684 |
| Sun & Abraham (event-study) | year::-9 | 0.0246 | 0.0157 | 1.570 | 0.124 |
| Sun & Abraham (event-study) | year::-8 | 0.0257 | 0.0108 | 2.376 | 0.022 |
| Sun & Abraham (event-study) | year::-7 | -0.0096 | 0.0088 | -1.087 | 0.283 |
| Sun & Abraham (event-study) | year::-6 | 0.0259 | 0.0082 | 3.152 | 0.003 |
| Sun & Abraham (event-study) | year::-5 | 0.0098 | 0.0087 | 1.131 | 0.264 |
| Sun & Abraham (event-study) | year::-4 | 0.0042 | 0.0082 | 0.519 | 0.606 |
| Sun & Abraham (event-study) | year::-3 | 0.0085 | 0.0071 | 1.189 | 0.241 |
| Sun & Abraham (event-study) | year::-2 | 0.0011 | 0.0059 | 0.191 | 0.849 |
| Sun & Abraham (event-study) | year::0 | 0.0006 | 0.0074 | 0.075 | 0.940 |
| Sun & Abraham (event-study) | year::1 | 0.0004 | 0.0072 | 0.053 | 0.958 |
| Sun & Abraham (event-study) | year::2 | -0.0022 | 0.0088 | -0.248 | 0.805 |
| Sun & Abraham (event-study) | year::3 | -0.0040 | 0.0083 | -0.478 | 0.635 |
| Sun & Abraham (event-study) | year::4 | -0.0099 | 0.0100 | -0.985 | 0.330 |
| Sun & Abraham (event-study) | year::5 | 0.0009 | 0.0117 | 0.080 | 0.937 |
| Sun & Abraham (event-study) | year::6 | -0.0038 | 0.0151 | -0.252 | 0.802 |
| Sun & Abraham (event-study) | year::7 | -0.0022 | 0.0118 | -0.189 | 0.851 |
| Sun & Abraham (event-study) | year::8 | 0.0056 | 0.0183 | 0.304 | 0.762 |
| Sun & Abraham (event-study) | year::9 | 0.0046 | 0.0136 | 0.340 | 0.735 |
| Sun & Abraham (aggregate ATT) | ATT | -0.0016 | 0.0071 | -0.232 | 0.817 |
The script also exports LaTeX versions of the cohort summary, the estimator comparison, and the regression output to output/tables/ta4_*.tex, ready to be \input{}-ed into a draft.
Question for you
Suppose you reran this exact analysis on a panel where treatment effects grew much faster with event time, and where most banks were treated early in the sample. What would you expect to happen to the gap between the TWFE coefficient and the Sun & Abraham aggregate ATT?
Sketch your answer in two lines, then look back at the event-study plot to back it up.
What to take away
A few things to internalise from this TA:
- DiD identifies the ATT, under the parallel-trends assumption. It does not identify the ATE, and it does not require treated and control units to be identical in levels.
- With staggered timing, the standard TWFE coefficient mixes good and bad 2x2 comparisons. When the bias is empirically large, the headline number changes when you switch to a heterogeneity-robust estimator.
- Always report a TWFE estimate and at least one staggered-DiD estimator (Sun & Abraham, Callaway & Sant’Anna, …), plus an event-study plot. The TWFE coefficient is a diagnostic, not the final answer.
- Cluster standard errors at the level of treatment assignment, drop always-treated units, decide the event window before looking at results, and remember that pre-treatment placebos are evidence about parallel trends, not a formal test.
A few questions you should be able to answer at this stage:
- Why is parallel trends untestable, and what do we use as evidence in its favour?
- What exactly is the “forbidden comparison” in Goodman-Bacon’s decomposition?
- Why must never-treated cohorts be coded above the maximum period for
fixest::sunab()to recognise them? - If the event-study plot showed a clear downward jump at event time = 0 followed by a flat profile, what would you conclude about the dynamic structure of the effect?
- If the pre-treatment placebos in the event-study plot were systematically negative, what would that imply about the design?
For the formal references behind everything above, the staggered-DiD literature is now relatively self-contained:
- Goodman-Bacon (2021), “Difference-in-differences with variation in treatment timing”, Journal of Econometrics.
- Sun & Abraham (2021), “Estimating dynamic treatment effects in event studies with heterogeneous treatment effects”, Journal of Econometrics.
- Callaway & Sant’Anna (2021), “Difference-in-differences with multiple time periods”, Journal of Econometrics.
- Baker, Larcker & Wang (2022), “How much should we trust staggered difference-in-differences estimates?”, Journal of Financial Economics.
- Roth, Sant’Anna, Bilinski & Poe (2023), “What’s trending in difference-in-differences? A synthesis of the recent econometrics literature”, Journal of Econometrics.