In matching for causal inference, we often focus too much on reducing bias and too little on variance. This has generalizability implications. This paper, while not focused on external validity, tackles the bias-variance trade-off in matching for diff-in-diff:
While matching on covariates may reduce bias by creating a more comparable control group, this often comes at the cost of higher variance. Matching discards non-comparable control units, limiting the sample and, in turn, jeopardizing the precision of the estimate. That’s a good reminder.
How about matching also on pre-treatment outcomes?
Here, the win is clear: it’s a guaranteed reduction in variance because the sample-size trade-off no longer applies once matching is performed. So, while a reduction in bias isn’t a mathematical certainty, this makes additionally matching on pre-treatment outcomes a potentially optimal strategy when both bias and variance are a concern.
The generalizability implications will be part of the matching chapter of the Causal Book.
PS: Yes, matching on pre-treatment outcomes reduces the diff-in-diff estimator to diff-in-means and may introduce bias, but that’s a discussion for another day (and chapter).