More on this book
Kindle Notes & Highlights
Read between
January 7 - February 14, 2023
Economists’ use of data to answer cause-and-effect questions constitutes the field of applied econometrics, known to students and masters alike as ’metrics.
Because, as this hypothetical comparison suggests, real other things equal comparisons are hard to engineer, some would even say impossibile (that’s Italian not Latin, but at least people still speak it).
By changing circumstances randomly, we make it highly likely that the variable of interest is unrelated to the many other factors determining the outcomes we mean to study. Random assignment isn’t the same as holding everything else fixed, but it has the same effect.
The methods they use—random assignment, regression, instrumental variables, regression discontinuity designs, and differences-in-differences—are the Furious Five of econometric research.
The United States spends more of its GDP on health care than do other developed nations, yet Americans are surprisingly unhealthy. For example, Americans are more likely to be overweight and die sooner than their Canadian cousins, who spend only about two-thirds as much on care.
So it is with any choice, including those related to health insurance: would uninsured men with heart disease be disease-free if they had insurance?
In the novel Light Years, James Salter’s irresolute narrator observes: “Acts demolish their alternatives, that is the paradox.” We can’t know what lies at the end of the road not taken.
National Health Interview Survey (NHIS),
A good control group reveals the fate of the treated in a counterfactual world where they are not treated.
More-educated people, for example, tend to be healthier as well as being overrepresented in the insured group. This may be because more-educated people exercise more, smoke less, and are more likely to wear seat belts.
This term, equal to − 2, reflects Khuzdar’s relative frailty. In the context of our effort to uncover causal effects, the lack of comparability captured by the second term is called selection bias.
The insured in the NHIS are healthier for all sorts of reasons, including, perhaps, the causal effects of insurance. But the insured are also healthier because they are more educated, among other things. To see why this matters, imagine a world in which the causal effect of insurance is zero (that is, κ=0). Even in such a world, we should expect insured NHIS respondents to be healthier, simply because they are more educated, richer, and so on. This positive selection bias runs counter to the negative selection bias we imagined in the parable of frail, insured Khuzdar and hearty, uninsured
...more
As we’ll see in the next chapter, if the only source of selection bias is a set of differences in characteristics that we can observe and measure, selection bias is (relatively) easy to fix.
The principal challenge facing masters of ’metrics is elimination of the selection bias that arises from such unobserved differences.
We must randomly assign treatment in a sample that’s large enough to ensure that differences in individual characteristics like sex wash out.
Two randomly chosen groups, when large enough, are indeed comparable. This fact is due to a powerful statistical property known as the Law of Large Numbers (LLN).
MATHEMATICAL EXPECTATION The mathematical expectation of a variable, Yi, written E[Yi], is the population average of this variable. If Yi is a variable generated by a random process, such as throwing a die, E[Yi] is the average in infinitely many repetitions of this process. If Yi is a variable that comes from a sample survey, E[Yi] is the average obtained if everyone in the population from which the sample is drawn were to be enumerated.
CONDITIONAL EXPECTATION The conditional expectation of a variable, Yi, given a dummy variable, Di = 1, is written E[Yi | Di = 1]. This is the average of Yi in the population that has Di equal to 1. Likewise, the conditional expectation of a variable, Yi, given Di = 0, written E[Yi|Di = 0], is the average of Yi in the population that has Di equal to 0. If Yi and Di are variables generated by a random process, such as throwing a die under different circumstances, E[Yi|Di = d] is the average of infinitely many repetitions of this process while holding the circumstances indicated by Di fixed at d.
...more
Because randomly assigned treatment and control groups come from the same underlying population, they are the same in every way, including their expected Y0i.
Random assignment works not by eliminating individual differences but rather by ensuring that the mix of individuals being compared is the same.
When analyzing data from a randomized trial or any other research design, masters almost always begin with a check on whether treatment and control groups indeed look similar.
In any statistical sample, chance differences arise because we’re looking at one of many possible draws from the underlying population from which we’ve sampled.
The standard error of a difference in averages is a measure of its statistical precision: when a difference in sample averages is smaller than about two standard errors, the difference is typically judged to be a chance finding compatible with the hypothesis that the populations from which these samples were drawn are, in fact, the same.
Differences that are larger than about two standard errors are said to be statistically significant: in such cases, it is highly unlikely (though not impossible) that these differences arose purely by chance.
The first important finding to emerge from the HIE was that subjects assigned to more generous insurance plans used substantially more health care.
More importantly, today’s uninsured Americans differ considerably from the HIE population: most of the uninsured are younger, less educated, poorer, and less likely to be working. The value of extra health care in such a group might be very different than for the middle class families that participated in the HIE.
The weak health effects of the OHP lottery disappointed policymakers who looked to publicly provided insurance to generate a health dividend for low-income Americans. The fact that health insurance increased rather than decreased expensive emergency department use is especially frustrating. At the same time, panel B of Table 1.6 reveals that health insurance provided the sort of financial safety net for which it was designed. Specifically, households winning the lottery were less likely to have incurred large medical expenses or to have accumulated debt generated by the need to pay for health
...more
This implies that the effect of being insured is as much as four times larger than the effect of winning the OHP lottery (statistical significance is unchanged by this adjustment).
The RAND and Oregon findings are remarkably similar. Two ambitious experiments targeting substantially different populations show that the use of health-care services increases sharply in response to insurance coverage, while neither experiment reveals much of an insurance effect on physical health. In 2008, OHP lottery winners enjoyed small but noticeable improvements in mental health. Importantly, and not coincidentally, OHP also succeeded in insulating many lottery winners from the financial consequences of poor health, just as a good insurance policy should. At the same time, these studies
...more
Quantifying sampling uncertainty is a necessary step in any empirical project and on the road to understanding statistical claims made by others. We explain the basic inference idea here in the context of HIE treatment effects.
Expectations can be written as a weighted average of all possible values that the variable Yi can take on, with weights given by the probability these values appear in the population. In our dice-throwing example, these weights are equal and given by 1∕6 (see Section 1.1).
The sample average, Ȳ, is a good estimator of E[Yi] (in statistics, an estimator is any function of sample data used to estimate parameters).
UNBIASEDNESS OF THE SAMPLE MEAN E[Ȳ] = E[Yi
The standard deviation of a statistic like the sample average is called its standard error.
The standard error summarizes the variability in an estimate due to random sampling.
One last step on the road to standard errors: most population quantities, including the standard deviation in the numerator of (1.6), are unknown and must be estimated. In practice, therefore, when quantifying the sampling variance of a sample mean, we work with an estimated standard error.
One miraculous statistical fact is that if E[Yi] is indeed equal to μ, then—as long as the sample is large enough—the quantity t(μ) has a sampling distribution that is very close to a bell-shaped standard normal distribution, sketched in Figure 1.1. This property, which applies regardless of whether Yi itself is normally distributed, is called the Central Limit Theorem (CLT).
Wielded skillfully, ’metrics tools other than random assignment can have much of the causality-revealing power of a real experiment. The most basic of these tools is regression, which compares treatment and control subjects who have the same observed characteristics.
Regression-based causal inference is predicated on the assumption that when key observed variables have been made equal across treatment and control groups, selection bias from the things we can’t see is also mostly eliminated.
Comparisons of earnings between those who attend different sorts of schools invariably reveal large gaps in favor of elite-college alumni.
