Study of the Week: Feed Kids to Feed Them

Today’s Study of the Week is about subsidized meal programs for public school students, particularly breakfast. School breakfast programs have been targeted by policymakers for awhile, in part because of discouraging participation levels. Even many students who are eligible for subsidized lunches often don’t take advantage of school breakfast. The reasons for this are multiple. Price is certainly a factor. As you’d expect, price is inversely related to participation rates for school breakfast. Also, in order to take advantage of breakfast programs, you need to arrive at school early enough to eat before school formally begins, and it’s often hard enough to get teenagers to school on time just for class. Finally, there’s a stigma component, particularly associated with subsidized breakfast programs. It was certainly the case at my public high school, where 44% of students were eligible for federal school lunch subsidies, that school breakfast carried class associations. At lunch, everybody’s eating together, but students at breakfast tended to be poorer kids – which in turn likely makes it less likely that students will want to be seen getting school breakfast.

The study, written by Jacob Leos-Urbel, Amy Ellen Schwartz, Meryle Weinstein, and Sean Corcoran (all of NYU), takes advantage of a policy change in New York public schools in 2003. Previously, school breakfast had been free only to those who were eligible for federal lunch subsidies, which remains the case in most school districts. New York made breakfast free for all students, defraying the costs by raising the price of unsubsidized lunch from $1.00 to $1.50. They then went looking to see if the switch to free breakfast for all changed participation in the breakfast program, looking for differences between the three tiers – free lunch students, reduced lunch students, and students who pay full price. They also compared outcomes from traditional schools to Universal Free Meal (UFM) schools, where the percentage of eligible students is so high that everyone in the school gets meals for free already. This helped them tease out possible differences in participation based on moving to a universal free breakfast model. They were able to use a robust data set comprising results from 723,843 students from 667 schools, grades 3–8. They also investigated whether breakfast participation rates were associated with performance in quantitative educational metrics.

It’s important to say that it’s hard to really get at causality here because we’re not doing a randomized experiment. Such an experiment would be flatly unethical – “sorry, kid, you got sorted into the no-free-breakfast group, good luck.” So we have to do observational studies and use what techniques we can to adjust for their weaknesses. In this study, the authors used what’s called a difference in difference design. These techniques are often used when analyzing natural experiments. In the current case, we have schools where the change in policy has no impact on who receives free breakfast (the UFM schools) and schools where there is an impact (the traditional schools). Therefore the UFM schools can function as a kind of natural control group, since they did not receive the “treatment.” You then use a statistical model to compare the change in the variables of interest for the “control” group to the change for the “treatment” group. Make sense?

What did the authors find? The results of the policy change were modest, in almost every measurable way, and consistent across a number of models that the authors go into in great detail in the paper. Students did take advantage of school breakfast more after breakfast became universally free. On the one hand, students who paid full price increased breakfast participation by 55%, which is a large number; but on the other hand, their initial baseline participation rates were so low (again because breakfast participation is class-influenced) that they only ate on average 6 additional breakfasts a year. Reduced price and free were increased by 33% and 15%, respectively – the latter particularly interesting given that those students did not pay for breakfast to begin with. Still, that too only represents about 6 meals over the course of a year, not nothing but perhaps less than we’d hope for a program with low participation rates. The only meaningful difference in models seems to be when they restrict their analysis to the small number (91) of schools where less than a third of students are eligible for lunch subsidies, in which case breakfast participation grew by a substantially larger amount. The purchase of lunches, for what it’s worth, remained static despite the price increase.

There’s a lot of picking apart the data and attempting to determine to what degree these findings are related to stigma. I confess I find the discussion a bit muddled but your money may vary. The educational impacts, also, were slight. They found a small increase in attendance, but this result was not significant, and no impact on reading and math outcomes.

These findings are somewhat discouraging. Certainly we would hope that moving to a universal program would help to spur participation rates to a greater degree than we’re seeing here. But it’s important to note that the authors largely restricted their analysis to the years immediately before and after the policy change, thanks to the needs of their model. When broadening the time frame by a couple years, they find an accelerating trend in participation rates, though the model is somewhat less robust. What’s more, as the authors note, decreasing stigma is the kind of thing that takes time. If it is in fact the case that stigma keeps students from taking part in school breakfast, it may well take a longer time period for universal free breakfast to erode that disincentive.

I’m also inclined to suspect that the need to get kids to school early to eat represents a serious challenge to the pragmatic success of this program. There’s perhaps good news on the way:

Even when free for all, school breakfast is voluntary. Further, unlike school lunch, breakfast traditionally is not fully incorporated into the school day and students must arrive at school early in order to participate. Importantly, in the time period since the introduction of the universal free breakfast policy considered in this paper, New York City and other large cities have begun to explore other avenues to increase participation. Most notably, some schools now provide breakfast in the classroom.

Ultimately, I believe that making school breakfast universally free is a great change even in light of relatively modest impacts on participation rate. We should embrace providing free breakfast to all students regardless of income level out of the principle of doing so, particularly considering that fluctuations in parental income might make kids who are technically ineligible unable to pay for breakfast. In time, if we set up this universal program as an embedded part of the school day, and work diligently to erase the stigma of using it, I believe more and more kids will begin their days with a full stomach.

As for the lack of impacts on quantitative metrics, well – I think that’s no real objection at all. We should feed kids to feed them, not to improve their numbers. This all dovetails with my earlier point about after school programs: if we insist on viewing every question through the lens of test scores, we’re missing out on opportunities to improve the lives of children and parents that are real and important. Again, I will say that I recognize the value of quantitative academic outcome in certain policy situations. But the relentless focus on quantitative outcomes leads to scenarios where we have to ask questions like whether giving kids free breakfast improves test scores. If it does, great – but the reason to feed children is to feed children. When it comes to test scores and education policy, the tail too often wags the dog, and it has to stop.