notes for 3/31/17

  • The first book review should be out for subscribers today! It’s a reprint of an academic review I wrote, but in the future they’ll all be new content. Just need a week to read a couple more books appropriate for this blog. This is the first time I’ve ever distributed a reward through Patreon so please let me know if you didn’t receive an email or can’t access the review. If you’re not a subscriber yet, think it over! I’m exploring some cool options for other rewards and I hope to let you know about some of them soon.
  • Be sure to spread the word about this project if you like it.
  • Some readers have pointed out that the rates for SAT participation in some states are so low (mentioned in this post) because those states require the ACT as a learning assessment. Which is certainly true! But note that the point of that post isn’t to say “look at how low these participation rates are” but rather to explore selection bias, which in the case of ACT-dominant regions would be even more pronounced – only the very motivated students, particularly those looking to attend elite private institutions, would be likely to take the SAT.
  • I have gotten a fair amount of pushback on the idea that randomized trials of charter school efficacy aren’t really random. I agree that this is an idea that I need to explore at greater length in the future. In addition to what I suspect is lurking non-random distribution, I think the bigger question is whether “charter school” even makes sense as a condition suitable for randomization. More to come.
  • On the other side, I appear to have been too kind to the CREDO studies. To call survivorship bias a demonstration of quality on the part of charters is just… not cool.
  • The first Study of the Week post should come out on Monday. It’s a big meaty one and I’m really happy with how it’s shaping up. Not 100% sure but I’m guessing I’ll distribute book reviews on the weekend and do Study of the Week on Monday or Tuesday. And feel free to email me with suggestions or requests.

looking beyond test scores in defense of after school programs

The Trump administration has proposed cutting funding for a program that provides after school programs for low-income students. At the Atlantic, Leah Askarinam defends the programs. I’m on board with continuing to fund them, but I find her defense counterproductive.

Askarinam’s argument is kind of strange. The Brookings Institution ran several large-n studies in the middle of the aughts that showed, without much ambiguity at all, that the quantitative learning gains from these programs are minimal. Askarinam fixates on the age of those studies as a reason to question their validity. It’s true that the latest study on the efficacy of these programs is about a decade old, which isn’t ideal, but also isn’t unusual; it’s really hard, far harder than most people think, to run effective large-scale social science research projects. More to the point, why would we assume that something fundamental has changed in the outcomes of these programs in the past 10 years? She notes that the federal policy situation was different then, but that hardly seems to be sufficiently explanatory for me – the federal education policy situation changes all the time, without seeing systematic differences in student outcomes. (Indeed, the irrelevance of federal education policy to student outcomes is the source of great lamentation.) Consider the standard here: if ten+ year old studies had shown robust learning gains, would Askarinam now say that they were too old to be trusted? Such a standard would cut both ways, after all.

And while it’s true that absence of evidence isn’t evidence of absence, absence of evidence is… absence of evidence. Askarinam offers some anecdotal evidence of academic improvement, discusses internal research, and speaks generally of gains not captured by those older studies. That’s fine as far as it goes, but none of it amounts to responsible evidence for the kind of quantitative gains the Brooking studies were looking for. More study is needed, obviously – you can use that phrase like a comma when you’re talking about ed research – but as with pre-K programs, I think if the question is “are test score and other quantitative gains in outcomes sufficient to justify the expense of publicly-funded after school programs?,” the answer is clearly no.

So am I opposed to funding for after school programs? No, not at all. I just think we should fund them for defensible reasons. Askarinam quotes David Muhlhausen of the conservative Heritage Foundation, “It’s a place to have their kids while the parents are at work,” Muhlhausen said. “That’s the real key to these programs and why they’re popular—not that they provide any benefits to the students. It’s basically a babysitting program for parents who aren’t home.”

Sounds good to me.

The birth of publicly-funded, federally-guaranteed education for children aged 5-18 was one of the greatest advancements in human well-being in history. It helped move millions of children into formal education, providing not only the various benefits of schooling to them but also the essential ancillary benefit of childcare. This in turn made it easier for both parents to work. While we might lament the fact that it’s now necessary for most households to have two incomes to survive, the fact is that it is necessary, and without the free childcare that public schools provide, family life would be impossible for much of the country. Public education also helps our slow, imperfect march towards gender equality. And in a world where digital technologies make it easier and easier to avoid interacting with people who are outside our immediate familial and friend networks, formal schooling can help make the kinds of connections between people from radically different backgrounds that are essential for a functioning democracy.

The cost of these programs is around $1 billion a year, or about one quarter of one percent of what we’ve spent on the failed F-35 jet project in the past 15 years.

In an era of stagnant real incomes for most workers and spiraling costs of housing, healthcare, and higher education, programs that provide safe supervision for children are worth supporting. “Traditional values” conservatives should embrace programs that make child rearing feasible for more families; liberals and leftists should appreciate expanding government assistance and taking more social goods (like childcare) out of the market.

Askarinam’s defensiveness, it seems to me, reflects the way that the widespread acceptance of test score obsession has boxed us in. Too many well-meaning progressives have adopted this reductive view of the purpose of education; they then end up unable to defend programs they favor when the results of those programs on test scores are inevitably small or nonexistent. The universal pre-K debate is a perfect example. The endless back-and-forth involves credible arguments from both supporters and skeptics, but few would question that the test score and other quantitative gains we’re arguing over are modest. So stop arguing through that frame. As long as test scores are taken as the criterion of interest, we’ll be playing defense. Instead, we should argue that the basic benefit of pre-K and after school programs is to provide essential childcare support to struggling families, and to provide social and personal enrichment that has value even if uncorrelated with test score increases. We need to expand our definitions of the purpose of education outside of the quantitative, rather than staying rooted in a frame that often doesn’t help us. Askarinam describes an after school program that offers social and emotional health benefits. That’s worth fighting for on its own. So articulate that case, and do the same with pre-K. Argue from strength, not weakness.

welcome to the ANOVA

Hi there, my name is Freddie deBoer. I’ve been blogging off and on since 2008. I’ve also written for many newspapers, magazines, and websites. (You can see some of my published writing by clicking the My Work tab above.) In my professional life, I work at Brooklyn College in the City University of New York in the Office of Academic Assessment, where I work with faculty to help them develop and implement faculty-led assessments of student learning, and as coordinator of the Writing Across the Curriculum program. This project, a new blog called the ANOVA, is designed to combine those two parts of my life while narrowing and focusing my engagement.

The ANOVA will be about education research and education policy. That way, I can continue to work and research in education in my professional life, and take the reading and engagement I’m doing and make them useful for a popular audience. I will discuss major trends in education, legislation and federal policy related to education, new and existing research in the field, and the philosophy and purpose of education. I expect I will post 3-4 times a week. One of these posts will be a Study of the Week, where I look at a prominent, problematic, or interesting research study in education, whether old or new, discussing the findings and what they mean for the broader world.

I will be attempting to monetize this blog through Patreon, so please consider pledging to support this project financially. Those who contribute $5 a month or more will get access to a weekly book review. If the amount of contributions exceeds my expectations, I will think of other ways to reward patrons. You can also make a one-time donation on PayPal.

Why “the ANOVA”? Because the term, which stands for Analysis Of VAriance, refers to a statistical technique commonly used in education research; because the attempt to define how variance in educational outcomes are determined by predictor variables is perhaps the essential question in quantitative study of education; and because it’s a beautiful word.

I will not avoid talking about the political dimensions of education. Education is an inherently political topic. However, this will not be a political blog and will feature no political writing that is not narrowly focused on education. I will not, for example, weigh in on the campus political wars in this space. When in doubt, I will err on the side of not engaging if a subject is not clearly directly concerned with education. Please bear that in mind if you’re thinking about contributing. It should go without saying that this project will not be affiliated with or endorsed by Brooklyn College in any way, and that I will not be working on it during my regular work hours.

I’ve gotten a lot out of writing online, but it has had downsides, especially concerning people targeting my employment. Online politics, are not good for my mental well-being. As someone with poor impulse control and bipolar disorder, it’s best to limit my political engagement in digital mediums that favor immediacy over thoughtfulness. I also have found much better ways to utilize my political energy in recent months. Since moving to New York I’ve gotten involved in my own union, in a tenant’s union, and in local education politics, along with attending many protests. This has been wonderful for my mood and sense of political purpose. Online politics leave me discouraged and unhappy; offline politics make me hopeful and energized. So I intend to keep my political engagement squarely offline.

This is a modest project with modest goals. I want an outlet where I can write for a small audience of interested people and share a little of my expertise and my opinions. I’m hoping to carve out a niche where I can engage productively and professionally about topics related to my expertise and which I am passionate about. I hope you join me.