In response to my last post I got another email of the type “you’re saying there’s no learning, you’re saying people are cursed, so why have schools/colleges at all, etc.” (I also got the typical influx of racist IQ science types, which, seriously, you guys, find somebody else, please.) I thought I made it pretty clear that I do think learning happens, and that in fact I’ve helped my students do it. My point is that this learning takes great effort, and that in my opinion dramatically reducing individual instructor attention and face-to-face contact will result in a serious reduction in learning. I particularly think this is true with the kind of vulnerable students who we need to add to the ranks of the college educated, according to the popular viewpoint that is often expressed by President Obama. It’s also my position that learning is possible and powerful but can’t change the relative positions of large groups.

So let’s get wonky and check out a really useful graph.

Council for Aid to Education

This graph shows two dimensions and two data sets. The blue circles are institutional averages for freshman. The red squares are institutional averages for seniors. The X axis shows the mean SAT score for institutions, and the Y axis, mean scores on the Collegiate Learning Assessment (CLA). The blue regression line shows the relationship between institutional mean SAT scores and institutional mean CLA scores for freshman, and the red, the same relationship for seniors. Both of these relationships are positive and quite strong.

This particular graph is from 2005-2006. (This happens to be pulled from the report for Concord College, which you can see individually in the filled in red and blue dots.) Having stared at more of these graphs in the last month than I care to admit, I can tell you that this is what you see in all of the available years of CLA data: a healthy increase in aggregate scores from freshman to senior year, with a highly explanatory relationship between ability effects, as measured by SATs, and performance on the CLA. An important caveat here is that this data is cross sectional, not longitudinal; that is, the seniors represented here are not the same students as the freshman. However, with careful sampling and rigorous controls, we can still demonstrate the average improvement within an institution with great fidelity, and indeed that limited longitudinal data we do have with this mechanism suggests just that.

So: are graphs like this one good news or bad news?

Well if you’re worried, as a lot of tongue cluckers are, that there’s no learning happening at American colleges, you shouldn’t be, provided you trust the CLA. Every school in this data set improved significantly. In the most recent data (which is what you’re likely most interested in), there was an average growth of .78 of a standard deviation, with no institution showing less than .62. From the time of the test administrations shown in this example to the 2011-2012 administrations, the institutional average has been .73 of a standard deviation. If that sounds small, trust me, it isn’t. For improvements over that many thousands of students, it’s quite large. Note that the CLA is the same mechanism that was used in the research for Academically Adrift, which famously argued that college students aren’t learning. The difference is that the CAE’s research is more larger, considers a longer time frame, and is altogether better.

Now for the other perspective: the fact remains that a very large percentage of the variation in CLA scores is explained by a test taken before the students actually started college. As you can see above, for the year in this chart, the r-squared is .75 for all administrations. R-squared is a statistic  that tells us how much of a particular change in one variable is explained by another variable. What an r-squared that high means is that even before accounting for all of the within-institution factors like demographics and funding, etc., institutions could only be responsible for about 25% of what the CLA can represent. This explanatory power of prior learning/ability effects/etc. does not decline from freshman to senior test administrations. (In this particular year, in fact, the r-squared increased, although it’s a trivial change.) This is why I say both that learning is real, and potentially quite valuable in a variety of ways, and that education has limited ability to transcend structural and individual inequalities.

A crude but effective way to look at a chart like this one is to say that the distance between the two regression lines shows education’s potential. The regressions themselves show education’s limits.

It’s essential to say that these are mean scores, here. Were we to look at a scatter plot of individual scores, there would be far more variability here. (Looking at institutional means — sample means — gives us the kind of pretty graphs we see above.) Plenty of students would likely exceed their expected CLA result– but, given that scores on both tests are roughly normally distributed, a roughly equivalent number of students would underperform. So when people accuse me of being illiberal in skepticism about the ability of education to change outcomes, they are perhaps mistaking what I’m saying about averages and trends for what I’m saying about individuals. Roses grow from concrete! Individual students come from backgrounds that deeply disadvantage them in many ways and go on to spectacular individual success. And there’s a certain traditionalist attitude that sees all of this as the way it should be: the point is to give individuals the opportunity to shine, and the cream will rise. (An attitude I will happily advance as a reason for robust affirmative action policies, in an opportunistic way.)

But what I care about from a political and policy standpoint is improving the economic outcomes of large groups. Matt Yglesias laid this case out brilliantly recently. I would put it the Eugene Debs way: “I don’t want to rise up from the ranks; I want to rise up with the ranks.” An individual rising from poverty can be a beautiful story, but it still leaves behind all the people left behind. And this, more or less, is the Obama argument: educating more and more people will enable more and more of them to improve their station. But the people who are not currently getting access to higher education now are far more likely to be at the left hand side of the chart, and given the strength of the regressions, they are likely to end up well below those who start out on the right hand side even if they attend a school with an exceptional average improvement. Again: not destiny for individuals. But statistically certain for groups. This is before we talk about dropout rates and the way the hardest to educate are self-selecting themselves out of the data.

Retention Rates by SAT Band, from the College Board

I happen to think that the growth shown in the CLA has a lot of benefits for individuals and societies alike. I imagine that this is a boon to economic productivity, and I happen to be one of those romantics who believes in education as a source of human flourishing. But the fact that the improvements shown are rather uniform, and are so bound to a student’s relative position before they go to college, should make us quite skeptical about college’s ability to rescue large groups from poverty or to reduce inequality. Even if we can dramatically expand the number of students from the left hand side of the SAT distribution entering and graduating from college, we’d still be left with a large portion of our society that would be left behind. And what people are calling for is expanding that portion while essentially eliminating individual attention and face-to-face intervention in college, when those students are those who need the most of both. I am neither a nihilist nor a romantic for saying that this isn’t going to work.

It’s nice when a rose grows from concrete, but I’d much rather that all roses grow from gardens. And the way you get there is with redistribution, not education.