It’s a frustrating fact of life that arguments that are most visible are always going to be, for most people, the arguments that define the truth. That’s certainly the case with Academically Adrift, the book by Richard Arum and Joseph Roksa that has done so much to set the conventional wisdom about the value of college. Many people assume that the book’s argument is the final word. I have, in the past, pointed to some critical words on its methodology, or the methodology we’re allowed to see. (One of the primary complaints about the book is that the authors hide the evidence for some of their claims.) Richard Haswell’s review, available here, is particularly cogent and critical.

However, even setting that aside, it’s essential to say: later research, using the same primary mechanism, the Collegiate Learning Assessment, has done a great deal to undermine the book’s claims.

My attitude towards the CLA and its successor, the CLA+, is complex and evolving. (I’m writing my dissertation on the CLA+, as it happens.) I can come up with both a list of things that I like about it and a list of things that I don’t like about it. It’s a long and complex topic. (Good thing I have a whole diss to work through it!) However, setting those controversies aside: that’s the primary mechanism through which Arum and Roksa make their arguments. Yet research with a far larger data set and undertaken using the freshman-to-senior academic cycle that the CLA was intended to use has shown far larger gains than those reported by Arum to Roksa. This report from CAE details research on a larger selection of schools than that measured in Academically Adrift. In contrast to the now-notorious .18 SAT-normed standard deviation growth in performance that Arum and Roksa found, they find an average growth of .78 SAT-normed standard deviations, with no school demonstrating an effect size of less than .62.

Like I said, there’s a whole host of conversations that could be had here. It’s very noteworthy and important to point out that the r² of the CLA’s correlation with SAT scores is .81, meaning that only 19% of the difference in student CLA scores can plausibly be ascribed to college effects, with some undoubtedly owing to demographic and individual student factors. But for here, I simply want to point out: uncontroversially more complete and wide-ranging research, undertaken with the same mechanism as used in Academically Adrift, had findings that contradict its central claim. It would be nice if that would seep out into the public consciousness a bit.