Consider this fairly recent NYT piece, titled “A Simple Way to Help Low-Income Students: Make Everyone Take SAT or ACT.” The essay talks about research that shows that making these tests mandatory in high schools raises the participation rate (duh) and in so doing identifies high-achieving students who ordinarily would not have taken the test. See, typically those who are screened out of taking a college entrance exam through self-selection are those who are less college ready and perform less well. But this is far from universal, and there are many potential high-scoring testers who are screened out through fees, lack of parental guidance, or a lack of information about when and how to sign up. The research discussed showed that low-income but high performing students are less likely to take these tests than high-income, high-performing students, and that making the tests mandatory will thus lower the relative disadvantage of those students. Mandating the test is thus a tool for equality – it increases the opportunity for students who are typically systematically excluded from college.
Let’s think about things for a second. First, and to be clear, the research does not show that low-income students are more likely to perform well. The opposite is the case:
(Data’s a bit old, but this is a durable outcome.) So having more low-income students taking the SAT will likely mean finding that many low-income students are in fact not prepared for college, to go along with finding those high-performing kids who we wouldn’t otherwise. Still, obviously I think it’s a noble and necessary goal to help identify talented students from poor families. The point is that it’s odd to think of this as a project for increasing equality as such. We’re simply looking for more “diamonds in the rough,” and hopefully helping to pull them out from their peers – who are thus left even further behind.
Here’s a point to stress: the very purpose of educational testing is to identify inequality. That is, we develop and administer tests precisely to better understand how students are not the same. In fact, the most precisely that tests are, the more unequal we understand the tested population to be. A 10 question test likely has less discriminatory power than a 100 question test, and thus the 100 question test is more likely to differentiate between closely-grouped students – that is, to identify how they are unequal. Progress in educational testing stems from designing instruments that are more sensitive to underlying inequality. That’s the very name of the game.
As I’ve said before, we talk about education as fulfilling two functions that are not just in tension with each other but directly contradictory: education is discussed as a tool for creating greater socioeconomic equality, and as a system for identifying excellence and rewarding it with status and economic opportunity. The problems here should be obvious.There’s a much larger conversation about summative equality and equality of opportunity here, which is too directly political to get into in this space. (I will say that I think equality of opportunity is not really a coherent idea when you pull at it a bit.)
But from the standpoint of educational policy, it’s not clear to me that we really know what we want to be doing. Some people tell me that our goal should just be to move everyone up in terms of absolute achievement, raising averages without necessarily changing relative performance. That might have lots of good effects, but it’s by definition not something that could help with inequality, as what’s rewarded by the labor market is relative educational achievement, not absolute. If everyone who ever went to an Ivy League school was sent to the moon, they’d simply look for the next rung down and hire accordingly. If the purpose is instead to shrink the variance, to narrow the range between the top and bottom of the achievement scale, we’d want to talk about limiting resources to the top-performing kids, and we’d still be looking for differences in what individuals can do. And we have no good reason to think that we can achieve either at scale, because while some interventions have helped different groups at different times, the general bell-shaped distribution of overall achievement on any identifiable quantitative metric of academic success has been persistent and unchanging over time.
As long as we use education as a system for sorting students into different tranches of ability, and as long as that sorting system is a key mechanism for placing people into different levels of income and joblessness, we can’t conceive of our system as being an engine of socioeconomic equality. We might sometimes use testing to identify areas where more resources are needed and distribute them accordingly. But 15 years since No Child Left Behind and the testing-heavy era it augured, we have seen almost nothing in the way of convincing proof that testing is a reliable tool for raising standards and increasing either equality or opportunity. Tests are powerful things, and modern test-development can produce exams of extraordinary precision. But they can’t be useful until we have a clear and coherent vision of what we’re testing for. To get that understanding, we have to begin to pull apart our basic assumptions about education and our economy, to ask ourselves if the system can do what we imagine it can do.
Not too long ago, I felt the need to change the stream of personalities and attitudes that were pouring into my head, and it’s been remarkable.
This was really the product of idiosyncratic personal conditions, but it’s ended up being a good intellectual exercise too. I had to rearrange a few things in my digital social life. And concurrently I had realized that my sense of the world was being distorted by the flow of information that was being deposited into my brain via the internet. I hadn’t really lost a sense of what the “other side” thinks politically; I’m still one of those geezers who forces himself to read Reason and the Wall Street Journal op/ed page and, god help me, National Review. But I had definitely lost a sense of the mental lives of people who did not occupy my various weird interests.
What were other people thinking about, at least as far as could be gleaned by what they shared online? What appeared to be a big deal to them and what didn’t? I had lost my sense of social proportion. I couldn’t tell if the things my friends were obsessing about were things that the rest of the world was obsessing about. Talking to IRL friends that don’t post much or at all online helped give me a sense that I was missing something. But I didn’t know what.
No, I had to use the tools available to me to dramatically change the opinions and ideas and attitudes that were coming flowing into my mental life. And it had become clear that, though I have an RSS feed and I peruse certain websites and publications regularly, though I still read lots of books and physical journals and magazines, the opinions I was receiving were coming overwhelmingly through social media. People shared things and commented on what they shared on Facebook and Twitter, they made clear what ideas were permissible and what weren’t on Facebook and Twitter, they defined the shared mental world on Facebook and Twitter. They created a language that, if you weren’t paying attention, looked like the lingua franca. I’m sure there are people out there who can take all of this in with the proper perspective and not allow it to subtly shape your perception of social attitudes writ large. But I can’t.
It’s all particularly disturbing because a lot of what you see and don’t online is the product of algorithms that are blunt instruments at best.
So I set about disconnecting, temporarily, from certain people, groups, publications, and conversations. I found voices that popped up in my feeds a lot and muted them. I unfollowed groups and pages. I looked out for certain markers of status and social belonging and used them as guides for what to avoid. I was less interested in avoiding certain subjects than I was in avoiding certain perspectives, the social frames that we all use to understand the world. The news cycle was what it was; I could not avoid Trump, as wonderful as that sounds. But I could avoid a certain way of looking at Trump, and at the broader world. In particular I wanted to look past what we once called ideology: I wanted to see the ways in which my internet-mediated intellectual life was dominated by assumptions that did not recognize themselves as assumptions, to understand how the perspective that did not understand itself to be a perspective had distorted my vision of the world. I wanted to better see the water in which my school of fish swims.
Now this can be touchy – mutually connecting with people on social media has become a loaded thing in IRL relationships, for better or worse. Luckily both Facebook and Twitter give you ways to not see someone’s posts without them knowing and without severing the connection. Just make a list of people, pages, and publications that you want to take a diet from, and after a month or two of seeing how different things look, go back to following them. (Alternatively: don’t.) Really do it! The tools are there, and you can always revert back. Just keep a record of what you’re doing.
I was prepared for this to result in a markedly different online experience for me, and for it to somewhat change my perception of what “everyone” thinks, of what people are reading, watching, and listening to, etc. But even so, I’ve been floored by how dramatically different the online world looks with a little manipulation of the feeds. A few subjects dropped out entirely; the Twin Peaks reboot went from being everywhere to being nowhere, for example. But what really changed was the affect through which the world was presenting itself to me.
You would not be surprised by what my lenses appear to have been (and still largely to be): very college educated, very left-leaing, very New York, very media-savvy, very middlebrow, and for lack of a better word, very “cool.” That is, the perspective that I had tried to wean myself off of was made up of people whose online self-presentation is ostentatiously ironic, in-joke heavy, filled with cultural references that are designed to hit just the right level of obscurity, and generally oriented towards impressing people through being performatively not impressed by anything. It was made up of people who are passionately invested in not appearing to be passionately invested in anything. It’s a sensibility that you can trace back to Gawker and Spy magazine and much, much further back than that, if you care to.
Perhaps most dramatic was the changes to what – and who – was perceived as a Big Deal. By cutting out a hundred voices or fewer, things and people that everybody talks about became things and people that nobody talks about. The internet is a technology for creating small ponds for us to all be big fish in. But you change your perspective just slightly, move over just an inch, and suddenly you get a sense of just how few people know about you or could possibly care. It’s oddly comforting, to be reminded that even if you enjoy a little internet notoriety, the average person on the street could not care less who you are or what you do. I recommend it.
Of course, there are profound limits to this. My feeds are still dominantly coming from a few overlapping social cultures. Trimming who I’m following hasn’t meant that I’m suddenly connected to more high school dropouts, orthodox Jews, senior citizens, or people who don’t speak English. I would never pretend that this little exercise has given me a truly broad perspective. The point has just been to see how dramatically a few changes to my digital life could alter my perception of “the conversation.” And it’s done that. More than ever, I worry that our sense of shared political assumptions and the perceived immorality of the status quo is the result of systems that exclude a large mass of people, whose opinions will surely matter in the political wars ahead.
I am now adding some of what I cut back in to my digital life. The point was never really to avoid particular publications or people. I like some of what and who I had cut out very much. The point is to remain alive to how arbitrary and idiosyncratic changes in the constant flow of information can alter our perception of the human race. It’s something I intend to do once a year or so, to jolt myself back into understanding how limiting my perspective really is.
Everyone knows, these days, that we’re living in digitally-enabled bubbles. The trouble is that our instincts are naturally to believe that everyone else is in a bubble, or at least that their bubbles are smaller and with thicker walls. But people like me – college educated, living in an urban enclave, at least socially liberal, tuned in to arts and culture news and criticism, possessed of the vocabulary of media and the academy, “savvy” – you face unique temptations in this regard. No, I don’t think that this kind of bubble is the same as someone who only gets their news from InfoWars and Breitbart. But the fact that so many people like me write the professional internet, the fact that the creators of the idioms and attitudes of our newsmedia and cultural industry almost universally come from a very thin slice of the American populace, is genuinely dangerous.
To regain perspective takes effort, and I encourage you all to expend that effort, particularly if you are an academic or journalist. Your world is small, and our world is big.
Today’s Study of the Week combines two subjects we’ve talked about recently on the ANOVA, college remediation and regression discontinuity design. The study, by the University of Warwick’s Emma Duchini, throws even more cold water on our efforts to fix gaps in college student readiness with remediation – and leaves us wondering what to do instead.
One of the basic difficulties in improving educational outcomes lies in the chain of disadvantage. Students who start out behind tend to stay behind, and it’s not productive to ask teachers to make up for the gaps that have been opened over the course of a student’s life. As I’ve said on this blog many times, most students tend to sort themselves into fairly stable academic rankings early in life, and though individuals move between those rankings fairly often, at scale and in numbers this hierarchy is remarkably persistent. So third grade reading group serves as a good predictor of high school graduation rates, which in turn obviously predicts college completion rates. Meanwhile, the racial achievement gap appears to exist before students ever show up in formal schooling at all. It’s discouraging.
This study comes from the United Kingdom, but it concerns a question of great interest on this side of the Atlantic: do college remediation classes work? We know that college student populations are profoundly different in incoming ability. The college admissions process makes sure of that. That means that institutions like mine, the City University of New York, face profoundly higher hurdles in getting students to typical levels of ability, as our admissions data tells us that many of our students are unprepared. Typically, this results in remedial classes, to the tune of $4 billion a year for public universities. But as Duchini notes, evidence for the effectiveness of remediation is thin on the ground. Her study takes another look.
Duchini’s study draws its data from the economics department of a public Italian university. This university implemented an entrance exam for potential students, consisting of a math section, a verbal section, and a logic section. The results of this test, combined with high school grades, determines whether students are admitted to the program. However, the math section alone is used to determine whether students need to take a remedial program. Because this involves using a cut score, the cut score is fairly close to the mean, and there are no other systematic differences between students placed in or out of the remediation program, this is an ideal situation for a regression discontinuity design, as I explained in this previous post.
Ultimately Duchini considers the exam scores and educational and demographic data of 2,682 students, sorted into descriptive categories like gender, immigrant or domestic, vocational or general track, or similar. Importantly for a regression discontinuity design, there is no evidence of student groupings tightly on either side of the cut score, which can indicate that there is student manipulation of placement that would invalidate the design.
There’s an interesting dynamic in the data set, perhaps an example of Berkson’s paradox. Students who perform better on the entrance test are actually less likely to enroll in the program, even though doing well on the test is a requirement for attendance. Why? Think about what it means to do well on the test: those students are more academically prepared over all, and thus have more options for majors to take, meaning that more of them will choose to enroll in a different program.
In any event, Duchini uses a regression discontinuity design to see if there is any meaningful difference between students on students on either side of the cut score and how the trend line changes, looking at outcome variables like odds of dropping out, passing college-level math, and credits accumulated. The results are not encouraging. In particular, the real nut is here, how remediation affects the odds of passing college-level math. Note that the sample is restricted here to edge cases, as we don’t want to get a misleading picture from looking at students too far from the cutoff – this is a last in/last out style model, after all – and bear in mind that because this is a remediation test, the treatment is assigned to those on the left hand side of the cut line.
The upward-sloping trend is no surprise; we should expect student performance on an entrance exam to predict the likelihood that they’ll get through a class in the test subject. What we want to see here is a large break in the performance of the groups at the cut score, with a corresponding shift in the trend line, to suggest that the remediation program is meaningfully affecting outcomes – that is, that it’s bringing students below the cutline closer to the performance of those well above it. Neither eyeballing this scatterplot nor the statistical significance checks Duchini describes provides any such evidence. I find that fact that the data points are more tightly grouped on the left side of the cutline than on the right interesting, but I’m guessing it’s mostly noise. Look in the PDF for more scatterplots with similar trend lines as well as the model and threshold for significance.
Duchini goes into a lot of extra detail, breaking the data set down by demographic groupings and educational factors, though in every case there is little evidence of meaningful gains from the remediation program. Duchini also speaks at length about potential reasons why the program failed to meaningfully prepare students to pass college-level math, including wondering if being assigned remediation might discourage students by making them feel like the work of getting their degree will be even harder than they thought. It’s interesting stuff and worth reading, but for our purposes the conclusion is simple: this remediation program does not appear to meaningfully help students succeed in later college endeavors. It’s only one study from a particular context. But given similar studies that also find little value in remediation, this is more reason to question the value of such programs. More study is needed, but it’s not looking good.
Clearly, if remedial classes don’t work, and they cost students time and money, they should be scrapped. But scrapping them won’t solve the underlying problem: students are arriving at college without the necessary academic skills to ensure that they succeed. College educators will typically lament that they’re trying to solve the deficiencies of high school education, but of course high school teachers can fairly look back as well. Ultimately the dynamic is applicable to the whole system: students are profoundly unequal in their various academic talents from a very early age, and we’re all searching for ways to serve them better. Perhaps the conversation needs to turn to whether we should be pushing so many students into college in the first place, and whether we need to look for answers to economic woes outside of the education system entirely. But for now, we as college educators are left with a sticky problem: our students come to our schools unprepared, but our programs to fill those gaps show little sign of working.
Hey gang, I’m back from vacation and excited to get back to work here on this project. Lots of cool stuff in the works for here, including hopefully some audio and video content soonish. (Not a podcast, don’t worry. The world has enough podcasts already.)
I wanted to take a moment and explain why I’m going to be moving away from freelance writing. I’ve had a pretty good run lately; I was in the print Los Angeles Times a couple weeks back and the print Washington Post last week. (You can always check out my published writing by clicking the My Work tab above.) I know we’re all supposed to be too cool to care about print these days but, well, I do care. And I have a couple of heavily-researched pieces coming out in some longer form journals in the next several months, and it looks like I might have a regular column-type thing to indulge my political side. But beyond that, I’m not really interested in freelancing anymore. The truth is that I just find the process so aggravating and dispiriting at this point, and the money so bad, that it’s simply not worth it to me.
I just find, at this point, that the process of pitching, composing, shepherding through edits, promoting, and trying to get paid sucks the life out of me. The commercial interests of publications require editors to ask for things that are tied to the news cycle in the most facile way imaginable. I get it, and I don’t blame them personally. But I’m opting out. And it’s increasingly hard for me to explain to editors what I want a piece to do and say without writing the piece. I’m just really not interested in the “beats” of a piece of nonfiction anymore; the argument, in the sense that people traditionally mean, is just about the least interesting aspect of nonfiction writing. So when asked to reduce my own prospective writing to a series of explicit moves, I’m forced to fixate on the parts that I find least interesting or valuable. What I want is to write in a way that is free of precisely the kind of paint-by-numbers literalism that editors require. Again, not a knock on them. It’s just not in my interests anymore.
Meanwhile, the money generally sucks. I am very grateful for the LAT publishing me in their print edition, for example, and I knew what the rate was going in. But writing and editing a thousand-plus word piece for one of the biggest newspapers in the country got me $200. So many younger writers I know think that the higher profile, more established places are where the money is, but often that’s not true. Not anymore. And if I don’t enjoy it and the money’s not good, what’s the point?
I also don’t have a lot of hills to climb anymore in terms of places I want to be published. At this point even my (many) dogged critics can’t really claim that I can’t get published in major magazines or newspapers. And it’s not like they changed their tune once I did, anyway. I started writing for big pubs in part as a way to prove to my detractors that, contrary to what they said, I could get published in respectable places. When I did, they didn’t retract their old insults. They just switched to new ones. So there’s little appeal there, at this point.
And, finally, I’m just exhausted by people not reading. I’m just exhausted. The WaPo piece is an expression of 100% straightforward left-wing values; it’s a critique of corporations and an endorsement of the idea that only the left can guarantee true freedom. I do write my fair share of left-on-left critiques, but this piece really is not that. It’s simply an articulation of basic left principles in a frame designed to make them more appealing to the unconvinced. But the piece has predictably attracted criticism from the left, people insisting that I’m a reactionary even though I’m making a standard left critique of corporate power. Some have claimed that it’s a defense of the Google memo writer, when in fact I explicitly justify Google’s actions in the very first paragraph. The great bulk of the piece was written six weeks ago, before that memo existed, and that situation is tangential to my larger point. Meanwhile, others saw the headline and immediately assumed that this was a defense of the Charlottesville protesters – which would have been remarkable, given that the piece had come out on Friday, before the event. Either of these misconceptions could have been cleared up simply by reading the piece. But this is, increasingly, a bar that many refuse to clear.
This is a long-winded way of saying that I’m happy to have this outlet, where my audience is small and sympathetic and where I can avoid so many of the headaches involved in professional freelancing. Never say never, obviously, and I’ll pop up here and there. But what was always a bad bet has only gotten worse since I started doing this and I just don’t really have it in me to continue the slog. I need to focus on academic writing, book projects, and this website. Thanks for coming along.
We’re in an interesting political era, to put it mildly. I don’t just mean “Trump’s America,” or the specific partisan aspects of our contemporary situation. I mean also that we’ve been publicly grappling with broader issues of how individual people can feel empowered and engaged in the work of deliberative democracy, when so many of our digital tools have made us seem further away from those we disagree with than ever before. We’ve been grappling with this broad idea called “populism,” a curious debate given the basic definitions of democracy; we’ve argued over the proper role of experts and expertise; we’ve worried over bubbles, fake news, and the death of the commons.
We’ve also asked for decades how the liberal arts can be made relevant and important again. These seem to me to be two questions that answer the other. It is precisely the humanities that has long concerned itself with these questions, and it is the humanities that is best suited to answer them. We should look beyond our narrowly vocational interests in education, and recognize that STEM-mania and the obsession with technical skills have something to do with our unhealthy public discourse. A healthy deliberative democracy requires work. It requires people to go out of their way to foster discursive spaces where we can have a truly democratic conversation. Dismantling the humanities, despite what you’d read in the average magazine article, has consequences, and we’re living with them.
A college class, obviously, is a little thing, and doesn’t have much impact on the national conversation. But I am naive enough to believe that teaching and learning still matter, and so I’m laying out a vision for a class I thought up that is designed to address precisely the crisis of conversation we’re seeing today. The liberal arts are constantly based for their supposed impracticality, but it’s hard for me to imagine a task more practical than that of teaching young people how to be engaged, involved citizens.
Seminar in Public Writing: de Tocqueville’s America
The class I’m proposing here I envision as a 400-level seminar in English or Writing programs, entitled “Seminar in Public Writing: de Tocqueville’s America.” The class will be a seminar revolving around Alexis de Tocqueville’s seminal text Democracy in America, and using the text as a lens to consider public writing, public formation, and deliberative democracy.
Public writing is a field concerned both with writing objects designed for public consumption and with the theoretical and practical structures within public writing. It foregrounds the role writing plays in various types of political power structures, with an emphasis on its generative potential within a deliberative democracy. Public writing is ideally designed to produce effects within the world. Those effects may be as passive as mutual understanding or as active as generating concrete expression within the political process. In every case, public writing looks out from the individual or small group concerns of the creator of the writing onto a larger public to which it is addressed.
Dr. Linda Shamoon, Professor Emeritus at the University of Rhode Island, once described the process of public writing creating social change as such:
In our democratic society we ordinary citizens (as well as professional writers and those in leadership positions) who encounter a problem we consider to be public in nature may use many kinds of writing to arouse the concern of others in our community. Some in our society say we are obligated to speak out—or write—about such problems or issues. Initially, we may get little or no response to our demands for a remedy to the problem, but those of us who track an issue and seek or develop forums for our voices to be heard may find ourselves involved in many different kinds of public writing in support of our cause and working with others for solutions we had just begun to understand when we started.
Public writing assumes various stages of success. Generally, we see public writing succeeding in four stages:
Recognition— the work of public writing is read/heard; the argument is recognized as having been made.
Inclusion— the person or persons who produced the writing are recognized as valid members of the public, permitted to make public statements.
Discussion— the piece of public writing is legitimately and openly debated in good faith.
Action— the public writing produces those effects it was designed to produce.
Note that any piece of public writing need not be successful at any of these stages for it to be considered worthwhile by the person or persons writing it. Political dissidents and other out-group members often participate in public writing with no expectation that their writing will be recognized, included, discussed, or will generate the action they desire. We should still see the effort involved in public writing as beneficial and worthwhile even if it satisfies none of these stages. Democracy involves failure as well as success.
The tendency for public writing to be created but to be denied entry into the space of public discourse concerns the second stage of success, inclusion. Public writing is deeply concerned with the question of who has the right to speak— that is, who is allowed entry into a particular public sphere. Publics formation is one of the key theoretical areas of public writing, and it is here that we intersect with Alexis de Tocqueville and Democracy in America.
De Tocqueville’s text is one of the seminal works of early political science and a definitive statement on early American democracy. De Tocqueville, a traveling French nobleman, was deeply intrigued by the still-young American republic of the 1830s. Commissioned to examine the American prison system, de Tocqueville and his traveling companion sojourned across the United States and into parts of Canada, documenting many aspects of early American life that were in contrast to the practical and political norms of continental Europe. De Tocqueville’s text is a useful historical account, but it is must valued today as one of the most important evolutions in the history of political science.
A pressing question animates de Tocqueville’s text: why had republican representative democracy succeeded in America when it had seemingly failed in many other parts of the world? As an intrigued and sympathetic observer, de Tocqueville catalogs the unique elements of American democracy and civic participation. Concerned particularly with the intersection of religion, citizenship, and democratic duty, Democracy in America attempts to understand the particular American equation for successful repesentative democracy.
Of course, the success of 1830s America was success predicated on a system of brutal and oppressive inequities in power and quality of life, which de Tocqueville does not ignore. (It is relevant to point out that de Tocqueville’s view on American democracy grew much darker in his later years.) Indeed, the question of slavery haunts the book. De Tocqueville does not ignore the fact that slaves, women, and native peoples were written out of the very democratic processes he praised, and neither should we. Rather, who is included and who is excluded from democracy is of central importance to the theories of public writing. De Tocqueville’s text remains relevant to a 21st century audience in part because it is so insightful about how democracies have always excluded as well as included, with the backdrop of 19th-century America providing a host of examples of how a public is formed and how marginalized people are excluded from it.
The following syllabus describes the course, its readings, and its goals. Students will learn basic theories of publics through philosophers like Habermas, discuss what it means to write for a public, consider the impact of the internet on publics formation, and read through Democracy in America, using the text to give the course shape and structure. They will participate in creating a journal of their own writing, to be hosted online as well as bound, printed, and distributed on campus – because it’s still a thrill to see your words in real print, especially for young students.
public writing and DIA syllabus (editable Word document)public writing and DIA syllabus
Group-level differences in cognitive ability are in the news again, thanks to the quickly-notorious Google memo on the company’s diversity efforts. You can find a lot of stuff written about these differences from qualified people already and I’m not gonna add to the pile, other than to say that from where I’m sitting, if there are any sex differences in intelligence or math ability, they seem not to exist in early childhood and there are plausible cultural and social reasons that they would appear by high school. The science on personality trait differences seems less clear to me but then those constructs are also less concrete. In all of this I’m pretty much in keeping with the liberal mainstream.
But I do want to voice a caution, here, because there’s a natural but unfortunate tendency to make an unjustifiable corollary to arguments of these kind. Regular readers will know that I reject the idea of biological or genetic explanations for academic differences between races. Instead I follow most progressive people in thinking that the differences are socioeconomic and environmental in origin. There, too, I’ve often seen people make the same bad leaps: they tend to reject the idea of innate or genetic differences in individual academic ability or intelligence too. It’s not hard to understand why; talking about genetic differences in intelligence at all may seem like fruit from a poisoned tree, and why not just reject the whole idea altogether? But understanding the difference in group-level claims and individual-level claims is hugely important, both analytically and morally. It’s the difference between contributing to stereotypes that have contributed to marginalization and injustice of vulnerable groups, and accepting the reality that not all individual people are equally gifted in all areas.
And the data here is really, really clear: there are profound differences in individuals in academic or intellectual ability; these differences are generally quite durable over the course of one’s life, although of course there is some variability, as there is in any measurable psychological trait; and there is very strong evidence that a major portion of this difference comes from genetics. I don’t think that boys are smarter than girls or that black kids are less intelligent than white. I do think, and think both the empirical record and common sense shows, that not all people are equally talented in different intellectual domains, and that if you believe that the brain is the product of evolution, we should expect a significant amount of that difference to be genetic in origin, which is in fact what twin studies, adoption studies, and GWAS data show. I’ve written about all this in this space many times before.
You can think about this clearly if you just eliminate the comparison between groups that are supposedly different and look only at within-group distribution. So, for the purposes of this debate, look at women and their various metrics for intelligence and academic success, whether generally or in math/STEM/computer science. Forget about comparisons to men for a moment: within that group, on any properly validated intelligence metric, we find a normal distribution of ability. That is, there’s a mean, and there’s a distribution of about two thirds of the data points within a standard deviation from that mean, and about 95% of the data points within two standard deviations, and the distribution is just about symmetrical. Some women are better than other women on the SATs, IQ tests, quantitative reasoning tests, etc., and in predictable ways. The same exact condition applies when looking at distributions of black students, Asian students, students from Turkey, students who attend public schools, students who are left-handed, students who play Little League, etc. – real, persistent, and predictable differences of ability between individuals.
Now these individual differences don’t have much to tell us about diversity efforts like those at Google, which for the record I support, other than to say that Google is probably looking for those in the very top reaches of these distributions no matter what. But they say a hell of a lot about how we should approach education from a policy level. Policy has to reflect our empirical understandings of reality, and right now, ours doesn’t, as it is based on the false notion that all students can be brought to meet arbitrary performance standards, that there are no intrinsic limits to how well any individual student can perform, and that the purpose of schooling should train every student to be a Stanford-education Silicon Valley superstar. That’s the kind of cheery, optimistic, utterly-unachievable policy goal that comes from thinking that, because there aren’t genetic differences in intelligence between men and women or between races, there are no such differences between individual people either. That’s wrong and destructive and we can’t allow our necessary efforts to oppose bigotry to lead us in that direction.
For a lot of great thoughts on how to ethically consider genetic influences on individual intelligence, I recommend the work of the brilliant Paige Harden.
If you found value in this blog post, please consider supporting this project financially by clicking on the Patreon or Paypal link in the sidebar to the right.
Hey gang, I am officially on vacation for the first time since I started my job last September. Posting this coming week will be light, though I expect to have at least a couple pieces up. Thank you for your continued support of the ANOVA. I’m having lots of fun and hope you all are too.
Like so. (Fake charts I made with fake data, btw.)
You already get it, right?
Typically, when we perform some sort of an experiment, we want to look at how a particular number responds to the treatment – how blood pressure reacts to a new drug, say, or how students improve on a reading test when they’re given a new kind of lesson. We want to make sure that the observed differences are really the product of the treatment and not some underlying difference in observed groups. That’s what random controlled trials are for. So we randomly assign subjects to test and control groups, look at what the different averages are for the two different groups, note the size of the effect, and determine whether it is statistically significant.
But sometimes we have real-world conditions that dictate that subjects get sorted into one group or another non-randomly. If we then look at how different groups perform after some treatment, we know that we’re potentially facing severe selection effects thanks to that non-random assignment. But consider if we have assignment based purely on some quantitative metric, with a cutoff score that sorts people into one group or another. (Suppose, for example, students only became eligible for a gifted student program if they score above a cut score on some test.) Here we have a non-random distribution that we can actually exploit for research purposes. A regression discontinuity design allows us to explore the impact of such a program because, so long as students aren’t able to impact their assignment beyond their score on that test, we can be confident that students just above or just below the cutoff score are very similar.
Regression analyses will be run on all of the data, with subjects below and above the cut score combined but flagged into different groups. Researchers will run statistical models to determine whether there is a difference between groups who receive the treatment and those who don’t. As you can see in the scatterplots above, a large effect will be readily apparent in how the data looks. In the above scenario, the X axis represents the score students received on the test, the cut score is 15, and the Y axis represents performance on some later educational metric. In the top scatterplot, there is no meaningful difference from the gift students program, as the relationship between these two metrics is the same above and below the cut score. But in the bottom graph, there’s a significant jump at the cut score. Note that even after the intervention, the relationship is still linear – students who did better on the initial test do better on the later metric. But the scores of everyone have jumped right at the cut score.
There are, as you’d probably imagine, a number of potential pitfalls here, and assumption checks and quality controls are essential. All of the people tested would have to be able to be sorted into the gifted program solely on the basis of the test, the cutoff score has to be near the mean, and you need sufficient numbers to see the relationship on either side of the cut score, among other things. But if you have the right conditions, regression discontinuity design is a great way to get near-random experimental design quality in situations where you can’t do that for pragmatic or ethical reasons.
Matt Bruenig critiques the concept of the “Success Sequence” quite convincingly here. There are a lot of just-so stories in our culture about what it takes to be a success. Typically, these stories are confusing the lines of causation all over the place, failing to see that confounds and covariates are doing most of the explaining.
I sometimes get anxious emails from parents, wondering what they need to do to make sure their children are going to be OK academically. And because of networking effects and the nature of who reads this small-audience education blog, I can mostly tell them accurately that they don’t really have to do much of anything; they’ve already set up their children to succeed simply by virtue of having them. Here’s the real Academic Success Sequence:
- Be born to college-educated parents.1
- Be born to middle-class-or-above parents.
- Be born without a severe cognitive or developmental disability.
- Don’t be exposed to lead in infancy or early childhood.
- Don’t be born severely premature or at very low birth weight.
- Don’t be physically abused or neglected.
If you are one of those lucky enough to tick off these boxes, congratulations. You’ve got the vast majority of the accounted-for variance breaking in your favor. Is everything accounted for? No. We’ve got a lot of variance in cognitive and educational outcomes that never seems to be systematically explainable. I actually think that’s a good thing – perfect determinism is contrary to the fight for human meaning – but it’s important to say that this variance is not only not currently accounted for, it is likely never-to-be accounted for. This is what the behavioral geneticists call the “gloomy prospect“: the possibility that large portions of unaccounted-for variation in psychological traits like intelligence are the product of truly non-systematic events, like particular psychological traumas, getting a concussion, meeting the right person, having the right conversation at the right time….
Thus it’s the case that some people can “win” in all of the above categories and still suffer from real hardship in life, just as some can be on the wrong side in many or all of them and flourish. Still: if you’re an educated, employed parent raising a healthy child in a stable home environment, the odds are strongly in the favor of that child’s eventual academic success. Of course, none of this stuff is stuff that individuals can control, and much of it is not stuff that parents can control either – particularly given that the parents were once the children whose outcomes were similarly conditioned….
Now many people will say, well yeah, of course these things matter. But what do we do beyond that stuff? How do we set our kids up to succeed? I’m not going to say that nothing you do matters. But in terms of moving the quantitative indicators that people are, sadly, most fixated on are stubborn and hard to move. Some things appear to work – intensive one-on-one or small-group tutoring seems to me to have the most promising research literature – but we’re playing with small effect sizes here, particularly in comparison to the influence of the factors listed above. Of course you want to bend as much of the variance in a positive direction as you can. But the effects tend to be so small, and thus so subject to being offset by minor random fluctuations in uncontrolled variation, that it’s just not worth worrying about them. The best thing you can do for your kid is to be present and kind and supportive and then stop stressing out.
The great irony is that we’ve seen this growing culture of panic on the part of bourgie parents about their child rearing practices at the exact historical moment that we’ve learned conclusively that these practices just don’t mean very much.
In particular, the Baby Einstein stuff, trips to museums, violin lessons, edutainment software – my understanding is that there just is little to no rigorous research that shows that this stuff works to move the needle on SAT scores or GPA or similar, once you control for the kinds of confounds listed above. Does that mean that this stuff doesn’t matter, that you shouldn’t do them? Of course not. Children should all have the opportunity to lead intellectually enriched, challenging, and varied lives. I’m very grateful that I had that chance myself. But you need to appreciate them for their own sake and on their own terms, not as a means to goose test scores. And obsessing over getting your kid into the right preschool is pointless too, as is worrying over selective high schools. It may make you feel like the right kind of parent to fixate on this stuff; it may, more cynically, help you feel competitive with other parents. But extant evidence suggests it just doesn’t matter. What does matter is giving your child commitment, love, structure, and a moral education, because life is about so much more than where you go to college.
Of course, many people in our society are not lucky enough to have been born into the kind of advantaged position described above. Given that fact, you’d think that our system would be set up to minimize the impact of these unchosen factors. Instead we work to maximize their impact and call the resulting system “meritocracy.”