oh no

Abraham Riesman for New York Magazine

It turns out that the progress of the counterculture is like pushing a canoe away from a pier; you generate movement until you run out of leg, and then you’re too far from the foundation to generate force, and you’re just drifting and directionless. The truth is that, even in the heady make believe of ye olden days when a man was a man and boys all had crew cuts, the school jocks and the prom queens were always outnumbered. Either way: today it’s no contest. No one today would admit to being the prom queen in a million years. Everyone is the outsider, now. San Diego Comic Con, jam packed with throngs of thousands, each convinced that they are most alone boys and girls to ever walk the blacktop. And with no inside to be outside of, all that’s left is jokers and their pose, this absurd posture of rebelling against that authority which is no authority at all.

Jokes are a horror story now, the banal, stuttering emptiness of a culture of class clowns and no teachers. Desperately, desperately invested in crafting a persona of disaffection and unconcern, endlessly A/B testing their zingers, trying to convince everyone else that somehow an entire new species has been born of people who have no internal life save for feelings of condescension and being self-impressed. They sell it all by misnaming it irony, an indispensable tool for life that’s been leeched of all of its vitality, all in service of the fear of those who cannot bear to betray emotion to the people whose approval they seek with white-knuckled intensity.

When I lived in Chicago I would go to parties with my buddy who was in the comedy world there. He did standup and improv and sketch. He’d take me to these parties and I’d meet others in that scene, people all trying to make it in comedy. And a lot of them were lovely people. But I’d come home from those parties and just feel exhausted. It was a consequence of being on the receiving end of everybody’s material for hours.  And there’s the future, for you, the exhaustion of people proving that they’re not trying by trying very hard, a dystopia of amateur comedians, and a nation of people trying to sit at the back of the class who can’t seem to find where the front is. Why are you doing it? Who is it for? And can you take yourself seriously long enough to ask if you even really want to do this anymore?

How Charter Schools Cook the Books

For a variety of reasons, charter schools have long been the darlings of American news media’s discussions about education reform. For one thing, our media is disproportionately neoliberal and inclined to believe that markets make everything better. For another, our pundit class draws disproportionately from the elite classes, who tend to have attended expensive private schools and who have no particular sympathy – and often outright disdain – for public education. For another, the funding apparatus of our think tanks is heavily bent against public schools and towards charters, as the do-gooding rich types who fund such institutions are often market-focused and antagonistic to unionized public sector employees like public school teachers.

Whatever the reason, the general state of affairs in education reporting is near-total credulity towards charter schools and their advocates, with few in professional media digging in to charter school rhetoric to find the flaws. To understand these failures, it’s important to look at how charter schools achieve good numbers at the cost of fairness and transparency.

Manufacturing Selection Bias

Generating responsible arguments about education is difficult for a variety of reasons. Perhaps the biggest lies in selection bias. Selection bias refers to when inequalities in how samples are gathered – such as “public school students” vs “charter school students” – leads to incorrect assumptions about results. I have argued in the past that selection bias is in fact the single most important phenomenon in educational statistics.

A classic example in selection bias lies in the common assumption that private schools are superior to public. Many parents send their children to private schools without any rigorous investigation about whether those schools are superior to local public schools at all. After all, they might say, look at the star students the private schools graduate. But there is an obvious and immensely important factor missing when we attempt to naively compare outcomes across school types: the incoming student bodies are not remotely the same. Private schools almost universally have more affluent student bodies than traditional public schools, meaning many of the most disadvantaged students are systematically excluded. And many privates also employ entrance exams or grade requirements before enrollment, ensuring that their student bodies will be predisposed to succeed.

Charter advocates tend to speak as if charter schools have demographically and economically similar student bodies to public, and act as though we have true random placement into their schools. Some claim that lottery systems are sufficient to wash out differences in incoming student bodies. Random assignment is extremely important in educational statistics, as it is necessary to ensure that our comparisons are fair.

But we know that charter school student bodies are very often not equal. And we know that many charter schools go to immense lengths to make sure they aren’t. A 2013 Reuters investigation found myriad ways that charters go out of their way to exclude the most difficult to educate:

Students may be asked to submit a 15-page typed research paper, an original short story, or a handwritten essay on the historical figure they would most like to meet. There are interviews. Exams. And pages of questions for parents to answer, including: How do you intend to help this school if we admit your son or daughter?

These aren’t college applications. They’re applications for seats at charter schools.

Charters are public schools, funded by taxpayers and widely promoted as open to all. But Reuters has found that across the United States, charters aggressively screen student applicants, assessing their academic records, parental support, disciplinary history, motivation, special needs and even their citizenship, sometimes in violation of state and federal law.

Note that even requiring parents to opt their children in to lotteries is sufficient to contaminate randomization enough to make drawing responsible inferences impossible. And clearly these schools go far beyond that.

These behaviors are not only important in and of themselves, as indicators of how unscrupulous actors can bend the rules to make charters look better compared to public. They also demonstrate that even among charter officials themselves, there is a strong understanding of just how strong a role student selection plays in outcomes. Otherwise, why would they go to the trouble? For all of their talk about how charter practices are sufficient to help any child succeed, their own behavior demonstrates differently.

In fact, we have a raft of research showing that, when we employ genuinely random distribution, perceived differences in school quality makes no impact on student achievement.

You’d expect charter advocates to be particularly stringent about charters that engage in these practices; if they really believe that charters are better on the merits, they’d want to ensure fair comparisons. But in my experience, reformers are in fact incredibly credulous about even the rosiest numbers that arise from the charter world, almost never engaging in appropriate, productive skepticism.

Refusing to Backfill

An important type of selection bias is survivorship bias. With survivorship bias, we only observe a given characteristic in those examples that make it past some sort of selection procedure. If you ever hear a speech by any successful famous person, they are likely to deliver some sort of bromide about how they kept a positive attitude and never gave up. Which may be true – but there are also plenty of people who kept a positive attitude and never gave up and didn’t succeed, but crucially they never get the opportunity to make speeches about it so we don’t adjust our understanding accordingly. This is survivorship bias.

A common type of charter school chicanery involves the refusal to backfill and in so doing create a type of survivorship bias. “Backfill” refers to schools enrolling more students to fill spaces created through students dropping out, failing out, or being removed for disciplinary problems. Backfill – backfill through random selection, of course – is essential for making fair comparisons. After all, the students most likely to leave are often the ones living the most difficult, most transient lives, and thus those most likely to struggle academically. Refusing to backfill amounts to creaming the best students off the top after the fact.

Who’s guilty of refusing to backfill? Why, Success Academy Charters, the darlings of the charter school set! Aside from the brutal working conditions and army of short-term “tourist teachers” looking for a foothold in New York City, I suspect that this accounts for a large portion of the supposed advantage of Success Academy. If charter advocates are serious about actually wanting real student gains, why have they not led the charge against this kind of practice?

You can also just routinely suspend the most vulnerable students until they drop out or are forced out, which many charter schools already do.

When In Doubt, Cook the Books

Survivorship bias strikes again. The 2013 CREDO study was widely ballyhooed at the time as a vindication of charter schools, showing significant learning gains relative to public. And with the credibility and prestige of Stanford’s CREDO project behind it, the report made serious waves. Unfortunately, few people seemed to dig into the fine print. As a (pseudonymous) writer pointed out at EduShyster, the CREDO report admitted that 8 percent of the charter schools in the initial sample had closed. And which schools are most likely to close? The worst performers! Of course your numbers are going to look good when the worst 8% of the sample simple vanishes into thin air, a vanishing act generally impossible for public schools. Again: why would serious charter school advocates tolerate this kind of thing, if they are genuinely interested in helping children learn?

There are many, many other examples of charter advocates playing fast and loose with numbers in order to attack public schools. For example, when discussing the supposed New Orleans miracle in post-Katrina schooling, charter advocates are prone to trumpet the rise in the number of schools receiving a passing grade from the state since public schools were closed and replaced by charters. They typically neglect to mention that the cut score for passing was lowered in between the rating of the public schools and the rating of the charter schools.

Just Giving Everybody A’s

I like this one the best, because it is the most brazen. At San Diego Metropolitan Career and Technical School, every student is above average. The grades are sterling. The graduation rate is top notch.

The test scores, sadly, are quite bad. Because they seem to be giving out great grades to everybody regardless of performance. Hey, that’s one way to achieve – just lower standards. Reform types love to argue that market forces compel schools to promote student learning, but this is incorrect on its face. Market forces compel charter schools to please parents, which is not at all the same thing. And you can bet if it’s happening at one school, it’s happening at another. There are thousands of charter schools in the country, and yet their advocates constantly talk as though any given school performs identically to the attention-grabbing, high-resource, big-city idiosyncratic schools they love to tout.

If I am hard on the charter school crowd, it’s in part because they’ve spent the last several decades attacking teachers, hundreds of thousands of public servants who make middling wages performing an impossible job. But it’s also because issues like these are simply not discussed by advocates, who tend to adopt a defensive position and refuse to countenance any questioning of charter schools at all. I am currently working on a book about these topics; my day job is in academic assessment; I wrote a dissertation about standardized tests; and I’ve taught students from kindergarten to graduate school in a variety of contexts. I have never found serious attempts to grapple with the profound challenges to charter school numbers that I have laid out here. If charter advocates actually care about improving education, rather than simply winning, you’d think they’d leap at the challenge.

my book is in fact an anti-race science book

Since this has come up – I am not writing a pro-race science book. I am writing a book that, among other things, is anti-race science. It unequivocally rejects the idea that different races have inherent differences in intelligence. Whatever you might think of me or my project, I have denied racist pseudoscience my entire political life, and that has not and will not change. Find a different angle of attack.

I am once again in trouble for things I haven’t said and don’t believe. This attack on me on Twitter is occurring without anyone quoting a single thing that I’ve ever actually said.

Related. I endorse everything in this piece and in this piece, save for the endorsement of charter schools, which links to David Leonhardt. Leonhardt is one of the most consistently biased voices on charters out there. The supposed charter advantage is actually the product of selection effects and survivorship bias. Such opinions, unfortunately, are not permitted in high-profile places like Vox.

the involuntary admission barrier to care

I am very far away from the news cycle, these days, but even I have not missed the horror of another terrible school shooting. As it should, the topic of America’s mental health system appears to have again come up. I want to very briefly note a serious practical barrier to appropriate care, which is the involuntary admission system.

When I reached the end of my ability to cope with my illness last August, I had a dilemma. I went to the hospital because of a long string of erratic and self-destructive behaviors. But the final event that drove me to seek emergency care was that I had accused a friend of hacking into my bank account and threatened to harm them in revenge. That they didn’t have me arrested was an act of mercy. When I got to the hospital, I knew that if I revealed that I had threatened physical harm to someone, I would be at risk of a 9.13(b), New York’s involuntary admissions policy. Most other states, I believe, have similar laws. I could not risk the disruption to my life, and the total loss of control, a 72 hour stay would entail. And since I was not willing to divulge that detail, which would have made my crisis clear, the psychiatrist who treated me would not allow me a voluntary admission and I was left to pursue outpatient care. This is the lacuna into which you may find yourself when you have a psychiatric crisis: how to receive appropriately urgent care without losing control of your life. This problem was particularly acute in years past because I was hiding my condition from family and friends and was terrified of them finding out.

This dynamic, I’m sure, would not have impacted the Florida shooting. And I recognize the need for some form of involuntary admissions. But I am convinced that many people avoid seeking care entirely out of fear of involuntary admission, and something has to change.

Miles in his dotage

Old man.

Dusty clawed, waddle-walker, tail kinked like the corner of a page in a book. Ears rimmed with white, eyes a little red, belly a little big. You can still trot, when you really want to, but mostly you amble along like a kid on his way to school, moving as if the purpose of moving was not to move. You can’t get onto the bed on your own anymore, but you can pull me right over in pursuit of a chicken bone. You look like you, in my head, and then I find old pictures of you in your prime and I’m taken aback. Memory is funny when you’ve been with somebody constantly for over a decade. I can’t remember what life was like before you and I have no intention of knowing what it’s like after you. You will not let me cut your nails. You grunt and groan like the old man you are. You are tired.

In time, after your stroke, when you had learned to walk again and the skin on your head wasn’t pulled so tight and your belly wasn’t so distended, when you went off the steroids and the hair the vet had shaved finally came back, when I was no longer carrying you around like a briefcase or holding napkins to your nose as it bled and bled and bled – afterwards, many people told me how good you looked, how you were good as new, how you seemed just the same. And it was true that you got better, in a way I never thought you would.

But I also knew and know that something has been broken in you ever since, that there is some deep quality inside of you that was changed, as if life itself insisted that your illness be written permanently on the record of your life. As an animal your body is your legacy, and the illness – and your brave recovery – are part of that legacy. And when you struggle to get up the stairs, or when your body shakes faintly in its quiet sad way, or when you lie in my arms as I scratch your belly and I can feel how you shudder inside, I feel again that feeling I felt when I picked up your contorting body and put you in the car.

A little brokenness isn’t the worst thing in someone of your age. You carry it well. You have gotten so much better; I can’t believe you were once on 9 medications. I guess the tables have turned. You have been there through more than a decade of mine – a decade of growth, a decade of experience, but mostly a decade of regret. Every mistake in a life of mistakes, a witness to my failures without judgement. When we lived in Rhode Island and I couldn’t afford to run the heat at night I’d put on two hoodies and you’d curl up in my arms on a tiny twin bed. Now you never want to sleep next to me, and it makes me a little sad. But I get it. It’s a little statement of your independence. You are, after all, your own, before you are mine.

Old man. You still hunt rabbits in your sleep. Squirrel chaser, bone chewer, baying hound of Lefferts Gardens, I know that you will never die.

significantly more than you wanted to know

They would not admit me, first of all, at RUMC. The more-or-less explicit reason was that I was unwilling to say that I was a physical danger to myself. This was not so much a matter of pride on my part as a matter of honesty; I was in rough, rough shape, but I was not in danger of self-harm. So I left with my brother, who had come up from DC to take care of me, sedatives in hand, and proceeded to look for outpatient care. I remember walking out onto a Staten Island street and just thinking, what the fuck do I do now?

It was a wearying ordeal to get into treatment. Maybe I’ll write about it someday. But I am in treatment, now, and have been for four months. I am blessed with a great prescribing psychiatrist, thanks to help from my friend Katie Halper. Finding a working therapy solution has been much harder, but I have done some short-term CBT and I’ve Skyped with a psychoanalyst.

I am on a comprehensive meds regimen. I am on the largest dosage of lithium I’ve ever been on. I’m on olanzapine, an antipsychotic. Accepting that I need to be on an antipsychotic medication has been a key step, for me, and one of many ways in which this time has been different than the past. I am on fluoxetine (Prozac). I’m on Wellbutrin. I was on Lamictal but I had an allergic reaction and we had to drop it. I have prescriptions for Xanax for anxiety and Ambien for sleep. (Don’t worry, I consider avoiding benzo addiction to be one of my hobbies these days.)

One thing you learn: there’s no such thing as just the right level of concern. Like, you tell people you’re bipolar, and they think you’re kind of a moody fellow. You tell them you’re on antipsychotics and they think you belong in Arkham Asylum.

The side effects are what they are. I have tolerated olanzapine in a way I have never tolerated Seroquel or risperdal and for that I’m grateful. But, god, the hunger. The olanzapine causes hunger pangs, omnipresent and gnawing, all day, every day. If I’m not careful I could just write about the hunger for hundreds of words. Hunger in the morning and at night. Hunger right after I eat. Hunger that dominates my mental life. I went to a meeting at my union hall and they had pizzas laid out on a table. I took three slices and went and sat down and while I was eating one – while pizza was literally in my mouth – I began to stress about when I could go back up to get more, when it would be socially acceptable to get seconds. I Googled around about olanzapine hunger and I felt so validated to read other people, talking about 50 pound weight gain, about being forced off the drug by the hunger. One person wrote that it’s like having a hole in your stomach and that sounds about right. I have all sorts of little tricks to try and deal with it; I drink coffee and seltzer by the gallon. (No joke, I average 5 liters of seltzer a day.) Still I’m up 20 pounds since August. It feels like a war of attrition I’m bound to lose.

Other than that, the side effects are alright. I go to bed absurdly early unless I really fight to stay up. Some acne. My hands shake sometimes. I have to pee a dozen times a day. At first I had some verbal dysfunction but it faded, thank god. Mostly at first there was just the permanent sense of being altered, the familiar sense that I was a few degrees off from everybody else, a haziness. I no longer feel that way, usually. The question I guess is if it went away or if I simply got used to it, if I have a new normal. All I can really perceive is just enough of an increase in mental effort to do everyday things, just enough to notice. Writing. Writing is so much harder than it was.

There have been some other annoyances. The logistics, the travel, the time off work, the expense. I struggled with agoraphobia for months after, scared to be seen, scared of people. They tell me it’s pretty common with people who go inpatient and so I feel like it wouldn’t be unusual for someone in my situation. Anyhow, it’s much easier now, although I’m still frightened of being seen in a way I can’t quite express. I mostly keep my world small.

I’m sober. Occasionally I go to AA meetings but I am not really doing the steps; it’s partly the god and partly the submission and partly I’ve read the research. But it is profoundly comforting to know that the meetings are out there, so many meetings. You have to give credit to the 12 Step people on this: I can look on a website and find a dozen meetings in a dozen neighborhoods at many different times of the day. The whole thing is set up to make attending meetings as easy as possible. The contrast with finding a therapist is really stark. Anyway: sobriety’s fine. It’s been easy. I mean it sucks, but it hasn’t been difficult. I’ve been to bars with friends and not had any temptation. I just miss beer. I made a pot roast while it was snowing the other night and it felt criminal to not drink a nice stout. But I’m committed to sobriety and it has clear and direct rewards.

Everything with me is rituals and rules now.

Getting off of social media was easy too, although I’m sure it would have been harder had it not come with such dramatic upheaval. There’s this unusual feeling that no one is selling anything to you, off of social media. In this era we’re all constantly trying to sell visions of ourselves to those around us, electronically, and cutting that out of my life has left me with relationships that seem at once more natural and more deliberate. Do I miss it sometimes? Sure. I hate to give Facebook any credit at all, but without it things do seem a bit more lonely. There are people I miss. I have this permanent sense of not knowing what’s happening, partly by design; I am not really reading the news lately and tend to avoid takes like the plague. But socially, it’s tough. I have no idea what’s going on in people’s lives. And people are in touch and then they stop being in touch, and I never know why. Probably because of something I’ve done.

I thank god for my union. My strong health insurance, that’s thanks to the NYC public sector unions and how they’ve fought. My prescription drug benefit comes directly from my union. My generous sick leave days are thanks to my union. Had I exhausted them, I would have been able to draw from a pool of shared sicked days that others have donated, thanks to my union. My boss has been great about everything, very supportive, but if that had not been the case, I would have been able to rely on the muscle of my union, on the grievance system. I thank god for my union.

I’ve thrown myself into activism work with my housing rights group. It’s a wonderful radical, deeply pragmatic organization. Housing is such a primal need, and the people who work for housing justice in this city are some of the best I’ve met. My group is a great opportunity to be a rank and file activist, to just be a foot soldier. I do phone banking; I hand out leaflets; I get signatures. I do all the mundane brick and mortar political work, and it’s a blessing. I’ve also been working with my union whenever we have a meeting or an event. Our contract’s up; it’s going to be a hard fight.

I am lonely. I see people and they treat me differently now, and my heart rebels. But my head knows that it’s natural. I am different, after all. Still, when I interact with someone I’ve known and they’re just completely the same, when they treat me the same, it’s such a wonderful feeling.

I get emails, lots of lovely emails from sweet people who reach out to me. I sometimes have a hard time writing back. I often don’t know what to say. I suppose thanks will have to do. I do get other kinds of emails, occasionally. It’s still the internet. On the internet, all truths are crowd sourced, even the most personal and intimate. And so I get people who patiently explain to me what my condition means and doesn’t. I get accused of faking it. I’m told what my disorder can and cannot explain, I am lectured to about what I am accountable for and am not. I am told that my illness renders my political opinions illegitimate. It’s amazing, not only how many experts in bipolar disorder there are out there, but how many experts there are in my bipolar disorder. The question of my culpability for my actions when I am cycling is one that I will continue to turn over in my head for the rest of my life. And I have the advantage of being me. I will keep my own counsel on the nature of my illness. I don’t need any volunteer parole officers for my mental health.

Is the question whether I’m sorry? All I am is sorry. I am so sorry. I have such bone-deep regret. I live in guilt. And I am paying for my mistakes. I have once again set friendships on fire, driven off good people who trusted and cared for me. Not with one big thing but with a long, drawn out series of minor betrayals, my constant accusation and demands, all drawn from a frantic paranoia, my deep-seated fear that people who care for me are withdrawing their affection and, when it gets bad, my belief that they are actively plotting to harm me. I mistreated people who were unfortunate enough to be close to me. You chip away at people’s trust, a chisel here and a chisel there, and then they make the rational decision to cut you out of their lives. And then you have to live without them. Yes, I am sorry. I am so sorry. I’ve changed my life in every way I know how. I should know better than to say that this time is different. But this time it has to be different. I can’t go off meds again. I can’t do this anymore.

And so I’m more sure than ever that I need to stay away. Long experience tells me that, sometimes, that’s the only way to make amends, to make yourself scarce. There are projects I want to work on, longform stuff, if I am permitted. But there will be no comeback tour. I have lost my having-opinions-on-the-internet privileges. I enjoy being off social media; I’m deeply uninformed about the news cycle and couldn’t comment intelligently; I have lost the pathological compulsion to always be writing and sharing. But most importantly, I have to remain offline as a penance to the people who I have mistreated with my words. I have to give up something to show people I understand that my behavior was not acceptable, to sacrifice something in the name of accountability. Are all of the things people got mad at me about legitimate? No. There has been irrational hatred of me along with rational dislike. But I have lost my right to argue about that distinction. I have to be accountable, and the best way to do so is to permanently remove myself from online life. It’s not much but it’s what I have to offer at this time. Besides I was online for 9 years. Better to let other people talk, to let other kinds of people talk.

Sometimes I dream of another life, or really another me, a me who wasn’t at war with myself, and the things I would have written then. But I’m too old for that now. I can only move forward. If there is any virtue to getting to a place where you say, to another human being, “I cannot go on like this, and I need help,” it’s that the part of you that cares for the opinions of other people dies, and however briefly, you live unmediated. I would like to think I can access it again, in times of better fortune.

I’ll be alright. I have become acquainted with the quiet. Time for books, time for snow, time for the library, time for campus. Time, like the man wrote, to burrow deep for a deep winter.

standardized tests are tools of inequality, not equality

Consider this fairly recent NYT piece, titled “A Simple Way to Help Low-Income Students: Make Everyone Take SAT or ACT.” The essay talks about research that shows that making these tests mandatory in high schools raises the participation rate (duh) and in so doing identifies high-achieving students who ordinarily would not have taken the test. See, typically those who are screened out of taking a college entrance exam through self-selection are those who are less college ready and perform less well. But this is far from universal, and there are many potential high-scoring testers who are screened out through fees, lack of parental guidance, or a lack of information about when and how to sign up. The research discussed showed that low-income but high performing students are less likely to take these tests than high-income, high-performing students, and that making the tests mandatory will thus lower the relative disadvantage of those students. Mandating the test is thus a tool for equality – it increases the opportunity for students who are typically systematically excluded from college.

Let’s think about things for a second. First, and to be clear, the research does not show that low-income students are more likely to perform well. The opposite is the case:

(Data’s a bit old, but this is a durable outcome.) So having more low-income students taking the SAT will likely mean finding that many low-income students are in fact not prepared for college, to go along with finding those high-performing kids who we wouldn’t otherwise. Still, obviously I think it’s a noble and necessary goal to help identify talented students from poor families. The point is that it’s odd to think of this as a project for increasing equality as such. We’re simply looking for more “diamonds in the rough,” and hopefully helping to pull them out from their peers – who are thus left even further behind.

Here’s a point to stress: the very purpose of educational testing is to identify inequality. That is, we develop and administer tests precisely to better understand how students are not the same. In fact, the most precisely that tests are, the more unequal we understand the tested population to be. A 10 question test likely has less discriminatory power than a 100 question test, and thus the 100 question test is more likely to differentiate between closely-grouped students – that is, to identify how they are unequal. Progress in educational testing stems from designing instruments that are more sensitive to underlying inequality. That’s the very name of the game.

As I’ve said before, we talk about education as fulfilling two functions that are not just in tension with each other but directly contradictory: education is discussed as a tool for creating greater socioeconomic equality, and as a system for identifying excellence and rewarding it with status and economic opportunity. The problems here should be obvious.There’s a much larger conversation about summative equality and equality of opportunity here, which is too directly political to get into in this space. (I will say that I think equality of opportunity is not really a coherent idea when you pull at it a bit.)

But from the standpoint of educational policy, it’s not clear to me that we really know what we want to be doing. Some people tell me that our goal should just be to move everyone up in terms of absolute achievement, raising averages without necessarily changing relative performance. That might have lots of good effects, but it’s by definition not something that could help with inequality, as what’s rewarded by the labor market is relative educational achievement, not absolute. If everyone who ever went to an Ivy League school was sent to the moon, they’d simply look for the next rung down and hire accordingly. If the purpose is instead to shrink the variance, to narrow the range between the top and bottom of the achievement scale, we’d want to talk about limiting resources to the top-performing kids, and we’d still be looking for differences in what individuals can do. And we have no good reason to think that we can achieve either at scale, because while some interventions have helped different groups at different times, the general bell-shaped distribution of overall achievement on any identifiable quantitative metric of academic success has been persistent and unchanging over time.

As long as we use education as a system for sorting students into different tranches of ability, and as long as that sorting system is a key mechanism for placing people into different levels of income and joblessness, we can’t conceive of our system as being an engine of socioeconomic equality. We might sometimes use testing to identify areas where more resources are needed and distribute them accordingly. But 15 years since No Child Left Behind and the testing-heavy era it augured, we have seen almost nothing in the way of convincing proof that testing is a reliable tool for raising standards and increasing either equality or opportunity. Tests are powerful things, and modern test-development can produce exams of extraordinary precision. But they can’t be useful until we have a clear and coherent vision of what we’re testing for. To get that understanding, we have to begin to pull apart our basic assumptions about education and our economy, to ask ourselves if the system can do what we imagine it can do.

recalibrating your sites

Not too long ago, I felt the need to change the stream of personalities and attitudes that were pouring into my head, and it’s been remarkable.

This was really the product of idiosyncratic personal conditions, but it’s ended up being a good intellectual exercise too. I had to rearrange a few things in my digital social life. And concurrently I had realized that my sense of the world was being distorted by the flow of information that was being deposited into my brain via the internet. I hadn’t really lost a sense of what the “other side” thinks politically; I’m still one of those geezers who forces himself to read Reason and the Wall Street Journal op/ed page and, god help me, National Review. But I had definitely lost a sense of the mental lives of people who did not occupy my various weird interests.

What were other people thinking about, at least as far as could be gleaned by what they shared online? What appeared to be a big deal to them and what didn’t? I had lost my sense of social proportion. I couldn’t tell if the things my friends were obsessing about were things that the rest of the world was obsessing about. Talking to IRL friends that don’t post much or at all online helped give me a sense that I was missing something. But I didn’t know what.

No, I had to use the tools available to me to dramatically change the opinions and ideas and attitudes that were coming flowing into my mental life. And it had become clear that, though I have an RSS feed and I peruse certain websites and publications regularly, though I still read lots of books and physical journals and magazines, the opinions I was receiving were coming overwhelmingly through social media. People shared things and commented on what they shared on Facebook and Twitter, they made clear what ideas were permissible and what weren’t on Facebook and Twitter, they defined the shared mental world on Facebook and Twitter. They created a language that, if you weren’t paying attention, looked like the lingua franca. I’m sure there are people out there who can take all of this in with the proper perspective and not allow it to subtly shape your perception of social attitudes writ large. But I can’t.

It’s all particularly disturbing because a lot of what you see and don’t online is the product of algorithms that are blunt instruments at best.

So I set about disconnecting, temporarily, from certain people, groups, publications, and conversations. I found voices that popped up in my feeds a lot and muted them. I unfollowed groups and pages. I looked out for certain markers of status and social belonging and used them as guides for what to avoid. I was less interested in avoiding certain subjects than I was in avoiding certain perspectives, the social frames that we all use to understand the world. The news cycle was what it was; I could not avoid Trump, as wonderful as that sounds. But I could avoid a certain way of looking at Trump, and at the broader world. In particular I wanted to look past what we once called ideology: I wanted to see the ways in which my internet-mediated intellectual life was dominated by assumptions that did not recognize themselves as assumptions, to understand how the perspective that did not understand itself to be a perspective had distorted my vision of the world. I wanted to better see the water in which my school of fish swims.

Now this can be touchy – mutually connecting with people on social media has become a loaded thing in IRL relationships, for better or worse. Luckily both Facebook and Twitter give you ways to not see someone’s posts without them knowing and without severing the connection. Just make a list of people, pages, and publications that you want to take a diet from, and after a month or two of seeing how different things look, go back to following them. (Alternatively: don’t.) Really do it! The tools are there, and you can always revert back. Just keep a record of what you’re doing.

I was prepared for this to result in a markedly different online experience for me, and for it to somewhat change my perception of what “everyone” thinks, of what people are reading, watching, and listening to, etc. But even so, I’ve been floored by how dramatically different the online world looks with a little manipulation of the feeds. A few subjects dropped out entirely; the Twin Peaks reboot went from being everywhere to being nowhere, for example. But what really changed was the affect through which the world was presenting itself to me.

You would not be surprised by what my lenses appear to have been (and still largely to be): very college educated, very left-leaing, very New York, very media-savvy, very middlebrow, and for lack of a better word, very “cool.” That is, the perspective that I had tried to wean myself off of was made up of people whose online self-presentation is ostentatiously ironic, in-joke heavy, filled with cultural references that are designed to hit just the right level of obscurity, and generally oriented towards impressing people through being performatively not impressed by anything. It was made up of people who are passionately invested in not appearing to be passionately invested in anything. It’s a sensibility that you can trace back to Gawker and Spy magazine and much, much further back than that, if you care to.

Perhaps most dramatic was the changes to what – and who – was perceived as a Big Deal. By cutting out a hundred voices or fewer, things and people that everybody talks about became things and people that nobody talks about. The internet is a technology for creating small ponds for us to all be big fish in. But you change your perspective just slightly, move over just an inch, and suddenly you get a sense of just how few people know about you or could possibly care. It’s oddly comforting, to be reminded that even if you enjoy a little internet notoriety, the average person on the street could not care less who you are or what you do. I recommend it.

Of course, there are profound limits to this. My feeds are still dominantly coming from a few overlapping social cultures. Trimming who I’m following hasn’t meant that I’m suddenly connected to more high school dropouts, orthodox Jews, senior citizens, or people who don’t speak English. I would never pretend that this little exercise has given me a truly broad perspective. The point has just been to see how dramatically a few changes to my digital life could alter my perception of “the conversation.” And it’s done that. More than ever, I worry that our sense of shared political assumptions and the perceived immorality of the status quo is the result of systems that exclude a large mass of people, whose opinions will surely matter in the political wars ahead.

I am now adding some of what I cut back in to my digital life. The point was never really to avoid particular publications or people. I like some of what and who I had cut out very much. The point is to remain alive to how arbitrary and idiosyncratic changes in the constant flow of information can alter our perception of the human race. It’s something I intend to do once a year or so, to jolt myself back into understanding how limiting my perspective really is.

Everyone knows, these days, that we’re living in digitally-enabled bubbles. The trouble is that our instincts are naturally to believe that everyone else is in a bubble, or at least that their bubbles are smaller and with thicker walls. But people like me – college educated, living in an urban enclave, at least socially liberal, tuned in to arts and culture news and criticism, possessed of the vocabulary of media and the academy, “savvy” – you face unique temptations in this regard. No, I don’t think that this kind of bubble is the same as someone who only gets their news from InfoWars and Breitbart. But the fact that so many people like me write the professional internet, the fact that the creators of the idioms and attitudes of our newsmedia and cultural industry almost universally come from a very thin slice of the American populace, is genuinely dangerous.

To regain perspective takes effort, and I encourage you all to expend that effort, particularly if you are an academic or journalist. Your world is small, and our world is big.

Study of the Week: More Bad News for College Remediation

Today’s Study of the Week combines two subjects we’ve talked about recently on the ANOVA, college remediation and regression discontinuity design. The study, by the University of Warwick’s Emma Duchini, throws even more cold water on our efforts to fix gaps in college student readiness with remediation – and leaves us wondering what to do instead.

One of the basic difficulties in improving educational outcomes lies in the chain of disadvantage. Students who start out behind tend to stay behind, and it’s not productive to ask teachers to make up for the gaps that have been opened over the course of a student’s life. As I’ve said on this blog many times, most students tend to sort themselves into fairly stable academic rankings early in life, and though individuals move between those rankings fairly often, at scale and in numbers this hierarchy is remarkably persistent. So third grade reading group serves as a good predictor of high school graduation rates, which in turn obviously predicts college completion rates. Meanwhile, the racial achievement gap appears to exist before students ever show up in formal schooling at all. It’s discouraging.

This study comes from the United Kingdom, but it concerns a question of great interest on this side of the Atlantic: do college remediation classes work? We know that college student populations are profoundly different in incoming ability. The college admissions process makes sure of that. That means that institutions like mine, the City University of New York, face profoundly higher hurdles in getting students to typical levels of ability, as our admissions data tells us that many of our students are unprepared. Typically, this results in remedial classes, to the tune of $4 billion a year for public universities. But as Duchini notes, evidence for the effectiveness of remediation is thin on the ground. Her study takes another look.

Duchini’s study draws its data from the economics department of a public Italian university. This university implemented an entrance exam for potential students, consisting of a math section, a verbal section, and a logic section. The results of this test, combined with high school grades, determines whether students are admitted to the program. However, the math section alone is used to determine whether students need to take a remedial program. Because this involves using a cut score, the cut score is fairly close to the mean, and there are no other systematic differences between students placed in or out of the remediation program, this is an ideal situation for a regression discontinuity design, as I explained in this previous post.

I can teach you regression discontinuity design in two images

Ultimately Duchini considers the exam scores and educational and demographic data of 2,682 students, sorted into descriptive categories like gender, immigrant or domestic, vocational or general track, or similar. Importantly for a regression discontinuity design, there is no evidence of student groupings tightly on either side of the cut score, which can indicate that there is student manipulation of placement that would invalidate the design.

There’s an interesting dynamic in the data set, perhaps an example of Berkson’s paradox. Students who perform better on the entrance test are actually less likely to enroll in the program, even though doing well on the test is a requirement for attendance. Why? Think about what it means to do well on the test: those students are more academically prepared over all, and thus have more options for majors to take, meaning that more of them will choose to enroll in a different program.

In any event, Duchini uses a regression discontinuity design to see if there is any meaningful difference between students on students on either side of the cut score and how the trend line changes, looking at outcome variables like odds of dropping out, passing college-level math, and credits accumulated. The results are not encouraging. In particular, the real nut is here, how remediation affects the odds of passing college-level math. Note that the sample is restricted here to edge cases, as we don’t want to get a misleading picture from looking at students too far from the cutoff – this is a last in/last out style model, after all – and bear in mind that because this is a remediation test, the treatment is assigned to those on the left hand side of the cut line.

The upward-sloping trend is no surprise; we should expect student performance on an entrance exam to predict the likelihood that they’ll get through a class in the test subject. What we want to see here is a large break in the performance of the groups at the cut score, with a corresponding shift in the trend line, to suggest that the remediation program is meaningfully affecting outcomes – that is, that it’s bringing students below the cutline closer to the performance of those well above it. Neither eyeballing this scatterplot nor the statistical significance checks Duchini describes provides any such evidence. I find that fact that the data points are more tightly grouped on the left side of the cutline than on the right interesting, but I’m guessing it’s mostly noise. Look in the PDF for more scatterplots with similar trend lines as well as the model and threshold for significance.

Duchini goes into a lot of extra detail, breaking the data set down by demographic groupings and educational factors, though in every case there is little evidence of meaningful gains from the remediation program. Duchini also speaks at length about potential reasons why the program failed to meaningfully prepare students to pass college-level math, including wondering if being assigned remediation might discourage students by making them feel like the work of getting their degree will be even harder than they thought. It’s interesting stuff and worth reading, but for our purposes the conclusion is simple: this remediation program does not appear to meaningfully help students succeed in later college endeavors. It’s only one study from a particular context. But given similar studies that also find little value in remediation, this is more reason to question the value of such programs. More study is needed, but it’s not looking good.

Clearly, if remedial classes don’t work, and they cost students time and money, they should be scrapped. But scrapping them won’t solve the underlying problem: students are arriving at college without the necessary academic skills to ensure that they succeed. College educators will typically lament that they’re trying to solve the deficiencies of high school education, but of course high school teachers can fairly look back as well. Ultimately the dynamic is applicable to the whole system: students are profoundly unequal in their various academic talents from a very early age, and we’re all searching for ways to serve them better. Perhaps the conversation needs to turn to whether we should be pushing so many students into college in the first place, and whether we need to look for answers to economic woes outside of the education system entirely. But for now, we as college educators are left with a sticky problem: our students come to our schools unprepared, but our programs to fill those gaps show little sign of working.