Study of the Week: To Remediate or Not to Remediate?

Today’s Study of the Week comes from researchers at my own university, the City University of New York, and concerns an issue of profound professional interest to me: the success of students who are required to attend remedial math classes in our community colleges. CUNY is a system of vast differences between and within its institutions, playing host to programs at senior colleges with well-prepared students that could succeed anywhere and also to many severely under-prepared students who struggle and drop out at unacceptable rates. In this diversity of outcomes, you have a microcosm of American higher education writ large, which like seemingly all things American is plagued by profound inequality.

Here at Brooklyn College, fully two thirds of undergraduates are transfer students, the vast majority of them having come from the CUNY community college system. (Those who get a sufficient number of credits from the community colleges must be admitted to the institution under CUNY policy, even when they would ordinarily not have met the necessary academic standards.) Typical academic outcomes data for these students, as distinct from the third who start and finish their careers at Brooklyn College, are vastly different, and in a discouraging direction. Since my job entails demonstrating to people in positions of power that students are learning adequately here, and in explaining why unfortunate numbers may look that way, this difference is important. But CUNY policy and the overall rhetoric of American college agitates against this nuance. Indeed, the recent adoption of the Pathways system of credits is based on a simple premise, that a CUNY student is a CUNY student, a CUNY class a CUNY class, and a CUNY credit a CUNY credit. This assumption of equivalence across the very large system makes life easier for students and administrators. It is also, I would argue, empirically wrong. But this is a question far above my pay grade.

In any event, the fact is that CUNY colleges host many students who lack the level of prerequisite ability we would hope for. Today’s study asks an essential question: is the best way to serve CUNY community college students who lack basic math skills to send them to non-credit bearing remedial algebra classes? Or is it to substitute a credit-bearing college course in statistics? The question has relevance far beyond CUNY.

Algebra is a Problem

When we’re talking about incoming students who fail to meet standards, we’re also talking about how they fared at the high school level. And the failure to meet entrance requirements for college corresponds with a failure to meet graduation requirements for high school. Among the biggest, most intractable problems with getting students to meet standards comes from algebra. A raft of evidence tells us that algebra requirements stand as one of the biggest impediments to students graduating from high school in our system. Here in New York City, the pass rate for the relevant sections of the Regents Exam has fluctuated with changes to standards, with 65% passing the Algebra I in 2014, 52% passing in 2015, and 62% in 2016. Even with changing standards, in other words, more than a third of all NYC students are failing to meet Algebra I requirements – and that’s despite longstanding complaints that the standards are too low. Low standards in math might help explain why 57% of undergraduates in the CUNY system writ large were found to be unable to pass their math requirements in a 2012 study.

Indeed, rising graduation rates nation-wide have come along with concerns that this improvement is the product of lower standards. You can see this dynamic play out with allegations against “online credit recovery” or in the example of a San Diego charter school where graduation rates and grades are totally contrary to test performance. Someone I know who works in education policy in the think tank world told me recently that he suspects that less than half of American high school graduates actually have the skills and knowledge required of them by math standards, as distinct from just formally passing.

The political scientist Andrew Hacker, himself of CUNY, has made the case against algebra requirements at book length in his recent The Math Myth. As Hacker says, the screening mechanism of getting through algebra, pre-calculus, and similar required courses prevents many students who are otherwise academically sufficient for higher education from attending college. He marries this argument to a critique of the funnel-every-student-into-STEM-career school of ed philosophy that has become so dominant and which I myself have argued, at length, is empirically unjustifiable, economically illiterate, and educationally impossible. Rather than trying to get every kid to be an aspiring quant, Hacker recommends replacing algebra and calculus requirements with more forgiving, practically-aligned and conceptual courses in quantitative literacy.

The question is, can we do what Hacker has asked without wholesale remaking the college system, a very large boat that’s notoriously slow to turn? That’s the question this Study of the Week is intended to answer.

The Study

Today’s study was conducted by A. W. Logue, Mari Watanabe-Rose, and Daniel Douglas, all of CUNY. They were able to take advantage of an unusual degree of administrative access to conduct a true randomized experiment, assigning students to conditions randomly in a way very rarely possible in practical educational settings. The researchers conducted their study at three (unnamed) CUNY community colleges. Their research subjects were students who would ordinarily be required to take a remedial non-credit-bearing algebra course. These students were randomly assigned to three groups: the traditional elementary algebra class (which we can think of as a control), an elementary algebra class where students were required to participate in a support workshop of a type often recommended as a remediation effort, and a undergraduate level, course-bearing introduction to statistics course with its own workshop.

In order to control for instructor effects, all instructors in the research taught one section each of the various classes, helping to minimize systematic differences between the experimental groups. Additionally, there was an important quality check in place regarding non-participants. In an ideal world, true randomization would mean that everyone selected for a treatment or group would participate, but of course you can’t force participation in experiments. That means that there might be some bias if students assigned to one treatment were more likely to decline to participate. Because of the nature of this study, the researchers were able to track the performance of non-participants, who took the standard elementary algebra class. Those students performance similarly to the in-study control group, an important source of confidence in the research.

The researchers used several different techniques to examine their relationships of interest, specifically the odds of passing the course and the amount of credits earned in the following year. One technique was an intent-to-treat (ITT) analysis, which is a kind of model used to address the fact that participants in randomized controlled trials will often drop out or otherwise not comply with the experimental program. It generates conservative effect size estimates by simply assuming that everyone who was randomized into a group stayed there for statistical purposes, even if we know we had some attrition and non-compliance along the way. (“Once randomized, always analyzed.” ) Why would we do that? Because we know that in a real-world scenario “subjects” won’t stick with their assigned “treatments” either, and we want to avoid overly optimistic effect sizes that might come with only looking at compliance.

(As always, if you want the real skinny on these model adjustments I urge you to reading people who really know this stuff. Here’s a useful, simply stated brief on intent to treat.)

The results seem like a pretty big deal to me: after analysis, including throwing in some covariates, they find that there is no significant difference in passing the course between students enrolled in the traditional elementary algebra class and that class plus a workshop, but there is a significant and fairly large (16% without covariates in the model, 14% with) difference in odds of passing the course for those randomized to the intro stats course compared to the elementary algebra course. That is, after randomization students were 16% more likely to pass a credit-bearing college-level course than a non-credit-bearing elementary algebra course. Additionally, the stats group had a significantly higher number of total credits accumulated during the experimental semester and next year, even after subtracting the credits earned for that stats course.

(Please do take a look at the confidence interval numbers listed in brackets below, which tells you a range of effects that we can say with 95% confidence contains the true average effect. Starting to look at confidence intervals is an important step in reading research reports if you’re just getting started.)

Additionally, as the authors write, “as of 1 year after the end of the experiment, 57.32% of the Stat-WS students had passed a college-level quantitative course…, while 37.80% still had remedial need. In contrast, only 15.98% of the EA students had passed a college-level quantitative course and 50.00% still had remedial need.”

Another thing that jumped out at me: in an aside, the authors note that there was no significant difference between the groups in their likelihood of taking credits a year after the experimental semester, with all groups around 66% still enrolled. Think about that – just a year out, fully a third of all students in the study were not enrolled, reflecting the tendency to stop out or drop out that is endemic to community colleges.

Of course, none of this would be inconsistent with assuming a good deal of explanatory power of incoming ability effects, and the relationship between performance on the Compass algebra placement test and the odds of passing are about what you’d expect. Prerequisite ability matters.

In short, students who were randomly selected into an elementary algebra class with a supportive workshop attached were no more likely to pass that class than those sorted into a regular algebra class, but those sorted into an introductory statistics class were 16% more likely to have passed that course. Additionally, the latter group earned significantly more college credits in the following year than the other groups, and were much more likely to have completed a quantitative skills class.

OK. So what do we think about all of this?

First, I would be very skeptical about extrapolating these results into other knowledge domains such as literacy, writing, or similar. I don’t think all remediation efforts are the same across content domains and it’s likely that research will need to be done in other fields. Second, the fact that a supporting workshop did little to improve outcomes compared to students without such a workshop is discouraging but hardly surprising. Such interventions have been attempted for a long time and at scale, but their results have been frustratingly limited.

All in all, the evidence in this study supports Hacker’s point of view, and I suppose my own: students can achieve better results in terms of pure moving towards graduation quickly if we just let them take college stats instead of forcing them to take remedial algebra first. But there’s a dimension that the researchers leave largely unexplored, which is the question of whether this all just represents the benefits of lowering standards.

Are We Just Avoiding Rigor?

The authors examine many potential explanations about why the stats-taking students outperformed the other group, including potential non-random differences in groups, motivation, and similar, but seem oddly uninterested in what strikes me as the most obvious read of the data: that it’s just easier to pass Intro to Stats than it is to pass even a remedial algebra course. They do obliquely get at this point in the discussion, writing

degree progression is not the only consideration in setting remediation policy. The participants in Group Stat-WS were only taught elementary algebra material to the extent that such material was needed to understand the statistics material. Whether students should be graduating from college having learned statistics but without having learned all of elementary algebra is one of the many decisions that a college must make regarding which particular areas of knowledge should be required for a college degree. Views can differ as to which quantitative subjects a college graduate should know.

They sure can! This seems to me to be the root of the policy issue: should we substitute stats courses for algebra courses if we think doing so will make it less likely for students to drop out or otherwise be disrupted on the path to graduation?

This is not really a criticism of this research, though I’d have liked a little more straightforward discussion of this from the authors. But I will hold with Hacker in suggesting that this does represent a lowering of standards, and that this is a feature, not a bug. That is, I think we should allow some students to avoid harder math requirements precisely because the current standards are too high. Students in deeply quantitative fields will have higher in-major math requirements anyway. Of course, in order to take advantage of this, we’d have to acknowledge that the “every student a future engineer” school of educational policy is ill-conceived and likely to result only in a lot of otherwise talented students butting their heads up against the wall of math standards. But unlike most ed policy people, I am willing to say straightforwardly that there are real and obvious differences in the specific academic talents of different individual students, and that these differences cannot be closed through normal pedagogical means. That’s what the best evidence tells us, including this very study.

Hacker says that many ostensibly quantitative professions, like computer programmer or doctor, require far less abstract math skills than is presumed. I don’t doubt he’s correct. The question is whether we as a society – and, more important, whether employers – are will to accept a world where some significant % of people in such jobs never had to pass an Algebra II or Calculus class. Or, failing that, can we redefine our sense of what is valuable work so that the many people who seem incapable of reaching standards in math can go on to have productive, financially-secure lives?

What We’re Attempting with College is Very Hard

Colleges and universities have found themselves under a great deal of pressure, internal and external, in recent years. This is to be expected; they are charging exorbitant tuition and fees from their students, after all, and despite an army of concern trolls doubting their value, the degrees they hand out in return are arguably more essential than ever for securing the good life. Though enrollment rates have slowed in recent years, over time the trend is clear:

Policymakers and politicians must understand: these new enrollments are coming overwhelmingly from the ranks of those who would once have been considered unprepared for higher education, and this has increased the difficulty of what we’re attempting dramatically.

What we’re attempting is to admit millions of more people into the higher education system than before, almost all of whom come from educational and demographic backgrounds that would once have screened them out from attendance. Because those backgrounds are so deeply intertwined with traditional inequalities and social injustice, we have rightly felt a moral need to expand opportunity to those from them. Because the 21st century economy grants such economic rewards to those who earn a bachelor’s degree, we have developed a policy regime designed to do so. I cannot help but see the moral logic behind these sentiments. And yet.

Let’s set aside my perpetual questions about the difference between relative and absolute academic performance and how they are rewarded. (Can the economic advantage of a college degree survive the erosion of the rarity of holding one? How could it possibly under any basic theory of market goods?) We’re still left with this dilemma: can we possibly maintain some coherent standards for what a college degree means while dramatically expanding the people who get them?

One way or another, everyone with an interest in college must understand that the transformation that we’re attempting as a community of schools, educators, and policymakers is unprecedented. Today, the messages we receive in higher education seem impossible: we must educated more cheaply, we must educate more quickly, we must educated far more underprepared students, and we must do so without sacrificing standards. This seems quixotic, to me. Adjusting curricula in the way proposed in this research, and accepting that higher completion rates probably require lower standards, is one way forward. Or we can refuse to adapt to the clear implication of a mountain of discouraging data for students at the lower end of the performance distribution, and get back failure in return.

programming note

Please forgive the light posting this (holiday) week. We’ll be back to a regular schedule next Monday, and there’s a book review I’m very happy with coming to $5 Patreon patrons tomorrow. I wrote something on Medium here that explains some aspects of my life that have been relevant to my written production. Hope you’re enjoying yourselves and see you soon.

Lesson Plan: Teaching Prose Style to Freshmen

This post, which is an updated version of a resource I wrote for the previous version of this website, is the first directly pedagogical resource at the ANOVA. I hope to make these a somewhat regular addition to what I post here – lesson plans, syllabuses, assignment sheets, classroom practices. I will try to vary the level and subject of the materials, and I hope to have a post like this once a month.

In one of my favorite in-class assignments from back when I used to teach freshman writing, I had my college students rewrite sections of classic literature (once class got the opening to Moby Dick, the other the opening to A Tale of Two Cities) in the voice of another author or genre of their choice. So I got, for example, Bill Simmons’s take on Moby Dick and David Foster Wallace’s spin on “It was the best of times, it was the worst of times.” Students always enjoyed this assignment, which is part of a long, multi-part sequence on prose style that I have been developing in writing classes for years now. I thought I’d talk about why I teach prose style to freshman, and how I go about doing it. This post will probably not be interesting to people outside of a niche audience.

Why teach prose style to freshmen?

These impressions are necessarily anecdotal, but from my perspective it seems that style is not typically a major concern in introductory composition classes in the American college.  One point of view that I hear often, and which I understand, is the fear that freshmen are too young to really grasp prose style — that concentrating on  how you say what you say is a layer of complexity that’s unhelpful when you’re still figuring out what to say. Some people feel like style is best addressed in an advanced composition class. I get where they’re coming from, but I think that a) this is the kind of disjunction of form and function that just makes acquiring these skills more difficult in the future and b) it contributes to the kind of mystification that I’m specifically trying to fight when I teach prose.

Probably the more practical problem for many writing instructors is the time commitment, and here I feel it’s necessary to say that pedagogy of this kind is quite out of fashion. I don’t want to get sidetracked onto one of my discussions about (what I perceive to be) the problems with college writing pedagogy. I will simply say that, in recent years, there’s been a major push to bring multimodality into the composition classroom. That is, there’s pressure on instructors, and often institutional demand in terms of program policies, to bring non-alphabetic composition into introductory  writing classes. This can take the form of units on infographics, posters, podcasts, videos, levels of video games, image editing, web design, even coding in HTML or similar. I don’t mean to dismiss the importance of these subjects generally, although when they should be taught and by who are a separate conversation. But if you’re looking for where the time went, for a lot of instructors, it’s probably there. It takes a long time and effort to teach a unit on visual rhetoric and Photoshop. Correspondingly, there’s been less time and attention given to what most people traditionally think of as the purview of college writing classes, and with that has come the perception that a focus on prose mechanics is old fashioned and not a smart career move for early career academics.

Finally, I think a lot of talented writing instructors feel unsure about prose style themselves. Style maintains a (wrongheaded) reputation as being somehow the product of natural  talent, as being unexplainable, and as being unteachable. That leaves a lot of instructors, I think, worried that they can’t teach it the same way they teach other aspects of writing. There’s so little attention to the topic in research, meanwhile, that the message often seems like this kind of work isn’t valuable or valued.

So why teach prose style? A few reasons. First, because style is an intrinsic aspect of prose and can’t be meaningfully disaggregated from content. An attitude that style is something that can be tacked on at a later date seems misguided to me. I certainly agree that developing a particular style takes time, and that having a personal and expressive style remains an advanced goal for college writers. Still, there’s no time like the present to demonstrate to students what style is and how it works. Second, style is an essential element of making students care about writing; it’s the site of inspiration. If students are going to learn to love to write (and such a love can be of immense value no matter what their academic field or eventual profession), then style has to be part of our pedagogy. How you write what you write is where writing  becomes intimate, meaningful, and fun. Too often, collegiate prose is taught in a way that ignores style, or even worse, that accepts the strangling nostrums of Strunk & White, “why is there personality in this thing, eliminate it immediately” orthodoxy. That’s a mistake.

In my own experience, broad generalizations about which kinds of college students like to write are never true. I’ve had students of all kinds who loved to write and of all kinds who thought it was a chore. But the students who have loved it have been, without exception, those who felt that their writing was a system of self-expression, a way to express themselves not only in what they were saying but in how they were saying it.

How to do it?

My fundamental learning goal in teaching style is this: to demystify prose style. Very often, students coming to a formal consideration of prose style with a sense that prose style is a pure “feel” thing, that they know it when they see it but can’t put their finger on it. That’s romantic, but it stands in the way of their adopting prose style themselves. My intent is to demonstrate to them that style emerges from the text itself, that we can observe style the way we do any other textual feature, and that we can in type write with our own style by becoming more attuned to how great stylists write.

To that end, I teach a three part assignment sequence. First, students write a brief analysis of a writer’s prose, identifying its salient features and the textual properties that makes it stand out. Second, students parody a writer’s style, typically rewriting an already written passage in a caricature of another writer’s style, or writing their own narrative in that style. Finally, students write their own text, perhaps a narrative, review, or editorial, in highly stylized prose, working to inflect their own writing with some of the features they’ve recently identified in that of others. Assignment sheets for this sequence can be found in my at the bottom of this post.

Typically, I begin the unit with a discussion of style as an abstract concept, of style in general. This often means discussing style in clothing and personal appearance first, which is a useful way to begin the discussion. Every class discussion is different, but I try to highlight a few points about style: that style is chosen (we don’t say that someone’s style is to be short or to have a particular skin color); that style is expressive, that it says something about us as individuals; that style is risky (you can be both praised for your style and mocked for it, sometimes at the same time); and that style is a matter of how one accepts or rejects conventions. All of these things are as true for your wardrobe as they are for your prose.

There’s no substitute for exploring style through looking at the work of stylists. Here, I’ve found, the best approach is to take a long walk through the style of many individual writers. This is, as you can imagine, a significant class time investment, but it really is essential. In early attempts at this unit, I figured that I could show students the individual elements of a few specific writers — sentence length, word choice, syntactic complexity, metaphors and references, spelling and mechanics “mistakes,” slang, profanity, imagery, symbolism, etc. — and then have them look for these things in writers they like personally. But I find that they need a lot more examples to really absorb these things. It’s very helpful if you can walk them through different writers, having them identify the (textual!) elements of their styles. I find that kind of repetition is necessary for them to really absorb why writers “feel” different in their expression.

Who you look at is up to you. I tend to mix both fiction and nonfiction writers, journalists and novelists, etc. I do tend to avoid translated work, just because it’s so hard to say where style ends and translation begins. Just make sure you introduce dramatically different styles, particularly at first, when students are still getting an handle on what style really means. I tend to use Hemingway, David Foster Wallace, Alice Walker, Grantland Rice, Hunter S. Thompson, Bill Simmons, Iris Murdoch, Gwendolyn Brooks, Shakespeare, Dr. Seuss, Tao Lin…. I also discuss styles that are communal or contextual rather than individual. So not so much Dashiell Hammett specifically as detective noir style, a consideration of gothic writing as an entity, and a consideration of “texting style”/emoticons/emoji. One thing these authors and styles tend to share: there’s often a lot of parodies of them available.

Why a parody segment? Because parody is a wonderful way to explore style. Parody, after all, tends to rely heavily on exaggeration, on taking stylistic elements and blowing them up. (“Taking them to 11,” as I often put it in class.) That makes them stand out more and thus make them more explicit to inexperienced writers. I stress in class that parody doesn’t have to be harsh or critical, though it often is. Sometimes parody is very loving. But what makes a successful parody work lies in how well the parodist has absorbed the consistent elements of a given writer’s style, and so a parody that’s accurate (and funny) is one which has demystified style in exactly the way I want for my students. Naturally, I go through a lot of parodies in class with the students, too. There’s tons of sharp ones out there if you look around a bit.

Students will often rewrite favorite texts from history or literature, or they’ll write stories of Harry Potter or The Hunger Games in a famous writers style, or they’ll write about their own lives and experiences in a famous writer’s style. So I get the “Declaration of Independence” written in texting style, The Hunger Games as written by Shakespeare, Grantland Rice confronts his calculus final, Dr. Seuss goes to a frat party, and so on. It’s always a joy to see what they come up with. You do have to give them opportunity to practice first, though, by giving them texts to “translate” from one style into another, preferably in the computer lab.

The final section of the project is really wide open; the students can compose pretty much whatever they want for me, provided they’re exhibiting strong style in the piece. Note that I mean strong style, as in obvious and unmistakable, rather than good. I don’t want to judge them based on something that they’re writing with a deliberate eye to exaggeration, after all, although I’m sure to say so if I think it’s particularly strong. You may find that the wide-open nature of Part Three doesn’t work for you and choose instead to put additional constraints on the project.

A few other things:

  • Students need to be informed, firmly, that although we can certainly discuss the conventions of dialects like Black English Vernacular, these are not styles that are appropriate for students to write in. Be frank. They’ll get it.
  • Go in fear of the word “tone,” which students will latch onto. It’s true of course that prose can have an academic/casual/formal/comedic/conversational/sad etc. tone, but tone is such a wooly term that without care it’ll end up being used without specificity. So ask: what makes a style mournful? Confessional? Old fashioned? Vulgar? Be specific!
  • Be prepared to manage student anxiety in this assignment. It’s very natural for students to feel a great deal of trepidation, given that many of them will likely not have done anything like this in the past. A lot of your effort has to be devoted towards just getting them to go for it — encouraging them to stop worrying so much about their grades and to just try out their ideas.

Resources

  1. Ernest Hemingway, “Indian Camp“; “Hunting a Tiger in Africa,” J.B. Miller ; this clip from Midnight in Paris.

I typically introduce Ernest Hemingway as an accessible writer with a very distinct style. While it’s true that I’m trying to develop prose style in students for the purposes of argumentative, expository, and other types of nonfiction writing, fiction writers are often very useful for discussing prose style. Hemingway is someone many of them have heard of, he has a very distinct voice, and he’s been the subject of many parodies. (He’s also a writer who is sometimes great and sometimes… not.) “Indian Camp” is good Hemingway, and indicative of his style, or perhaps of the public perception of his style. It’s moving and students grasp the style pretty quickly.

Moving to the Miller parody, then, is natural, and you can really easily draw the connection between the original and the parody, and show them how a style that seems quite meaningful in one context becomes ridiculous in another. (The Miller book, The Satanic Nurses, is chock-full of great parodies that are really funny, if you’re into reading fiction.) Finally, the movie clip really shows the difference between a person in their writing and a person as a person; Midnight in Paris, after all, is both a portrayal of the limitations of our dream versions of the famous figures we admire, and a very loving portrayal of the romance of those dream versions. Students really dig this.

  1. National Council of Teachers of English, “Literary Parodies: Exploring a Writer’s Style Through Imitation

A nice resource from the NCTE on this type of assignment.

  1. Tracer Bullet” comics from Calvin and Hobbes by Bill Watterson

A great exploration of hardboiled detective fiction, and exactly the sort of transposition I often ask students to do.

  1. Frank Sinatra Has a Cold,” by Gay Talese, and “American Marvel,” by Edith Zimmerman

It can be tricky for students to navigate the difference between genre conventions and style, given that these are often intimately linked. By putting these two great celebrity profiles in conversation with each other, students can see where one ends and the other begins. These are the same genre, and yet the effect of reading them could hardly be different, and the reason is style. This can be a really great opportunity to discuss self-referentiality, celebrity culture, and the confessional mode in writing. Note that the pieces reflect two different approaches to celebrity culture, which are a product of their respective eras: celebrity as the creation of intentional distance and celebrity through the impression of artificial familiarity, respectively. Students in particular love Zimmerman’s piece, and because it shrinks the distance between them and a celebrity (they are forever telling me that Zimmerman seems like a friend of theirs, or that they could be her), it also shrinks the distance between their writing and writing that is professionally published.

4.“Wow, Fiction Works!,” by Colson Whitehead

There are a lot of parodies that I use in this unit, and I leave it to you to find the ones that work best for you. This Colson Whitehead parody of James Wood, though, is probably my favorite. I will often read this with students in class without first having them read any Wood, and have them try and identify what Wood’s writing is like just from a parody. This is an exercise in back-to-front thinking that I find useful after we’ve introduced and discussed a concept for a good while.

Style Assignment Sequence