we don’t, in fact, know what works in teaching composition

Doug Hesse is the latest compositionist to take to the Chronicle to defend the honor of the field against someone arguing that students can’t write. Hesse is a really genuinely kind and considerate scholar – I talked to him at a conference once where he was sitting around gabbing enthusiastically with grad students before I realized who he was – but I find his essay unpersuasive. Some of it is just the conventions of the genre; yes, “college students today can’t write” is a hoary old cliche, but that doesn’t mean the statement isn’t true. But more, I just don’t agree with the titular claim that we know what works in composition. And I don’t agree because I don’t think we know what it means to “know,” in composition.

Hesse’s article says that his perspective on what works is “informed by decades of research.” There’s no citations provided for a laundry list of claims he makes, which is fine – it’s an essay, not a journal article. But what the average reader might not understand is that the research Hesse refers to is, in dominant majorities, completely lacking in the kind of statistical and methodological controls that have been steadily developed in the social sciences over the past century. Adequate sample size, stratification of those samples, use of control groups, reporting of statistical significance levels and effect size, adjusting for the distortions of alpha common to nested data through multilevel modeling, utilization of techniques like quadratic regression to counter data that doesn’t conform to the assumptions of typical multivariable models like linear regression and ANOVA – these things are almost unheard of in contemporary composition scholarship. Indeed, much of the research published in the most exclusive composition journals lacks even the most basic statistical controls at all.

There are, of course, a host of problems with contemporary social science and its methods. I am not at all naive about those problems. And despite a frequent accusation, I’ve never disdained strong qualitative work. A good ethnography or case study can fill in the gaps left by large-scale quantitative approaches, in a way that truly deepens our knowledge. The problem is that we don’t have that quantitative work, in college writing, and that which exists tends to come from research fields other than our own. On a fundamental level, I believe that claims about what works and what doesn’t in writing education writ large have to emerge from carefully planned and executed empirical work that reflects a vast body of research on how to make generalizable observations about student performance. The claims Hesse endorses mostly don’t.

Instead, if they are like the vast majority of composition scholarship, they come from very small-n studies utilizing convenience samples from instructors studying their own classes, involve that instructor-researcher making subjective observations based on vague and underdeveloped criteria, and utilize no statistical controls whatsoever to account for the vast number of ways educational research can go wrong. Such studies can provide insight when they are paired with rigorous large-n quantitative work, but on their own, they are not a way to “know” anything about our students and how best to teach them. This is my position.

Why have compositionists disdained that quantitative work? As has been documented in articles like Richard Haswell’s “NCTE/CCCC’s Recent War on Scholarship” and Susan Peck MacDonald’s “The Erasure of Language,” composition scholarship was once welcoming to quantitative and empirical work, but saw that side of the research tradition shrivel. That shriveling was in large measure the consequence of the “cultural turn” in composition, in which cultural studies essentially ate the field. Scholars like James Berlin, Elizabeth Flynn, Carl Herndl developed a research literature that insisted that quantitative scholarship was a tool of hegemony and bigotry. To put it in Flynn’s terms, “feminist critiques of the sciences and the social sciences have also made evident the dangers inherent in identifications with fields that have traditionally been male-dominated and valorize epistemologies that endanger those in marginalized positions.”

That vocabulary is neither idle nor accidental: the attitude is not merely that quantitative (“scientistic”) research is limited or contingent, which is of course true, but that it actively endangers students and scholars from marginalized backgrounds. By 2005, Richard Fulkerson could write of composition, “in point of fact, virtually no one in contemporary composition theory assumes any epistemology other than a vaguely interactionist constructivism. We have rejected quantification and any attempts to reach Truth about our business by scientific means.”

Composition’s public spaces, at conferences and on listservs, are generally deeply antagonistic to any arguments for rigorous, large-n quantitative studies. Even carefully worded appeals to a both/and approach that values qualitative or theoretical work while it pursues more quantification often meet with defensiveness and politicized rejection. The field does see periodic calls for more empirical work and more quantification, but somehow those rallying calls never seem to inspire more space in our prominent journals, which of course is profoundly relevant to what is professionally valuable to individual academics.

To take a pertinent and depressing example, the premiere journal College Composition and Communication published a special issue in 2012 documenting all the exciting new methodological possibilities for the field, including various forms of quantitative research. The journal then proceeded to published almost no numbers-based scholarship whatsoever for the next 4 years. When I was coming up in my MA program I was told that I should look at Research in the Teaching of English as the kind of journal I might one day write for, but in the past half decade or so RTE has published almost no large-scale quantitative work at all. The annual 4Cs conference has become something of a joke for those of us who favor quantitative approaches, in that it has adopted an extremely loose approach to what counts as writing research… unless that research involves statistics. Young academics eager for recognition as researchers – and the professional incentives that come with it – cannot help but notice these dynamics.

It’s not hard to understand the social dynamics of this equation. When one side says “we value the narrative, theoretical, and qualitative work being done, but we need to balance it with rigorous quantitative work to understand our students and their learning,” and the other side says “quantification is part of a lineage of racism, sexism, and colonialism,” it’s not hard to guess what the prevailing sentiment will be in today’s humanities programs.

This dynamic – a divide between the methodological arguments of quantitative scholars and the political arguments of cultural studies and critical theory scholars – plays out in far more realms than just composition. I identify as an applied linguist because the majority of my graduate training, my independent reading and tutoring, my research, and my professional life have occurred in that field. The divide there is stark and growing too: the fields of ESL/TESOL/applied linguistics is marked by a growing inability for differing groups to communicate with each other. In my experience, those fields feature small and shrinking groups of quantitative scholars undertaking empirical investigations into linguistic and assessment data and a larger group of scholars who call the former tools of colonialism. I have been in academic spaces, in the real world and online, where the two sides differ so greatly in vocabulary, methods, and culture that they are essentially unable to communicate, but are forced alongside each other by the small and shrinking professional and academic landscape.

Ultimately, the definition of knowledge and how it is produced are the product of incentives. The professional incentives in composition today simply cut too hard against quantification.

Suppose a young compositionist were to ask me for career advice and said that they wanted to do an empirical dissertation. I would tell them to identify some small and impossibly specific sub-population of students and conduct a case study on two or three of them, to grab a convenience sample of whatever students were ready at hand, to assign arbitrary and vague codes to what they’ve found, and to draw vast conclusions from their sample of two or three students, despite the fact that the kind of researchers who love case studies typically disdain generalization. I would tell them that because experience tells me that’s the kind of empirical work that gets people jobs. I would caution them against conducting a large-n, quantitative study, because experience suggests that this is a bad way to go about getting a job. Write a dissertation like that in composition and the most likely outcome is a lot of defensiveness and dismissal in your job interviews, the weird, “you think you’re better than me” attitude that academics who don’t do quantitative work often harbor for those that do.

I’ve said this before: from a purely careerist perspective, it would have been far more lucrative for me as someone working in composition to write a dissertation on the rhetoric of Doctor Who than an examination of the history and theory of standardized tests of collegiate learning. I don’t say that to insult scholars in pop culture studies or to suggest that there is no value to that kind of inquiry. But in general there appears to be an inversion of the expected, in the field, when it comes to what is a common approach and what is a niche concern. I value and respect work in critical pedagogy, cultural studies, minority rhetorics, pop culture studies, “the digital,” and critical theory. But that work has so crowded out efforts to rigorously examine how students learn to write that I’m surprised Hesse can be so confident in what he knows.

I don’t disagree with most of Hesse’s prescriptions for composition. In particular, when he writes, “Students learn to write by writing, by getting advice and feedback on their writing, and then writing some more,” I want to applaud. Writing is like playing a sport or learning a musical instrument: there is no substitute for repetition. You must practice! Students need to be writing, a lot. I would personally prefer that they be working at much smaller scales than is typical in contemporary composition classrooms, taking apart their own paragraphs, finding what doesn’t work, and rewriting them until they’re polished and strong. But yes, there is simply no substitute for practice, for repetition, in training young writers. What I question is Hesse’s apparent confidence that students are actually getting that much opportunity to practice.

My own take is that in fact in many introductory composition courses students produce very little writing over the course of a semester. Indeed, my limited, anecdotal impression is that the average American composition class involves a few weeks of unfocused intro-to-argumentation lessons, a couple of short papers, and then weeks spent on podcasting, Photoshop, pop culture analysis, and whatever else the instructor finds fun to teach. Indeed I’ve been at conferences where composition grad students bragged to each other about how little actual writing their writing students do. In part this is a labor issue: with so many composition instructors coming from the ranks of adjuncts and grad students, there is a widespread fear of angering students by giving them too much work. In part it is an incentives issue: it’s far better to go on the job market as a compositionist who teaches podcasting and video game design than it is to go on the job market as a compositionist who teaches writing as it is traditionally understood. Universities may fund writing programs in the understanding that writing papers is important, both for other college learning and professionally, but it is not universities who hire professors. It’s other professors, and they gauge status and esteem through what’s novel, current, and “innovative,” not what’s pedagogically necessary.

Of course, there is some fact of the matter about what is actually happening in our composition classes. We don’t have to rely on my impression vs. Hesse’s. We could use quantitative methods to better understand what actually gets taught in composition programs. But that requires all of the stuff that composition has disdained for decades, those techniques and standards that people like Flynn insist are just an excuse for patriarchy and white supremacy. To get that work done, established scholars like Hesse must actually fight for its presence in the field – fight for it as peer reviewers at journals and conferences, fight for it in the hiring and tenure process, fight for it even when it is politically and socially uncomfortable to do so. My experience in composition tells me they are deeply resistant to doing so. After all, once you’ve climbed to the top of a reward structure yourself, why would you try to chance what precisely it rewards?

Hesse dings Joseph Teller for spreading “lore” and refers to “overwhelming empirical evidence” against him. But in fact, as someone with a broad grasp on what composition has called knowledge for decades, I find the evidence very far from overwhelming. I would in fact call it lore myself. Pull aside different compositionists and ask them what we definitively know as a field that we didn’t know ten years ago; you will more likely get a lecture on the hegemony of knowledge than consistent answers based on a shared reading of rigorous and replicated research. There are ways that human beings can answer questions about how students learn to write effectively – never perfectly, never without doubt, but constructively, with greater and greater confidence over time. The question for Hesse and the field he defends is whether or not to use them.

15 Comments

  1. I’m ignorant about the literature on composition pedagogy, but it seems to me that the situation is more clear-cut than you’re admitting here. You admit that Hesse is right when he writes, “Students learn to write by writing, by getting advice and feedback on their writing, and then writing some more.” It would certainly be nice to have good empirical research, but there shouldn’t be anyone who disagrees with Hesse’s prescription: one could replace “writing” in that sentence by any other human activity and it would be just as true.

    So the only real question is: is this happening in composition classes? I understand that my formulation here is imprecise, and I agree with you that it would be best to have empirical research figuring out exactly what the best way to implement this is. But at the moment those sorts of details are often quibbles; it isn’t difficult to look at the syllabus for a composition class and ask: are students doing a lot of writing and getting lots of feedback? In a class filled with beside-the-point activities like podcasting and Photoshop, the answer is obviously no. In a class of the sort Hesse teaches, I have little doubt that the answer is yes.

    Quantitative analysis is important, and it could be used to fine-tune already good classes in writing. But this sort of fine-tuning won’t be useful if instructors just don’t feel like teaching writing in their writing courses. Shouldn’t we focus on fixing that first?

    1. There are in fact many people in college teaching who regard giving students lots of writing to do as foisting grunt work on them.

      “it isn’t difficult to look at the syllabus for a composition class and ask: are students doing a lot of writing and getting lots of feedback?”

      At A class, yes. At scale it’s a much more difficult proposition.

      “Shouldn’t we focus on fixing that first?”

      Sure. But please understand, this is a very controversial stance of mine within this world – people react with visceral anger when I complain about how much time students spend playing video games in comp classes.

  2. I agree with almost everything you say, Feddie. There isn’t enough writing repetition and practice in higher ed. But I don’t think that’s rhet/comp’s fault. The field works within the (typically) two-semester parameters it’s given. Despite the prevailing and deeply entrenched belief that people write well after a semester or two of writing class, that isn’t enough time for students to become “good writers” without regular guidance and feedback from faculty (or other qualified people), which most non-comp faculty simply will not do. (And maybe the shouldn’t have to. I don’t know; that’s another discussion.)

    For the record, I’ve taught comp at four institutions and reviewed literally hundreds of comp syllabi for research , and the 2-3 short paper lots of critical theory scenario you describe was not the norm—far from it. Most FYC programs require 5 – 6 papers, ranging from 3-4 to 6-8 pages and culminates in a (revised) portfolio of that work.

    All that said, rhet-comp has more problems than I care to discuss—I “got out” last fall–but it shouldn’t be burdened with bulk of the responsibility when it comes to students’ shortcomings. I’d say institutional/structural issues are more–or at least equally–responsible.

    As usal, enjo reading your stuff.

    Dan Smith

    ps – apologies in advance for typos, etc. written in haste.

  3. Interesting post. I am reminded of a truly excellent AP English teacher I had in high school. (I had two excellent AP English teachers, actually, but this was my first one.) She always sent back my writing assignments full of comments in red ink. I think she tried her best to give an equal amount of feedback to every student, regardless of their skill level. She threw a lot of advanced shit at us — _The Education of Henry Adams_ for example — and treated us in every way like young adults with brains capable of expansion.

    I’m not sure what part of her teaching was best, but I am thankful she taught me. I didn’t take any English classes in college, so the AP credit was meaningless to me, but as I plug away at a fiction project right now I am so very glad of this early experience.

  4. Very nice response to Hesse’s article. I’ve run against this attitude in our field as well on two different occasions. Once on the WPA-listserv when I suggested that validity as understood by educational measurement was a good and useful term for writing assessment, and when I have tried, and failed, to get a large-n for a replicated study that is a decade old on writing placement.

    In the first instance I got some of the cultural studies resentment tossed at me in a vein similar to how Lynne criticizes that discipline in her book. It wasn’t Lynne, but another “writing assessment” scholar who believes that ed. measurement is hegemonic and reduces the experience of literacy to numbers. Or, something like that, since the argument ended with, “well, if you read my diss, then you’ll understand (and presumably agree with me).” And all I was arguing was that the concept of validity is useful and thus can be used meaningfully theoretically.

    In the second instance, the survey data that Gladstein et. al. collected seems to be all the empirical data our field needs. As if that’s the best one-stop shop for answers to questions that quantitative data can answer. And, more shocking still, the n for placement is less than 1/4 the n that the survey I sought to replicate.

    I dunno man. How this will shake-out in the time of tRump, and how our field, strange and diverse as it is, will respond is anyone’s guess. I suspect that like all good-intentioned fools, it’ll be our collective folly to fight this fight with more theory. Just watch for the comp equivalent of “It’s time for some game theory”.

  5. What would a well-designed, large N study of teaching composition look like? I ask this as a non-expert in your field.

    I’m a quantitative social scientist who uses statistics, but I am skeptical that large-scale surveys are a good way to identify successful teaching practices. We have a decent idea about how use surveys to study age, race, gender, income, household composition, and so on, but can survey instruments really capture pedagogical techniques or the nuances of classroom experiences? Please change my mind.

  6. “involve that instructor-researcher making subjective observations based on vague and underdeveloped criteria,”

    Your writing reflects a surety that moving away from this model to a different one is possible. What’s the different model of judgment? How would it work?

    Anyhow, my issues with the “cultural hegemony” people: Minorities are disadvantaged by life, so they become disadvantaged in their work, but if you tend to give higher grades to majorities than minorities differently, that’s racism (!!!), and any system which does this is inherently wrong, so instead we have to stop redressing the imbalance and move towards a system that denies minorities skills they could actually use (which majorities tend to have already) so that no one has to feel bad about themselves.

    Writing’s really a perfect example of this – I’m upper-middle-class, white, male. I’m great at writing, recently having taken an English class and gotten 99/100 without too much effort, wrote each of my essays within 1 day or 2 at most. I had the excellent resource of my mother to check over my work (I do the same for her, by the way). If that 99/100 I got was illusory, and a result of a system desperate to avoid cultural hegemony, who cares? But in these classes you have to occasionally peer-review an essay, and given my locale I got Latino students both times, and…I mean, I liked the essays, but they weren’t good, as such. If they got 99/100 based on that quality of work (obviously these were rough drafts, though), then they’re going to stay at that level, and that’s probably a bad thing. (Though I personally think that essay writing in general is a useless skills, but clearly a lot of people disagree, including you, so there it is. Actually, if you end up reading this comment and responding, you could tell me why you disagree – greatly appreciated.)

    1. “I personally think that essay writing in general is a useless skill…”

      Allow me to respond since Freddie is probably busy, would you? I think you can castigate literally any skill as ‘useless’ and come up with good reasons for it.

      I am willing to say that essay writing teaches discipline, improves one’s mastery of a language (any language that you choose to write in!), and is morally worth pursuing over many other things, such as attempting to rig Bitcoin markets and selling illegal drugs to broke college students.

      1. Sorry, didn’t notice this. My ability to write nonfiction – to write essays – has helped my life in every sense: personally, emotionally, socially, professionally, een romantically. Essays are good.

        1. “I think you can castigate literally any skill as ‘useless’ and come up with good reasons for it.”

          How about “in no real-world situation do you ever have to write an essay, ever, at all”. That can’t be a function of motivated reasoning, which you could turn around and apply to any skill. Though it can be applied to many skills taught in school, but there you go.

          Even pieces like this are not, strictly speaking, essays, and diverge from the essay format in a number of significant ways. Normal articles diverge from that format even more significantly, as do business reports, anything else you can think of. So teaching the skill of essay writing seems to come down to teaching auxiliary skills such as

          “discipline, mastery of a language”

          and I can’t believe that the most efficient way to teach these things is through what’s basically a dead art form. Not to mention

          “morally worth pursuing over many other things, such as attempting to rig Bitcoin markets and selling illegal drugs to broke college students.”

          i get what you were trying to do here, but you failed. Moving on:

          “My ability to write nonfiction – to write essays – has helped my life in every sense: personally, emotionally, socially, professionally, een romantically. Essays are good.”

          But can you quantify, specifically, how this has happened and why, and why it couldn’t have been done more efficiently in a different form? We’re literally talking about the core of one of our biggest school subjects, and very rarely does anyone offer a solid, fleshed-out defense of why this core is relevant. It just…is.

          1. I’m thinking about juggling right now. “In no real-world situation do you ever have to juggle five objects at once, ever, at all.” That’s probably true for the vast majority of people. Yet some skilled jugglers can do it and they may even get paid to entertain others while they do it! Strange how that is.

  7. A good piece, and I agree with you. One quibble, though: if we could get people to erase “today” or “nowadays” every time they write sentences like “college students today can’t write”, that would be very helpful. College students have *never* been able to write, even when they were an elite, mostly affluent minority who attended prep schools instead of public school before college.

  8. Thanks Freddie. I am looking forward to the book. As a sort of counter-example I thought of while reading your piece (which to me seem the be the argument you’ve been making for sometime, and one I agree with at times) are a couple initiatives going on in our department (unnamed). One: A large grant that was given to a database project that will fund the project and grad student stipends for years to come, and Two: an initiative in the writing center to actively promote its work, and sell it to the university, in part, but showing the “value added” of the labs’ services via mounds of statistics. Both of these are being done by relatively new hires.

    Now, this administrative and fund raising work I vaguely describe above is certainty different than what you’re discussing, i.e. research in the field and teaching, but the connection here is that it’s an example of R/C scholars being actively aware of the current conditions of the university and doing what they can to adapt. These initiatives are not research in and of themselves, but they both use, and enable more empirical research to be done. My point here is that I imagine things are changing out of necessity, though slowly, and further, I wonder how much the march towards privatization and the corporate university will render debates about disciplinary agendas as quaint relics of the late-modern period.

    But back to the subject at hand, I also think that many people are doing empirical research in composition, and that there will be more in the future. Looking at the kinds of empirically focused faculty that have been hired here in the last three years tells me as much. In the mean time though, if you are really concerned for the health of the field, I’d also so say (in addition to your writing, is keeping at presenting and publishing empirical work in the field of R/C. It’s totally cliche, but it really seems true to me: be the change you want to see. Take up space. Occupy. Then negotiate. Don’t just tell them/us what to do. Find a way to co-exist. The die has already been cast for the old guard, and many of them weren’t trained in those methods. There’s nothing to be done about that. As we move forward however, let’s stay on it, and create a more balanced and visible field. Amen. Sorry for preaching. Thanks for your work,

  9. You cannot learn to play the violin without practicing.
    You cannot learn to play the violin well without listening to violin music A LOT.

    Likewise, I guarantee you there are no pro basketball players who did not grow up practicing and watching basketball.

    Or racing-car driving. Or any other sphere of human activity that society values.

    Of course that does not apply to all countries in all eras. In England basketball is not played well, or practiced much, or shown on TV. In India they play other instruments but the violin. In certain cities in North America, literally nobody is well dressed. And it happens that we live in a world in which almost nobody reads, respects reading, or — surprise, surprise! — writes clear, precise prose.

    But it doesn’t mean that, if the goal is to produce violin-players, you can succeed without having people listen to violin music or practice; or, if the goal is really to produce people who can write, that that can be achieved without getting them to read and practice writing.

    Writing is not an inborn human right, it’s a skill.

Leave a Comment

Your email address will not be published. Required fields are marked *