a scientific definition of causation

Whenever I get into these correlation and causation battles (and I do frequently, both in the university and online), they seem to go wrong in two ways: one, people often insist on an entirely unhelpful definition of the word “implies,” and two, people often presume some quantitative signifier of absolute causation that does not exist in most fields.

For the first case, there’s a widespread and strange contention that “implies” is synonymous with “proves.” I find this out of character with conventional use: the word implies seems to exist specifically to indicate a softer claim that the word proves. “Officer, did the suspect specifically state that he wanted to buy drugs?” “No, your honor, but he implied it.” My boss implied I would get fired if I didn’t do my work. The woman at the bar implied she’d like me to buy her a drink. Etc etc. To say that an implication amounts to proof positive just seems contrary to the way we use that term, to me. If you’d prefer “suggests” or “provides evidence for,” then I’ll use that instead. But in each case, we are make a probabilistic judgement call, not a simple quantitative conclusion, because of point two: in the large majority of fields, causation is a philosophical, epistemological concept, not a mathematical one. In those fields where there are communal standards of causation, from my limited knowledge, they tend to be beyond the reach of a vast majority of the social sciences.

Take a field where the stakes for research are high indeed: medicine. Robert Koch, a pioneering epidemiologist and physician, proposed four criteria for demonstrating that a particular pathogen caused a disease.

1. the microorganism or other pathogen must be present in all cases of the disease
2. the pathogen can be isolated from the diseased host and grown in pure culture
3. the pathogen from the pure culture must cause the disease when inoculated into a healthy, susceptible laboratory animal
4. the pathogen must be reisolated from the new host and shown to be the same as the originally inoculated pathogen

You can already imagine the host of problems here, even for a field as “hard” as the medical sciences. We know that there are agents in medicine which contribute to diseases or conditions but which are not necessarily present in all cases. (10% of lung  cancer victims, for example, have never smoked.) Growing pathogens in cultures is very straightforward if that pathogen is a bacterium, but far less clear when we’re talking about many kinds of environmental and behavioral risk factors. Replicating certain behaviors or conditions in lab animals is often a practical impossibility, and in many instances, human epidemiology varies drastically from that of mice or similar lab animals. Reisolating a causal agent again makes sense if we’re hunting for a bacteria and no sense if we’re asking if exposure to soot causes scrotal cancer. And so on. So eventually, Koch’s requirements had to be discarded in some avenues of medicine such as carcinogenesis; the bar was simply too high, and the need for more cancer science far too great.

In some fields, the only responsible way to assign a cause is through a controlled experiment. That often means that the researchers must themselves control the given exposure or other independent variables, introducing them into the test group themselves. The ethical quandary is obvious: no institutional review board in the world will allow you to deliberately expose a test group of babies to tobacco smoke in hopes of determining strong empirical proof of causation. (I hope!) Similarly, no one could or should divide babies into a cohort to be raised in affluence and a cohort to be raised in poverty for the sake of experimental value, even though we might learn more in doing so that we have in decades of educational research. Instead, we are left to muddle through with observational and quasi-experimental designs in which researchers cannot themselves control exposure to a given independent variable, which some serious epistemologists would say is a requirement of truly demonstrating causation. And we do alright, sometimes, if we’re careful, limited in our claims, and we replicate. (With lung cancer, at least, we can see the physiological changes that occur from smoking. How could we ever look for a physiological sign of causation in the brain of an impoverished child struggling to succeed in school? Where would we look?)

So I will again go out on this limb: I believe that poverty causes poor educational outcomes. I think by a rational, fair standard of what we mean in common human language by a cause, that statement is true. The evidence? Decade upon decade of studies that demonstrate a strong correlation between the socioeconomic class of students (or their parents) and educational outcomes. Across a broad variety of contexts, for a number of different age groups, in all manner of different levels and types of schools, we see that basic dynamic. I’m not suggesting simplicity here. While I believe poverty is cause of educational failure, it is surely not the only cause. And while this basic dynamic is present in reams of data, the effect size is not always the same, the effect is not always equally distributed across the income spectrum, the effect sometimes changes according to age cohort, and on and on. Yet I feel confident enough in the relationship I’m describing — and in my readers to understand nuance and appropriate limitation — to say that poverty causes poor educational performance.

If you think that my use of the word “cause” here is problematic, or simply wrong, I’m very happy to have that discussion. The literature on this topic is vast, and I’m nothing resembling an expert. But for practical purposes we have to allow for sufficient linguistic and epistemological simplicity to actually grow the “storehouse of human knowledge.” We might get experts together from a variety of fields to debate and develop fair, pragmatically-useful definitions of cause that are reasonable for those fields. But I find that debate so much less fruitful than many other forms of inquiry we can undertake if we all agree to understand causation as contingent and complex.

I again recommend The Emperor of All Maladies for a very cogent discussion of how all of this played out in the realm of cancer where, despite being on the forefront of modern science, with incredible resources, the best-trained researchers could not prove causality to Koch’s standard, which the tobacco companies used to nefarious ends. Siddartha Mukherjee writes on page 256, “Rather than fussing about the metaphysical idea about causality (what, in the purest sense, constitutes ’cause’?), [Bradford] Hill changed it to a functional or operational idea. Cause is what cause does, Hill claimed.” If that’s good enough for investigating cancer, it’s good enough for me.

Update: Forgot to mention that this essay by Greg Laden is just terrific on these topics.

22 Comments

  1. Koch’s Postulates are used to prove causation for *infectious* diseases. Cancer is not an infectious disease (while oncoviruses cause cancer, they’re nevertheless infectious microorganisms so Koch’s Postulates still apply).

    Not sure how Koch’s Postulates are relevant to demonstrating the mechanism of action by which a carcinogen causes cancer.

    1. They are relevant because of history; they were posed as a necessary set of conditions that oncologists and cancer researchers had to satisfy in order to prove causation, and later abandoned for the reasons I’ve laid out here. Thought I made that clear.

  2. I guess what I am saying is that using a method for proving the identity of a disease-causing infectious agent to prove the identity of a carcinogen is wrong on its face, not merely too burdensome a standard for the field of oncology. Whether that standard was erroneously employed in history or not, I’m not sure how that is a good example for what you seem to be claiming – which, if I follow, is that because we can’t randomly place (genetically identical?) babies in purposefully lavish or severely deprived environments just to see who gets better grades we should presume that poverty is the cause of poor educational outcomes. Not a bad presumption, I just don’t follow.

    Sorry if I’m wasting your time here.

    1. First, your positing something that you haven’t proven, which is that Koch’s standard for causation was only utilized with what we narrowly define as infectious agents. More importantly, whether you find this standard useful for oncology, ex post facto, is far less important than the fact that the field of oncology itself grappled with the application of this standard or similar standards in the past. That historical example has relevance to me because I’m interested in the tangled, complex ways in which people attempt to determine causality in field-specific ways. This is an example of an approach to causality that proved far too rigorous even in a hard science.

  3. “First, your positing something that you haven’t proven, which is that Koch’s standard for causation was only utilized with what we narrowly define as infectious agents.”

    But that’s all it’s useful for in the first place; it’s all it was ever useful for in the first place. I don’t get your objection to my point at all. Sorry, Freddie, I like your blog and your writing generally but this post of yours is all over the place.

    1. Listen, man: I have tremendously little patience for when people play the internet commenter game of spending 30 seconds considering an issue of great complexity that others have thought about and debate for years, arrive at some facile complaint, and treat it like profundity. You’ve spoken with great certainty about a post that you’ve misread and keep misinterpreting, arguing logical connections I never made and claims I never advanced. Try LISTENING to someone else. Try slowing down. Try not assuming that you are the smartest human alive. Just try those things, for a little while.

    2. It would indeed be very weird if I were arguing that oncologists should be utilizing Koch’s standards of causation. But I’m not arguing that and didn’t imply that I was. I said that the field of cancer research once tried to apply those standards and found them to stringent to be of use in their field. That’s a historical claim and I’ve provided a citation for it. If you think Siddartha Mukherjee errs in arguing that this was once a preoccupation of cancer research, you could provide evidence to support that notion. It would also be weird if I were saying, as you suggest, that the fact that Koch’s standards were too stringent for cancer research meant that we shouldn’t keep kids in poverty for research purposes. That would be VERY weird! But not as weird as you glossing my post that way, in that it neither accurately represents the claim I’m making, the evidence I’m providing, or the inference I’m drawing between the two.

      As the post makes very clear in the beginning, I’m reacting to a common tendency I encounter where people seem to assume that there is some standard statistical definition of causation that is mathematically derived and straightforward. And I’m arguing that, in most fields, this is not true. I’m further arguing that causation is in fact a very complex notion, one that involves philosophy and epistemology. As evidence, I’m using a historical example of a standard that had been applied in a related area and ultimately was discarded because it was not tenable for the kind of research to be performed. Historical examples can be used to illuminate contemporary controversies. I am further pointing out places where certain definitions of casuality are not appropriate or practically applicable, given the conditions of different fields of inquiry. Again, with the intent in troubling a simplistic notion of causality that I think is unhelpful in these debates.

      You may find some or all of my points incorrect or not useful, but I would like it if you would attempt to make sure you have correctly read and understood them before you dismiss them. You know?

  4. Taking imply to be equivalent to proves comes from logic (the discipline). A implies B is the equivalent of if A then B.

    Material Condition

    It is also where the original statement Correlation does not (¬) imply (→) Causation comes from and the disconnect between imply the term of art and imply the everyday usage is probably what causes a lot of the misuse of the phrase.

    1. I’m not sure it’s right to talk about a “term of art” vs. an “everyday” usage, because, to my ear, using “imply” to mean “suggest”/”hint” only works when the entity doing the implying is a person with the intention of indirectly expression. Criminal suspects, your boss or the woman at the bar can imply/suggest something. But when one fact implies another fact, that typically means something closer to logical implication. I mean, if I said “the fact that the grass is wet implies that it rained”, and it turns out that it was actually the sprinklers that wet the grass, then my statement is simply wrong. It doesn’t matter whether wet grass somehow “hints”, “suggests”, “indirectly expresses” or “provides evidence for” rain. Most people, I think, would just say that my statement was wrong.

    2. “Material implication” doesn’t mean “logical implication” either. It’s a symbol used in logic, but it’s meaning is entirely truth-functional. “P→Q” is false if P is true and Q is false, and true otherwise. That’s it. If “implies” means “materially implies,” then “Freddie DeBoer is the Queen of England implies grass is purple” is true.

      What you’re looking for is “logical implication.” This is not a truth-functional notion, and there’s no connective for it in the language of formal logic. Rather it’s a meta-linguistic notion. The exact characterization of logical implication is a matter of controversy among philosophers, but a popular characterization goes something like this:

      A logically implies B iff under all possible reinterpretations of the meanings of the non-logical expressions in A and B (i.e., words other than “and,” “or,” “not,” “if… then,” “some,” “all,” etc.), either A is false or B is true. In other words, the material conditional “A→B” is true under all reinterpretations of non-logical expressions.

      (This sense can be given a precise mathematical characterization in model theory.)

      For what it’s worth, I think “implies” in English sometimes means “logical implication” and sometimes doesn’t (it almost never means “material implication”). An example where it does is “The fact that the tallest person in the room is a spy implies that there is a spy in the room.”

  5. Judea Pearl has a nice approach to this problem.

    http://bayes.cs.ucla.edu/home.htm

    His underlying idea is that causality is the component which remains after you mess with the underlying composition in all the possible simpsons-paradoxical ways. In economics and marketing I find this a highly effective idea – probably would also work well in sociology. He provides a formal calculus as well.

    So for your “poverty causes low educational outcomes” theory, to study causality you’d need to check if the effect remains after you vary the composition of the students. I.e., hold parental race fixed – does the correlation remain? Hold parental IQ fixed – does the correlation remain?

    1. This is what I came here to talk about.

      Pearl’s book Causality: Models, Reasoning, and Inference is a masterpiece.

      He provides a formal mathematical framework for discussing and proving causal claims, and for simulating causal interventions and counterfactuals. The math neatly extends from basic Bayesian probability calculus.

      The theory is still developing, since the book was published in 2000, but it’s popping up in economics, information theory and statistics, machine learning, social science, etc.

      Orthogonally, is it odd that I enjoy your methodological/meta posts the most? I would totally read more posts about yoir research.

  6. I think it does make sense to talk about separate technical and colloquial meanings of ‘imply’, but that the colloquial meaning of imply actually isn’t all that different from the technical one in formal logic.

    I find that ‘imply’, generally speaking, is weaker than ‘prove’ (except in limited cases where ‘imply’ is used technically in formal logic), but also stronger than ‘suggest’. Consider a case of referring to a tree in a front yard. Calling the tree a “tree” is an explicit reference to the tree; calling it “the largest plant in the front yard” is a little more implicit; pointing at it and calling it “that thing over there” is more implicit still.

    I take ‘imply’ to be related to ‘implicit’, as in implicit vs. explicit language. People imply things all the time in ways similar to the way the people in the hypothetical tree example do, and still manage to successfully communicate. Even the person pointing at the tree and calling it “that thing over there” probably won’t be misunderstood to be referring to the street, a house, or anything else is in the direction being pointed at. People can reason with each other even when using implicit instead of explicit language.

    This idea of ‘imply’ differs from ‘suggest’ in its open-endedness. ‘Suggest’, while interchangeable with ‘imply’ in some instances, is more open-ended. Note how making a suggestion that X indicates a greater degree of receptivity to X not being realized, whereas implying something is more decisive. ‘I suggest X’ entails that I won’t be bothered much or experience much cognitive dissonance if X doesn’t happen or if X is discovered to be false.

    This is related to the technical sense of ‘imply’ in formal logic, as ‘logic’ is sometimes used in a somewhat archaic, broader meaning — to mean something like ‘reason’, ‘rationality’, or ‘critical thinking’. To be logical, in this broader sense, is a very low bar to clear, and it is ubiquitous. ‘Imply’ may have developed two meanings as a result of ‘logic’ becoming associated specifically with formal logic: the technical meaning of the word, and another meaning associated with what would nowadays tend to be called ‘reason’.

    So all that said, I think ‘imply’, unless used in its technical sense, has a meaning somewhere between the level of strength of FdB’s definition, and its intended usage as a synonym of ‘prove’ by some users of “correlation does not imply causation” (which may in some cases be informed by formal logic).

    I agree that “correlation does not imply causation,” with ‘imply’ used as ‘prove’, is a problematic retort to statistically inferred causation, but since ‘imply’, while weaker than ‘prove’, may not be weak enough to evaluate utterances of “correlation does not imply causation” in any way besides on a case-by-case basis, I don’t think the line in the sand that ‘imply’ is closer to ‘suggest’ is going to dispel concerns about the meaning of ‘imply’ enough to get the point across more universally.

    However, I think Freddie did something interesting (probably on accident) in his previous post that might help. In the title of that post, he declared his opposition to thoughtless retorts of “correlation does not imply causation,” but in the body, he slipped into quoting “correlation is not causation” and “correlation does not equal causation,” similar statements common on the internet but which lack the word ‘imply’. The issue of the meaning of ‘imply’ may be sidestepped by hinting that “correlation does not imply causation” is part of a group of similarly problematic statements, showing that whatever ‘imply’ actually means in general, at least in those retorts, it is intended to be used as ‘prove’.

    This would call attention to the meanings of the whole phrases “correlation does not imply causation” etc., rather than the individual words like ‘imply’ that compose them. I think it would also be a better solution than replacing ‘imply’ with ‘suggest’, since ‘correlation does not suggest causation’ is uncommon (perhaps due to reasons relating to the relative weakness of ‘suggest’ detailed above).

  7. I think your statement: “I believe that poverty causes poor educational outcomes.” is incomplete. My belief can be summed up as follows, “I believe that Under our current education system, poverty causes poor educational outcomes.”

    I see nothing about poverty per se that would cause poor educational outcomes. To say so would be to say that the poor are somehow incapable of being educated irrespective of anything else save poverty. Im no expert, but to me, this is nonsense. There is nothing inherent about the poor that should prevent them from getting a good education, or to put it another way, you can imagine a system where by poor kids could get a top rate education, and its hard to believe that they would fail simply because they come from a poor background.

    The problem is, we have a system that says “you people there in that poor neighborhood have to go to that shitty school while these people here in this rich neighborhood get to go to this good school” Then we all scratch our heads and wonder why poor kids have worse outcomes.

    The ironic part is that this system is defended most vocally by those who claim to be opposed to racisms and classism.

    1. Your causation is backwards, as I’ve written many times. Poor students don’t do poorly because they’re systematically excluded from the best schools; the best schools are perceived to be the best schools because they systematically exclude the worst-performing students. When you say, “I see nothing about poverty per se that would cause poor educational outcomes,” you’re ignoring the fact that even within the schools that have the best overall performance, the poorer students do worse than the richer, and in fact just moving poor students to rich schools via vouchers or busing has a terrible track record. If you were right and the only cause of bad academic performance were bad schools, then swapping rich kids into poor schools and poor kids into rich schools would see a reversal in their performance. But we see nothing like that. Indeed, across a broad swath of the research, we see student-side factors overwhelming school-side factors.

      1. “even within the schools that have the best overall performance, the poorer students do worse than the richer, and in fact just moving poor students to rich schools via vouchers or busing has a terrible track record.”

        Thats a fairly amazing statement, one that im skeptical of. A couple of points in no real order. One, based on my very limited exposure to it, id say that educational research is much like gun control research in that there is no shortage of studies designed to advance a certain agenda rather than advance our understanding of a subject. This is not to say that all educational research is useless, only that im (I think rightly) way more critical of educational research (or gun control research for that matter) then i would be for other scientific claims. So when you claim that there are broad swaths of the research that prove whatever, im mostly unmoved. There are, after all, broad swaths of the research that claim that smoking does not cause cancer for instance. Unhappily, scientific research is often just another form of propaganda.

        Two,

        Two, your claim in the OP is that “poverty causes poor educational outcomes” but in your response you say that ” even within the schools that have the best overall performance, the poorer students do worse than the richer” There is, however, a difference between a “poor” outcome and one that is simply worse that someone elses. To me, a poor educational outcome is, say, being unable to read after you graduate. Compare that to an outcome that is worse that a rich student, while that could still mean “I cant read” it could also mean “i got C’s whereas Richie Rich was the valedictorian. I dont know about you, but ill take ‘I got C’s” over “I cant read”

        Third, the notion that its ok to force poor kids to go to different, often worse schools than rich kids is wrong even if where you go to school has no effect whatsoever on their educational outcome (which, again, seems unlikely). I just dont see how anyone can justify saying “you kids get to go to this nice school with lots of resources and mostly good outcomes because you live in this nice neighborhood and you kids over there, you go to a school that has substantially fewer resources and consistently worse outcomes, because you live in a poor neighborhood. How is that not racist and classist on its face? Hell, if where you go to school has no effect on your educational outcome, then where is the harm in letting people go to whatever school they want, even if the choice does nothing. Hell, a Ford and a Chevy might be almost entirely the same, but i still appreciate having the choice between the two.

        1. First, if you’re simply going to say “I don’t believe the research,” then there’s not much point in having the conversation. I assure you: that poor students whose parents were uneducated suffer significant and consistent disadvantage relative to others is a near-universal finding with decades of profound evidentiary support in a large variety of settings and through a vast number of experimental and observational mechanisms.

          Second, it’s not merely that poor kids continue to lag behind rich kids at schools with good reputations, but that poor kids tend to do almost exactly as badly at good schools as at bad. Again: though it may seem crazy to you, parent’s income, race, and parent’s educational level are hugely determinative of student performance, and are in fact far more predictive of student outcomes than what kind of school they go to.

          Third: why stop there? This is the bizarre part. You acknowledge class and race inequality, and yet your unit of analysis stops at the schools. Of course I want equity in school resources, but I don’t mistake that as the equivalent in equity for student lives. Sending a kid to a rich school for 6 hours a day, and then sending him back to poverty, drugs, and crime for the other 18 hours a day has not been proved to improve their educational outcomes. I know you find that hard to believe, but again, that’s the overwhelming anecdotal and empirical result. But more to the point, caring only about the kid for the 6 hours he’s in school is bizarre. And this is the far deeper point: asking for educational equality in a world of profound economic and racial inequality is crazy. We’ll never get there. We need to build a more equal society, and then more equal educational outcomes will come. A universal basic income or other program of market socialism will do vastly more for students than any charter schools or private school vouchers ever could.

          1. “if you’re simply going to say “I don’t believe the research,” then there’s not much point in having the conversation.”

            Im not saying anything like that. Im skeptical of the claim that research proves all of your claims for all the reasons i cited above. Skeptical != dont believe. Moreover, im not too impressed with an argument that is basically “Im right because research” and im surprised you seem to think otherwise.

            “You acknowledge class and race inequality, and yet your unit of analysis stops at the schools.” Of course, i thought schooling and poverty were the topic at hand.

            “But more to the point, caring only about the kid for the 6 hours he’s in school is bizarre. And this is the far deeper point: asking for educational equality in a world of profound economic and racial inequality is crazy.”

            You seem to be making perfect the enemy of better. Even if we cant fix every problem the poor have it is still worthwhile to try and fix some of their problems, in this case, a system that produces unequal educational results. I dont see the value in accepting a so obviously flawed system merely because there are other problems that also need solving and i cant really understand why so many others do. Would you accept your own logic applied to, say, police brutality? Would you defend a justice system that systematically murders and imprisons poor blacks way more than it does rich whites? Would you be satisfied with the argument that this is the way it has to be because some unspecified research says so? Would you really be content with saying “Poor blacks have so many problems that there is no value in trying to prevent the police from shooting them (a bad outcome if there ever was one)”? Or that “asking for judicial equality in a world of profound economic and racial inequality is crazy.”? If so, then i have badly misread your world view.

  8. Also, i wouldnt be suprised to learn that some poor students have issues that make a good educational outcome very challenging (like a tendency towards violence, for example), but i would be stunned to learn that all poor students have issues which make a good educational outcome overwhelmingly difficult.

  9. Here’s the best explanation of causation that I’ve found. It’s the one actually used in practice by insiders, not just the speculations of outsiders.

    “How can we test hypotheses on the etiologic relation of particular viruses to particular tumors? There is no simple answer to this, but perhaps an important part of the answer is a negative statement: we do not test such hypotheses by being bound to Koch’s postulates. Scientific proof of an hypothesis consists of elimination of all conceivable and reasonable alternative explanations, not in filling in the blanks in a prescribed set of rules. Koch’s postulates are a precise formulation of experimental requirements for eliminating alternative hypotheses in the testing of one particular pathogenetic model, that is, that an infectious agent produces disease reaction during its period of active multiplication. When pathogenesis involves delayed onset of symptomatology, Koch’s model just does not apply. Instead, we must formulate or predict the host-parasite relationship we think may obtain, examine this model for what inferences can be made, and then devise means of testing for the expected outcome of those inferences which are unique for that model.” (A Survey of the Tumor Virus Problem from an Epidemiologic Standpoint. Wallace P. Rowe, Laboratory of Infectious Diseases, National Institute of Allergy and Infectious Diseases, USPHS, Bethesda, Md. Cancer Research 1965 Sep;25(8):1277-1282, pdf page 5.)
    http://cancerres.aacrjournals.org/content/25/8/1277.long

    But this is not the sort of definition that the Surgeon General et al. have employed for decades. They use a fill-in-the-blanks definition of causality, with an extremely serious omission. They do NOT eliminate other possible explanations, and the explanation they systematically ignore is that of infection. They use studies based on lifestyle questionnaires which cynically exploit the fact that smokers (and passive smokers) and more likely to have been exposed the pathogens that cause cancer, heart disease, and other illnesses, such as Helicobacter pylori, cytomegalovirus, Epstein-Barr virus, and hepatitis viruses B and C.

    As a result, the Surgeon General et al. have gotten away with systematically deceiving the public about the health risks of smoking for decades – and the tobacco companies let them. For example, they let Jonathan Samet lie in the Minnesota tobacco trial that smoking causes ulcers, and never mentioned H. pylori, despite the fact that the National Institutes of Health had issued its consensus statement on the subject four years earlier. Samet, in fact, has been a ringleader in every Surgeon General report since the 1980s, as well as chairman of the IARC report, and even the EPA and ASHRAE reports! As if all this isn’t bad enough, he’s now the chairman of the scientific committee of the FDA committee on tobacco! And all this fraud piles higher and higher, while supposed people of science drivel about the supposed nefariousness of the tobacco industry, who have never done squat to oppose it!

Leave a Comment

Your email address will not be published. Required fields are marked *