it’s not the crime, it’s the cover up

So this may get my Hater Card revoked, but I actually have found a lot to like about the early days of Vox. There have been some really good pieces on there– I liked the Passover explainer, and Yglesias’s distillation of Pikkety’s Capital in the 21st Century was well done, and an example of how that type of piece could function well. I haven’t read the book yet, but the post seemed to do a good job of relaying its central arguments and criticism against them, certainly informing people who were never going to read it, and I doubt there is anyone who was going to read the book who is going to skip it thanks to reading Yglesias’s work. Of course there are also pieces I find of dubious quality, but that’s the nature of the beast. Dylan Matthews failing to mention the loudness war in a piece on the benefits and drawbacks of vinyl is just bizarre to me; he calls dynamic range merely a drawback of records, when in many instances, the limited dynamic range on records is a benefit, since it keeps mixers and engineers from squashing the mix in the race for more and more volume. (Is it insulting to have pieces that ask “what is marijuana?” Sure. It’s a little insulting.) Like any publication, Vox will likely have some good, some bad, and it’s up to readers to be discriminating. So far, I’ve been pleasantly surprised.

That being said: this piece by Kelsey McKinney is really bad, and worse, it’s bad in exactly the way that people have feared about Vox. (People like me.) The legitimate fear, with a site like Vox, is that the urge towards simple explanations and data-driven writing will result in pat, seemingly authoritative answers to questions that are actually far more complex. And that’s exactly what McKinney is guilty of, cooking the books to make the world seem far more orderly and simple than it really is, and doing so in a way that seems transparently designed to reach a particular conclusion. Matt Novak of Paleofuture has an extremely effective, thorough, and utterly fair dissection of McKinney’s piece that really could stand as a model of how to write a rebuttal of this kind of piece. I highly encourage you to read it. There’s a lot of territory that Novak covers, but most damningly, he shows that McKinney’s dates for when prominent technologies were invented are almost entirely arbitrary. Arbitrary dating can sometimes be OK– many technologies, after all, don’t really have a specific invention date– but when your variable of interest is entirely dependent on length of time since invention date, that’s just an existential problem with your research.

McKinney’s graph alone should make us skeptical. Check this out.

That’s almost perfect! Which is enough to make the hairs on the back of my neck stand up. Real data does not look like that. Not DVs, anyway, not data the researcher doesn’t directly control. A stair-step graph of such uniform precision is only going to be found under rare research conditions, ones where we’re dealing with variables that are highly constrained or mechanistically related. And the time to adoption of prominent technologies simply is not that kind of data. This chart should raise red flags. The first question anybody should ask is: what about the technologies that aren’t on this chart? How does the selection of these technologies affect the model? I’m willing to bet we could find 7 technologies that did not fit this pattern, even if we have a more rigorous method for determining invention date.

Now just getting things wrong is okay. (I do it all the time!) After all: Novak wrote that response. As pessimistic as I can be about online journalism and commentary, this is how it’s supposed to work. When unsupportable claims or irresponsible analysis happens, hopefully, somebody else will do the work to refute it. There’s nothing disqualifying about McKinney’s piece. However, Vox apparently heard the criticism and reacted to it– but didn’t come clean. Here’s Novak’s update:

It looks like Vox has heavily edited their original story by creating a new lede, referencing research by the FCC and Pew Research, and correcting factual errors like their conflation of the invention of the internet with the invention of the web. They’ve also swapped out their original “Technology Adoption” graph with a new one and added an additional graph with data from the FCC. None of these edits are noted in the piece. But they appear to be sticking with their original argument anyway, so the critique below still stands.

That’s just bush league, in my opinion. It’s okay to make corrections– better than okay, actually, it’s necessary and responsible. But you have to come out and say you did that by writing a brief section (a paragraph will do) saying “we changed X, Y, and Z, and this is why.” If you don’t, it just looks dishonest, and it risks contributing to a sense of imperiousness that is not a good look. Worse, it gives you less incentive to not make the same mistake in the future, if you just disappear the old problems. There’s an “Updated by” line at the top, but no other information, and for me, that doesn’t do enough. Don’t compound the problems, guys. Just own up to them.

Update: The post in question now has a brief correction at the top, which is progress. As with Novak, I don’t think the correction addresses the fundamental problem.

12 Comments

  1. The most dubious thing about that graph is its selection of start dates. What does it mean to say electricity came into being in 1873, or the Internet in 1991? As you say, we need a “rigorous method for determining invention date,” or at least one that isn’t numbers pulled out of a hat that happen to produce a nice graph.

  2. Not hugely impressed with Vox so far. A lot of their stuff seems to be information I could as easily have found on Wikipedia (more easily, maybe, since Wikipedia loads faster). The writer who is doing Ukraine pieces seems to be baking in Western assumptions before the analysis part begins. And so on. True, the Yglesias piece you link to is a good explication to a non-specialist audience in a non-condescending tone (sometimes a problem for him). So, hit and miss.

    Similar reaction to 538. To my eye they’re doing more factoid aggregation than analysis. Did we really need to know what percentage of American women groom their pubic hair?

  3. Freddie,

    Good points about the Vox article.

    Will you modify/correct your post now the the article in question now includes an “update” paragraph at the beginning of the piece and a “correction” paragraph at the end explaining the changes that the author made?

  4. Why use invention date, which is inherently problematic, and not use something like time from 1% adoption to xx%?

  5. Freddie,

    Let’s talk. No I mean really, let’s do a podcast. I say this an expert in rhetoric. I’ve forgotten more than you know. Ask James Hikins. I beat Ted Cruz in debate. I sold all the coal off the Titanic. I literally sold every wheat ccent penny in America (17M of them). I coached 20 kids to nationals in speech / debate by time I was 21. I’ll add more things if you won’t stipulate to my expertise.

    Because this vox take down you think is god’s own prototype is ridiculous.

    The “Internet” and literally NO technology “exists” when it is under government control. Having been on a pc since 1979 DAILY, let me say this as again as an expert: the Internet is 1994, not even 1991.

    It’s just f’ing silly to say otherwise. WE COULDN’T GET IT FASTER. We all knew what it was, but there weren’t any commercial interests trying to SELL it to us. UNTIL it is for sale, it is not a product.

    Do you understand the phrase, “government should do what the private sector can’t or won’t do”?

    What that means is, government is only justified IF AND ONLY IF it is begging private sector to do it instead, and private sector won’t or can’t do it.

    Smartphones are the next example. Start with iOS and Android.

    Then do Netflix streaming.

    The next one is VR rigs.

    I’m not saying the VOX analysis is great. I’m saying it happens to be right, and you shouldn’t be looking for takedowns of this assumption – it doesn’t HELP you in any way.

    Selah.

Leave a Comment

Your email address will not be published. Required fields are marked *