“Initially… the translation was perfect, but when I started to speak in longer sentences, it basically fell apart and got a lot of it wrong. As I tested with others who spoke in Greek, German and French, we noticed the same thing. We could never completely rely on Google translate to get the words right.”
This is, I think, a constant dynamic in tech circles: astonishment, followed by growing dissatisfaction and frustration. So what makes machine translation difficult?
I could go on about these issues forever. (Put a beer in me and I’ll talk your ear off about why both the Chomskyan approach and the Peter Norvig approach are at an impasse when it comes to actually decoding language as a system.) Let’s pick one particular issue for true machine translation: the dilemma put forth by Terry Winograd, professor of computer science at Stanford. (I first read about this in this fantastic piece on AI by Peter Kassan.) Winograd proposed two sentences:
The committee denied the group a parade permit because they advocated violence.
The committee denied the group a parade permit because they feared violence.
There’s one essential step to decoding these sentences that’s more important than any other step: deciding what the “they” refers to. (In linguistics, we call this coindexing.) There are two potential within-sentence nouns that the pronoun could refer to, “the committee” and “the group.” (Note that both are singular and “they” is plural, so one thing machine translation has to overcome is problems with formalist grammar!) These sentences are structurally identical, and the two verbs are grammatically as similar as they can be. The only difference between them is the semantic meaning. And semantics is a different field from syntax, right? After all, Chomsky teaches us that a sentence’s grammatically is independent from its meaning. That’s why “colorless green ideas sleep furiously” is nonsensical but grammatical, while “gave Bob apples I two” is ungrammatical and yet fairly easily understood.
But there’s a problem here: the coindexing is different depending on the verb. In the first sentence, a vast majority of people will say that “they” refers to “the group.” In the second sentence, a vast majority of people will say that “they” refers to “the committee.” Why? Because of what we know about committees and parades and permitting in the real world. Because of semantics. A syntactitian of the old school will simply say “the sentence is ambiguous.” But for the vast majority of native English speakers, the coindexing is not ambiguous. In fact, for most people it’s trivially obvious. And in order for a computer to truly understand language, it has to have an equal amount of certainty about the coindexing as your average human speaker. In order for that to happen, it has to have a theory of the world, and that theory of the world has to not only include understanding of committees and permits and parades, but apples and honor and schadenfreude and love and ambiguity and paradox….
Some might say that this is a particularly bad example to pick with Google Translate, because it is a probablistic engine; rather than trying to parse the syntax-semantics interface for these sentences, it would merely see how these sentences or parts of sentences have been translated in the past, assign a certain probability to a given set of translations being correct, and act accordingly. (In terms of pure translation, anyway, it would only have to faithfully provide an equivalent language-specific reading of the English text to speakers of other languages, but I’m afraid in some languages that would entail having to coindex the pronoun itself.) That’s true– but it’s precisely that probabilistic nature, that reliance on chance, that leaves Ulanoff and his partners frequently unable to understand each other past a certain level of complexity. In order to do that — in order to go from pretty good to legitimately astonishing — I believe machine translation would have to move beyond Bayesian probabilistic approaches and towards developing an actual theory of the world for their models, which would entail a functioning theory of mind. Outside of Doug Hofstadter, hardly anyone is even trying to do that. (As my friend Alex Waller says, “It’s okay to discuss the pros and cons of AI, but we need to admit actual AI will almost certainly not exist in our lifetimes.”) So for now we’ll have to settle for OK and recognize that there’s always going to be the odd WTF awful translation popping up, because of what the human language capacity can do and computers can’t.