 So this presentation is what happens when you do a PhD in machine translation and I could just leave it there when and You just watch so much Star Trek when you're meant to be riding up and also at all other times I Ranted so many times about the translator in Star Trek to my partner that eventually she was just like Why don't you just do a presentation on it and here we are So I Imagine you've all encountered machine translation before Google translate among others This is a somewhat optimistic headline from ten years ago and Unless someone has been keeping a lot of secrets from me and the rest of the machine translation industry This is not the case But there have been a lot of advances in the last ten years so the first half of this translate of this Presentation will be a somewhat non serious look at how machine translation works these days and How it struggles these days and the second half will be some of my top rants about Star Trek's translator Okay So machine translation today. It's basically based off neural networks sequence to sequence neural networks as they're known and The very high-level idea is you have some giant matrices and they map from a sequence of numbers to another sequence of numbers and The maths behind that gets a little bit complicated and I won't go into it But Effectively you show a bunch of examples of one sequence of numbers You show some other sequences of numbers and it learns that when these numbers appear here these numbers appear here This is useful for translation because if you associate each number with a word you suddenly get a translation system in theory so how do you actually Train these you need as I said lots and lots of examples of machine of translations Sentences in one language Sentences is another language show them to the system at one point. It adjusts the values in the matrix To make one likely given the other This is actually a Fairly typical example of the sentences given to a machine translation system a lot of them are Trained on sentences scraped from the internet. I see so much Star Trek plot summaries in my day job All of that Star Trek was worth it To say nothing of the fanfic So this is a fairly typical example But the idea is you show millions of these or tens of millions of these if you have them available Sentences with different vocabulary sentences with different word patterns in them so that the system can learn These words occur and these words occur. These are the patterns that make up language and And Like I said the system doesn't know what is a verb and what is a noun and what is what it just sees numbers But it sees numbers that occur in the same patterns It doesn't have to know that what I've highlighted are names if it sees that the same Terms if a term appears in one sentence it has to appear in another sentence Doesn't need to know that's a name It just knows that it has to copy it across if it occurs for example Before the phrase is a fictional character probably a name Can anyone close to the screen see a potential problem with this example? You don't have to know French Kate Mulgrew is in there and she shouldn't be Kate Mulgrew is in the French sentence, but she's not in the English sentence. This is a genuine real problem not Kate Mulgrew, but Well, maybe if you've seen Voyager Not Kate Mulgrew. She's lovely The Examples should ideally be good translations of each other They're not always in fact. They're not even usually What we have to hope when we're giving all of these tens of millions of sentences scraped from the internet and aligned urine using statistical heuristics is that they are mostly good translations of each other and When it sees that there is this random Little phrase that is in the output translation that should not be there If it sees enough other good examples with good translations of Kate Mulgrew, we're fine if the only place it sees this kind of sentence is in the context of fictional person in Star Trek Then every time it sees fictional person in Star Trek it might start putting Kate Mulgrew in there This is how we end up with hundreds of rogue Kate Mulgrews in our translations. This is a real again Not Kate Mulgrew. This is a real problem. I've had to contend with I was once training a system to translate biomedical papers now funny thing about biomedical papers The English side Typically had the title of the paper in square brackets the non-English side did not I found this out when any Text in square brackets was just triggering absolute goldy gook on the output any text in square brackets bad So that's one potential problem with machine translation We really have to have this data well aligned and you can't look at millions of sentences. You just can't Another potential problem. I Said that these matrices are mapping sequences of numbers to sequences of numbers and they are But they can only map so many numbers usually the number is about 50,000 There are a lot more than 50,000 different words in a language so We don't map words to words Instead we normally break words down into sub words This is cool. It lets you represent unseen words. You don't have to Have seen a word very very frequently in the worst case possible you can just spell it out break it down into letters But the other cool thing about this is That the words don't even need to be in the same language once again these systems don't know what a word is They don't know what's English and what's French and what's whatever They can just spell out if you put in a German phrase and say that it's an English It can just spell it out and make your model think that it's going to try and translate English that goes badly But it is a way towards Universal machine translation going from many languages into one it helps if you have a little tag that says this is X language But you don't have to another thing about kind of what we call Rare machine translation pairs They don't usually actually translate directly We need millions of sentences Examples to get these systems working well often for a language pair like French to Chinese We may not have those good examples so what these systems are often doing in the background is They are translating into English and then they're translating back out of English and that Sometimes causes some errors because as you may know machine translation is not perfect It's not perfect when it goes into English And it's also not perfect when it goes back out of English again So you get even more opportunities for it to go wrong if you've ever wondered why some of these language pairs are Little bit worse translating twice What do you do when you're seeing a new sentence? When we're training this model we see sentence in one language sentence in another language When we see a new sentence, obviously we have no translation Otherwise, we'd be fine so what we do is translate based off the input sentence and Everything we've translated so far and that's great if what we've translated so far is sensible And if we know how to read the input sentence If we don't know what to do with the input sentence if we've never seen this kind of language before We just fall back on what is likely given what we've output so far or What is likely given the language we're outputting so far if we had no idea what this input was this sentence might finish today It's a good day to do the dishes If it knows the input sentence, we're fine you know What is likely to a machine translation system in almost all language pairs What data is used in almost all language pairs and what gets fallen back on if you have no idea what to do with an input The Bible this is a real problem This isn't this is an old example, I think they fixed it now for Specifically really confusing inputs, but for a long time there was a known bug in machine translation Not just Google generally where if you put incomplete nonsense what it would output was oh, I don't know what to do with that quick Let me just produce a sentence. That's likely in English and for for the very rare languages The data that is available is the Bible so the sentence that is likely given the training data is from the Bible And you get some absolute nonsense. You also get probably my favorite headline of all time Not just for the phrase the spooky nature of neural networks, which I think is my new band name Okay On to the Star Trek Rants so I'll be rating these out of ten where ten is plausible I'm assuming there's no telepathy if there's telepathy all of this makes sense But brings in some ethical questions, which I guess we just don't talk about So no telepathy Explaining the example and then I'll explain why I've ranked it as plausible or not Okay, this section is rated 12 for language in this example and in many episodes There are Klingons speaking Klingonese and it's not translated. They're just having a conversation And it's not translated This doesn't make any sense. Okay, so So the thing about trying about training a system to not translate something if you if you do it too much The system will refuse to ever translate that language It just learns to copy it all of the time Because copying is easy It's very easy to leave something untranslated and these systems like to do the easy thing the the optimization algorithm favors it So There's a couple of options here one is that You have some kind of Spoken register which indicates that you don't want this to be translated by the universal translator on a Klingons ship Where there's one federation person sitting there The other option is that it's directional and it only translates it for you if You're being talked directly at but we see people over here in conversations through the translator. Look, he's sitting right there. He's listening This isn't very plausible Um Next Individual untranslated words. This is a classic example of a Klingon being somewhat uncomplimentary towards someone This is perfectly standard. This this makes perfect sense. There are loads of words even in our human Existing languages, which we wouldn't translate under a machine translation system because in Context you don't have a great translation, but maybe more relevantly The cultural context of that non English word or non target language word is perfectly clear to us in the federation They presumably know what that means and what it means when a Klingon says it to you and If a system is trained not to leave that Word not to translate that word. It doesn't translate it. That's completely plausible The distinction is that the system doesn't have to copy huge tracts of language if it copies huge tracts of language Then you run into the risk that it just forgets about that language completely But one once in a while One untranslated word it gives it gives the right cultural vibe well, oh Spoilers Okay in the episode Darmock we have a culture that communicates solely in memes Wouldn't know anything about that. This is pretty plausible This is pretty plausible like I said systems are good at copying words. They're good at copying names They're good at figuring out what is a name and copying it across sometimes they garble it a little bit But the point is the system doesn't like Picard It doesn't need to know who or what Darmock or Jalad are it just needs to know that They occur in this place in the sentence. They should occur in that place in the output. It's fine. That's fine We're getting to the good ones Translating words in isolation Makes me sad and it makes all machine translation researchers sad What does shields mean okay? Are we are we talking about a part of the ship? Are we talking about medieval armory are we talking to someone called shields in this case? It's a command. It's it's actually an instruction to raise the shields, but how do we know that it's just one word you can't So people often use machine translation systems like dictionaries They are not trained as dictionaries and they don't really work very well as dictionaries So pro tip for today Going to Google translate to put in a single word often doesn't work very well Because they're not trained to translate without context Actually Google translate does I think now provide a dictionary underneath for this reason because people are doing this But the fact that the universal translator can handle words in isolation and make it make sense is Okay, this is an episode from deep space nine where a completely alien language turns up And the universal translator is able to slowly learn the language in the first in the first Picture no idea what it's saying in the second picture. It's picked up on this one word That's pretty pretty. That's pretty plausible. This is Actually how machine translation learns if you look at a system early on in training It's worked out maybe one or two mappings across The reason I've marked it down a bit Is because mostly what it does for the rest of the sentence is output nonsense in the output language So I would expect to see a bunch of random English words in that output But I'm being pedantic and when would I ever be pedantic? It occurred to me that need was a bit of a weird word to translate out of nowhere But actually you can work out when someone is saying that they need something For example, this is my cat. I Don't speak his language, but I know when he's saying that he needs food He is always lying This definitely wasn't just an excuse to get a photo of my cat When I speak English, I do not look like I'm speaking Japanese Mike that I do not know how to explain to Star Trek That your mouth makes different word different shapes when you're saying different words Okay, dubbing is difficult Dubbing is really difficult the amount of time we spend saying a sentence is different in different languages Some languages are incredibly rich. You can say a whole sentence in one word. Some you'll take two or three times that long some have loads of syllables per word some have loads of valves some have loads of Plosive consonants which make your lips make different shapes You emphasize your words with body language at different points in the sentence It doesn't make any sense if you're hearing everyone in your own language, but their lips are moving With a completely different language How? This upsets me Yeah, this is the one that I can't get around without telepathy But even with telepathy like how would you Are you like mentally cutting together different parts of the of your vision it? Yeah, no Okay That is my talk So machine translation can do cool things It can also break in I've said amusing ways I think they're interesting ways the ways machine translation breaks tells you a lot about the ways it works Universal translators are not unique to Star Trek. You can see this kind of behavior in a lot of sci-fi But I hope I've given you some little ways to become nearly as pedantic as I am about this and If I have one takeaway for myself, it's that I Should learn to stop worrying and love the telepathy. Thank you very much. I'll be around here if anyone wants to talk