 This is Expert, a visual tool to explore the learned contextual representations and attentions of BERT and really any other transformer model. You can find this work at www.expert.net. This is the landing page for expert. At the top left you can see an input sequence box that allows users to input their own sentence to view in the attention visualization below. You can select a model on the online demo, filter the attentions to show only the attentions you care about and remove noise that you don't care about, change the layer to show different embeddings and attentions throughout the model. Down here we have the attention visualization. On the left we have each of the individual tokens. By hovering over any token you can see the attention out of these tokens on the left into every token on the right. By clicking on a token you freeze the view and update the right head box accordingly. By hovering over any individual column you can see the contribution of just that head to the attentions in the middle. By clicking a column you can toggle whether or not that head's contribution is displayed in the attention box in the middle. To select only a particular head, unselect all heads and click on your head of interest. The first demo we'd like to show is a demo we call behind the mask. In this demo we'll be using a BERT-based model which is trained with a system called mask language model. Essentially the way it works is given all the input tokens, mask a single token and ask the model to predict what word belongs here using the context before and after the word. We have implemented this feature in expert with a double click over any of the input tokens. Here you can see that when before it was masked we wanted to predict the word local because it knew the word local belonged there. But after masking we see that it wants to use the word nearby. That's an input local. This indicates that BERT has a good grasp of the English language. In this example we'll focus on the word escape. We know that it wants to predict the word escape at the output but we don't know when in the model it encodes that information into this mask. By selecting this token we enable us to search by the embedding across an entire corpus in this case the wizard of us where every word in that corpus is annotated for its attention, its embedding at every layer and its context at every layer. And its context in every layer is defined by a concatenation of all of the head embeddings. Searching by embedding then at layer one we see that we have no information encoded it matches punctuation not the verb that we expect. We can confirm this by scrolling down to the histogram where 45 46 of the top 50 results that are closest to that embedding are punctuation. However by the output of the model at layer 12 searching for the embedding we see that we match words like bother and kill and the histogram here for the part of speech confirms that verb matches 46 or 47 of the top 50 nearest embeddings. So here's the question at what point in the model did it become very confident that the masked word escape should be a verb. If we go back to layer six and search by the embedding we see that we have no verb information in histogram and skimming over the qualitative results can confirm we match punctuation determinants. If we search at layer seven we will see that we're matching a tiny bit of verb information but mostly determinant. So we've started learning that this ought to belong at least to a little bit to a verb. Now by layer eight searching by the embedding we see that we're matching verb majority of the time and we're matching words like kill and have and see and fight. In this way we can explore at what point transformer models at least in the context of natural language processing learn what meta information should belong to particular tokens. The next demo we want to do is exploring gender bias in GPT2. The first thing we'll need to do is select from the model drop down this GPT2 and then we input the sentence that we care about. In this case we'll input the man visited the nurse and told him to attend to his patients. To explore gender bias the easiest way for GPT2 is to ask what would you like to predict next given the context before. In this case if we click on the word hold and hover over it we'll see that the model wants to predict the word her. Know that it does not know any information after this word because it's not a regressive model. We want to search for heads that connect told to nurse and see if they encode gender bias across the larger corpus of the Wizard of Oz. The first thing we'll do since there's not many heads in the first layer that connects told to nurse is go to the output and we see there's not much here. Exploring a couple layers before we see there are two heads actually. First thing we can do is inspect one of them. In this case head three which very strongly tends to nurse and search by context. Do we see anything that makes it clear that it's attending to feminine nouns or masculine nouns. In this case no it connects heart, wear, blood, bitten. So maybe it's not that head that's encoding the gender bias. Let's check head eight. In this case we see that the max attention is to girl or Dorothy or which. This one more likely has some impact on gender bias. We can check if this head encodes gender bias also for men by replacing the word nurse with the word doctor and repeating the experiments. So the sentence structure hasn't changed. However by clicking on the word told and we're still in the same head we see that there's significantly less information from that head. And we also know that it would predict next him with 68% confidence. Doing a similar context search and we see that we're essentially matching no op characters. Often the word when. Let's explore a different head to see if a different head includes masculine bias. Head three of this layer matches the very similar things head three did for nurse. So maybe it's not in this layer that masculinity is encoded into particular roles. Let's check layer nine. In layer nine we have a different head that's very strongly tends to doctor. Let's search by context. Searching by context here we actually match traditionally masculine roles. Farmer, grandfather, man, field, Oz. In all these ways we can see that gender bias is not encoded consistently across heads. We can check one last time by seeing if this head encodes feminine gender bias. Already we can see that the attention out of told into nurse is less than the attention out of told into doctor. However what we want to know is if head nine dash four that is layer nine head four encodes both masculine and feminine bias we would need to check more than the single example. So once again let's search by context. If we look at the qualitative results it doesn't seem like this head encodes feminine bias. In fact it starts with farmer. However as we scroll down we see that her is matched in the top three results. Her again soldier girl is again matched. Woman is frequently matched. Soldier girl is matched. Because of the amount of feminine words that are included in the top results this might be a good indication that perhaps pruning this head or modifying the behavior of this head might mitigate the gender bias you see at the output of the model. This concludes our demonstration expert. You can access the live server running at expert.net. There you can link to our open source code into our original paper and play around with demonstration similar to the ones that we just gave. Alternatively you can also go to huckingface.co.expert to explore community uploaded models and newer model architectures using a lighter version of the same tool. We hope that you guys can uncover a lot of information and hidden behaviors inside of the attention patterns and contextual representations of your own transformer model. Thanks for watching.