 Hi, my name is Hubert Baniecki and I would like to show you how to open the machine learning black box with Model Studio and Arena. I'm a researcher and data science student at Warsaw University of Technology where I developed tools for explainable machine learning in R and Python. And I would like to highlight Model Studio and Arena, which based on the Daleks R package, which is one of the most popular R packages for explaining predictive models. If you are not familiar with the Daleks package, I really recommend you to check it out. There are a lot of changes in the process of explaining predictive models, mainly connected with the fact that we present the explanations to various stakeholders. Explaining predictive models might have a high entry threshold in a way that we need a lot of knowledge to understand the explanations, but also a lot of practical technical skills in coding them. One could say also that there is quite a problem with reproducibility of creating the explanations for predictive models. So to change these problems, we would like to provide tools that would automate the whole process, maybe provide some versioning system for explanations. But most importantly, we like to create an interactive experience in which users can customize their explanations in a given way and explore a lot of them. So when we talk about interactive explanatory model analysis, what we really mean is a process in which we compare multiple explanations next to each other. So a single aspect model explanation shows only one side of the black box model. Well, we would like to compare multiple explanations to create, to gain a lot more information and broaden the context. There are a lot of explanations that we can use and that benefit one from the other. But also we can't forget about the data exploration visualizations that can really enhance the explanatory model analysis process. So the Model Studio R package creates a dashboard for model analysis. Here we see an animation of exploring such a dashboard where we can choose multiple variables to explore or different observations for our local explanations. So we see a grid of four plots that we can customize in any given way. And this dashboard is created automatically by our function. And what is the most important is that this dashboard is serverless, meaning that it's only an HTML file that you can easily save and share with others. In saving these dashboards for multiple models might serve as a versioning system for explanations for all of your experiments. So maybe you compute some models in the cloud and at the end you would like to create such a dashboard and then this dashboard can be sent to you by email and you can only focus on analyzing the model, not coding the explanations. Here I present all code that's needed to create such dashboards. So given data and model, we create an explainer object using the Daleks R package. But then we can create a panel and then we can create the Model Studio dashboard using the explainer. If you're interested in this example, you can check out my GitHub repository where I host the code connected with this talk. The Arena dashboard is an advanced tool that allows you to compare multiple models and show various explanations next to each other for them. What is more important, you can also compute these explanations on multiple datasets. Sometimes it is quite important to compare the explanations on the training and validation sets. There are also some fairness posts available for assessing the model bias. And here we see that we have a lot more pages available for us. So we can save more explanations and more views of the model. I would say that this Arena dashboard is still easy to create as it takes only a few more lines of code as we need to create more models and more explainers. But finally we are presented with a dashboard in an offline or online fashion that we can really use to open the machine learning black box. Any feedback would be appreciated, so do not hesitate to contact me if you have any questions.