 Hi, I'm Bruno Scaronne, presenting our work Understanding Search Behavior Bias in Wikipedia, which is joint work with my advisor Ricardo Vizai Yates from Nordistan University and Eric Bernhardsson from the Wikimedia Foundation. Let's start with our motivation. Extensive research over recent years has recognized the importance of studying the existence and impact of biases on the web and its search process. In particular in this work we focus on one of the most visited sites on the web, Wikipedia, which as you probably know is the world's largest open encyclopedia, providing content to millions of readers across the globe in more than 300 actively edited languages. Due to this we believe it is indeed relevant to understand how its users browse the content of the site, in particular in this case using its website search tool. The high level objective of the work was to identify elements affecting or biasing the search behavior of Wikipedia users. For this, based on Wikipedia's server logs, we generated datasets containing millions of search sessions to first characterize its website search and then compare it to generic web search. In this work we focus on the potential sources of bias shown here inside the blue rectangle on the slide, which are the type of device, namely whether it is a desktop or mobile device, the page lengths of the articles, as well as the ranking position of the pages. We guide the study by three research questions, namely whether the client type affects the observed search behavior, whether the page length affects the time a user spends reading a page, and finally whether the ranking of a page affects the time it takes a user to click on the link to it, which is also known as time to first click. Now I'll give you an overview of our results starting with the first research question. First I'd like to note that our analysis is based on clickstream data as a proxy for search behavior and the dwell times derived from it, which is common practice in the information retrieval literature. As a reminder, dwell times are computed as the time difference between two consecutive clicks within a search session, as you can see on the equation shown here on the screen. Related to dwell time computation, in our data sets we observe that many sessions are single clicked, and thus we cannot compute dwell times for those. The main observation here is that for the analyzed variables I introduced earlier, which are also shown inside the orange rectangle on the slide, we did not see any significant difference in the results when segmenting them by client type, which we know is not the case in web search, as we will see later in the talk. Now I'll continue with the dwell time distributions. Here for visualization purposes we see a sub range of the distribution of dwell times for desktop on the left and mobile devices on the right, both exhibiting the same general pattern. In particular, both have two modes, the first as you can see occurring around zero seconds and the second close to seven seconds. The first has been previously reported as accidental clicks in the context of mobile devices. And given that we also saw several consecutive clicks to a page were being made from the same referrer in our data sets, we filtered them and indeed observed the first mode vanishing in this filtered version. What is perhaps surprising here is to observe the same behavior for desktop too. Regarding the second research question, we analyzed the duration of dwell times and the size of the pages where these were measured. Again here the pattern is the same regardless of the access method, namely there is an initial peak, closely after zero seconds given by accidental clicks, as you can see here on the charts, followed by a steady increase until reaching roughly 60 seconds and after that the page length oscillates. This means that it is only the case for dwell times shorter than a minute that the longer the dwell time the bigger in size are the pages being accessed. This may be what one would intuitively expect, but as I mentioned before it is not true in general for either of the device types we have analyzed. To further confirm this we did two things. First we plot the data without being in dwell times and that's no averaging page length, as you can see in this scatter plot shown on the right section of the slide. And also computed the Pearson correlation coefficients, which as you can also see on the table, confirm the anchor relation between the variables. This behavior was known in the context of genetic web search for a small sample of 25 users. But in this work, as I previously mentioned, we use logs containing millions of sessions, greatly expanding this sample size. Regarding the relation between the ranking of a page and the time to first click, we analyze this both at the search level and also at the session level by averaging over the different searches within a session. And what you can see here on the charts is what happens for all cases, namely that the variables are uncorrelated and that the two aggregation levels show similar results, meaning that the search behavior tends to be uniform within a session. Here you can see all the correlation coefficients that confirm the anchor relation, which imply that in this context the ranking has no significant impact on the time it takes a user to click on a result. Given that we know users rarely view more than 20 results, and in particular not the entire list of results returned by a search engine, we hypothesize that users keep results without really reading them. Otherwise, we would observe a linear time increase in the collected times. To conclude, we envision this work as the first steps towards identifying ways to improve the search experience of Wikipedia users. And so future work includes, on the one hand, providing insights into what produces the observed behavior, as well as including additional elements into our analysis, like the semantics of the queries, to obtain further insights. The first point can be addressed by segmenting users, for example, by languages, and assessing if the average search behavior differs among them. Or tokenally, given the in general transit nature of the search behavior we have observed in this work, we would also like to explore measures that encourage longer page stays in the articles, or alternatively, ones that assure the essential information of them is located at the top of the pages, so that it is not missed by readers. We thank you for your attention. Our code is available on GitLab, and we are happy to receive any feedback you may have.