 So our first presentation will be by Cif Waffen Schmidt and Elke Hausner, both at the Institute for Quality and Efficiency and Healthcare in Germany. And their talk is entitled search strategy development at a German health technology assessment agency, our experience with our from an end user perspective. Yeah, hi everyone. Elke and I, we are going to talk about search strategy development. First, we're going to give you some background information, then we take a look back. Because we used our from 2008 to 2011. And we are going to talk about our current situation and give an outlook. So, our agency is conducting systematic reviews to inform the German healthcare system on the base of the systematic reviews decisions in the healthcare section in Germany will be done. And Alkandai, we are from the information management department at Igwik. That's a short version of the long version Igwik. And in this department, we have eight information specialists, and we are conducting or developing around about 200 searches per year. And from 2008 onwards, we were looking for support in the development of search strategies. And it was because during the first years of a quick we had a lot of interaction and discussions with the project managers about the searches. And the project managers would say, are there. This is a term, or these terms they are super important and they have to be in the search strategy. And we have the feeling that these terms might not be so important. And they would really increase the garbage of the results of the search. So, but we had no way of, we had no arguments to solving this discussions. So, we started to implement text mining, because we wanted to have because we are from an evidence medicine or work is based on evidence based medicine so we thought we would like to base our approach on evidence based and we our work is based on Jenkins 2004 that is a publication that describes the development of search filters and we did the same. We generated test sets, and we base our search strategies on discussion around the test sets, and it's more. We have more objective approach and then we also have more way on discussing and making decisions about the search strategy. I would like to give you an example. This is a search we did in 2006 about prostate cancer without text mining. You could see you don't really have to understand what the each of the search lines do and what they include just just a look on it. You can see it's very complex and complicated and you don't really know where other different search concepts, what is done. And that is also a thing because, of course when you add search terms because someone says, Oh, I woke up in the night and there's another search show we have to add. What happens is the search gets really big and the search, the project manager tells you, No, yet that's not possible. I can't really have a look at 10,000 hits. So you have to find a way to reduce the hits and by doing that. Before that it was really by chance, what you're doing and you were not really sure if that is really the good way. So we updated the search in 2010 with our new more objective approach and that is what the search looked like at the end after the update. So they are really, you could see less search terms, and it's a more clear search. So that is just an example why and we did that and what we have been doing. Yeah. How did we get there. I take a look back. So we were looking in 2008 for a tool that supports our work. We were looking for a text mining tool. And there were some tools already available. The pub reminder, it's still online. So maybe you've heard about that. We were looking for a tool which has a little bit more functions. And we were having a colleague in our team he was using our for his statistical analysis and he suggested why don't we try out the package team. And we started working with TM. The author is Ingo Finera. And also this tool is still available and it's being updated regularly the last time in November 2020. How we use this tool what we did we use it for. We used it to split the text in separate files. That was one thing we did. We prepared the references for analysis. It was called transformation. And we performed the text analysis itself. Also, we already had the possibility there to compare our references to a population set. So references, we could compare our references to see which one, which terms are overrepresented. And on the next slide, you can see a screenshot from TM in our, and you see this lines of coding. And then on the next slide is basically the output. So we use the frequency table you can see on the opposite side. Starting from that we choose the terms for developing the search strategy. We identified when we use it really some advantages using a package like TM. It was very flexible with almost unlimited functions or tokenize a word stemming on all this and we also saw that there are many options to further develop packages to use the other packages in our and so on, but we did not use it by them because we did not have the knowledge to actually further develop it. We also saw some disadvantages on the other side, it was that there was just no graphic user interface like a shiny app, and we needed training in our environment and in the TM package. And that basically led to some kind of rejection by the information specialist because they did not have this programing skills. And that led, please press the button that led to the situation that we switched to word start another tool in 2011. And that's our current situation. We have been working with word start for the last few years. It's also a very established tool and program. It first released in 1998, it's available in version nine, and the developer is provided research. It's content analysis and text mining software. We basically use word start very similar to TM, but it has this user friendly interface. I show you on the next slide. You can see here you have this different options and you just click click click and then you can work with it much easier than going through this programing lines and try to adjust the code. On the other hand, the output from word start, as you see here is very similar to the one in TM. So it is very, yeah, we almost use it in the same way. So we have the overrepresented free text terms and the keywords we have the TF IDF values set value. So it's very similar to TM. Yeah, so this is the last part of the outlook. So we were looking for an alternative for quite a long time now, because one of the disadvantages of word start is that it's very expensive. And also with each update you have to pay fees again. And the other disadvantage is that word start is not was not really made for what we are using it for. I think most users use it for qualitative evidence to analyze qualitative evidence. So there is no real way to cooperate or to work together with the software or to have different or new functionalities. As I said, it's not free of charge. And because of that there is no cooperation possible with other users. And that is, we found out that is really a limitation for us. And we have no way of have further developments because the software is exactly like it is, and we have no way of changing it. One thing is that in the last years. It is clear that the future lies in a tool that combines several steps together and the words that would not be able to do that. And it has to be integrated into other applications we have. So that is a limitation as well and we are hoping for a tool that have these possibilities. So, our requirements for a new tool would be that it has a graphic user interface so we that we are actually can use it. There are further functionalities and that we can develop the tool. Another thing that was obvious is, and we did not know that 10 years ago, but at the moment it's it's a real big topic is that then should be no barriers for it security. That is a limitation as well. And it should be open source or it should have an easy way to access so we have the possibilities to exchange with our information retrieval community. Yeah, so these are our experiences with our insert strategy development. Thank you.