 Good morning everyone. In my talk I would like to address issue of reproducibility of the published behavioral results and the paper mentioned in the title while it is paper that introduce a software tool for analysis of such behavioral data it also features as examples analysis of actual behavioral data so maybe also a subject of a case study. As you probably have heard there is a growing concern about reproducibility of published results and many factors has been blamed for the inability to replicate published results from scientific misconduct and poor publishing culture to the misuse of statistics and in case of behavioral research there are very often mentioned such factors as lack of standardization of laboratory equipment or interaction with human experimenter for example experiments performed by men in case of mice a bit different results than when experimenter is women also observer bias is very often mentioned and as a countermeasure for such issues automated phenotyping systems like in telecage has been proposed and now is a problem see the pointer okay so in telecage system is a housing cage in which there is a group of animals right here and all tests are performed in the housing cage that way there is no novelty stress as animals are in the familiar environment because they are group house and group tested there is also no isolation stress and what is important part of the system are these four apparatus is in the corners they are called condition chambers or just corners and in every such corner you have two drinkers access to which is controlled with a programmable programmable door every animal is tagged with an RFID transponder so its identity is known when into enters the chamber and then the specific paradigm may be applied for example animal may need to now spoke the door several times or perform certain time sequence of no spokes in order to access a drinker for example you may put in one drinker sweetened water and in the order pure water and then check how fast animals would learn to no spoke to the bottle containing the reward so we have a system which greatly reduces stress by providing by testing animals in familiar environment and in social context the system provides quite good isolation from human observer and as the system records data automatically it really reduces observer bias so what can possibly go wrong as I mentioned you proposed the paradigm in which animals are learned to distinguish between left and right side of the corner such data from the very same experiment the very same data set was given to two PhD students in our institute and they were asked to calculate the same statistics which was fraction of visits which begon with a no spoke to a door behind which corner with sweetened water was as you may see a major part of the data cloud is of the diagonal so some errors has been made it required investigation of every data point in this cloud and verification of it before publishing so once again quite obvious solution to that is automation as a computer program especially non interactive one is a formal description of the performed analysis every step is recorded of such analysis and even if there is some mistake it is documented and may be found it is also much quicker and more precise than analysis performed by a human and most importantly it can be repeated repeated for different data set with a very little effort but there is also the disadvantage which is a great effort which needs to be done if there are any technical details for example of the data format in case of the files which are generated by the interrogate system your data are scattered across multiple files which needs to be to be merged first before you can make any sense of the data in order to get rid of that great destruction during writing or analysis the programs we developed a Pymas library which allows you to the focus on the analysis itself and imagine that you want to perform the analysis that I told you before in Python programming language it requires six steps the first one is the loading the library itself the second is loading data then you need to extract all the visits of the particular mouse then for every visit you need to take the first no spoke then check what side it is to and then you just need to account incidences of left and right sides and that conceptual code translates one to one into six lines of Python code if you use the Pymas library so we have automated system we have a library and the Python programming language which allows us to make our data analysis automatic so would our published research reproducible the problem is that the analysis that obtained results are not reproducible if you don't have your your program we generate the features or if you forgot parameters of the program which led to published results and the program doesn't need to be eaten by a dog but it is also possible that it would get lost like a needle in a haystack when for example after several iterations or several dozens of iterations when you introduce small changes to your code and send results to your experimental colleagues they send you back feedback and you modify your script and and you end up with many versions of the script and then you are asked to repeat some analysis but the information which script generated results that you are required to repeat is lost so what makes our paper really reproducible why do we there claim it to be so the first I would like to distinguish between scholarship and advertisement as was defined by in a paper by bouquet and the paper was about a wave lab and reproducible research and they put a statement that article about computational science in a scientific publication is not the scholarship itself it's merely advertising of the scholarship the actual scholarship is the complete software development environment and the complete set of instructions which generated the features so we decided to go that way and first of all in our paper we put a statement of reproducibility as advised by Pang in which we mentioned all the software which was used to obtain our results with the versions of the software and to this approach there are alternatives for example you can just ship a container of that containing the environment for example Docker and what is more important we decided not to include any results to in our publication by itself but rather we write text of the publication and use the literate programming paradigm which means that we put code snippets which were analyzing the data during compilation of our paper and they were generating the features which by the way a reproduction of behavioral data which analysis was published before in a paper by Alicia Pustian so as the source code of the paper is publicly available then everyone can download it and try to reproduce this figure or any other result that is contained in our paper and moreover they can inspect what kind of analysis has been performed which greatly increases the transparency of such research so I would like to thank you very much for your attention if you would like to read the paper itself it is available at this digital object identifier you may also use QR code that by my celebrity you may reach by using the research resource ID given here and also if you would like to talk to me personally after talks I would be at my poster which is p43 during the lunch break just after I grab some something to eat once again thank you very much thank you for the talk and I will start with the questions okay thank you for the great effort to enhance reproducibility and my question is behavior data is intrinsically very noise noisy so with all your effort let's say you release all the code even my interrogate is commercial available so have you thought about asking other lab in geographically completely different location like us or Japan let them get c57 completely independently and then but use entirely same intel leakage and tell the same code try to reproduce what was published and still would you expect to see the similar results or would you expect to see some different response well that's a very good question the point which I wanted to make is that when you have very the very same data and you obtain different results due to errors then obviously you are doing something wrong but in the case if the different laboratories obtain results from different runs of even the same paradigm in the same apparatus but on different animals or even in the same lab if the same animals are run through a paradigm twice of course I expect that there would be some minor differences between obtained data sets and this is when statistical analysis enters and it tells us whether we may consider such differences as something that has some practical meaning or something that can be blamed on the noise that you mentioned does it response to your question or I just I just want to see how much improvement would you expect to see by doing this way because if there is a built-in like a animal to animal variability lab to wrap variability no matter how much you push the acquisition analysis side of it if there is like c57 used in the America versus c57 used in Europe they separate long time ago and they diverge quite a bit then no matter how you standardize that your acquisition and the data analysis it start from the different animal right so I just want to see how much variability would you expect to see at the animal side as opposed to improved how much degree of improvement by doing your efforts I just want to see some like I see so for sure there was a publication addressing this issue I don't remember at the moment who was the author but if you type in a in a in a browser that the phrase into the maze then you would probably get that publication that was in the title we may check it during the lunch break if you wish