 Hi everybody and welcome to ESMACONF. We're really happy to have you with us. We're just going to close up the conference now. We've got a little bit of a presentation bringing everything together and some viewing statistics as well for those who are interested and we're going to also have a bit of a catch up with our two Hackathon projects which you've heard a little bit about during the week. We're going to speak to those project leaders and see how they got on and then I'm really happy to be joined by most of the organizing team for a little bit of a discussion just to close out. So I presented at the start of the week some of the viewing figures from 2021 so last year's ESMACONF. I just wanted to show them again so we had 514 registered participants last year 39 presentations 10 pounds of questions and four workshops with about 650 unique viewers 175 subscribers and about 3,500 total views during the conference and in the year after we've had a total of 9,000 views and 339 subscribers since last year. It's amazing how much you have engaged with the conference. Thank you so much. We had 28 presentations, eight special sessions and six workshops this year and we've had 842 registered participants so far. You can see the spread of people across time zones that I already showed you and as of last night this was we had 3,375 views so you can see we're already way ahead of last year. We've had 643 viewers over the last few days and 221 subscribers so thank you so much. This just shows the viewing figures for the first two days of the conference for the last two days of the conference so showing that 2,500 people have been engaging over the last two days and you can see there that some of the live streams have been particularly popular and we're really looking forward to seeing how this evolves over the next week or so and you can also see that our workshops have been particularly popular. Alison Bethel's workshop on Monday had 50 people in the Zoom and an additional 450 viewers online which is amazing. You can see that other sessions have been really popular as well so thank you so much for engaging, for watching. We're really happy for those of you who've been able to watch us live and for the people who are watching in catch up. We know that a lot of you watch in catch up and we're really happy that you're able to engage with this in your own time or if you stumbled across it later in the year as well. Welcome to Esmer Comf whenever you are. We also wanted to give a shout out to our so far most popular video. This might change this top position. There'll be some presentations fighting for top place I'm sure but so far from the viewing statistics we have congratulations to Elina Tacola and her presentation on automated research reading which is so far her individual talk has had 60 views and that session was really popular as well but we're looking forward to seeing who might be the most popular video over the year and we'll feed back to you on that at Esmer Comf 2023. So again just to summarize we've had 28 amazing presentations across review processes, graphical user interfaces, quantitative synthesis with NMA, other quantitative synthesis methods, Beijing lens and quantitative synthesis, building an evidence ecosystem for tool design we heard earlier today and just now developing the synthesis community and I really recommend that you watch those live streams all of those sessions are really fascinating if you've not been able to watch them yet. We've also had six amazing workshops, one full day workshop from Wolfgang Wittbar, an introduction to meta analysis in R and a workshop from Alison on searching for studies, one on collaborative coding in GitHub from Mac Granger, a really interesting session introducing systematic review coordinating bodies and the collaboration for environmental evidence by Ruth Garcide, a wonderful workshop on structural equation modeling by Aaron Basu and the really insightful workshop this morning by Martin as an introduction to writing functions and packages in R. Those workshops that are recorded, that's workshops two to six are available in catch up they'll be available all year and if you want to catch Wolfgang you'll have to wait for one of his other many online training events so there's a huge body of training material there that we are really indebted to our presenters for providing and that's a really wonderful resource. We have also had two hackathons, I introduced these at the beginning of the week, Trevor Riley has been leading the team trying to produce site source and Trevor will jump on shortly to talk to us about that and Wolfgang Wittbar and his team have been working on metadata which you heard about a little bit earlier today in the last session. So I just wanted to invite, I'll stop sharing there, I wanted to invite Trevor and Wolfgang back up to the stage just to give us a bit of an update on how you got on and how it went and what you think the future might hold. So maybe I'll pass over to Trevor first. All right awesome so I just want to say thank you to the whole evidence into this hack team. It really has been an incredible experience coming into the coding world as it is from the outside. I was a little nervous and even submitting the initial R package. I thought it was a good idea but you know I'm often in a bubble doing my work so it was really cool to even hear that initial feedback about and the excitement around it. So yeah so I didn't really know what to expect jumping into this and I think there was actually a lot of the same feelings I think from even the coders because well they may have been involved in a hackathon in a previous instance doing it online and again across many hours as a lot of the group is spread out was something new but it was really a fantastic learning experience and again even within the folks who are doing the coding I think that there was a huge difference of views in terms of like how to attack problems and different levels of abilities too so there was a lot of learning within the group even in that initial meeting that we had to get things together and we have put together who stood up an initial shiny and started to populate it with some of the diagrams and built it off of some wireframes so we have an idea of what it's going to look like the functions that are going to be behind it I think a lot of it is still going to be in production but if I can share my screen I do have it pulled up and so again the initial are you able to see fantastic all right so the initial idea was to be able to analyze you know databases based on their coverage when you are reviewing different strings or different strategies and have an idea of the you know best databases in a specific field but a lot of more ideas came out of this in terms of analyzing things so not only do you can you know I think will this tool be able to analyze database versus database we're also talking about methodologies and strategies different strings so a lot of the ideas just began to snowball when we started talking about that so we do have some use cases up on the shiny right now if you have any feedback or want to learn more I'd say reach out to me on Twitter and I'm always happy to talk here's some examples of what some of the data might look like in terms of crossover percentages with databases from search results and this would be that in the number and then again there are some really other cool visualizations that we talked about so this one looks at the number of records per source and then shows the crossover through these dots and also unique records from particular uploads so so there's a lot of information in here I'd say if you if you want to dive into it take a look we'd love feedback and if anybody's interested in working on this tool who wasn't on the initial build definitely welcome welcome more people in and I just can't say it enough about the team it was really just a wonderful working experience and I really look forward to working with folks into the future I don't know what that looks like but I know that the team is really excited about getting this out and I'm sure we'll have more updates and probably a presentation for next year for you there Neil. Really thanks so much Trevor that was really interesting to to hear and I have been involved in in this team a little bit whilst trying to also coordinate things so I didn't have as much time as I would like perhaps but it was really interesting to hear the whole team start off and say you know I'm not really that much of an expert and I don't know that much about this but together I think the team is really strong and brings lots of different things to the table as well it's been so wonderful for me that we've had a tool that was proposed by someone who's not a software developer because I think it's really important to like as Alex said in in the last session I think well maybe the session before it's really important to have tools that are being produced to answer needs as well as ones that are being produced by people for their own need because otherwise there's these gaps and niches that aren't being filled so thank you so much for proposing this idea. I was as you say really excited that my idea of what it might be used for shifted because we spoke to Alison and to Sarah and to you who are on the team who who are the end users so you know the tool that I would have produced on my own wouldn't have done what you needed it to do even though it might have answered a tiny bit and I think that's that's the real strength in the team as you say that's brilliant so in the in the future you said you are happy to have people come forwards and ask to join the team and so this is going to be a project that you're you're taking forwards it's not just going to be finished here it's going to be presented at next year's as my company say that's great awesome. Yeah absolutely I'm dedicated to making sure this is the the package is a successful package and that it can be used in the ways that that we envisioned it and then also I think that there are a lot of unique ways that somebody may be able to take this and apply it to their own research questions. There's always unique use cases for tools like Litsearcher we use it in unique ways like Citation Chase or even using that for metadata completion in unique ways that wasn't really built for so I'm sure there's going to be a lot of great applications. Brilliant thank you so much and Wolfgang over to you do you want to tell us what's been happening with metadata? Yes all right so the idea was to make some improvements to the metadata package there are a couple lessons learned here trying to do a hackathon while at the same time following a conference is a bit tricky so I think having a dedicated hackathon would be beneficial then the other thing is why people were spread out over time zones so that makes it difficult to really get together but we did manage to do a couple things and then I'll be happy to show that so let me share my screen here. So the first thing that I wanted to do is to make a slight improvement to the dot search function right so this is this function that allows you to find data sets maybe you are interested in data sets that are tagged with the term medicine okay oh there are a lot okay but if you wanted to search for multiple search terms you would have to supply like a vector here so let's say you want data sets medicine and arts ratio okay that cuts it down a little bit but this is a bit clunky here having to specify like a vector of strings so now what you can do is you don't have to do this you can just put a comma in between and and that works equally well so that by it just splits it up by comma right or a semicolon or you can put an end there so that's a bit nicer then the other thing is that when you find a bunch of data sets like those that you might want to look at it used to be the case that if you make a selection then it would pull up that data sets and let me just pull up that first data set here so then you start looking at this data set and you say well maybe that's not quite what I wanted but the function would would exit once you selected the data set now it allows you to select another data set so I mean just a small improvement but you know it's a bit nicer this way I would say so just a little bit of an improvement here to to the dot search function we also managed to get two new data sets into metadata so just two more so it's slowly growing um hopefully we'll get to a hundred eventually and then while the the real goal or the primary goal was to build a shiny app that does something along the lines of what that search does and Matthew and Kyle were really sort of the leads on this and they just kept crank cranking out like uh new versions of this sort of prototypes and I just want to sort of demonstrate two of them so um that's the first one one second here all right so basically here you have these fields so these concept terms right each data set is tagged by concept terms and let's say you want to find data sets from the field of education and maybe with standardized mean differences now you see here four data sets right so that's a bit of a nicer way of searching I would say right it looks a bit nicer um the one thing here is that uh data sets might be tagged by multiple fields and now they're getting here comma separated and so this is not quite how we want things to be right that should just be one long list and you select from those um so we have a little bit more work to do here but this is one version and let me stop this and another version works a little bit different um so here you can select from a drop down list but you can also enter like meta regression and then it does tap completion or completes this that's maybe a bit big here let me make the smaller so so then it pulls up these data sets here on the right hand side so so this would be sort of just yeah a different way and of course you can always also in both of these like enter a search term doesn't work right here but it does work in the other one so you can find some data sets through just by entering some search terms up here so that's the idea having sort of an app really basic it doesn't require like something super fancy but does make it a bit nicer to search for data sets I would say all right that's pretty much it that's uh that's amazing that's really uh mind-blowing that you've been able to do that in a week where you say that you haven't had much time that's amazing progress I it's so cool to see something functional come around uh and just light up a package in a way that people who I mean as we've had in a organizing team discussion we can see that you light up when you're using R which is really lovely to watch but it's so nice to see that what you are so good at producing what the whole team is amazing at producing can be made available to so many other people with an interface uh quite well I mean I wouldn't say easily but in uh quite a short period of time we heard about Eliza Grahame's really popular package earlier today litsearcher that has been made so much more widely usable by having this user interface so that's really commendable and an impressive piece of work well done and um so I think we're gonna we're gonna move on now that's amazing work to to hear about how those hackathon projects have have got on if you want to know more about them we'll be putting links at the bottom of the video and in the chat and you can find them at the bottom of the Esmerconf website as well they're listed there you can see their github repos and follow their progress and if you want to know more about how to use a github repo you can check out Matt Granger's workshop um so I'll press on now um just wanted to highlight a few other things um I wanted to really thank the organizing team who are listed here Emily, Kira, Yana, Alex, Chris, Kyle and Matt for their help this week would not have been possible at all without their amazing MCing and moderating and the emotional support amongst the team as well which has been really lovely and is the reason that Esmerconf is I think such a valuable community um so thanks to them and their incredibly hard work um I also wanted to really thank again the people who produced these workshops and can't be overstated just how much effort went into preparing and providing these workshops um you know they're hours and hours days and days of time and they did it all for free so thank you so much um to all of you we really hope that you carry on enjoying those ones that are available for the future um we'd like to curate a database of of workshops and the same with all the presentations as well that will be available afterwards that you can dive into wherever and whenever you are watching them so those resources are incredible we really hope you enjoyed them as much as we did and also I wanted to thank again our funders Code for Science and Society who've been following us on Twitter a little bit it's really nice to see their support as well during the conference but their financial support this year um has been invaluable we've been able to support 27 bursaries for people who otherwise would have found it really difficult to attend so thank you so much for that and also for the transcription of presentations which opens things up and also to our donors who provided individual support we are really thankful if you've enjoyed Esmerconf and you have the ability to support us to keep it free for everyone then pay it forwards if you like by visiting opencollective.com slash Esmerconf and you can see how you might be able to support us there going forwards we will be around in the future this is our second event but we will definitely be running Esmerconf 2023 um so Esmerconf 2023 well what will it be um it will probably be in January or February 2023 the clues in the name we'll have more presentations we'll have more workshops we'll have more hackathons although we might reflect on some of those observations about the hackathons that we we need some safe space and dedicated time outside of the main program so we might think and learn we always want to be improving the event this year is quite different from last year and we really value your feedback in improving and continually growing in how we provide Esmerconf so if you do have feedback please do provide it we have a survey that we'll be sending out where you can provide anonymous feedback um and give us comments of how you think we could be doing more or what you liked as well we always like to hear nice compliments so please feel free to uh to to make us feel that it's worth it um and we will have more people maybe we went up from 514 to 843 um maybe we'll get more um we can cope with more that's the beautiful thing about Esmerconf it's scalable very easily because you are watching um in your time that you've made free for this conference thank you for that you're watching in whatever way works for you um so we're really really thrilled that so many people are engaging so many people are tuning in and also that this is useful to you we want this to be useful um that's the whole point so we uh we're really happy that uh that you're giving us more that we're giving you more we want to see more so Esmerconf 2023 see you there but how can you engage with Esmerconf more generally as I said we've got a survey coming soon if you want to provide any anonymous feedback whether you've got concerns or complaints or something you just want to mention please do feel free to follow this link bit.ly slash esmerconf underscore feedback and the capitalization there is important it's linked to from our website and from the code of conduct if you want to provide feedback then um that would be really great whatever you comment we've already had some feedback so far this week which we are definitely going to implement um it just really helps us grow and helps us serve you better as I said if you have available funding and you want to support other people uh with uh funding that could help them come to the conference when they otherwise might not be able to then please do consider that maybe even consider joining the organizing team you can hit us up on twitter at es hackathon with a dm a direct message or a tweet you can email the es hackathon gmail accounts um at es hackathon at gmail.com or me directly neil hadaway at gmail.com and please do also spread the word this is your conference as much as it is ours um so please tell people about it uh all of the materials from esmerconf 2021 and 2022 will be online fully and forever and always free so we really encourage you to dive into the materials from last year as well there's a whole suite of like 39 seven minute presentations about loads of different packages that are still just as valid today as they were last year so we'll be uh featuring all of the presentations and the workshops and content from every esmerconf uh front and center in the esmerconf website as we redesign it so shortly after uh this presentation you'll see a nice searchable table of all of the presentations the sessions and the workshops so that you can dive into content as in when you want it so uh check out the website join us next year register please send us your abstracts when abstract submission opens um and i just wanted to take some time to thank you thank you so much for coming along um and so if my uh my organizing team colleagues um and the others want to uh join for a little discussion we can maybe close with a brief um chat about what we've particularly enjoyed what our take home messages are wolf gang and trevor if you want to say something as well what's been a real take home for you anybody we we need to thank neil for all of his hard work again this year um always in awe and inspired by your enthusiasm for evidence synthesis and um yeah how much you put into building this community um we really appreciate all the work that you put in neil so thank you thank you so much it's um it's been such a pleasure this week it's it's so nice i think i i really really am reflecting on terry's presentation which i i really really love from tuesday um if you haven't seen it watch it it's it's mind blowing i would say because it the when you stop and realize that you're in a presentation you're in a session with people who are clinical uh psychologists or you know they're from health care there's an ecologist in there um like the diversity in a single session is nothing i've ever really experienced in a conference before and it's it's so cool that you don't even think about it because you're talking about the tools that we use and that bridge of being able to say hey you're in a different discipline but we're doing things kind of similarly should we learn from each other and share resources um that's that's really cool for me i don't know if anybody else has got any take out take home messages what what really stuck in your mind i think the the hackathons um i think i think um wolfgangs may be right in that perhaps we run them just before or just after the next esmer comf but i think it's been really valuable to bring people together with a kind of common goal and i know i haven't had as much time as i'd have liked to have contributed to the hackathons this week but it's certainly something i'll be contributing to going forward and i think it's it's really valuable to kind of have that space for someone who's got a bit of an idea for a tool um or perhaps something conceptualized and they want to make some improvements and being able to really bring people together to drive that is is just brilliant and what i've loved about um sort of our community with with with site sources kind of the diversity in in membership again a bit like like you were saying with terry's um talk the the fact that you don't really think about the fact that you know someone's in health care someone's in psychology someone's in um ecology um it's just it's all about the tools and that's that's valuable yeah that's a really good point it's um it's really really stimulating and um yeah it's just great to have discussions with people that you otherwise wouldn't me well i just wanted to um reflect on the sense of community and belonging across the conference and i also want to thank you because i've been behind the scenes and i've seen how you know the the equity diversity and inclusion is so high on your agenda and i think it really shows um you know as a woman and being interested in these areas you've really made me feel welcome we've seen that diversity across and i just want to really congratulate you on that thank you very much that that really means a lot try and hold it together someone else say something now i think for me that that's really important i'm um i'm honored to be a research associate at the africa centre for evidence and um from the interactions i've had with amazing colleagues across africa and the the work that i've done i've been very lucky to do with the global evidence synthesis initiative that was based in or is based in beirut um really highlighted for me the conditions and the the context that people are working in and just different and if you don't bring that diversity into teams you don't get the diversity and and the usability in the product so um i'm really i'm really happy that that shows because it i think it should be a priority for all of us and i i think um one of the most important things is is feeling that you do belong um and i want really everybody we want everybody who's uh taking part in asma comp to feel that they have a place and that's something also um that was highlighted in uh in one of our sessions as well about how as a community uh of tool uh you know tool users tool developers those interested in tools how we can also make sure you know we're always interested in new tech and what's the latest thing that we can do and you know cloud pipelines and blah blah but also thinking about how we can make our tools as accessible um as possible and to people all over yeah that was it was really interesting to to be watching that session i was really excited i was making loads of uh loads of uh notes about questions and things to to ask i thought that was really valuable that um yeah i mean just as an example the the cloud-based reviewing management tools are great but if you don't have constant access to the internet you can't use them so um i think your point about you know these shiny apps shiny apps are cloud-based but you can also run them locally which is great so you can go to get a repo you can clone it you can run the the shiny in your own system um and i think maybe there's more that we can do for lowering the bar in being able to run a shiny app locally um so that you can actually run it like a normal package without having to open up um would be would be cool maybe that's something that is we can look into but um yeah i think that's a really great point i want to say kind of while we're on that um the topic of kind of accessibility i think the fact that we've been able to to get funding from um i can't remember the organization name because my mind's just gone blank but um to provide those those bursaries um and i think you know some of the the stories behind you know the bursaries that we were able to get out really demonstrates the value in not only these being free and available for everyone but also um that we're actively trying to get people who otherwise wouldn't be able to join to to be able to to come along to learn and and hopefully in the future to also contribute as well yeah that's a really great point you bring people in who otherwise wouldn't be able to and they will be able to take part and and take over i think that's um yeah that's really great awesome oh who was that emily yeah sorry um i i i wanted to reflect a little bit on i'm kind of going back to terry's presentation with the opening just about um the creativity that's required in this kind of work and and this group in particular um it was just so wonderful to see in all the presentations the presenters kind of talking about where what the problem was and what they did creatively to to address it in a way where people could could learn from that experience and say okay this is how i could be creative about this other problem that i'm facing and learn from this this group so to me even though they were presentations um there was a real sense of kind of pushing people to think creatively about their problems and you know let's like think about a way to move forward with this as a community and solve these together which is just a spirit that i've always loved with the hackathons and the conferences and um i'm going to be delighted to be a part of it next year so brilliant thank you so much i think that's a really good note to end on then um thank you all for uh that really interesting discussion we've had loads of discussions this week uh in the organizing team we're going to carry on with them uh we might have a little bit of a pause of like a couple of days before we start organizing their work up 2023 but really do reach out to us if you'd like to be involved in the organization um it's one of my favorite things to do um so thank you all for coming um it's been an absolute pleasure um and i look forward to seeing you all at Esmeralcom for 2023 goodbye from us