 Hi, thank you, Michael. Can you hear me? Okay. Yes. Thanks. Okay, wonderful. Well, thank you It's a pleasure to be here with everybody. My name is Neil Batra I'm the president of the non-profit organization applied epi and one of the co-founders of the epi our handbook project Which some of you may have heard about and I'd like to talk to you today at the our medicine conference about Are in the public health side of medicine in the and and more specifically in the applied epidemiology side of epidemiology, so I'll be talking about some of the innovations that our group has been doing the wide community that we've Built over the last year and a half Some of the tools that are available to you for learning are and for using are And I think you'll appreciate and and I'm I'd be excited to talk with many of you about where this can go in terms of getting Are more widely accepted in this space, so I'll go ahead to my next slide This is the our medicine conference, but I think it's still worthwhile Just describing in very crude terms Epidemiology just the different spheres in epidemiology because it it actually has an impact on the work that we've done as applied epi And that is just to say that generally speaking we have academic epidemiology where perhaps it could be said the Primary objective is to advance the knowledge base this work often involves complex mathematical modeling And in this space, I would say are is pretty widely accepted and resources are available many of which have been displayed at this conference Now the space where applied epi works is in the partner side of this where it's about the practice of Epidemiology to actually implement disease control and in this in this space The objective is usually operational insight for disease control This often takes place in emergency settings where time is short resources are short people are stressed and You know producing a situation report is is crucial and it has to be done fast or something like this The analyses are mostly descriptive. They're usually very simple analysis tables plots Descriptive maps simple dashboards this kind of thing But in this space Our adoption has been hindered by a lack of training that's relatable to the kinds of people And their backgrounds that work in this side of epidemiology It's not widely adopted yet. Although the momentum is there And I'm speaking from from a perspective of somebody who's been working in epidemiology for over 10 years And I've been in this applied space the whole time. So I've worked For organizations like local county health departments in the united states With uh, who and doctors without borders internationally And you know, this is kind of the the world I inhabit So apply that be focuses on that latter part It's the frontline public health that we really want to strengthen and although What we do is not restricted to r r is a large part of what we are doing right now So that's the tools that we're building. So we're talking about our packages But largely the biggest tool we felt we've built is the epi r handbook, which I'll be talking about later Training that we think is relevant to field practice that really meets these field Practitioners where they're at in terms of their comfort with coding and with the content and making that transition accelerated Um, the support that's needed beyond a one-off training or beyond the tools that the need to build this ecosystem of support to really escort people In their journey into maybe what might be first time programming for many people And as they as they sort of explore the depths of the of the r community The the vastness of it and then on top of all that we want to leverage the transition of public health to r To take an opportunity and make the methods the science behind public health and sort of interventional applied epidemiology Make that more robust because I think that's that's a key part of epidemiology. It's not just about coding. It's also about all of the The science and methods behind what we do So who are we? Let me just take a minute and describe a picture of our group here So at this point we have about 170 people who are contributing to our our various projects At this point, we're mostly away from using volunteers and and actually have paid consultants at this point And they're scattered across the world in over 40 countries The thing that really defines us and sets us apart from say a group that's based in an academic university Is that we are built of practitioners So people like I said who are working at district or country or county level health agencies or NGOs And so it's that extensive front line public health experience that we're bringing to the table And that really comes across in the training materials we build Whether it's the epi our handbook or our courses or our tutorials So we really are a movement for epi's and by epi's The group consists of our experts some of the people on our team are building the cutting cutting edge our packages such as You know transmission chain packages modeling packages Etc, but we're also made up of learners because that's really the vast majority of the public health audience We're trying to serve Many people on our team are data science educators and we have many languages represented We've strengthened many partnerships over the past year. So whether it's the global outbreak alert and response network linked with isaric The uscdc we have a very strong relationship with the field epidemiology training programs around the world the fetps And teffy net which is a major funding source for those and many others And doctors without borders has also been a strong supporter of our of our program And we can actually put some numbers to this, you know However crudely that the need for our training in this sphere is very clear So we recently did a survey with teffy net that group I said that supports field epidemiologists training programs and in the survey Low-income field field epidemiologists in low-income countries overwhelmingly said they felt constrained by their current tools Right. This is an astounding number and that these were primarily excel spss Epi info either point and click tools are still broadly being used around the world in applied epidemiology and public health 90 90 percent said that our training should be a high priority So they recognize that r is the way for them for their careers and to make their work more efficient and better But only 12 percent said that their agency had the sufficient capacity to actually train them in r So that's the problem. They recognize that it's the gold standard. They want it But there's a there's a training gap there and the impact can be huge Now I don't need to talk too much about why r is being adopted now in public health It's um, I mean this audience knows it's it's free the advanced capabilities are incredible specially visualizations Those who are learning are coming in at a time when it's easier to learn than ever with tidyverse and r studio People like me who learned it 10 years ago. Wow, that was a it was a different story It was a little more difficult to pick up As a novice The fact that it's community driven and the collaboration benefits are huge So a few examples in public health the r markdown feature as you know Is is really the a game changer for public health for for folks who are used to Editing microsoft word documents in a harried manner right during an outbreak the idea of in this case For example producing over 300 COVID situation reports in a day And doing that every day at the beginning of COVID-19 is Was was was so powerful right and this is why many jurisdictions are switching to r right now I was working in Haiti last September during the earthquake response there And I I actually met a Haitian epidemiologist there who It's much to my pleasure said I've been using your epidemiologist r handbook for three months teaching myself r and I've I've transformed my workflow From collecting the data and well not collecting but you know receiving the data Processing it cleaning it and producing reports that used to take me 80 percent of my week Over 30 hours a week and he had turned that into 30 minutes or so right and and I think all of us know How that happens with r but for him this was a dramatic change It allowed him to think big picture about the health systems and ways to actually improve instead of always playing ketchup And cleaning data in excel And he also felt that he trusted his analysis more He trusted his analysis more because he said I I know when there's a mistake I know when there's an error because r will tell me So I think the the the other thing I want to dwell on before moving into what we do is that As you know r is not just another tool coming in it's a culture And this is a culture shift for a for a discipline that's used to working with sass and stata corporate software To do these kinds of analyses and this is inherently democratizing It's decentralizing innovation and ultimately it's going to empower Epis from the global south and I think that really drives our organization So what have we done? This is a little diagram we've made that sort of pictures all of the different pieces we're working on for r There are other pieces we work are working on that do not have to do with r And let's just walk through this piece by piece So the first item that really launched our organization was the epidemiologist r handbook Now this is a book down It was launched in may of 2021 And I think within 24 hours it had been viewed 11,000 times. So it really went way farther than we expected It's now been used by 230,000 people. It's coming in at about 1400 hits a day And we're just really really pleased with how many people this has helped What we thought would originally help being being help maybe you know a couple hundred friends And it really just kind of took off It's available online at the website on the slide. So epi r handbook.com But we know many people are also using it offline in settings with very poor internet connectivity And I'll get to the purpose of that later It's 50 chapters in the book down. I think if you take the book and you print it out in microsoft word It's over 1200 pages long. So it's it's quite a large resource But it's written to be easy to read. It's got sample code Basically for the tasks that at these do every day, whether it's making certain visualizations doing our markdown grouping data deduplicating data And I think it's written in a very sort of Practical way for those of you who have at least a little bit of our experience We now have translations going into 11 languages And this has really expanded our network all over the world The vietnamese translation is is is up and live and the french is coming soon as is the turkish and many others This was built by a group of epidemiologists who essentially were volunteers And there were people probably 120 people from around the world who all contributed into this book And I think that's really special and it's something that makes sense within the r community And it's something that I think the r and open source spirit really fosters So the epi r handbook Many of you have used book downs before so I won't I'll go through these slides very quickly But in case you haven't seen it you have sample code You have narrative text essentially tutorials basically and walking through the outputs Some of the chapters in the book these are little graphics that are at the top of each chapter So if you're transitioning from excel or from sass or from stata We have content especially for you all of the importing and exporting you could ever imagine whether you're working with apis or google sheets Or little tips like code to to automatically import the latest csv file that arrives in a folder right very very crucial for those of us who are receiving data every day And we need to make that report every day And a little bit about the basics as well of r like r projects and all of this We're using here and and rio and these kind of things that many people are who still peruse the internet They still don't know about some of these more recent developments that make coding much easier Cleaning data is the bread and butter of an epi's work And so we really we focus on cleaning data how to apply those core tidy verse functions To those crucial things like working with dates the the intricacies of deduplicating And grouping data with strings and all of this So this is things you also find and say the r for data science book But I think the lens here is very much driven by the tasks that we know that applied epidemiologists are asked to do On a day-to-day basis and so that's the framing and that's the scenarios and that's the code that we give And so also plots I mean we're that's communication of information is what we do right and so whether it's gis basics Some more analysis like survival analysis or we have a very robust chapter on time series and outbreak detection And we've been connecting with the the group at cdc that does syndromic surveillance to to revise this chapter even And like I mentioned epidemic modeling How to handle missing data and all of these things Tables just presenting simple descriptive tables remember I said that's that's really what most of this is all about But also epidemic curves, which as epi's were very particular about how these are shown And and so there's a lot of code in there about how to make them exactly right You know diagrams transmission chains for outbreaks or phylogenetic trees They're all in here. They link to other resources as well But the idea is it's enough to get you started if this is all you have Reports, um, I think some some gems in here are the the report factory package Which helps automatically categorize our markdowns that are produced on a regular basis our get chapter We're quite proud about and even this chapter 49 How do how do you troubleshoot using r on a workplace computer that deals with one drive or that deals with Other kinds of firewalls. So this kind of thing. Uh, we really wanted to get out there get our experiences out there to the world Um, some of the challenges that we had making this book Well, we did it in a pandemic and it turns out in a pandemic epidemiologists don't have a lot of time Um, so that was a challenge, but we did want to keep independence from major institutions So we were an NGO a nonprofit and that really made us more agile and fast to produce this We had to think about our audience and say Is this really for a true r beginner and the answer is no the handbook It assumes you have some basic experience with r And we'll get to how we address the those novice true novices later We made a lot of editorial decisions That we detailed in a chapter called editorial decisions About which which packages we chose to to highlight and how we decided to show different solutions to for example Making an epi curve. Did we choose wrapper packages? Did we go with gg plot? Did we do both? Um, it was a massive bookdown. Uh, and that was a challenge just to get it rendered and it's still a challenge with the translations today So that's the FBR handbook. I encourage you to check it out and like I said, there's a gap there though It's not for the novice. Um And so that's why we built this tier of of material and there's two components to this The first is our live course our gold standard And I'll get to I'll describe it in a moment and the second is these free self-paced Learn r tutorials that any of you have probably used before something like them the course itself Is 35 hours long? Um, so it's it's basically a week And we try and put as much of the the really important our basics in there as possible focusing on data cleaning on Making plots and making reports because that's really what most people want to learn how to do quickly It's all public health case studies. Uh, it's live coding demonstrations The the the feedback we've gotten in this course is is fantastic and it's always being improved I won't dwell on it more than that just to say this is out here If this is the course your organization is looking for then, uh, send us an email Um, we've supported at this point a lot of health agencies and just in the last year And we have we're running trainings essentially every day for the remainder of the year Um, and so I think that just speaks to the demand among the public health space for this kind of thing Um our educational approach Again, it's about delivering the content that we know as frontline practitioners We're very intentional about the vocabulary we use, right? You know things that can be very quickly alienating to someone who's never been in this computer sciencey world Even just using the word string for example. Well, people might think of their shoelaces So how do we how do we introduce that language in a way that doesn't make them feel? You know less lesser Building that confidence to tinker tinkering experimentation that's built into the curriculum I think that's really important And we actually spend the time to help them use our we dedicate Up to an hour per person to help them troubleshoot their installations their one drive Sink problems, etc Which I think is much more important than putting them in a cloud environment That's very sterile because we want them to actually be able to use our We do have these five free tutorials online that cover much of the same content because we know that our live courses won't reach everyone And those are accessible at our website and it's just a login a free login that you can use to to get into those tutorials We do have some advanced courses in case studies. So these are advanced courses Whether you're looking to do GIS dashboards You know survey analysis or learn statistical modeling in the in our these are also available Although quite frankly, we've been overwhelmed with the amount of introductory courses that have been in demand The idea with case studies is participants would ask, where do I go to get extra practice and Well, so we said, well, let's go work with all of the different health agencies We are affiliated with around the world and gather the free case studies and put them in one place So this is a work in progress, but we have several up there now and I'm sure we'll be adding more in the next few months We do make some packages, but really it's not our priority We make a sit rep package, which was originally made by a number of different partners and msf is This is really for msf the doctors without borders But the our markdown templates and some of the helper functions that are there are very applicable So the original makers of that recon msf and then we've taken over the management and maintenance of this The support piece I mentioned this we all know that training is not adequate It may be required, but it's not adequate and we've addressed this in two pieces One is the applied epi community. So this is a discourse forum, which is different than discord But essentially it's similar to what you would find in stack exchange But the idea here is that it's much more friendly much more welcoming of mistakes And it's really embracing the public health context behind questions and not just stripping it down to coding alone Because quite frankly many people myself included are intimidated to post on a forum like stack exchange Whether we've made our reproducible example exactly right It can be really scary, especially for people where this is all very new And so we have topic areas in here about our code, but also about epi methods other software math modeling this kind of thing The community you can post a question You can get answers within a day or two from several people Around the world the community builds it and again you can read it for free or join the discussion with a login What we're piloting now is the extension of this and the goal here is ambitious The goal is that like we have 30 40 instructors now around the world that are teaching classes We also have a Co a cadre of technicians who are going to man on our public health desk So this is something we're piloting with doctors without borders and with a number of these field epi training programs right now And we hope to roll it out within a few months But this is the idea that Agencies can subscribe and we'd have discounts for low income settings. And this is where you can call and get a An answer whether it's oh, I can't figure out and oh, it's just a comma that you misplaced You know or maybe it's can you help me build my data pipeline because I'm responding to an ebola outbreak And I've just learned r and I'm not really sure how to do this So we've been working with who and talking about how we can support Those kinds of responses which many people on our team have already supported But this this kind of help desk feature would be really helpful in those kinds of settings So that's something that's in the works. We're piloting it like I said and we hope to roll it out soon So if anyone's interested in helping us build this or supporting it You could have a massive impact if you help this if you help us bring this into reality So in the interest of time, I want to just bring some parting thoughts here Firstly public health is a discipline that In most of the world is relying on sub optimal technology and I think we've all seen in the past two and a half years How problematic that is That a discipline that's been chronically underfunded and really is a part of the infrastructure of society for emergency response and for just general functioning We need to make this an easy transition The moving from point and click to a scripted reproducible analysis is a huge step for people who are have not been Embedded in computers their entire life. And so I think we need to discuss How do we make our space as our users more friendly? to people like this to practitioners and make sure that this Transition like this is actually led by practitioners And mostly I see led by academic folk and led by You know, and I think that's fine, but it needs there needs to be a big seat at the table for the end users And I'll wrap up here in a moment The friendly communities of practice. I think is important Relevant training materials. So, you know, most public health epi's are not doing modeling. They're doing very basic descriptive epi I think linkages between developers and front-end users is extremely important so that the tools made are actually useful And that I think it needs more funding because ultimately we need our champions at these health agencies to be Institutional champions and that leads to train the trainer campaigns that we're trying to do And I wanted to just say that the translation teams have been for us a wonderful thing hundreds of people involved in translating our work And that's really expanded our global reach And lastly our website if you're interested in applying to be an instructor or some kind of technical contributor otherwise We do have an open call for that Here's our email address our website and the various twitters and things like that And I will leave it with that. So Michael over to you. I don't know how much time we have Oh, we have we do have a little bit of times. Um, so I'm gonna uh, there was one question Or at least one question. All right So, um, there's a question. Do you teach data collection best practices and software like, uh, red cap for example So the quick answer is not yet. Um, we what we're building out our our curriculum right now But as I mentioned our organization, we called it applied epi because Like it or not someday in the future R might not be the tool of choice anymore Um, and so teaching all everything around it is important to us everything from basic epi methods how to do surveys how to do sampling how to You know what even things like leadership in emergencies and how field epidemiology works in practice setting up surveillance systems And that will involve data collection practices. Um, and so those are curricula that we are Right now working on we're actually working on a companion manual to the epi our handbook right now That will be focused on epi methods and really a practical Um, implementable, you know down to earth manual of epi methods as applied in mostly low-income settings And tightly linked to the r-handbook. Um, so we're calling it Tentatively the epi methods manual, but the idea is that we're we already have many partners who are interested in contributing to this Um, and we're going to start writing it in the same way we did the epi our handbook And I hope you'll see it sometime next year But I think that also gets to this collection about a question about data collection. Sure. All right. Thanks Neil I see a question from peter about do you have companion learn our apps for the chapters? um I would say we we have learn our tutorials So those are the five tutorials available on our website apply to epi.org slash tutorial They don't exactly correspond with the handbook chapters because well, there are 50 handbook chapters and only five tutorials So that tells you something right there But uh What we are looking at doing as we revise the handbook this fall With an update is making it much more interactive so you can sort of jump into a learn our Environment and practice the coding with the data sets that you see in the tutorial Making that much easier to do So that's on the way. Thanks for the question Let's see Another question about data collection best practices um Let's see is red cap often available. I would say yes. Yeah, a lot of people do use red cap Who we work with? And I there was another question in the chat. Do you think r should be taught in medical school? Lawrence, um Well, having never gone to medical school. I'm not sure I'm the right person to answer that You know, I studied epidemiology, but I'm not a medical doctor You know, I think Yes, I think it probably should but I'm definitely not the most qualified person and I think the thing what we teach is the The application to public health disease control and there are many physicians who end up in leadership positions in public health disease control Now whether or not they end up doing the coding themselves It's probably important that they have an understanding of what a tool like r can do Um, and I think that's so that you know alone would be useful. Maybe it's not an intensive course, but maybe it's an exposure So they can understand how scripted languages like r work and what they can do in public health and what they can't do Um, great. I'm seeing from beth that many residents are coming in knowing r. That's fantastic I think the the the critical piece there though is if they're going to go work in public health They shouldn't have only received training in r for academic modeling and research Then that's missed in so many Academic curricula is the the data cleaning that you that you don't get because you're given a very standardized Dataset to work on which is very understandable in an undergraduate class or even a master's class But when they step into their first job in a public health department Whoa, that data is not going to be so clean Um, and they need to know how to do that and so that's why I think we find so many people looking at the fbr handbook Is that's one reason Thanks, joe, uh, wonderful to hear about that call for proposals