 Welcome to USAID 2021. We are truly happy to connect to all of you globally. This meeting is live stream. We are very grateful to all our sponsors who make this conference possible. Today's sponsor is RStudio. Hello to introduce myself. I am Aditya Patia, Geospatial Data Analyst at Inflabs in India. Today we have Meenakshi as the keynote speaker for RM Action. Meenakshi is an Air Quality and Environmental Health Researcher from India currently living in Taiwan. She works on projects based in India and the US. In 2017, Meenakshi co-founded the Women's Own Consulting Organization Inflabs with a focus on providing reliable data for environmental health projects and education focused on global grant challenges. She is passionate about reproducing research and enjoys teaching art and stats. Meenakshi earned an MPH in Environmental Health from the University of Washington, Seattle in 2015 and an MS in Biomedical Engineering from the University of Alabama at Birmingham in 2009. Her work on pollution in the produce was recently featured in the New York Times. Before we start requesting all of these to ask questions and upload questions in the Q&A feature here, welcome Meenakshi over to you. Thank you so much and hello everyone who is here. Thank you so much for joining. I'm really grateful to use her for this amazing opportunity. Today I'm going to talk about how we use our to answer questions about air quality and much of the work that I will present today is led by Aditi. We have just been very fortunate to get a chance to work with her on these projects. So yeah, I'll get started. So we are a small team interdisciplinary team with very different backgrounds from GIS, atmospheric chemistry, environmental health, basic science, and we have all come together to work on air quality projects. The reason I named our talks today. Can we do this in our is because we all we've all had very different our journeys and we are all in very different places are team in very different places. We have had no like many our users we have had no formal training and our and we have learned it on the job. And so every time we have a new computational question we have a new visual visualization challenge. We ask, we ask this because we have not done this before, can we do this in our and for the past four years that we have worked in our and on these projects the answer has always been like a resounding yes, yes we can do this and we can do more. So this talk is about that what just a sample of what all we do in our and yeah. So, to specifically what we do in at it lapses, we measure air quality to answer questions like is one block dirtier than other is one season different than other in terms of concentrations. To low cost instruments or consumer grade instruments do they answer questions reliably when compared to more expensive reference grade gold standard instruments, and if they don't can we apply statistical corrections to make their measurements more reliable or more informative. We adapt cutting edge methods from different context mostly developed country settings to low or middle income resource settings like India, when it comes to monitoring, and we also analyze national level household surveys on energy use indoor air pollution, and we work on building models for pollutants using different sources of data with satellite and ground data. So I'll spend the next few minutes in contextualizing, giving a background on how we collect data that we then analyze and are so it will be there with me for some air quality primer. When it comes to measurements, there are two main types of air quality measurements. One is stationary monitoring. So these are two duplicate high end sensors installed on a rooftop. And this is typical on how of how air quality stationary monitors are placed they measure your ambient air and give the concentration at that particular site. What is becoming more and more popular with time now is mobile monitoring, where sensors are placed in a car, and then or any moving platform, and then they are used to map pollution at street level. So what you see here is a is a Google Street View car. So these Google Street View cars are the same cars that Google uses for maps for mapping your street and some in some countries some of a few cars from the fleet have these really sophisticated air quality instruments fitted on them so that while the street is being mapped air quality on those streets are also being measured. In India, Google Street View cars are not allowed for privacy and security reasons. So we have adapted this technique working with the same team for our Indian settings. So here's an example of what a stationary monitoring network looks like. So this is a map of Bangalore where we work which is in South India, a city in South India, and all these circles, the each circle is one stationary monitor and the color and the number gives a concentration of a particular pollutant there. So this is what a network looks like. Many sensors distributed all over the city. And if you have enough sensors in a particular region, then it can give a city scale or a macro scale idea of what the pollution like that there is. So this is one of our stationary monitoring projects. This is the schematic of our mobile monitoring project. So what goes in our, the car that carries the sensors. So we have now we're coming to these different data sources that we have to combine together to analyze. So they're different groups of devices, we have portable instruments for air quality, different each instrument is for a particle or a particular gas. So we are in the air. And since we are moving on a mobile platform we are calculating we are collecting spatial data and that spatial information is very important we have a GPS for location information. We also use probes for weather conditions stood so to measure humidity in the air because that impacts our readings that impacts the readings when you're measuring particles, especially. So these are the different instruments that go into the mobile platform and one of the main products is building a street level pollution map like this on the right, which is similar to your Google traffic maps except the color here. Instead of traffic it denotes the level of the pollutant so the darker street means they are more polluted than the lighter or the blue streets. So what the setup means is that are each each of these instruments have their own data frame like they have their own data download method and from the point of view and analysis a different data frame in a different format with the time stamps in different format and such layers of complexity complexities that we get to analyze them. So this is the inside of our moving our platforms it's less, less fancy than a Google Street View car, but what we what we have done here is used to remove the backseat of a car fitted with these metal shelves and all the instruments that I showed in the previous slide. They are all secured in this blue bin. And they have inlets coming out of them, which measure the air that measure the air and the particles inside. And the laptop laptop here is used as a logger because all instruments do not log data within instruments so they need additional laptop. So this is the, this is our data collection setup and this is how we collect the data that we later analyze in our. This is what it looks like in practice we take this car out on the road and make repeat measurements several times, but sometimes it looks like this. So this is an image of one of our roads where we had planned to take the car for measurement. And there is a store block as you can see the road is dug up it was. It's typical it's a very common site in many Indian cities, it's unplanned unannounced, and we don't know when we would be able to go back on the road so it is indefinite. So, the all that to say is if you have collected field data you know that it is complicated there are many things that are out of control. It can be chaotic. And, but, once we have surmounted those and finally painstakingly collected the field data using our in the next step for data analysis really allows us allows us real control and the flexibility that our provides is just is very, very empowering. So now from here I will spend I will show a few slides on packages are packages that we use specific packages that we use and what do we use them for. So, we all love tidy words. So for all our data cleaning tasks, we use many tidy words packages and for plotting as well. We have, as you saw the different sources of data in the mobile monitoring platform, they all have timestamps. And so it's very useful to use reader and read underscore CSV functions to read, read those in because it automatically parses the time data time objects so you don't have to specify separately that this is not a character and it's a daytime column so it's very super useful ggplot for all our that's our first go to for all our visualization plotting tasks. And then we're always working with a lot of big data we are collecting high frequency data per second or per minute for many, many, many days and so using per and map functions instead of the for loop whenever we can makes the analysis more efficient makes our code faster. So that's very helpful. And beyond tidy words there are some packages that we have. We find particularly useful for our analysis. So first is consistent type and stamps using lucidate. So we have, we have our data from different devices. And our first step is to match them together to join the data sets together based on the timestamps so the location data with the pollutant with the pollutant data and with the weather data. Now, for this, the timestamps need to be consistent so they can be matched. So the timestamps could be of different formats that just based on device manufacturer preferences. Sometimes the timestamps are in different time zones and that's because that depends on sometimes where a machine where the instrument is manufactured, or some manufacturers prefer just standard time. So it is different from the Indian standard time that we would be using or would be counting on for analysis. So for all of this lucidate the function functions in lucidate are really, really intuitive. For us, for example, parsing date times with the YMD underscore HMS set of commands. If you have not used this before, it's year month day underscore hour minute seconds and there are several all permutations and combinations of this. Again, you can be used and they are quite intuitive because this order represents what your input is in assigning time zone with underscore TZ is also super helpful so we use lucidate all the time to for timestamp manipulation before we join our data sets. Next is something that makes our life very easy is using the janitor package for cleaning up column names. So this is the I have a before and after here. So these are column names of a data set before, before using the package. So this is a this is again an air quality data set that we received. And this is elevation, you can see elevation month season, these column names are really descriptive which is great. So these are not bad column names they're very descriptive and from this point of view of English like they're understandable. But when you start typing in these as variable names or column names you see some inconsistencies. For example, there is dash, there is dash and a gap here. Then there are capital letters and small letters and then there is again gaps again gaps in dash and then many column names starting with digits instead of alphabets. So, I can load the janitor library, and then just ask to clean names. And, and this is, this is what happens after that. So, by default, it will clean up all the names as in convert them consistently to snake case so all lower cases separated by underscore, all the gaps are gone the dashes are gone they are all replaced any separation is replaced with underscore so this is more consistent and when you're calling each column it's just more systematic than to include in analysis. Next, we use plotly a lot for to make interactive plots. So ggplot is the default here I have an example of ggplot and then what happens if we use plotly after that. So I have a simple ggplot time series object it's called plot one here. I'm sorry about very fine font but stay with me the y axis here is time it's sorry x axis here is time it's a time series and y axis is pollutant concentration so this is what we get is a useful plot a static plot from ggplot. And if I use plotly so I've just called library plotly and then wrapped the my ggplot object plot one in ggplotly, which is a function so that sticks in a ggplot object and then converts into interactive plot so now look it's the same plot it's the same information. Now I can scan I can click on points and it shows the date and BC so it shows more information on that. There's a lot you can do. I can zoom in. I can pan around. So suddenly I have this very nice interactive plot with just one command, and it's very useful if you're working with again high frequency data big data and you want to do more with a plot. Next set of interactive plotting is making interactive maps with the leaflet package. So let me just do a quick demo of that. Yes. So this is the. This is a sample map of our study area. It's a background of open street maps and on this there is a leaflet map overlay, and the colors show pollutant concentration and if I click here, I can click on each point and see what the what the attributes are there. So I have one pollution pollution at 72 I have 55% humidity, and then I have this particle concentration. So this is, again, better than a static map. I can, I can pan, go to a different area can zoom zoom look at this. This is just just so much more increased functionality using the leaflet map. So we also use our for statistical modeling. There are many things we can just do with base our linear multi linear regressions with the function logic models when you have categorical or binary outputs with GLM PCA principle component analysis using PR calm. There are many options for linear mixed effect models we use the LMA for package more frequent, more frequently, and then the generate general additive models or gap models using the gap package. We have not started exploring tidy models yet but that's that's on our on our list. Since we're talking about air quality. There is this one package which is really cool and very helpful called open air by car slow and rockins and it's for air quality specific data there is some really cool visualizations that you can do using that so this one that we make very often. This is called a calendar plot as you can see this is a calendar from July 2019 to February 2020 and for each day. Each box is a single day and we can see this is the color signifies the pollutant concentration and the arrow here is for the wind direction so you can see the wind direction and magnitude from the size of the arrow. So this is just like this one single plot is doing a lot of work and it's just really amazing and very informative to use. So if you work in an air quality and if you haven't checked out yet, definitely check out the open air package. So you may you may have figured or imagine that in our in this work we do a lot of there are a lot of repeat data analysis tasks. For example, for mobile monitoring we take a car in a neighborhood and we are. We map a neighborhood several days for do many repeat measurements so that we have all. We have accommodated for all the variations in that neighborhood. So it's one neighborhood measured several on several different days many different times, then we go from one neighborhood to another and then measure that several different times. And so these repeat measurements so that these daily measurements that we take and the data analysis tasks are pretty much similar for from one day to another. Make sure the timestamps are consistent, join the data, plot the data to see if what happened that day whether any anomalies to quality assurance quality checks. So to simplify these repeat tasks we have developed some shiny apps. One is for the mobile monitoring so the data from multiple monitors. And what this looks like is on the left you have the option of uploading your data from different sources so for the location data the polluting data, the weather data. And once you do that the app will automatically generate a joint file and not only that but also a statistical summary and time series plots maps. So this really reduces the computational time from many hours per day to a few clicks per minute so this is here is a link for that it's an MMAQ shiny. Another app that we have developed for stationary data. So this is called pollute check and it's to facilitate analyzing open source air quality data from India and from many other from other global sources as well. So once again you can upload the data and specify data cleaning parameters. And it will generate a summary statistical plots linear regressions and many other and we have also integrated open air so the calendar plots. And generate those using the open air functional functionalities through this package as well. So if you have in your area you have open air quality data sources take this for a spin and you know definitely give us feedback. And if you want to learn more please attend Aditi's talk elevator pitch which is in a few hours and yeah ask more questions on that and give us feedback. Okay, so from data analysis I just I want to switch gears and talk about using our as a workflow tool to organize and share our work. So the first thing and maybe you already do this but before we used are we just like organizing our work in our separate folders on our computers and everyone had a different system but using creating an R studio project for each new project and just each new bundle of analysis has been super helpful. So that's how we start every project and now they're organized like that and using GitHub for version control. And then initiating the GitHub with a read me file, which in the beginning it contains a project description file name description so what are the file naming conventions every project may have different naming convention based on the dates on the region on the purpose and other data so once after any like initializing we keep the read me file continues to evolve as our analysis involves and as we completed so using the R projects version control plus read me it's really helped streamline our organization. And then reporting with our markdown. So that has been a really game changer I think in terms of if you already use our markdown you know how useful it is to have, it is to have code and outputs side by side in facilitating sharing and reviewing by other people who are coding or were reviewing your results or collaborators to for reproducible analysis, you can reproduce an analysis with a click of a button, make few changes and reproduce it again, and different output formats are possible with dogs PDF we of course HTML format has the most of the functionalities so that's really useful. Some features that we are we find particularly nifty for us is just this is inline text. So I want to summarize include a summary of this is the mean or this is the standard deviation of this parameter inside my text and I don't want to write it manually but have it computed I could use the inline text function so here is if you've not used it. Here is a simple here is an example so I have this. This is this is all code and this is a generated output because this slide show is also our markdown document so this is all code and output here that you're seeing so I have assigned is to and I want to say, I have a number of cats and a plus one number of dogs, and I include those expressions in this codes, along with our, and then our mcdonald knows to evaluate that code. So I have it will this is the output so this is what you type it as your code and your our mcdonald report output will have two cats and three dogs which is which has taken the value from a and this from a plus one so if you include an expression it will be evaluated. Like that and in your final report it just looks like plain text but it's just like the clever thing that if you want to include a number which has been computed and don't want to type it changes this is this very nifty. This is the tick feature which is really cool is just having table of contents and and then a floating table of contents so it's just with just these two commands to see is true and to see float is true. I'll just give you a demo here. So this is a report with floating table of content by floating I mean as as you go further down the report, the your table of content stays with you so you can actually use it to navigate through the report, no matter how long the report is. So that is very useful. And then something else that we have been playing around with a little bit is remember our stationary map of Bangalore low cost sensors. So this is, we have around 40 plus sensors and for each sensor we have to very frequently every few weeks or monthly we have to generate diagnostic reports like is the sensor healthy or is it with data reliable so that just that means data cleaning analysis and then starting some time series regression, comparing it with reference regression and the analysis looks the same for all of whichever sensor you choose the analysis with the same. So we are playing around with the idea of using parameters using the each sensor as a parameter and then generating individual diagnostic reports using a single arm on file. So that will be that will again really simplify the analysis. So that's all today for the analysis part. I'm really excited to share with you what this the final part of the talk which is stuff that we have learned from the art community. These are the things that you know, we did not set out to learn these lessons but we are so glad that we did because it has really, I think, improved our work for the better, and just our approach for the better. The first lesson is just, you know, importance of community. Today, if anyone asks me if we are on the fence about our and would do learning are be worth it or should spend time and I, you know, I my usual responses, it's definitely, it's worth your time. I don't feel that the time is wasted. But, and also, our community is amazing so you should definitely learn art for that for the community. And five years ago if someone had told me that you learn this technology because I'm a community is amazing. I would not have been able to relate to that because that's just like that's just two different things that were two different things in my mind that I couldn't connect like I just I want to make cool plots. I want to learn this statistical method. So I can use it in this paper. I don't know, I don't know about this community aspect I can't relate to that, but that could not be further from the truth today. And, and, you know, you, all of you are attending here who are part of our community so thank you, thank you for that and it's just just the support and encouragement that we have received from the our community has been amazing. And when I said in the beginning that we learned everything on the job. In terms of our, I think the other thing is also that we learned many things from the community, just many techniques, many approaches things to think about, and we are just so grateful, whether it is the ladies like channels when you're stuck you post a question there's so many helpful responses. The minorities in our user community which is just amazing and encouraging and if you identify as a minority in our please definitely consider joining this amazing community. The book clubs the meetups the tidy Tuesdays and the and the code of sharing. The norm of sharing your code is that's just like such a good spirit, and we all shared that now and we've learned so much so much from that. So, that's an amazing, very invaluable lesson that we have learned. Second lesson is using artwork for teaching. So, when you think of data science or even like air quality that an artwork it just doesn't fit together at least you know, in my mind it didn't used to, but now I have. Now, my mind has changed our mind has changed and the main inspiration for that comes from Dr Allison hosts work on her, you know, amazing fuzzy delightful are monsters and her artwork in for beginner concepts as well as like really sophisticated concepts like clustering and statistical analysis and all of this are just so useful in making, not just learning but also teaching our and just making it so much more friendlier and more fun and more accessible. So, this, this has been very very inspiring for us. I just want to give a shout out to the take up giraffes project by volume and Leon. The link is right here. In their own words. It's a delightful series of modules to learn statistics and our coding for students scientists and starts enthusiast and it's truly truly delightful. So they start if you're not familiar with this they start with these cute giraffes measuring their length on each of these two islands, and then looking at the differences of population so through through like these through this kind of storytelling they really introduce distribute basic concepts like distributions population distribution standard error ggplot. And so if you are if you're a beginner yourself or if you're looking to teach beginners, please definitely check check this resource out so examples like this like they have made our learning fun. And as well as really made us think about how to push this envelope of teaching and making it less dry when it comes to our and starts and you know, a bit more friendlier. So that's, that has been great. And the third lesson is this open source and collaboration. So how to operate in this ecosystem and you know, like how to think about sharing code and making making that collaboration less painful and less painful and easier by following some coding practices. So, again, this is what we try to follow to make practices easier and if you have more insights of this stuff, this stuff that you do I would love to hear that in chat. So, using projects and version first like I mentioned, version control for organizing and really really exhaustive descriptive read me files so if people read them, especially in this pandemic situation when you are not there to explain. Many times and people are working in time inconsistent manner, having that read me file that is exhaustive to hand hold someone new is has worked well. And just coding like following tidy word style guide for code so your code looks consistent is easier to understand and consistent naming with functions and objects. And commenting commenting code for yourself and commenting code for others for our collaborators where and definitely explaining the why of the code like why did we do this analysis and why did we do that analysis when we did that one. And especially when you're adding new packages into code when the new packages and new libraries that are function tasks specific so just come commenting that what that library is for. And last but not the least, not hard coding file locations so no file locations which says like my C drive and my computer and my folder inside the code because that will not work on someone else's computer so using a little file parts with the package here that's been that has really simplified this using relative file parts in the code. So these are some practices I won't say that every code on my computer right now follows this, but at least before we share it with others we try to go through this checklist to see that all of this is satisfied satisfied. So what next we have. We have a, this is some things from our list so one thing we want to do is a booktown project to organize all of our field protocols and code project wise so for each project this has been kind of this ambitious project that we have. I think we are working towards so so that like we can bundle bundle it in a way that if you know next another group wants to work on mobile monitoring, we know we could just transfer this do this knowledge transfer in a way that it is can be well followed. And we want to explore tidy models as we get more into modeling and machine learning for building a quality models and the third thing that this is, I've written it as a bullet point but it's more like a reminder for all of us, our team that is thinking about accessibility. This has definitely been a blind spot and really it's the interactions with our community on Twitter and slag that has sensitized us to this to think more and to you know to to have considerations for accessibility in our workflow. So, for this presentation I've tried to add all text in the images if they don't work for you please let me know because I would like to definitely like to correct that and that's like this would be a work in progress but we would like to be conscious and start thinking about that now. So with that, I would like to acknowledge the resources that I've used for this presentation sharing in and sharing in extra sharing and extra has some really cool features, like the tab sets and style views and really adds a lot of functionalities to sharing in all the images and the one like on the side. The background images are from raw pixel which that are publicly available and the logos in the schematics are from the noun project. And this is a Creative Commons project and these are publicly available. I would also like to thank our research partners and funding collaborators without whom the work would not have been possible. And with that, I leave you with this image of what it feels like to be a part of the supportive art community and thank you so much. Thank you so much for being here today. Thank you so much for a wonderful talk. I have a few questions lined up. The first question is by David. Can you give some examples of research questions that we have been able to answer with art. So some of the, yes. So some of the questions that one thing that we wanted to ask is, you saw those stable the maps that we are calling stable maps. So, one thing was that we wanted to know how many drives in a neighborhood is enough to make reliable, reliable stable maps that capture most of the variation. So that's, that's one thing I think that's one of the first questions that we wanted to ask and that. So we did that using our it involves. It involves first taking a data sets of say 30 drive 30 drives and in a neighborhood and then you take two drives and three drives and 10 drives and see if they match with the 30 drive data set. And so it involves sampling and resampling and so all of that work. We did that in our and that like the answer the answer is just like in researchers have a definite answer but you get some answers and more questions. So, the answer was it depends on the neighborhood and depending on the variation in the pollution, the number of drives that you need will change. Yes, thank you. The next question is how often will update the stationary data for shiny. For the shiny or the stationary data. So the stationary data we don't collect that data that for the shiny up that data is publicly available already. Speaking of those speaking of those collective platforms it depends on who is putting out the data. Sometimes the federal agents and depending on the type of monitor. Sometimes, most of most of it these days is real near real time so it's every day or every every few hours, but if it's depending on the technique sometimes the older monitors which are not real time. The data can be updated on in a matter of days or in a matter of days. Okay. So, thank you. The next question is by are not you are working with several people that in this project, how do you organize that you all work in the same day and how do you get up and collaboration. So, it, I think it looks like actually we are not many people. So we are actually a very small team. So in terms of the people who do coding is really like right now is three of us who are who are doing coding so it's not, we're doing. So it's like, yeah, it's like a small team that's doing a lot. So, we use GitHub, not in the beginning towards the later later end when we are when one of us is like, say hey I'm done with this analysis and I think I've checked everything do you want to check if this runs the same way on your computer. So that's when we use GitHub to like just like whatever my project bundle is to just use GitHub so that nothing is missed missed and it all goes. So it's like I'm doing other people just download all of that and then clone that and then see if something is missing and if, and then she can check for all if I'm doing the best practices if I have hard coded filing file directories and stuff then she would change that and then I could make those edits. Okay, thank you. I hope Arnold we have answered your question. The next question is, did you run into memory issues when storing all your quality data in the laptop also since the analysis can be replicated all around the world. Have you thought of turning your analysis into a package. For this I would invite Aditi to answer the question Aditi that's okay because that work is led by you. Yes, so we did run into memory issues, but not with data but just by when we analyze a lot of data I would run into memory issues but then I also learned about packet processing so I keep on my course running for this mega wealth happening in my studio. So that is the first part of the question and the second part, we have not thought about making it into a package it never occurred to us but then I'm sure we can definitely give it a try. Thank you for pointing that out that's a good initiative. Yeah, I hope we answered your question. The next question is, I wonder if R is fast enough for data analysis in your case. I think you're dealing with big data. Do you use R in a cluster or on a single PC. I think that was something similar and yeah I'm actually going to add anything. I don't know I think you answered that. The next question is, could you please explain a bit what the here package does. Yeah, sure. So, if you just download the library here like the here package. You will get as soon as you download that you'll get a message that here starts from your current directory. So here basically encapsulates what your current directory is so if you're using it on a different machine. It will encapsulate that so basically here makes the starting point of your directory as your current directory and then from there I can write that in this folder my file my file is like within data within I have data and then I have raw and in that there's my filing so I can just say here. In within brackets data raw here so as long as I'm even if I transfer the same bundle like if I upload on GitHub and someone downloads it. It doesn't matter where on the directory their file in their computer the file is because when they use here that then again that path starts from here which is their current directory and from data and raw and the filing. I hope I hope that vocal explanation was useful. Yes. So the next question is, finding models is a great tool for trying different model classes, but the gains in the model performance are often marginal at best. What are your expectations from that. Thank you. Thank you for that information. I didn't know that I think it is just because we are working in tidy words and just like having a sort of streamlined way of doing modeling is something that we like to learn. And so that's that's just like a different way of writing code. In the tidy words or like the tidy tidy way of doing this. So that's that's why it's it's on our list. Okay, so there's no question. Do you need to update the shiny code for creating data. Do you want to answer that. Right now we don't have to update any shiny code, but you can definitely put a request in the data report if you want any more features of training data to be added right now it has a few specified cleaning options already inside the application. Yeah, I hope I was able to explain it clearly. Anything to add in action. No, I think that's that's good. Yes, we have one more question. Do you think there are enough to do this for going from beginner intermediate user or do you see some gaps. I feel like I have gone from beginner to intermediate are just using online resources maybe maybe not necessarily tutorials. But what have helped are the book clubs so our for data science book clubs or advanced our book clubs like within the R community so being part of those book clubs so it's more active learning rather than following tutorials. And, and I think once you start using our then you're getting you get better at asking the right question. So again it's like you get yourself in the data. Like, this is a question I need to solve. I know this much beginner are and then I can do these things to become for intermediate are also something that has help is just reading other people's code so that's why I just love tidy Tuesday and you know the there are screencasts by so so so many people that are helpful, not just for visualization but also for tidy modeling. So, I do feel like there are enough resources, not maybe not in form of tutorials, but if not in form of tutorials many other ways to go from beginner to intermediate. Thank you. This one last question which will be taking. So, Ashish says, I'm working on job-related regression. Can your research help in my subject. Actually, I don't know. Are you familiar with that. I mean, graph, sorry, job-related regression happening in for land use regression models or another type of language regression models, but you can definitely get in touch with us. What do you say? Yeah, and you know, the link to our GitHub profiles is are there like online on Twitter. So just like definitely get in touch. I'm getting into land use regression soon. So, you know, maybe maybe there's something that will be helpful for you. Can I add just one more question. So, what did you use before there was an alternative to our month down. How did you get around before our month down. So before our month down, I would just like share share my code, like the .r file and if anyone asked for it, because most of the time which is not needed like unless unless the collaborator is another person who is actively coding they are not asking for it. Really just word like word doc or Google docs or power presentation. So it's it's like what I'm able to share is only outputs. We still do that. We still do that and it's it's still useful to share with a larger like with a larger set of collaborators who really what like this doing higher level of at a higher level of decision making where maybe they are not coding and or they just want to look at the results. So what helps in Google docs is what we can do in our markdown yet or is like this this real time commenting. So if I want to say that even if someone is looking at in a time inconsistent way they can leave their comments and then you can come back and address them. So for such functionalities we still use Google docs and the combination of our markdown but just with the HTML reports and with the interactive plots it's just so much more information that we can communicate. So now, I think a hybrid of both is what we are trying we just we are doing and it it works depending on the who who we are communicating with. Thank you. Thank you so much again and actually for the very interesting and engaging team. And thank you everyone for joining us today. Please keep the discussion questions going on in the staff. There is a channel key underscore push for how and you do the great we want to again our studio or the sponsor of the day, and the next session will be in 15 minutes on these tracks and see you all again.