 Hello everyone, welcome to the NPTEL course on remote sensing and GIS for Rural Development. This is week 11, lecture 2. We are coming close to the end of this wonderful lecture series where I have been getting a lot of comments on how easy it is to use remote sensing for applications for rural development. And as I said earlier, I've met a lot of people who have expressed their interest in this course and multiple other ways that could be used for collecting data and working on rural development. On this note, I've also looked into certain aspects that are new for rural development and we'll be discussing this in the current lecture series. So while we are on the penultimate week, week 11, I will also showcase one aspect of data collection that is less acknowledged and less used at the real world. However, wherever this methodology has been used, has created considerable impact of this. I will be first talking about the synergized mapping. So let us go into this remote sensing and GIS for rural development, week 11, lecture 2 where we will be looking at data from other sources that have been used for rural development. Let's say, for example, we have looked into this pretty much in detail. What is synergized mapping is there are multiple players, multiple agencies that come in and help to map the data. And there are institutions and NGOs that create capacity training and update the knowledge of people through on the ground exercises. And farmers, stakeholders, rural communities are those who actually engage with the data and the results because it is for them these data is collected and used. So we have looked at satellite and drone data, but now we will look at crowd sourcing plus satellite data in a synergized mapping scenario. In this scenario, what happens is a lot of data can come through. However, not all data can be in a usable format. It is because there could be some data issues, quality issues, what data has been collected is not clear. So we will look into a schematic where actually this has been done at scale and a lot of people have been using this for the last past 10 years. So of the crowd sourcing data collection method, let me introduce what crowd sourcing means. So crowd is a general public and sourcing means you are sourcing the data from them. For that to happen, there has to be a platform that is given to the locals or crowd, we call it here. And they participate voluntarily. There is no payment here. Like for example, they have to give data and then they get money or it is not a mandate for them. This is purely voluntarily done. So what we have here is a group of people who are cautious about the system, what they are monitoring and they collect data and upload it or share it to the platform. So the platform has to be robust enough to capture the data and then put it at a larger scale. For example, the farmers might give data on the crop growing scenario. Is it at bird stage? Is it at intermediate stage? Is it at fully grown stage? So these stages can be monitored and then given back to the crowd sourced platform. Let's say for example, we have a database I've created where we can put the data and then see if it triangulates with what is happening on the ground like other satellite data like rainfall, land surface temperature, NDVI. We can we can triangulate if the data is working and or if the government schemes are addressing these issues. For example, the geology or mission is there where every house in the village is to be connected through a tap. And also there is the grid scheme where across India the rural electrification program has been well received but the quality could be compromised because of other externalities. So the pipe might be there but water coming in or not is very very crucial and that relies on the resources available. So how do we monitor it? It is very highly impossible to put meters everywhere and then monitor it given the population we have. So the best way is to work ground up. So all the other data is top down where we call it top down approach where the government collects data through the system and then takes it back to the top and then analyzes it for policy. When we talk of bottom up the people the crowd who's at the key stakeholder level they will collect data and then put it up in the system. So how is that useful? This is useful because the data goes from the user's end. For example, I am drinking this water so I should be able to know what is the quality of the water because is it affecting me, healthy etc. Rather than trusting the supplier who's giving me the water. So at the supply end there could be data collected from the tap how the water is coming the quality but at the end user how is the water quality I should be able to give and this helps in intermediate issues that can happen the water supply. There could be leakages there could be mixing of dirty water and drinking water etc. So all these can be done if the end user gives the data. So census is an end user data right you don't monitor from the top people go door by door collect the number of people living number of households and then issue the data and from there the electoral data is created and from there your population is also assessed. These are important scenarios for any government to develop both urban and rural but very much it is important in the rural entity because at the rural level what happens is we have to have clarity on the data we need to have better infrastructures planned and for which you have a lot of data that is needed. In the cities it is almost easy to collect because the representativeness is there there's a community of houses so one house reflects all the other houses whereas rural it's not it's very scattered and the hamlets are there where a group of 30 to 20 houses would be there and then suddenly after that there's a big land of agricultural land and then another 20 houses 30 houses 10 houses etc. So to address this the best way is to go bottom up where crowdsourcing is key and I've been advocating this a lot because if you use open source software open source instruments then you could definitely collect data at a very very high spatial and temporal resolution in rural villages for the fraction of the cost. For example the farmers can take just an image they don't have to use an app to take an image already the camera is there in the smartphone and almost 90% of people have smartphones if they can afford it like economically along the average line there multiple phones now available which are not that too costly and does the job. So one quick check of the fact is that you do see a lot of TikTok videos and Instagrams Facebook videos YouTube videos from villages and all of that using smartphones you don't have to have fancy devices. So moving on there is potential to collect data and put it so now why don't we use this for geospatial location data GIS data is the key and this is what a company has been working on and looking at this data in a long-term fashion. Let us look at this in a app that has been created in recent times so open street map has been widely used it is a very very sophisticated system we will discuss this system in today's lecture and also download the plugin in the QGIS software and see how it works and reflects the data. So I would like to install the plugin in the current series and then we will take a call on how this map works. So what we have here is open street map is a tool that everyone uses for looking at certain data that has been collected and shared by public. So what is OSM? It was found by Steve Coast in 2009 it is a free open geographic database it has data attributes we call them right so some data about whatever locations amenities schools crops everything okay you'll be amazed to see the quality of the data not all data is there because this is not run by a organization to collect data it is pre open source volunteer data so only if people pitch in data you will get data. So let us look at this at a very close angle my point here is when I say that all area is not covered the point I'm trying to make is most of the data is covered and we'll be able to use it wisely if we know that most of the data is covered so starting with no data at least we have a data that can be used that is the question. So we will be able to look at in the next lecture a particular state and a particular district we will download the map and then we'll apply it in this OSM software app we'll install this plugin in QGIS and then we'll extract data so this is based on community of volunteers this data collection initially it was very small in 2009 when Steve Coast had started it but slowly the momentum grows so like any other company any other product it starts slow but then when people momentum start lot of data comes in the data quality could be an issue I will open this link pretty soon now and it is really licensed but again let's not talk about the negatives right away there's a lot and a lot of positives I would say that this is one of the highest updated data that any government would like to use because it comes from the bottom not from the top where top down is you will send a person to go collect data so he or she will travel go collect data and then come back sit and map it takes time whereas here with the click of a button from the volunteer it comes straight to the location database and the volunteer is already sitting in the village so you don't need to have you know super time taken for these kind of activities so let us open the free open state map.org and what we'll be looking at is we'll be looking at the data set the metadata about the data as I usually am telling you that it is very very important to have metadata for all the data products that we have the boundaries may not be correct because they may use different boundaries but we are more important and we are more focused on the attributes okay so let me quickly open it and while we go to the website this is the OSM plugin that I'd like you to install I hope you know how to install it because we have given you the tutorials on installing plugins however if there is time enough we could also show quickly how to install the quick OSM so I will go back to this opening this links please allow me to share right so this is the first link when we open OpenStreetMap.org you have the map coming up you can search a location so let's say you can say it's Pune as we have been doing for the past lecture series yes I would like to see the Pune and there's multiple other things in Pune but let's go here okay so here you have the boundaries also so fix Armin boundary for Kalyani Nagar fix boundary for Pune these are the data that are there in different languages also you can have and in Tamil also it is showing so there are different languages and parts available okay so this is the one when you when you open Google and then search for OpenStreetMaps most probably this will come up because this is the basics you can say learn more and then see where it goes in terms of who's the local knowledge and then how the community is driven but I would like to see here show you here it is hosted and supported by UCL UCL is the London's Global University homepage so it is it is actually pretty good that a university is giving these spaces because someone would have asked me okay how do you get a big a big database and managing the database all of that is done in this UCL which is good because some some these cannot be by itself done so UCL stands for University College London and a lot of research activities have been done here you could see that the the database it stores is pretty historic and it will it'll actually do a wonders because they cannot afford to do it as a volunteering company you cannot afford to do all of it right so and also this supported by Fastly, Bitmark hosting and other partners, Bitmark hosting is also a cloud space but other partners are also there so let's first acknowledge all the partners from the many several have been supporting OpenStreetMaps from the beginning at the Barlett Center and very very very great Barlett Center is the hopefully the data center that creates all these locations a built environment is is having some data input and data access Fastly, Bitmark for many years have been extremely helpful for supporting them and many many others many many others are here you will see mostly you don't see the proprietary software here because for them this is kind of not they're just cutting in their business and profit right so proprietary software have always business these whereas this is open source it is free for everyone to use you can learn more about OpenStreetMaps so let's go quickly and learn more so you see the local knowledge what does the local know about the particular data set let's say water body what is the name of the water body everything is is contributed contributors use aerial imagery GPS and low tech field maps to verify OSM is accurate and up to date so for example OSM might contact you after you give data to show that this image we have taken can you check if this image is correct let's say the Poway Lake boundary we know that the Poway Lake boundary has been changing because of the developments around the lake so they'll ask me okay is it correct is it changing and then those kind of things we can take on aerial imagery then we have the GPS devices some people may not have a handheld GPS device to monitor and take data however your phones are pretty good your phones can take a very good amount of data so please try to see if you could use your phones for these kind of data collection and stuff there are multiple apps for GPS please look into your play store Apple store for downloading accurate GPS free open source software that gives you the location and in the location you can mark school education crop time anything that you can mark low field tech maps so some maps that needs to be updated and then those are being used and it is community driven a lot of passionate youngsters are giving a lot of their time to develop this database and then they're very and who are they they are very fond of mapping GIS professionals engineers humanitarians disaster relief workers NGOs industries colleges institutes etc and open sea street maps are free data OSM is free open data they're free to use for any purpose as long as you predict opens street map and its contributors I have I don't I don't see them pull you down but as I said it is it is ethical to thank the contributors and acknowledge them so please feel free and take it as a as a duty to acknowledge them and use their data so you'll see that all my slides when I'm using OSM I do have their logos so the logo is meaning of not showing that it is my work it is OSM work okay so copyright and open data statement tells that it is free and open for everyone to use so this is very important because a lot of people give money for them right so as a as a free open source system they cannot sustain by themselves a lot of people have put in money when they put in money they want it to be not commercialized but available for the people if they start commercializing then it becomes a company and I demand share on the profit right so if I'm putting money and they are using for commercial then I demand money for my services and budget but here since it is a full open source for our open source it is it is important to understand that there is a need for acknowledging them and it has to be kept open source okay it's the legal documents are there if you would like to see and then these are the partners hosting partners again UCL as I said hosts the data a part of the data it's better to break it into smaller bins and then store it let's say Asia region is in one one database the US region is in one database etc or some attributes so they'll say all the schools are in one location all the hospitals are in our location etc so the key contributors if you to see you would see multiple countries you don't see India yet maybe it's not picking up yet so hopefully it will pick up and then there are trademarks for the brand or OSM as a brand has been trademarked okay so in the open street communities if you go to the community part okay I'll go back to what is open street maps we've learned a lot so you can see the history of your your searching and how people have updated as I said in this location who's updating so if I change the location and then go to let's say Chennai and then okay so Chennai is there nine months ago someone has input some some data and then we can go to see the history in the history also you can see now the name changes for so for this region so the box you see for the box who has input data and how old so close about 11 hours ago someone has input data on accessibility added elementary etymology data added buildings so you can click here and see what buildings were were added so someone added this building in open street map which is really good and other nodes locations etc can be taken up right so in the history side we do see a lot of these data two three days ago three days ago and somewhere your name and other stuff is being added because they say how many edits map knows etc so here is where sometimes you may not get very very accurate data but it is your role as a researcher as a student to make sure you verify the data because it's open source people can put data that is not correct and accurate and that is not purposely done it is done also because maybe they didn't know the correct way of doing it so it is up to you to correctly map so when you start mapping you will have to sign up display name your profile and then you can create the maps where you want to put data and other sources so this is it about open street maps you have gps spaces where it tells you someone has traced a route from drag point Toyota price from a car and then a lot of other people have done traces using their gps data so basically they have taken their gps location and road and path and they're putting it into it so user diaries are about users and how they have mapped how it has helped etc hospitals locations if you see it is mostly very very useful in developing and underdeveloped nations because you don't have a set agency like NASA or ESA for helping them using remote sensing data so crowd sourcing is the way forward and then we have the communities as we said we already saw the communities there are multiple open street communities we don't have one for india yet and i really hope a lot of communities can be set up and users can like you who are taking this class can also be part of this community and then copyright yes there is a lot of copyright that can be shared and used so we can see this i have already said what are the copyrights that are available for public you can you can pick and choose your copyright status depending on how you use the map and for what you use it the trademark is there the magnifying glass logo and the state map are registered trademarks of open street foundation we will not go into the open street foundation but understand that it is part of the logo and trademark okay so we will now go into the other aspect let me where we have the help section where we can see like beginner's guide help for a mailing list you can be part of the community you can open the open street map wiki to look at documentation about the person so you can see here the beginner's guide developers and users platform etc so this gets back to your browsing the open street maps how to browse what are these low points it's like a tutorial for you to use and then use it for your learning and understand we'll do some quick tutorials i believe in doing it and learning it so it's better to do that way also okay so we have come here and then we also have other platforms forums asking questions so normally as i said there is a lot of open source community helps where people just help you to do it so for example here show coordinates on map download international boundaries so maybe someone has an issue in downloading completely new with osm i tried to download all international boundaries is that possible and yes how and then the answer is given so on so we get the answer is not given because it's a very recent question let's go to some questions the answer is there routing distance from open street map so someone is asking i use osm back to api right here routing waypoints but there we use an image what should i do you can set up the graph hopper or osm on your infrastructures then load the pbf file for calculating the distance okay but the point here is so basically any questions you have if it is a legit question and a serious question they will be answering it because people take their time looking at it and answering it so let me while we before we go to the next session i would like to open the plugin and then show you how to download your your plugins because that is very important at this stage for which you will have to you have to make sure you know that is it useful and do you have the bandwidth for keeping keeping the stored data in your computer okay so with this let me open my qgs page so let it let it open in the background i'm going to open my qgs now while we go back to the previous slide and then open this link so we will also look at the files the metadata as promised let me open so which is here in the web in the page bottom right i'm going to open the the planet osm which is the metadata download space are currently restricted so you can download the data so here's where you can learn about the data and download the data so latest torrent file on data that is existed 125 gb created two two days ago latest weekly change sets all in a torrent side you'll have to use torrent to download it and again this is if you want to download the entire data but i'll be showing you without this how to download the data as a qgs package so for that let me open my qgs and share with you so okay so i'm going to share my qgs yes we have my qgs and yes share my qgs so what do you see here is an empty slide i'm going to go to plugin manage and install plugins it'll first open that pop-up which says i'm going to download the repository let it download and then this happens so in the install i have all these installed for my previous maps now i'm going to type q quick osm okay so quick osm is the tool that we'll be using so let's first read about it here okay so it says download osm data thanks to the overpass api you can also open local osm or pbf file this is also the logo trademark by them in terms of using it for data parser top ogr is used to let you see all the osm keys duty ratings look at the number of downloads literally a lot of downloads and then the tags are open street maps etc so open street map is what we want we will click it and then once this happens we say install plugin let it install depending on your internet speed and internet bandwidth it will take some time okay so now it has done so quickly this will get updated just see it okay here is what happens plugin install successfully and we have the qsm page that we have so it says the version so usually these were not there now it says 2.1.1 updated august 21 and these are the versions add a warning key well how this is improved then this so we had 2.01 2.10 2.1.1 we're going to use the recent one so this is about learning on what are the issues in the previous and how they fix it fix an issue with installing q quick qsm plugin good and then we'll close it so now if you go to vectors and then say see the vector tools you will see qsm okay do not open this now we don't want that but qsm is what we'll be using for our lecture in the next lecture because right now let us set it up and I also wanted to show you something that is needed for collecting data quickly from osm so I'm going to add my India layer so we'll go here in my GIS layers we have so I already have some data that is downloaded for this class but I also want you to download some data that is very important for you let's say you're going to download a particular database which is which is having a lot of information on your project so let's say you're going to do a rural project in India, Tamil Nadu and within that particular district so how do you go about it is the question so here we will we will look into a particular database and then we'll see if how we could download the data use it in the following ways okay so I'm going to now download first attach my district and full states I'm going to show you quickly how to take data so this data was taken from a previous geological the survey of India data okay the previous version because I was doing something previous before the Andhra Telangana was divided okay so you could see that the full Andhra is there so this this was used for my particular research interest and then you also have vector data you can add more vector data click on what else data I have you have India district shape file add and then you'll see the whole of India with my districts okay so let's say I need a particular district I have a particular district in mind that I would like to use for my analysis so I'm just going to open my attribute table you can search for it using the search box or you can actually if you know the district name you can come down and then look at it so this is the name zero is the India level then you have the state level and then you have a district name in here so let's first click the button here so once you click the open attribute table I think my open attribute table is not visible because of my screen so I hope now it is visible yes so all I've done is I've opened the India districts file and then the open attribute table has come I have clicked the name zero as India name one is state and then name two is your district so I clicked this arrow mark so that it does it into alphabetical order let me do it again just to show you how to do it moving the layer I am adding a vector layer okay now the vector layer has been added I'm going to open the open attribute table okay so this is a name a state name zero is India country name and then the district name is name two so you can see that randomly it has been arranged let me click that button and then it goes out so now I want p so I'll let me come down to Triciapalli because that is what I am going to use for my study ST okay so we have Tenu, Tanjavur, starting from VL, Sri Rangam, Tutukudi, Tiruvallur so we can just come down to see which districts that we want to use so Triciapalli is what so right here I'm going to click it so that whole line is being selected so now if you come down and then see the the district has been selected correct so all you have to do is right click export export save feature as in this page make sure this is clicked save only selected features because you have clicked a entire district shape file we're not going to do the entire district we're only going to do a particular district so I'm extracting it it's kind of extracting so I'm going to say that I'm going to save it as Trici the short form of Tirupalli so to save and then the default other things are default can be say add save file on to it yes so now if I remove the India district you still have to cheat okay so the idea is you can take a whole data set and from there just extract what you want and keep it ready for your analysis in the next class so OSM does require this aspect a shape file to feed in or you can use your internet and then use OSM to have good data extraction so with this I will stop here I'll see you in the next class please be prepared with the OSM plugin and also some shape files of your interest thank you