 Hi, my name is Xiao Yan. I'm a research staff member at IBM Teacher Watson Research Center. Today I'm going to give a tutorial on IBM Pears. It's a large-scale, big geospatial data and analytics platform. Pears' website is athttpspears.res.ibn.com. IES stands for Research. If you have any questions, please feel free to contact admin account pears.ibn.com. Here is the outline. I will quickly go over the sign-up process and then give some live examples for doing queries on the user interface. And then in the end, I will demonstrate the API course using the REST client plugin. Let's get to it. Okay, here's the Pears' webpage. On the left-hand side, here's the link to the user menu, which is very useful. It shows you how to use Pears, how it's designed, and the details of each dataset. And then this tutorial video will be hosted here as well. On the right-hand side, here's where you can sign up for a new account. For IBMers and academic or non-profit users, you can request a trial account. Once you click Subscribe, it should bring you to the subscription page. We need your work email and your institution name, because the approval process involves validating the email versus the institute. And we would love to hear about your use cases using Pears. Once you've submitted, you should receive an email saying we are processing your request. Usually this takes less than one business day. Once the account is approved, you'll get a welcome to IBM Pears' email, and you need to use the link there to activate your account. Basically, you need to set your password. Please note that this link is only valid for one use. The second time you try to use the link, it will not work. It will give you an invalid link. And this is for security reasons. But if you already activated but forgot your password, you can reset your password and it will bring you into the system. So let's get started. Your login ID is your email address you used. Here are a few main menus. The query page lets you define the query conditions. The job page shows you all the queries you have done. And because this is a big data retrieval platform, you actually don't need to wait around for the query to complete. You can log out and log in and the queries will be saved under your account. The metadata page, I'm going to go into details because it explains the architecture of the data organization. First is the dataset. As you can see, these are the available data sets like ECNWF, European Weather Model, Elevation in the US, Global Weather Model from NOAA, Crosscape is historical crop planting mapping in the US. We also have a drone dataset for experimenting. And MODIS satellite, we have global satellite coverage of MODIS with two different products. PRISON is the historical weather dataset in the United States. You can go back more than 10 years. IBM Analytics is where we put our own developed model from PERS data and put it back on the system for users to use. This one is called reference evapotranspiration. A climbing forecast is based on NOAA's CFS model and it is a long-term forecast with six months to log out. This one is SMT self-learning model. It's our own statistical weather forecasting model for two days ahead in the United States. And then NINE forecast is also United States weather forecast from NOAA. Then there is data layer. Inside each data layer, inside each dataset, there are multiple data layers. Let's pick USA weather. And as you can see, there are parameters such as temperature, relative humidity, solid irradiance, pressure, etc. So we covered dataset and data layer. The next one is data table. Data table is something new we're developing and it's going to be useful hosting Internet of Things type of IoT sensor data. Basically, this will enable hosting our point data that it's a long series of temporal events. We'll save the data table for our next tutorial. The next one is data region. Data region is most useful for satellite data. So for example, if I choose MODIS, because satellite data is organized by tiles, what it means is it's just like a snapshot picture of taking by a camera. And you can look up the tile of an area you're interested. For example, here the horizontal tile 7 and vertical tile number 5. And it's located around in latitude 40 and this longitude. For MODIS, we have global coverage. So it has all the tiles. It has a complete list. The last one under meta data is color table, which is the color scale we use to visualize data. You will see later. Only administrator can associate a color table with a data layer. So please let us know if you have a favorite color scale you want to use. Please do not change anything in here. Just for one example, radiator shows spectral data scale. And help menu has three submenus. One is the user menu, as we just mentioned about. So this gives you access inside pairs. And then there's tutorial page. And the tutorial page will host all the tutorial videos. Our video today will also be put in here. And then of course we have our acknowledgement, acknowledge all the data sources that allow us to use and distribute the data. Okay, so let's get started. So query is under submit new, submit a new query. As you can see first, what query you need to define at your spatial coverage. It can be a single point, can be a polygon, it can be a rectangle area you draw on the map. Okay, let's first take a look at single point query. Click anywhere in the world. It will fill in the latitude, longitude for you, or you can choose to type in your values. Then you can choose the temporal coverage. In our case, I want to take a look at climbing forecast, that's six months out. And this one give you the interval. And you can also just do date. And what this means is it will bring you the data that's prior to or equal to this timestamp. All right, we can choose all the parameters. And then submit this query. And this came back right away because point query is very fast. And the data is displayed on the table is very convenient. You scroll down here, B, this column is a unit. And then the last column is the value. You can download the data in CSV format or in JSON format. I just want to point out here on the bottom, there's a feature that's very useful. It's called the API string. You can copy the API string to a clipboard and save it to use later for API calls. You just need to attach the web server address prior to the API string. We'll get to that in the end. You can also say send to query page. What this does, it remembers all the query conditions you're pointing, so you can modify it any way you want and read some meta query. This is how you modify your query and it's very convenient. So point query is very straightforward. Now let's go on to polygon queries. There are three different polygon groups of polygon, personal, group and repository. Personal are the polygon that you actually uploaded yourself. So I'm going to show you how you add your own polygon. Under the query menu, the second item is called add area of interest and personal. As you can see here, it listed all the polygon that I uploaded. To add a new polygon, just click the little add icon and give a name, a key and a name. Here I want to explain share with group. The concept here is you can choose to share your polygon with the group of the user under the same category. This is a way of collaboration. I always share my polygon with the group so everybody should have a base set of polygon show up under their group category. So let's choose a KML file. Right now we only support KML shift file format for now. And then add. You can see we just added test and go back to the submit query page. Under polygon, you should see the test I just put in. And for everybody else, it should show up in the group. So for example these are the shift file that other user actually shared in the group. The third group of polygon is called repository. This is very convenient. We have uploaded all the shift files for all the states in the United States. If you put in USA, you will see all the state. And you can start typing and search for the shift file you need. And very soon we will upload all the country shift files as well. Now let's do some queries. The first example I want to show is for orange farms in a county in Florida. So in my personal polygon, I have a polygon called Florida Isoto County. And I'm interested to take a look at the latest NDVR satellite image of the oranges. So what I do here, the satellite is one of the satellite product. NDVI is normalized difference vegetation index. It's a measure of how green the tunnel is. And I can add a condition on the historical planting map in the US. I want only take a look at oranges. The index for orange is 212. You can find this index in the user manual and submit. And this is the new query we just submitted here. And this brings you to the jobs page. Most of the area of the jobs page is the map for visualizing the results. And on the right hand side is the details of your job. See this one is complete retrieving the data. And I just want to show you one useful tool here. Click there and click on the little pan here for editing. You can give a nickname. For example, I will say Orange, Florida. And you can also copy the API string just like we did and save it for later use. And what we got here is you click on the file name and it will show you the data layers. It got back. Check on the box. For example, here is a crop because we use crop to do the filtering. There are lots of oranges in this county. And if you click on the timestamp here, the color scale bar will show up. And this is the one we define in color table as I mentioned in the metadata. And we're interested to take a look at the NDVI value, the distribution of NDVI. And click on the timestamp here and it will show you the NDVI color scale. Click on it and it will show you the value where you click. And this is very useful. You can set up a time range and to collect a long history of NDVI evaluation evolving. And you can even correlate the NDVI to your yield of the year and also correlate with all the aggregated weather information. So that's the first example. So now let's do a query outside USA. One feature I just show you here I didn't mention. You can also do send to query page. That's the same concept as a point query. So here I'm going to save some time to do the query. It doesn't matter. For example, 4 is fire. So for this query, I'm going to show you how you choose an area and query the area just by clicking on the map. So I'm interested to take a look at the impact of this awful forest fire in Canada this year. It happened, I think it started in May 1st of 2016. So I'm going to go zoom in and go into that, go into the region first. You know it's in Alberta and it's around Fort McMurray. So what I can do is click area and I can just drag and draw a little rectangle. That's the third way of defining the area. And it was happening in May 1st. So I'll do the query from April to maybe June 1st. June 15, let's say. Then you will capture the before and after picture. And this is actually choosing both products for modis. Aqua and Terra, these are two equivalent satellites. While this one is working, the next query I want to show you is let's do something in Africa. This one, I'm interested to take a look over the weekend. What the precipitation is going to be like. And I want to see what the evapotranspiration is like. So I can prepare for irrigation in some of the areas in Kenya. So this is the Kenya. The time is for this weekend. GMT time. And I choose ECNWF. That's the European model. It's global weather model. And I choose in precipitation rate. And the filtering condition is the reference evapotranspiration is greater than 5. So this one is working. We can take a look at the one we just finished. That's one about the forest fire. So as you can see, this is March. I think it kind of still pretty cold. And it gets greener. Time goes by. And then this is the day before forest fire. And then you'll see two weeks after the forest fire. It's a whole massive impact. The fire as you can see here. And you can see it should be the same story on the other satellite. After winter, get greener. And then the fire. And this is very useful. These two. Just a few click, you'll know what's happening. This is very useful for monitoring environmental issues and events. The one for Kenya also complete here. Kenya we're trying to take a look at the weather pattern. And you can see these areas has low precipitation. Basically means low evapotranspiration. Because we said evapotranspiration to be less than 5. If I click on the timestamp, it's less than. These areas have high evapotranspiration. They are higher than 5. Okay, I was clicking, sorry, I was clicking on precipitation. So you'll see the moving of the rain cloud, raining area. And yeah, I want to prove to you the evapotranspiration. It's the query area is higher than 5. Okay. So the next query, I actually want to do something for a much bigger region. Like China. And you, this is a really large area. And I'm interested to take a look at the weather pattern, the seasonal forecast, and around New Year's time. During the weekend, what's the weather it's like. I want to take a look at precipitation. I want to take a look at temperature. Let's submit this one. Yeah, I just want to point out there a few things that's pretty useful to do. For example, look at the modis satellite image. When we look at the one of the five, that's after the forest fire, you can change the color scale. You still use the green, you know, the green color bar. But you can change this to make it, you know, to make some area more standoff. For example, this is this. And I want all the area that's below 0.5 to standoff. See? So really show up the area that got impacted by forest fire because NDVI value is much lower than the other areas. Okay. Yeah. Okay. So this query also returned already. This one is going to turn a little bit because it's generating visualization files. The first parameter here is ground temperature. And you can see the ground temperature in China. Very cold and very warm in the south. So it's a lot of fire files we generated. And this is the precipitation pattern. So this way you can plan your vacation accordingly. Okay. Some areas are more likely to have rain. I will avoid for my vacation if I'm going there for vacation. Okay. So let's do one more query. This one is quite special. It's our experimental drone data layer. So this coordinate system, this little square is actually our research center location. And I'm going to take a look at the latest drone we have. And I'm going to take a look at the band. But in addition, I actually, let's make it a little bit more fun. Let's compare to satellite data. See how they compare. Of course, this is extreme. A drone image, the resolution in our case is about 20 centimeter. And the satellite image is about 250 meter. Of course, there are other high resolution satellites out there commercially available or publicly available. But drone really offers an opportunity that is because satellite image one is very far and to the resolution couldn't really get to sub meter. It's very hard to get to 20 centimeter, but maybe it's possible. And then three is it's impacted by cloud cover. So if there's cloud, you're out of luck. So we had that experience, especially with the coast area of California, because it's so cloudy, we couldn't get much clear sky images of the area. So drone can overcome that if we have a very reliable and consistent operation of drones. The data is back, it's 100%. I'm trying to cover as diverse data sets and use cases as possible to give you a good feel on how this works. One rule of thumb is when you create query, try to create query that's not so big. For example, instead of you do, okay, 10 days forecast for the entire United States, you can do five days and then the query size will be half. And then you can launch both of them at the same time. That's how you get the data back faster. And for all the query you have done, you can download the data you want, you can save the file. And we encourage all users to use other GIS software for additional geospatial analytics, for example QGIS and S3. Our platform, if you watch the introduction video, we focus on bringing the content, bring the data to user and make it very easy to use. We take the heavy lifting of data scientists shoulder so they don't have to processing and curating all the data. Okay, so this is the drone data. You can see the cars very clearly. Actually, this data set reaches the limit of the open stream map rendering resolution. The initial, the raw resolution is actually higher than what I show here. And for comparison, yeah, this is modis. It's just something to keep in mind that drones really can be very powerful. Okay, so next we're actually going to go into the API section. So I like to use a little web browser plugin called REST Client. You can see here. So instead of typing in the API code string on the browser, I put it in here. So we have, the first one is the point query. Okay, we're already prepared. So you can just copy and paste and send. I will ask me for username and password. Yeah, this is a lot of data for it to put it into JSON format. The default here also is JSON format, but you can, in our menu, you can find, you can actually ask to return our CSV instead. Yeah, this is how it looks like. Value latitude, longitude, data layer, that's the temperature. Okay, now let's check the polygon query. So that one, the type is point, this one is polygon. Yeah, so this one, it will give you the importance in here, give you a job ID. This is the one we need to retrieve our query, the jobs. And then put in our job ID. And they say it's succeeded already. And what you need to do is just after query jobs, put into the browser that you download the file. Yep. So that covers the basics of API call as well. And I hope that is useful. And thank you very much for watching. Please let me know if you have any questions and welcome to Paris.