 Hi, my name is Xiaoyan. I'm a research staff member at IBM T.J. Watson Research Center. Today I'm going to give some demo on IBM Paris. It's a big-scale geospatial data analytics platform. So today I will quickly go over the sign-up process and then give some examples for you doing the query on the platform. And then in the end I will demonstrate the API call using REST Client. So the website is Paris.ies.ibn.com. IES stands for Research. And if you have any questions, please feel free to contact admin account, Paris.us.ibn.com. So let's get to it. Okay, the Paris webpage here, on the left-hand side, there are links to the user menu, as you can see. The user menu, you can find pretty much everything you need to know about how to use it, how to design and the details of each data set. And then we also have this tutorial video will be hosted here as well. And our system maintenance weekly schedule is 5 p.m. to 7 p.m. Eastern Standard Time. On the right-hand side, here is where you can sign up for a new account. For IBMers and non-profit or academic users, you can request an account. We need your work email and your institute's name, because the approval process involves validating the email versus the institute. And we would love to hear about your interest in UsePairs4. And once you use this subscribe, it should bring you to the page, this is the subscription page. It should send you the email, say we're processing your request. Usually this takes less than 24 hours in a business day. And then once the account is approved, you'll get a welcome to IBMPairs email. And you need to use the link there to activate your account. Basically, you need to set your password. And the link is only valid for one click. So if the second time you try to use the same link, it will not work. It will give you invalid link. This is for security concerns. So the link is only activated for once. But if you already use it and then for some reason don't remember your password, you can reset your password. It will bring you back to the setting password page. All right. Okay, so let's get started. Okay. Your login is your email address. Here are a few main menus. The query page let you define the query conditions. The jobs page show you all the queries you have done. And because this is a big data retrieval, you actually don't need to wait around for the query to complete. You can log out and log in. The query will be saved under your account. Metadata. I'm going to go into this a little bit because it explains the architecture of the data. First is data set. As you can see, these are available data sets like ECNWF, European Weather, Elevation in the US, Global Weather Model, for our cross-gib is historical crop planting map in the US. I think 30 meter resolution. We also have San Joan data set. We're experimenting. And MODIS, we have global satellite coverage of MODIS, two different products. And present is historical weather in the United States. You can go back, I think, about 10 years. I've been in analytics. This is where we actually can develop new models and put it back on the system for users to use. This one is called reference evapotranspiration. And climate forecast is based on Norway's CFS model. And it's a long-term forecast, about six months look out. This one, the SMT self-learning model is our statistical weather forecasting model for short term in the United States. And then forecast is also United States weather forecast for Norway. And then inside data set, inside each data set, there is a data layer. So for example, let me pick one that has a lot of parameters. You can see these are the parameters like the wind, precipitation, ground pressure, solar irradiance, relative humidity and temperature. And these are called data layer. So data set and data layer. And then data table is something new we are developing. It's going to be able to host Internet of Things IoT sensor data. So basically it can host a point of data that has a very long temporal series. So that one is not ready, but it's coming up soon. Data region, this one is mostly for satellite data. So for example, satellite data is going by tiles. What it means is it's just like a snapshot of one part of Earth. And you can look up, like, see, okay, if you know the tile of the area you're interested in, you can take a look. Okay, is that tile available in the database? So for modis, it's horizontal tile number seven. For example, here is the number seven and vertical tile number five. And it shows you the center of the tile, the latitude and longitude. So for our modis, I think we have all the tiles, it's a global coverage is complete. And the bottom one is color table. Color table, this is where you define the color scale you use for the data. For example, blue, maybe you want the blue color scale for the blue spectrum. For example, in satellite, you can associate them. What you do is, I think it's in the somewhere, in the data, maybe in the data layer, you actually can associate that. And of course, administration, you can change your password. Under help, there are three menu. One is the user menu, so you can access past menu inside your account as well. And then the second one is the tutorials. So here, in addition to the demo video, we also have the introduction. The introduction, hopefully you already watched that one. It covers the design, the concept, and the architecture of pairs. Okay. And then on the bottom is the acknowledgement to all the data sources we use in our system. Okay, so let's go back to the query page. And go to Summit New. And as you can see, excuse me, for define a geospatial query, the first thing you need to define is the spatial coverage. So it can be a single point, a point anywhere in the world. And it can be a polygon. Now, you know, you preloaded the polygon shapes. And it can be an area. You can draw a little rectangle and define an area. Okay. I'll get into a little bit more detail. So let's take a look at just single point. Just click anywhere. So it will draw the latitude and longitude. And for this one, I want to demonstrate something that's, you know, how fast you can get a lot. You can get a lot of point data like six months ahead. For example, for the weather, climate focus, okay, long term focus. And you can choose all the parameters in there and see Summit. And this should come back right away because the point query really is, it's very, very fast. For point query, you can demonstrate, you know, you can display your data in a table very easily like this. So you see you have the unit information, you have the value. And you can download the data in either CSV or in JSON format. So point data is simple. It can be handled by that. And I want to point out to you is here on the bottom. This is a very useful item. This is what we call the API string. You can copy the API string to your clipboard. And that one you can use later on say, okay, I actually want to construct an API called routine lay for my, for my code. So you can do that. What you can do is you can just put in the address. Just attach the address here and add to the right before the query string. And then this one you can, you can call in the API code. I'll get to that. And you can copy it and also you can send to query page. What this does is it fill the, you know, the conditions that you just put in. They don't have to redo it if you want to modify something. This is make it very convenient. Okay. All right. So point query is very straightforward for polygon. The polygon here, there are three different group of polygon, personal polygon. These are the ones you uploaded yourself. So I'm going to show you how you upload your own polygon. It's under query. They say add area of interest and say personal. And you can see where all my polygon came from. You can say add. Let me just say a test. I want to add a test. And right now we actually only accept KML, shape file format. So for example here, I can pick, pick either one, anything, right? So I can say Italy. Okay. And they add. So this shape file just got added. And then when you go back to submit new and your personal polygon, it should show up. Yeah. So test. So basically it showed up. And of course you can also do group. What I shown you here is in my case, I don't see any group. When you were adding personal polygons, you can, you can actually decide to say, okay, I actually want to share this with the group. For example, this one I added. Here I share with group. And this one will show up in everybody's in the group, under the group. So that's a way of collaboration with people. So I have all my polygons shared with the group. So I think everybody should have these things set up a polygon under your group. In my case is in under personal. So the third one that's very, very useful is in the repository. So we actually uploaded all the shape file for all the states in United States. So for example, you say USA, Alabama. Yes. And you can query any state name, just start typing and it will show up for you. And you can use any of these to do your query. So for example, the first thing is I actually have an interesting area in Florida. It's called DeSoto County. And it grows a lot of oranges. And I interested to look at the current orange, you know, the conditions. Of the orange, orange yellow bombs. So what I did was I choose for the time here, you can do an interval. And for me, I'm only interested in the latest. So the day here, it doesn't mean the time need to equal to this time. It means the time that's closest to this time before, before this time step. For interval, it's the same way. So you start a time and then end a time between these two. Okay. So I only want to get one file of NDVI value. So this is under satellite and product 13. There are two satellite equal of their equivalent, Aqua and Terra. And I will say, okay, let me take a look at the NDVI value. But you can add a condition. For example, I can say, okay, I want to take a look at just the orange bombs. So you can pick historical crop planting. The crop is equal to 212 is for oranges. And submit the query. And you'll see this is a new query. We just submitted because it's initializing the query job. Okay. And we can go back and take a look. I just want to see it. Okay. I'll show you. One thing that's very useful is once this job name is created, you can select it. And then click the little edit job icon. It's a pencil button. Okay. And then it show up. It basically lets you, you can put in a nickname. Say this is orange. And the same thing, you can copy the API string. Okay. So I'm just preparing myself for the API section here. So you take a look is the query, the type is polygon, interval, deep layers, et cetera. And the filtering condition is this layer equal to 212. Okay. And then you can send it back to the query page to modify your query, just like I mentioned. Okay. So it actually is ready. So it's very, pretty fast. And here to click here, it will show you the, basically show the layer. So this area has lots of orange fonts. And you can click on the, on here, it actually happens to be orange color. That's pretty fun. And if you click on the timestamp, basically that's a file name. The color scale will show up. Right. So if I click on the orange area, it should, it tells me it's 212. This one, because they are like 255 different crop. That's why the scale bar we actually specifically made a scale bar for it. So it can show up. I think 212, yeah. It's about the orange color, orange color. Yeah. So, and we also, we actually query NDVI. This is normalized, different vegetation index. It's about the most, you know, useful parameter for vegetation index. That's, you can gather from satellite images. So if I click this one, it actually shows the NDVI of the orange field. So you'll notice that, you know, this NDVI is not continuous. It doesn't show any other places. If I click here, it only show you the area that orange is planted. That's how we do filtering and joining different data layers. And this can be very useful because you can monitor over a long time period historical and take a look at the NDVI historical data and learn the, you know, each year's yield versus the NDVI how it grows. And you can even add all the historical weather conditions. Okay. So that's the first one. And by using that trick, we actually can do something very similar. For example, I have just tested one of the query is for cornfield in Iowa. So I can just say edit job. And this one, I want to send it to query page. So I don't have to repeat this definition. Say spatial coverage, this is Iowa. Be familiar with U.S. geography. And this is the day. I think I was interested to take a look at last year's corn around July timeframe. And we are using the same thing, the NDVI value for modis. And here the crop is equal to one. That's where corn is. And you can submit. And I should give you the same results at the corn NDVI. So what I'm showing you here is the crop is corn. The crop mapping is very high resolution. I think it's 30 meter resolution. So as you zoom in, you will see a lot of details. There are a lot of corn fields. And then you can take a look at NDVI value. For the corn field. So again, you can learn a lot about what happened for the corn farms year by year by accumulating all the data. All right. The other one that I think I'm interested to show is a composed query in Kenya. This is for agriculture. So I do the things I want to edit the job. Send it to query page. So I have the exact same condition. So the Kenya is the polygon I chose. And the timeframe is for the weekend, for the past weekend. And then there is the weather. Because I was interested to take a look at the precipitation rate of Kenya over the weekend. And I want to filter it with another condition is reference evapotranspiration. And it's the ECNWF based model. That's greater than five. So I'm interested to see the area that there's a lot of water loss on the plan. And at the same time, there's no precipitation. I can submit this one. And that's the new one we submitted. I can show you quickly how that looks like. We'll check the life query a little later. So this is, you know, I think every three hours. Take a look at the precipitation rate in Kenya. And you notice that some area didn't have anything because that's the area that evapotranspiration is less than five. So if I go to the next parameter, evapotranspiration, then I can show you, prove to you, yes. Everything shown here has higher than five millimeters per day water evapotranspiration. So another, I just, I also want to show you how to do, how to define an area query instead of using a predefined polygon. This is even straightforward, even easier. So for example, I'm interested to take a look at the area in Canada. That's the area this year, I think you may, may first try the awful forest fire. And we can take a look before and after the forest fire for this area will happen. So I found the area and go to the area definition. And I'm just going to make a rectangle selection. Okay. And then that will define the area for you. And then the date, because I know we were looking at around May 1st, before and after May 1st. So I can do is, okay, I will do April 1st till the end of May, maybe till the beginning of June. Okay. And then we can take a look at the satellite. And EBI, of course, is very important. We'll see the vegetation index before and after. And we can also take a look at near infrared. This actually can tell you when it's burning, the infrared is going to be high. Right. And also the red, we can do both. Okay. And submit. Yeah. So this one is working. And I think the previous ones, I just want to show you. Yeah. It's working as planned. Yeah. So that's how this works. Okay. I think we have more coming back. Yeah. So while it's spinning, it really is just taking the data and generating the geotip file so it can render on top of the map. Visualization. And in our case, visualization is not the core of the project because there are so many really awesome softwares out there that can do GIS visualization really well like QGIS3. And so we encourage our users to download the data. These will be saved geotip files and use that to do first analytics and so on. Yeah. So that was our call. Very well did. I just want to make sure we got, I show you all the ones we did together. Okay. And the new one. I think that for McMurray, the forest fire. So this is before the forest fire. This is end of March, beginning of April after the forest fire. So you can see pretty clearly where they, so this is just, sometimes there is cloudy conditions. So this is the red spectrum. I think the one, that one actually shows really where it's burning. It has a much higher signal, I think. Yeah. Higher red signal. And near infrared should portray a similar story. Yeah. So this is before it looks quite uniform. And then after this area. So NDVI will tell you even clear story monitoring. So this is April 30th and May 16th. Look at this large burnt area. Okay. The next one I'm going to show is for a large region. I just want to show how powerful this can be. For example, for China, it's a continent of China. It's really a huge region. And we can take a look at, say, one month from now for nine days. What's the climate forecast is going to be like? So for example, I'm interested to take a look at precipitation rate and then the temperature, ground temperature. And then we can submit this query. And I can show you, you know, the sync query I just did right before I start the session. This has a lot of file because a lot of files because it's every six hours. So every six hours or 10 days, it's a lot of data. And it can be done actually pretty straightforward, pretty easily. So this is a temperature profile. You see how many files we have. And then this is precipitation. And you can see how the precipitation moves every six hours in the region. So the last query, see, that's already come back. So it's amazing. The last query I actually want to show you is something very interesting. It's a drone data. So satellite a lot of times has cloud problem, then it can block the image. And also it's so far away. I think the highest resolution satellite is three meters for some private companies. And the publicly available one, the European Sentinel satellite is 10 meters. That's about the best. But for drones, what we can do with drones is it's another level of resolution that we can achieve both spatially and temporary. So here is, so I actually want to do the same query just to show you what this query is about. Okay, we go back to the query page. And this is the location. Actually, that's how I find the location. So we experimented with drone data that we collected ourselves in our research center. So we are in New York, outside the city. Pretty easy to find because we are on the reservoir. So we have a very special building. It's a curved shaped, a beautiful building. And you can define the area very easily, like I mentioned before, use the area to and just use selection. And what we are doing here is I just want to take a look at the latest data we have collected by drone. It's called IBM drone data. And it has a spectrum called red spectrum. And we want to compare it to the satellite data just for comparison purpose and also compare to the red spectrum here. So that's what you'll see too in some of the query. And what you'll get back is data like this. So this is the research drone data. The drone, you know, the image is, you know, a tiny little bit image. This is about like a couple hundred pictures stitched together. We have automatic program that can do it. And you can see the building and all the cars. Actually, the resolution of the drones is about two centimeters. It's actually higher than the open stream map that the rendering map we can let us do. So it just offers something that's beyond our, you know, regular satellite resolution. It's much higher. And for comparison, of course, I have a medium resolution satellite here. It's modest with land set and with Sentinel. The resolution will be much better, but still nothing compared to the drone. And this is how a 250 meter satellite image look like because we're just only a few pixels big. Basically, something like six pixels. Okay. Yeah. So I hope that gave you a pretty good introduction on how to use our user interface and how to use the jobs page. The jobs page. Okay. I think I forgot one thing was when we have, for example, when we have a picture, you can also change the color table. Remember the color table we actually talk about. You define it. You can choose one and then the color will change. And for some of the data, it's, it's a can be very, very useful to distinguish features. Okay. All right. So next section, we actually going to talk about API. So I like to use a little tool called REST client. So remember we, we actually saved some of these. Okay. So it's very simple. You just copy and paste the API string plus the web server and send. And of course, it will ask you for authentication. That's the same as your username and password. And okay. So this is the results. It comes back. And there's a little bit, you know, JSON. It's the JSON format. The web browser is very slow today. Okay. I guess, yeah, because it's a lot of data. So that's, that's just how basic API call. And then you can, of course, you can convert this into a curl command or W get command in your code. Very easily. So this is a point query. And point query, you can actually also make it a CSV. I think it's called type format. Let me see. I have to check. So this is also in the, in the menu. Okay. Yeah. Yeah. Some, some of like uppercase, low-case, I probably messed up. Okay. Okay. So the next type is, is how do you deal with a polygon query? Because it's not going to be like this. It's going to be a lot of data. And it's going to be packed in a geotape format. So I'm going to copy this. I'm just going to take this part and paste. And when you do copy and paste, be very careful because windows sometimes will add, you know, blank space onto your string. So be very careful. That can cause a lot of errors. So this is good. Okay. The type is polygon, this interval, this data layer, and then this filtering condition. And then we'll say sent. And what this get back is it gave you the very important is a job ID and also give you the status of your query. And once the query is done, you can access the status very easily. I think it's called query jobs. Yeah. Very jobs. And I need to copy this. Okay. All right. And then it says the status is running. I think it should be done very quickly. And then once it's done, you just need to put in download. And of course, this doesn't work in the rest client. I think the method should be, I don't know. But if you put in the web browser, it will let you download. But I want to take a look at the status first, make sure it's complete. It's still running. So let's just give it a couple minutes. How do you want to go back to? Yeah. I think it should show up in the, yeah. I think it should show up in the, is it? This one, 34747192. Okay. It says succeeded. Okay. So now I put in here download and say go. And it just say tell you save the file. Then you can save the file to your computer. And you have, you have all the geo tips. Yep. So that concludes the tutorial for today. And then you can see, yeah, actually in your user interface also show it's available. And you can download now. Yeah. And let me know if you have any questions and welcome to pass.