 I would like to start to welcome you all, first by welcoming you all to this webinar. My name is Sibrand van Beema. I am working at CloudFaro as a remote sensing data expert and as such I am quite regularly giving webinars about potential uses of the Crayodious platform. So I would like to focus today on Crayodious. So first the agenda for today, I will start talking a bit about CloudFaro and Crayodious. Then I will focus a bit on the use of Crayodious software or the Crayodious software in general. Then I would like to tell a bit more about how Crayodious is integrated on the Crayodious platform and I will spend most of the time of this presentation a live demo of Crayodious. For people that have been attending the webinar end of August about two months ago, this is pretty much a rerun of the same webinar I don't, I will talk a bit about the updates of the Crayodious system but on the whole it's fairly the same story. If you have any questions during the webinar please feel free to post those in the chat box and we have two moderators, Sylvia Nazjewowska and Marcin Bioewewski who can help answer your questions during the webinar already. And I will after the live demo also run a Q&A session and briefly go through the questions that might not have been answered or to take some questions. So I will start with my presentation. First a bit about CloudFare. Who are we? CloudFare, we can call it now a European private technology company founded in 2015, headquartered in Warsaw in Poland but as you can see on the map of Europe there's already presence in three countries besides Poland, that's also Germany and the United Kingdom. CloudFare is the provider of cutting edge computing services and focusing a lot on the European space data sector and they're focusing on the uses of big data sets like multi-pattern bytes, repositories of Earth observation satellite data and yeah it's one of the leading cloud service providers for the European space sector. Well as you can see it's growing quite nicely, there was a 66% revenue growth in 2019 compared to 2018. So the company is growing quite quickly. Then I will focus on Creeodias, Creeodias is one of the cloud computing platforms that's managed by CloudFare. There's more, CloudFare is also involved in the Wekiyo platform which is one of the other Dias platforms but also the Code Today platform from the German Aerospace Center and a few others as well. But Creeodias is one of the five Dias platforms which stands for data information and access services which is initiated by the European Commission to build European cloud computing platforms for the utilization of processing of satellite data primarily from the Copernicus program. So on the Creeodias platform there's direct access to the full archives of the Sentinel repository, Sentinel 1, 2, 3, 5P and Zoom 6 that was launched last week. Also we provide access to the Copernicus service data so you have the C3S which is the climate change data, CAMS, atmospheric data, CEMS which is I always get confused emergency mapping, CLMS is land monitoring service and CEMS which is the marine environment monitoring service. So all these Copernicus service data is also present on the Creeodias platform or at least if it's not there it can be ordered and we can make sure it's being put on the platform. That's also what I would like to stress is that it's not a static platform, we have often users that are interested in certain data sets and we can provide a way to integrate these data sets so it can be accessed by users or any users on the platform. Besides the Copernicus data we have ancillary data mostly DEM data but also data from non-Sentinel missions or non-European missions like Landsat or ESMOS easy Europeans and NVSAT as well, European missions but they're more specific as well and we also have now a few contracts with some very high resolution data providers like Jilin, KazeoSat and CompSat. So this data sets can also be made available through our platform. At the moment I checked yesterday we have 19.7 petabyte of data, it's almost 20 petabyte of data now and as there's more data recorded every day it's growing. The data can be accessed through APIs, can be used through GUI like you can see the screenshot on the right which is the Creeodias Finder where you can be more like a traditional data search engine where you can search for data and download it there as well. Any registered user can download data from our platform, you don't even need to have a paid for subscription, you just need to register. So the data is freely available to download but you can, the idea of the cloud computing platform is that you can process it on the cloud platform itself without the need to download data. So for that we have a wide range of different types of virtual machines that can be used. So we offer very scalable computing services and flexible billing options as well. Then we would like to move on to a little background, introduction to the Sennfor CAP software. So the Sennfor CAP software is the capstance for common agricultural policy, it's a European union policy which aims at improving agricultural productivity and ensuring a decent standard of living for farmers within the EU. So it's currently in a reform and modernization process, the CAP, which, yeah, Earth observation data has been acknowledged as or recognized as a very useful tool, data source for checking and checking compliancy to the CAP policies, the CAP requirements and, yeah, while also and it will also make it more cost efficient is the expectations. So the Sennfor CAP project is initiated to facilitate this uptake of satellite data for the CAP by providing algorithms products and workflows that will help to monitoring and management of the CAP. So more information can be found on the web link below. So the Sennfor CAP software package can generate a number of products. This is, first of all, you can calculate a cultivated crop type map, which is discrimination of crop types or crop type groups. This is based on, the system will need some input data to build this crop type map. But yeah, it uses a number of, well, more advanced machine learning algorithms to classify satellite data into crop type maps. So the second product that they have available is a grass mowing product. So it can discriminate grass mowing events per parcels or per agricultural field. This, yeah, let's go to the third one. There's a vegetation status indicator. So it can, this is more a maps of the, yeah, the status of vegetation, how healthy it is. So it's more traditional, I would say, vegetation indices like NDVI, but also more advanced, modern biophysical products like LAI, fraction of cover that can be created, that can be calculated by the Sennfor CAP software. Then we have the product of the agricultural practices monitoring product, which means the identification of agricultural practices of crop harvesting and plowing of grasslands, as well as comparison of farmer declarations of fellow lands or the growing of catch crop nitrogen fixing crops against EO data. That's nicely read this definition there. So it's quite a wide range as well. They are at the moment also integrating a new product, which is a tillage detection, and this will be available in the latest version or in the new upcoming version, which is foreseen to be released in January. So how does the system work? Well, it basically is an automated ingester of large numbers of satellite data, which are either Sentinel-1 radar data or optical data from the Sentinel-2, as well as the Landsat-8 satellites. After ingesting this, the system can apply atmospheric correction to the optical data sets. So they are converted from level one to level two product. And for the SAR, the Sentinel-1 data, it calculates terrain corrected backscatter, as well as coherence time series. So these are quite complex calculations. The system runs this completely automated and from these level two products, these are used to calculate the higher level products that I mentioned in the previous slide. The crop time mapping, the grassland mowing detectors, et cetera. So now we'll talk a little bit about the integration of Centfor-CAP on the CREODIUS platform. We have built a custom-built Centfor-CAP virtual machine image, which means this is basically a ready-made image of an installation of Centfor-CAP, which by any user of the CREODIUS platform can start a virtual machine with this image and have all the data, all the software installed and ready to use. The big advantage of using Centfor-CAP on CREODIUS is that we do store the complete European collections of Sentinel-1, Sentinel-2, and Landsat-8 data. We also start to have more and more users of the Centfor-CAP software that are focusing on areas outside Europe. In that case, we have a need to download some data, usually from the Landsat repository, but also the Sentinel long-term archives. Those systems are now ready in place and we can download quite quickly all the data to our repository. So this means that the data is sitting basically next to the software and there's no need to download the satellite data through the, well, quite often, slow official data hubs. We provide, as I mentioned already earlier, we have scalable storage and computing solutions, so if you need to process a lot of data, you can enlarge or make use of a more powerful virtual machine with a larger storage capacity. If you are less in need of big data processing, you can scale it down easily. Well, we provide active support with regard to technical Centfor-CAP related issues or we can forward queries to the Centfor-CAP developers if we also don't know the technical solutions completely, which I also mentioned at the last point with a direct communication with the experts and the developers if there are any specific technical issues. So this I will show also in the live demo, but I quickly want to show a bit what this Centfor-CAP image is. So in this image, we have included installation of all the necessary software to run Centfor-CAP. So besides Centfor-CAP, you have the Maya software, but also post-grad SQL, Apache, etc. It runs on a CentOS 7 operating system and we have a GUI included as well, so you can easily run it. You can visualize the virtual machine quite well. It doesn't need to be all command line based. The Centfor-CAP software is so set directly so that the access to the Sentinel-1 and 2 and Landsat data repositories are ready to use. There's no need to update the system, so it will know where the data sits. The ancillary data that's necessary for Centfor-CAP, which is the SRTM and the WBD, so the water body data that's needed for the Maya processing are included. These are quite large data sets, but they're already on the image. It's ready to be used by, for example, X2Go client, which I will show in the live demo as well, which is a way to access a virtual machine in a GUI version, so through a graphical user interface. What we offer is that we provide always the latest version of Centfor-CAP. We are at the moment updating to version 1.3, but we expect that that image to be busy, to be ready soon enough for any existing users that are interested in upgrading to 1.3. We can provide some installation scripts to run this upgrade. We also provide a flexible billing model for Centfor-CAP users, what I mentioned earlier. If you need more computing power, more storage, we can scale you up. If you need less, then we can scale you down. We have remote sensing data expert and sales support as well. We make sure that the data set is always all the Copernicus data up to date. It is basically a mirror of the official sign hub or the Copernicus hub data. All the Sentinel-1 and 2 data is there as well as the Landsat data, but then for coverage of Europe. As I mentioned, we can also order and download the data for out of Europe. We also offer customized contracts if needed. We have more information on the web page. There are two links below. Further information, please contact our sales expert for Centfor-CAP, which is Marcin Biaweczki, and technical support can be found under support at cloudfair.com. This was the introduction into Centfor-CAP and how Centfor-CAP is integrated on the platform. I would like to switch now to a live demo, which will consist of three parts. I will quickly run through installation and setting up a virtual machine with Centfor-CAP. Then I will show how a Centfor-CAP project can be started. Then I will also focus on analyzing a number of processing results, which I generated in the last few days. I will end my presentation for the moment and make it small and go to my browser. This one, so I will go first to the, wait a minute, I would like to, yeah, okay, there you go. Here I'm on the web, the home page of Crayolias. There you go, I've already logged in. That step has been done automatically. If I would like to start a virtual machine with Centfor-CAP, I can go to my cloud environment and then I will log into my console that any Crayolias user will see. Here I can set up a virtual machine, change a virtual machine, delete a virtual machine, or shelve it if you're done using it. This is where you can manage all your virtual machines. If I go now to instances here, you can see a number of virtual machines that are currently running under my account. If I want to start a new one with Centfor-CAP, I can go to the button launch instance and I can build one. I will give it a name that makes sense of the 26th of November and I will go to the next. Here, I will show you here the virtual machine images that are all available to users of Crayolias. There's quite a long list nowadays already. You can use a Windows virtual machine or Ubuntu or CentOS. Here's the one with Centfor-CAP. I select that one, activate it with the button to the top. You can see it's now here at the top. I can go to the next one. Now I have to choose a virtual machine flavor as they call that. Basically, what kind of machine does it need to run and how powerful does it need to be? How much RAM do you need? How many vCPUs? Centfor-CAP does need some RAM. It does need some storage space. These values are related to the root disk. If you need bigger storage, you can easily attach a volume to this virtual machine, which can be used for storage of data, but this is just a root disk where all the software is located on and is running from. I decide to take the HMX large, which is big enough for Centfor-CAP. It's selected here at the top again. You can go to the next one here. We need to set which networks I'm using. I take this private network and also EOData, which is very important. I connect the virtual machine directly to the EOData repository, which is the data server where all the satellite data is stored. The private network is a way to connect with the VM to the outside world, if I'm completely right about that, but it needs to be selected as well. Then we have a few security groups, which I need to set. This is very standard. There's very good tutorials on the website, which tell you now that we also have some YouTube movies that connect new users on how to set up a virtual machine. This might look very technical, but it's not that complex. There's always the support that can help you. Now we need to have a key pair, which is basically a password to access the virtual machine. I take this one, which I use often for Centfor-CAP. Then there's a few steps, which I don't really need to change anything for now. I'll go to the end, then launch instance. Now I get already on the top a message that says success. The instance is launched, and at the moment it's falling, which means it's starting up. It's being installed or built, basically the virtual machine. This takes about a minute or so and not much longer. There we go. Well, that was less than a minute. Now I can almost start using my virtual machine. I just need to assign a floating IP. A floating IP is an internet protocol. It's basically just the address of the virtual machine, which is the access point to the way to access the virtual machine from the outside. I will need to select an IP address. I think this is the wrong code. I need to make a new IP address available. It's thinking now. I expect it to be done very shortly. There you go. That needs to be connected to the Centfor-CAP IP address. It starts in one line too. Then if I associate it, it should be ready to use after this. It always takes like 20 seconds, half a minute. I hope it works in a live demo. There you go. Few. Always exciting when it works. I can use this floating IP now to access the virtual machine. I copy it. I can do that in a browser, but it takes a few minutes for the IP to be registered so it can be used through the internet directly. I've learned from the previous webinar. I will go instead to X2Go. X2Go is a software to access virtual machines or other external computers through a number of protocols. Don't pin me down on the specifics. I'm not an IT expert. But for that, I will need to make a new connection. I put as a host here the IP address that I just copied. The login should always be EO user. I can give it a name. I just do it send for CAP. Today's date. I need to also define here which key I am using to access it, which I also defined in the setting up of the virtual machine and in what kind of GUI environment I'm using. We have the XFCE installed on the send for CAP image so that that should be good. Here I just say OK and then I see there's a new connection right here. I click it, then it asks me, do I trust the host key? Yes, I do. Then it will hopefully create a connection. There you go. Connect it and then it always takes also like 20 seconds to start up. We build the connection and there we go. We have a connection to there you go to the virtual machine that it just created. So I will use a default config. So here's just the desktop Linux environment, but it looks fairly friendly, not some scary command line. I would like to show here quickly the send for CAP, how it looks like. So we can go to a browser. We have Firefox installed on the virtual machine, but you can install any browser that you would like to. For the first time it always takes a few seconds to start up. But there you go. Some standard tabs that you get. I just take a new one and I just type in local host and press enter. And there you go. Here's the send for CAP platform ready to use. I can use the login which is standard send for CAP with password send for CAP for a new installation. Any user can make a new account and change the password to make it a bit more difficult. But here we have the platform all ready to go with all the tabs that can be used for users. So that's how simple it is to set up a send for CAP virtual machine. So that's what I would like to show here. Now instead of starting a project here, I would like to change to a other virtual machine that I started three days ago and if it's sent for CAP and I've been running some processing on. So I can show some processing results and there I can also show how to build, start a new project, for example. In the meantime, we might as well just see if we can use this IP that I had before. Let me copy it again. The IP address that I, let's see if it's already registered. Yeah, there you go. So now I use this, just copy and paste this IP address to the browser and there I can access my virtual machine through a browser which is great if you're not really interested in diving under the hood for. It's great to show users how the system is working for example without having to deal with difficult setups through X2Go and so on. So that's a very neat way to access the platform. But I would like to show a bit more advanced results. So I go to my other send for CAP virtual machine that I started a few days ago in X2Go and I started up again and there you go. Connection has been restored and there we go. Here's my other virtual machine. I go to the browser again, Firefox. Let it start and then type in instead of all these samples of page localhost and there I can go into the send for CAP platform yet again. And as you can see here, there's already, I just would like to show quickly how we can start a new processing site. So if you see there's already one here defined here, NL, so it's in the Netherlands. There's some good sample data that's available on the send for CAP platform with sample data for the Netherlands, paramedic of the Netherlands with field polygons and also actual CAP declaration data. So it's very good to test and to show how the system works. So just for the sake of it, I will create a new site which I call NL2 because I will reuse the same sample data. So here I select a shapefile which defines my site extent. So this is basically just a polygon. So I will, I can now save a new site and I can edit the site. And it's from the extent file that I sent. It already identified which Sentinel-2 tiles, so 31 UFU and 31 UFT are covered by the, are covered by the site of interest, area of interest, as well for the Landsat-8, the tiles, the row and path numbers. So 198 and 199, 024, 024, yeah. So they are, they're also automatically identified from the area of extent. And here I can add a new season which I will quickly do. So I'm interested in, for example, I call it 2020. And I'm interested in looking at satellite data that's my growing season, covers my growing season from let's say the 1st of April this year through to, let's say June, it was the mid of the season until the 1st of August this year. I would like to enable this site. And here is a list of processors I can use. I can use L3B, which is the vegetation-stated data, so the LAI, NDVI data that can be calculated. And here we have the level four product that is called crop type, the grassland mowing, the agricultural practices data. You can set it that automatically process the data, but you can also run it later. So I'll leave it off for now. I just want to show how this works. I press the Save button here. And in order to start processing the data, I just enable the site again and click Save Site. And then it will need a minute and there you go. Site has been successfully modified. So that's good. The system now starts to search for data that corresponds to the area of interest and the period of interest. So the Netherlands between the 1st of April 2020 and the 1st of August 2020, it will, because we're on the CREO, this platform, not download the data, but it will create a symbolic link to the data, or direct link these days. And then it will run the different processors. So for the optical data, Sentinel-2, Landsat-8, it will calculate atmospheric correction. And then for the Sentinel-1, it will calculate terrain corrected backscatter and coherence data. So here we can see the data sources. We have a tab here for CREOdios. It tells you that we, and it's enabled. So we would like to use Landsat-8 data, which is stored on CREOdios. And here are some settings. So we take a direct link to the product. And this is where you can find it on the network. So under the EOData directory, which is mounted to the virtual machine automatically. And then Landsat and then the type of data. So this data, yeah, this is a real big advantage of using the CENFACAP platform or software on our CREOdios platform. You can directly start processing without any need to download the data. So if you go to System Overview, please don't save. Yeah, you can see it's slowly, it's still low. But in a few minutes or so, I expect the CPU usage to go up when it really starts to pre-process the data. So that's nice. This will take a day, depending on how large an area you're interested in, how long a period you're interested in, it will take a day, maybe two days, one and a half days to process all this data. And if you look at it, there is now about 880 products that are downloaded or not downloaded, but there's direct links created and they will be pre-processed or be used for pre-processing. So we're talking already about a significant amount of data. So that's that. And now I would like to go to my last step of the last part of my life demo. I would like to show some processing results. So for that, I have used the data that is processed in the first site, the NL site, not the NL2 that I just created, but the NL site. There's already a list of products that have been processed. So here you see the level 2A atmospheric correction data. There's a long list of Sentinel 2 and Landsat 8 data that has been pre-processed. Based on those optical data, we have a lot of level 3, the LAI data. If I click on one, you can see a brief screenshot. So this is quite a cloudy image that has been created. It's the Netherlands, so you expect a lot of clouds. So here you see, it's just a quick screen dump of the data. But you also see there's a list of Sentinel 1 level 2 amplitude data, which I can also click on and then you see also a simple screen dump. Yeah, the screen dumps are not fantastic, I should say, but you can see that there's data. Let's take one more. And also you can see here I have created some level 2A crop type products and also a level 4B crossland mowing products. So I would like to take a quick look at these two products. I've simply opened them in QGIS, which I also installed on the virtual machine. So you can open the data directly in QGIS. So this is an example of the crop type data that I've loaded, as well as the crossland mowing data. So I quickly would like to look at this one. You can see here there's a very large range of different crops that are grown, it's unfortunately in Dutch, but fortunately for me. But please believe me that it's ranging from potatoes to corn to wheat to pumpkin. So a very diverse range of crops that are that are fed into the crop type mapping system. So what the crop type mapping does, it basically creates a classifier which is trained with a subset of the total set of input data and then goes through all the satellite data or adds the satellite data to the classifier and then comes up for each parcel the likelihood based on the satellite data that this declared crop is actually really there. This is one of the important checks for the CAP monitoring system. So here I changed the, this is the same dataset, but I changed the setting. So all the green parcels are where the crop that's been declared is by the crop type classifier as agreed to, this is slightly that this crop grows in this field. So anything green is conform and anything red is non-conform. So you see there's a few red. I have to say here that I've only used two months of data, so this classifier is probably not very reliable. The more data you feed, the more satellite data you feed into the system, the more reliable the results are, the more accurate they will be. So I can for example click on this red field here where it's red, which is non-conform. It's also funny that, let's try this one. I need to zoom in a little. That huge edge has a tendency to select a lot. So here you can see in the attributes and then we go to the bottom and show, can I make this bigger? No, I need to just pull this a bit in. Yeah, there you go. So here the CT decal is the declared crop type. And that's a type, it's a numeric crop type. But there's also, yeah, that's expressed in numbers, but the declared crop type is crop type number 153. And according to the system, they predict that it would be crop type 3000 that's growing there. So that's why they say it's non-conform. And for any user of the platform, these are the fields to be interested in to see what's going on there. It might be that if you feed more satellite data into the system, it will change to conform. So that's something that's always useful. I just processed two months as an example. So so much for the crop type mapping. I'll make this small again. This one a bit bigger. And then I have the second product, which is the grassland mowing area. So here all the parcels, you see there's fewer parcels visible here. And these are all the parcels that were identified as grassland by the system. And the different colors correspond to different dates where the system thinks, yeah, it's quite likely that a mowing event took place on this date. So you see the different colors. Anything red is quite early. That's the third of May. And anything more towards blue, we go from green to blue is like 17th of June. So in this period, the system identified like this, it's likely there's a mowing event in this period. So you can see also, this is only two months worth of data. Although this process are working a different way. But yeah, you can get a good idea for what the dynamics are and for the cap controllers and the paying agencies, it's very useful information. So with that, I already quartered to three. So most of that time is gone already. I would like to end my live demo with this and go back to the presentation. So there you go. There you go. Go to end sharing and go back to the presentation. Yeah, so that's the live demo. Then I would like to alert to upcoming webinars on cloud.verbal.com slash webinars. We regularly update the website with the latest upcoming webinar. So I would like to recommend everybody to regularly check. We have another one coming up on the 8th of December, which is run and go together with Cloud AO and the German Cloud company. And please all the links to subscriber on the website there. So I would like to go now to my Q&A session. I will make my questions visible here and see if there's any questions that are not answered yet. So I will quickly run through them. Yeah, will the webinar be recorded? Yes, it's currently be reported and it will be placed on YouTube on our official channel. A good question. Is there an API to control send for cap with Python? Martin has responded to that, to the send for cap manual, point 2.5. From the top of my head, there is, yeah, you can run the entire system through the command line. And so that's it. There's no need to use the GUI that I just showed. You can start the system, you can restart the system, you can assign a new area of interest, a period of interest, you can do it through the command line. So in that sense, you can run it in Python. Yes, if you connect to the virtual machine through Python, which is possible, then submit a few commands to the virtual machine so that that should all be possible. A question, where is the LPIS data imported from? That data that I used is just sample data that's provided by the send for cap consortium. If you would like to look for test data, yes, you can maybe look in the inspire portal, which is already suggested, where you want to go. But yeah, if you want to test the system, you can quite well use the sample data that's on the send for cap website. I can provide a link to that later as well. So it's just, yeah, but you need to upload LPIS data to start the processing, the scroll type, and it's cross-linked mowing data. And then you need to have data that's in a certain format. It's all explained in the send for cap manual as well. And also, I would like to refer to the send for cap forum, where there's a lot of technical questions being answered. So there's also quite a wide range of information there. And you can always send us a message and then we can maybe figure it out ourselves or directly ask the developers. Yeah. So that's that. See if there's any more questions. Thank you. Yeah, are there any publications about the classification algorithms used in the crops? Yes, there are. There's detailed information on the send for cap website, which is Sylvia put the link to that as well. Yeah, they're all there. It's quite a extensive description. So you can find it on the website of send for cap as well. And that's a good question. What would be the best easiest way to combine send for cap and high resolution data on create us? Good question. I would say let's talk to us and me and marching. And then we can think of how to do it. There's ways to integrate planet data in send for cap. But we don't offer that on our platform. But I think there's definitely a way where we can figure this out, maybe in conjunction with the send for cap developers as well, because they are also working on integration of several very high resolution data sets to this. So good, good, good to hear. Okay, that's the bottom of the questions. If you're interested, there's a lot of information on the chat links as well. So if you want more information, have a have a good look. And switches, I would like to thank everybody who's still around for joining the webinar. If there's any questions, please feel free to send me an email or have a look on our website cloud for dot com or create us dot EU. And at the bottom, you also see our Twitter, LinkedIn and Facebook accounts. Not very clear. But yeah, if you just Google, you'll find us there as well. So yeah, I would like to finish up with that. And thank you very much for your attention. And hopefully you'll tune into our next webinar.