 Today I will talk, I will tell you about Krodiya's platform, so let's start. So today's agenda is the following. I will show you the some information about architecture of Krodiya's platform. I will tell you about the data that we are collecting and sharing to our users. And tell you something about the tools that we are using and you can use for browsing the data, selecting and processing the F observation products. And also at the end I will show you how to start the virtual machines and how to use it. I would try to answer your questions if you have on the chat, but if probably the best way would be if I would answer these questions at the end or my colleague probably will try to answer this question during the webinar. So first, what is Krodiya's platform? Our platform consists of servers and data storage computes and we are collecting the data from the, so we use servers to process the data so every user can start his or her own virtual machine and have access to your data that we are collecting from European, from ISA repositories. Every virtual machine can have access to block storage and object storage and you can of course make backups of the data that you process. The image here shows that we are constantly collecting data from various hubs, so SciHub, 5p, Hub, ECM, WF, ElmetSat and all this data is stored in our data repository. We have about 17 petabytes of data already stored and every day we collect several terabytes of new data from satellite products. These products you can access using various protocols like S3FS or NFS and you can also use OGC, you can set up your own WMS server and you can collect the data using the existing WMS servers as well. Now let's talk about the data that we are collecting. We have both free and paid high resolution imagery. The free data is the data collected from ISA, European Space Agency hubs. This is Sentinel-1A, 2A, 3A, Sentinel-5P, also Landsat, 5, 7, 8 and NBSat and depending on the products some of them are full archive, some of them are stored for six months and after six months they are deleted from our repository but some of the older data you can order using our tools that I will describe a little bit later. We have also contracts with Chinese and Kazakhstan companies so we have also very high resolution satellite products that can be ordered on demand. This has to be agreed with our sales department and also Copernicus services data and some other collections. This data is available from our virtual machines. So generally speaking everybody who is registered to CRODIAS can have free access to all the satellite data delivered by European Space Agency and they are available both by the browser and directly as a file system on the virtual machines. So the system is like this that we collect data to the big storage which is maintained by open source software called CEF and any additional data that you can order will be stored on a special big cache and kept for several months depending on the capacity that it occupies on the cache and you can access this data using NFS protocol, S3 protocol and access it using WMS and WMTS and WCS. This is done by our partner who is maintaining the Sentinel hub software. So now let's talk about the tools for data in the processing of observation products. I will switch my screen to the browser. So this is the main area that you can enter when you try to find information about CRODIAS. So this is the main page. You can see from this place you can access various functions and applications. Here in the menu you can see the information about CRODIAS, about our computing resources and data processing, pricing plans and here in tools we have your browser, your finder and you can access the cloud dashboard, the place where you manage the virtual resources and you can use also Jupyter Hub for your processing that you can directly make some processing on satellite products without starting your own VM. So this Jupyter Hub is the tool that you can use for free and you can in various other menus you can see this is one place that I would recommend. This is frequently asked questions. This is the repository of the various articles about how to use our resources, how to deal with some problems that you encountered during starting and maintaining the virtual machines. It is also categorized on various subjects, so for instance subjects regarding accessing your data or regarding the access to OpenStack. OpenStack is software that is used for managing all the virtual resources. So first I will show you the finder. You can access the finder from tools menu and clicking on your finder. You will see in the following dashboard and with this dashboard you can choose what kind of collection you want. I will make the screen a little bit bigger so that the fonts are a little bit bigger. So for instance if you want to look for let's say sent in two products you can choose what kind of processing level you want. You can also filter some other parameters of this product. You can choose the dates of when these products were observed. So for instance here since June to today and if I click on search then I get the list of products. If I click on this so here I can see that it is showing the first 10 products out of about 1.4 million of total results found in these dates and for sent in a tool. You can click on the on the products and see whether how it is some parameters like method data that is connected with this product. So you can browse here you can also download this product yourself. And the only thing that is that if you want to download the product you need to log in. So now I'm browsing the products as anonymous users so not yet logged in. And here in help you can also see some additional information how to access these products in some other ways like using the API. If you want to process and find more data then a browser will be not enough then you can use the API of the finder. You can also choose some selection by hand. So for instance if you look for some products that are in Africa you can use polygon selection so you can point area that you are interested in and then search again and it will show only these products that are covered by this area. So this is the main some general main information about this finder. And if you want to look for some you can also search using search phrase. So like here the example was to show wintering Quebec and it will also try to find the products that are connected somehow the metadata is filled with for these products. The next important product is EO browser. EO browser is done is made by the company that we are cooperating with it's called Synergize. You can see the logo of Synergize here on our main page and using Synergize you can also look for several collections and you can make some basic sometimes even more advanced processing for the product. So you can choose the place on the map. So for instance let's see I'm looking for some products in Canada and I can collect I can choose the collections that I want to check. So for instance in two and three and then search click search of course I chose this is showing the latest month the products during the latest month but you can also choose other dates and now I can see that it is showing the products that are covering this area so I can click on some of them and I can make some visualization of this product. It sometimes takes a little bit time just a moment I check once again depending on the the product visualization can be sometimes just a moment. Okay so this is one of the example of visualization of certain products glued together to make some bigger picture and you can choose information you can choose some filters additional filters but I'm not a data specialist so I think this more detailed presentation will be covered by other webinar and now I will show you the way to access one more tool is Jupyter Hub but Jupyter Hub you can use using being logged in so I will now use my credentials of my sample user and when I log in as this sample user I will have the service started that is every user gets containerized Jupyter Jupyter virtual machine and you can use some you can use some for instance Python scripts to access the data and to process this data this is the very simple script in Python showing how to download one of the products to the cache of this Jupyter a notebook and this will also be presented more deeply in in fact it was presented already I think but probably it can be presented once again by our data specialist okay I'm coming back to the main page of Claudia's and now I will show you access to the cloud dashboard so clicking on cloud dashboard I will be redirected to the page of it's called horizon this page is the main page for accessing the virtual machines virtual storage and as I was if I'm logged in if I can use two methods of logging in I can use either open ID connect or keystone credentials keystone is the name of the authorizing server of OpenStack and open ID connect lets me connect using my credentials which I already entered during logging into Finder so this browser already remembers my credentials so I'm logged in as John Doe now to my virtual resources and this is the starting page of horizon from here I can create new virtual machines so for instance these are some virtual machines that I I have already created some some time ago I can choose various projects that is I can create new projects that that in this project the resources are separated one from another so for instance I can choose the project that is with access to EO repository so for instance I have some virtual resources here that are virtual machines and so one way to create virtual resources is from this page and this page is a little bit more flexible than buying resources from the main page of Kradias so for instance if I want to launch a new instance I click on launch instance choose the name so for instance webinar choose source of course all this information is available on our main page in among frequently asked questions so you can you can find this information here that is the information in the frequently asked questions and now I will just briefly show you some example how to create a virtual machine I can choose the image and here are the various images that are available in in our repositories so mainly we have Ubuntu, CentOS and also Windows servers we have also dedicated software like argis which is licensed and there is also osgo live and there is also an image for CentOS 7 and Cent4CAP by the way the webinar for how to use Cent4CAP will be tomorrow so I invite you to enter the webinar tomorrow so now I can choose the image let's say I choose the image simple Ubuntu image so I click here now this step is for choosing the image so I already chose the image now I can use the flavor and the flavors are various sizes of virtual machines so let's say I choose the small one next step is to choose the network we have first of all access to the EO data and the necessary step is to add one of the private networks each project can have one or more private networks this is connected to the ability to create virtual resources on virtual network resources and this networks can be separated or can be virtually connected to each other so let's say I choose the private network external 2 and access to EO data network ports this step is not necessary to choose this one is very important because using security group is like opening is in fact is allowing some ports on IP ports so if I choose allow ping SSH RDP I can later access this machine using SSH protocol or RDP protocol and I can also ping it so I can check whether this machine is responding to ping so I can monitor whether it's working properly if I do not add this the machine will be created how I will not have access from the external world so just for the sake of this presentation I will not add this now I will show you how to add this later and in case of Linux machines and as I and as I chose the Ubuntu in case of Linux machines I can I should add also so-called keeper so SSH keeper and this keeper in my case I already have one keeper stored in this area however you can also create your own keeper or create as many keepers as you want you can add keepers that is public key and the keeper is a combination of public and private key used for SSH communication so in my case I will add this key that I already have have stored here and having this I can launch the instance the instance is being built at the moment it takes a while depending on how big this machine is and how big storage you used and having this I would make it a little bit okay I will okay so having this I can see that the virtual machine is already created and I can first of all I can log into this machine using this area so make it access it through this console through console here are here is the menu of various actions that you can perform on on the virtual machine so you can access the you can associate floating IP because at the moment the machine has only private IP fixed IPs attached and only after associated floating IP the machine will have access from from the external world and but if I want to just see how the machine is working I can access it from this browser so when I click on console I will see the place where I can make some basic actions so if I the two different users on on each Linux machine one is your console so from the browser if I login from the browser I should use your console and it will always ask me for the first login it will ask me for the for entering the new password so I have to enter the password twice sorry I see again okay so now now I'm logged in as your console user and I can do some basic administrative actions like seeing the for instance I can see whether I have access to your data so if I go to your data and make a list I can see that I have access to I can see that I have various catalogs like Sentinel 1, 2, 3 so all this data that is available on you can see on in the finder you can see it also here in in the in the console the the difficulty of using this your console is that you cannot copy and paste data to this console so it's much easier to use the accessing the Linux machines using SSH so I will come back to the instances and now when I attach I will try to attach the floating IP and now I can select IP from various pool address pools I have also already used some of them so I will try to attach one of them and this is a little bit difficult for new users to to choose which network you should use so generally you should use the private network and the differences that for instance this 10 112 is your data subnetwork and 192 is the private network so if I choose this one I will have error and I need to choose the other one so this one okay so now I can see that you can see that the floating IP is already attached to the webinar virtual machine and I can access it from my terminal so now I open a terminal on my desktop and I can try to attach the I can SSH to this address and now I use the other user not your console but your user so these are two users all this information you can find in frequently asked questions and if I try to SSH to this machine I will get at first I will not get any response and the reason is and I can check the other way so let's stop it and let's try to ping to this machine and this also shows nothing so this means that we have some problem and in our case is that we forgot to attach the security group so the way to fix this is to go to edit security groups for this virtual machine and now I can choose the various security groups this the other ones are defined for some other purposes and this one is by default attached to every project so I but it's not it's not yet attached to the virtual machine so when I click on plus on this and make safe then this security group will be attached to the virtual machines it means that it will allow pinging to this server and SSH to this server so let's check once again so let's check ping as you can see it is already responding and I can also SSH to this machine it asks me for confirmation and after this I'm already logged into to this server so again I can do a last so for instance cd your data and ls and I can see the various catalogs with satellite products so let's say that you want to use that you want to process or see in your virtual machine you want to process some product that you found in finder so let's say that this product I switched to finder and let's say that we want to see this product on our virtual machine because we have some script for for processing these products for instance so here is the here in product identifier is the path to this product as you can see it's your data sentinel 2 and some other metadata parameters so I can copy this path go to my terminal and cd control v contour shift v in my case is pasting this path to the terminal when I enter I'm already in the catalog showing this product so when I do a less I can see some information and elements of this product so this is the way you can access satellite products directly from a virtual machine so once again you can use finder to to find some groups of products for instance and then you can use some processing directly on on the on the virtual machine so so this was the information about how to create a virtual machine there are many options in in this so-called horizon so this dashboard opens the dashboard for instance you can attach new volumes to the virtual machine if you see that you do not have enough space on the on the virtual machine you can attach additional volumes you can make some backups you can make some snapshots snapshots are the momentary images stored on the disk of the virtual machine so when you decide that for instance you can free you want to freeze the state of the machine you can make snapshot and then later from this snapshot you can create new other virtual machines you can see the network topology of of your resources so if I click on network topology I will see my various resources attached to I have in this example I have two private networks and various for instance this webinar in a virtual machine which I created is attached to your data and to private network external too and you can see also your routers and you can configure some elements of of these routers so these are for instance private networks available you can create a new private network and make some additional connections between them you can also edit the security groups so for instance I will show you how the allow ping SSH RDP this is the this default group allowing access from on TCP port 22 and TCP 3389 so RDP and also ICMP so this allows pinging the server and there are some other elements which which you can you can do in this so now I will show you the the other way to create virtual machine because this was done in a so-called horizon but another way is do it directly from from creodias portal so if I'm already logged in and as here I can buy virtual machines from directly from from this place so clicking on order now first I will maybe before I will show you the place where you can check the the pricing for pricing for the machine so clicking on pricing you can see the price list and basic information about usage of the costs and and some information about discounts and here on this table you can see the various flavors so various sizes of of the machines how many cores virtual cores how big the RAM how big SSH disk SSD disk so the storage that the system operating system is is on on the virtual machine and of course most of these machines you can create with bigger system storage but this this is the information about basic configurations you can also have virtual machine with argis and so having this information you can order virtual machine directly from from this portal so if you click on order now you will be directed to the place where you can step by step choose the resources so for instance in in this area you can buy additional credits because if you want to use the machine in paper use mode you need to have some credits on your account and then each machine will be created will be charged by the time that it is running on the important information is that if you want to stop the machine for some time and not be charged for this you you should make so-called shelding so I will show you maybe first how to shelve the instance because this is quite an important function so for instance this webinar server that I created now it is in the state of running and you can see it's running for 17 minutes if I want to stop this machine I can choose various options and one of the option is to pause the instance the other is suspend shelf and you can also shut off the instance and you can delete the instance and pause suspend will stop the machine however it will not free the RAM and CPU so the machine will still be charged for occupying the computing space on our infrastructure and if you shelve the machine then it will I will show you how to do it I just click shelve no okay okay so it is it is now shelved it shows shelved instance and now status is already shelved so when I try to SSH to this machine I will not have any response because this machine is already stopped and not only stopped but it it is not occupying the place the space on our virtual machines resources so not occupying CPU and RAM it already freed and now it will not be charged for using the CPU and RAM it will be only charged for the storage it occupies and it does not occupy big storage you can see see it here and so the so the disk was eight gigabytes so in fact you will be charged in when the machine is shelved it will be charged only for eight gigabytes storage which is much less than than the using of the server itself so so this was information about how to use the machine when it is in paper use mode and also coming back to this buying resources from the Claudia's main area from the user's platform you can choose also you can buy storage you can buy also bare metal machines you can buy dedicated servers dedicated servers have additional functionality that it has special local disks attached so if you need to have some additional speed on storing and retrieving data on for instance on some local cache you can use this dedicated servers because they are both more powerful with CPU and also have this local storage which is much faster than this virtual storage used on the main virtual storage area and so let's say we want to buy some also small server here it will show me the price per hour so in in this case price per hours you can see here price per hour you can see the price for per month and if i order this i will need to perform some additional steps okay so i will not probably go through the whole process because it it doesn't take much time however we have only six minutes to the end of the webinar so probably you have some questions already i would just show you that here step by step you can choose the various elements which are similar to that that i showed you on creating the virtual machine in so-called horizon so open stack dashboard and this is more this way is a little bit more dedicated to simple configurations so so that's i think that's it for the moment thank you very much for your attention and please let us know if you have any any additional questions thank you