 Okay, Emmanuel, the floor is yours. Thank you. So, I come from the European Center for Medium Range Weather Forecast, and we have a pretty large archive of our forecast. In fact, it is the largest in the world, and so extracting data from there is quite tricky. What do we do? We are an international organization financed by 34 member states, and we perform operational services, namely weather forecast, and we support the national weather services in the exploitation of our data. We do research in this field, and we also provide some Copernicus services, so free access to some of our data and some data that we compute, especially for that. Obviously, this is not just computing, we also have to acquire quite a lot of data. In fact, weather data for us are two main categories. One is made by your observation, so we collect data from satellites, from radar, from weather station, and whatever we can fetch, and we archive those data in our system, and then we use those observations to create an interpolation of the status of the atmosphere. Then from that interpolation, we start our numerical models that are couplet models. So, we have an ocean model, and a sea ice model, a land model, and an atmosphere one. Obviously, we are more interested in the atmosphere. This model is producing a simulated set of variables for each cell of the hour grid. Obviously, the variables are temperature, pressure, windspin, and direction, humidity, and so on. We are interested in both 2D fields, mainly on the land surface, and 3D fields like the wall atmosphere. The computational cost is linear in the number of cells, so we have to somehow optimize the data grid that we use to reduce the computational cost. In fact, now I will forget for a while, the observation I will focus on the model out with the part that we really are required to archive and assess quickly for our users. Again, the grid. Instead of keeping a regular lot long grid, we optimized a little bit the computational cost by using a grid that is decreasing the number of points as we approach the poles. We call the grid the reduced Gaussian octahedral grid, why octahedral? Because it's essentially based on an octahedron that is just outside of the earth, and so we can somehow alph the number of points that we use for describing the each layer of the atmosphere. Obviously, this is really helpful from the computational point of view, while all the geometric computation are somehow affected by the fact that we have to interpolate from this grid to the one that our user wants. Another aspect is the z-level. We cannot use a constant z-level when we have a mountain. Obviously, we are not interested in the atmosphere under the ground, so we have the layers, that are following the shape of the earth, so the digital elevation model up to a certain pressure, and then we go on constant layers. So even on z, we have to interpolate in a somehow custom way. Having those constraints, we designed a system that is able to scale with a resolution. So the idea is that we want to provide higher resolution simulation for our user, but usually increasing the resolution is quite costly. Doubling the resolution is usually costing us eight times the computational power. Two for the dimension, one because we have to reduce the time step to get the accurate calculation. So the impact on the computation and on the output file is pretty heavy. Right now, we are running our system at nine kilometers per cell. So essentially, every cell is 81 square kilometers on the global simulation. These generate layers that are 6.6 million points, roughly, and so each field is stored in about 50 megabytes. The problem is that we store 137 layer for each variables, and we perform 51 different simulation every twice a day, to be honest. So essentially, we generate millions of fields every day. So our archive is increasing nearly 200 terabytes per day. So in five days, we had a petabyte. So a little bit more in detail. We perform every day, twice a day, so with two different timing at midnight and noon, a simulation at nine kilometers. Then we perform an ensemble of 50 simulations at 18 kilometers of resolution. Again, twice a day. Then we have a lower resolution, but extended in time simulation. And then we have also quite a lot of research activity that is also generating new data sets. This is the amount of data that we distribute. And in the last few years, the amount of data has been increasing almost exponentially. And this is our forecast. So right now, our simulation that is performed in one hour is generating roughly 70 terabytes of data. So it means that we are writing to this roughly 19 gigabytes per second. And this is our major constraints. Our model can scale way better. We theoretically could already produce this amount of data, but simply we cannot store it. The IO that we are using today, that is a parallel system with Laster and so on, is not able to cope with that. And we are working to improve on that size. But in any case, we are committed to increase to five kilometer resolution by 2025. So we are still working on that. Our computing facilities, we have a redundant system with two supercomputers. We already signed for better ones that are going to be deployed this year. But right now we are using those that are still in position 42 and 43 of the top 500, so not too bad. But again, the bottleneck there is the Laster parallel system. Then we have also some resources, some cloud resources for disseminating our data. One is under the umbrella of the Copernicus services and the other one is still experimental, is a European weather cloud that is useful for our member state to exploit those data close to the data source. So instead of fetching the data from our mass, moving the data in their own facilities and performing the simulation there, they can move the computation close to the data and hopefully reduce the overall latency. And the most interesting part, our archive. Right now we have 300 petabytes. Obviously, we cannot keep everything on disk, so we have a large tape archive, but we have also some nice caching policies. So essentially, only 4% of the requests are hitting the tapes. All the 96, all the remaining 96 are either performed from an object store that we designed or a disk-based cache. Again, we are adding nearly 250 terabytes per day, so four, five days are an additional terabyte for us. And also, our archive will meet the capacity of our Oracle tape libraries because the four tape libraries that we have are going to store up to 370 petabytes, so we have to extend it somehow. And in any case, we are going to move all the archive in another computing center during this year, so we have to manage carefully all those data. Okay, quite a lot of data. How our user requests the bit they need. They give us a user request, we have a query language designed for that. They can specify the number of levels they want, the sum of the parameter, temperature, humidity, or whatever they need, a range of dates because they may be interested in simulating the evolution over time, and also the accuracy. We store a description of the atmosphere each hour. So they can even decide to sub-sample. In this case, they are requiring 10 days of simulation with a file every three hours. And also they can specify a regional domain. Usually our services are interested in downscaling on a special area, so we provided the simulation worldwide, and then they focus on their country, their area. Okay, we can easily split those requests in two parts. One is related to the hypercube data access in the sense that we consider our data as filling an hypercube with the several dimension that are the data, the level, the variables, and so on. And we have to index those data according to that. And then a geometric query, up to now a box but probably something more interesting will be arriving pretty soon. Okay, how can we cope with the hypercube data access? We have a domain-specific object store that we call the FieldDB, FDB, so each data bit in our FDB is a layer, so is a field described in the atmosphere. The model is writing directly to the FDB, and this is also required to support the throughput that we need from our model, in the sense that the parallel system cannot guarantee the 19 gigabytes per second that we need. So we have this cache that is supporting the IO operations. We are also adding several kinds of different backend to our object store. Right now, again, in operation, we have a POSIX file system, and we are adding something really fancy using NVRAM. And in that case, we can reach 100 gigabytes per second. But we are also considering some cloud-friendly layer like a set. Set is still not performing to the level we have from POSIX, so is roughly four times slower, so we are still working on that. In any case, the object store is supporting our hypercube queries, so all those parameters are selected from an index, and then we just hit the disk for loading the field with a length and an offset. We get nice properties like is fully acidic the system. All the data are fully flushed, so we get some nice guarantee on the data that the data are stable, even if we have a crash in the computing system. Moreover, having a large computing environment, we may have trouble like a node that is dropping or something like that, so we may need to rerun a computation, and we still want to guarantee that the data are fully accessible and reliable, so we have a right once policy. So in case of a rerun, data are not overwritten, otherwise we can risk to have two instances of the data that are not fully consistent. We write in a newer location, and then we purge the space that is no longer required when the new write has been completed. Okay, the idea is that we produce the data, we store on the parallel system, and then we archive on the tapes. With FDB, all the process has been modified, so the object store is taking care of all the IO, and then is forwarding the data to the parallel system to the archive, and eventually we will provide also a cloud consumer that is really interesting for assessing the data and performing computation close to them. For the geometrical part, again, we have agreed that it is not user friendly, in the sense that it is not the regular lot alone that our user needs, so we have to interpolate to a regular one. Up to now we are interpolating the wall layer, so the globe, and just then we crop the user selected domain. We do that because using Laster, we cannot byte address the data, so we have to read the wall field and compute on that. We are all working on nicer data storage for byte addressability, and so for being able to read just the subset of the data that are required for the interpolation, so essentially the portion of the Gaussian grid that are required to interpolate just the sub-domain. This is still in work, but hopefully it will see the daylight in a few months. Again, what can we do for giving all the user access to our archive? We are developing a cloud-as-a-service system for letting the user to use our query language and real-assess the data. The system is under development and is financed by a couple of European projects, namely I'm paid by one of those that is Lexis. The idea is that the data are accessible even from externally by simply eating a REST API, all from the European weather cloud that we are hosting. Everything is a nice RESTful API with both a command line interface and a Python client that is connecting and supporting all the queries that we offer. We give through these polytop, add access to the archive and to the real-time data. There is a license issue up to now, so essentially the data are not freely available. We are selling the data for a living, but we are committed thanks to an effort of our member state to release our data, but it will take a few years. More details on the exploitation of those data from the cloud will be in this talk given by my colleague, John. Yeah, obviously for question, if we have time. Thank you very much. Any questions in the back? Okay, we have an open source version of the model that is OpenIFS. We have open access for research to the archive. The only thing that is not accessible for free is the real-time forecast in the sense that for research usually having the data with a few days of delays is not an issue, but you can easily ask for a research access and play with the high resolution and whatever you want. And the project I'm working with, we will have an open call for exploiting those data, and so you can apply for the Lexis open call and ask for access even of the real-time data. It will be time constrained for the lifetime of the project, but still something relevant and useful for playing with the largest archive that Aptona is available on the world. Okay, another one? Okay. How is the interface? Sure, sure. The possibility to get the data on a regular lat-long grid is already there in the sense that the interpolation is a parameter of the query language. So you can select the desired resolution, one degree 0.23, whatever you want, up to 0.1 degrees up to now, and essentially the interpolation is performed on the fly for you. So you already can get the data on the grid that you like without the need to implement the, oh, sorry. So you can start afterwards if you want. Sure. Thank you very much. Once again. Thanks.