 Now I need your support. Who of you knows where you can look for the weather for 40 years ago? Not even a percent of the audience in attendance. Who knows what warming stripes are? That's probably just as few and probably the same people and who wants to know if there has been must know in history. Probably all of you You won't get to know this in this talk, but you get to know how you can find out. So this is what Manuel will show you how you can use public climate data. Big applause for Manuel. Hi everyone and welcome and a very good morning for you. Two years ago I was allowed to talk here about how you can use satellite data to see how mass data changes in Greenland for instance. So how ice melts How we're using too much groundwater in a few regions of the world. Usually I'm working locally. Which means I want to know what the gravity is like little g exactly here. So sun, moon and stars usually are in annoyance and I can easily easily ignore them. So from a measurement perspective atmosphere is very annoying, but usually we can change a lot about that. So for myself I usually calculate the effect of the atmosphere changing in time. So the simplest way to do that is just measure it myself. This is my little gravity meter. So I can measure how the gravity changes over time. And this is something that measures pressure, air pressure over time. So I subtract the standard air pressure and then have a delta air pressure. So by a rule of thumb I usually have three nanometers per second squared per hectopascal. Then I have the correction I need to know of a time. Well almost. Well of course atmosphere air pressure isn't exactly measured at only this one point. So say I'm measuring 1000 hectopascal and then I have lots of vapor over me. This will probably be not the same. So just imagine there's Yeah, just imagine the water and the atmosphere and the humidity. It has mass, so it has effects on the local gravity. Coefficient and so it contributes to my measurements. So you can do a level two correction to find how the atmosphere layers above me look in my climate model. For example one of my results is here on the right. The plot on the upper right shows the atmosphere height until 50 kilometers. It's separated into 37 layers. I can do this into 80 kilometers if I want to. And according to a few variables from like climate model with humidity, temperature, air pressure, I can calculate the density of the atmosphere. If I know the geometry of these atmosphere layers. So I can calculate the attraction of this atmosphere and how this draws up the gravity coefficient. Usually we have something like 9.81 meters per second squared. So we are calculating until the ninth floating point, which we can do. And if we look into this on the x-axis with time and the y-axis the height, so the upper plot, then each of the you see in the individual layers you have something like plus minus 15 nanometers per second squared and in the lower layers you have something like positive attraction and over that you have negative contributions. So we have success. I have a better correction than before. You can see this in lots of comparisons. And now you're rightfully questioning this is a very special problem you have. Yeah, that's true, but what do we do with this? I want to talk about the huge data we have. So positive data volumes you can use. I'm talking about various services where there are weather and climate data and you can use it for your own projects. So I'm talking about FTP service with ASCII data. I'm talking about API APIs for data download with two boxes to download and analyze. And I don't know if we get to the third point. There will be a little color in my presentation before this. So I'm showing what I found in this data because I'm a user of this data. So the description cannot be complete even close, but there was a nice talk about climate modeling which I can recommend it. They were much more detailed in describing this and with her descriptions. I cannot do this in this scale. And all this is also extensively documented. Everything I'm talking about is in the FAR plan. There's a document with all these services with all these resources. And if you're looking for 363 underscore climate on GitHub, you can find the services I use to download this data and produce these plots. Okay, so let's start with measuring yourself. It usually works quite good. But usually because it's quite good because it's usually working with computers with Windows PCs. So you have to look, oh, is there maybe someone else who made these data or measured this data? So there's the German forecast service. They create like their own climate models themselves. So this is just weather data which I can visualize. They will be distributed on different channels. The first one is the climate in the center portal. It's an interactive web tool. So there I can click which parameters do I want to know and then I can click onto a map which station I want to know and then I get the data. So there are also some interfaces. There are also some web services available and then you can get data. I don't want to talk more about this. So I said there will be ASCII files from a from an FTP server. So let's take a closer look at this. So there are a lot of stations with time series and meta data and documentation and Rasterized data. In this data there's metadata to used used measurement equipment and much more. So we have an overview of all the stations available of the German weather service. The service itself also provides data from Europe. 500 of these stations have contributed data in this month. So I want to talk about the air temperature and the height of two meters above ground. So when you look at these time series, they're about 25 and that's also on the very right. They are there's data available from more than 110 years. That's Potsdam. So there you can download the the data of the temperature because the daily measurements are just like a thick blue band. I created the middle so so there are also some color month and at the end of the year they're like most of and the year middle increases So you see that the median increases slowly So the usual way to display this are the warming stripes so if you looked at the things of Friday for future you see in the background blue white red stripes which So you can get this from the yearly averages So 19 So you you take the average So this was in the case of Potsdam. It's 8.7 degrees Celsius and then you have the upper time series The the average plus minus two and two and a half degrees and the below colors scale Is limited to two plus minus two two and a half degrees So in the lower lower scale you get blue deep blue scales and the upper upper values you get red So I created these with show your stripes So you go to the side and then you see that the first stripe you see is for the global global temperature Interesting is the information it tells you where the data actually comes from So let's look at as us Europe and then Germany So the build looks roughly as we had before annual average temperatures so it source annual average temperatures from Germany from 1881 until 2018 using the data from the German weather service So when you do this for a different region you get different information So there's obviously data available at the German weather service from back to 1800 something So everyone Could go there and takes some coordinates and thinks okay. How does this look like for my my favorite place? So someone had this idea before With the this following article So they created a tool where you can put in your favorite place for example Leipzig and then you get this stripe so you see So when you look at this thing at the lowest Line it's inspired by at Hawkins the sources this German weather service. That's the time series And when you click on methodic you see what I actually told you just before So that actually I put in Leipzig and not the actual coordinates, so let's have a quick look at this data data set So it's surface data, so there's a lot of variables So I show you the air temperature in this animation So in every pixel is one times one kilometer. That's the resolution of this data set and for every pixel the Average is subtracted from the actual value So the coordinates are unfortunately Gauss Krüger, that's a rectangular coordinate system. It's quite difficult to interpolate there So for the script you you can get on GitHub I change these to geographically coordinates So the surface data from from the German but the dean's just It's quite interesting So that was all just surface temperature, but I actually wanted the third dimension So we need some Climate models And that's continuous continuous So Decided on using the medium-range weather forecast European Center So that's a research service they create their own climate model So the models go back to 1997 So it's they want to try to enlarge it to 1950 So that's where I look for the weather when I don't want to look at Target show in the Copernicus program. That's the you program for Research so that you can find at Copernicus that you So the Copernicus program of us free access to three three stuff One one once one thing they offer us security, but So let's have a look at this climate change service I want to demonstrate something Welcome to the climate data store You're welcome. So and you to the climate data store and you have three buttons and we Check out on the first as the first and second one So the most actual model is Yeah, I five and it's Resolution is 25 degree to 25 degree So if you're talking about soil humidity and you want to talk about something like that then you need your five land By standard they're giving their you hourly and monthly values from today and until 105 days to go The data can be accessed with net CS and Nest you have a grip data format and then you can use mud lab or whatever you want to use for this No, I don't want to install all of this and download all this data so the climate data style helps us with this so Copernicus There's an online toolbox to use Sorry, it was already loaded On the left side you see what has already been written by people And you see the documentation of those functions Not as important right now So I'm pressing play because this takes a while this is accessible over the far plan This can take 10 to 20 minutes. I used it before The script or its own Has a part where I want to where define what I want to have so three climate variables To meet the temperatures no depth and total precipitation This is variables. I want to talk about soon. No, I'm defining Coordinates where I want to have this plot. This is Leipzig And now I'm defining the grid I want to use so Point to five degrees If I'm doing a higher resolution than data Access will be smaller. I'm taking all years all months. I can use and now I Looked at the area. This is Germany if I want to use a bigger area. This will probably take a while And if we're talking about monthly data, this can take on six or twelve hours until this is even downloaded So we see the result how it's presented here so you can zoom into this and look into segments We talked about three different variables, so now we're looking into snow depth. So this is water equivalent So just imagine the snow has molten completely. So we use the water equivalent and then we see how much meter this will be over our cell in this grid And if I downloaded the data for other coordinates as well, then I can also change the coordinates But I have only looked into only downloaded German data before so I can't change them right now So I can plot this data, I don't need to download the data myself. I only need to get a grip on this scripting language So let's look into the data catalog Because my calculations are too complex to do it in the toolbox So this is how you get also more insight into what's actually there This data is clement data store doesn't only have the era 5 data it has a few other products to For instance arctic sea roots, how does it develop for naval navigation? Here's the era 5 This is all sorts of positions for era 5 we have So this is hourly atmosphere parameters since 1970 until today Today five days ago If you are installing a few things you can download this with that and with bigger Downloads you might want to think of the download policy because they recommend to do it In monthly box because you will be in the download queue for a few days if you don't do that And on the right you see the scrolling bar it's small This is the list of all parameters which are there. It's quite a few But we do know what we want Here the things are chosen that I already used So to meet the temperature the total precipitation And air pressure For all this very other variables this has been sorted by topic Then you can click through time ranges Data points in the time and now if you had your own Python script Then with show API requests you can see How you can request this in Python to get this data set that I now clicked for If I want to do this with the toolbox that I just showed you Then you look into show toolbox requests and then this is what you get and you can just copy paste it into your toolbox code So you don't need to do this by hand And if you just want to download link you say submit form And then you can see what you wanted to download And then I see okay. I already wanted to download this Depending on the data size this can take a few hours or maybe even a day In particular until the data has been collected and until it's ready to be downloaded if you want the highest resolution Data this is on can be on magnet bands, so this can take a while Then there's a few topics that have already been prepared pre-analyzed for instance The size of the Arctic ice and how it's covering the sea So there's applications who can use live data for this because they are prepared And you see I'm always looking at the same page, I'm just using different tabs We're seeing applications So when is the not East Passage passage free of ice and you see this gets longer and longer every year What's in here Survival ability for a specific mosquito in Europe So when will tropical illnesses come into Europe, but I'm a climate researcher, so let's look into the era 5 Explorer This is presenting us with a map Or let me click into what I already loaded then you have a map This is a global map. You can zoom you can look for your place You want to understand where you want to have the climate data for the last 40 years And then on the right you see the result the first plot shows For these 40 years the monthly average of temperature the minimal and maximal daily values So if you want to travel somewhere or get to work and you don't know how the weather gets in March You can just have it visualized for you This is what we talked about and now for the long-term evolution on this place. I clicked on to Leipzig And then we see the whole time series and take a five yearly average five year average And then we see some of these tropical nights and frost days Frost days are the ones with temperatures below zero degrees and Tropical nights the ones where the nights had at least 20 degrees Within these 40 years you can see The number of summer days Increased so the average temperature has been higher than 25 degrees Celsius and the frost nights get fewer and What you can actually see even though it's quite a few only a few so far you see that the tropical nights are also increasing in number So this is what came from the climate data store in the era 5 model for different topics you might want to have For this stuff you want to program yourself. I said I had a few example scripts. You can look into those And there's a few example applications. You can also use like these So if the demo doesn't work This let's skip these slides The last thing I can also look into the missions of NASA and ESA This is The NASA AOS is the Earth Observing System Data and Information System Someone who has worked with NASA data before knows they're not Anymore freely downloadable you need a user ID to get to the earlier archives The visualization tool is world view Yeah, it's public you can just use the south code and Build it yourself on your own local platform and then download this data Now let's click for this and then you see you already have a topical segmentation With a few pre-prepared analysis Which is supposed to show you what this visualization tool can do for you in the left column you see this several layers You want to have it's something like 900 For time reasons, let's just look into California and British Columbia wildfires. So Canada and USA you see the satellite image You see the description on the right and you see the smoke how it's traveling over the continent And this is combined with several satellite missions And each point Has been identified so this fires have been identified with modus moderate spectrography something and the rest of it is you can look into yourself and the Deciding feature is that you can actually scroll through time and scroll back And so you can actually produce an animation So if you look into one specific Value that interests you so now here the fires or you're looking for aerosols or temperature You can play a few days with these The data with that's in the background here is free for download you need an account But as far as I know, this is not a problematic thing So as a summary There is a big treasure of data we can use Which is explicitly for public usage If the hurdle to get into this is not that high it's ftp it's ask a data So the biggest hurdle you have is how you want to visualize or Report and analyze this even if you don't want to program You So there's quite a few tools where you don't even need to be We don't need to program this to do the analysis Which even show you what's possible So the answer to the question has there been more snow in the past then now every one of you knows how to research the answer for this the resources on the far plan on get up you have the interesting Resources and you can talk to me all via decked and email Thanks for listening to the English translation of this talk for public climate data on a 36 KS Communication Congress here translators were was ago and Bubbler, we appreciate your feedback. Please email us at hello at C3 lingo and use C3 t as a hashtag on Twitter enough a question and Microphone Micro 9 number please on On slide 12 you had Germany with a kilometer times kilometer resolution, and there was like a red point in North Rhine, Westphalia. Do you know what that is? I don't know enough about this this data is from Observations that stations had and however the German weather dean service has been called at the time In the description of the product on the server You get further information for the relevant literature Thanks for the talk and thanks for the cool sources. I have a question to your to your field of Science so in the beginning of your slides you argued that that the effects in the atmosphere which Decrease the earth gravitation No increasing so the scale goes in both directions. How is this happening? This is Since we can't remove the atmosphere and since we don't want to do that We usually do their calculations with a constant atmosphere a normal atmosphere 1,000 hectopascal And then you can talk about what station has what kind of expected pressure So if I'm doing this probably at zero Level at sea level then I'm expecting 1020 hectopascal So this is the difference of 20 nanometers per second squared This is the attraction we get from air because air has mass of its own and this is giving pressure on to the surface And if I have a high pressure Zone then the surface Will not have will have a lot of pressure and a low pressure environment where we have Little pressure so we need to correct for a bit As I said, this is a rule of thumb So with methods as I talked about with the atmosphere measuring it we have a better correction So let's imagine where the coast or at the sea then we have a better correlation with my method With the sea level So more water is more gravity less water is less gravity Can you can you give us Can you give us a grab on the measurement of the on the of the speed? How does that how how how can you feel that? That doesn't have anything to do with earth rotation We're just talking about little g so the 9.81 meter per second squared So how Mars is held in the atmosphere So how atmosphere is distributed over the earth surface has a different has an effect on earth rotation But this is only a millisecond correction Depending on the mass distribution I grow to please I also want to thank you You said that downloading the data takes quite some time sometimes Does that is that because of the size or is it just because they need to get all the data? Yeah, the last thing Depending on how big the data collection is you want to have It needs to be collected from different data service And that's when you're able to download it so you have a bit of a time in the queue It can be a few hours until the data has been collected So let's talk about a gigabyte data set this might be a month Europe three values over the Height of the total height of the atmosphere. This is something like 10 gigabyte So you can just query it once and if you now query that with a script a second time in the morning Then the server will now know. Oh, yeah, they already asked for this. So this will be faster than Also have a question about the weights net How do we need this data which is like in the eighth or ninth? floating point number We're talking about New sensors so high precision barometer, so let's look into this precise Device and then we see the strange arc in your data And I'm seeing there's a strange arc in your data and you're more precise and then I need a more precise model for this data So So if there's a few things you can't directly measure but is of interest so something like grant water And you're talking about the levels of the swatters and the offices who are Measuring this if you don't have if you're not in luck that there's an office who is actually doing these measurements then you can do grab your metrics And then you're doing that with high precision But then you need these high correction models and then you need these atmospheric models Is there also for these ground level? Well, yes, are there public available data? Yes, there are but there's no such pages where this is all centralized it's Quite diverse from federal state to federal state, so I'm from Hanover So there is a group in the city who are Doing I'm collecting this and then you can ask them for this if you're doing research on it, but But yeah, usually this is a topic for the city you're in So this all climate model data So there's no one flying in two kilometers above and the measures this how are these calculated? Are they internally calculated? Are they before or after calculated? So each of this is five days back so the model is Calculated with the data The tip until then so until recently this has been something like 30 days ago And currently you can assume That the data will change with a bit of delay How such models are produced this was very well explained in the climate modeling talk so when the data is calculated Does that mean? So that they will be on the long-term like corrected back into the past They do They have every ten years a correction of the former model And then you usually have two mod the two models and then you take a value of the 1980s and look into the climate models and see are there any discrepancies How this is done I can't tell you in all this position if I now want to Compare my results to the ones of a colleague a few decades ago. They can just take her model And Then I have the good resources that these models are usually accessible in parallel I wanted to ask if Europe is Like leading in Giving this data to the public or are there other states or governments who And so that everyone can use these data I can't tell you this in a big I'm quite sure there's quite a few other material logical services doing this other like the German weather service So something like error 5 Which is reasonably public For the American version, I would guess that this is the data service I showed to you But precisely I can't conclusively answer this For German models, I'm also not quite sure if all of these are publicly accessible We use the RCFF because this is something I and a colleague of mine could just use In particular this model is labeled with not just commercial but also public interest explicitly So everyone is supposed to use this So if I'm doing a commercial product with this, I'm not quite sure what how this is allowed