 Hi, my name is Donald O'Leary. I work for the National Ecological Observatory Network as a data science educator. Thank you for joining me today for the AGU 2020 Data Help Desk. Today, we're going to go over NEON's instrumented systems data, how they're stored and organized, because it's not always intuitive at first. So I hope by the end of this short video, you'll have the tools and knowledge you need to move forward with using our IS data in your own research. So what we're looking at here is the Explorer Data Products page. Megan Jones gave a great discussion of how to use this in a previous video. So today we're going to be looking at, rather than phenology data, we'll take a look at photosynthetically active radiation data, or what we call PAR. So if we type PAR into the search bar here, we'll see we start to get some filtered results. And this first one here is the one we're interested in, photosynthetically active radiation. Something I want to point out is in this description, it says that observations are made by sensors located at multiple heights on the tower infrastructure. We'll get to that in just a second, but I wanted to highlight that before we get started. So if you click on the download data button over here, it'll take you to the data download page and just really quickly I'm going to go through this. We'll select three months of data from the Wind River Experimental Forest. Let's do September through November of 2019. Some good example data I'm used to working with here. So we select the site and the date range. We will want to include the relevant documentation. And let's use the expanded package. We'll agree to the citation policies. If you click download data, that'll initiate your download. I already have this ready here. So I'm just going to open this up and show you what it looks like inside of this zip file that you're going to download. So if you open that up, let's make this a little easier to see here. You'll see that within the zip file, we have our five PDFs for our relevant documentation that will have all the information that you need, every little technical detail about how we collect these data and how we validate them before publishing. And then you'll see three more zip files here. These are the site months of data that we downloaded. So we can see 2019, 09, 10, and 11 for September through November. Let's open one of these up and take a closer look inside. All right, we have quite a lot of files here, so I'm going to zoom in, make it easier to see, and let's go through them here quickly. If we go down to the very bottom, there's four kind of metadata files down here. This is an XML file of an EML. If you're familiar with that metadata style, you'll know how to use this. If not, you can read more about EML files through your favorite search engine. Next we have the read me file. If you open this up, it'll describe all sorts of things, including the naming convention for these very long, complex file names and the data files we have up here above. The next two are really relevant to this particular data package. So let's open up the variables file and take a look in here. So this is a CSV that contains information about the different variables that are reported in this data set. So it'll tell you which table it comes from. These par data are separated into one and 30-minute averaging intervals. Then we have the field name here. So this is the field of the particular data set within the actual data CSVs. Some of them, you can kind of guess what they mean just by the name, the start and end data are pretty obvious. But we have this nice human readable description here that'll give you more detailed information about what that variable is actually reporting. This one is really useful. The data type will tell you specifically what type of data these are reported as. The units, of course, are essential. Something I didn't know before really digging into this data product is that we're actually counting the micromoles per square meter per second as incoming energy, not just wattage or something like that, but a more raw basic scientific metric here. Then you can see which download package it's a part of. You can see our basic download package is really just the relevant variables to the data products. But then in our expanded data set, we report all sorts of quality control metrics. So anything that ends with a QM is a quality metric or a quality flag as we like to call them. This gives you an indication of just how rigorously we validate our data before we publish them and send them to you. Last one is the publication format. This will tell you if it's like a date time stamp, exactly what format we're using, are the values rounded, should they be integers. This is really valuable, so you can get a sense of the level of precision that we are reporting in this data. Next one I'm going to look at here is the sensor positions file. So before we go into this, let's talk about the PAR sensors in particular. I chose this data product because it is one of our many instrumented data products that are collected at many locations. Some of our instruments we only have one of at each site, but for things like soil arrays and these PAR sensors and temperature sensors along the tower, we have multiple locations. So you can see this is a schematic of our many different flex towers that we have across the observatory. All the way over here on the right is the Wind River Experimental Forest. This is in southern Washington and the temperate rainforest. So these trees are huge and this flex tower has to be above the trees. So it's a very tall tower and it has a number of these booms that come off with the side of it. Each one of these booms has a PAR sensor on it. So you can see that we have PAR readings for many different vertical positions. Those vertical positions are reported then in this sensor positions file. So if we open that up, you can take a look here and see the first variable being reported is the horizontal dot and vertical positions that kind of concatenated together. They're all in the same horizontal position. They're right there on the tower, which is horizontal position zero. And then we have many different vertical positions, including two at vertical position eight, which are the upwards and downwards pointing PAR sensors at the top of the tower there. You scroll over to the X, Y and Z offset. This will give you the spatial reference for exactly where these sensors are relative to our datum spike in the ground. And the Z offset you can see is the height of the sensor in meters above the ground. So this flux tower goes all the way up to 74 meters tall. And that's where these data are reported from. Last thing I'm going to do is point out the actual data files, because that's what you're interested in working with the data, right? So again, check out the README file for the exact naming convention for these data files here. But now you might have a better understanding of why we would have 16 different data files reported for this one data product at this site. We have the one and 30 minute averaging interval. And across these eight different vertical positions. So if you open up one of these, this is maybe more what you expect to see when you're looking at our data here. We have a time of the reading, the beginning and ending of that averaging interval. And then the different variables as reported in the variables file. So that's a quick overview of how these data are laid out. Of course, if you have any questions, please contact us at neon. Thanks for joining.