 Today's presenter is Julia Frankowski who is the government information librarian at Michigan State University. She received an MIIS from Wayne State University and a BA in history from Michigan State University. And today she's going to talk about the APIs of data.gov, which if you're interested in this topic, I'm sure she's going to mention this is actually an article in the Journal of Documents to the People, Code Orts Journal. So if you're interested in more of this, definitely take a look at that article. All right. Good afternoon, everyone. Can everyone hopefully see my slides? I should be sharing my screen right now. I want to thank you all for joining us for this webinar today on the APIs of data.gov. So at the end of today's webinar, you will be able to understand and explain what an API is, know how to navigate data.gov to find APIs, and have an understanding as to how you might use APIs in your projects. There will be time for questions at the end of today's presentation. So what exactly is an API? APIs are everywhere. They're in our apps. They're in our websites, our software, our online widgets. But unless you're a web developer, you might not really understand their purpose or their importance. So here's the formal dictionary definition. Application programming interfaces, or APIs, are a software tool which performs a particular computational function. And APIs act as building blocks, allowing software developers to create new applications without having to code every function from scratch. Technical stuff, but what exactly does that mean? In non-technical words, what that's really saying is that APIs allow for seamless access and use of information and data stored elsewhere. So this means that developers don't need to write every piece of code from scratch. They just need to write code to retrieve information or data from the API that contains that necessary information. So because the code is retrieving information that's elsewhere, it means that this data and information does not need to be stored and updated on your own database. It's stored and updated centrally. Because of these aspects, it makes the development of websites, apps, widgets, software, et cetera much more efficient because you're not doing everything from scratch. So here's some more technical information about APIs that you don't necessarily need to know, but I'm going to point it out just to be thorough. So APIs are built with a series of protocols such as SOAP, which is an older protocol and REST. And these protocols are combined with formats like JSON and XML, which are both human and machine readable formats. With the combination protocol and format, an API is born, and this API has the ability to offer a variety of functionality that other developers can tap when those developers are creating their own interfaces. In order to actually use the API, though, you'll need to know a scripting language such as Python or PHP. Using these languages, developers can write commands to access the information and data from the API. So good APIs have documentation for how to use them, such as example queries and commands, so that developers who are trying to take advantage of all that the API has to offer don't have to guess what command does what. So they would have example code of using this parameter gets you this information that's stored in this API. So that way you have an idea of what exactly the API contains and what it can do and how to write that code without having to do once a trial and error. If there isn't good documentation, it makes the whole process a whole lot harder. So why would anyone want to use an API? You don't need to download, store, or update any of the data or information in your own database. This means that every developer is using the exact same data and information. And since it's in an API, anyone can access and use that data. This is great for making sure that there's consistency across different apps or websites. And just the process of having to make sure everything is updated in all the places can be really tricky. So having it in one central location, such as an API, makes it a lot easier. You update it in one slot and it updates everywhere. APIs also help create a more seamless user experience by enabling developers to create interrelated systems. So this seamless integration is one of the reasons why we often don't realize we're accessing information from an API. An example is when you're on a store's website and you're looking for the closest store to you. You can type in your zip code and define in your store. And chances are you're using Google Maps API when you type in your zip code in order to find the closest location. So that's one example of how there might be an API behind all that, but you don't see any of that. You just feel a little search box. You type in the information. There's stuff in the code that calls to the Google Maps API, retrieves the information, generates it on that webpage that you're on, and you have no idea any of this is going on. Additionally, APIs help save developer's time since they won't have to rewrite code for every little thing. They just need to write code to access that API. So if you wanted to find near store without accessing a Maps API, for example, there would be a lot of geolocation and things like that, and it would be a huge horrible mess. So with the API from Google or any other mapping related software thing, it makes it all a whole lot easier for everyone, and then you don't have to spend a lot of time writing all this new code and everything. So now that we've talked a little bit about the basics of APIs, we're going to move on to talking about data.gov and how to get around it to find APIs. Data.gov is a clearinghouse for locating not only governmental data sets, but also APIs, and an attempt to foster a more open government. By creating APIs that the public can access, can use to access government information and data, agencies and other governmental organizations, can allow others to spend time and money developing user interfaces to access this information. So developing apps and websites, it uses a lot of developer time and time means money, and so by allowing anyone to make these apps, websites, software, whatever with open government information, it allows these agencies to go and use their resources to do other things. But that doesn't mean that the agencies and other government organizations aren't making their own apps and interfaces. It just means that other people have the option to make something. So maybe they see government information in a way that the agency may not have seen a use for it, integrating it with other resources that are outside of the government to try and make a better product that maybe falls outside the scope of the agency. So it can really add to more widespread usage of free government information. This is what the homepage of data.gov looks like in case you've never been there. You can search in the giant search box for certain topics, and this will pull up everything on that topic, or you can click on data up at the very top center of the page to give you more advanced searching options. Clicking data gives you a lot of faceted options on the left-hand side. If you scroll towards the bottom, you can select API as the format to pull up only items listed as having API as the format. And as you can see, there's quite a few data sets with API that are available, but there are some caveats that go along with using data.gov. The APIs and other data are not actually hosted on data.gov, but on other agency websites, and this means that sometimes when you click on a link, you might not get taken to the page you were expecting. If links change and it's not always updated in a timely fashion, also certain APIs like those from the Census or NASA require you create an account first in order to get an API key to use on these keys are pretty common when using APIs. The keys are usually used for tracking your activity so that the creator can see who is using their API and in what manner, which can sound a little bit scary, but it can also be good for the agency to see that the API is actually in use. So they can warn you if they decide that they want to discontinue the API, for example, or if they're going to be major updates or if the URL is going to change, they can always get back to the users by looking at the key and seeing who's actively using it. And let them know that a few changes are coming or it's being discontinued or what have you. Another huge issue is that not all APIs have documentation on how to use them, which can make them very, very challenging. It's sort of hit or miss on data.gov as to whether there is any documentation, if the documentation is really poor, if it's acceptable, or if it's actually pretty good. So sometimes you'll look at an API and there's absolutely nothing there to tell you how to use it, and so you kind of have to play around with it yourself trying to figure out what you can do with it, what exactly does it contain, what calls it and commands, it responds to that type of thing. So here's an example of what you'd see when you click on the page you find on data.gov. This is for NASA's astronomy picture of the day API. And if you click where the arrow points, in theory you should get taken to a page with information on the API. But as you can see, it takes you to a page that says you need an API key. So you actually have to go to api.nasa.gov to request the key, and it'd make a lot more sense for the link to automatically direct you to that page rather than let there a page, but it doesn't, so you just kind of have to deal with it and copy and paste. Requesting an API key from NASA is really painless. This is part of the page that you get. You just add your name, your email address if you want. You can put in your purpose, but it's not required. And then they supply you with a key almost instantaneously. Now all agencies are the speedy when it comes to supplying keys. I requested a key from the Bureau of Labor statistics a few days ago, and they still have not responded to me. So it's one of those things where I don't know if there's human review going on for those things, but NASA is very speedy, and they actually have a very nice API documentation to show you what you can do. So this is an example of part of their API documentation. It shows you the HTTP request. So you would copy that link, and that's the link that you would use to get information from, and then any query parameters. So you can use latitude, longitude, date ranges, and then you put in your API key, and this gives you an example query of what it should look like when you are writing your query. So for this example, what this particular example does is if you've ever wondered when was the last time NASA took a picture of your house, your institution, or a location that you can think of, this query would tell you, and it would actually give you a URL to view the picture that was taken of whatever location you're plugging in with the latitude and longitude. So it's kind of neat that you can figure out that type of thing. So we'll come back to the NASA API a little bit later in some of our examples, but right now we're going to move on a bit to some examples of how others are using data.gov APIs. The nonprofit organization Sunlight Foundation is those building really neat tools and websites to keep track of different aspects of government and things like that. A clear spending uses the data.gov API from USASpending.gov, among other sources, to show where money is going, basically. And so they give different scorecards, and it's really neat to play around with and to see how exactly they're using information that they get from USASpending as well as other resources to see where exactly money is going. And this is an example of an iOS for the congressional record. While the API.data.gov might be free, and the information that they're providing is freely available, that doesn't mean that developers who use the API aren't going to try and profit off of their creation. So here in this app, these developers are actually charging money to users to access different issues of the congressional record. So you can buy this one from 2014 for 99 cents and others. All for 99 cents was subscribed, which is a little ridiculous, because there's also another iOS app from the Library of Congress that does the congressional record. And it provides all the information for free on your phone. So it's one of those things where people are always trying to take advantage of reselling things that are free in order to make a profit off of people who may not realize that this stuff is freely available. And the Department of State has created the smart traveler app that adds the latest travel warnings, alerts, and country information for those planes travel outside the U.S. So once again, you update the information in the API one place and then anything that's accessing that API gets updated automatically. So you can get the latest warnings and any other similar app would have the latest alerts and things of that nature for traveling. So those are just a couple of examples of APIs in use, but there are tons more since APIs are really everywhere. Now that you know a little bit more about APIs, where can you go to learn more? Code Academy is a great interactive training website that allows you to learn all sorts of code. So here's a listing of all that they offer. For working with APIs, I recommend learning PHP, Python, and hey, look, they even have a learn APIs course. And look at that. That's one that's highlighted. One of the options is to learn how to use Python to access the National Highway Traffic Safety Administration's API. All these courses are free. They're at your own pace. You just mess around with it when you have some time. And it's a great way to expand your skill set. And another fantastic site to use is W3 Schools, which is also free and also at your own pace. And here is a listing of all that they have to offer. PHP, JSON, and XML would all be helpful for working with APIs. Learning to code is all well and good, but what if you don't want to build your own tools, but you still want to see what an API can do so you can recommend it to those who are developers, maybe at your own institution. There are some tools out there that let you see the functionality of the API and what you play around with it without needing to know too much in the way of code. So this is a great way to play with an API, especially if it has good documentation to pretty much walk you through what you're doing. One such tool called Postman. So head on over to that page. So it's a downloadable app for Chrome or Mac. I've already downloaded it. I'm a Mac user. And so this is what the interface looks like. It's nice because it saves all of your queries that you've done. On the left-hand side, you can go back and see what you've done. It also has basically the start of any command. So you have different options. And so one of your most likely to be used when you use an API are get and post requests. But mostly you're going to be using get request. So basically you're going to be asking the API, get me this information. So let's go back to the NASA page. So this is the NASA's assigned signature of the day. They give us our HTTP request. So this is where you would put it says get. And then you would just copy that link. Go back to Postman. Enter your URL. And if you just send, you'll see it gives you an error message because you need an API key, which I had already requested. This key is not any sort of authentication or anything. So it's okay if people see it. It's just basically tracking my usage. So I would click on params, which means parameters. And this allows me to start building my query. So you don't have to build some long, crazy string up here. You just have to start adding parameters to kind of build your question that way. So it's kind of like advanced searching in a database almost. So we do API underscore key and then value, paste in my key, send. And it gives you in JSON format information about the picture of the day for today. The fault is going to be today. So it gives you an explanation of what the picture is. And then it also gives you a URL. So we can open up the URL. One weird thing about this is once you click on it, you do have to click send again to get it. And it will ride you with the pretty picture. Going back to the information about the API, you can do a couple of different things. You can also add different dates. So default is today, but you can also see what pictures are for different days using this. So let's go back to our original query. Let's add in under key. We can add in date because as it shows here, date is a parameter. The format is going to be year, month, day. So I want to see what the picture was for my birthday this past year. So we can send. It once again gives us the information with text explanation and then also the URL. We click on the URL, have to say send again. And that's the picture that it provides. And so you can do this with pretty much any date. So one thing you can do with this is you can build like a little widget or something for your website if you so wanted to display the picture of the day. Or you can have it so people can put in different dates. And it would call to the API and show you whatever the picture of the day was for that particular day. So in addition to Postman, which is an app that you have to download while it's free. There's also Hurlet, which has this lovely vomiting unicorn as their mascot. But this is a browser based tool. And as you can see the setup is very similar to what Postman looks like. The only difference is you don't have to download anything. It doesn't save what you've done in the past so you can't go back and see what your previous requests were. But that's okay. So once again, we can do and we know we need to start adding some parameters because we need a key. Once again, it would be API key, API underscore key, and then we'll add another parameter for date. And let's see what the picture of the day was for the first of this year. I'm not a robot, so I'm going to launch the request. And it gives us a whole bunch of information about this particular file. Once again, the explanation, the URL, which we can open. And so this is the picture of the day for 2016. So what's nice about well done documentation is that you can, like I said, you can easily look at this and see what exactly you can do with it. You can see it's a get request. You can see the URL. You can see the date. You can see what to do. And it gives you the example query. But because you're using one of these tools, you don't have to do all of this, you know, writing the URL yourself. You can just build it basically using these different parameters. So that's just one aspect of this particular NASA API. They also have an option where you can see when was the last time that NASA took a picture of my house, for example. And it shows you how to do the request. You have to get the latitude and longitude, which you can pull from Google Maps. And then you can set, like, a begin date and an end date if you want. And then see the URL and generate the picture very similarly to what we just did with the picture of the day. What's weird about this is if you wanted to do something where you wanted to create, like, an app or something that allowed people to just plug in their address and see what was the last time NASA took a picture of their house, it would actually be a number of API calls that would have to happen. Because when we just do one request, it gives you a list of all the pictures that have been taken. So it makes it so there's a lot of different steps involved in that, which makes it rather confusing. So let's do, and it's working on pulling up all the different options to show you when were the times when NASA took pictures at this latitude and longitude. And for whatever reason it's taking a while to load, maybe this is a very busy time for NASA. I'm terribly sure about what's going on, but like I said, it would give you a list of all of the different times that NASA took pictures of your house, and then you would be able to see, to do options to view the URLs and see the pictures. So that's just some of the stuff that you can do with this. I said there's tons of different options in this, as I mentioned earlier, really good documentation. So I would recommend playing around with the NASA API just because it has so much to offer. The documentation is very easy to understand. It's not many tons and tons of pages, and it's not very technical in nature. And because they give you nice examples and everything, it makes it a very approachable API to use. Like some of the others that you may find on Data.gov where it's just kind of hit or miss. That is all that I have for you all. I didn't want to get too in the weeds with this because I know what APIs are a very complicated topic. So I'm happy to answer any questions, and I encourage you all to play around with APIs. You can try Postman, you can try Horlitz. There are also other API tools out there, but they kind of just give you a sense of what an API can do and get you an idea of what you can do with it, and if you want to try integrating it into any of your own. Widgets, software, websites, things like that. There's actually a question, Julia, while you're doing that. I was just going to read this from James. Are there any libraries? And this is for, I think for a lot of attendees, too, because there might be people who know about this happening. Are libraries building APIs into their collections? I don't know if there are any who are building them into their collections. I know here at Michigan State we have people who are working on building APIs in order to access different data sets that we've acquired from other places. For example, like ProPlus and things like that. But we're not necessarily putting it in the collection the way you put a book. We just kind of have it out there as an interface in order to interact with these different collections. So I guess in some ways it is kind of being integrated into the collection, since it's an interface to access these data sets and everything. But I don't know, it's kind of an early stage of the game at this point. I could see collecting data sets from government agents using them to offer an API as one other way to access. Yeah, definitely. I think that would be an interesting way to do this. And I think it would definitely take all of us a little bit plugging around and then playing around with it to be able to get to that point. But that would be really cool. Yeah, definitely. Yeah. I mean, why would you use API versus just giving the link to the webpage where API is already in use? Well, the API is basically just – it's structured not really human-friendly data and information by building like a widget or something to access the API. It gives it a pretty interface for users to interact with. So APIs are kind of all in the blog code that most humans look at and don't really know what to do with. And then the interface that you build in order to interact with the API gives it the pretty face. And the approachableness that our users expect. Have you had any data reference questions where accessing data via API came in useful to you or the researcher? That's a great question. I have not encountered that situation. We just hired our data librarian, so I don't know if they've had any questions. We've been without a data librarian for a while. So I'm not sure if our previous one ever had this issue or anything like that. This is just something that's kind of – I like to play around with. And I haven't yet seen questions, but I wouldn't be surprised if in the future I got a question from like a political science faculty member who needs help accessing information that would be best handled by using an API in order to get at and then having to have some sort of interface in order to access that more easily. But we have several programming librarians on staff, so they would probably be the ones responsible for doing the actual like work with building the code and the interface in order to interact with that API. My job would basically be locating an API that would be suitable for them for that purpose. You're wondering if other sites besides NASA have good documentation if you run across particularly good sites for – because NASA does seem to have a lot of assistance for our guidance for people who are trying to do this. Yeah. I was looking through a bunch of them, and many were not very good at anything at all. It was really scant and kind of disappointing. Census has pretty good documentation. And then there was one that was – it was something about food atlas, but it used ARC GIF, and it was very good thorough documentation, but it was very, very long, very, very complicated because it was GIF mapping stuff. It used census track data and all sorts of information to find out basically where there was food insecurity and things like that in their API. And it was – while good documentation, it was a lot to get through. Most of what I've come across on data.gov – one could almost say barely APIs. They may just be a dataset, and you'll look at it and it's like, well, it's just one dataset. This isn't really much of anything. It doesn't have any documentation to it. So like I said, it's really hit or miss. I would recommend you can sort in data.gov by most popular, and usually the most popular APIs on data.gov have the best documentation to look at, which is why they're popular. People are actually able to use them. Yeah. And James is completely right. It doesn't make sense to have an API for one simple spreadsheet that's never going to be updated. It's going to be the same information over and over again. It's for more complex datasets, basically. Things that will be updated frequently, things that have multiple variables in it, things like that. Well, what about pulling from different datasets as Jenny asked? You can do that. An API allows you – if you built the API, you can pull from different datasets in one place, and so you're not having to go from one dataset to another. This basically eliminates that step and puts it all in one spot. But you could pull from different datasets if you wanted to, but there's just a lot more like work. Yeah. And so there – as Jenny pointed out, that she means more like one dataset, like to pull data and then pull mapping data. So there are APIs that do pull, for example, like numerical data, and then they pull GIS data. And then they put that together in a nice API that you can access that will combine those two. And so it makes it so you aren't having to go to both places and trying to mesh those things together. The API could pull those all together. There's a question, would it be viable to use APIs as a collection development tool? I think that would be an interesting one. Let's see why you couldn't use the API in order to make sure that you're collecting all of the EPA datasets. Yeah, the problem is that it's possible that the API could go away, whereas web archiving, at least, you know for sure you're saving it somewhere. So that would be my main concern, is if you're only relying on the API, it's possible it could just disappear or be discontinued, and you may not know it. So you're kind of, since you're not downloading any of the data when you're using the API, you're just sending a call to the API, you're not saving anything. So you don't have any sort of backup with that. All right. Well, unless there are any more questions, say thank you to our presenters. You're welcome. And thank you all for attending.