 All right, I think we'll go ahead and get started. Good morning and thank you for coming out today. I'm Sarah Mills. I'm a postdoctoral fellow and lecturer in the Center for Local, State, and Urban Policy here at the Ford School. Center for Local, State, and Urban Policy Close-Up is one of our research centers here. And this event is actually part of our Close-Up in the Classroom Initiative. That is a project that's funded by the Provost Office Third Century Initiative to help us better integrate the research that's going on in our center with what's happening in the classroom here. And so as part of that grant, we were able to develop two different courses. This is a course on environmental policy research. And so it really links up with Close-Up's efforts in we have an energy and environmental policy initiative. And so many of the students in the class are engaging with that data. And others are doing work through the class that's directly applicable to this type of stuff that we're doing in the center. In addition to Close-Up sponsoring this, it's also being co-sponsored by the program in the environment, the School of Natural Resources Environment, and the Energy Institute. So I want to thank those partners. And she slipped out of the room. But I also wanted to thank Bonnie Roberts. My colleague got close up for pulling all of the details together for this. Because this is part of the class, there are, as I said, a number of students from that particular class here. And pedagogically, what I really wanted to present them with was the nuts and bolts of research. How do you pull together a policy research process? And while I was in DC for a conference last year, I met Reardon Frost. And he was telling me about this project and really making pragmatic decisions on what categories do you include if you're trying to measure the states against each other in terms of their level of environmental policy. And so that's really what I hope that we'll be able to get out of this in an introduction generally to his environmental index. Reardon is finishing up his PhD in Public Administration at American. Hopefully, it'll be done by the end of the year. In addition to this work, his dissertation is also looking at urban sustainability and urban policy. And I understand he's a blogger on urban policy and planning specifically in DC. So I look forward to your presentation. Excellent. Thank you very much. Thank you, Sarah. Thank you to Barry and Bonnie as well for hosting me. This is a project that I'm working on with the Center for Environmental Policy at American University, which is within the Department of Public Administration policy there. And as Sarah mentioned, it's really about trying to create a modern state environmental index. And I'm going to go over both the results from that index, but also how we're trying to construct that index, the challenges that are inherent in constructing an index. So just to justify why we're doing what we're doing off the bat, we're focusing on the state level. William K. Riley, not to be confused with Bill O. Riley, is the former EPA administrator under George H.W. Bush. He's kind of our Center for Environmental Policy's figurehead. And he gave a speech to the Environmental Council of the States back in 2015 in which he said, climate action and adaptation has been most notable and imaginative at the state and local levels. And if the clean power rule is overturned, it will continue to be. So he said this about a year before the 2016 election. And the clean power rule now is, I would say, on even shakier ground. So the state and local levels are really going to be, I think, the new focus for these types of policies, especially as they are either ignored or worked against on the federal level. So if we agree on the state level, then, why create a ranking? I understand that the students in this class will be creating somewhat similar things, but not necessarily exactly like this. But we wanted to do a ranking because it's great for benchmarking. You can see the progress that you've made or lost, especially over the years. You can see if you're in the leading states or if you're at the bottom of the ranking room for improvement. You can pair yourself to your neighbors. Everybody loves to do that. And then also, everybody loves and hates rankings. I put a logo of US News and World Report college ranking. I went to undergrad at Connecticut College. And when I was there, they signed on to a list of colleges that were boycotting the US News and World Report college ranking because they felt as though the methodology was unfair or maybe not transparent enough, whatever. And I'm sure it probably had nothing to do with the fact that we slipped rankings or something that year. But the fact of the matter is, and as I'll get into later, rankings really can rile people up as to we're really proud that we're at the top. We're really mad that we're at the bottom. Why did you have this methodology this way? So that's what I'm going to explore. So when it comes to environmental rankings, there's quite a few on the national level. This is Yale University's Environmental Performance Index. They've been producing this for a while. These are the 2016 results of environmental performance, blue being good, red being bad. Basically, it looks like a developed country, developing country map. But this is just to show that there's a lot of work being done here on the national level. On the city level, there's a lot of work being done as well. Kent Portneys taking Sustainable Cities seriously is a great look at it. There's an organization called Sustain Lane that was ranking cities in the US. There's a lot of global city rankings, for instance, from Arcadis here. But on the state level, the most recent one that we could find was the Financial Services Blog Wallet Hub, for some reason, has decided to rank the states on environmental policy and performance. But the most recent comprehensive ranking that we could find is the 1991 to 1992 Green Index, which was the Institute for Southern Studies, I believe, Bob Hall and Mary Lee Kerr, and had about 256 indicators in it and was somewhat surprisingly to us one of the most recent we could find. So 1991 to 1992 is quite old when it comes to data, not when it comes to people. But when it comes to data, it's very, very old. And so scholars have been looking at what have people been using in lieu of a good modern index. So Knisky and Woods in 2012 wrote an article that categorized what environmental scholars have been using since then, because there hasn't really been any good ones since then. So they have it in four categories, state government expenditures on environmental protection, private sector pollution abatement, state environmental regulatory enforcements, and then, of course, environmental indices. And the funny thing is that some people, as recently as 2009, including the scholars who wrote this very article, have been using the 1991 to 1992 Green Index, even though it's almost as of 2009, 20 years old. So what has happened since then in terms of indices is one that appeared in Forbes in 2007. It didn't really have its own dedicated website or anything. It was just kind of an article that appeared on it. America's Green Estates, there were six indicators listed there, carbon footprint, air, water, et cetera. The states that led in that ranking were Vermont, Oregon, and Washington. The states over the bottom, Alabama, Indiana, West Virginia. And then more recently, as I mentioned, the Financial Services Blog Wallet Hub, which gives you credit scores and other things, has decided to rank the states. And surprisingly, in depth, actually, as well, with three categories with 17 indicators total, five in environmental quality, eight in eco-friendly behaviors, and four in climate change contributions. The states that lead up on there are Vermont, Washington, and Massachusetts, and the states that are at the bottom of Montana, North Dakota, and Wyoming. So we set out, at the Center for Environmental Policy, to create a new, modern, comprehensive state policy index. And so we were trying to mirror the 1991 to 1992 Green Index, which, as I mentioned, has 256 indicators, 179 condition indicators, and 77 policy indicators. So that seems like a very high bar. But the fact of the matter is that some, and it is a fairly high bar, but some of their variables were somewhat questionable. For instance, they had things like, what percent of your state is federal land? That's not really anything the state has any control over, necessarily. What percent of your state is forest land? Some states are more forested than others. The climate, the geography, is vastly different across the country. You could argue that you should punish more agricultural states that have torn down their forests or urbanized states that have torn down. But then there's also states that are more deserted in general. So there's just some interesting things that you could take issue with. The number of motor boats was the one that I thought was most peculiar. It was in their kind of fun and lifestyle part of their index, where they're trying to see how much leisure, how good the leisure and recreation is in a state. But I'm from Minnesota, and there's going to be a lot more motor boats in the land of 10,000 lakes than there are in, say, Nevada. So some double counting as well that they were using in their methodology, so it wasn't as rigorous as it could be. And then the biggest problem, though, that I would say is that replicability is very difficult with this index. So if I wanted to just look and say, OK, I just want to make a 2016, 2017 Green Index, I'll use all their same indicators, I actually can't do that. Because a lot of their indicators were publicly available government data, but a lot of their indicators were other indices made by other organizations. And the big problem with that is that those organizations stopped making those indices. And so if I wanted to replicate it, then I have to go to the World Wildlife Fund and say, hey, remember back in 1990 when you made this index, can we make it again? So it quickly becomes an unmanageable task. So when we set out to create our index, we were looking both at policies and conditions. So policies are the things that the state legislatures pass. They funding conservation, water nutrient standards, carbon cap and trade like the regional greenhouse gas initiative in the Northeast. And then conditions are kind of the results, we would say, of those policies and also of things that the policies haven't yet addressed. But carbon emissions, energy consumption, so on and so forth. So we started looking at collecting in both of those. And our guiding principles were, especially for conditions, to normalize the data to be able to compare states to one another because otherwise comparing Rhode Island and California doesn't make a lot of sense. So the most common ways of normalizing that is by GDP or by population. We chose to do it by GDP because it answers the question, how much air pollution do you need to emit to generate $1 million of GDP? And that's what we like to call ecological efficiency. But it actually has pretty similar results when you do it by population as well, which I'll show you. We also wanted to collect data from publicly available sources that were very easily accessible, very transparent, very replicable. So we're just looking for kind of like off-the-shelf data from the Department of Transportation, the EPA, the Department of Energy. We're not really looking for anything where you would need, if you wanted to replicate it, to go to every state and say, hey, what is your data on this specific thing? And that's a problem when it comes to the policy variables. So we're a lot of those policy variables in order to see which states have these policies. You need to go to each state's legislature website, so on and so forth. So what we decided to focus on first was the conditions, the ecological efficiency side normalized by GDP to compare. And we came up with eight to nine indicators in these different categories. So in energy and climate, we have carbon dioxide emissions. We get that from the Energy Information Administration, energy consumption from the same source. Air, we have criteria air pollutants. Criteria air pollutants are six air pollutants that the Clean Air Act requires the EPA to regulate. And so we have those, which is from the EPA. Transportation, vehicle miles traveled, this is from the Department of Transportation. That's kind of a proxy variable to get it the numerous effects of the transportation sector from the carbon emissions from it, air pollution from it, runoff from it as well. And then in the water category, again, kind of proxy variables. Fertilizer purchased, we were trying to get it runoff from that, some nutrients from that. Water withdrawals, surface or ground, we can split that into two variables, which is why this is eight or nine indicators. And then in the waste variable, we have toxic release inventory and hazardous waste generation. And these variables aren't perfect, and I'll talk about that as well, how we got quite a bit of feedback on these variables. But these are the results here from the index, including all those variables that were just listed. And as you can see, just so that you can actually read it, I just put the top eight and the bottom eight here. But the top eight are mostly small Northeastern states, and the bottom eight are a little more geographically spread out, but in general, more resource extractive states or fossil fuel dependent states, a little less developed than the kind of small Northeastern states in terms of urbanization. So, and then this is also showing you per, GSP just means gross state products, so same thing as GDP, but comparing that with population and you're seeing that you're basically within one to three ranks on either per GDP or per capita. So, the challenges that we experienced with this particular index were data availability and data quality. Since we were looking for things that were right off the shelf, easily available, data availability was a big problem, and it was basically a big problem in anything except for energy. There's great data in energy. If you're doing a project on energy, congratulations, it's gonna be much easier than, for instance, water. If you're going to do a project on water, reconsider, because water is just a very interesting thing that is not very available and the data quality itself is not high. And additionally, sometimes you'll just have to use variables that are proxy variables. So, vehicle miles travel, as I mentioned, is a proxy variable. It's not directly measuring what you're interested in, but it gets at it through just saying this is the vehicle miles and then that's associated with the different effects of the transportation sector. But data quality, as I mentioned with water, one of the things that we were considering for a policy indicator was what percent of your water is in a state have you assessed? And we thought, okay, maybe that would be a good proxy variable for how good of a handle do you have on your water quality? How committed are you to addressing your water quality? What we noticed in that data set, which is reported to the EPA, is that some states would say, oh, how much percent of our water bodies have we assessed? 120, of course, 119 percent. And that is impossible. So, we asked the EPA about it and they basically just kind of shrugged and said, if we send it back to the states and say, no, do it again, we get a call from a congressperson that says, hey, stop bothering my state. So, we just kind of tossed that variable at the door. Surface water withdrawals is another thing that has data quality issues because surface water withdrawals we were trying to use as a proxy for water use in a state. And for one, there's the issue of some states are gonna depend more on their surface water, some states are gonna depend more on their groundwater, but surface water withdrawals also includes water that is being drawn up into hydropower dams despite the fact that it's then being put right back. So, we don't really wanna punish states for using hydropower in that variable. So, that's an issue that we ran into as well. There's also the issue of extreme outliers. One of the most important things you can do in projects like this is really kind of delve into your data and see what's happening in terms of the range of the outliers, et cetera. And one of the things that we noticed is that Alaska, for instance, was a crazy high outlier on toxic release inventory. So, number one on toxic release inventory is Vermont. Releases about 271,000 pounds a year of toxic releases. And Alaska, on the other hand, is at 970 million. And Alaska is also much higher than anyone in 49th or 48th place as well. And we don't really know why this is. We've hypothesized that it could be because of the mining operations in Alaska. It could be that injection wells and so on are, you know, that's included in toxic releases. So, you know, it's unclear, but this is something that we've seen in the data. And it troubles us because we're not sure if it's a reporting error or if it's just something that maybe we don't want measured. And when it comes to things that we don't want measured, we also noticed that in carbon intensity there were places like Alaska that had very high carbon emissions, much, much higher than anybody else, even when we're controlling for GDP. And I noticed when I delved into the data that the miscellaneous tier had a lot of the carbon emissions. So I emailed the EPA, and that's another piece of advice is that, you know, emailing these agencies directly and just asking them can be very helpful. And one of the things that the EPA told me was, oh yeah, in some of the earlier years, you know, like besides our most recent data, we are including emissions from forest fires in your carbon emissions. And that's not really something, you know, that I want to punish a state for, you know, because it's not from their direct industry, and it's not necessarily something that they have control over. So those are two issues that we had. And then eco-efficiency index, the feedback that we got, people critiqued the appropriateness of the variables, including like the fertilizer variable, the water withdrawals, proxy variables, people sometimes take issue with. And we also had some critiques about the fairness of comparisons. We presented this index to the Environmental Council of the States, which is an organization of all of the heads of the environment departments in all the different states. They were gathered in D.C. because they were actually considering making their own ranking. And one of the things that a representative from North Carolina told me was, you know, I would be comfortable being compared to Tennessee. I'm not comfortable being compared to New York State. So that's another thing that you can do, is you can do a nationwide comparison, like we've been doing in this project, or, you know, you can kind of break things out by region to see more specifically how people are doing against their neighbors. People also critiqued how it was kind of unfair against certain economic industries. You know, you've got small northeastern states that have a lot of financial services, which don't have a lot of direct environmental impact. And then you have other states like North Dakota, Wyoming, and Montana that are all more dependent on natural resource extraction. And this was kind of punishing them. I was also pointed out to us that fossil fuel dependent states do poorly in this index, but, you know, that's actually pretty intentional because if you're a fossil fuel dependent state, then I think that you shouldn't do well in an environmental index. So additionally, people wanted us to show performance beyond rankings because rankings don't tell you as much. So that's one of the things we changed first is that, you know, rankings are great. You can say, great, I got first place, I got second place, whatever. But it's impossible to tell, you know, what's the distance between 22nd and 23rd place? It could be very, very close, or it could be, you know, a wide chasm that is there. So what we decided to do was start assigning state scores. And this is something that a lot of indices and rankings and everything, like a lot of places assign scores, but people are like strangely proprietary about their score formulas. You know, they say, oh, we based on this, this, and this, but they never show you the actual calculation. So since transparency and replicability was important to us, we came up with a very, very simple formula that I'll show you on the next slide. But you know, this is an interesting thing as well because there are more decisions to make here. Do you base it on the average? Do you base it on the median? Do you base it on the standard deviation? When you base it on the average, it's susceptible to outliers. The median, to a certain extent, is also susceptible to outliers if there's enough outliers to bring it up. Standard deviation is something that was suggested to us at APAM, the policy conference, when we presented this there in the fall. And so we actually then ended up using that because standard deviation is based on the total distribution of the state's performance. And so it's not as susceptible to outliers and it works a little better for us, we found. The other thing that we did with the feedback is we saw that there was definitely issues with the appropriateness, with data quality on several of the variables like the toxic release inventory one or the water withdrawals. So we decided to focus on a subindex and really focus on the variables that had the highest quality data that had the most favorable feedback on them and that were also thematically linked. So we created something called the air, climate and energy subindex, which we'll just call the ACE index. And that has energy efficiency, carbon intensity, vehicle miles traveled and criteria air pollutants. So things that are all connected to air, climate and energy and things that we also had the most confidence in. As I mentioned with energy, a lot of confidence in their data quality and data availability. And that's two of the four variables here. So our scoring formula, as I mentioned, is based on standard deviation. It's super simple. I don't even have any Greek on the board. It's basically just standard deviation, divided by performance times 10 so that you can read it a little easier is the times 10. But, and then we weighted everything in the ACE index except for the vehicle miles traveled variable and our justification for that was that vehicle miles traveled is more of a proxy variable. The other ones are more direct measurement variables. So we weighted vehicle miles traveled a little less. So the advantage of this is that the magnitude between the states is revealed. The rankings don't tell you that, but the scores do. The results are comparable across indicators because the performance and the standard deviation are both within that measurement, but then they produce a score that you can compare across energy efficiency and carbon intensity. So then, just as an example of how simple this formula is and how it works, Michigan consumed about 2000 megawatt hours per million dollars GDP in 2013 and Alabama, which was just the first state in the alphabet, consumed about 3000 megawatt hours per million dollars GDP. So the standard deviations about 806 for energy efficiency, for energy consumption and then you see the results from that. The higher the score, the better you are. So very easy to understand, pretty simple. Based on standard deviation, it's not as susceptible to outliers. So you could do more complicated things that might give you more interesting results, but we really wanted to kind of keep it simple and have something that was easily understandable. So now we are closer to the results, which is something that people always like to see. And when it comes to results, there's a lot of different ways to display your results. So obviously there's tables the classic way, but as you saw earlier, I'm only really showing the top eight and the bottom eight of the states in these tables. And so you can't show all 50 states to a audience like this. So then there's maps, great for comparing the states, but less specific information known on that. Radar charts, I'll show you one of those for Michigan, which kind of shows within state performance. And then scatter plots as well can show you good trends. So on the ACE index, again, this is just the four variables, carbon intensity, energy efficiency, vehicle miles traveling, criteria, air pollutants, these are the results. So the results of the scores and the rankings there. So top eight again, mostly small Northeastern states, but we also have California now, which was not even in the top eight on the Ecoefficiency Index, is second here. We have scores that range up to 47 and down to two. On the bottom we have Montana, North Dakota, and Wyoming, more natural resource intensive states, as well as Alaska. Mississippi is not dead last, so they would be happy to see that. But it's still in the bottom eight there and some other states there as well. So this is then showing, again, per GDP versus per capita for the ACE index, not the Ecoefficiency Index, which I showed earlier. And as you can see, it's really pretty similar across the board. There is one bigger difference that I'll point out with Mississippi. They have, I think, a below average GDP, so they do a little worse in the GDP and a little better in the population moving from 46 to 39th place in that particular ranking. So this is what I mentioned earlier, but the different ways that you can do your scoring formulas. So on the left I have standard deviation, which is what I just showed, top four and bottom four here. And these are the scores that are produced when you base it on standard deviation. These are the scores in the middle that are produced when you base it on average and the rankings. The top four are the same there with three and four switched, so New York moves up. And then the bottom four are almost the same, except Alaska does a lot better. And this also you see in the median. The top four stay the same. Bottom four, Montana, North Dakota, Wyoming, holding down the fort across the board, same rankings. But Alaska moves to 38th place and 40th place when you base it on average and median. And I'm thinking that that's because Alaska's benefiting when they're skewing up the average or the median because then it's a higher, it's closer to them. The average and median are both closer to them when they're doing that, when they're biasing it upwards. Whereas standard deviation, it's less susceptible to that. So everybody loves maps. I say that because I love maps. I'm not sure it's actually true, but there's a bunch of maps that I can now show. And this is the overall ACE index. So we have, again, the variables on the right. For anyone who's interested, Criteria air pollutants, the six are ozone, particulate matter, carbon monoxide, sulfur dioxide, nitrogen dioxide, and lead is actually a Criteria air pollutant. But since the successful legislation in the 90s, since 2005, lead has been so negligible that it's been hardly reported, which is really good news. So this is what you see. Darker green is better. Gold is worse. And this is something that you know, you don't think that colors on a map is necessarily gonna make that much of a difference. But one of the most common feedbacks that we got was be nice to the states that aren't doing well. So at one point in time, I listed the bottom states as something like middling performance or something. And I got quite an earful for that. So now we don't use green and red or anything like that. It's just kind of a yellowish color if you have room for improvement. So it's something that I did not think I would run into, but it's definitely a useful thing to keep in mind. So again, you have here the overall results and then you can specify this to each variable. So I used a free mapping application used called Tableau Public. I highly recommend it and you can make these great maps pretty easily. Carbon intensity, you have a pretty similar to your last slide, but there's a couple states like Idaho doing better than usual because they've got their hydropower, that's what we think. And then the northeastern states doing well and everything. And one of the reasons that we're making this ranking, this index, is so that researchers will look at these maps and look at these results and say, oh, I wonder why this state is doing well or poorly or whatever. I wonder why these results are the way they are. And that's something that the index can then motivate is further research on these things. That's one of the reasons that we wanted to create it is that researchers would be able to use it as an indicator of their own if they want or to just get into the reasons for these different levels of performance. So criteria air pollutants is a little more interesting. There's outliers on the other side in criteria air pollutants. So there's people who are states rather who are doing really, really well on criteria air pollutants and then other states that are just doing pretty well. So the middle of the country in general is a little lower on these rankings, northeast and west doing a lot better. And again, those are all the criteria air pollutants that are included here. So then energy efficiency states in general do better on this one. But again, you're seeing the same pattern of the northeast and the west coast, but then you also have some states like North Carolina, Virginia, Georgia, Florida that might be doing a little better than you might expect. North Carolina in general actually in this index does better than you expect and a little bit in the Midwest there as well. And then vehicle miles traveled. You'll notice that Alaska and all of these was doing poorly. Alaska is number one in the country for vehicle miles traveled per GDP. And the reason I think that this is because there aren't many roads in Alaska. Intercity travel in Alaska is largely done by ferry and by plane. And so it's not like a place like for instance in Montana where if you wanna get to anywhere else in Montana you're gonna be driving long stretches. So that's what we're thinking, but it could be easily interesting to get more into that. Same thing with Hawaii. But you see a lot of the kind of southeastern states, more driving dependent states not doing as well on that. And the thing too that I should mention is that some people have registered surprise for instance that Texas is doing better on this variable. And there is a little bit of bias given to states that have very high GDPs since we're controlling by GDP. Texas has very high GDP as does California. So in general they have more room to have like higher vehicle miles traveled for instance because of their very high GDP. So, specifics on Michigan. We have the, Michigan's basically in the middle of the pack but a little better than 25th place on most of these variables. These are its scores. Again the ACE index is with weighting so that's why it's not bigger than the sum of the others. But 23rd in ACE index, 28th in carbon intensity, 31st in vehicle miles traveled, 22nd in criteria, air pollutants, 26th in energy efficiency. So this is on the left an example of a radar chart and this is something that in our experience with the feedback that we've received, states like a little more because they can see where they're doing better or where they're doing worse. But basically as I mentioned before the scores, higher scores are better and what you can see here is like if higher scores are better then a bigger shape is better and Michigan's doing pretty well on criteria, air pollutants. Not as well on vehicle miles traveled, a little better on carbon intensity and energy efficiency. And you know, Michigan and Ohio I wanted to compare and I don't want to get kicked out for this or anything but they're neck and neck and Ohio may be better in some areas but for the most part across the rankings they're one rank away usually. But this is just another way too of showing this is just stacking the scores of all the variables on top of each other which you can do because the scores are comparable across indicators and so then you can just kind of see how that looks and when the states are slightly more different, for instance, Alabama and Michigan you can see the difference better but this basically looks like the same two bars there. So another thing that you can do is you can start to look at variable correlations and so this is something that I just plugged into Stata but you could probably also do this in Excel but it's just seeing how much these variables are correlated, how much they move together and everything and so as you can see vehicle miles traveled, criteria, air pollutants across the board were always higher than 0.5. These are like pretty strong correlations, energy efficiency and carbon intensity are very highly correlated at 0.93 and that's actually a bit of a concern. We like to see some level of correlation with all of these variables. Dan Fiorino who heads the Center for Environmental Policy uses this to make the argument that when you improve one thing you're improving these other things across the board but one of the concerns is that carbon intensity and energy efficiency are so strongly correlated that we might be double counting a little bit there because energy related to carbon emissions and so that would be a concern that we could kind of explore more. So one of the other things that we can explore more is what are the reasons for the variations in state performance? So this is just again the overall ACE index map but why is it that West Virginia's not doing well or Montana's not doing well? And as I mentioned this is the type of thing we want our index to motivate research on but we can come up with our own hypotheses as well. So a few listed here, natural resource extraction I mentioned, fossil fuels I mentioned but two interesting things that I wanna mention as well is that there's what's called a race to the bottom among states and that's something where states will race with each other to cut their environmental regulations to attract businesses to their state. Vogel, this scholar called this the Delaware effect not because of Delaware's environmental regulations but they basically use like their tax regulations and cutting those to attract more corporations to their state. And so that's something that's studied a lot and studied a lot especially with Southeastern states that are trying to create business friendly environments by cutting a lot of those regulations and making it really cheap to set up there but there's also a thing called race to the top and that's where states that are already dark green are basically trying to race to be the best in all the environmental rankings and everything. California comes out with a policy and other states are trying to come out with similar policies because they're also not only businesses but residents that are looking to locate in states with good environmental policies. And then there's also the fact that occasionally the feds will come in and say, we wanna create a new policy, we're gonna base it on for instance, California's policy. And so that's a bit of a race to the top because then California's very stringent policy brings everyone else up as well. So, or it could be as much as we're conjecturing about all these other things, maybe it's just red states are not as good and blue states are better. Or maybe it's a policy thing. So we'll get into some of that here. So, natural resource extractive industries, undisprisingly the top three states on the ACE index have less than 1% of their GDP and their employment in the extractive industry. And then the bottom three states have much more than that. With Wyoming having the most 22% of its GSP is from the extractive industry. This is all from the US Extractive Industries Transparency Initiative which is led by US Interior Department. These numbers though, we're still kind of surprisingly low on the bottom three in terms of like percent of employment especially. But this is just an interesting thing that you can see. Maybe that's the reason that Montana, North Dakota and Wyoming are always 48, 49 and 50. So it could be a policy thing. So one of the things that we did is instead of including other indices within our own index is that we can just compare our index to theirs. So our ACE index is based again on performance, on conditions, so it's like the conditions that are actually occurring. Whereas the American Council for an Energy Efficient Economy has created a whole index on the policies that each state has passed, specifically relating to energy. And when we compare our index on the bottom there and the ACEE index, it's a pretty up and down positive correlation. Correlation score specifically of 0.67 and this is something that we can use to kind of confirm all results. This looks right. Our air, climate and energy index is matching up pretty well with an energy policy index. And so this is something that if we wanted to then we could get into it even further with regressions and trying to see how strong that relationship holds up. So red states and blue states and those in between, we noticed that we saw this article by Jacob Hacker and Paul Pearson in the New York Times in July of 2016, a couple months before the election and they color coded the states on a zero through four scale based on how many times they had voted democratic in the past four presidential elections. So again, this is before the 2016 presidential election. So the past four presidential elections, you get dark blue, if you went Democrat all four times, you get light blue if it's three times, purple if it's twice, light red if it's once, and then dark red if you didn't vote Democrat at all in the past four elections. So then they put this and ranked the states on a variety of different indicators. So we have like on the right, that's education, we've got on the left, income, innovation, life expectancy, so on and so forth. And the point they were making is that the blue states are pretty much at the top and the red states are pretty much at the bottom of these rankings. So we shamelessly stole this and gave them credit for it and everything, but this is what it looks like with our indicators. And so to compare that, I would argue that it's even more strongly divided with blue states on the top and red states on the bottom. And I updated the color code to include the 2016 and exclude the 2000 elections, so the past four elections, which actually turned more states blue than red because of what happened in 2000. But this is the result of that and it's probably hard to read, but this is carbon intensity, vehicle miles traveled, criteria, air pollutants, energy efficiency and the overall index all the way on the right. And as you can see with vehicle miles traveled Alaska there, the Alaska and Texas actually only deep red at the very top of the rankings there. So I also did a simple correlation with this, so carbon intensity and all of my variables with that color coded variable, just the zero through four variable. And this is what I found, pretty strong correlations across the board. The overall index is almost 70% there, energy efficiency and carbon intensity very high as well. So in conclusion, my advice especially for the students in the room, but for anybody who's looking at these index indices or wants to create their own indices, seek out feedback and listen to it. It can definitely be harsh, but it's really very useful. And I've often said with these types of projects and with life in general, never take yourself too seriously. If someone says, I'm not really a fan of that particular way of ranking it, it's like, okay, maybe they're right. Maybe it's not an insult to your character type of thing. So it's a really interesting thing and anybody can give you feedback on these types of things because everyone has kind of an intuitive sense of what makes sense and what doesn't. So I was talking to your friends or for my case, I talked to the Center for Environmental Policy's advisory board, other professors, so on and so forth. So looking into alternative approaches also really helps. Actually calculating per GDP and per capita because the most common feedback we got is, well, why didn't you just control by population? And then I can show I did and here's what it looks like. Knowing your variables and indicators is very helpful. Obviously keeping good notes to remember why you did what you did. And then being patient. I often kind of chuckle myself when there's news articles about, this is the era of big data and everything like that. But for a lot of the data that we're trying to work with, it's not that great yet. We're not really, we've moved quite a long ways but we're still working with a lot of data that has a lot of quality issues, has a lot of availability issues, everything like that. So just being patient with that and understanding that you maybe have to use proxy variables or you maybe have to use a variable that doesn't report exactly what you want it to. These are just limitations that every project has limitations. So you talk about them and you acknowledge them and you just try to do the best you can with the data that's available. So that's it. I'll be happy to take questions.