 Once more, if you're just joining, my name is Charlie Warnstein and I'm a managing editor for ProPublica. Welcome to today's webinar. Close captioning of the program is available and can be enabled by clicking on the closed caption option on the bar towards the bottom of your screen. As an additional note, this session is being recorded, and a link of the video will be emailed to everyone who registered. For those new to ProPublica, ProPublica is a nonprofit newsroom dedicated to investigative journalism. You can find a link to our healthcare coverage in the chat. Today we'll be walking you through how to use ProPublica's updated nursing home inspect database. As a reminder, today's webinar is primarily for journalists, but we understand others may be joining us to learn more about what information the database offers. We'll do our best to address as many questions as we can at the end of the session. My colleagues will be joining me shortly, but first I wanted to open the conversation with some background on the database and how it came to be. So as many of you know, nursing homes are an important healthcare option for folks who have either serious disabilities or older and need additional help with activities of daily living. About 1.1 million people live in nursing homes across the country, according to the Centers for Medicare and Medicaid Services which is a federal government agency. In July 2022, there were about 15,000 certified nursing homes in the United States. Most nursing homes are certified as both skilled nursing facilities, which means they provide managed recovery period after a person's illness or injury. As a replacement. And there are long term care facilities that deliver healthcare and services or resident needs for mental or physical conditions that are not rising to the level of skilled nursing care so those are. I apologize, those are the recovery services. These nursing homes are among the most overseen facilities in the country in part because residents are so vulnerable. It is both where they live and where they receive medical care. So the Centers for Medicare and Medicaid Services, which is the federal agency which is known as CMS regulates these facilities in a couple different ways. These are regular surveys where they go in about once a year to look at the conditions in a facility. And the second is based on complaint investigations. The reason I'm sharing this information is that our tool which you will hear about in a couple minutes is based on the information from these inspections that CMS conducts. I wanted to just take a minute to show you the resources that are available from the federal government on their website so you can understand what is available on the federal site and how what we offer is different than what you can find on the federal site. So the federal government runs a site called here compare and I'd like to share that. I'm going to share my screen so you can see, see that site here. So the care compare site enables you to look up either a nursing home by name or by location. And in this case, let's let's just type in Michigan. It's where I'm from so I like to use as an example. So you type in Michigan and you did search. And you'd see here that there's 300 nursing homes in Michigan. And it is, it looks like it's alphabetically searched for you. You can further some look for by a county by facility name, you can sort it in different ways by an overall rating, or by other ratings and so in this case we're going to be talking about health inspections so we can say we want to sort it by to look for facilities that are much below average. And then we can apply that. You could see that it's going to pull up 82 facilities that is labeled as much below average, and you can click into a facility. And within the facility, you can click into view their inspection results. And within the inspection results, you can click into view various reports. So this can be difficult if you as a journalist are trying to compare different facilities in your community, you know it can be tough to understand how these actually stack up to one another. And we try to make that easier in our tool, which again will be showing in in a couple minutes, when you click to view one of the facility reports, you can see here. And it will pull up a PDF of a particular inspection report here as well. So I'm going to stop the share for a minute and CMS offers a variety of additional data points that you can access with respect to nursing homes. So they make their nursing home data available online, and you can find that we're going to put a link in the chat to that I'll also share my screen. They make available data files on nursing homes and rehab services and you can see, you can download individual files from this menu and again we're going to make that link available on the screen, but you have to know what you're doing with data. You can also go across different data sets and it can be a disincentive for sort of diving in and understanding where to start on a particular facility or in a particular community. There are also a series of zip files that are available on our site, which enable you to actually get the words that comprise the CMS inspection reports. And those are, again, require you to have some knowledge of how to use Excel and look across files. I wanted to mention one other site that I think is helpful to have as a context and a backdrop before I turn it over. And that is that the US Department of Health and Human Services Office of Inspector General inspects and does a number of reports on nursing homes, and you can go to the Inspector General's website and find all of their reports that are available. I think it's a great helpful resource in terms of thinking about issues affecting nursing homes overall and context to put your stories in if you're looking at a particular facility, for example. Okay. One other separate note. If there are questions, there is a Q&A box at the bottom please, please feel free to drop your questions in there. I'm going to turn it over and introduce my colleague Ruth Talbot who will be taking us through a walkthrough of our Nursing Pumins Back Database walkthrough. Hi everyone. Thank you for joining us. So I'm going to walk us through kind of a live demo of the tool. But before that, I'm just going to share a few quick overview slides. Let me know if you can't see the screen. I'm not entirely touched on this but this tool is kind of aimed right now at people who want to deep dive on these inspection reports and the individual deficiencies within them. So deficiencies within an inspection report are just the individual issues or failure to meet standards of care found in those inspection reports. So our tool kind of focuses on searching and understanding these reports. But we also hope that it is accessible for people who are maybe more casual consumers who are thinking about putting a loved one in a nursing home, and just want to kind of do a deeper dive on that home. I'll mention this in a second, but we are actively investing in nursing home inspect. And so we are one of the things that we are hoping to do is to make it more accessible for that kind of average consumer who may not have a ton of background on what these inspection reports mean. And then just a few data notes before we dive in. So our data only includes Medicare certified nursing homes. So if there's a nursing home that isn't in this program, it may not appear in our data. And then also, there are some kind of longer term care facilities that you might kind of get confused with but if it's not a nursing home, it is also not in our data. Again, if there's a home that you think should be in the data, feel free to reach out. But those are kind of common reasons you may not see a home you expect to see in our tool. So we have our data about once a month. And as Charlie said, we pull all of it from the Centers for Medicare and Medicaid Services CMS. Some of the data that they have is self reported by nursing homes. It's not a lot of it, but just a nuance if you are using this to report it's good to be aware, you know what is self reported by home versus what is like result of oversight by the government. I believe like COVID vaccination data is still self reported. So just a little note that I will be reiterating throughout this walkthrough, which is just no one data points summarizes a home. It's always good to get as full of a context as you can. We hope that these inspection reports really provide a lot of information, but you should always read them in full reach out to a home if you have any questions. If you're thinking about putting a loved one in the home. It's always good to visit the home. Everything we present is just one piece of hopefully a large picture that you will be considering. Like I said before, we are investing in nursing home inspect. I will mention a few things throughout the presentation that we plan on doing relatively soon. But we want to hear from you we want to hear what would make this tool better for you more useful. And please feel free to get in touch. I would love to hear from you guys. Awesome. So now I'm going to switch over to demoing. So, wait, actually, I just got ahead of myself. I'm going to give a brief overview of the kind of main pages I'll be demoing and then I'll switch into the live demo. So we have first our landing page. This is a good place to go if you kind of are looking for a national overview or kind of don't know where to start. So it kind of provides an intro to searching the tool. It also provides some summaries on states and homes that may have a lot of deficiencies or fines. We also have state and county pages. So these maybe allow you to kind of do a deeper dive on a specific area if you are say a local reporter based in New Hampshire. If you are looking to put a loved one in a home, maybe like in the county that you reside in. These are good places to go. And they have a list of homes and summary information that you can sort by. And a few other things that we will get into during the demo. We also have specific pages for each nursing home. These provide again some summary stats on the home and then they also provide more in depth information on these inspection reports and try to make them a little bit more easily consumable. And last but not least, we have our searching feature which as Charlie said is kind of the secret sauce of this tool, it allows you to search across every inspection report in the data. So if you're looking for trends, look for things that might be common issues, and really filter that down to find what you're looking for. And, okay, now, not getting ahead of myself when I say we are going to jump into the live demo. Awesome. So this is the homepage. As you can see it has some summary information about states. And summary information about homes, ones that have the most deficiencies so like I said, those deficiencies are the individual issues found in inspection reports. Highest fines by home. If you want to just see all homes sort by various categories we have we have an all homes page that you can filter by various things lowest deficiency count highest serious deficiencies highest deficiency count things like that. You can kind of quickly find homes in it with a nationwide focus that might be ones you're interested in looking at. We also have these sort of summary information about states that are maybe high on some of these indices so serious deficiencies I think maybe I'll use this term throughout the presentation so if anyone isn't familiar with it. So serious deficiencies are ones that are causing immediate jeopardy. So these are kind of the ones that maybe are most worth looking at there when a resident is put in maybe immediate harm or immediate possibility of harm. And then one thing I want to highlight here is we've added this focus on inspections delays throughout the throughout the app. And that's because I think post the pandemic, a lot of states and homes are behind on those scheduled surveys that Charlie mentioned. And we think this is important because those are sort of the routine inspections that cover a comprehensive view of what might be going on at a home, whereas those complaints surveys or the infection control surveys are very specific in response to say a complaint made So we want to highlight that some states are still catching up on their inspections. I think this problem is getting better I think it was worse a few months ago, but this is still an ongoing thing that some states are really struggling with. The only other thing I want to highlight here is that on nationwide and on a state by state basis, we have started to surface this section called most recent serious deficiencies. And those are the ones that are causing immediate jeopardy to resident health and safety. This is hopefully to help reporters get a better sense of what issues have recently come up in homes either in their state or nationwide. Get an overview of what decisions might be and explore them more thoroughly. So you can always pop into the homepage here or go to explore the report directly if you're interested in learning more about those. And then if you want to do a deeper dive so you want to start looking at a state, and we have a search tool here that should make it easy for you to find what you're looking for. So, you can go here, search by either state county territory or nursing home name. So if you want to get a little more specific in your search generally searching for nursing home names you can narrow it down by this same with counties. If I'm specifically looking for Illinois, I can hop into Illinois. The state pages offer some summary statistics on the homes. And then also they offer kind of a more focused version of that most recent serious deficiencies section. So just specific to Illinois. If you want to see more of those means recent serious deficiencies you can pop into the see more and you can see up to 30 of those if you if you want to just explore kind of all the ones that have happened in the last few months. Lastly, you can sort of search by home sort those see what you know again like with those list of all homes you can search by most serious deficiencies. You can search by lowest deficiencies, all those things to kind of help you narrow down to what homes you may may be interested in. If you want to get more specific in your search. You can go down to the county level so say I'm based in Chicago and I want to go down to cook county. I am not from cook county and like Charlie but using it as an example. And this has the similar summary information and same level of ability to search by homes filter those. We do not have recent serious deficiencies on the county level yet because a lot of counties don't have any but we may add that for the counties that that do. So anything you might want to do here as I've said is kind of look for a home of interest. So if I'm looking for a home that maybe warrants further scrutiny. I might sort by highest count of serious deficiencies. And if I do that, it'll filter down this list, starting with the ones with as you can see here the most serious deficiencies down within Cook County. And then kind of go down based on that. I could also filter by something like highest efficiency count. So as you can see this one, this, this home has a lot of total deficiencies, but very, very few serious deficiencies are zero in this case. And those are the ones that are causing immediate jeopardy. Maybe I will filter again by highest count of serious deficiencies because I'm really interested in the ones that caused immediate jeopardy to resident health or safety. And then so maybe I want to explore this home, and we'll look at that one in a second. I can also filter by things like lowest efficiency count if I'm interested in homes that are doing really well, or at least to that have very few total deficiencies. And so we can see the all these homes have a relatively low number of total deficiencies. And basically all of them have zero total deficient or zero serious deficiencies and very low fines or no fines. Again, no one data point defines a home, you can't just be like, okay, it doesn't have, you know, it doesn't have a lot of serious deficiencies or no total deficiencies therefore I know for sure it's a great home. But those are good signals that maybe the quality of care is good there. And so that's the county page. And then if we want to explore those two homes and learn a little bit about the more about the homes page, we can hop into the first one. This was again just a reminder this was a home that had very few total deficiencies. On every homes page, you're going to see relatively similar information. You're going to see summary stats about the home, where it's based, you can pop out to its Medicare page which is the page that Charlie showed earlier or the site that Charlie showed earlier. So you can see it's overall rating and other things about it. You can also see it's ownership type. I think I mentioned earlier that we are going to be adding new features to nursing home inspect and one of those new features we plan on adding is expanded ownership information CMS has made more ownership information available. And we plan on improving what we offer around that making it easier to search by owner and see you know everything a certain person might own or a certain LLC might own and just just kind of figure out things from there. We also have staffing and capacity information. We have surfaced these two metrics here nurse hours per resident per day and nurse turnover. We've surfaced these because the MS as they are strongly linked with quality of care. And what you want to look for here is that this number nurse hours per resident per day is relatively high, hopefully above the state average which we show down here. And for nurse turnover, you are hoping that that number is relatively low, hopefully below the state average which again we show down here. And then you can also see things like COVID staff vaccination information is the staff have they gotten that initial round of vaccinations are they up to date with their boosters. And then you can see more information about the inspection reports here but we'll get into those in our second home that I'm about to show. So this second home is the one that had a high count of total deficiencies and a high count of serious deficiencies. And you might see that we have some of the same information but some things are different. We have the same ownership information we list the staffing and capacity. But you'll see with this home, the staffing capacity is much lower. It is below the state average. And this is generally bad. Again, no one number defines a home but the more the more nurse hours per resident per day generally the better outcomes for residents. Nurse turnover is also a little higher. It's definitely below or above state average. COVID vaccination data it is it's below the national and state average you can see but it is not well below. The other thing that we have added is this flag section so flags will be different or home you'll see different flags depending on the home you're visiting. And what we think is worth noting about it. In this case, we have noted that this is a special focus facility. That means the government is subjecting it to kind of an increased level of oversight. That means more standard surveys more standard inspections. And that is because it has a history of substandard care. I'm going to switch to another home which has some more flags just to show more examples of what these might be. There's the special focus facility candidate, which means that it meets the criteria to be that special focus facility but the government has not yet flagged it as that. There's also a flag that indicates a low staff vaccination rate that just means that the staff vaccination rate is well below the state average. There's also this inspections delayed flag which gets back to what I was talking about on the homes page it just means that this home has not had a standard survey in quite some time. I think they are supposed to happen, roughly every 12 months at most are 18 and this one has not had one in more than two years. And that's, you know, it's just a little bit of a warning that if there are issues at this home, maybe they aren't being caught because there isn't that kind of routine inspection going on. It doesn't mean that there are necessarily more issues it's just something to be aware of, especially if you like look down and you don't see recent deficiencies recent reports. That might mean not that there aren't problems but just that they haven't been inspected recently. So switching back to this home the one with lots of deficiencies, just diving into the inspection reports section, which is like I said kind of the focus of the app. You can see a little summary about the deficiencies found against the home and again those are just individual issues within inspection reports. And Charlie showed one of these a second ago but I'm just going to show it again. And this is one of those inspection reports from CMS. And they are pretty wordy, pretty terminology heavy. If you're familiar with them I think people are pretty good at parsing them but if you're not, they can be a little hard to read initially. So our goal on homes pages is to really make these easy to skim in a way that those reports may not be. And that means sort of surfacing summary information about the reports, the data was filed the type of report, the number of issues found the seriousness of these issues. And so if you're familiar with this data you might know kind of the letter grade seriousness, which goes from a to l a being serious l being the most serious. And also break that down for people so if you're here, you can see this one is of seriousness J. If that doesn't mean anything to you we hope that this will help seriousness is made up of two things, severity and scope. So very is kind of the level of harm and scope is how many people were affected by the issue. And so we hope this makes it kind of easy to judge really quickly, you know how serious of an issue was this, how many people did it affect. We also have the category of the deficiency, and the description of the general type of deficiency that it was just to give you a sense of maybe what category was it administrative, was it a residence rights, was it a quality of care, etc. If you want more information can always click through the full report that will be that CMS report that I discussed. Yeah, and so you know we just hope that it's easy to kind of scroll through here and get a sense of which reports you might want to do a deeper dive on. And the last thing I want to highlight on the homes page is this penalty section. It just lists all the fines or payment suspensions against a home. And so this is when CMS either finds a home or stops the Medicare payments to a home. And if that in that fine or penalty is linked to an inspection report, we make it easy to just sort of scroll up and see that report associated with it. So that you can kind of link you know what the fine what what the fine was with maybe the issues that were related to the fine. And if not, that tends to be because it was a data reporting issue, or maybe an infection control program issue there's been a lot I think more of these fines that haven't been associated with inspection report during the pandemic. But most of them are associated with inspection reports so you can kind of scroll up and see what's going on in those various reports. Alright, so the last thing I want to highlight is just this advanced search feature. So this is what Charlie was talking about that lets you search across all of the inspection reports across all homes nationwide. And we've updated this to do some more advanced text searching, as well as kind of adding advanced filtering where you can filter by multiple locations right now this is just states or territories but we're hoping to expand that. You can also sort by date. So say you're interested in reports that were filed early on in the pandemic, you could search for you know three months at the very beginning of the pandemic. You can search by report type seriousness which is that letter grade that we talked about earlier. So if you're only interested in say deficiencies that count as immediate jeopardy, you can check these boxes. You can also sort by deficiency category. I mentioned this later when they're where they're speaking about their reporting but we want to expand the filters as well and we're probably going to add filtering by F tag, which if you're familiar with nursing home reporting is just kind of a lower level of category within these. So I'm just going to do kind of a test search show you how that works. So say maybe I'm in the Pacific Northwest and I'm interested in Oregon and Washington. And like I said I'm interested in those kind of deficiencies that constitute immediate jeopardy. So in terms of the tech search. And this is maybe not a real query I would use but it's just to kind of show an example. You can do some relatively advanced stuff you also totally able and welcome to just search for like basic words. Apologies for kind of the dark, dark search term that is went to right there, but you can do you know your basic word, but you can also do more advanced things. In this case, this should search for the word pressure. And either the term ulcer or sore. So if you want to do kind of more advanced queries where you're looking for a number of different things. You can do those now. One thing to note I have put the words pressure and sore in quotation marks. And that is because words now that we search for in the app that are not in quotation marks are subject to something called stemming stemming is where it searches for various forms of a word. So if I went back and put in the word choke. So I want to search for chokes choking choked again sorry for choosing my example. But we know that maybe that will be useful for some people who are searching kind of broadly for a concept, but that also some people might want to search for a very specific bird. If that's the case you just want to put it in quotations, like if say if you want to search for medication, but you don't want to search for medicating or medicated, you just put medication in quotes. So I'm going to go ahead and submit this. And it looks like we found 11 inspection reports that matched with 11 matching issues or deficiencies within those reports. And we surface the seriousness of the deficiency that was found to match, and then also some text from the deficiency to try to give you a good sense of what's going on in the report see if it's something you're interested in exploring further. So if you are interested in looking at the full report, all you need to do is click through to here on the PDF. Let's see if it loads quickly. And it should take you to that full inspection report so you can look at the at that text in its full context. One thing to note here is that we only surface in each of these reports which is which is the each box found in your results. This is the deficiencies that matched your search criteria. So this report might have more than one deficiency in it, but only the one that matched what I searched which was this ulcer or sore and pressure. You can also click through to the home to see more information about all deficiencies found against it. And once you're here you know if you're scrolling through and you're not seeing what you're looking for or you realize that you want to search more things. You can go ahead and update the filters while you're in there so I'm going to say oh maybe I'm interested in Idaho as well. And the update I'll add in seems like it found two more reports in Idaho that matched my criteria. So maybe I'm also interested in like less severe deficiencies, but ones, you know, ones that are still still quite severe. So I added in kind of GH and I, in terms of seriousness, and now it's found quite quite a few more reports to go through. You can filter down by date, or if you're interested in like the most ones that match, or starting by home by state you can do that you can also see by most recent reports to say I want to see just like the most recent report that matches this criteria, search by that. And it looks like it's one from June of this year. Yeah, so we hope that this gives you kind of a lot more in depth control over your searches. But like I said, we really want to hear from you, we want to know what what works for you what doesn't. If there's anything from the old app or the old searching in the app that you miss any features you want to add any filters that would be useful. We are really excited to hear from you and keep improving the tool. Yeah, so I don't know if there are any questions in the chat I will look through but in the meantime, I would like to pass it back off to Emily and Charlie, we're going to talk about using this tool to report on nursing homes and also just some of the nursing home reporting that they have done. Thanks Ruth. Emily is going to kick us off. Okay. Hi everyone I'm so glad you can join us today. I just wanted to kind of go through a little bit of the landscape of nursing home regulation and sort of what agencies and individuals might intersect with with with nursing homes. The sort of make everything that Ruth just went over really kind of human and matter to your stories so how can and how what Ruth has mentioned it can support your investigations so broadly speaking. As earlier said they're about 15,000 nursing homes across the country, and the federal level they're regulated by the Centers for Medicare and Medicaid Services which also, and the federal government also provides funding to these facilities through Medicare and Medicaid. So if you zoom in down to the facility level you've got state agencies which regulate nursing homes as well there may be more strict rules around, for example, staffing in different states that legislators have enacted. And at the state level, nursing homes can play a really big role in the legislation related to the facilities they can tend to be potentially very large lobbying groups at your state legislature. And then at the local level there's, you know, especially if you're a community journalist if you're really local journalists like individual facilities can have their own stories and, regardless of what level of nursing home kind of strata you're looking at, you'll probably end up looking at individual homes and kind of the people what people experience there and how that interacts with your story, I'm going to go through a few examples of how that has worked out. And then I have a note here says, these can all kind of blur, you know they're nursing home owners or companies that manage the care at nursing homes that span different cities, counties, states, and it all kind of comes, it can all kind of mix so it's not a hard and fast rule that's generally the landscape. To build off what Ruth said I just want to say one of the most useful things when you're getting into this space is to really like learn what your f tags are and those are basically codes for different citations or deficiencies that are used by state and federal regulators so you can kind of just I kind of usually just Google like cheat sheet f tag cheat sheet and something will come up where you'll get a list of like all of the f tags and like little descriptions for what they are, and I can kind of introduce you to some ideas for like what your story might be about. So, there's a wide range of f tags I think, and a lot of inspiration that you can get from them, probably for the most part you would be interested in an abuse and exploitation category. And I have in the past also reported on the infection control which is obviously become really important in the context of the pandemic. And I heard from Ruth that they are going to be making the tool. You may soon be able to actually filter by specific f tag with this tool which is like really great news because that is something that would make I think it will make our lives a lot easier. So, so outside of the f tags I would say also get into reading these survey reports, because as Ruth said there is a lot of vocabulary that you're not probably going to be familiar with. And to build on Ruth's example the pressure ulcer was something I wasn't really familiar with it's something that you'd probably know as a fed sore. But in the literature or in the in the service reports that's how it's referred to so getting familiar with that vocabulary through reading a bunch of reports will probably get you started. So I'm just going to go through pretty quickly, some of the stories that I've contributed to that have sort of intersected in this space. So last year we wrote a story about medical exemptions to the COVID-19 vaccine for nursing home workers and sort of the backdrop of this is that the science tells us that the reasons to avoid the vaccine are very, very, very, very rare. The CDC reported that sort of maybe six out of a million doses or fewer than six of a million doses had an adverse reaction that was severe enough to warrant avoiding the vaccine altogether. So that's really really really small point 006% of those doses. So comparatively we found with this new data that CMS has begun to collect and publish that there were nursing homes where a large, large proportion of their staff had actually turned in a medical exemption for to avoid the vaccine. And so this is a really crucial issue at this time especially because by when we wrote this story, it was about a year after the vaccines have been widely available for nursing home staff. And it was really troubling the idea that there would be a nursing home worker who could file for a medical exemption erroneously. And so this story was really a story that was about science about a federal mandate about shifting regulations and COVID. And using the data that from the nursing home is back to one from CMS we were able to compare states to find some states had really large proportion of their nursing home workforce that we're claiming a medical exemptions, or just weren't getting a reason. And then we were also able to take advantage of the surveys to find a, I think this was a new new F tag, where a nursing home worker had had claimed claimed a medical exemption but the inspector found that that was an exemption had sided the facility. So that's sort of how all of these things came together. And you could kind of think of this as like an industry kind of way to approach a story like this where you're thinking how is this industry adapting. And when you're looking at all the different relations and some of the things you'll be reading in the service report you might want to ask yourself, okay, like how is this best practices like how is this. to reflect the what we know about how to treat nursing home residents and to make sure that they are they are able to thrive in these facilities, not to mention, you know, just survive. So that is how we sort of approached this story. Prior to coming to ProPublica I was part of a team that looked at a very obscure funding mechanism of Medicaid. And so to try and give you the shortest version of this very, very, very wonky topic. You know, nursing home care is paid for by Medicaid, Medicare and private pay. And Medicaid rates at nursing homes are lower than Medicare rates so if you're an individual on Medicaid, the government is paying the nursing home less for the same services as someone who receives Medicare. So you have that sort of reality. At the same time, you have county nursing homes or state nursing homes publicly owned that are considered the sort of safety net nursing homes. These are where these are where the sort of poorest of the poor who require medical care around the clock or who are elderly, end up going. And as such, they have a disproportionate number of low income people so they don't have maybe the balance of private pay and Medicare recipients to balance out the lower rates of Medicaid. Such a crash course into what's called supplemental payments, or almost because a supplemental payment is where Medicaid to compensate for those lower rates at these public facilities. CMS will pay a higher rate to a public facility to compensate for the fact that there are so many Medicaid recipients at the facility. So Indiana's county hospitals figured that out, and about 20 county hospitals bought up 93% of the nursing homes in the state, making them all qualified to receive this extra money. And as you can see, it had a big impact because Indiana became like the largest recipient of this money. And it's pulling on, you know, billions of dollars and extra Medicaid dollars. This tip had originated from some different kind of other tips that we had gotten. But when we looked at the system we kind of kept asking ourselves like, Well, where's this money going. And we actually still don't really know because the hospitals won't tell us and I think there is a an ongoing lawsuit still by Indy star my former paper, where we're pushing for the release of this information. But what we said was like well if it's going to the nursing homes, like it's supposed to the erratic in theory, then there would be really good care because we're getting all this extra money. Well what it turned out is that the care was still not very good. And in this case we focused on staffing as our metric of whether care was. There was quality care at the nursing homes because we were told by CMS by experts who research nursing home care over and over again that the staffing was the most critical component of care at these facilities. So we kind of use that as our balance we looked at the funding, and we looked at the care, and we said, if we're getting our kind of our thesis was like if we're getting so much money. Why is the care lacking so far behind all of these other states that don't receive the money. And so that's sort of a way we were kind of we that's the way we approach the story. We supplemented this story with MACPAC data and research MACPAC is the Medicaid and chip payment and access commission it does congressional research on Medicaid. Super interesting if you're into that which you might find yourself if you look at nursing homes. We looked again at nursing home survey narratives and we looked at lawsuits. We looked at how we built our story. And I just wanted to highlight a couple of kind of brief snippets that we pulled out. I did not know Ruth was going to show an appearing facility but ours are also appearing for some reason. And these were just a couple of the kinds of, you know, reports that came out of these facilities which were pretty awful. And one of the things that is difficult about nursing homes is kind of, it's difficult to get inside. I mean, theoretically we could just walk in but there's also this it is, you know, technically private property all the things. These survey boards allowed us particularly as the pandemic progressed in 2020 and insight into these facilities that we otherwise would not have had and that's really that those are what really helped us drive our stories about the quality of care there. So then as a result of our nursing home coverage and even here at ProPublica I've kind of continued to sort of use these surveys as in my reporting on different issues. And this is where you really get into the more local opportunities. These are individual nursing homes or perhaps they might be companies that you're maybe they're based where you are. I mean, there's a lot of different ways to look at it. But this is where you can kind of get into like a more local story. And so a large part of my job in 2020 it was just reading all of these survey reports, which is when I discovered that there was an LPN who tragically unhooked an oxygen mask from a patient who later died. She was ultimately charged and I believe pled out was charged with a charge related to acting without a license because she was not, she was not licensed highly enough to do that sort of action. So this story was both an opportunity to, you know, something really horrible happened but we also made sure in our story to build context into that. And part of the context of that was that this occurred at a time when the nursing was really, really taxed. And when she herself, she told health inspectors that she was dealing she was caring for 40 COVID positive patients at the time that this happened that she made this really awful choice. So it had the context of the wider industry and, and, you know, I remember one of the things that we were quoted in the story is in a nursing home advocate who said, you know, ultimately this is this is a facilities, you know, responsibility. So I think it's important to kind of keep that all in mind when you're looking at these different individual acts. Again, surveys became really helpful when we were reporting on the Minnesota Board of Nursing. Starting out did not expect really to interact with nursing homes in this story at all. But what we were, what we were looking for in our reporting is actual harm that had been done by a nurse that had been reported to the Board of Nursing, but who had not had the complaint process quickly enough and then went on to cause more harm. And we were able to find just that. And I think the most interesting part of the survey report that we reviewed for this story is that the administrator at that facility actually called out the state in two inspectors and said this is not my fault. We did a background check on this person. We did not see anything in their background check that would indicate that that she had diverted drugs, which is a kind of technical term for like stealing drugs. In this case, narcotics, we had no idea that she had done that. And there's nothing on her license, nothing flagged for us to know that. And, and yeah, we're getting excited. So that really helped, you know, build color and kind of like a real life situation to what we were dealing with on more data driven numbers of complaints and lag time kind of story. And as Ruth has mentioned, I think some of the real opportunity for the search tool that nursing home inspector allows you to do is searching actual individual keywords which can maybe aren't captured in the idea of the f tag but something more specific than that that you want to look at. So my examples here, social media, law enforcement, police drug diversion, if you're looking for something kind of more specific than an f tag look at you. And then what's next for nursing homes, there's probably there's endless reporting to do on nursing homes. The pandemic was a hugely disruptive force, we are still very much kind of reeling from it. Nursing home residents are still among the most vulnerable to COVID and we're at a time when it feels basically like society has kind of moved on from any kind of large scale precautions. So you have to wonder sort of like how our company is dealing with this how our, you know, how our facilities handling this is, you know, what what is the staffing situation after a years of burnout, kind of level of pace for these pandemic also spurred legislation across this country, some states boosted staffing requirements, many others provided immunity, civil immunity to nursing homes. How's that working out. Is it benefiting the residents is it benefiting the companies I'd love to know if you know how these things have sort of started to pan out now that's been a couple of years and some of the laws have been passed. The Supreme Court recently upheld the right of nursing home residents in publicly owned facilities to file lawsuits, civil rights lawsuits. Probably not as important to you unless you live in a state like Indiana that does have so many public nursing homes but it may be something that has repercussions down the line. So, you know, private equity is, I think increasingly moving into the space and where there's profit off of vulnerable people there's probably a story to be told. So it's another thing to examine and kind of keep an eye out in your communities. So, with that I will turn it over to Charlie. Thanks Emily. I wanted to show a few additional examples of how keyword searching can really help you to, you know, explore what's on your mind and really lets you use your imagination to find interesting stories and story ideas so I'm going to share my screen and show a couple examples here. So one of the most serious issues in nursing homes today is when you have residents who elope and elope and means that a resident has left the left the facility, and has left the facility when they shouldn't have left the facility. This can cause, put residents in danger, residents who have dementia, maybe confused you may not know how to return home. And there are situations where residents who have eloped, I have died. And those deaths have prompted reports and you can see when you look up elope and died, as I did. And then I sorted it by most recent reports interestingly. The appearance here Chicago Heights was also came up in Ruth search is there again in mind so if you're from Illinois. The fact that this is come up a couple times maybe something to look into. When you look into these reports, you are able to find homes where what happened is quite serious and may raise a question so I wanted to show you something that I found in preparation for for this webinar, which was this particular inspection report from capstone health care states on Orem in Houston, Texas. When you click on the link, which you're able to see in here is that a resident eloped in very, very hot weather, a 63 year old resident was outside. And then subsequently was found outside with a temperature of 108 degrees Fahrenheit was brought to the emergency room and subsequently died the resident had heat stroke and and did not survive this. So as we think about all of the heat issues going on all across the country. And you look at this you may in fact, you may say like you know I'm really curious to see residents who have been affected by the heat and our tool enables you to do research and searching to find examples of where you know where you can look upwards like heat stroke you can look upwards like air conditioning, and you're able to to find examples where where those come into play. Here's an example from a nursing home in Maryland, which is the opposite. I'm sorry in Missouri, Maryland Heights, Missouri, where a resident walked outside when in the winter. And this was from January of 2023. And the resident was outside when the temperature was seven degrees Fahrenheit with a windshield negative 12 degrees wearing a T shirt, light, light sleep pants and no shoes. They didn't find any staff when they entered the home. And when the resident was sent to the emergency room, the resident was found to be hypothermic and required emergency medical treatment. So they declared what's called an immediate jeopardy here which as Ruth noted, is an immediate jeopardy to the health and safety of residents. So again, you can see that elopements are not just a paperwork error but in fact put residents in serious danger and when you just when you're able to keyword search in nursing home inspect. And again I'll show you my keyword here, where I look for the word eloped, and then died in quotes. And so what Ruth said about sort of structuring your query so that in the loop I wanted it to find a loop and elopement and eloped, and it died I really wanted the word died. So there may be others where they use other words other than died, but from from this search we found 87 matching deficiencies 67 reports in which both the word elope and the word died are in there now that doesn't mean someone died during an elopement, you will look through some of these reports and find that that's exactly you may not find that exact hit, but what you will be able to find is able, it will narrow down the pool of reports for you to take a look at. We had a question in the chat earlier about a resident who had been deemed an imminent danger, and whether or not there are ways to see how often that it happened and home cited for this so I took the phrase imminent danger and put it in quotes and put it in quotes for it, and we were able to see where it comes up within inspection reports so simply knowing the the phrase you're looking for is able to bring that up. Another example for you, I used the word terminated in quotes to look for staffers who were fired by their nursing facility for potentially improper conduct at work and I sorted it by most recent reports and we see there are 344 inspection reports in which terminated is used just in immediate properties. So these were cases where employees were fired. So if you're trying to sort of narrow down your universe of, you know cases we see the first two examples most recent ones from June of this year so we're talking really recent May of this year in Kansas so if there are any reporters here from Kansas, you may have a story trying to understand what happened at these couple facilities that perhaps you want to, you know, take a look into. So what you can see here is really are getting you to employees who were terminated by by their home so if that's something you're interested in, you could do that you could you could set dates you could choose your state location and narrow the field down but there could be really good stories that are within there. Here's an example of a termination where we could see that the nurse was terminated for stealing narcotics that were supposed to go to residence. And you will find lots of reasons for terminations but by clicking through to the inspection reports, you're able to get the report and then just keyword search the word terminated in that report and it will take you right there. The last example I wanted to show you involved a story that I did at ProPublica a number of years ago, where nursing home workers were sharing explicit photos of residents on Snapchat. And then across an example of a nursing home employee who have been arrested for sharing not nude photos of of residents on Snapchat, and I wondered how often this happened. And so using the precursor to the relaunched nursing home inspect, I was able to find 35 cases in which workers have been accused of surreptitiously sharing photos or videos of residents on social media. And so it's, you know, you think about all the things that can happen at a home and just how this strips residents of their dignity and it just breaks your heart actually. And in just looking in our tool for the phrase Snapchat, we see 32 deficiencies at 21 homes and, and you can see some of them involved similarly really horrible things where they're putting photos of residents on Snapchat. So if you do a search for, you know, for tick tock, you will similarly find that tick tock has mentioned in reports so there's real real power here in nursing home inspect and what I like to tell folks is it really is a matter of just letting your imagination dictate where you want to go and there's just such power in the, the various ways in which you can sort and filter within nursing home inspect so that is what I wanted to share. I hope that was, that was useful. And I think we are all ready to, to come back on here for, for questions and answers so we're going to turn to the Q&A portion and before doing that I'd like to share a link to our event survey in the chat box. We appreciate your feedback. And again if you'd like to ask a question, click the Q&A icon at the bottom of your screen to submit it to us. And we will, and if Emily and Ruth can come back on, we will go to our first question. Okay. We have a question here from from Anita Lee about, do you have a key for shaping queries any advice for how to shape the queries you use. Ruth, do you want to offer any advice on that. I mean I think this is a combination of your skills and the capabilities of the tool but I would say we published sort of an advanced guide and there is some some advice in the advanced search about how to do that. It's a combination of trial and error like looking at reports and kind of seeing what terms tend to go with each other and tend to be describing the issues that you're interested in so you know if you're interested in like Charlie said termination, searching termination seeing what two terms tend to go along with the kind of terminations you're interested in and going back or finding your search. And then just taking advantage of some of the more advanced things that we can offer now you can search by multiple terms as long as you string them together with and or or. You can do kind of those more advanced searches where you search for, where you search for, you know, one, one term or the other or a combination of terms. So just getting creative playing around with what you can do and again just kind of, I found it really useful to be like oh I'm interested in this broad concept search for it, read a few reports and realize that okay maybe that term isn't the right way to get it maybe it's this term and just kind of keep refining. Another question for you, Ruth. Does the ProPublica database support direct SQL queries and what about exporting data from the reports. Was this you know are you able to integrate this with other data sources. We do not currently support exporting your searches or anything like that, but that is one of the things we are discussing in terms of especially for searches how to make it easier for people to extract information from the tools so stay tuned. Okay, another question, will a reader or user be able to identify if the inspection report is an annual one, you know the 15 18 month routine cycle, or if it's a complaint investigation inspection. Emily, are you done. I was, I'm happy to answer because I can see that you can yes see that on nursing home inspect and you'll also see in the survey document itself that will refer to complaints and it, you should be able to determine which citations are related to which complaints, although you probably you'll not know much outside of that or what the complaint contained since it's pretty anonymized. Great. How can I tell if there is a downward trend and how a nursing home, you know, if there's a downward trend in a nursing home to detect a potential indicator of elder abuse. And in terms of what we offer in the tool I think there are just a few things to look at. There's those flags that we surface that should surface if they're you know if it's a special focus facility and things like that one of the flag that we do currently didn't mention before is a change in ownership flag so if the if the ownership of the facility has changed in the last 12 months. We flag that in the hopes that you know maybe that could be an upward trend maybe it could be a downward trend but maybe there's some kind of changes happen the facility you want to keep an eye on. And then I also just like to look at the recent inspection reports and see are there more severe or serious deficiencies. More recently are all the serious deficiencies in the nursing homes, past several years ago. Are there more complaint reports recently things like that just kind of looking for recency within those inspection reports. One of the other key things and then everybody should keep an eye on is that different states have very different, you know, inspection mechanisms that while the Centers for Medicare and Medicaid services overseas, the nursing homes across the country they contract with inspectors the state and local levels to do those inspections Emily talked a bit about this and, and those state inspectors have different levels of strengths. And so, some states are pretty aggressive at citing and finding homes with problems. Others are very light touch. And so, you know, it's very hard to compare nursing homes across state lines to one another, which you want to be able to do is sort of look within a state and look to see, you know, how performance within a state on now certainly if you're looking for incidents where residents for example choked on food, potentially died. You know, you've crossed the country for examples like that, but it's very hard. You know you see Ruth showed the slide of how in Maryland I believe 75% of homes are late for their routine inspections right roof. I hope I'm not miss 74% right. Yeah. But so in Maryland right you would expect they're not going into homes as often so they're not going to be finding as many problems. You wouldn't want to compare that to a state that that wasn't, you know in place with but within the state I think what Ruth said, you know, looking at different timeframes and seeing just the home come up, you know, in a recent timeframe as well as the overall timeframe can be really helpful to understand trends. Another question, does the tool provide access to 990 forms which are the nonprofit tax filings, or financial information of nursing homes and these apply to only nonprofits. Ruth. We do not, but you can feel free to check out our nonprofit Explorer tool which lets you search by nonprofits and search their 990 data. I think I could drop a link to that in the chat in just a minute. And, and, you know, maybe there'll be some fun crossover we can explore in the future where we link out to it on individual nonprofit nursing home pages. I don't know how to find the nursing homes payments to their affiliated parties, there's a, you know, a note that the inspector general plans to look into this. I don't believe that this information is in the data that Medicare makes available on payments to affiliated parties. This is where perhaps looking at the 990 form if it's a nonprofit home. I don't see, you know, payments to their contractors, the payments to their executives. I don't believe this is in the CMS data though. Does the nursing home inspect database support a public API. And as we've already addressed the SQL question but is there a public API. We don't have a public API. Another thing that we are considering CMS does have a public API that you can check out for pulling some of their data. So if you're just looking for like a direct data poll, that might be a good place to start. Does our tool allow you to track coven infections hospitalizations and deaths among nursing home residents and staff. So some of that information closer to the start of the pandemic. We transitioned it to vaccination data as that became available. In part because some of the data we had felt outdated, like, if nursing homes are now doing better when they'd previously done worse at the start of the pandemic. But we, we are considering adding lots of new data and and if we if we determine that that is still useful and relevant data we may read some as we continue to expand the tool. Here's a question for me here was Snapchat the keyword do you use to find all the explicit photo cases. I actually tried to use various words, understanding that it wouldn't just be the Snapchat platform and also, like, sometimes inspectors who are unfamiliar with using it as two words, snap space chat. And so you need to think like how might an inspector who may not, for example, use the tool, like refer to the tool. And this is where when you use direct quotes, you know any variation can be meaningful in the sense that you want to try different combinations of things otherwise you may box out something you're really looking for. For photos you could look for videos like I have a tend to have a broad search strategy I think Emily you talked about having a broad search strategy to as you look for things. Yeah. In terms of trying to find I think, you know, I think that the keyword demonstrates sort of like what when you're looking for something very specific. There's something that is more like a general harm than understanding kind of like what inspectors would pick up on again back to those f tags like that is kind of like what is helpful to to kind of pursue your story. There's a question about can you compare facilities violations to star ratings. So star ratings are Medicare's effort to group facilities within a five star rating system where five stars are the best and one star is the worst. We have not incorporated stars into our own presentation of this information, you know, in part because we have felt that having access to the raw data on individual inspections we don't. So some inspectors not attempt to be everything for everybody. And so we really focus on inspections because this is the, you know, they can't be gained by facilities and I saw a couple questions about gaming and you know this is where inspectors go in there the eyes and ears for the public and we really want to surface what they find when they go in so we are not sort of relying on the star ratings and don't include those in our presentations. So I could just add a little bit to that that each of the star ratings actually takes into account. A lot of the raw information that's available on the tool are available from CMS. And then kind of does some kind of calculation that was decided upon, you know, at some point and then produces a star rating. So what you're actually looking at is the actual info that goes into that star rating so it's not completely divorced. And it probably is a little bit more useful to like the average person's brain to look at the, the, the literal, you know staffing and what up and whatnot. Is the database searchable by funding source so like Medicare Medicaid private pay, etc. It is not but again we're planning on expanding kind of the ownership and I don't actually know a lot about what funding data is available but if there is kind of in depth funding data we can definitely try to incorporate that. Yes we said at the outset this applies to Medicare certified homes only and so many Medicare certified homes also take Medicaid and private pay but for homes that solely take private pay they would not be included in the nursing home repair tool and similarly. So they have to be a Medicare certified home in order to be included. I don't believe Medicare releases information other than on Medicare cost reports which are highly technical reports about the sources of funding within the homes. And so that that is not something that we've incorporated into our tool. We're going to talk about whether you can filter for location by state county city zip code within 25 miles like can you put in sort of the circle the radius of what you're looking for. Yeah you cannot currently the closest we can get for that is the county level search. I really said you know we're kind of trying to balance. Being everything for everyone versus you know being really specific on these inspection reports and we decided not to prioritize that right now just because there are websites like CMS care compare that do kind of let you do more of a geographic based search. But that you know that is something that's on our radar for an improvement down the road but we have kind of higher priorities right now. So all for questions and folks have it please feel free to to put it in in the chat. Oh, we have another one. Can you find facilities that have successfully challenged violations and we're able to get reductions in the severity, or get them erased. So this was actually a bit of an issue. I know that the New York Times reported on, I believe how facilities were able to avoid having their penalties show up, because they were appealing them for a long time and CMS made a change where they're now showing the finds that facilities base before the entire process, you know, has played out which will bring a level of transparency to those, they may reduce like change the fine if they are successful, but I think they're not waiting to have those appear until the point at which they had in the past I don't know if that's actually taken effect yet or not but that will that is a win for for transparency. We, I don't believe though that we have the specific data on which finds were subsequently changed. We've noted that we update our data every month so a CMS releases new data, we update the data, which means a couple things one, which I'll find is sometimes older inspection reports from say more than three years ago will disappear from our site if they've disappeared from the CMS site, they're going to disappear from our site. So it's a very dynamic site that is dependent on CMS's data similarly for fine is updated, we will see the updated fine, but I don't think we're going to note on our site that that fine number has changed from prior reports. This there's another question here about when we are referring to Medicare or we're referring to Medicaid way or facilities. I think specifically a CMS says the data in their Medicare compare and that the inspection reports they're making available are for Medicare certified sites. And so that's, that is their definition of the data that's available. And that's the definition we're using as well. Is there a shot we're going to expand the application and database management team. And that feels like a question for you, Charlie. Well we're really proud of this tool and clearly we've put a lot of roots time and others time into, you know, really overhauling it from top to bottom the, the back of it at back end of it the front end of it making it super useful and you've heard about adding additional features to it. If you haven't checked out ProPublica's entire suite of news apps it's pretty amazing and we have a robust team of six or seven news app developers and a news app editor who just work on these custom tools and other, you know, interactive tools available to reporters and so if you're interested in a job we post them on our, our site ProPublica.org slash jobs we don't have any open right now. But we do support a really robust news app team. Okay. Thank you. What are you able to share the general timeline of your next rounds of improvements in search features? I think we are not. We haven't really scoped out exactly like the dates but we're hoping to start in those right after this prioritizing improvements the filters first and then adding in the ownership information so hopefully in the next month or two. We're hoping to kind of see new features rolling out. But, you know, I don't want to overpromise. Yeah, we've had a couple comments also in the chat about additional resources for reporters to turn to and I think it's really important what, what Ruth said earlier which is, you know, this tool alone should not be the sole basis for any reporting you do on nursing homes or should it be the sole basis upon which no family members select a home. We are, we encourage you to chat with you know knowledgeable folks. There are long term care coalitions that are available in different communities. And really, you know, in here in New York City the LTCCC is a really great group that has good data and good expertise. But there are similar groups all across the country. There are long term care ombudsman's offices that are also good resources in every state, you know, in the country to be, you know, who are experts at this and so we would encourage you to sort of reach out to, you know, folks to put anything you find here in context and of course to reach out as well to the facilities or comment if you're going to include what they, you know, information from the reports on inspection reports are generally two sides one is, you know what's called the statement of deficiencies and that's what you see in the data that we're using. The other side is called the plan of correction, and the plan of correction is not actually in this document but is available, either through a freedom of information request to CMS. So the homes are required to make their inspection reports available for public inspection in a public area of the home. So if you're a reporter you can go to the facility and ask to see the report, and that will include their plan of correction. There is another question here about whether we can address whether a facility has a read ownership or other types of land ownership structure Emily I think you wanted to tackle that. I can't answer the specific question but what I can say is a couple things about ownership which is that CMS does publish and as recently like bumped up it's published publication of ownership data. When we were doing the story in Indiana what we relied heavily on the legal business name, which is located in the data if you end up working with that part of what CMS offers. So that is that is available and seems to be increasingly so and I think just one word of caution I think like Charlie said that you'll always want to like expand your reporting beyond to nursing comment spec or CMS, or whatever you end up using to just look at different records for LLCs and see if you can, you know, pull together these different companies that way. It's not really an easy answer for ownership unfortunately because there's a lot of ways to kind of separate those into individual entities but but it's possible to do a lot of research would be combined these different record sources. Thanks Emily and thanks Ruth. That is our time for today. So thank you to our audience for joining us and for your really thoughtful questions. Again, this event has been recorded so you'll receive an email with the full video of today's event will also post this recording on the ProPublica YouTube channel. Please take a moment to fill out our event survey linked in the chat box. Stay up to date on all of our upcoming programs by visiting our event page, which is ProPublica.org slash events will drop a link in the chat. And from all of us at ProPublica or Emily and Ruth and myself. Thank you for joining us. Have a great rest of your evening and see you next time.