 So let's go ahead and get started always and live in fear of these academic myths that if the professor doesn't show up by you know if it's. Professors 10 minutes late classes dismissed and these other things and if people might just shorten that time frame as we've gone on so I don't want that to happen. So, my name is guy Palmer I'm the chair of the committee and in just a minute minute I'm going to have the different committee members that are present for this call introduce themselves. But just very briefly want to appreciate all of the speakers and panelists that are meeting with us this morning, the issues that you're addressing are ones that the committee has looked at and is really interested in your input. Let's go to the next slide please. So, as I think probably all of you know, this is actually the second phase of a of a two phase process, a report. So our phase one report which was published in January of this year really looked at kind of an overview of the system that was implemented kind of in rapid fire during the emergency response to COVID-19. And actually looked at what we've learned from COVID-19 and how that would apply to development of a national wastewater surveillance system. So we're now in phase two, which I think is the next slide Justin, which we're kind of structured by a statement of task and basically what we're doing in phase two is we're really having seen that there is value in having a national wastewater surveillance system really now to kind of drill down on what really makes that system implementable, sustainable, and able to provide timely information for public health action. So just looking at, as you can see on the statement of task which is present on our website, really to just look at the characteristics of what that system is in greater detail, things such as quality criteria, what it takes to report data accurately and timely, looking at technical limitations, and then actually looking forward as someone said how to kind of time proof this system so that it's not obsolete in one year as technology changes. So that's the committee's charge which is a little bit daunting, but we really rely on experts such as yourselves to help us accomplish that. Next slide please. So, we have multiple committee members with us today. I've seen Lauren, Raul, Reka, Chris, Chuck and Krista but somebody else may have joined. So I'm going to actually have you, while Sandra there she just jumps right in. I'm going to have you each introduce yourself. And I'm going to just go on the order that I can see people on my screen so Sandra I'm going to start with you. Good morning everybody. I'm Sandra McClellan. I'm at the School of Freshwater Sciences at the University of Wisconsin, Milwaukee and involved in the Wisconsin wastewater surveillance program. All right, Chuck. Chuck Haas, Drexel University Professor of Environmental Engineering. Let's see Raul. Hi everyone. Raul Gonzalez. I'm a scientist at Hampton Road Sanitation District in Virginia. Krista. Hi everybody. I'm Krista Wiginton. I'm an Associate Professor of Environmental Engineering at the University of Michigan. Lauren. Good morning. Lauren Hopkins. I'm the Chief Environmental Science Officer here at the Houston Health Department and I run the wastewater program from the Health Department side here in Houston. Thank you. Marissa. Hi, I'm Marissa Eisenberg. I'm an Associate Professor of Epidemiology, Complex Systems and Mathematics at the University of Michigan as well. And I work with Krista on the wastewater program here at you know. Christine. Hi, Christine Johnson, Infectious Disease Epidemiologist at UC Davis. Let's see Lauren. And Reka. Hi, good morning, everybody. I'm Reka Singh. I'm serving as wastewater surveillance program manager for Virginia Department of Health. I think I got all of the committee members as you can see we have quite a diversity of expertise. If I've missed anybody or somebody has since joined. Speak now or forever hold your peace or jump in wherever you want. Okay. So let's just get right into it. And our first speaker is Shenzhen Yu from from Harvard and going to address one of the issues that we've already been kind of grappling which which is the sewer connectivity and the implications for equity in a wastewater surveillance system. So Shenzhen I'm turning it over to you. Great. Thank you. Let me just share my screen real quick. Can you see this okay and hear me okay. Yes. Okay, great. Hi everyone. Thank you so much for having me here as a part of the discussion today. I'm chin chin I'm a postdoc at the Harvard Chan School, and I'll be sharing some work that we have done assessing sewer connectivity and its implications for equity. This work was done in collaboration with my postdoc advisor, Jonathan grad and two really amazing collaborators, Scott Olson and Claire Duvalet. So one of the major benefits of wastewater based epidemiology is that it can capture a more equitable and representative sample of the population and case reporting. And in fact, the Health and Human Services project of wastewater surveillance during the COVID-19 pandemic was able to capture a more representative distribution of ages, as well as the Hispanic and black population in the United States, compared with the vaccinated population. However, one aspect of wastewater sampling that's often not thought about or talked about is who is actually connected to sewers. And about 16% of homes in the US are not connected to sewers, with the most common alternative being septic tanks which collect and process waste on site. Thus the waste does not get into the sewers and does not get sampled in wastewater based epidemiology. So septic tanks are more commonly used in rural areas than metropolitan areas because of the difficulty of having sewer systems in more sparsely populated places. However, the previous work on equity and sewer connectivity suggests that this is a complex problem with a lot of heterogeneity and variability within the country and within different populations. So this study from Biobot had found that both within metropolitan areas and within rural areas, those that are on septic actually earn more than those who are on sewers. However, this report from the Environmental Protection Agency in 2017 found that households that earn less than or equal to the median household income were more, were almost 10% more likely to lack access to a treatment works so to lack access to a sewer system than those that earn greater than the median household income. Furthermore, in Florida, Hawaii, Delaware. There was a strong correlation between income and household decentralized system usage, so primarily septic tank usage, but this was not the case in Rhode Island. So because of the complexity associated with sewer connectivity and the fact that there are very few studies looking into this, we found that there was a gap in assessing the factors associated with sewer connectivity. So because of this, we were interested in answering three questions. First, to what extent is their demographic and economic inequity and sewer connectivity. Second, which geographical areas have low sewer connectivity. And third, what is applicability of wastewater data to neighboring unsampled communities. So just the first question our approach was to analyze household level data from the US census of sewer connectivity connectivity and septic connectivity. So from this data, what we found was that at the census division level households with an American Indian and Alaska native white or older householder which is the person who was completing the survey, and households not in poverty are less connected to sewers than average across the census division. And this seemed to also be the case when we looked within urban and rural areas, with the exception of the trend of lower connectivity for American Indian and Alaska native seem to only be present in metropolitan areas that were not in the central city. So this actually included areas that were urban as well as areas that were rural. Additionally, we saw similar trends in individual large metropolitan areas with some exceptions for households with a Hispanic householder and households in poverty in particular metropolitan areas, suggesting that there is geographic variability across the country. Unfortunately, the data from the US census of sewer connectivity was not enough to look at smaller geographic areas there just wasn't enough households that were sampled to look at these smaller sample sizes. And so as a result to address this question about geographical areas that have low sewer connectivity, we had to turn to additional data sets, and we compiled data sets that we thought could help us address this question. But unfortunately, we were not really able to find any data sets that were really perfect for answering this question. So the data sets that we found were those that assessed connectivity to plumbing in general, so sewer septic and other forms of plumbing inspections for septic tanks and data from wastewater collection facilities about their population served. So first, this data about overall plumbing connectivity can give us a lower bound to the fraction of the population that's connected to sewers. And from this data we saw that there were large parts of Alaska, as well as the Navajo Nation in the Southwestern US that had very low connectivity to plumbing in general. So there were many counties that had more than 10% of the population that were not connected to plumbing. And so this already helps give us a sense for areas of the country where the access to sewers and access to wastewater sampling might be quite low. We also were looking at the septic tank inspection permitting in the state of Florida. And here we saw that there was a lot of variability in usage of septic tanks across the state with the most usage of septic tanks in the panhandle and inland. We also were looking at the environmental protection agencies clean watersheds needs survey, which is a survey that assesses the needs of wastewater facilities, specifically financial needs. And as a result, the wastewater facilities also submit data about their population served. And here what we saw was that in addition to the state of Florida, there were other states that had large areas of the state that lacked sewer connectivity. So in particular less than 20% of a county receiving sewer services. And this included Michigan as well as Minnesota. So, in light of the inequities in sewer connectivity across the country and across different populations, we wondered what, if any, is the applicability of wastewater data to neighboring unsampled communities. And to address this question, we took the approach of using simulations of interacting populations with different sewer connectivity. So, we used a deterministic compartmental model of two populations and be each with susceptible in fact it and recovered individuals. In the simulation, population a is entirely connected to sewer, whereas population be is only partially or can also be not connected to sewer at all. And we allow some level of interaction that is tunable between the two populations so that individuals from one population are able to infect individuals from the other population. And in particular we were interested in the question of how well can wastewater data predict the time of peak infections in the unconnected or less connected population. And this might be useful in a policy setting. For example, if you're thinking about the question of one you can relax restrictions. But with this model, we think you can also address other types of questions relevant for policy in the future. So, first, when population be is entirely unconnected to sewers. Not surprisingly, we see that the wastewater concentration is able to capture the time of peak infections in population be when the level of interaction is high enough, such that the disease dynamics in and be are similar. But we wondered if more population be were sampled, for example, if individuals and be were traveling to a workplace that had sewer sewers there or traveling to some other public area. With this, then allow the wastewater concentration to better reflect the time of peak infections and be. And what we found was that. Yes, this is the case that if more of be is contributing to wastewater than the wastewater better reflects what's happening in be. However, the strength of the interactions between a and B had a much larger effect on the ability of wastewater to capture population be compared to how much of population view was sampled. Additionally, if population be is smaller than population a, or if population be has a lower basic reproduction number. We start to see that the wastewater sampling cannot capture be as well. And this is because the dynamics in and be start to become very different from one another. So, to conclude, we found that access to wastewater based epidemiology varies in the US within and between communities, demographics and economic statuses. The geographic areas with low connectivity are existing in Alaska, Florida, Michigan, Minnesota and the Navajo Nation, and likely other locations that we lack the data for. So, for example, data sets are here and exhibit biases and discrepancies, and the data gaps are very, very much, especially prominent for small communities tribal lands Alaska native villages and at the state and local geographic scales. Even weak interactions between two neighboring communities can allow wastewater monitoring and one to predict the time of max infections in the other community, when the population sizes and parameters of disease spread are similar in the two populations. So the implications of our findings are that the data limitations point to a need for data consolidation and local analyses. Where the data does not exist or it's not accessible. Another approach that might be useful is mapping the catchment areas and their population sizes, which has been demonstrated very nicely in this recent paper. Estimating the extent of mobility between communities can help to inform how generalizable wastewater data can be. However, in regions without sewer connectivity or little interaction between connected and unconnected communities wastewater data from the neighboring communities will be much less informative. In these cases, sampling wastewater outflow at frequently visited non household locations might be very useful and valuable. And so, the takeaway is that analyses of sewer connectivity in combination with assessments of mobility demographic and pathogen transmission patterns can help us with the design of wastewater sampling systems and the interpretation of epidemic trends to yield equitable and informative sampling designs. And so, thank you so much for your time and I'm happy to take any questions. Yeah, thank you. And that was that was really interesting. One of the questions we've had is, is obviously the relation between income and whether the disadvantage and it have you kind of layered income into this model, because my just, you know, guess would be that your, you know, your wealthier households are much likely to have connectivity through work or other things into communities that are sewer versus some like Navajo Nation or some of those remote areas in Alaska, which I think might just guess would be there much less likely so have you looked at that or obviously you've thought of it because your data kind of takes you there so Yeah, that's a really great question I think also very complex question as you alluded to. So, we did look at whether there were associations of income with sewer connectivity and we actually found potentially results that could be slightly conflicting depending on what geographical scale you look at. So at the national level. I sort of flashed this really quickly at the beginning but what we found was that households that were not in poverty tended to be better connected to sewers and those that were in poverty. And this was at like a very coarse green geographical scale. And so, we think that there can be very different effects at a local scale. Navajo did look at within the state level data sets of Florida and the EPA clean watersheds needs survey, whether there were associations of income with sewer connectivity at the county subdivision and county levels. And there what we found was that there was actually a positive correlation between income and sewer connectivity so suggesting those that are more. Connected tend to be wealthier. Again, this is aggregating at a county level so it's really hard to know what's actually going on and I think this gets to the point about the need for data to actually look at this at like an individual household level, which unfortunately currently we don't have but the EPA has submitted a request to the American Community Survey from the US Census to ask if they could include a question about or expand their question about plumbing to ask what type of plumbing. The households have and that might help alleviate some of the data limit data limitations if that request goes through but I think there's still a lot of data gaps that would still exist. So, yeah, really good question and sorry, I don't have a better answer to. I think my question really is, is this connectivity between populations in that these the lower in the hypothesis that I would have is that the lower income or especially. In Alaska and and First Nations communities are much less likely to have an interaction with a sewer population that you would gather that information from that that's. So, yeah, that that then it splits apart. Because the wealthier communities on their own septic systems are more likely to interact with the community that it is sewer that. Yeah, totally. Yeah, and I think maybe looking at mobility data could help you start to look at that. But yeah, definitely not. I think we need to do a lot more work into this question. Okay, other questions. That's all Scott had one in the chat. We have a question from Chuck and Marissa. Okay, and Scott has one in the chat as well. So, I'm, well, you can look at that change and the one in the in the chat. I'm going to go to Chuck. Okay. Actually, let me let me have Marissa go first. I have a sneaking position. Our question may overlap. We'll see. I was curious. So in the in the model is a one. This is fantastic data. I'm, I'm like, really excited about all like we've been wondering a lot of these questions and it's super cool to see this. The second thing I was, I was curious about. So in the modeling portion of what you're talking about, it looked like you were sort of looking at the wastewater signal. And then kind of like visually assessing in some sense of how or like, you know, comparing peak times and things like that. I was curious if you had thought about other sort of inference ways of looking at that, you know, there, you would imagine that you could ask the question differently and say, I don't know if you might already be doing this but like, did you do any sort of like fitting of the model to the simulated data to see if you can recover. Like, are there signals from the connected thing, even when it doesn't have exactly the same dynamics, you would imagine that maybe the place you are measuring the peak gets shifted or there's a second wobble or you know like various things like that. I don't know. That was not well formed but hopefully you know what I mean. Yeah, I think so. And please correct me if I don't. But yeah, I think we can do a lot more with the model. So so far, like you said we only looked at the time of peak infections. But I think you could imagine looking at other aspects of the disease spread that you might be interested in like when you start to see uptake in the cases. Yeah, so defining some threshold for when you would say there's like a certain number of cases has been reached and asking when that occurs in the two different populations. And I think also, sort of what you're alluding to is maybe whether you could maybe even make a model that helps incorporate some of these parameters like mobility and population size, etc. To then use that to predict the timing of whatever you're interested in. Yeah, or to like infer the population that you aren't directly measuring right. Yeah, I think that would be really cool and we certainly have not dug into this at all so that would be a really, I think interesting direction for future work. But yeah, it would be great. Awesome. Yeah, super interesting. Sorry check go for it. I don't know if we overlap. No, no, it wasn't actually although we're getting the same thing. I wonder if anybody has looked at outbreak dynamics in communities that are seward versus not seward to see in fact if they are similar. Yeah. So, at the time that we were working on the model we did not see any other studies that had looked at this. But I think maybe since then, it's possible somewhere has been done I know some other people on this call. The group alley boom might be thinking about this question of how like spatial structure or like spatial subdivisions and other aspects of disease spread might affect like how how spatially dense your sampling needs to be so I think she'll talk about that later. So, yeah, to our knowledge, there hasn't been work looking at that but I think that would be really interesting. So we actually did think about trying to do this analysis and one challenge that we ran into was that the wastewater data was just so noisy that it's really hard to, or at this point, it's very noisy so it's really hard to actually do these comparisons and see a signal on top of the noise. But I still think it would be interesting to do and yeah maybe somebody else on this call is thinking about these questions. I feel like I haven't seen anything like this in the US I wonder if there has been stuff internationally with like polio, you know, like people did some of this kind of thing for polio I think when for looking at, like, measuring from not, you know, for unsuered populations where you're potentially taking environmental samples, you know, and even, you know, like canals and things like that. Anyway, I don't know off the top of my head of one but I wonder if there's something in that literature. Anyway, yeah. Okay. Yeah, thank you and please stay with us because we're going to have some additional questions. Thank you, Ron. So now I want to introduce Colleen Norton and Miriam Nuno. And I think they're going to present together they're going to they're going to work out how they're going to present. I think Colleen is going to go first but we'll give you control. Okay, great. Thank you. Can everyone see the slides okay. Yep. Great. Well, thank you for having us and it was great to listen to the first speaker as well. I'll be presenting along with Dr Miriam Nuno on equity and sampling optimization for wastewater surveillance in the Central Valley of California. Right. I just want to acknowledge that this is a huge collaboration of, you know, past and present students in different institutions and organizations that help make this possible. So I'm presenting on behalf of a large team and wastewater scan Dr. Ali Bame and Dr. Marlene Wolf will be presenting later they do. So our monitoring is run through them for our eight wastewater treatment plants and just want to acknowledge California Department of Public Health and our communications firm JMMB as well as all the wastewater treatment plants. So I'll be part one and just kind of a background I know you cited some of our work in your first phase report so I won't. I'm going to give you a brief update where we are on some of that research and talk about California monitoring equity and some initial analysis for United States. I know Dr. Bama will talk about that a little bit later. And a mention of global equity I know you're focused on the United States but it's important for, you know, pandemics don't know borders. So I'm going to give you some examples of what we've learned from all this analysis and then Dr. Nuneau will present about optimization modeling some initial modeling she's doing for California to optimize and have time for Q and a. A research for wastewater based epidemiology in California, one of our publications. And there was less representation of wastewater treatment plants from disadvantaged communities in California initially in 2021. A year into the pandemic when we did this analysis and so only 11% of the treatment plants in disadvantaged communities were monitoring for wastewater. And there was less representation also in rural communities only about 15% of the 48 plants at the time or in rural areas most were in urban coastal and Southern California so only 21% were in the Central Valley and like about 33 or 38% in southern and coastal California and only about 4% in Southern California. So we saw this as a need for healthy Central Valley together and so Dr. Heather Bischel she was doing leading healthy Davis together and we kind of expanded it for healthy Central Valley together. And then like in Merced, I had started COVID poops 19 and monitoring like where everything was all the treatment plants that were monitoring for wastewater and SARS COV2 and wastewater and I was like, really, we really need it here since the Central Valley is kind of like a healthcare desert almost and like a lot of disadvantaged and underserved populations and that that could really use this information. So this was, we have like eight wastewater treatment plants that are part of healthy Central Valley together to increase this representation in the Central Valley, and started kind of doing the needs assessment and consulting with local public health departments in 2021 to help identify plants we did, we did contact some plants that gave us a hard no. And actually, one gave us a hard no like Turlock where I live near Merced, but then there was a change in leadership and they also saw that we had done monitoring and Merced and Modesto, and that it was giving really good information so then they got onboarded later. And so then we've been monitoring, I think when we started with the scan or California scan it was like seven days a week for Davis Merced and Modesto and now we're then we shifted to four to five and then three times a week is still robust enough per week to get good trends over time. And so this is where we were currently still monitoring for SARS-CoV-2 and a lot of different pathogens as well flu influenza RSV and others. And this is kind of overtime the update from that 2021 publication. There's a lot more representation in California I think from 48 sites now to like 78. You can see there's a bit more in the Central Valley from our project as well as I think UC Berkeley had added like Fresno and Bakersfield before but then now they're part of California Department of Public Health and there's some more northern California sites so as your report said kind of equity increased over time but it's beginning we're kind of focused on like urban areas or the main wastewater treatment plants and we saw that for the United States and globally as well. This is a recent publication and Frontiers in Public Health where we, you know, we have all this data so we're finally getting around to analyzing a lot of it and for Merced, Modesto, and Davis we did some analysis this is just a graph of Merced showing that the wastewater data really aligns well with the cases and hospitalizations and ICUs. We did lag the data to match up with each other. We know that hospitalizations usually come a bit later after cases and same for ICUs. And so this shows that even in like underserved areas with less access to testing actually in vaccinations like Merced was like 38% when everywhere else was over 50% for vaccinations in the beginning of the pandemic. So there's still a good alignment between the wastewater data and the health metric data and it's important that programs have kind of equity based factors that they consider so that we can improve health equity with wastewater monitoring. And this is a master's student I'm advising at Loma Linda University, Dr. Ryan Sinclair so this is a bit, you know, older data takes a while to do this analysis and we'll see a little bit more from the wastewater scan project. But from the COVID poops 19 data that collects like we collect all the sampling locations. So in 2022 for social vulnerability index kind of and then buffering each of the plants. There was a little less representation in the higher percentile so from the past speaker this might also be from people being attached to like serve by septic tanks so not necessarily able to have a local wastewater treatment plant. And I know CDC has mentioned before that they had like 25% representation now. This was before the kind of more expansion of the CDC news program. But as you can see, like, yeah, initially in the pandemic, even the first couple years we didn't necessarily have kind of representation across the US for for social vulnerability. And this is global monitoring so updating I know like our original preprint that's now published was cited in the report. And that's when we had 58 countries of based on like one year analysis and now we're up to 72 countries so we went from 65% of monitoring in high income countries to 57% so we gained some equity, but there's still like a lack of monitoring and low income countries. By updating the dashboard I have seen kind of the dashboards that we have to remove or the sites to remove are the ones that are kind of in low and middle income countries, as well as we're tracking for flu and RSV. And I see those more in high income countries so people expanding targets really doesn't happen as much in low and middle income countries. And so it's important I think the CDC representative at the call on Friday was mentioning how they kind of support airport monitoring globally or at least network with them so it's important that the United States support these programs a lot of the programs are supported by you know like international funds. I know CDC does support some international like health projects and monitoring so it's important to keep doing that because it gives us an early warning of what's to come into the United States. This is some of the lessons learned from the monitoring for, you know, over three years, just about, you know, setting and publicizing and evaluating continuously or equity targets I know the prior speaker presented that there is more representation for race and ethnicity with wastewater monitoring. So it's not really like publicized very like much on like CDC's website, or like some of the health department websites like it's, it would be good that we keep this at the forefront of, as we have our programs for wastewater monitoring. And even if we use something like social vulnerability index or EJ screen, it's still good that we evaluate for race and ethnicity overall because it can be washed out and some of these indicators. And as the prior speaker mentioned there's kind of like lack of data in certain areas and we find our analysis difficult to do even for equity because we don't have a lot of the sewer shed boundaries for all the treatment plants in the US, or like their population served. This is the EPA 2012 database that's a bit out of date. So it'd be good to have those to do more of this evaluation. And we found that consulting local public health departments also helps. I'd better identify treatment plants they identified like Los Bonos in Merced County it's like kind of this commuter population between the Bay Area and the Central Valley and they often we see the virus levels increasing in that treatment plant that then goes into community more into Merced that they kind of bring the virus from more in the urban areas. And so that's been an important plant for us to monitor that we wouldn't necessarily know, unless we talked to the health department. And we provide a bit more service to rural and underserved treatment plants, like there's a courier service that would be difficult for them to mail the samples for example they have one of our plants is like a mile away from the mailbox and they're also like over 20 minutes from the center of town so it'd be hard for them to do the monitoring if they didn't have a courier or like and we've provided a lot of like kind of training and just overall support to those treatment plants that are more rural and underserved treatment systems or that only have like for staff. And to get more representation in these underserved areas you can really leverage as we saw in the pandemic, the universities that are placed nearby, and also just a reminder to support rural and underserved treatment plants and low and middle low and middle income countries continuously, even as we optimize I hope we don't lose some of this representation and rural underserved areas and in low and middle income countries. So, I'll let Dr. Mariam Nuneau go next for part two on optimization modeling so apologies if I went a little over. Thank you Colleen. Yeah. Hi everyone. I'm Mariam Nuneau, Professor from UC Davis and my statistics and just a really brief outline. We will show a little bit of the work that we've done before. We will focus on the optimal spatial allocation of wastewater treatment plants, and we will, you know, have a focus on thinking about balancing colleges and population vulnerability, and we'll share some thoughts about possible future. So, you know, our team was really lucky to connect with people like Colleen and Heather to get on board when COVID-19 hit and sort of the focus of our work has been, you know, how do we use math and statistics to provide actual information about the COVID pandemic and in relation to wastewater. And so one of the recent publications as Colleen just mentioned was this first one when they looked at expansion of the wastewater disease surveillance to improve health equity in California, some of the work that we've done is looking at Bayesian spatial approaches to monitor COVID variants through test positivity rate from the wastewater. We've also looked at the importance of training periods and how that impacted COVID incidents. So we were lucky in the sense in the city of Davis that the city had mass testing for the entire city, which was not true across the country. And so we were able to do the exploration of, you know, what does it look like when you train your models when there's very limited testing and what does it look like where you have high levels of testing and how does that impact your projections. And so we had some work on that. I won't spend too much on the social vulnerability index since Colleen has already explained, but we will be looking in when we're thinking about optimization of locations for wastewater treatment plans, we will look at social vulnerability. It's one of the options for thinking about vulnerability and just to know that these measures, the community is vulnerable compared to other communities at the same level. So it's a rack or percentile. This discussion today will be based on California 78 wastewater treatment plans. And this is what the breakdown of California looks like. So we can see. And the, and the shading of green, the locations that have, you know, high vulnerability in terms of the social vulnerability index. And we can also see the population that is served. So the areas with higher volume of population serve are the lighter areas. In terms of the data they will be using, as I mentioned, it's going to be 78 wastewater treatment plants around California, but easily one can think about the expansion of these work across the nation. And these are the areas that are actively monitoring wastewater data and is publicly available through the city pH. In our specific analysis, we will look at geographic coordinates. And, you know, thinking back to the discussion that Chin Chin and Colleen said is, you know, the data that we have is often a different scale. So for geographic coordinates, we will be looking at seed codes for population density. We will be looking at the corresponding cities for the population that is served. For population service, the estimated number of persons served by the sampling site. And in terms of the SARS RNA concentration, we will be looking at at the plant level. In terms of the social vulnerability index, we will be assuming 1 at the county level, but as we know, it could be different levels. So, you know, thinking about the problem of solving optimization is thinking. You know, for what variables do we have county level for what variables do we have simple level or city level. So, you know, in terms of accessing the data on the same scale, it will be an ideal situation. But we're just taking what we have. In terms of the problem formulation that we have, our goal is to optimize allocation resources for wastewater monitoring across California. Thinking to account at special distribution coverage and prioritizing communities that are vulnerable. The first part of the problem is we're proposing this as an integer optimization problem. And we will use a simulated annealing, which is a probabilistic approach to find a maximum to a proposed objective function. And I will share with you what the objective function is. These approach allows us to explore the solution space effectively and find an optimal solution to meet the needs of our objectives. And so the objective function we have is a function that takes into account population serve population density, spatial coverage, social vulnerability, and disability of similar signals. So often, you know, I think as Chin Chin was mentioning, you know, when you're doing modeling and you're trying to borrow information from neighbors. If the information is similar, the trends and similar, then that's really not adding much value. And so we wanted, you know, to look at, you know, how when we do the modeling, you know, having different trends and similar signals. How does that impact our optimization problem? And our problem is subject to constraints based on our priorities. So, you know, our priority may be to say, you know, let's take 70% of the current plans, 50% of the plan. So if we have a specific goal of, you know, achieving this number of plans that we can have that to be a constraint, we can also have a constraint that is based on the population serve. It could be a constraint based on cost. Among other different constraints. So the function for population serve, when we prioritize on population serve, then, you know, we're prioritizing plans with the higher population serve. And this piece serve here represents the population server plant A and SP serve corresponds to the sum of those populations served by all plants and the set of eye and this case we have 78. And we go to the next term, then now we take an approach in which we prioritize density, then it takes into account those areas with high population sensory when there's possibly this is transmission is more likely to occur. And so, similar to the previous life population, probably of density represents the city's population density of plant I and SP density is the sum of the population densities that we're considering. We similar can think about the possible third term where we are penalizing selecting plans that are too close to each other. While promoting a more geographical spread distribution of plant. If we wanted to prioritize that on that that we can do that as well. Since, you know, a lot of the work that we have been doing has been prioritizing and vulnerable populations, then, you know, taking into account this term here would be, you know, a favoring the representation of vulnerable populations in our optimization problem. And I mentioned that, you know, depending on the question and the priorities, the different items they're going to this optimization function could be taken into account. So this fifth term will focus on say, assessing the similarity between different wastewater signals produced by different plants over over time. And the primary goal in that scenario would be to group wastewater plants that are similar based on their signals, and taking into account different, you know, trends, taking into account different peaks, and the magnitude of the signal variations, and taking into account this and adjust, you know, reduces the redundancy that different signals may provide. So this aligns really well with what she was mentioning about, you know, possible way to think about that. So this is sort of right off the oven calculations. On the left, we're looking at three different scenarios. So we say, you know, assume that we have a scenario where we wanted to reduce the number of plants by 40%. So instead of having 78 we say, you know, we want to plan for 43 plants. Scenario A, B, and C are taken into account three different situations. Scenario is, is we prioritize the social vulnerability index. And we can see the, you know, when we prioritize the social vulnerability index, there's higher coverage in the communities in the, in the red and the high region, which is a high social vulnerability index communities. If we prioritize population serve, then, you know, we see that there's a higher density in the Bay Area. And then if we say, you know, we just want to reduce the number of plants and we want to go equal across, you know, so population serves social vulnerability index, and we equally weight those factors that this is the solution that we will have for that scenario. And so for future work, as I mentioned, this is just something that we're in the process of developing. We haven't published this work. And, you know, we want to sort of think about solutions, something that, you know, Guy Palmer mentioned when we started is, you know, he mentioned that we're thinking about systems that are implementable, that's sustainable. And one thing that I want to add in there is, you know, system that is adaptable, because what we've learned from the pandemic is that there's nothing, you know, that is fixed, everything changes. And he actually goes to also the approach that Chin Chin took in terms of her domestic model, you know, thinking about ways in which you might implement Bayesian approaches where, you know, you learn from the past to improve your future. And so I think that this is sort of what we're thinking, you know, how do we, you know, solve the problem of, you know, surveillance, but also make it that it's adaptable because things will change. And so in terms of things about the future, we mentioned that, you know, similar ways, whether signals is something that is important that we might be able to borrow from the neighbors and we don't have to monitor everyone and just optimally do that. You know, we're thinking about really taking this problem and trying to think about that, including terms that are selected that are strategically distributed across specific region regions considering both demographic and geographic factors while really reducing redundancy and the characteristics of the signals. So, no, not that, you know, have also been interesting to me is, you know, when we think about the CDC and what the CDC is trying to do, there is a problem of local versus national. And, you know, how do you give power to individuals locally while still, you know, monitoring nationwide. And so I think this is something that is really something that it's important to think about. Considering regional minimums to ensure geographical, geographic distribution, considering potentially healthcare regions for research planning. So all of the work that we're doing for monitoring what is it for at the end of the day is it to reduce overwhelming hospital, you know, admissions. And if that's the case, they know we can think about, you know, including in our optimization of terms, maybe health regions that could be relevant. And also, you know, our team has done monitoring of relevant early indicators such as Google trend, Twitter's Twitter data and Dr. Visas and we've seen a very strong correlation of those signals with wastewater. So if we're thinking about downscaling on some locations, continuing to monitor them through these signals that it's, you know, available could be an alternative solution. Thank you. All right. Thank you very much. In fact, if you want to chair this committee, I'm willing to give it to you. You tell us an excellent presentation that addressed some of the, the really things you've struggled with. Sorry, I apologize very much because I really wanted to encourage everyone here, especially would like to acknowledge Ali and Marlene because they have been fantastic and leading all the efforts in California and if it wasn't for them, we would have zero data so thank you all. All right, thank you. We have about five minutes for questions. And we're going to take a 15 minute break but please stay with us because we're going to have plenty of time for questions. Later on we just want to stay a little bit on that schedule, just for the for the other speakers so they they'll know when to join. So questions. Yep. So Krista then Marissa. Great. Hi, thanks Miriam that was really interesting. And I wanted to ask a question about that this dissimilarity and wastewater signals which is really, really, I think it's potentially really powerful. Like, in your analyses when you have 78 plants and I missed if they were all analyzed by the same group and if they were all analyzed with the same methods, wondering like when you start to think about different groups and different methods being used different pathogens being looked at. How do you then like tease out this dissimilarity. Thank you for that question and I think this is something that, you know, I will say that as a person that doesn't process the data. I'm always just like give me the data and we'll start doing things with that and then we have, you know, calling ahead or like, well, hello, I'm Mary and this is in Florida hold on Mary and this was processed by this, you know, company. And what I have what I have seen with that, you know, in very much detail is that, you know, it's not a doesn't matter because it does matter. But when the signals are, you know, coherent is really overwhelming. You know, and we have some graphs in which you can see, you know, you can, I can just plot the data and without knowing, you know, who processed it or so. And many times, at least, you know, when in the past we've had, you know, because we said a lot of, you know, is signals right when the signals are smaller than is harder but when the signals are imminent. It really is, you know, overwhelmingly coherent signals and and so we've done that for the whole California we can see, you know, within regions that, you know, overwhelmingly that 80% of the time they all look the same. When you start looking, I say, within a city or within a county, where you're dealing with, you know, small sample sizes of populations being served. Then I think we've noticed a lot of noise at the county level. So often we say, you know, of course, you know, for Yolo County we might have four, but really just taking the signal where the highest population has been served. We often feel like, you know, the other signals are not really adding anything but noise. Right. And so I feel that thinking about, you know, the scale that is informative. It's important, but to answer your question really simply, it's, you know, really, those things are like highly correlated with each other when you see a signal. Marisa, do you have a question? Yeah. Well, I'm sorry, I got, I had to do a thing in the middle of this. So it's possible you answered some of my questions already. I guess my question, I had two questions. So one was, in the optimization process, I was curious if you've looked at all about whether the optimum sampling, you know, structures that you get are unique. And if not, if you've looked at like sort of like what alternate, you know, like given that they're very correlated, sometimes they're quite similar. I would imagine that there's many situations where you have five plants and you could take any one of these five and it would be about the same. And so I'm just curious if you kind of looked at that. And then on a related question, if you've looked at when you, when you, if you're going to reduce by 40%, I don't know if you've done anything with like mutual information or anything like that to look at the information loss that you have when that happens. Anyway, that was all. Yeah. Thank you. Thank you. Yeah, we were, you know, we've started playing with this approach, you know, literally like maybe a month ago, but one things that have become clear to us is, you know, as we discuss with the colleagues that oversee their plans is, you know, after what they were saying to us is like you look at the population density, but then the plan is located in a place where population density doesn't seem relevant. So I think, you know, to answer your question that there are a lot of nuances that we definitely have to keep in mind. In terms of, you know, what it means to keep one or other. One question that I have as really answering the real question for me when we think about optimization is, you know, if I'm going down 40% in terms of the monitoring plans. What power does that give me to predict or forecast. Things, right? Infections, possible hospitalizations. If you can show that reducing your plans by 10% or 20% gives you similar forecasting, you know, approach, you know, solutions. And I feel like that's where it becomes valid because I'm thinking that, you know, I don't want to be in that conversation when you go to a county and say, hey, sorry, we don't need you to be monitored anymore. You know, people don't want to hear that probably, right? But if it's there, you say, look, we have some some work in which when we do reduction or done scaling by 10, 20%, we still are able to give you information that is important for you to make actions. Then I feel like people will be like, okay. And for me, this optimization problem is actually just passing through where the actual question is actually bigger. Like, is they going to give us the information? You know, that's 40% less give me the same information as the whole thing. That's where I feel like the question is becoming more relevant. But we're still young in that. But yeah, we will connect with you to pick your brain. Oh, no, it's fine. I just was curious. Yeah, I mean, I would also be super curious. Like, I can imagine in decision making questions, the non uniqueness question is relevant because it might be convenient for various reasons to pick this plant over that plant. And so, you know, it'd be interesting to think about how we communicate that and those kinds of things is just absolutely. Absolutely. Christine, you have a question. Okay, last, last question before break. Really quick. I know you presented a lot on the prediction and trends for COVID, but I'm curious about your work to apply this to other emerging infectious diseases, especially ones that might be more rare. Thank you. So, you know, I would say that that's a question that I think is important to us because, you know, as I was mentioning to guy. Implementable, sustainable and adaptable like, you know, how do what we do is not just covered, you know, is flu is RSV. And I feel like, you know, I've been thinking to myself, like, how do I not run out of business with ways water surveillance is like keeping it real right and there's, you know, sort of something that is still relevant. And so I think you're, you're, you're perfectly right in that we need to think about how this could be used to other, to other pathogens. Because we've learned so much and then now we're just going to hopefully, you know, having with surveillance for I don't know how many years into the future so I think the more relevant that we make it to other diseases. And then more, you know, sustainable and prepare we would be in the future so definitely I think thinking about other applications. Okay, let's take a break until a quarter after there are two questions in there. I think kind of to everyone one specific to Miriam, but we can get back to those and let's reconvene it a quarter after. Thanks. All right, we're at a quarter after so let's go ahead and get started. Thank you for staying with us. I'd like to introduce Alana Chan who's going to speak about sampling optimization at the national scale. And I think she has control of her slides. I actually I'm going to start and then a land. Okay. Present slides and I'll present and so my name is Ali Bame and I'm excited to share this presentation with the Atlanta, who's a PhD student at Stanford. We've had some contributions from Arlene Wolf as well to this presentation. So wastewater scan is a program that measures concentrations of infectious disease targets across the United States and we have all these different sampling sites, and we're using a consistent method that we believe is comparable across sites so this allows a really great data set to probe some of the questions about temporal and spatial sampling as well as redundancy in targets actually that we're measuring so we're going to talk about planned research and previous research that we have done with this data. And the frequency of sample collection varies between the sites from most or three times per week but some are daily which also offers a really rich data set to look at some of these research questions related to frequency and spatial scale of sampling. So we just instead of having like a really complete story here we just have like some teasers and some thoughts to share with you all because we're not going to pretend like we have all the answers here. So first you know we were asked to think about how often and where should we sample and how to reduce redundancies, and we think that the answer to this really depends on the use case which I think has not been necessarily clearly defined so, for example, here are two use cases and maybe there's more one is for like bioterrorism threats emerging and imported infectious disease agents in biological agent discovery. So wastewater surveillance for these may not require frequently spatially dense and equitable sampling instead maybe you're targeting ports of entry densely populated urban centers, areas adjacent to dense animal operations, etc. And you're not maybe doing frequent sampling because you can bin everything together and just screen it for presence absence of like Ebola or measles or something. In this case to this is I think more in line with how we're using wastewater surveillance right now, we're doing surveillance on common and recently emerged circulating infectious diseases to generate a quote weather report, or a health report for the public and targeted public health messaging, inform clinical decision making and allow the public to take protective actions. So, in this case, the temporal sampling should be often enough to identify changes in levels, trends and frequency of detection of the targets. And spatial sampling should consider equity issues because if the public is ingesting this information and potentially benefiting from it, you don't want somebody to be left out and feel left out. So we're really focused on use case number two for the thoughts in this presentation. And I'm going to pass it to Elana who will take over for a bit. Yeah, so for temporal considerations, one of the main questions we have is how often do we actually collect these wastewater samples. And so, if we think of daily sampling as the gold standard for wastewater based epidemiology, we can kind of down sample to kind of explore these questions. And we know that daily sampling isn't feasible for most places, which is why this question is important. And so, like Ali mentioned, some of these sites have a really rich data set where we have daily sampling. And so taking those data sets we explored these questions. So, if the use case is trend analysis, our down sampling analysis that was published in plus water earlier this year suggests that collecting samples four to five times per week is sufficient to correctly or identify trends in wastewater that were obtained with the daily data. And so you can take this framework and apply it to see if reduced sampling is sufficient for other use cases, such as modeling cases from wastewater or characterizing levels of wastewater as high, medium or low. Next slide. And then there's several considerations on where to sample. So I'll go through each of these next but briefly they relate to mismatched jurisdictions, attributes of the contributing population and spatial coherence and wastewater concentrations and disease occurrence. Next slide. So yeah, beginning with the mismatched jurisdictions sewer shed boundaries often do not align with other surveillance data or public health jurisdiction so there's this mismatch in geography. And so a sewer shed can cross multiple cities or counties, and one city may contain multiple sewer sheds. There's also changing inputs to consider so wastewater can be diverted between a couple of different locations, depending maybe on like emergency situations or just that area service by multiple treatment plants. And then another thing to consider is that case data is often geocoded to a person's place of residence, but wastewater kind of depends on where people are actually contributing. So someone may reside in one sewer shed but work in another sewer shed. And so their data is contributing to multiple sewer sheds, even though if they're infected they're really only one case. There's also this mismatch in responsibility. So wastewater systems are often at the purview of cities whereas public health responsibilities typically lie at the county or state level. And so along those lines, I'm thinking about strategic site availability. If a public health agency has strategic goals for sites, but those sites are unable or unwilling to participate. There's kind of this mismatch in responsibility and kind of what can actually occur. And so, and also just like as I mentioned before not all sites have a high enough networks, sewerage coverage. And so at this time we don't have any clear recommendations on these points but these are all just things to continue thinking about as we think about national wastewater surveillance system. Next slide. Yeah. And so then another thing to think about is attributes of the contributing population because if we're thinking of a national system we want it to be both equitable and representative. So as I start with the wastewater skin network, we decided to look at two metrics that are calculated by the CDC, the social vulnerability index, which has been mentioned previously, and the Environmental Justice Index. And so as you've heard, kind of how these indices work is they take all the census tracks in the US and give them a ranking value between zero and one, with zero being those of the lowest vulnerability and one being those of the highest vulnerability. And so this overall social vulnerability index contains a whole bunch of variables that go into this calculation but they're grouped into four different categories, as you can see here, like social economic status and household characteristics. So what we did is we took all the census tracks in the US and grouped them into those that fall within our wastewater scan network, and those that do not fall within our wastewater scan network to see how representative things were. So overall, looking at these box blocks, you can see that census tracks within wastewater scan for the most part seem to be representative of those gotten wastewater scan. With the most clear exception being this racial and ethnic minority status category, we can see that the social vulnerability index is higher for the wastewater scan network. So this means that there are more racial and ethnic minorities being captured by wastewater scan compared to the general population. And a likely reason for this is that the wastewater scan network is more concentrated in urban areas where racial and ethnic minorities are more often resigging. Next slide. So yeah, we also did this for the environmental justice index which has three overarching themes that go into the calculation of social vulnerability, environmental burden and health vulnerability. I also pulled out one of the variables that goes into that environmental burden domain, which is air pollution, since if we're thinking about respiratory viruses. This could be an important environmental characteristic to think about. And so once again, for the most part, things look to be pretty representative. So these are all just important to consider as we think about a national wastewater surveillance system. Or if we were to down sample sites and just to pick a few sentinel sites, we want things to be representative and also equitable. And so as I mentioned in a later slide, we're also considering other variables to kind of characterize the wastewater scan. Next slide. So then the final point for sampling or spatial sampling was spatial variation and disease occurrence. And so on the graph on the right is influenza onset and offset and multiple sites that are span over 100 miles. And you can see that the onset and offset of that peak was nearly identical so there was really no additional insight gains by measuring multiple sites. But on the plot on the right, which is a heat map for M pox, you can see that currents of M pox really differed across these sites. So we were able to gain additional insight by sampling at multiple sites. And so the kind of takeaway here is that different spatial resolution of sampling may be needed for different diseases and also different use cases. And there are a few reasons for this and this includes the property of the disease. So maybe it's effective reproduction number property of human behavior. So how the disease is actually transmitted and then mixing a population. So maybe rural areas. There's probably less mixing compared to urban areas. Next slide. So yeah, we have multiple kind of questions that are planned and ongoing to address these data using or to address these concerns using the wastewater scan data. And ultimately we hope this will inform where sampling should occur. So in terms of equity, we're planning to see how wastewater concentrations are correlated with determinants of health. And so taking those markers of social vulnerability and environmental justice and seeing how they are synced with wastewater concentrations. And then other determinants of health that we have in mind are looking at urban versus rural sewer sheds sewer sheds with that have a lot of ports of entry so especially looking at airports perhaps, and then also sewer sheds with a high number of healthcare facilities. In terms of sampling site redundancy we plan to look at whether wastewater concentrations exhibit local spatial autocorrelation, which essentially means seeing whether sites closer together are more similar in terms of their trends and the relative wastewater levels, compared to those that are further apart, and seeing if this changes over time. Because if we see that sites exhibit spatial autocorrelation, this could suggest that picking just a few Sentinel sites would be sufficient. In terms of the different diseases we hope to explore these questions for different targets to see if the patterns are similar or different and this hopefully will inform whether the sampling recommendations we have are disease dependent. So kind of like I mentioned with that difference between influenza and epochs, the kind of takeaways can be different depending on the use. Okay, and I will do the last two slides. So the other thing to think about is optimizing what to measure. And this slide shows some data from a wastewater treatment plant there are three axes the top one is SARS CoV two, the middle one is four different enteric viruses. One is six different respiratory viruses. And I guess the thing that I wanted to point out here is that we see cohesive, we see coherent patterns in almost all these enteric viruses so the axes are scaled from zero to one just to look at their relative types. And so they all sort of go up around, you know, late winter peak in the spring and come down. And then for the respiratory viruses it's a little more complicated but you this was also like during the omicron surge but we do see that there is some coherence and some of the respiratory viruses like RSVB rhino virus and human metanomavirus and where they sort of go up together and then they come down and stay down. So I guess the point here to bring up in thinking about redundancies and optimization is whether we actually need a network to measure all these different viruses or if we can have a representative enteric or a couple representative respiratory viruses that would represent the behavior of more than one disease at once. So something to think about. So lastly, just some concluding thoughts is there are a lot of challenges here. But we do think that representative sites or networks of sites can provide regionally and nationally relevant information beyond exactly where they are located. And there are other examples of surveillance networks that are at the national scale that use Sentinel sites like the emerging infections program, which some of you may be familiar with at CDC. We're 10 states collect really intensive surveillance data on infectious disease and coordinate and share it to inform national recommendations and insights into disease circulation. So maybe that's a model to think about and look more into. And then we probably don't need complete coverage to be regionally and nationally representative. So that's the last slide of our presentation and we're happy to take any questions. Any other time I guess for question. Oh, thank you. No, we do have time for questions. And then we will have some other presentation but we have we have time here for questions to see if I can see hands coming up. Sandra. Thank you both Ali and Marlene. There's really differences between what the national system goals are versus a local system. Can you kind of compare and contrast how you would optimize the national system versus maybe optimizing within a state. And what some of those differences would be. I know it's a broad. That's a science question Sandra. I don't know I'll pass it to Marlene you can take that one. Yeah, I think it's it is a little bit more of a policy question. Yeah, I have a couple things all related to though I guess that people who are maybe our policy minded would be considering. I think the use cases are a really important part of this and that whenever we talk about the representativeness or the usefulness of these systems we really have to say for what right. And so if you are, you know, a local county public health department, then you know your determination of what information you need in order to track COVID surges or the start of flu season or, you know, an Mpox outbreak that you want to be on the front of awareness of that you think could be possible. You know that's going to be a different answer that you're going to come to then CDC saying you know what are some some key sites that we can look at throughout the US to understand the start of flu season. I think that when you have, you know, those local local priorities and local public health departments, determining their needs they are often going to mean that every single one of those departments is going to have a set of sites that have specific priorities, whereas from a national perspective, you know, we see some of these features that Alana and Ali just talked about that give us some spatial coherence for example, or you know we can see I think some trends overall at a high level, but that isn't necessarily going to give every county health department the information that they want about their local community. So, I think we can use some of the things that we've learned about sampling frequency about the thinking about the you know what it means spatially for us to have good information about a certain community and then the policy question about well what communities do you need that information from and where is that placed. I think that that is, there are probably many answers to that that question depending on your goals. Thanks, and just a little bit of a follow up, I guess I've thought a lot about, you know, is the national system. Not redundant, but how can it be coupled with the local systems because there are those two competing interests so how do you remove the redundancy between the national system and the local system when they're serving kind of two roles and you might have an answer for that but I think that is part of the optimization, you know question that you maximize both what the states are doing with what the national systems are doing. I'll just say quickly, right now the system systems systems as they exist are that state systems are the national system right that there is the every state and their decisions about their system and I've talked to many people of you know they're different good strategic lenses that states are using right to to select sites to reach out to sites. And then those are the sites that are in the national system because that data is getting funneled up and so one of the challenges and opportunities I think is that there are a lot of different sort of approaches to prioritization out there within these these different systems. And so bringing together these conversations about you know what are, you know different public health agencies using in some cases as their prioritization along with, you know what maybe CDC or the EIP program for example, is an example of you know sort of CDC working with states to say here's a network that will be representative but it's 10 states you know and in every state wants to have a wastewater program we think right so yeah that sort of what you need at a state or local level may not match what we need for the national big picture but they are functionally the same system most of the time. And so that crosstalk, like in this conversation today I think is really helpful to optimize both. Thanks. Thanks, Marlene. Marissa. I wanted to sort of follow up on that set of questions. This question is mostly actually me pestering alley and Marlene maybe or whoever wants to to talk about in a call a few weeks ago. I had a discussion about levels and metrics, one of the scan calls. And one of the things that came up briefly was the idea of like abstracting levels or metrics or whatever to larger national scales and I think that's really important here. When you're in a national kind of a use case thinking ahead for like news, you often want to understand what a region of the country is looking at. And so how do you how are you all thinking about abstracting upscaling, you know those kinds of things. Yeah, I, you've asked that question on that call and at this point we're working on mathematics of, you know, population waiting data and making sure that, you know, issues related to the mathematics of that and I haven't forgotten to share the math with you I just, I was glad that you all were thinking about it and I was hoping you could, because I think that's something we're going to have to grapple with and and is really well positioned for that so yeah. So, one of the, one of the things I think that we've been able to overcome with everybody using different methods to make measurements is that trends are this kind of the same between methods. And that maybe isn't the same necessarily our levels right and both levels and trends are really important as is frequency of detection right depending on your detection limit and your sensitivity of your assay so you know we have identified three different archetypes of pathogens that we're measuring and pathogens that are very rare like where we look at the frequency of detection as being an indicator of whether something is a low, medium or high wastewater level. We look at things like SARS-CoV-2 that seems to always be around we look at trends and levels to identify whether we're in low, medium or high category for wastewater. And then the seasonal pathogens which are either like influence that either don't have it or you like have a lot of it and and that's a different archetype. And so, if you think about how measurements relate to getting that information out, there are certain things that you simply are going to have more trouble with if you're using a different method for measurements across across the country. With our project we have the same method for measurements so we don't have we can eliminate that problem and so we can sort of think about binning things and aggregating things but we definitely have grappled with that Marissa like how do we make a we're not going to provide aggregated information at the state level but we're going to do it for we think we're pretty sure regionally like the census regions and nationally. And so to do that we are aggregating data by population and we're also thinking of waiting by state population as well so that one state isn't over represented by having more wastewater treatment plants so but there will always be that concern that the regional or national characterization does not characterize things correctly at the small town where you know you're planning to visit so which is true for any kind of aggregate surveillance system anyhow but yeah yeah yeah totally. Yeah. Yeah, I'll be very nice work. I wonder if you have enough data to differentiate between combined sewer systems and separate sewer systems in terms of how they behave. Yeah, that's a great question Chuck. We have just finished compiling all that information for all the hundred and 80 sites in the network and we're working with some researchers at Carnegie Mellon who are interested in PMM of variation between sites and so they're including that variable in their model. I'm sure Elana will include that in her work as well, but not yet not nothing to share yet but certainly possible. Any other questions at this moment. If not we will have still have more time for questions after our next presentation which Ariel if you're ready. Yeah, I'm ready. Okay, perfect. Great. Well, thank you all for having me. I'm going to give a little bit of our state perspective so I think we're kind of coming back around to those. Those questions my pieces well timed here. Can you all see that okay. Yes. Great. So, I'm Ariel Christensen I'm an environmental epidemiologist at the North Carolina Department of Health and Human Services. I'm our lead epidemiologist for our North Carolina wastewater monitoring network. We were actually one of the first eight states to be funded by CDC news. So we've been around for a little bit of time and have really just, you know, really tried to do a lot of work, obviously at the state level with building an equitable and an optimal kind of framework effectively for COVID-19. And as you all kind of know from being involved in this work, the kind of questions around wastewater have really shifted and I think that's the biggest thing that I've noticed in the past few years has been. We really built the plane while flying. During COVID it was standing up a massive surveillance system for states who've never done this kind of monitoring before and it was a collaboration between infectious disease folks like environmental health folks like totally across different disciplines and trying to do that in real time during a pandemic when we all were really, you know, pressured and for a number of different things and experiencing our local officials, you know, experiencing a lot of pressure and a lot of, a lot of, you know, terrible kind of things. And so I think, you know, really just seeing like how much the questions have changed, you know, trying to build that plane while flying for trying to understand what was going on with COVID trends. And now the questions have kind of shifted, you know, since the public health emergency has, you know, ended. We're really trying to think about how to use the, what we built to be able to still look for other pathogens and kind of evaluate what we did during COVID as well, and see if this, if we can use some of the lessons learned during this time frame to be able to kind of move us forward. So that's my, I guess, little caveat background to everything. So in North Carolina, we actually started with only 11 sites in January of 2021. And this is because they were really well positioned around our academic institutions in North Carolina. So around the Raleigh Durham area around like Charlotte, around, you know, ECU Greenville area where we have like major universities. And we're doing a lot of this academic research. And so we tried to leverage on what they were learning and what they were able to do to be able to have they had this these collaborations with our wastewater utilities. So we really just tried to use what they already had to be able to start our network. And we now have expanded, you know, to 50 sites across the state and all at the municipal like sewer shed level. But we really tried to and adding sites to be really intentional about who we were adding to the network. And as you can see, we've really increased our geographic range of different sites. We have, you know, when we were doing this during the pandemic, we were trying to look at things like social vulnerability that's been mentioned so much today, we've been looking at, we were looking at COVID testing rates and vaccination rates, when that started to come out. We were looking at like overall COVID burden in some of these areas. And obviously looking at what available data we had for our sewer sheds, which was really sometimes difficult to obtain. And it's still really an ongoing process of trying to get sewer shed level data and accurate networks in in our jurisdiction and I think a lot of other states are grappling with that as well. And so we have really tried to expand, you know, with those kind of parameters in in mind, using just like a general basic ranking, and have been able to get a number of these different sites involved in the Biobot program also like wastewater scanned, as well as news to be able to bolster like our outreach and the amount of areas that we're able to oversee. But this question about representation and you know what the kind of limitations of wastewater were kept coming up. So we, and rightfully so we should be thinking about these things, especially when building a new surveillance system and kind of continually coming back to this question of like representation and how we can interpret the data, you know, for public health action I think that's really important component I want to go into as well here but so we partnered with Mathematica and NC State to do an analysis of our sewer sheds in North Carolina so we had two main questions we were trying to look at are the sewer populations that are monitored across North Carolina, representative of county wide and state populations with respect to different demographics and social vulnerability. And the team that did this, Mathematica and NC State and a person from our team, Stacy Reckling who's now part of the news team use 21 different census indicators to look really extensively at this data. And we were lucky enough to have a GIS analyst who has been a part of our team Stacy since the beginning of the program and she really has pioneered a way to build sewer networks using geospatial resolution data that is, I think a really important component to this conversation is just making sure that we have like the most accurate data possible to look at these different geographic scales. And the second question that we were looking at were, are the demographics and social vulnerability of sewer sewer populations, irrespective to whether or not they were included in our program. How are those compared to the unsewered populations. So, this publication is still in process. So, I'm just going to give you all like a very basic teaser. But overall what we did find is that we were looking and this was only when we had 25 of our monitored sewer sheds. So we're hoping to redo this analysis with all 50 of our sites now. But overall what we did find is that, on average, looking at SPI, which you all know is a combined aggregate of a number of different census measures is that our monitored sewer shed populations were similar to our county populations and there were a few. And then overall they were really similar to what we were seeing in our state. And I thought that was a really interesting and helpful takeaway for us, obviously, as we're considering like how we interpret these data and what they really mean from both the county perspective as well as the state perspective. And we did see, when you compare to the state, fewer actually white populations in our sewered population compared to our unsewered population. So that was really, again, interesting as well and kind of contrary to some of the findings. I think at the national scale and just speak to needing to do these kinds of analyses at different geospatial resolutions in order to understand kind of the complexities of what your state or what your jurisdiction is dealing with. So we found, you know, overall the sewered populations were more Hispanic, greater African American population, lower household income and lower educational attainment, as well as lower and social vulnerability. So I think that is just something that we're going to continue to look at. And I think it also, if you understand a little bit about North Carolina, we have a lot of septic. So about 50% of our population is on septic. And I think that makes some of these findings very, very difficult and kind of speaks to not needing to continue to do this analysis and get data on where our septic tanks are in North Carolina, which is kind of an ongoing problem is like where, where are these located where are these people located. And I think, you know, I think we need to be thinking about those kinds of questions as we interpret this data because there's, there's, people don't quite see themselves represented all the time in the data right if if they know that they're connected to the septic. So I'll just end with like a few questions that we're still kind of grappling with in North Carolina, which is, you know, we're kind of we're coming back to this idea of Sentinel sites which I know have been discussed a lot and I think we're really trying to repeat this analysis and make sure that any sites we do retain in the program and, you know, continue to do monitoring for our representative of North Carolina's overall population but also that we might be needing to oversample certain vulnerable groups in North Carolina because of kind of what we found with the rural versus urban. There may be like a need to maintain some of these smaller cities in our populations because or in our sewer network because I think historic redlining in the area is something that, you know, have bound have dealt with like a lot of unbounding in our cities and that ends up being where a lot of our higher minority populations reside so I think we might need to be thinking about how to oversample like certain populations so we can continue to have at subset and like aggregate and look at these vulnerable groups. I think we've also talked a little bit about this today there might be like other ways we can be thinking about getting at these vulnerable groups like are there sub sewer shed or sewer shed or even neighborhood level sampling that we can be able to do to help understand some of the impacts that we're seeing that are really different. And would that kind of approach vary by different pathogens as we talk about antibiotic resistant organisms. And, you know, Candida or us are those things that need to be done at like a with the medically vulnerable population so looking at like our long term care facilities and such. And I think we're really, you know, continuing to ask this question about, you know, even if we do see kind of our more are more Hispanic or higher African American populations and higher social vulnerability populations captured in the sewer sheds. We still know that rural populations are dealing with a lot of challenges in terms of health equity and access to care so how can we make sure that they're still that they're still covered and understood and feel like they're represented in the data that we that we're generating. And I think, you know, that can get that kind of talk really brings us back to making sure that we have that community buy in and making sure that we have the wastewater utilities buy in as well because they were huge unsung heroes of the public health emergency and continuing to engage with them and make sure that they're, you know, they're supported as part of this massive effort to build the national network is really important and we should be, you know, continuing to advocate for them in terms of not only public health infrastructure investment but also wastewater and access to water investment as well. So, I'll just, I guess stop there. I think I was too far over but hopefully we have a question. That's fine area. We're fine on time. Getting some applause from Colleen. Are there any questions for Ariel at this time. Or we can wait. Oh, there's a question. Yep. You've got your hand up. Sorry, I was just doing the pause. Okay, you're just clapping awesome. Sorry. If there's no questions at the moment will kind of continue with this kind of looking at this from a state level. I do want to throw one question out just for people to think about don't please don't answer it now. But, you know, if we look at the current data and we look at kind of downsizing this to to what may be an optimal size based on covert or whatever. There's also a cost to, you know, one of our goals is that we have a flexible and adaptable system national system. Obviously you need to scale it back up. There's a fiscal cost to that, but there's also a sociologic cost to that that is if you've, if you've told a waste of water. Your data is not really that important to us meaningful to us. You know today and then a year from now we say you know we really need you on board. I know that's a very hard question to put into some of the things we're discussing but if people would think about that and provide some thoughts a little later on after we finish these presentations I would really be appreciative because I think that's a major issue as we as we look at right sizing. At the same time, maintaining this this flexibility and adaptability so with that. Susan Peters is our next speaker is going to speak on the lessons learned in Michigan. All right. Thanks everyone for having me today. Is the slides set showing up. Yep, perfect. All right, great. Thanks everyone, glad to be a part of this amazing group and yeah we're happy to share our experience here in Michigan. I think, as I've heard some of the other presentations today we're starting to get some common themes that perhaps I'll touch on as well. So for the group I just wanted to kind of set the stage where we're at in Michigan because I do think we had a unique perspective. And definitely, we are thinking about downsizing and recognizing that some optimization will be needed in the future. We're really lucky in our state back in 2020 when everything happened that we had a ton of people jump on board and be interested in wastewater testing, you know, from local health departments to, you know, our utility partners to our academic and health partners. And so we took a decentralized approach, and we basically said okay we have a set amount of money. You know who wants to jump on board with this and and essentially you needed to create a locally based project. And that project had to have a laboratory partner. It had to have all of the local health departments that you were proposing sampling locations in to be on board. And then obviously those utility partners and others as needed. And it didn't, it didn't matter who wanted to lead the project as long as those components were there and everybody was coordinated. So we ended up with a pilot project that had 20 locally based projects. And we looked at over 200, 270 sites in 37 of our 83 counties, plus the city of Detroit. And then this eventually morphed into what we call our sewer network or our wastewater evaluation and reporting network. And that's currently ongoing through next summer. Now we're at 19 local projects. And as I will touch on in a minute we actually expanded the number of sampling sites. And now we're up to 36 of our local health departments. And we also have participation from five of our tribal nations. And so in Michigan again when we were thinking, you know, in a very quick off the cuff manner during those initial stages of the pandemic, it was really important to us to get that geographic representation. We certainly span a range of populations in our state from very urban in the city of Detroit, you know, to very rural in some of our more northern counties. And we wanted to make sure since, you know, we all kind of knew at that time that that unfortunately COVID wasn't sparing anyone that we did have some of that representation and different populations in different geographic regions across our state. And we did well with some of our populations that, you know, sometimes we have trouble reaching or our historically underrepresented like our tribal nation partners. So we were, we were really pleased that, you know, everybody kind of, you know, wanted to take advantage of this opportunity and for those of us in public health we know sometimes these don't come along often, you know, as was alluded to earlier, you know, it's, it's very rare that we're building whole surveillance systems from the ground up so it was an exciting opportunity. And I'll just dive into a little bit more what went into the decision making process and selecting the sites. And again, I will say there was an overarching strategy but then at the same time we wanted to allow flexibility. Again, with this being, you know, kind of a new concept in terms of, you know, wastewater monitoring being done consistently in the United States. We weren't necessarily sure what might give us the best data, and as well what might be most actionable. And it was very important to our department to have this network result in an action in public health action. I mean, data is great, but if nobody's using it, you know, it, you know, isn't serving its purpose. So we came up with two overarching strategies. And that was to look at the larger community level data so whether that was going to the wastewater treatment plans or in some situations are large sewer sheds. And then we also told the participants that they could look at congregate living facilities. You know, we again as we think about where we were in the pandemic we knew that we were having facility level outbreaks of COVID-19. We didn't exactly know what that would look like potentially in the future. And we wanted to make sure we were getting data on some of those vulnerable populations. So again, it was, you know, fairly loose, but as long as you had a building level facility, you know, you could propose that particular site so we have long term care facilities assisted living K through 12 schools. And as you can imagine, we have a lot of university and academic sites, just because of, you know, not only interest in that population but also participation by by those in academia in this work. And then also our prisons, and we've even had like a couple of work sites and we have one like local airport, you know, we have a smattering of interesting sites. And as I said, you know, if you can make the case to us that you are going to use data on this population at the, you know, to inform your decision making, then, you know, within reason will will support your test date. So, you know, we have some variability and in the number of sites depending on, you know, if if somebody's dropped out in a given week or something but you know we have about 430 sites in the project right now, and you can see the distribution there. And so I just wanted to highlight a few findings from our network. This is again the work of many, many very talented individuals within our network and I don't pretend to take credit for it. So we've got their, their references there. But as I go through these I think the kind of overarching concept we have found so far is that we're seeing a lot of sites specific variability at least as it relates to SARS-CoV-2 data. And again, when we think about the usefulness of data, the utility of data, everybody wants to know, you know, how, how much of a lead time do we get? And as you can see here, it's been a little all over the place. Like when we looked at our data from our pilot project back in 2020 and like averaged every single kind of building or sampling location that met our QAQC criteria, we got about 17 days for our lead time. But then one of our collaborators at Michigan State University, Dr. Irene Zachararchy, she does a lot of work in our Metro Detroit areas where they have some very large interceptors and some of those systems will pull from, you know, a population that's over a million people. It doesn't just cover the city of Detroit, they go up into the metro area. And they were actually getting up to a four to five week lag time as they looked back at the data, which, you know, is really powerful. But then another one of our papers, you know, they looked more at their rural sites. So our wastewater treatment plants and large sewer sheds up in some of our upper peninsula sites. And again, site specific was like zero to seven days. And then even we had one example, kind of in the middle of our state where town where they were sampling both at a campus location, and then offsite at other places within the city. They were getting a mismatch there in terms of their lead time. So, I guess, you know, to summarize all that saying is, is that I sometimes kind of like to play devil's advocate in that. I don't know, at least from what we've learned so far if we could potentially, you know, scale down and feel like we're, we wouldn't be losing data just because we've seen differences between, you know, sites, and we, I also didn't put it in here because we also have seen some indication that even within a site, we've seen differences in in our lead time between the wastewater signal and cases within the different waves of the pandemic so depending on the different so again just I think a lot to consider as as we think about, you know, having to downscale and and what we might be losing. I also like to throw this other example in here this is from Dr. Aaron best at Hope College, which is kind of in like West Michigan. And this is from fall 2020, as you can see, and this is one of our examples that we like to show to kind of advocate for testing at those building levels. And so, at that point they were testing for N one and e genes, which is the blue and yellow bars. And ultimately what you see here is that they started getting increasing signals, which informed their public health actions were, which were to first increase the frequency of their clinical testing in their dorms in the affected area. And then the orange areas arrows are when they would remove some of those positive cases or their close contacts that had been identified by that testing. They had gap in the detection of this or the detection in the wastewater signal which you would expect, and they had some more cases popped up. They did the same thing they took, took people remove them from the residential setting. And then you got the corresponding decrease in the wastewater signal and and as they went through their, their quarantine or isolation period and actually returned back to the buildings, you could see that signal pop back up again. And so, again, as we talk about utilization of the data, like I feel it's been our experience that as you get down to a more granular level. In a lot of situations it's been easier to implement public health actions such as, you know, clinical testing and potential isolation and quarantine. I also pulled this bit from a paper that one of our local projects published. And I didn't, you know, I wanted to make sure that they were getting credit for it so it's, it's quoted there. Okay, this is one of our local health departments talking about how they started out doing more community level testing, and they found it wasn't useful for them for a variety of reasons, one of which has been discussed in some detail here is, is they had a high proportion of their population on septic systems and so they felt like they were not accurately capturing that population. And then again as you know they got further along in the pandemic, they felt like they were having less ability to implement some of these community level mitigation strategies. Instead they found the more receptive population to some of these potential interventions was at a facility level. So they decided to kind of switch strategies and focus more on their nursing homes and their long term cares, and really work with those clinical people who were going to utilize that data and, and work with the health department to have them come in and do some of these follow up interventions once they, you know, reached a certain threshold. I just put that out there is, is this is some of our experience and, you know, why we have appreciated having data on both community level but then, you know, building our facility level sites. And for those that are interested, we do have some of these data up on our website, we kind of call them success stories from our pilot project. And so again as we focused on uptake and utilization of the data. Here were some of the, the outcomes that came from that. And then I'll touch a bit on several sites, which others have as well and I think hopefully we can add to that part of the conversation. You know, again, with so many sampling locations, it did become and still is a bit overwhelming to look at all of that data. And so from an analysis perspective, we did think it was useful to pick some, you know, quote, Sentinel sites across the state to try and do more of that detailed analysis on. And so this is available up on our website, but we took 20 wastewater treatment plants from around the state. And the selection criteria for that was, you know, we wanted about two per region across the state as well as we wanted sites that were testing at least two times per week. And from there we tried to get the most population coverage that we could. And so I think that's worked well from a communication standpoint, it's certainly much easier for some of our statewide folks that look at this data to look at this. I think what we haven't had a chance to do and it sounds like others are interested in as well is is looking at are these sites that we chose truly representative of the other wastewater testing sites within that same region. Could we essentially downscale and use those as a true Sentinel site? And, you know, we're not there yet, but we would like to get there in the future. And then something else that again has been touched on today that we wanted to highlight from our experience is definitely that flexibility of the system, you know, alignment with that CDC priority of being able to respond in some type of an emergency or emerging capacity. And definitely, I think it was an eye opening experience for us in that we were one of the jurisdictions that decided to start doing some polio virus testing. And, and we thought it was going to be easy like slam dunk, you know, we could just go to our existing sites we could, you know, start testing for polio virus and start churning the data out and getting the answers we needed. And unfortunately, that very much wasn't the case where we needed a different population size in a particular county. Then, and we didn't have those sites we we had ones that sampled larger populations and we had ones that sampled smaller but it was like, you know, Goldilocks and the porridge we needed to get one that was just right according to CDC's recommendations and and so we had to select those sites and go through environmental engineering and the whole nine yards. So, again, I just put that out there that I think, you know, related to the earlier discussion on what are the goals and the outcomes of wastewater testing. If it's it's to hone in and be very intentional on optimizing testing for certain pathogens. Then I think, you know, we'll have the data to get there. If we need to remain flexible, then we may need to either remain larger or or just think about we may have a slower response capacity. And then also as well put down their range of interest and participation and again that's kind of echoing on some of those those comments from earlier. Yeah, we, we struggle with, you know, continuing to maintain interest and participation in the program. And I think, yeah, if we're to try and turn it on and turn it off. I think that's going to be real difficult for for a variety of stakeholders. So, so that's it. I just want to put up a picture of those of us who participate in the network that we're at a conference this year. And thanks for your time. Thank you, Susan. Are there any immediate questions for Susan. I see applause. I'm thankful to tell you just applause and a hand raised. All right. You're getting a lot of applause. So that's good. And there's a hand up record. Very nice presentation Susan, I was just wondering about your local projects. So you mentioned you had 20 local projects and can you share some more detail on that that was for the pilot projects right and how you were using those as as compared to your sentinel sites. Like that data, how that was useful. Yeah. So, yeah, we had 20 local projects in our pilot and then we kind of combined two of them together. So we now have 19 local projects. And so those sentinel sites that I was referring to are a subset of those. They're, they're not different. They're, they're taken from within those projects. And, and yeah, the idea was just that, you know, we, we had a feeling we weren't going to be able to, to manage all of those sites at the state level. So at the same time we were trying to ramp this up our state public health lab was overwhelmed with clinical site, you know, clinical samples and wasn't able to help contribute and so we were really thinking about what model would work best for us and we already had like a really significant model with a lot of these university and academic and other laboratory partners for our beach monitoring efforts across the state. And so we just kind of piggybacked on that we said okay here's here's a group of people, more or less, you know, who already kind of work together, we'll just, you know, and they wanted to help they're like what can we do. So, so we kind of took that framework and put it on to wastewater. Great. Thank you. And I was also wondering about your airport data. How you were using that data in any type of decision making. Yeah, you know, I can't say that we are at the state level it's it's definitely like a small local airport, but the local health department there was interested in. I said, essentially, I think kind of looking at what type of virus, you know what might be brought into the community, especially it's a it's a seasonal tourist area. And so I think they were curious to see if they would also get a seasonal change, you know, as as they got an influx of summer tourists. So, I think it's been more for awareness and on their part as opposed to like, you know, having an actual intervention or action with the airport itself. But yeah, we said, well, you know, if that's important to you at the local level it seems reasonable so Exactly. So we also got one request to sample at one of the airport, but it was very hard for us to understand how we will be sampling. So can you share some information about the sampling you conducted there? Like was there a separate wastewater treatment plant for the airport? They do not have a separate wastewater treatment plant. I believe they're sampling at a manhole there, but I would have to check in and get back to you. Thank you. Hi, I guess I was curious if you could talk a little bit about the like one of the things I think is kind of cool about Michigan is this like this, like elevating of the importance of local decision making and that that's a like a valid, you know, reason to select sites that are not necessarily as critical from a statewide view. And so you have this kind of smaller network that's more relevant for the statewide with sweep and a larger network with sewer that's maybe more relevant for local decision making. I don't know if you can share any more about sort of like how you balance the tensions between those when those two things come up but like, I don't know if that's, yeah, that's not a really well farmed question but hopefully you kind of know what I mean. Yeah. Yeah, I guess I would say maybe we've been lucky and in that we, I don't feel like have, have had too many tensions. So far, again, you know, those that want the local data have it. And then, you know, again, I think for for higher level for statewide consumption, it's, it's too much so everything we've heard like, you know, from our administration and whatnot is is a like you know that snapshot those those 20 or so sites to look at I think that's about what they, they can handle. But I will say I think that time could be coming as we potentially you know have to have to think about some of these decisions that that have to be made. So, so I guess maybe I'll defer say we've been able to avoid some of that tension so far, but it's probably coming. Rebecca. Thank you guys I forgot to ask one more question I think I put that in chat. I was curious about your lead time variability. And I guess most of us have witnessed that and on and off we keep wondering about like is it changing is it the same is it still useful as a leading indicator. So, could you share some more information on like the reason behind that or it's just a different type of population and different size of population which might be contributing to that. I don't have an easy answer for you but I think it's a little bit of everything. I think some of it, you know is the different systems, the different wastewater systems themselves. I think, especially when we look at those big interceptors that serve those really large populations in Metro Detroit. You know, my understanding, I'm not an engineer is that you know it moves very differently through that system compared to like some of our, our smaller treatment plants or our more rural treatment plants. So I think that plays a part. I think there could be some some population differences. Certainly, I think we could dive into that more we haven't too much yet. And then, yeah, I think in terms of the different waves of coven 19. Again, I'm kind of paraphrasing but the general sense from across our different projects in the network green home across was that we lost some of that early warning signal everyone started saying oh, as we're looking at our cases now, you know, for instance, if we had a week and in our early morning now we're starting to see it go down to like two days, you know where we're not seeing that early morning. And then we tried to relate it to what the virus was doing clinically, and it seemed like you know it was spreading faster through communities as well so that made sense. I don't know that there's an easy answer. I think it's a little bit of everything and potentially cider region specific. And it is definitely very complicated. I do understand. Thank you. I appreciate your answer. Ariel, do you have a question you wanted to join in on that answer. Either way works. I was going to join in on the previous answer and just support Susan and kind of the question around state versus local kind of, I don't know, push pull, I guess. And we actually had two of our two of our local health departments, specifically region for which covers 11 counties around the Charlotte area and then Wake County which is our biggest county that covers Raleigh. And they put in their own coven relief funds to support wastewater. And we thought that was really interesting so we kind of had like an over sampling and in some of those areas and had to obviously consider how that worked when we were aggregating up to the state metric or our state level because it's one of three metrics that we still monitor for COVID. But it was really interesting to see kind of that local buy in and they were, as we built up our capacity of our state lab of public health they've been really invested in trying to make sure that we're continuing sampling at that those sites so I just supporting Susan and that kind of there wasn't really like a push pull but I do think like the state wants our state leadership wants to see things a little bit more boiled down and then our local folks would really like to see them. Their communities represented in these data as much and I think we can accomplish both you know if we, if we handle it strategically. I think it's a great points area. I think our communities that have really bought in and are invested in it. Yeah, are very much so and, you know, we may lose some along along the way but but those, I mean, I can't tell you how many people say, when are you getting more funding or what's the next thing we're testing for and and yeah they, they want it to stay. The power of that community is so powerful for our sustainability, you know, and, and I think people really like that they can look at the data on our dashboard and say hey that's where I live or that's where I work and there is a really huge power in that and as much as we can, you know, keep that moving momentum moving forward while also thinking about these questions is our our our task. Yeah, thank you and let's I'd like to introduce Kathy Ensor who's going to provide some of their experiences from Houston. Share my screen. And get it in. Okay, does that work. Super yep, perfect. So, thank you for the opportunity to speak to this really amazing group and all the great work that you're doing. So I've been involved with the project in Houston since the beginning, and as the statistician and the head of the analytic litics piece. So I'm going to focus this. I thought I just when I was thinking about this presentation, or this discussion, you know what what are the kinds of questions that we would ask so first of all in the early days you know does a virus just actually exist within a city. And if so where does the virus exist. So once we begin to understand if something's actually there, then we would like to quantify the level in the trends within a city. And most importantly as a statistician I always want to look at the uncertainty of our estimates and I want to make sure that we keep that in mind. Are there are there communities within a city that are experiencing an outbreak. Are there trends within communities of concern for public health departments. Are there facilities that require close monitoring and going the other way how do we scale all of this information up to a national level. So constraints and sources of variability on all of this will first of all it's incredibly expensive first of all there's a physical cost of collecting samples. The lab cost of analyzing the samples and as your data gets more complex the statistical analysis will become more complex but they also have costs to translate those lab measurements to information. And then health department costs to translate the information to action. So, so this information comes with sources of variability and I just want to remind all of us of what those sources are. First of all the variation in the population samples so we're all relying on census information and generally assuming that it's known and constant. So that is a huge assumption and as we move forward with census and census products then it's also it's very important to take into account the margins of error associated with those. Also, you know the demographics and the behavior and the travel within these census groups that were whatever we're focusing on can change and are we taking that into account. Either but I think we should always have that in the back of our mind as an important consideration. Then then there's the variation in the virus itself. And so, so that's what we're trying to understand that's our target and how well are we understanding that target. And then when we go further there's the sampling variation and then the measurement and lab variation and I'm just reminding us not to ignore these significant sources of variation and I'll speak to that just a little bit. And then when we're sampling for rare events as new viruses come on is that more than a hit or miss. We need to be taking into account the probability of false positives and false negatives as well. So some of the questions that I'll try to address like where. And so we'll look at, as many of you know the city of Houston system to understand the city trends is developed from the extensive wastewater treatment plants that we have. And then we looked at sub sewer sheds and then Dr. Stadler spoke on Friday from speaking to some facility level sampling so I won't really address that as much. When do we sample as needed do we sample monthly do we sample weekly do we sample daily, and I'll try to give a little guidance there. How one lab or two, it's wonderful to have two measurements from two labs but is that really practical. And then, are we going to obtain replicates and then the measurement process in general. Okay, so first of all let's look at the how the enter and enter lab variation. And I think the takeaway from this slide is, I really want to encourage everyone to think about replicates. Keeping replicates in their sampling plan. If we look at this, just the coefficient of variation for the intro lab variation based on the replicates so the reminder that the coefficient of variation is just a standard deviation divided by the mean. And so, on average it's point one for and this is all based on this very extensive, you know, 20,000 measurements that we've taken in Houston over these years. So it's a very rich set of information that we have. So, so indicating that the variation in our in a specific from a specific sample can be as much as 14% of the mean value on average. And if I move up to the upper quartile then it can be above 18%. So that's that's a high level of just variation in the measurements themselves. And, and these are exceptional labs that we're working with they're not. I'm just convinced that the labs that produces information are a plus labs and so, so this is inherent in the process that we have. So let's go to the inter lab variation so this is the variation between the two labs. So you have a scatter plot down here. Lab one and lab two and you know in general you know that correlation is really close to one but it's not perfect and so, so, and this is all taken on exactly the same sample. And especially if we have no replicates and only one lab that it has a lot of inherent variability in it. And then one piece of variation that I think we, we are forgetting in this conversation is that a sample is just that a sample. So the variation from sample to sample is not directly observed. And so it helps helps us forget about it because we're not actually exactly measuring it. But we can bring in some higher level time series models that can separate measurements and sampling variation. And if there are replicates the sampling variation can be estimated separately from the variation in the measurements. So here's just an example of this and I'm going to set this example in our comparison of. We have one really large wastewater treatment plant that serves 5.5 550,000 people. And so we looked at four lift stations, subsurge sheds within that wastewater treatment plant and so this is the comparison that we have. A station is about two thirds of the population of this very large wastewater treatment plant. And so, so what you're seeing here in terms of the time very colorful time series that I have is a comparison of the. If I had perfect information so it's a retro perspective study on what went on within each of the lift stations, and I can't compare that to what with what went on within the 69th Street wastewater treatment plant. And, and so you can see that the signals are very similar but they do differ in some places. But what I want to point out on this particular slide if you go up to the table, you actually see that the can't actually move the people here. So I've been able to tease out the sampling and lab standard deviation from the standard deviation of the trend. And the standard deviation of the trend in all of these cases is basically almost the same, but the sampling and lab standard deviation for, say, apt and village into priest is almost it's an order of magnitude larger than what we would get from these two large areas of the 69th Street wastewater treatment plant, and the Clinton Drive region and so. So this is important to consider and it's not uncertainty that goes away. And so, we really do want to try to tease that out in our modeling efforts. Done. There. Okay, this is that same set of data and I'm just giving you an example. So what these plots to describe is the blue lines with their confidence intervals are the retrospective estimates if I observe all the data whereas the light blue lines are correspond to a online estimate so it only uses data up to the current time point and not any future data. And again so this is the uncertainty that we would see in each of these locations and the where the confidence intervals really expanded when we have missing data. Okay, so how could we take this information and and try to ask ourselves, well, should we really be spending our time sampling these sub sewer sheds do they give us additional information. And so we've rolled this into a control chart type approach. And so to see if there's really that hugely improved value in the information so let's just look at Clinton Drive which again is two thirds of the population represented by the treatment plant. And we see that the peaks are basically similar, but the levels in Clinton Drive do not fall off as much and so that's where we're seeing the difference between the two series. I'm not it's not clear to me that that's important information from a public health perspective. The physical process of sampling is very difficult for these lift stations as opposed to its relative it's easy to obtain samples for the wastewater treatment plant. The need time for surges seems not not existent or not substantial. And so the recommendation for the sub sewer sheds would be to use only in times of high concern and not as a part of routine monitoring, mostly due to the fact that they're very difficult to actually physically obtain a sample. And if I move to that, that question of what is the trend for the city or region I'm just pulled out some plots from one of our recent papers. And a reminder that the city wide trend is a population weighted some of the wastewater treatment plant trends and you can talk about how the population is incorporated. It's important to incorporate who's who's represented in each in each group we actually do copies per day rather than copies per leader to obtain our weighted some, but a national assessment should consider the population as well it's a really you know it doesn't make sense to average something estimates that cover 500 people versus 5 million people so so you really should be taking into account what the measurement the population the measurements represent when when you aggregate them. It's important to sample the wastewater treatment plant serving a large component of the population. And so if you can see these are these lines of the squiggly the the trends for each wastewater treatment plant are actually included, and then the heat map gives you a different perspective of how to look at that. And the total of course is the top curve with with the confidence bounds, but you can see if I'm just going to be summing up these trend estimates for each wastewater treatment plant, then the larger ones are going to contribute more strongly to the trend within the city. And as we think about downscaling what we sample then we want to sample specific wastewater treatment plants based on importance of public health intervention needs equity and budget. The remaining wastewater treatment plants are sampled in batches grouped by similarity. So, so we know a lot about Houston and I just want to point to this paper from 2021 with where we conducted a serial prevalence study to really understand the demographics in the of our community and how hard they were a hit in the early ages of COVID and so this was an incredibly well designed study and strongly scientifically valid results but it gives us a good perspective on sort of which parts of the city were hit hardest in which parts less so. And then again, I encourage you to take a look at that paper if it's of interest to you. So, going from that early understanding of our city, then let's think about how we're going to select our batches and understand understanding our region. And again, I'm focusing on Houston because this is where I have the data and where I have the access, but these, these comments generalize more broadly. I think of a case driven groupings for SARS-CoV-2. So, looking at the zip code level which is the level of focus for our public health department. So, basically group our zip codes that have similar characteristics with respect to the evolution of the virus and so it'd be the time course of cases, hospitalizations, deaths, demographics, community vulnerability index. And so this is an illustration of how the city breaks down if I, if I do a grouping of six clusters. However, maybe I'd also, you know, I'm really just trying to understand what is the best economical use of obtaining wastewater epidemiology information. And so I could also define my batches by the evolution of the wastewater treatment plant time series for COVID. But the obvious caution not all viruses will have the same evolution as SARS-CoV-2. And so my favorite plan is a health department defined groupings so based on zip code demographics, socio-economic vulnerabilities, population to the geographic continuity and measures of equity. And the batch sampling of the wastewater treatment plant sample respects the groupings. And it's important to have robust and stable groupings with the health department interpretation regardless of the strategy that you use to come up with these groupings. So if we want to look to the presence of a new virus, it just makes common sense to initially target those areas where the virus is most likely to be present within a city. I know that this was part of our discussion with IMPOX as the city looked into the presence of IMPOX. So it's easy to sample the largest wastewater treatment plants and so we should definitely do that because that covers a lot of people. But then also look to health department defined areas. And in this case, facilities may be most helpful in the early stages because you're really getting at sort of that local piece. So you're going on both ends with sampling large groups with the wastewater treatment plants and then targeting facilities that you think might be of highest concern. And then consider lift stations if in regions of high concern. And so those regions of high concern can almost be defined by the type of virus that you're looking for or pathogen that you're looking for. And then once you've established that the presence or absence, then you would want to move to routine monitoring and training quantification. And this would be based on the regular sampling that we plan that we scaled back regular sampling plan that you put in place. And then I would argue that a national perspective is driven from the bottom up, so larger international cities have the highest probability of serving a sentinel sites. I want to speak just a moment to temporal sampling considerations. So for the wastewater treatment plants, there's no reason to sample more frequently than weekly. So it really is sufficient to sample weekly. If you sample more frequently than those could serve as your replicates because you're getting information that's very similar to the previous information. So I've included two plots, the autocorrelation and the partial autocorrelation that shows how the measurements change week to week. And so you have a very strong correlation from one week to the next. I mean, it's 0.9 and this is observed in all of our wastewater treatment plants. And so that really speaks to the fact that sampling weekly is sufficient to understand the evolution of the virus in these large wastewater treatment plants. I would argue that a lower temporal resolution potentially daily could be beneficial at the facility level where the population dynamics aren't quite so strong. So, let's see if I've answered the questions the where. So for large cities, I would say wastewater treatment, your larger wastewater treatment plants are important. And then also key locations that are important. And then everything else batch that so think about batch sampling it's maybe it's easy to collect the wastewater but it's expensive to actually analyze it. I would recommend that only in times of high concern and is it that subsphere shed had a specific message with the pathogen that you're considering. And the facilities as budget and impact allows, we really do see strong public health intervention impacts from sampling facilities. When for your larger areas weekly is sufficient and for your smaller facility level areas maybe daily or at least by weekly how to labs is a luxury but it's extremely nice. But absent it's up to labs, let's at least, let's least have a replicates of your sample I think that that's really important, and then well documented processes so we can combine across labs nationally. Since we don't have a lot of standardization yet. So we'd like to, if we want to build up a national perspective, I would say that it's best it built from the bottom up. There are all of these uncertainties that are present and, and understanding those and understanding the data is incredibly important and that's only going to happen if you build it from the bottom up. The larger international cities, certainly Houston is one, had the highest probability of serving a Sentinel sites just because we have more of the world moving through our communities and, and more the world represented in our communities. And with that, I'll stop and answer any questions. Thank you Kathy. That was that was really informative. And we have a look and see if we've got some questions coming in. Scott. Hi Kathy, great great presentation. So it's curious, you were saying there's 14% coefficient of variation. And to me that doesn't sound like very big. That's pretty, pretty minor. Good. Yeah. So, well, I mean, in typical biological assays like on plates, 20% variability between technical replicates is considered pretty acceptable. For PCR, she's about one CT value for a QPCR. So 14% not not so bad for me, especially are these like what Ali is defining as biological replicates or these technical replicates on the PC. These are these are technical replicates. So just the QPC are not the sampling the extraction all that. Well, maybe I know I. This sampling and extraction, you know, so this is the, so you have one sample and and then single sample repeat me replicates we have, we have repeated measurements on that one sample. Okay, from the very beginning of the method. That would be my understanding, but that is a lab questions. 100% sure but yes, you mean in terms of whether the, the dilution and that both pieces I don't know I'd have to whether it involves the nucleic acid extraction the concentration. I'm not 100% sure Scott. Okay, thank you. Right, any questions from any of our for any of our panelists. Ali. I just thought it would be good to say or early. I think we have to be really careful about recommending a frequency of sampling without very clearly identifying the use case. And the metrics for coming to that conclusion. So, you said that you, you felt that weekly sampling was sufficient, but I just, I just think we need to be careful. I'm not saying it's not, and that that's not the right answer I just think we need to be very clear about how different use cases or uses of the data could require a different frequency of sampling and be really specific about that. And that there are quantitative approaches to evaluating what frequency is needed which as a statistician I know that you know that but I just, I just wanted to say. Let me. So I think I clarified that it's for the larger population areas and so it and it's, it is a quantification and it is a quantified answer in terms of just the system isn't changing isn't evolving that much where the goal is to like maintain the correlation between case data and wastewater. That's one use case another is to identify trends in the wastewater or the levels like there's different like actual uses of the data or frequencies of measure of positive detections or detections above a certain level like. I think that your description right now is very qualitative and I know that there's probably a lot more quantitative work behind that but overall I just think we need to be careful about. I would agree with the care but I would stand by my statement that the quantitative answer for large treatment plans even if you're looking at trying to compare to noisy case information that. 550,000 people that's a large set of people. And so the evolution of that large set of people is not going to happen. I have a publication showing that if you want to have a trend result be similar to what the trend would be with daily data that you need to sample more than once per week with very large sewer shed so again like I'm not saying that you're incorrect. I just think that depending on how you're using the data that the required frequency is going to vary and that's something that this panel is tasked with like what are the use cases and for different use cases what are the things to consider so I just wanted to point that out. We also have a publication with on daily data comparing to weekly and it shows very strong correlation between the cases and the treatment plant information and and Ali I think my my sense is coming at it from where is the value of the information and so we're we need to scale back the system and so the idea of. Sampling more frequently at these larger wastewater treatment plants may not be where the best use of the dollars would be I maybe that the best use of the dollars might be at the facility level where I do think that more frequent sampling pays off in a big way. But. You know because because we do have to look at sort of a constrained system now where we can't sort of do everything that we want to do. And I appreciate that I think constraining. There's so many degrees of freedom on this right and so I think. So ultimately it comes down to defining what the use cases as the number one thing and then starting to constrain on the different degrees of freedom related to that so I think it's sort of a nebulous quite like we were given a task to present without maybe clear actually what the use cases but yeah I'm going to stop talking thank you for engaging in this short debate with me and Sandra I see wants to chime in. Well actually I want to follow up Kathy just to pick your brain a little bit because it does come up to you know trade offs and that is going to be a big part of the discussion is. You know the timeliness of the data and the different phases of the pandemic when it status quo or we think flu virus season is coming but Houston is really unique and that you do have. I forget if it's 37 or 39, you know wastewater to implants servicing the city. If you had your choice, would you go to twice a week within a single or a few surveillance plans or would you expand the number of plants that you would start sampling if you thought, you know, we need to ramp up surveillance because we wanted to expand or we want to we think some things on the horizon. All right, so, so there are, I think I skipped one of my slides but there are a couple of locations that have. You know, it's not the airport but the airport sits in that location. And then there's a couple of other treatment wastewater treatment plants where, you know, like I 10 goes through it and we know that there's a ton of traffic that starts stops at Bucky's and so we know that there's a high, high volume of new people into the city and. So, those particular locations it might be helpful to sample more than once a week but I we're sampling all up currently sampling all of our wastewater treatment plants I don't feel like we need to generally sample them more regularly I think what would be more useful would be to add more facilities like schools or nursing homes and, and we're not, you know, from a public health perspective, where you can actually do a large intervention, then getting local is helpful. I will agree with Ali on the use case and so I sort of identified those use cases at the beginning of my presentation and so but if you're just after trends then this weekly information is, is pretty good. So, are you know are you going to get that small blip that much earlier. And it's not clear to me that that given all the other uncertainty that you have in the system that you're always going to routinely see that. So, so I would. So, what I've recommended for Houston is that we continue to sample sort of 15 locations are large ones and then strategic locations regularly, and then the other locations will be grouped or batched. We're sampling them but we're not analyzing them individually. And so that, you know, that's a common environmental sampling approach when you can't afford necessarily to sample everything. And so, so I think that this batching will continue to capture anything that's there in the minute that we become concerned for those regions that we've identified that we're grouping together then we can roll them back and sample them individually. Does that answer your question Sandra. It does. Yeah, because I think my question really centered around increased frequency versus increased sites and I hear you saying you're keeping all of the sites and going kind of to once a week or 15 of the sites and then just to follow up defining batching is that compositing wastewater treatment samples and then extracting. Okay. Okay. Great. Thanks. Thanks for that. But I will add that the temporal frequency at the facility level. Weekly is probably too far apart for that. All right, super I wanted for people I know people top of the hour usually have other commitments and will begin to drop off so it gives me an opportunity to thank all of you. Both the presenters and those who've asked questions and given us kind of a robust discussion. I think you've hit really some of the key things that the committee's been working on and has questions about and challenges to optimize the system so that I'm sure there'll be some follow up around these areas of kind of spatial and temporal frequency and and incorporating equity into this as well. These are all really critical elements that the committee is going to address. So with that, I'd like to thank you all and hope you have a happy and productive day. Take care.