 Good morning, and welcome to this public meeting of the United States Consumer Product Safety Commission We have one item on our agenda this morning The CPSC staff will brief the Commission on the recommendations to allocate Future unexecuted balances should balances be available to support extra project work The CPSC staff members briefing us today at the table are Dwayne Ray the deputy executive director for safety operations And Jay Hoffman the director of the office of financial management. Thank you both for being here and rolling us to Provide us with the information and answer our questions at the conclusion of the staff briefing We will turn to questions from the commissioners and those rounds will last 10 minutes each We will now begin with the staff briefing. Please go ahead Good morning, and thank you chairman Burkle We could go to the slides, please Okay, this morning that this briefing will be kind of in two parts. I'm going to spend just a couple minutes providing you with an overview of the Mid-year request and I'm going to spend some additional time on mid-year projects one and two because I think those Require some additional context in terms of what the staff is recommending With respect to the overall mid-year staff is recommending that the Commission authorized projects to fund from unexecuted balances if available Currently we are forecasting an unexecuted balance up around three and a half million dollars The sources of that unexecuted balance generally consist of pay and non-pay items These are fluctuations in in staffing levels essentially and sometimes it's operating funds that For one reason or another are unexecuted We've identified approximately five million dollars in projects that have been presented here in priority order It would be our intention that if approved we would fund these projects in order subject to available funds and acquisition acquisition feasibility In the mid-year memo that was provided to you last week there was This table along with an attachment that described in detail each of the projects I'm not going to go through them in detail here But I would just note that the first two projects are for enterprise data and systems The third project in cares is for exposure data Project for is to modernize safer products dot gov adding some mobile capabilities and improving content Project five on lithium-ion batteries project six our business process review that will inform needed upgrades to the IFS system Seven is enabling testing of Internet of Things enabled projects at the electrical lab Project eight is follow-on work for our smoke alarms work that was started in a prior year Nine gas appliances ten the child strength measurement. This is augmenting existing work an existing contract that we have going on Project 11 ATV stability. This would complete the multi-year ATV plan that we put together a few years ago 12 is safe sleep warning labels and finally 13 is an implementation plan and roadmap for our e-commerce work So with that I want to transition now and spend some the remainder of my time on Mid-year projects one and two These projects are components of a larger future vision That staff is proposing for the CPSC for an enterprise data management and analysis capability I want to take some time to provide you some context on how these two Component projects fit into that larger enterprise vision So first off This has obviously been an emerging priority and topic of discussion in these sessions for the last several months And I think this has been discussed in a whole variety of ways I've heard it described in terms of enabling an enterprise data management and availability capability for the agency I've heard people talk about how to leverage data across the enterprise so that we can have better insights There's been a lot of really robust conversation around using new tools advanced analytical techniques like Pattern recognition and artificial intelligence And I've also heard conversation around establishing new positions and and new roles and new skill sets I think all of these are great But what I've tried to do is pull together a team of people and say okay given all of these ideas What can we recommend? Can we recommend a path forward? So Staff has proposed two mid-year data projects to advance the agency analytic capabilities Let me describe these in some detail here, and then I'll provide you the larger context So first is mid-year number one an IT data strategy and implementation plan There are two components to this mid-year project. It's important to understand Component number one is what is the bigger picture strategy? How do we make data available at an enterprise level level? How do we make it scalable? How do we then leverage that enterprise data to improve our analysis to improve our analytical outcomes and What are the tools that we can identify in the specific use cases the analytical questions that we're trying to answer? And I think there's actually some clarity on this people are there's an appetite for a big picture strategy And that's what this is looking for the second component though is what I've heard less Conversation on and that is once we have this larger picture strategy What is the technical implementation plan described in business terms in order to implement it? What are the resources required? What are the tiered levels of funding? What sort of timeframe are we going to be looking at? This is likely a multi-year Large dollar initiative, so that's mid-year one Mid-year two is a is recommending a pilot study on machine learning and artificial intelligence In other words, we want to try to pilot something to inform the overall strategy and have some learning by doing What this project would entail is using open-source code and open-source tools Probably artificial intelligence and pattern recognition And to rapidly deploy these tools in the next six to nine months So that we can learn from the results to inform the overall strategy in mid-year number one We have proposed three use cases and I'll talk about those in a subsequent slide around automated coding data matching and data translation I think by Tackling this to do a proof of concept to see if these technologies and these tools and these techniques can work for us That will inform a larger dollar initiative and let me emphasize that the goal is to award this and to try to have Results by the end of the calendar year or if not the end of the calendar year early next calendar year All right, so now let me provide you some background in the overall context on how we got to these two ideas several months ago I Formed a sort of an informal team of senior leaders here at the agency to try to Continue the conversation that I think has happened in this room And we began that by trying to define the problems that we were hearing and the problems that we were perceiving So I think it's worthwhile to just sort of take a look at those so we know what we're trying to fix Here's this is probably not an exhaustive list, but hopefully it sounds familiar One our we have disconnected data our data is very useful within specific CPSC Organizations for example import surveillance and the work we're doing on the RAM But it's less useful across organizations the the data and analytics and CPS RMS are not part of the RAM data set We all know that to our systems are disconnected major program areas maintain their own data sets and business intelligence tools But those tools are not shared across organizations so CPS 360 is different than the RAM system Our processes are labor-intensive with a lot of labor-intensive processes Specifically anomaly detection is highly dependent on people. It's highly dependent on the integrated product teams For let me emphasize this one because it's going to come up again the cot solutions available to us have proven to be limited There's no out-of-the-box software solution that fits CPSC's unique analytical problems It likely requires high levels of customization and I would go a step further the market research is actually suggesting that Other agencies don't really have our problems. Certainly. There are big data problems at other agencies, but not really like ours Five limited agency skills and in areas like data science sass development artificial intelligence Data management not saying we don't have great people. It's just we don't have very many people Insufficient funding this is this is a the kind of investment that will be a major investment That's needed to fundamentally evolve our analysis systems and approach We're going to need to think in new ways in terms of how to fund this at the scale It's necessary and then lastly where to start do we start with the strategy? Do we start with quick wins? Do we start with modernizing our existing systems? Do we do some process mapping? I think that this lack of clarity maybe has created some false start So I simply identify that as a challenge Now I do think there are some opportunities in pursuing a modern data analysis strategy and new analytical tools at CPSC Hopefully some of these are evident to you, but number one I think there the opportunity is to develop a big picture plan Establishing an enterprise data strategy and forming the technical plan to implement that strategy over time That is what mid-year project number one proposes to do is to check the box on item one on this list Number two, there's an opportunity for enterprise analysis To create the environment for cross application of predictive analytics to identify new insights and correlations With in and across data sets. So making the tool set available setting on top of an enterprise Data paradigm CPS 360 DCM IFS RAM all pulling from a common source Trying to solve this sticky challenge of anomaly detection Enhancing the ability to identify occurrences involving low incidents in large volumes of data Applying new analytical techniques and tools like artificial intelligence pattern recognition to replace or supplement integrated product teams and so on The next one is is dealing with more data I think the term of art is boosting data ingestion. So if you want to feel cool with your kids This is the this is the term of art Essentially making it scalable. How do we expand our capability to access and process new data streams without? overwhelming Overwhelming our systems and overwhelming our people It's it needs to be timely. It needs to be relevant. We need to enable an enterprise data availability making data available across Our programs having the methods tools and standards to do that I've talked about deploying new analytic tools These are custom business intelligence use cases developed using open-source programs that can be customized for our particular needs and problems This presents an opportunity to standardize our data practices if data is going to be available at enterprise at an enterprise level in Enterprise systems, it's going to have to adhere to some standards And finally, I think there is the opportunity for some quick wins our market research has Validated the idea that we can test some of these tools now on a limited scope and Limited scale and have some results in the next six to nine months and item number ten is what mid-year number two? Proposes to do there are a number of attributes for a successful project I have listed five here that I think are critically important if we decide to pursue this proposal They are here. The first principle of success is that this is likely going to be an open-source and customized tool That is specifically configured and implemented to meet our unique needs things like anomaly detection Product categorization and data matching to name a few number two We're going to need a high-quality consultant with a demonstrated track record in this space for developing open-source customized solutions I will tell you the market research. We've done thus far. This is a fairly narrow Market space right now Number three, there's going to be a lot of learning in this We're going to have to accept some learning by doing that will be an important element of the approach Not everything that we learn will ultimately work or fit into the final enterprise solution But I think the initial use cases will be incredibly important to informing the overall strategy and again That's what mid-year number two proposes to do Funding we're going to require a multi-year funding strategy with support throughout the Commission Oh and be the White House Congress for the resources that will be required to implement an enterprise solution And lastly, we're going to need a scalable procurement vehicle once we identify a quality contractor that can support a phased implementation All right, so now let me provide you kind of the Bring it all together here in the three-step process On how we would propose that we move forward So our objective is to formulate implement and execute an enterprise data management and analysis strategy to drive improved mission outcomes step one We would first develop an enterprise data strategy that identifies in business and technical terms What the CPSC would like to be able to do with its data? And then we would develop the technical multi-year implementation plan to achieve that strategy while supporting existing capabilities We estimate that that'll cost around seven hundred and fifty thousand dollars the time frame would be to begin immediately and have this by The end of the calendar year and start formulating resource plans against it early next year Step one is what mid-year number one proposes to accomplish Step two we want to solve one to three tangible mission focused use cases that can be developed and deployed quickly We want to learn from those results results refine the strategy demonstrate quick wins and build momentum to step three We believe that this can be accomplished for around seven hundred thousand dollars The time frame would be this calendar year or early next calendar year This would be the point of departure for the scalable procurement vehicle that I described on the prior slide Lastly step three would be to build the enterprise data implementation plan from step one This would be this would be everything this would be the enterprise data management data availability across the agency This would be the business intelligence and new open source analytical tools This would be defining More use cases and analytical questions that we're trying to resolve The cost of this is obviously not known yet until we have number one completed But this is likely a multi-year effort requiring multi millions of dollars and a scalable procurement vehicle So what might this look like? So step one is what I have here is this is a conceptual Framework this isn't necessarily what it would look like but this is to try to hold the picture in your mind So mid-year number one proposes to build this picture into reality, whatever it may look like I think it's likely to look something like this. So let me just kind of focus on step This this middle portion here data sources. We have a lot of data sources. We know that we have IDIs We have hotline information. We have nice data We have retailer reports information from safer products gov information from the ram and import system and there may be Dozens of other data sources. We haven't yet considered that ultimately we'd like to add to our our ingestion capability The idea would be to take all of these data sources and then the second piece of it is to create a data ingestion capability using automatic data extraction and organization to index categorize and Format the data in a way that isn't so labor-intensive where the computer does some of the heavy lifting to get it ready for analysis You know adding product and severity codes Doing some of the things that we're talking about in mid-year number two The next component of this is then data storage This is the idea around enterprise data management that that we've had a lot of conversation around in this room Whether that's a data warehouse a data lake But some common platform where all of the information is a stored is stored and available across stove pipes as opposed to just being available within stove pipes The next piece is where I think there is more clarity and that is one of the analytical tools that we add on top of this Enterprise data capability and there's a lot of open-source tools out there. I didn't include it in this Slide deck, but our CIO provided it. There's literally hundreds of different open-source tools that can be customized for our use We're obviously using sass now There's a lot of interest in our and python and how does the RAM system sit on top of that and then lastly this this next piece I think it's easy to conflate with the analytical tools piece But it's really the results of the analysis that we want taking the time to form the use cases the analytical questions That we want to answer and how we want to use the enterprise data to answer them so in our three-step process step one is creating a Strategy that is essentially this picture or something like this picture in the implementation plan to achieve it Step two is tackling one two or three of these use cases on the right hand column And then step three would be pursuing a multi-year funding effort to implement this vision I promised I'd spend just a couple of minutes on mid-year number two the use cases that we could pursue now These are some ideas that staff has had that were included in your mid-year package. We've come up with three One would be automated coding so product code and severity code assignment to retailer data using machine learning This could provide a framework for getting initial classification on data that currently are beyond our capacity to clerically code A second use case would be data matching connecting recalls in DCM cases and CPS RMS So that it would be clear when looking at an incident whether it involved a recalled product This could accelerate our predictive modeling capabilities as a recalls a critical outcome and then the third proposed use cases data translation translating product information and CPS RMS to HTS codes to allow for incident data to be a signal for inspection for import surveillance We think pursuing a limited number of use cases using artificial intelligence tools for deployment and initial results could be accomplished By the end of this year or early next year these pilot scale results would then inform the larger data strategy I put together a timeline For how this could all unfold of this is something that the Commission was interested in pursuing this summer We would initiate development of the enterprise data strategy and implementation plan through mid-year project one We would simultaneously initiate one to three use cases to pilot this technology and demonstrate results By September I Would think we would want to include at least some initial startup funding and our next OMB budget request We may not have a Complete fidelity on what the final plan looks like, but I think a startup request would be in order By December January timeframe. I think we would want our use cases deployed and see the initial results from those What's working? What's not working? By February we'd review just and discuss and approve the enterprise data strategy and implementation plan the big picture And I would want heading into the April timeframe to start Pricing out that plan in time for our budget formulation process that will run through the summer and ultimately result in a a larger funding request in the FY 2022 OMB budget, I would note that the mid-year next year could present an opportunity to sort of get started if you will So that's a lot of information I know is a longer briefing than we normally give at the beginning of a mid-year But I will pause there and Over to you for any questions you might have Thank you very much And with that we will begin our ten minute rounds of questions and I will begin the questioning Jay, let me say first of all, thank you to you and your team for Taking the time to putting that for putting this together and coming up with this these two projects for the agency and consideration by the Commission I Would like to recognize that committee that you put together. Maybe you could just tell who was on that committee so that they can all Yes Let I wrote that down because I don't want to leave anybody off. This was a terrific team Jim Jiholsky Steve Hanway George Borlase Dwayne Boniface Jim Rolfe Steve Fabry Eddie Ahmad Adam Graham and James Baker were all Instrumental and working over several months to put this together Thank you very much. I have several questions about it and you alluded to it be in your opening comments just with regards to Informing one informs two and two informs one but more broadly in the 2020 budget discussion There were some initiatives and some amendments. How do these? How does project one in project two tie into those two amendments? Right, so this is around kind of who do we need to implement the plan and and I didn't do an Organizational assessment, so let me make that clear. This is simply my views and in the context of these projects It would seem to me that in pursuing this data strategy specifically mid-year one that that would ultimately inform the Request that were put forward in the FY 2020 budget. What kind of people do we need to hire to implement? This strategy, I don't know that we need to Bring in folks to help us refine the strategy further Thank you. I want to just turn to your Graphic on slide 11 With regards to that first column under project one where you talk about the different sources where that inform the agency with regards to incident data Will will project number one or two for that matter? Make it clear because I've often thought about this are we missing opportunities to find other data I you know I have been a proponent of retailer reporting I think that that is a very good source of information and data if we can get that to a manageable where we're getting meaningful data and As you talked about In your comments, but would this would project number one inform us of any potential other data sources? I Think is in vision chairman the project it could we could add that to the scope if that was something the Commission Wanted to do the way that I think we were thinking about it was really this this Data ingestion box the issue isn't so much are there Other data sources that we might want to add we know there are right there's retailer reporting has been talked about there is the urgent care Center data that we've talked about there's probably a dozen more than I'm not familiar that we've talked about But the the stumbling block is always the same. We don't have the capacity to deal with it So I think what we want to do is try to position the agency with the tools and technology to be able to ingest data Whatever it may be and whenever that data may become available So I think they'll always be an evolution in that I wouldn't limit it to just sort of the soup to juror You know that we're focused on today. I would think that the idea here is whatever we come up with We want it to be scalable and flexible for the future. Thank you in the Well, I'll say the fourth column over the under data analysis tools and examples of some of the open source Products that are out there. You mentioned we're using SAS But can you just distinguish how we're using SAS now versus what conceptually I think would be the result of Project one and two Well, so I'll give you a cursory, but we may want to bring Mr. Hanway up for a detailed Description of that I think at the end of the day that the SAS tools are proprietary tools And it's that it's the amount of customization that we can do is limited But if you'd like a more fulsome answer, I think I'd need to defer to Mr. Hanway I think that would be helpful and I think for me My understanding was and maybe it isn't correct, but that currently the SAS requires someone there And it's a very manual process to develop the rules and I just would like to understand This versus what we currently have Yes Yeah, so SAS is is useful for a number of things and we've gotten some tools to get get some greater capacity for handling unstructured data But it's still built in a way that was built for the many And I think the big shift in our mentality as we got together with this group is thinking about things that were built for us Rather than built for the many because we do have sort of unique data issues We have unique organization requirements a lot of the things that have to do with data and Management of unstructured data is about it improving your batting average, but our standards are much higher than that we want Near total accuracy when it comes to how we classify and categorize data And what we want is something that could give us some confidence that we could we could get to that place Many places when they get data in Can view it only as an asset for us it always comes with a certain Responsibility to know what's in it and to take action if action is required So we want something that can signal to us when action would be required And we want things that can help us with the with the often very laborious task of getting things classified and categorized correctly And we think some of these open-source tools can not only help us with some of those problems At least in these use cases But also teach us how to apply those solutions elsewhere because they are open source Which means we can look into the code and look at the approaches that they use and then apply them elsewhere Thank you very much And that brings me to my next question and any one of you can answer that but just in terms of because you mentioned One of our challenges is that we can't use these open-source projects or products without Some customization maybe you could just expand on that a little bit the challenge for the agency Well, and again, I would also look to Steve a little bit But I think in Just from the project manager perspective and bringing this team together and really talking about these sticky analytical problems that we have The questions are very specific and the out-of-the-box technologies Are a little bit course and we really need to bring in folks Not only with the tools that they're able to customize But also with the sort of data science skillsets in order to help us define how to organize these Organize our data and also organize our analytics So that's really what I was trying to get out in the larger context of the briefing But Steve may want to add to that with the specific challenges he has Yeah, I think the thought was well We what we could say is we could ask for FTEs and try to bring people in who would only Practice in this area or we could leverage the expertise of people who are already experts doing it Have them work on problems that we already have and then allow the people who are here to learn to do it By looking into their solutions I think kind of where we've been with sass is to say let's make as much progress learning all the tricks that you can do with With some of these these more advanced modules Kind of in your spare time or as time allows what we're not getting nearly the sufficient time We need to get as far enough fast enough as I think Organizational we want to be and I think this was a way to accelerate where we're headed But also what seems to have become Often the most common and best practice in this area These seems to be the tools that people who have a choice of tools seem to choose most and that's why we thought we might follow their example Thank you, Jay. I just wanted to Mention briefly or ask a question. You you've mentioned a couple of times learning by doing And what's the practical implications for the agency with when we're learning while doing? So the good news is if we're able to demonstrate quick wins the learning by doing I think helps us make our case in terms of more resources That we have some tangible results to take to OMB in Congress and say hey There is a new set of tools. There's a new set of techniques and then justifying that approach The flip side of that is learning by doing means occasionally taking some risks that don't pan out We might find that some of these tools are not as robust are not as informative as we had hoped And I think it would be better to know that for a few hundred thousand dollars rather than a few million dollars Thank you very much my time before I expire I'll turn to Commissioner Edler Thank you very much Let me say first of all that the sounds exciting. It sounds like it's new innovative creative dynamic and exciting And it may well be but I'd like at least to start with a couple of sort of fundamental questions First of all Jay you said other agencies don't have our data problems, but I could think that NHTSA would have similar problems I could think that FDA at least with medical devices would have similar problems What is it? I'm missing. What is it that's unique about this agency that Creates a need for something like this So we desperately wanted to find an analog of someone that was using these tools in a way That's similar to the way that we're using them and certainly data ingestion data categorization. There are similarities So it's not up It's not a universal issue and I'll let Steve expand on this but I think the one thing that we've learned when we start talking to other agencies is the Threshold for our level of accuracy is so high And to simplify for a lot of agencies sort of the 80% solution is good enough Identifying 80% of the problem for us We're really looking for the needle in the haystack and sometimes the needle in multiple haystacks I mean, did you want to expand on that challenge? Yeah, I think the The piece is unique for us I think retailer reporting is an example of that where I think people would be very willing to transmit to us large volumes of data that include a lot of Sort of narrative description that's completely unstructured people use language in many ways There's no dramatic governance. There's no spelling governance So you can do some things searching for keywords and things that that can make you find a lot of things But you still miss a lot of things through that approach What we want to do is get something that we trust as much as we would trust someone going through record by record And we haven't found other places that have both that issue of needing to scrutinize it that way But also with the volume that we have And that for example is even why some of the private sector people that are sending us things are sending us far more than We really want them to because even their training their models can only do it in To such a degree of specificity they still have to have to grab a lot that that we don't really need well, well you've identified one in particular, which is retailer reporting and It's something I've been skeptical about for the very reason you mentioned that I look at the years that we've been involved in this and I say how many recalls really Were triggered in whole or in part and it's really hard to pin that down So I could see retailer reporting being one unique aspect But I'm having and I could also understand that other agencies may not be doing what we're proposing to do But I'm just having trouble seeing that other agencies have similar problems that we don't aside from something called retailer reporting But I guess the next question I would ask is Somewhere somebody sat down said we got to do something because we got a serious problem What was the serious problem? What was the thing that lit people's minds about the need to do something like this because as I'm looking through some of these I'm saying, you know, like tying recalls to other systems that that doesn't seem to me to be that massive a challenge It's a challenge, but not that massive a challenge So could you sort of enlighten me is what where the catalyst for this developed? I can jump in The integrated product team is an example that we've described in the past and where we have this group of subject matter experts reviewing incoming data reports to signal early on that we have Something of concern to trigger some action going And I think we all realize that is not a sustainable model if the Volume issues that we've described. I think we have a new term anomaly detection But the the idea there is that how are we finding out about things early and taking action early? That is the problem in my mind that we need to solve And I think we have examples of that through this larger strategy to try to to get at that But if there's one thing that that I believe We could solve and you can get some tools and help be on, you know, this massive Effort by subject matter experts. I think that would be worth doing Thank you. You mentioned one other thing that is a constant source of concern to me And it's if you've followed me over the years. It's something I've always raised with it Which is the hazards of customizing because what I've seen at several organizations is they've developed For lack of better terms some new enterprise some new data analysis system and the next thing you know they've customized it and then there are two people in the world that know how to work it and One of them gets mad and walks off and the other one. We're afraid getting hit by a truck So I guess one of the things that does concern me is the degree of customization and Jay you said that at some point Even if we're customizing it we're training our own staff so that they will be able we'll have a large inventory of staff Capable of running this system because to the extent that we're having a high quality consultant high quality Consultants first of all charge higher prices sometimes or they leave So is there some way of making sure we don't get stuck with the one person? We're worried about getting hit by a truck Well, and I'll let Steve expand on this answer But I would begin by saying the customization that as I see it is really on the far right-hand side of the picture that I Described it's it's the use cases and analytical questions that we're trying to solve that are likely going to have unique definitions and answers the Overall architecture and toolset that we're recommending. I think that would be Pretty pretty I don't want to say standard, but I think that would be fairly ubiquitous I don't know that it's going to be something incredibly different than what other agencies are doing on that side of it So I think obviously we would want to structure the contract in such a way where the entire process wasn't so custom that we couldn't manage it But I don't know that this isn't that the item that would keep me up at night. I mean Steve do you want to? Yeah, I think so we've talked about customized in terms of how they're deployed But things like are in Python are actually very commonly used and the open-source nature means that anyone can look in and see the work That was done and if they have a familiarity with our Python, then they're able to see This is the code that does this. This is what does that which means you could replace People who are working on it. So that that's one of the nice things about it The other piece of it is they've also come into favor a lot in that academia because they are also inexpensive or free and so In that sense, then you've got generations coming in who are learning how to use them And those are the people that we see ourselves likely to hire over the years Well, you did talk about sass, but there are a lot of the sass stuff is proprietary So if we want to use it we have to pay for it. Would we not get into issues of this high quality consultant saying Well, this is my proprietary software. I'll customize it for you, but I'm gonna own it My understanding that the approach is to have someone develop a strategy who would be independent of the deployments So that they would not be in a position to be rewarded for whatever advice that they provide One of the question It sounds like you've thought this through but if we start this and Jay you were saying it's and I agree with you When you start big projects, you do drill dry holes and you just need to be prepared for that But do we have a point or several points along the line where we say? Okay, it's not worth continuing in other words several break points that are built in for assessing whether this is turning out to be useful Well We absolutely will have to build those points in. I don't want to overstate where we're at right now slide 11 is is I Hesitate to use the word cartoon. It's just a vision of what could be I think once we had mid-year number one in that Implementation plan we would need to Specify specific gating criteria for moving forward, but we're nine to twelve months away from knowing what those would be I'll just say slide 11 was the first time I looked at something. I said I think I kind of maybe vaguely understand it so Especially sort of the different data sources and integrating them and I realize we've had issues with siloed information all along It does at some point raised the question of how Critical is it to have these integrated if I were a manager coming to work? How would my life be different assuming this Succeeded and it worked well. What what would be different about my life? Well, I think the integrated teams is one place where we've imagined a different world Where instead of relying on people to catch everything by reviewing each individual case We have something that that would review language in a way similar to human would and take notice of the things that we Want a human to take note of and so then that would provide a signal to us that there's something here that requires further attention You know if our current state is it roughly three out of four pieces of data that come to us do not require additional action That rather than having discovered that by going through them manually You have something that's able to do some of that sorting for you So all the time that you're spending looking at the data you won't act on can be deployed to other activity Thank you my time's up and again, I'm not opposed to this. I'm just struggling to understand it. Thank you so much Thank you very much commissioner Kay. Thank you madam chair. Thanks to the staff for a thoughtful presentation So I guess mr. Ray is probably better directed at you Let's just say we do all this and it's all funded and it's all implemented What does the American consumer get for this investment meaning? How is the agency? What's the return on investment and what are the tangible actions the agency is able to take That can be tied directly back to this work Such that it's worth this investment relative to other things because obviously this is this is an all relative decision This is not like hey, we can just take whatever money we get just for this and it's all free We're gonna have to not spend it on something else Sure, so I'm we're talking one and two right Well one and two and then all because one and two are down payments if we accept this now We're basically saying We are buying into a system that will require Significant resources in the future and we're sort of going down that road So we have to make that decision in the next couple of weeks So I'm curious to know if we accept that pay that down payment We buy into this plan recognizing it might move in a bunch of different directions But there's a vision in a certain direction overall I'm just trying to understand what is the end benefit in terms of the agency's output right and I I Categorize them as Being able to make decisions quicker So I think obviously when we're reliant on these human processes to review incident data and make decisions There's limitations number bodies, you know if data grows and continues to grow which you know there There's obviously more sources out there that we're not tapping into You know, we've got a challenge there on it's not scalable. So And the impact being we find out we take action sooner and we resolve Hazards earlier in the process before more consumers are affected by so I think if we hit the sweet spot on this and We can make these tools work. I think that would be an outcome that I would Expect to see from this price. So I can accept the idea that we would potentially learn about a hazard earlier But since so much of our authorities involve Certainly on the recalls just talk about recalls for instance requires Individual cooperation from a recalling firm Knowing about it earlier doesn't necessarily mean there ain't gonna be any more willing one to do a recall or two to Provide the level of remedies that we believe are necessary. Correct. It's it I can get the time factor But I'm not seeing that it's gonna give us any additional leverage in terms of our ability to actually Effectuate a better recall. Yeah, I don't Timepiece I take I'm not sure it changes the interaction on on how we do with recalls With the exception of may we have other data that we're not currently getting and that that data helps in Getting to a cooperative correct, but I think mr. Hoffman said this plan is currently constructed is not Contemplating at this moment that additional data, correct. That's that's correct. Okay, and so then let's put recalls aside So it may end up having sooner engagement of a negotiation, but no additional leverage. How about rulemaking? How would this tangibly impact our ability to? Affect hazards through will fit through mandatory standards You know if we're able to get better eyes on what's happening I think some of the examples I've seen in the past on On some of the retailer reporting data, maybe didn't trigger an action on a corrective action plan But we're helpful in developing the 104 Standards and requirements And so, you know, they weren't you know injuries or deaths that were triggered by this But there were issues that were identified through that process. So potentially that could feed into any Regulation work or updates to 104 rules, okay, so 104 is obviously we have that additional leverage that we're back to leverage We have additional leverage where we can move forward in the absence of industry cooperation What about on seven and nine rulemaking do you see a tangible benefit on seven and nine rulemaking? I Would just put them all into we have potentially more information. I'm not sure that it gives us It would depend whether there was leverage additional leverage by the data I think I think it's just helps to refine the data that we have to make okay, and the same answer for civil penalties Yep in the same. Okay. Thanks, and then let me and thank you that is helpful And so mr. Hoffman, and I guess mr. Hanway. I want to turn to you on this Probably mr. Hanway Do we have enough data? Do we get enough actual data in volume of data to make use of this meaning? Do we is there enough for a statistical value for us to actually try to implement all of these tools? Yeah, I Think even today our current state There is stuff beyond our capacity to clerically code that comes to us And that's why one of those use cases was to try to do that immediately Because the concern would be that that's some of the stuff that's hardest to get eyes on and we would really like to have a high degree Of understanding of what's coming to us To signal anything that requires action got it So you feel comfortable making statistical projections though off of the data that we have now You just don't have the capability to do the coding to do the that type of statistical projections And when I think about projections then I get into the world of the probabilistic data versus the non probabilistic data So I don't see this as mainly something we're looking into because a nice I think it really is more because of the other incident data Although if we do get into the world applying this outside my domain places like import surveillance where there's enormous volumes to deal with in terms of Customs border data There are many places I guess that are rich environments to use it I think just in terms of the way that we'd actually use it again I'm thinking mostly signal detection for the incident data in my senses That's also where we're thinking in terms of import surveillance where we're just trying to do a better job Identifying shipments that ought to be inspected got it and then probably for the two of you So within the last five years we have the agency obviously had hearing on data We heard from a lot of different folks both at the federal level and in private sector people were hawking their products people were explaining their success stories and My sense of the collective takeaway from staff was We're sort of doing all that we're in pretty good shape Was that an accurate assessment of the takeaway from that data hearing and all the work that went into looking at the comments and everything or Is this a direct result from that hearing or the work that occurred around that period? I don't know if I'd characterize that hearing as central to this. I think there were some other Elements that we had to consider I think it was both some of our concerns about the integrated team process and its scalability and Also this sense that that it didn't seem like we ought to be in a position where we didn't want to invite In further data simply because we had limitations on our ability to process especially when there's so many people outside telling us that that they're good solutions that exist and Not wanting to completely turn a deaf ear to that that message that that there may be ways to cope with larger volumes of data Even if it is very unstructured and very hard to work with got it So it sounds like what you're saying isn't I don't want to put words in your mouth Is that the the sort of continued calls to expand our data sources might have been a trigger For looking at our processes to realize how we could actually do better because we didn't want to just add data If we couldn't actually do anything with it I think coping with much larger volumes requires bigger solutions and greater investments certainly got it and sorry go ahead Mr I would just add Even if we don't add additional if we just status quo We cannot continue to do this manual process. We need tools that help identify this. Yes, and I hear that Mr. Hoffman so in terms of lessons learned and I want to pick up some of the stuff that commissioner Other was talking about we obviously went through this with the RAM system Where we went the whole route of sort of hiring somebody and then they own the proprietary Technology and then we ended up getting captured by that and then we decided to bring it in-house and then we realized that we had expected to get returns in terms of specific steps that never really occurred or a cost a lot more money or what they What we ended up getting out of our investment ended up being relatively pedestrian in terms of what we were hoping for Are there lessons learned from that? Process so that we don't go down this road and end up in the same place where we overpromise and under the under deliver Yes, I think mid-year number two That's that's the hedge against exactly that problem not to litigate RAM But that's where we were really trying to start building the picture and then decided the picture that we built And we were too captive to it So I want to the lesson learned is be methodical. What's the strategy? What is the technical implementation plan to implement that strategy? What are we getting ourselves into should we decide to proceed and can we demonstrate any Any results for small dollars before we make that commitment? I would I would just want to Recaracterize what mr. Adler said I don't think we're making a cut and you as well I don't think we're making a commitment to to start this multi-million dollar initiative I think we're making a commitment to Do some additional research kick the tires demonstrate some results and make a decision next year My time has expired so I can't rebut that but thank you Wait 40 minutes Thank You commissioner Bianco. Thank you Jay, this is terrific to you and everybody who worked on this. Thank you I think we should have been talking about this 10 years ago 15 years ago, and maybe it would have You know foreclosed some of the things that my fellow commissioners are talking about But I'm ready to do a happy dance that we're at this point right now. So going forward I think that this we need to move on this We are so far behind and I think this is a legit legitimate place to start I think there's a lot of good things here And one thing that I heard you say that you know made me a little nervous is When I think we all know this is we don't have the capacity now to deal with the data We're getting whether it's retail reporting or anything else So we're we aren't dealing with it and we're not getting the benefit of it So take keep continuing to take in data Under the current structure is not gonna do this agency or the American consumer any good. Is that right? Yeah, I think our concern has been Taking on more data without having the ability to process and deal with it is problematic Okay, and I think we all know that and I I appreciate mr. Handway's comment about you know our level of accuracy is high of course it's high But right now we don't have anything so at this point anything that we can get that moves this ball forward Is a good thing and I would encourage everyone in this room including my fellow commissioners not to let the perfect get in way of the good because this needs to be done and It needs to be done forthwith Sorry, it's a legal term. I could come up with anything on the fly on that note And are you so I didn't completely understand mid to mid-year one and mid-year two the two projects here They're not they're not mutually exclusive, right? We're gonna do those together is what you're proposing is or is it my wrong? So they could be mutually exclusive I think there is synergy between the two was the point I was trying to make and this goes to hedging some of the risks that commissioner Kay was talking about I'm confident we can bring in somebody to come up with a a plan and even an implementation plan But I'd like to have a second contractor do some demonstration while that's ongoing and have that Completed to inform the plan As it's going as it's going and at the end point Okay, because what I what I would what I would hate to do But you know beggars can't be choosers But what I would hate to do is wait another yet six to nine months to get to get answers And then we have to wait another six months or however long it takes This particular body to vote I think that we're so far behind the curve that it would do this agency justice to do them both Simultaneously and as you point out we can learn by doing you make tweaks along the way. That's how any project works Am I right am I off? Just to put a fine point on it that the sense of urgency from the diocese has been expressed in multiple meetings These two projects were staged in response to that. We understand that people Plans are great, but we want to see results. Thank you. And and do we this is probably I probably know the answer to this But I wrote it down Do we have somebody identified who would head this up right now? I mean I know it's not that's not set in stone But to get this moving What I think we would we would use our existing organizational structure and these are it projects that would be led by our project managers and it but the reason I Form this team in an ad hoc way is to bring in the program perspective and bring in the analytical perspective And I think that the folks that I named and there's that's probably an expanding list are vested in this now And I think they want to be involved We're gonna it'll be a cross-program initiative for sure one of you mentioned and I remember who but you know It's a labor-intensive issue and we don't have a lot of people so while I agree with you that the people who are invested should stay I do think that we need to bring in everybody spread then we all know that so we need to bring in I think Mr. Hanway you mentioned that bring in some fte's to actually fortify that effort correct That was one option you could consider so if you didn't if you didn't have External parties building the use case you could hire people and designate their their tasks to be to be building this use case We would just have to make sure that as we put together operating plans that we didn't then assign them to something else, right? Yeah, okay, I think that's important and and just to address some of commissioner case questions I think the way I understand this You know the the importance and what the American consumer gets here by by moving into The century is that we can start predicting some of these hazards before the injury occurs So we can get to a recall before the injury or Before more injuries occur Do I understand that correctly the I mean information is power the more information we have and the sooner we have it We can go to a manufacturer for example and say hey we see this Do you see it and if you don't see it we see it and we can use that information to Move our mission forward and to enforce the laws and the regulations that we are put in charge of yeah I think in me or to that second item the data matching and that predictive modeling is trying to get some experience with that We've we've had that desire for a long period of time to be able to try to connect Hey, we've seen a hazard. It's similar. We see something close. Should we start looking at it now? And so I think that aims to get at that that that capability For as far as money So if I look at this list of things on here 13 things and and certainly not going to suggest that one is more important than the other But you know we have here the enterprise data you have a number associated with it And you have some different things that apply to what we're discussing here What if we took all the money that's available at the mid-year and put it into this plan? Doesn't that move us forward? No, it doesn't I tell me why sure and it's an important question and I appreciate you asking it We don't want to be in the ready shoot aim mode. I totally agree and I think that In the market research we've done the importance of and just the discussion we've had here today There's been some really good questions. We need to think through What is the enterprise strategy? What is the implementation plan and then do we all agree with it and It's not How much money we can put on it? It's how much money is enough to put on it to get a good result and I'm comfortable with the estimates that we have here for now And we'll probably come back with a bigger estimate of people get excited about this nine months from now Okay, terrific. And what what market research I've heard you refer to that a couple times Can you tell me a little bit more? Sure, and I'm gonna intentionally be a little bit opaque So it's not to bias the procurement process but we have done market research in terms of Speaking with vendors to understand what the capabilities are that exist in the marketplace and how they're applying those capabilities to other agencies We've done market research in terms of talking to vendors that have been tasked with coming up with Strategy and implementation plans for other agencies with with similar problems Maybe not identical problems, but similar problems And we've also done market research in terms of talking to customers of those vendors and other vendors I guess I can reveal organizations. We've talked to NIH for example. That's done some rip. They're got a lot of money They've done some really exciting Work in terms of applying some of these tools What have they learned where where have they drilled some dry wells? They spent the money they can share those things they can yes They can share them to the extent that they're applicable, but it's just further back to this So we do have some unique are you our use case pieces unique? I think I overstated the customization piece the questions. We're trying to answer incredibly unique Okay, I think It would be helpful just gonna throw this out there to me and I'm not speaking for everybody else But I'm guessing that if those who are the People in charge or the experts to come in and just show us put up a little demonstration This is how this works. This is how this would work. This is what this would do I think would help inform at least this little group up here and Everybody who will be involved going forward so that we're all speaking on the same on the same page because I can tell By the questions up here and your comments that we're all operating on different technology Knowledge levels and so I think you know again information and education is very important here so that we can make good decisions sooner That's all I have right now. Thank you very much. Thank you commissioner Feldman Thank you, and thank you all for being here today the discussion so far about the path forward on expanding CPSC's data capability and analytics capability is encouraging. I think it's an important discussion It's something that's come up Previously in the context for instance of expanding the retailer reporting program in the context of taking a look at some of that Epidata that that drives our epidemiological work and ability to spot Emerging hazards and in actual hazards with with product in the marketplace From time to time and I'm not quoting anybody here but but you get Excuses and concerns along the lines of it's just too much data. We can't We we don't know how to process that I think in 2019 that that's no longer a valid excuse given the amount of Applications tools resources that exist for handling large data sets And I'm hearing a bit of that today, but I'm glad that it's out on the table and I really think this is at the heart of the discussion Expanding our data capabilities bringing in large data sets particularly when we're talking about spotting anomalies and trends Even if the data set gives the agency a better sense of what normal looks like Then you've got a richer basis on which to spot even the Smallest anomaly so that you're seeing the smoke before the fire and you're able to get out ahead of an issue before it man Fests itself into a full-blown crisis I think with the speed with which the global supply chain is operating right now with Asian Manufacturing and the ability to get stuff into the United States not just through the traditional you know seaport container Distribution models but via e-commerce and surpassing our ports and sort of directly into commerce increasingly we're gonna see a situation where Where product that that was an idea just a couple weeks ago is all of a sudden Everywhere and therefore our ability to spot and and and stay ahead of that curve is is important So I think we need to expand the agency's capabilities and that includes taking a look at our systems our human capital and our expertise Really encouraged to hear the discussion today. You know right now. We're struggling under the current system Not just to make sense of the data, but to keep that data confidential So I think structural reforms are needed. I think this is a longer conversation I think it's something that we're not going to solve in in one pass at Making decisions with respect to un-executed mid-year dollars in 2019 But that's as good a place as any to start so I appreciate the recommendations I want to pivot and talk about something a little bit more discreet one of the top complaints that I hear from staff is Concerns about the reliability and functionality of the agency's VDI system It's a clunky system. It's temperamental and it's having a material effect on Material negative effect on staff morale and the ability to execute CPSC's important safety mission When I've had an opportunity to talk to staff in the field You know, they find that often they're kicked off of the VDI system that there's a direct loss of productivity associated with that and You know, I'm talking about inspectors and compliance staff that's that's out there across the country doing the important work of the agency And it's something that we've experienced here at headquarters when staff at headquarters is kicked off of VDI the agency grinds to a halt this happened most recently on March 7th of this year when the agency essentially went dark for more than 24 hours And that's not the first time that this has happened. My question is Because I'm hearing directly from the rank and file staff and I'm hearing from CPSC supervisors I'm hearing from folks on the seventh floor and I would candidly include myself on this group that that it's a problem That it's a top concern that it's demoralizing when it happens Where does addressing VDI in improving the system reliability fall in the staff priorities that you've listed here? That is not listed in one of the projects that is before the commission. Okay, that wouldn't be something that would be covered in the IT assessment Okay My next question would be, you know to the extent that alternatives to VDI exist What those alternatives might be what the cost associated with transitioning to a more reliable system may be? You know, whether there's been sort of a cost-benefit examination of that kind of transition I'm assuming it's not cheap We definitely did the benefit cost analysis to transition to VDI And I don't have that information right in front of me But that's something we'd be happy to brief the Commission on. Okay. I'd love to follow a briefing on that when you're ready My next question had to do with project number two You mentioned how AI could be leveraged to help improve the enterprise data analytics implementation and putting together a Pilot study to use machine learning and AI to benefit any number of mission-critical items What's the agency currently doing with respect to AI? Is it technology that we're deploying in any way right now? I would say there are things that can be iterative, but they're very manual So it's it's very hard to say that it's really the machine doing the learning so much Is it's it's that you can train the machine and so it's still very intensive in terms of training I'd say many of the solutions look like that the hope is though that once you've built something That's that's very mature in terms of its training Then it's doing more of the work than the person training it for right now It's almost a one-for-one in terms of the time you invest if you were to go through things manually You might take a similar amount of time in many cases as it does to train the machine to get to a similar place So we are currently deploying AI at CPSC In a limited way, I would say yes, but but but not to the extent that I think We'd like to okay. I have no further questions. Thank you Thank you. I believe we'll go for a second round of questions So I'll start that second round. I want to just clarify a couple points I think that in one of the conversations we talked about the uniqueness of this agency And I think if you look at us versus say a NHTSA We have 15,000 types of consumer products That's a far different than just a car and I understand there's a variety of cars But that's one of our challenges is the scope of our work and our jurisdiction and so that is a challenge There therefore is more data coming in Dwayne something you said is of concern to me and I'd like you to just expand on a little bit with regards to the Integrated teams and the sustainability of those integrated teams. Can you comment on that and? How this these two initiatives at least and before you get to your question. I just want to because I want to clarify something Project one and two of this mid-year is Only I think what it will provide to the agency and to us as the decision-makers the art of the possible It's to understand To give us a roadmap It doesn't commit us to any future actions You know we may you know But it informs us and it gives us a strategy forward But to your point Jay, and I think this is important That ends with that project one and any deployment of a strategy is a separate Initiative and it's not tied into The contractor that we're working or we would work with in the in the first project I think that's important to because I think Bob raises excuse me commissioner Adler raises You know valid issues there. There's you know sales people and they want to sell us their products and to keep those independent I think is critical I'll go back to Dwayne now Yeah, when when I say it's not sustainable. I think as good as our teams are and as Highly qualified as our experts are and looking at this data we still don't catch everything and That is the part where we need we need help. We need some tools We need improvements in that that I do believe will help in that area And then the data will continue to grow just by you know the fact of where we are the way The way consumers interact with with products how they purchase products all those kind of things are changed over the years And you know, I think it's one of these where we need to Modernize how we look at and view that incident data and process that and that that's what I meant by not not sustainable for over the long time Thank you the We talked a little bit about how this would benefit the consumer The other area where I think this could be beneficial is informing voluntary standards more data more incident data Understanding that incident data having the chance to review it I think and Dwayne if you could comment on that it would be Informative to the voluntary standards process sure and the example I was using one of four you can translate that right right to a Voluntary standard, you know, we get we get data that may not trigger the trigger a system substantial products hazard threshold, but there is Problems that we identify and improvements that we could see through that and we can make those recommendations to the standards bodies for sure Thank you. I'm gonna turn away from project one and two in in the plan that was given to us and Would like perhaps Steve or Dwayne to comment on end cares This has come up before and I think it would be beneficial to all of us to get an explanation of end cares And how that would improve Our information here at the agency. Yeah, as an agency we have some history collecting exposure data But it's typically been one-offs that we identify a single product where we feel like it's necessary to reach out directly to consumers about the products they have and what they use and then that goes through the entire cycle of Identifying the funding, you know preparing a statement of work specific to that product Getting it paperwork going through PRA and getting that kind of clearance for it getting the data back doing the analysis And I think if this sort of recognizes that that might be an activity that we do Or we'll want to do with some regularity So we start building something that allows us to do that on sort of an ongoing basis For things that are on our horizon to where we think that kind of data would be useful What it also does is takes advantage of economies of scale So much of the cost born of doing any kind of nationwide survey is the time you spend in getting Things sampled and getting cooperation from a respondent and if you're doing a single product, let's say for example It was like the nursery products You were asking people if they had any of these products or if they had any children you get a quick No, that's the end of the call you hang up But most of the cost was was born and just getting to that point Well, you could then be asking about other products that they might have or use and if the answer is then yes Then you could go into those modules and ask about the other products that you might be interested in for other reasons So so it does kind of combine efforts on these to make them at least on a per product basis a little more affordable And so it's one of those things we identify as a gap in our knowledge when you talk about risk There's sort of the numerator of the adverse circumstance but there's also this sort of denominator which is how often was a product used safely and that's sort of informative for answering that question as Well, so so n cares basically without even getting products specific at this point is just a vehicle for getting that kind of information Thank you and Dwayne with regards to Item number seven iot the lab out at 5rp. Can you just expand on that a little bit sure? I think the idea was without spending a lot of money get some experience with Expertise from outside the agency through nests some some computer scientists that would focus on looking at what we've Put out there as a potential risk, which is this hazard is Asian. I think the term we've used Get some products see the potential there of actually Exposing these hazards through those products and so this project is you know a small Investment just to try to kickstart some learning on that with some practical products connected through Through a network to to get at that. Thank you Item number four safer products gov modernization. We had a hearing on The modernization of it and what the users of it and the constituents were interested in In seeing us do which was extremely I thought informative hearing that we had I'd like to talk about that a little bit because that is a priority for many and for us It's a good source of data and so can we can you one of you expand on that a little bit? And in what how we would approach this modernization of safer products ask Jim if he would Coming up on this one. Thank you. Yes, so there has been work that's been ongoing I'd say probably over the last course of the last year including the feedback that came from the that came from the the hearing and also requests for information asking for input and advice on Modernization and making improvements for safer products gov we received a number of of Suggestions and things to that would make that platform more useful Chief among those are making it a more mobile friendly interface so it functions currently on a mobile device But making that more modern and also looking at ways to streamline the Way the data is entered and see see if there are ways to make that easier My understanding is that much of the data that is collected through that is required and so there may be Some constraints to quite how far we can do with that There have also been some other requests that we're considering including things like Making making the data and visualizations of the data more more available on the site Thank you very much Commissioner other Thank you very much madam chairman It isn't just that NHTSA has one product category It's there there are thousands of pieces that go into Manufacture of a car, but in terms of broad product categories I think that if you look at the medical devices that FDA regulates They've just got thousands of products that are under their jurisdiction, which is why I'm just curious why and when you say We're unique. I still have trouble Figuring that out. You needn't respond to that but one of the concerns that I am entertaining is that And let me talk specifically about Retail reporting because that seems to be the source right now of big data that we're confronting is that I don't want CPSC to suddenly assume responsibilities that the manufacturers and retailers have They've got obligations when they discover products that have defects that could present a substantial product hazard to notify us and right now I still feel that we're getting data dumps from Big big institutions like Walmart and Amazon that should be doing their own data analysis to red flag Products of concern and I would hate to see us then take that responsibility on to ourselves It's anybody thought that through and is there a good response to to that? I I think Even if we set aside the retailer reporting piece We we currently pay Contractors to code all of our incident reports coming in so there is an efficiency Even without taking on new data if we were able to use some of these tools to use the machine to do the coding vice paying people So I so set aside that policy question I think there's value in trying to get to that answer and if there's a policy direction to bring that in We've got tool sets to manage. Well, I do see the policy question is a big question one of the other concerns I have is that Looking for anomalies looking for subtle changes in the market looking for things before they've become evident Questions, what do we do with that when we get it? It seems to me one of the big problems We have at this agency is we have no problem identifying products that are Injuring people we have problems addressing them for example e scooters I see is a serious issue and I'm not clear. We're on top of that I was not happy with the response. We had with hoverboards So part of it is yes, let's find the anomaly. Let's find the subtle hazard But then the question arises about what we do with that and I'm particularly concerned about Things like chemical hazards. Is there anything in this? Proposal that would address chronic hazards associated with toxic chemicals. I Would say that's probably not its central strength So I don't want to rule out that there's some benefit. We could we could derive in that domain But I would say centrally it's difficult when So often they're not thinking about the single incident so much as perhaps just the presence of a material that they're concerned about One of the other questions that was raised is the whole issue of import surveillance, which is I think a huge issue In particular the issues arising from e-commerce, but I notice e-commerce was listed as 13th in priority I would have thought that e-commerce would have been fully integrated into this proposal And so can you explain the disconnect between the concerns there? I think with regards to number 13 on the list right part of the part of the Ranking on that had to do with there's a dependency on the work that's ongoing right now and that's not complete But there's a window of opportunity to get the ideas and projects forward So I think that was part of part of the thought and thinking on that however switching to Meteor project to I think the idea there of linking Our dynamic case management system with the harmonized tariff code is trying to see if we can tie in Deep while I'm using the term defect kind of items for inspection vice just staying in the regulated product So I think that's a way to try to get into that universe to try to see if there's a way to connect the data and make those Assessments at import. Well, I think that's a proper way to approach it Ultimately if we're not doing dealing with the defects and imports to me that means that we've got the serious gap And I did want to go back to the end cares because Picking up on a point that the chairman was making we've got 15,000 product categories I assume we're not going to be doing exposure studies of the 15,000 product categories so I'm curious how we would decide which products we're going to look for exposure data But it still strikes me and I will admit I am a skeptic about this that what we need to be doing is assessing What's hurting people and not necessarily what the exposure is? The it's exposures interesting information. It's nice to know I don't see that it's essential to know when it comes to doing either recalls or safety standards And so I'm just curious this almost a million dollars of expenditures. Can you give me a more concrete? Idea of how this is going to help us do our job. Yeah, I think this is often where I get into a Place where I want to cite specific products, but then it seems like I'm stepping into policy areas So I'm trying not to do that But for example if we were concerned about the uptake of the Internet of Things and how many people actually have things Things and how many are using them? You know didn't know when I bought my oven that it actually was enabled for this stuff, but it is not using it that way Could I be I don't know why things things of that nature that we think would be usefully informative to understand the risks That are out there and especially if it's your task to decide what's an unreasonable risk Then you might want to consider how often things You know don't end up in an adverse outcome as much as you're considering how often that they do But again, that's that's certainly your judgment to me Yeah, and again for me the starting point is what's hurting or what's killing people and then we can look at exposure, but Picking up on a point that Commissioner Bianco made I don't think we need to let the perfect defeat the good here I think we generally have an idea about what the exposure is even if we don't know precisely We can go to the industry you can find out what industry sales are and we can get a general sense This is providing to me a greater level of specificity than I that I'm I'm not certain We would have and I wanted to just ask about this modest 75,000 expenditure on IOT and Maybe just because it's in the news, but it's been in the news constantly about how the Russians and the Chinese are hacking every System that exists and I maybe we're just too small not to have been hacked in a more fundamental fashion But without detailing the specific security steps We're taking it if we're gonna do this project have we set up proper security to do this to understand Issues relating to IOT Yeah, I think the plan would be there would be an isolated network not attached to any of our working networks to To do that kind of testing that would provide those Security I Don't mean to jump around but on item 7 the business process review for IT modernization of the integrated field system Can somebody give me an example or two of the topics that would be addressed Is this going to be placement of field staff training of field staff? Computer sort of support organization of field staff. Can you give me a better idea of what's involved here? But this Six I beg your pardon. It's seven on my list of questions item. Yeah, item six is really related to Modernizing and and providing a case management system for the regulated Area and within compliance and this process review before we went out and built a new system based on our Process that we've been using Was to make sure and do review to say is there a better way to do this have we you know Do we ask for? 10 samples because that's what we've always done Is there a better way to process that and then that feeds into the requirements piece for the development of the system? So it's part of let's make sure our processes are good before we get to the business requirements for system development One last question at this point I'm looking at project 10 child strength measurement We went out to 5rp, and we talked to staff and we were asking would additional resources move the tip-over project More quickly, and I think we were told that throwing additional dollars wouldn't necessarily help So these are obviously Incremental dollars above and beyond what we're we need for doing the tip-over standard. Is that correct? I Think this one specific on the child strength is broader than than just the tip-over piece It can be used in other But is this something that would be necessary for the tip-over project in other words? I don't want to leave it contingent if this would be essential to the fundamental work on I told you no Not necessary. Thank you Thank you commissioner Kay. Thank you madam chair Mr. Hoffman if we can look at the chart with the 13 projects, and if we do the math Where? Is the sort of above the line below the line where we have a reasonable certainty of funding versus uncertainty? Is it between six and seven? Based on the estimate that's in the memo that's about right and so Unfortunately dr. Borla she might have to jump up here since mr. Ray just bolted So I'll wait till you get to the table So dr. Borla's if we don't fund accepting what mr. Hoffman just said that the likely versus unlikely line is Six up likely to get funded seven down unlikely Recognizing mr. Hoffman. There are all those caveats of That go into all of this contract work and that what we end up approving Might not be the list of likely and unlikely I get all that but what we know today Dr. Borla's if we don't fund for instance the smoke alarm project number eight What's the safety cost to not funding that? For for number eight, there's no impact on what we're already planning and doing so For example, the commission has previously funded the smoke and seal alarm survey that is undergo that is underway right now this The completion of that is not dependent on this money. This is actually funding Follow-on work after The survey is completed. So the impact of not funding it now is we could reevaluate it as part of the 20 operating plan Whether it could be funded then but it will not impact the current ongoing seal and smoke alarm So got it. Okay, but number nine was the safety impact of not funding the gas appliance. See a sensor work. That's proposed Again, it doesn't impact any of the current Commission direction in the 19-op plan For example, this is not impacting the ANPR that the Commission's directed to be brought up. This would impact the follow-on work in terms of When we would do it again, it would be something we would evaluate as part of the 20 operating plan If it's not in the 19 mid-year and I noticed in the description for this project that it was deemed that this was necessary to Answer questions that were raised on the voluntary standard on this issue on furnaces in terms of the durability of these sensors that are being used in Europe and Japan is That's something assuming the Commission approves an ANPR on this and we want to move forward on the rulemaking Is that work though that we will have to address at some point to do a mandatory standard? Certainly in addressing the effectiveness of the current voluntary standard for If we were proposing a rule yes, okay, and on the trial the next one the child strength measurement Commissioner other was asking about this. I can envision of course this being involved in PPA PPP a packaging and child resistance work and other things. What's the direct safety cost of not funding that project? This is actually just trying to supplement funding that the Commission's already provided. So really what we're just trying to do is expand The number of questions we can ask or you know the number of Different data measurements we can get from the children as they're being tested, which is already funded separately so the impact is we would just have a Slightly rest slightly less rich data set at the end of the testing if this was not funded And how about on the ATV 11 what's the safety cost of not doing that? This is the last piece of that multi-year data the Direct safety would just be really understanding For what's out there for example designs that are coming from Australia, etc What is the impact? But there are a number of other technical reports that have already been funded that will already be be published So this just doesn't complete the effort that we've invested the time and the money on and it would Limit in some way the amount of technical information We then bring to the voluntary standards organizations this whole ATV effort has been focused on Putting together all this technical information to go back and work with the voluntary standards organizations It would just provide Kind of a less fulsome picture then we planned and going back to them and Unlike maybe the child strength measurement But maybe like the smoke alarm one or even the gas appliance Do you envision coming back and looking for 2020 funds potentially to do the ATV work if it's not funded now? That's something we would then evaluate as part of the the mix with everything else You know that's part of the mid-year processes if it's Not funded in the you know VxHR funding at all plan time We've got that list of projects that then we come back and look for it mid-year, right? Okay. Thank you That's been helpful. Mr. Ray you benefited from dr. Boyle's jumping in in your absence. Mr. Hoffman the There are three it sounds like there are three IT related projects in this there is the IFS modernization There's the e-commerce work, and there's obviously what you've described as mid-year one and two I Understand that as you're presenting it Or maybe for all three of them that we're not really committing if we fund these or are we committing to some of them But not all of them if we fund these projects I'm trying to understand what our commitment level is and then To the extent that there is a commitment level What are we looking at in rough dollar terms that will immediately be walled off in the staff's proposed 2020? I plan because we've taken this step today like that is now off limits going forward I like the way you phrase that question. I think that's provides clarity So first let me tackle the results, and then I'll tackle the fencing question. So let's just go through them so product project one The commitment there is to have a plan and an implementation plan that we can then get back together and evaluate whether It's in a forum like this or through the executive director And then ultimately probably the operating plan with respect to project number six One of the things that's been discussed is hey We it's great that we want to go for a new enterprise data approach, but we have existing systems That are becoming antiquated. So I I see the IFS as a separate item I would note that this is a this is not point has not been made the IFS business process review Was a commissioned directed assignment in the operating plan and as a result of the shutdown We do not have the man hours to complete the project so this is simply the complete work that has already been directed to do some process mapping and see what a Modernized IFS might look like so that's all we're committing to is essentially a pre requirements And then with respect to e-commerce concept of operations. There's an IT component to this I see this as a more programmatic Proposal I don't want to misstate it. It's in Duane's area But I would say that I wouldn't characterize that one as an IT with respect to fencing for 2020 operating plan in the timeline slide that I proposed number 13, I think I I think I made this I tried to make this clear that there wouldn't be money necessarily fenced in the 2020 operating plan I think we would be coming back to you in May of 2020 is part of the mid-year process for Additional work you'd be much more likely to see a startup request in the next OMB budget this September So none of that if I'm hearing right none of these projects would fence any 2020 Money in the staff proposal at this time There's no recommendation to do that the 2020 has already been formulated We're gonna put a 2020 operating plan before you I don't want to prejudice the recommendations of the executive director But I also don't want to put the cart before the horse. This is a concept Get your reaction to it put the plan together and then I think my advice is the CFO would be to follow the budget process at that point Yeah, and that's very helpful I appreciate that response and then dr. Borlase in the remaining time back to you for a little bit there's no There were no adjustments proposed in the mid-year and obviously a mid-year is a Half-year adjustment to our operating plan. It doesn't stand alone This is in essence an amendment a proposed amendment to the commission's operating plan, but there's no Proposed amendment to any of our mandatory standards tables work or her voluntary standards table work Does that should we read into that that? Staff anticipates hitting every objective that is in the operating plan for both the mandatory standards and the voluntary standards I think it's fair to say that we're executing that commission direction. That's in the operating plan and Certainly for ex HR if there's Anything as the years gone up and you know furlough time Etc. I lost for working through the executive director's office to make sure y'all are apprised of any changes that are gonna happen from what the Commission direction was in the operating plan to what we're able to execute And I guess I'll ask the flip. Is there anything that you staff knows now at this juncture is it's highly unlikely that that Target will get met that's currently in an operating plan for Either in the mandatory standards or the really mandatory standards because voluntary standards. There's no target is just a dot Well, we've had the previous discussions about the crib mattresses the supplemental crib mattresses NPR working like I said to the process Executive director and then also in our briefings to each of the individual offices that one specifically talking about the challenges We're having in meeting the Commission direction in 19 on that one But it's still you're not proposing any changes at this juncture to the mandatory standards table You did not propose any no, okay. Thank you. Thank you. Commissioner B. Oko Thanks, I'm sorry if I jump around here Okay, I gotta ask you this because I'm a little confused maybe it was my notes and I know you're very precise So when I add all this up, so we have in the summary, there's four four point nine million to spend Projects add up to five point something that's a hundred thousand bucks difference, right in 75 We have a project here that For IOT, which I think is important and then when we spoke around the budget time I was under the impression that there was five million available That we would be anticipating a mid-year now. I'm hearing three point five. Can you just help me put all those numbers together? Sure first off I've been caught we the Let's see which one was it Project number four safer products gov that project had a last minute Change in the estimate and we updated the table and didn't catch the the math. That's okay I mean, this is not a challenge, please don't take it that way I'm just trying to see where I can get the most money at the most use Yep, and then with respect to our conversations and Commissioner Kay with his experience was kind of alluding to this So the current unexecuted balance if the year were to end today That's the number that I've given you that's my best guess and and what we talked about is sort of what Happens between now and the end of the year and what I the reason I think that number could grow in fairness is I don't know What ultimately won't get executed in the procurement process every year There are things that are in the operating plan that the Commission has directed us to do that for one reason or another Either a contractor doesn't bid or the bid proposal that we received is just so wildly different than what it is We wanted we make the tough decision to just say we're gonna We're just gonna cut this and that results in additional funds in the major process So that's why there's some fungibility in that okay That helps and so the the line that we were talking about between six and seven could definitely move and that would move Depending upon the reorder if we decide as a Commission to reorder them Yes, okay, so let's talk about this a safer products gov because you mentioned that we have 590,000 noted on this do we really need 590,000 not 500 Right 590,000. Do we really need that much to do what what mr. Rolfus was talking about? I mean what does that cost a lot of money? Well, I'll let Jim respond to this specifically if you would like to come to the table I would just note that this estimate did go through the investment review board process You know, I'm just trying to figure out sure what costs so much. This is my ignorance. I apologize So so it is an estimate and we won't actually know what the what the actual cost of the Of the project itself will will be until we actually get the bids from from the contractors This is based on Estimates from my team based on similar work Looking at the extent of changes that need to be made to the site and Looking and calculating that against Labor hour costs and so that's how that's how we generate that estimate the estimate could end up going higher or lower And there's also the possibility of making some adjustments in terms of which requirements actually go into the formal contract or not Right now This is this is the best estimate that we have and it is somewhat of an evolving process Okay, so so my questions here are really designed so I can make a decision as to if I have this pot of money and this pot of money Where do we get the the most return for for that expenditure? And so when I look at, you know, 590,000 on saferproducts.gov, which we all agree needs to be updated And compare that against right now We have seven point four one point four five million on the mid-year one and mid-year two Which I think everybody agrees needs to be done And and we're investing in that so comparing those two I'm just trying to get a better feel and then let me take that a step further So I heard about two projects one was the smoke alarm Survey which we have a hundred fifty thousand attributed to Excuse me. We have end care, which is also a survey that we have almost a million dollars attributed to and Projects one or mid-year one and two We need to do to analyze that data. I mean getting more data does us nothing if we can't analyze that would you agree? I would say the analytical challenges that we've talked about with our incident data are sort of independent in the state that we Do expect to get from the survey analytically it doesn't present to us the same challenges Why not? Because it would be more structured and more confined in terms of what we're looking for We would have a set of questions that we were asking people and having them answer it and so just the the scope of the analytic task Doesn't compare to finding anything potentially dangerous in the incident data. So oh, I see okay So we're not going to really find out anything for example an end cares about whether something's dangerous or could cause harm We're getting it's what people are using right? So To be fair I mean analysts do this stuff all the time I mean can't we reach out and there's other forms other than spending a million dollars Which is a significant amount for this agency to find out what products people are using It's a gap in our knowledge, but it's not as big of a gap as it is as our system We have big gaps in our system. Yeah, I think it's I felt that my responsibility to bring this to you is something to Consider but it's certainly up to you to decide how to move forward I just want to make sure that I'm not making assumptions. That's all Which one of the projects Which project or projects on the list of 13 best compliment mid-years one and two I would say the one that I wouldn't say I don't have compliments. I would say the one that to me is related would probably be mid-year 13 Because I do think that that e-commerce Programmatic strategy would inform the system piece. I don't know that it's his urgent Okay, and how about number six is I think When when Elliott was asking these questions, I wrote down. Well, why isn't this part of mid-years one and two? I mean shouldn't this already be funded this identification or modernization of our Integrated field system, or do I just not understand? I think we just viewed that as a Different need so this is we got it. We we don't have one in our regulated Product area we need one we've got it We put that in the mid-year or it was in the operating plan as Jay pointed out and We need we don't have people to throw at it, but we need to we need to make that move ahead So I think that was why it was broken out as a separate process. Okay on the ATV Stability and rollover you have a six hundred ninety five thousand on here Is there any suggestion that the voluntary standards are not moving on a positive basis forward? this This specific project was part of a multi-year ATV plan that That the XHR staff should develop several years ago And so this was bringing that last project to bear. It wasn't necessarily tied to Voluntary standard activities or any of the work I thought I understood George say that it would give us more data to go into the voluntary standard process and be you know have more information That's what sure. Yeah, it gets the proof of concept on these Occupant protection devices and that may Drive some recommendations in that area. I wouldn't want to say there There was nothing that received but what I was trying to get as I don't think the voluntary standards were driving the activity No, no, I know that I mean I'm just trying to see if you just say we might give us some information that may help us for 700,000 and I wanted to know is there any reason to believe that we really need this that there's some Reason that the voluntary standards that are the process isn't going to adequately address the the issue Okay, I'm happy to get back to you. All right, that's fine. I Write my stuff down as we're talking and then I can't go back I wanted to go back to something that that Bob said. Oh IOT So IOT 75,000 I think is is a small number and I think I don't think we're really developing a Project to see if we're being hacked rather It's to look at projects that can be hacked and to see if there's a safety component to it. Is that correct? Yes, okay. I heard on the radio just today, which I had never thought of up until today made me a little nuts That there's a concern that your camera on your PC that everybody has can easily be hacked imagine the safety Risks and harms that can result from something like that So I point that out because I think this is a very important project And I just wanted to make sure that we're gonna use that money if we vote on that to actually look at the safety Components of the product not necessarily the CPSC being hacked It is focused purely on the safety of the product and whether the product by some external Action could become dangerous And I just read this week a 2017 report from the staff that talked about some of the hazards that they've identified a few years ago I think Elliott instructed the staff to have it done and there was a lot of IOT stuff in there So I think this is important. Thank you Thank you commissioner Feldman Thanks in following on the discussion of the IOT lab I understood and I don't want to speak for any of my colleagues, but I understood Commissioner Adler's concerns about Security is sort of coming from a place of operational security. You're absolutely correct It wasn't that we're trying to develop something to prevent ourselves from being hacked right that that it makes sense You know obviously CPSC is a safety regulator. We are not a privacy or security Regulator, but there's a nexus between all of those things and safety potentially that if we're putting together a lab to take a look at a Connectivity that it be air-gapped or that there be some sort of Security measures put in place to make sure that we're not inadvertently Opening up a potential attack factor against the agency that would create security or Privacy concerns with respect to our broader systems The light bulb just went off. So I'm sorry. Sorry Bob I Listening to the questions that were asked I think I've had a lot of what I wanted to push on addressed I still when my eye runs down the Right hand column here the number that jump jumps out is this nearly million dollars for end cares This is the largest ticket item on on the the table here. It is You know almost 30% of the total projected on obligated funds the 3.5 million that exists right now I don't fully understand What it is what the survey is looking to accomplish how this fits into our broader mission So if you could take some time and just unpack that a little bit that would be helpful Yeah, and just to give you some history on this So for example, I think it was 2001 the last time we did an ATV exposure study And so that that cost about a million dollars at that time to reach out nationally Hear from everyone about what they had what they used and I think it's because there are times where we find ourselves In trying to understand an issue That where where the elusive piece is the use so for example, we had something come up I think in the past year where we weren't sure If you know that there was an outcome that might have been driven by say a product life assumption And this is an opportunity to you sort of observe that directly and say okay Are people really keeping these products as long and are they actually using them? Many of us may have a bicycle in our garage that yes, we bought it Yes, we still have it, but we haven't touched it in three years and it's got a quarter inch of dust So it's addressing that sort of gap that exists between simply talking about sales and product life and getting into actual use And then also it can get you to towards some of these other frontiers whether it's thinking about chemical exposure or it's thinking about some of the some of these newer products where maybe a lot of the The adverse outcomes haven't manifested themselves yet and yet we are concerned about that so we may Want to sort of understand how people have and use some of these newer things that it that have just come out Okay So it would be a survey that would increase our insight and expertise into product life cycles and sort of overall exposure based on Sort of how products are distributed in the marketplace and what the Sort of exposure ratio would be or simply if we would just say what products do people have in use and so if we've got some That are on our radar and we're Questioning what that universe looks like for example within our ATV report We pulled out certain tables because we felt like we no longer could use data that was that old to talk about You know what people might have in terms of their exposure And so it's it's just bringing to our understanding of any particular product that element of not just what's gone wrong But what what people have and how often things haven't gone wrong and and again I think that's for all of you to judge whether that's valuable to you in making your insights or whether you feel like the Other sources are useful. Okay. That's good background information. I appreciate that. Thank you I wanted to ask about item number 13 the the e-commerce con ops Can you unpack a little bit who would be involved in putting together this concept of operations? I see that this is an Exis program within within the agency So I would imagine mr. Jawolsky would be intimately involved with this but is this in developing this concept Does this contemplate? coordinating with other entities in the federal government outside of CPSC are their Industry stakeholders that are sort of integral to how E-commerce flows in the United States that that would necessarily have to be a part of this and are there some force multiplier or cost savings associated with Reaching out to a broader and more diverse group of stakeholders to help actually draft to this concept A lot to answer. I'll try to get them all but I've got four minutes and 45 seconds. Okay. I got First of all, yes, Jim's team is definitely Would be the lead on this project This anticipates contracting that workout so getting put issue in a contract And likely use one that we already have in place And and that would would feed that As far as other agencies Obviously, we're gonna be engaged with CBP in that Any other entities that you know, I think part of this hedging is the current work that we're doing is identifying stakeholders that would be relevant in this space to To to talk to and get their perspective and I think that's gonna influence how this plan and this concept of operations would be developed Okay, I think it would make sense to loop in a broader set of stakeholders at the earliest possible opportunity when you're developing the concept to make sure that whatever is ultimately developed Make sense in the real world is implementable and it's something that we then don't have to recreate the wheel In terms of deployment when we get to that stage, it's not if it's it's a when this is something that's gonna be on the Agencies table for the foreseeable future. I just I want to go about this in the most efficient way possible Sure, I agree with that Lastly, I wanted to ask about item number 10 the child strength measurement This is $100,000 for a quantitative analysis of postural data Dr. Borla's Where are we deficient in our understanding of child strength measurement? What is this expenditure going to ultimately tell us that we don't know currently? So we already have a sorry. We already have a funded effort underway on doing child strength measurement and As part of the research in that they are measuring the forces associated children opening things Gripping things etc. The idea here is not just gathering some of that force data But then getting additional postural data of how children are standing how they are orienting their bodies when they do these forces, so In terms of the deficiency like I said It's really a supplement to an already funded effort to try and gather some of this additional Information on how children stand position their body when they're doing these forces. Okay, and what would that tell us that? We don't already know Sorry, maybe more to the point. How is that useful? It's it's useful, especially for the anthropometric data as we're looking at updating some of this older information on children's strength For CPSC's purposes looking at you know for different ages when they're interacting with products What do we expect the greatest, you know pull force on a string would be if it's You know gripping a doorknob, you know or to the point on packaging You know, how do they actually grab things and twist them? Some of it is how where they position themselves when they do it also, okay when we look at the Child strength and human factors data going back. I mean I imagine some of this is fairly dated When we look at the data sets that's that's currently informing our human factors analysis at the agency How much does this data change over time I know that when you look at height and weight data There's all kinds of nutritional factors that that go that you see sort of wider swings and fluctuation But when you're actually looking at child hands strength for instance, is that something that you would expect to see a Large amount of fluctuation over time or is that relatively constant? Like for that specific example like we'd have to get back to you a part of the Original like funding and the and the push was to get this updated because it was older and Is data that is used beyond CPSC, you know, so CPSC? Whether it's using height weight data that's published by CDC Updated regularly or just trying to use the best available data This was an effort of CPSC working with in this case University of Michigan to develop the best available data But I can get back to you on specifics of for the different sources What they are because it does vary depending on what we need. Okay, I see I'm almost out of time But this has been a good discussion. I appreciate y'all being here. Thank you Thank you. I think we are going to go one more round of questions I think there's just a couple of cleanup questions from everyone I have one and then we'll continue on down the dais I just wanted to go back to the IOT and the funding as it's as as a parent Everyone at the dais is interested in IOT something that with Patti Adair's work being spearheading the IOT work the intergovernmental agency work is Is the 75,000 sufficient number to get the lab to the point that it needs to be at to be able to accomplish its mission and Is the three-month period to contract with the NIST folks? Is that sufficient amount of time to inform us as well? I think Where we're at at this point when we were looking at what's a ripe project for mid-year I think we made that assessment that this this would be good. It would help us advance our knowledge of that I think as our team is always capable of coming up with lots of new ideas and broader projects, but I think this is ripe for for where we're at it in the process Thank you, and one last question because there has been quite a bit of discussion about end-cares and it is expensive So in addition to what you've talked about and I notice here in the description of the project it does refer to The approval that we have to obtain to do these is is onerous, and I understand that But it does refer to pilot survey pilot analysis and then full-scale implementation Is there a way to begin this work or to do it without the entire allocation of the 950,000 I think there would be so yeah What we've described would actually get us all the way to collecting some pilots data pilot data So that's what that estimate included if we were to stop short of that and simply have them provide to us a design costing and paperwork PRA package to send to OMB. I think it would come in something something less than that and something quite noticeably less I don't have that handy, but I could get that to you. I would appreciate that. Thank you Commissioner Adler Thank you very much madam chairman. What's a data like I've been meaning to ask that and I forgot If you can tell me in 30 seconds or less Because otherwise Maybe a separate conversation Sure a simple simple explanation of a data lake is it's similar to a data warehouse Except the data that is contained in it tends not to be as formally structured as in a data warehouse and There's some advantages to that. All right. There's some advantages. It's more fluid and more liquid in there Yes, I did want to echo something that Commissioner Bianco said This the the two big proposals the data item one item two I still think I would like to have a separate briefing So I'm just gonna ask that as a courtesy and I would appreciate it if you wouldn't mind doing that I did also want to address the whole the issue of ATVs because This is culminating many many years of work with ATVs And I'll just say that we've been addressing ATV hazards for 30 years and they still persist and We still see far too many horrible injuries far too many deaths and far too many children killed on ATVs I don't know that I'm prepared simply to say let's let's work with the voluntary standards Community on this. I still think that we need to have the possibility of doing a mandatory standard So this to me is probably a bigger project Especially if it's the culmination of work that we need in order to decide whether we move forward Either prodding the voluntary standards sector or moving to a mandatory standard I did want that's just a comment with respect to project number eight the smoke and CO alarm local codes Is this designed to go and find out not just generally what are people doing? But what are people doing when the local code mandates smoke alarm and a CO alarm? Yes, it is as you You may know Different codes throughout the country. I do know different pieces and this is the correlate Our data that we had with where they were and what were the code applying to that and I'm curious about the protocol And I don't want to get too granular But this is one where we tell people in advance that we're coming to interview them about whether they have a smoke alarm or a CO Alarm are we going to say and by the way, we're checking to see whether you're compliance with the local code I don't think our intention was to be the code compliance No But is it to even alert them because to me this I hate to get into anything political But it's like saying are you a citizen when we're doing the census survey You're not likely to get an honest answer if somebody understands that the code requires them to have a smoke alarm or a CO alarm I think the idea on this is more to just assimilate all the codes and correlate the data as opposed to the individual person getting the survey Or the back-end piece. Okay. I well, I certainly appreciate that I wanted to go to the safe sleep initiative and forgive me It first the first question is if we were to abandon Incline sleepers as a project. I'm not suggesting that for a moment But if we were is this project still a critical project Yes, it because it is broader than just that class of products It is the sleep environment and the warning labels of messaging associated with that And I just want to share an impression. It's my impression that we have gazillions of studies Assessing the impact of warning labels on consumer behavior consumer purchase. What possible additional information Could we get from this or am I missing something? Obviously I am missing something, but what is it that I'm missing I Don't think you're missing the additional other than the description staff has in there. I Think those are my questions. Thank you very much Thank you very much commissioner Kay. Thanks madam chair Mr. Hoffman just one more question. We're in our last round. We were talking about fencing and I think by that you were not talking about the offloading of stolen goods. You were talking about Carving out budget Terry funds on a going forward basis one project. I didn't ask about was end-cares and whether if we Fund end-cares Now is there a fenced money in the 2020 operating plan because that does say it's a multi-year effort I would need to work with Steve and evaluate that if if if this item was ultimately improved approved and funded But we haven't prejudiced the next operating plan with fenced funds at this time, okay? So if it sounds like and I'm not I don't want you get ahead of yourself sounds like it You haven't anticipated that this would be funded and put it in the proposal at least that you're working with We're still in the early stages of formulating the operating plan. So that's a separate issue at this point Okay, great. That was it. Thank you madam chair. Thank you very much commissioner Beko just a hypothetical when I add up the things that I would like to see us do and Take out some of the things that I'm not a big proponent of is there any one of these or any particular Project on this list that you would like to have more money for if we took some from one thing because I have to agree with Commissioner Adler that I don't know what else we're going to get from that particular Initiative just wondering and it's a hypothetical if you haven't thought about it. You don't have to answer just wondering And I don't think we do okay. That's it. Thank you. Thank you very much commissioner Feldman Thank you Turning back to item 7 the IOT lab That $75,000 buys us what is this increased staffing and bringing on new FTEs to help service the lab is this Equipment and and and and device acquisition is this sort of rent and physical real estate I'm still a little unclear on What exactly we're purchasing with this 75 grand sure so Ultimately, it's a study but that involves Tech technical advice from NIST and in that and also the purchase procurement of products and Some limited networking piece to put it all together to do the study on on the products Okay, I thought this was a little bit more broad based than just a study But it sounds like it is it sounds like there's yeah, I mean there's there's actual acquisition to get a lab up and running We got a bright we bought got to buy some products We got to put them Network them and then we're gonna do a study on the products, but George can mr. Boyd dr. Boyd Is did you want to jump in just that? the a lot of the funding we're talking about NIST is to an interagency agreement, so It's not us hiring an FTE. It's an interagency group NIST to have some of their expertise come in Okay Mr. Feldman, would you yield for a second if I can follow up on what you're asking I'd be happy to so Thank you. Is this scalable meaning if we move money away from one of these other projects Let's say that a hundred thousand dollars, and it's now one seventy five Could you do a lot more and if so, what would that be? Thank you Commissioner Feldman. I Don't know and I don't know in that when we put it together When we discussed with NIST the idea the scope of what we came up with was the 75k so You know in the discussion of what we wanted to do The answer was 75k Okay, it would be great if there could be follow-up on the scalability if possible, please. Thank you I think as staff proposed it This this was the right thing to get ourselves into this I think if we start looking at broader and scaling we may have issues with the ability to accommodate this Product talking about physical infrastructure impacts within the laboratory that are that are not Anticipated, and I don't think we can react to that at this at this point Commission Bianca did you I? Just I'm still not clear, and I'm glad. I'm glad. Thank you for letting us follow up We're not gonna study whether we need an IOT lab. We've already made that decision, right? This is gonna be what the next step of that like what at the end of this 75,000 Can I go out the 5rp and see no matter how small it is that we're moving forward? You will see These devices in a lab what devices whatever the the IOT connected devices that we procure Okay, so we are gonna look at actual consumer products actual consumer products. That makes me feel better. Yes, and So I think where I'm hedging here is this lab term, and I think it's You know you can take a little piece of a part of a building and call it your lab Or you can create a whole lab and that those are very different costs Structures, that's why I'm a little hedging on that Which is why I'm asking about it because I can't visualize it either. Okay. Thank you. Thank you Peter Reclaiming my time. I have no further questions. Thank you Thank you. Does anyone else have any additional questions? Okay? Having heard no further questions. I again would like to thank the staff for their time and expertise this morning And everyone who had a part I know that you're the face of the mid-year But there are many people behind the scenes who also contributed to the mid-year plan, and I also want to thank them as well I also want to thank our secretary Alberta. Thank you very much for keeping the time and To rock who managed to keep the lights on today and OCM So Having heard no further questions this concludes the public meeting of the United States Consumer Product Safety Commission