 Good afternoon and welcome back. This is now panel two of our Data hearing we'll get right into it, and we have four panelists. We have Linda Potter Policy director for the National Center for the review and prevention of child death Mr. William Wallace from consumers Union. Thank you for joining us again. Mr. Don Mays product safety quality and compliance from Deloitte and Miss Kathleen Wigan also from this morning. Thank you for joining us again Senior vice president legal and regulatory fairs the retailer retail industry leaders association and with that Ms. Potter if you could please begin Thank You mr. Chairman and good afternoon to all of the commissioners I'm Linda Potter as he said policy director at the National Center for the review and prevention of child deaths a data center force that supports child death review Programs in all 50 states the District of Columbia Guam the Department of Defense and the Navajo Nation Thank you for the opportunity to address you today about how child death review data might be useful to CPSC Child what is child death review? It's a multidisciplinary team review of the circumstances of individual child deaths The purpose is to understand how and why the child died So the next death may be prevented there are more than 1200 review teams across the country You've gotten a packet I believe from me and this particular brochure says more about CDR I know that Commissioner Robinson is very familiar with our work, but I don't I'm not aware where the rest of you are familiar I want to say at the outset that We have had a productive relationship with CPSC for many years last year for instance a representative of CPSC attended each of our five regional meetings of State CDR coordinators and spoken attended and presented about CPSC to our To our coordinators and also many local child death review teams have a representative of CPSC who attends the reviews when the Product is at issue. We're very appreciative of this long history This brochure in here in your packet lists outcomes or results that have come about because of child death review And one of them. There's this huge bed CDR one CDR team is very proud of that a child died when trapped in that in the toy bin of that bed and the CPS or the child death review team referred it to CPSC and the bed was recalled the National Center for Review and Prevention of Child Deaths was fun was has been funded since 2002 primarily by Initially and primarily by the health resources and services administration, but also by CDC To be a support data center for child death review programs around the country in 2005 We also developed the web-based child death review case reporting system Which is now used in the 44 states The purpose of the system is to systematically collect Analyze and report comprehensive child death review data on the death of each child and as a result It's a very rich resources about the circumstances of individual deaths The system contains child family supervisor and perpetrator information actions taken to investigate the deaths Services needed or provided by the child or family before the death risk factors by cause of death Recommendations for an actions taken by agencies or others to prevent the next death Relevant to today's hearing the database includes the variable whether death was the result of or the problem or a problem with a consumer product The case reporting system was developed by a 30-person work group from 18 states over a period of two years It launched in 2005 and there have been four subsequent releases The most recent just this month The purposes of the releases is to update the information and to add additional modules About particular types of death for instance the most recent one added was about sudden death in the young Particularly from sudden cardiac death The system is web-based user friendly comprehensive prevention focus stable and secure with the state of the art operating platform managed at Michigan Public Health Institute where my agency is housed which is a Organization of 400 employees and a state-of-the-art system We provide support for all 1800 users across the country who are from both local and state child death review teams Users can enter search print download data and create standardized reports. We also have an extensive training site and Extensive assisted materials this map shows the states that are using considering or moving towards participation Only four states now aren't thinking about it at all But one thing we've learned is that it's always possible. There's change happens quickly and we never rule out participation of any state There are now a hundred and fifty more than a hundred and fifty-four thousand individual Records of individual child deaths that were reviewed by teams and entered into the system Some states migrated old data, but most cases are from 25 2005 forward On this slide you can see the breakdown of the cases entered by age gender race and manner and cause of death This next screenshot is next slide is a screenshot of the questions in the database Concerning consumer products a hard copy of the report tool. This is in your pot is in your packet The trigger question in our database is was death a consequence of a problem with a consumer product? We have seven hundred seven hundred and fifty-four deaths in the system for which the answer to that question was yes If you exclude migrated data, there are five hundred and forty-two cases Those are the ones we use to run the following data Was death related to the sleep environment? Yes and 35 percent of the cases was the product used correctly Yes, 26 percent. No fifty-four percent is a recall in place. Yes five percent. No fifty-one percent Did the product have a safety level label? Yes, 16 percent. No, 8 percent. Was the consumer product safety commission notified? Yes, 30 percent. No, 20 percent The next slide shows the age, race and sex of those 542 deaths related to consumer products, which are for the most part similar to the deaths in the system as a whole The causes of death From consumer products entered into the database are 35 percent from asphyxia Just to give you a few examples, which won't be strange to you sleep aids like the boppy pillow and always of course blinds Motor vehicles fourteen percent. I know you don't cover actual motor vehicle accidents or cars But these include car seats and crosswalk defective crosswalk lights Fire twelve percent the usual heater stoves and defective smoke detectors Poison nine percent button batteries toilet bowl cleaners Prescription drugs drowning six percent flotation devices and other bath devices Faller crush TVs that fall over many many times over Ferris wheels and car windows Other 14 percent of those 28 were medical and 12 were SIDS How has CDR data been used? We have a data dissemination program Researchers apply to use the data and our external data dissemination committee, which is is composed of scientists State CDR coordinators and our federal partners Determines the quality of the proposal and whether our data that the project or Articles have been published through this method and we're always looking for more researchers and more applications One of the brochures brochures in your Packet describes how researchers can apply a number of other articles have been published using CDR data Specifically on cardiac deaths suicide death in the sleep environment and maltreatment deaths in 2011 The injury prevention journal published a supplement devoted entirely to child death review three of the research articles in that issue Utilize CDR data Our data have also been used in three matching studies with respect to cardiac deaths The national violent death reporting system of the CDC and the National Center for Health Statistics, which looked at maltreatment deaths There are a number of possibilities for a data collaboration between Child death review and CPSC We would be very much interested in attempting a matching of our cases in your cases to see if we're getting the same cases although we cannot provide identified data to you or Without permission at the state level because the states own the data We can't provide aggregated data now and we could work with you to facilitate Relationships with the states so that you could get specific data that you required one final note Since 2010 the CDC has been providing us with funding and small amounts of funding to CDR programs in nine pilot states to improve the quality of their data Specifically to reduce the number of missing and unknown responses to the variables The results have been dramatic with just that small infusion of money Their data quality has increased significantly particularly with respect to missing and unknown data Thank you for the opportunity to speak with you today. I look forward to Additional contact between our two agencies. Thank you Miss Potter. Mr. Wallace Thanks On behalf of consumers union the public policy and advocacy arm of consumer reports Thank you for the opportunity to comment on sources of consumer product incident related information That could be used to inform the consumer product safety commission's important work At consumer reports where we cover test and advocate for the safety of many of these products We are keenly aware of just how much variety there is in the types of products and hazards that CPSC reviews Also, consumer products are changing all the time with new product characteristics and entirely new products Continually reaching the market that could pose safety risks It is therefore crucial for the agency to have access to not just a large quantity of data But also to high quality data that provide enough information about the affected individual the product and the incident To enable CPSC to identify whether the product might pose a risk of harm to the consumer Regardless of whether or not the agency is previously familiar with the product and the possible hazard The following comments address our perspective on this consumer product incident information Including one this CPSC data sets that we use and the purposes for which we use them as well as possible ways that CPSC could improve how it obtains processes processes analyzes or shares this information and To the non CPSC incident data of which we are aware that may help the agency in its work To start with the CPSC data sets Consumer consumers union and consumer reports frequently make use of CPSC incident data as a part of our work to analyze product risks assess the effectiveness of voluntary and mandatory standards in In sorry voluntary and mandatory product safety policies including labeling and develop consumer advice and policy recommendations To publicly accessible data sets including nice and safer products. Gov are among the tools we use Nice which is composed of injury data coded from a sample of hospital emergency departments as you know is Particularly useful for identifying new products that pose a risk of harm or documenting longer-term product safety hazards that have not been adequately addressed For example as discussed on the last panel research based on analysis of nice data was instrumental To alerting CPSC in the public about the serious risk to children from ingestion of high-powered magnets Also research based on nice data has demonstrated that the number of children in the emergency room from entanglements on window covering cords Has not significantly reduced over time Indicating that successive voluntary standards for these products have not effectively addressed the hazard While recognizing nice's utility We reiterate a previous request of ours for the public web interface to be upgraded to allow for multiple year queries And for the ability to graph data over time Currently this must be done by hand, which is a time-consuming process We also suggest that the front front page of the nice query builder Should include a few sentences from the excellent CPSC safety guide on the website to explain an understandable language the methods That are used to calculate national estimates from the data provided by the a hundred reporting hospitals Safer products.gov is of course also a very useful tool whose creation and development We strongly supported and which has made consumers medical providers and safety professionals better informed about potential safety hazards in the marketplace As I mentioned at this morning's hearing consumer reports often urges consumers to submit safety related issues the experience to the database and Also when our when our testing indicates a safety risk The database is often a useful tool for us to see if consumers may have already been affected by the potential defect Where additional analysis is warranted we petition CPSC for information from the death certificate file DTHS the in-depth investigations file INDP and the injury or potential injury incident file IPII We use this data to look into individual product categories for which we have ongoing testing projects as well as to analyze the whole set for trends and product hazards When we receive a response to a petition, however, and in fact sometimes when we request data from nice or safer products.gov We're often concerned that the data released to us by CPSC is not entirely complete With information missing or limited due to gaps in the original data sources or Perhaps further limited by what is released by the agency to us a Good first step that we would recommend would be for CPSC to make clear to us Why certain data is missing or limited? However, given the data's significant utility in identifying potential hazards and ways to mitigate risk We believe the agency should eventually find a way to allow public access on its website To INDP and IPII data sets as well as product related fatality statistics We look forward to working with the Commission to determine what needs to be done in order to make this happen We also use CPSC reports on specific topics These are useful, but it'd be helpful if the agency would also release the underlying data analyzed for each individual report To the extent possible As for DTHS INDP and IPII data sets downloadable files with dates of recent updates would be ideal Turning to non CPSC data sets as a trusted resource for comparative information about goods and services consumer reports receives a Significant number of reports from our print and online readers regarding safety issues. They have experienced with a product and Also as discussed at this morning's hearing earlier this year We worked closely with CPSC to establish a system where we could Regularly compile these reports and send them to the agency and as useful and a streamlined a fashion as possible Now we share our raw data with CPSC each month with the caveat that the information is unverified data From anyone who wishes to submit information via the form As a secondary tool consumer reports also receives alerts if a user review posted to our website Contains key safety terms such as fire smoke or hurt among others That way we are informed on a very preliminary basis of potentially unsafe Potentially unsafe consumer product incidents which we can then look into in greater detail In certain circumstances when a consumer wants to inform other readers about the product But does not report the problem directly to us CPSC may be able to establish something similar in partnership with online retailers or may already in fact work with retailers To carry out something like this We also regularly use the valuable aggregate Poisoning data from the annual report of the American Association of Poison Control Centers Recently a APCC data has been particularly useful in identifying the risk to children from exposure to the highly concentrated liquid and single use Detergent packets as well as the poisoning risk from liquid nicotine We appreciate a APCC's diligent work to collect this information as And make clear to policymakers and the public the consequences of household products that have a foreseeable risk of causing illness from poisoning However detailed a APCC data is quite expensive We believe in the interest of identifying hazardous products and prevent protecting individuals particularly children and seniors From poisoning risks CPSC should receive this information free of charge Eventually though and particularly in light of Canada's creation of a public surveillance system for poison information an additional goal for CPSC Should be the creation of such such such a system here The Commission is likely familiar with other data sets that we use to assess product safety risks These include the National Fire Incident reporting system Various City Department of Transportation agency data sets that report bicycle crash statistics and data from the National Health and Nutrition Examination Survey or NHANES Which is helpful to track chemicals of concern such as phthalates and heavy metals in the US population We're also aware of emergency room data sets and to touch on To touch on a distinction dr. Quinlan made earlier some are general and some are specifically related to pediatric care One other type of data that consumer reports uses frequently and which may interest the Commission is survey data We are pleased to know that CPSC is developing a proposed survey to collect data on progress toward its goal of increasing awareness of CPSC and Specifically on how effective the agencies information dissemination efforts are We understand the survey will measure the percentage of the US population that reports awareness of CPSC and the percentage that report acting on a CPSC safety message We agree with the Commission that increased awareness of CPSC and increased consumer action on a safety message Likely have a direct positive relationship to consumer safety. So we believe obtaining these numbers is important We would encourage CPSC on additional efforts it may take to acquire survey data Particularly if it helps improve our joint understanding of how consumers approach key safety issues once they learn about them in Conclusion modern safety regulation is simply not possible without high-quality data on products incidents and injuries We thank the Commission once again for the opportunity to comment today And applaud it on its forward-thinking efforts to ensure its work is guided by the best data you can find Thank You mr. Wallace mr. Mays Good afternoon chairman Kaye commissioners Adler Robinson Burkle more aerobic I appreciate the opportunity to speak to you today about data sources and consumer product related incident information I'm Don Mays. I'm the director of product safety and quality for Deloitte and Touche I've spent many years of my career Mining and analyzing data related to the safety of consumer products My past research uncovered scores of product safety hazards Some that were serious enough to prompt recalls of probably more than dozen products I worked been well-published in technical journals national magazines and white papers and for more than ten years I also served as a public spokesperson Before the media on product safety issues often invoking injury statistics provided by the CPSA Over the years I relied heavily on data stored in the CPSC's databases specifically the nice database and more recently safer products.gov in addition, I've used data managed by National Capital Poison Control Centers the National Fire Incident Reporting System and the National Fire Protection Association But I've also used other data sources for product safety incident information including technical papers mass and social media reports Consumer complaint websites as well as retailer customer review sites The product safety data that I've mined and analyzed have been very effective in influencing the development of various ASTM safety standards as well as influencing public policy Now the the structured data There resides in databases such as the nice database are relatively easy to analyze But unstructured data often in the form of free-form text is much more difficult Unstructured data often lacks the attributes that would help focus the analysis For example the absence of product model names or numbers or even the source of the information creates challenges Now although purists May be tempted to disregard data that may be missing important attributes. I believe that analysis of that information could provide early warning indicators That could help prompt immediate consideration leading to intervention now by intervention I'm referring to anything between an engineering investigation and a full-scale recall Today we have technology that can harness data from disparate sources and help analysts make sense of it You may have heard the term big data analytics It's also called predictive analytics sensing analytics advanced analytics among other monikers the technology involves integrating structured data such as that in nice with unstructured data such as that on Twitter To provide a more comprehensive view of potential emerging product safety issues Through identification through identifying keywords and phrases along with recognizing their relationships Natural language processing methods can be applied to create richer data sets than previously Plus big data analytics can also help process contextual text through what is called sentiment analysis It can discern the difference between a hot product that sells well and a hot product that can burn you furthermore technology that we use allows us to process the data in near real time and Visualize the data making it much easier to follow trends and spot emerging issues the challenge with this technology is to separate the signal from the noise and Our experience of providing similar services to our clients in industry such as automotive Manufacturing food safety supply chain the technology has allowed us to sift out meaningful clues and deliver early warning and Insights to our clients that our clients need in demand Now what was once a labor intensive process of scanning line after line of data entries relegated to voluminous spreadsheets Advanced analytics technology makes the process of picking out the emerging issues much easier Now you can spot the issues much more quickly using an automated process guided by human intelligence There can be many different data sources for analyzing safety issues using this technology whether it from From tweets blogs news articles consumer complaints the method can increase the size of the data sets by Magnitudes over what the CPSC has stored in their databases Plus I believe the consumers are as likely to complain about the safety of a product on social media as they are to write a letter of complaint to the manufacturer or to the CPSC in many cases Social media may serve as a leading indicator of the emerging safety problem Now we caution the commission on setting your requirement for data input on what you've termed as high quality data Setting a standard for acceptable data as having all the details on an alleged incident You may be leaving key information on the cutting room floor Say for example that you receive only two direct from consumer reports of a product safety problem Your staff may elect not to investigate the issue since the priorities are placed on dealing with products that have a higher incident rate But if you pull in any incomplete data as well as data from the unstructured data sources that I previously mentioned You may identify an emerging issue much more quickly Allowing actions to be undertaken to potentially prevent further occurrence of incidents Early detection of emerging issues can potentially save lives prevent injuries and reduce property damage From a manufacturer's standpoint early detection can also reduce the scope of a recall and lessen the potential of a product liability lawsuit In fact If a safety problem is caught early enough it can aviate the need of a recall altogether Now a word about your retailer reporting program I understand that the CPSC is reevaluating the program right now The voluminous amounts of data you receive is simply hard to analyze Or pick out infrequently recurring or emerging issues Here again advanced analytics can help separate Inconsequential issues from serious hazards And assign risk scores that could allow you to focus on the issues that present the greatest level of risk to human health and safety It's hard to pick up a newspaper today without seeing an article about the safety issues that have been plaguing the automotive industry as well as NHTSA Manufacturers and the government alike have been faulted for a lack of robust analytic processes That can provide early detection of safety issues I suggested the CPSC consider incorporating big data analytics into your risk assessment methodology We believe the technology will allow you to include much more data for better decision making and provide early warnings about emerging safety issues Thank you for your time and attention Thank you, mr. Maze and ms. McGuigan. Welcome back. Thank you for joining us twice today Good afternoon chairman commissioners and staff and thank you for the opportunity to represent Riela and its members and appear for you Before you again today to testify About the commission sources of consumer product related incident information that could be used to inform the commission's Hazarder identification identification risk management and regulatory enforcement work Realist testimony today will focus on one data source that has been the topic of much discussion lately including two on this panel A pilot CPSC industry partnership program known as the retailer reporting program or our RP Under this program five retailers and two manufacturer Participants provide the CPSC with real-time product incident information on a weekly or bi-weekly basis Realist testimony today will focus on the value of the RRP reports the need for the agency to Improve the RRP and the fact that the reports submitted by retailers Under the RRP meet the requirements of section 15b reports Retailers are in a unique position to provide valuable information to assist the CPSC In early detection of product risks and emerging hazards Retailers in the normal course of business gather a large amount of information about consumers interaction with the products that they sell In many instances customers were customers report their experiences with products Including experiences that may be a potential safety hazard to the retailer where they purchase the item rather than to the product manufacturer Retailers sources of product information include consumer product reviews Consumers stated reasons for product returns Complaints to the company's consumer product service department saferproducts.gov insurance claims and product liability cases The retailer reporting program was meant to capitalize on the fact That information quickly submitted to the commission from retailers Concerning safety issues with products may be among some of the first reports that the agency would receive about that particular product Reports from retailers are not only valuable because of their timeliness But also because there are incident reports that contain valuable information Currently each rrp submission contains up to 64 pieces of data Related to specific product related incidents The minimum data retailers provide for each product related incident under the rrp includes product description information product manufacturer UPC code item cost Description of the potential product safety issue or incident The date the issue was reported to the retailer Customer contact information is available The retailer's internal case management number For the incident and whether the retailer is the importer and thus considered to be the manufacturer of the product These rrp reports Fill many of the gaps that have been identified earlier in earlier panels today in the nice data By providing detailed product information Manufacturers names and product related injuries or risks of injuries that are not treated by hospital emergency departments This type of data is valuable because it includes both product specific information Rather than an allegation regarding a category of product or type of product and customer contact information Which can be followed up by the cpsc or by the retailer or manufacturer The cpsc can also take the rrp data information and use the data along with other Sources of information that it has including Other rrp reports to conduct a risk analysis to help identify emerging product safety hazards These rrp reports have previously so have previously proven their value As cpsc staff has acknowledged that recalls have occurred as a direct result of incident data received through the retailer reporting program Due to the unique position retailers occupy In the consumer product supply chain the regular frequency with which rrp reports are made And the usefulness of this information contained in the reports Rila and its members believe that the retailer reporting Reporting program already serves or could serve as one of the agency's most valuable tools to detect product risks early The retailer reporting program has operated as a pilot for over 10 years Rila is aware that the cpsc staff is continuing to evaluate The efficacy of the retailer reporting program and will be making a recommendation on whether it should be discontinued Or converted from its current pilot format to a formal program with broadened participation Rila believes that part of this review Should include direct interaction with interested stakeholders including current program participants and the over at least not personally Know over six companies that have pending applications to participate in this program This discussion should include Topics where the cpsc sees strengths and weaknesses within the program and how the report content and transmission process could be improved So that the cpsc can better use the data Rila is prepared to engage with the cpsc directly on this issue on enhancing the current program And is hopeful that the agency will make efforts to improve the program rather than discontinuing the current pilot Or continuing it in a way that lessens the incentives for retailers to provide the reports to the agency Given the value of the data that retailers possess Rila believes that it would be a mistake to do anything other than make improvements to and expand the program One of the fundamental underpinnings to participating in this type of program with the cpsc Was the understanding that the information reported to the agency would have sufficient data In order to qualify as a section 15 report As previously noted rrp participants currently provide up to 64 different data elements for each product related incident A review of the minimum data elements required to be submitted as part of an rrp report Reveals that these elements meet the required information of 16 cfr 11 15 point 13 c To constitute a section 15 b report Thus pursuant to the commission's own regulations the reports retailers have been submitting via the retail reporting program Have contained the necessary information to constitute a sufficient report Accordingly the information must be protected as confidential in the same manner as any other information reported under section 15 b of the consumer product safety act in accordance with section 6 b 5 of the consumer product safety act Rila has learned that these bed rocks of the retail reporting program have recently been called into question It would be a mistake and extremely detrimental to the retailer reporting program if the agency were to contradict not only the arrangements previously made with program participants But also not to honor the agency's own regulations regarding these reports submitted pursuant to section 15 b of the consumer product safety act While the duty of retailers to report substantial product hazards to the agency of course is required by law Much of the data reported to the cpsc through the rrp falls into a gray area Where companies are unsure whether the information indicates that there could be a potential Substantial product hazard requiring a section 15 b report Under the retailer reporting model rather than make a reasonable determination that information may not be required to be reported to the cpsc the information is submitted to the cpsc pursuant to the Quote when in doubt report and quote guidance often repeated by cpsc staff By no longer considering these reports submitted To section 15 b of the consumer product safety act the agency would be removing a strong incentive for early reporting And punishing retailers for following the when in doubt report guidance Rila believes that the agency should provide clarification whether the retailer reporting Program reports are currently considered to be submitted pursuant to section 15 b of the consumer product safety act If the cpsc believes changes need to be made to the underlying reports in order to achieve section 15 b status Or to make the data more useful to the cpsc Then rila and its members are open and willing to help collaborate on what those changes might be In conclusion rila's members and the cpsc share the common goals of ensuring the safety of products sold in the united states Early detection of product hazards and the speedy removal of unsafe and non-compliant products from the marketplace and consumers homes It is in both the cpsc's And retailers interests to have a retailer reporting program that provides valuable easily usable and timely data to the agency And in doing so provides a strong incentive for retailers to report product safety information That falls into the reportably reportability gray zone area Thank you again for the opportunity to provide comments on this important cpsc source Rila shares the commission's commitment to improving consumer safety and looks forward to its collaborative relationship with this agency on this issue and others Thank you. Thank you. Ms. McGuigan. Thank you to the panelists for your oral testimony We're now going to turn to five minutes questions per commissioner and miss. McGuigan. I'm going to begin with you again Uh, I think you mentioned seven participants currently in the retailer reporting and six or so that have pending applications Is that correct that is correct and for each of those I think you also mentioned 64 different data elements At least certainly for the seven that are participating. Is that correct? That is the template that is the maximum template that I am aware of. Yes, and so Why I'm curious in the user fee context or in the E filing context and we're talking about data here in the accessibility of data Why is it easy in the retailer reporting? But it's not easy to transfer one six of the data in terms of 10 day elements in one context But in this context Transferring that data the agency 64 data elements is not an issue and there's actually more clamoring to do that That makes sense I'm not sure I understand your so you've got you've got a line of retailers as I understand it eager to submit Up to 64 different data elements electronically to the agency and the retailer reporting process In the e filing Side of things the agency is asking for 10 data elements Which is which is less than 160 amount that we're asking for or that we've agreed to accept as part of retailer reporting What's the difference? Why in the retailer reporting context? Is it so easy for the retailers to submit this information? But in the E filing electronic filing context one sixth of that information or one sixth of the amount of data elements would be challenging Does that help? Yes, thank you I think you're comparing apples and oranges. Okay So in one context it's information regarding product related Incidents from consumers which is a very narrow scope Of products that are reported and I don't have the numbers in front of me regarding specific retailer reports How many line items that they report either weekly or bi-weekly? I'm happy to survey at least my members that are participants. They're not all real members But I'm happy to try to survey that or certainly your staff has that information. So that's on one on one hand the other Issue is as we've talked about Not only would a retailer be required on on for the e filing process not only would a retailer be required to provide these 10 or more data elements It would be required to file it for each line item for each Entry that it makes and as we indicated in our public testimony and in our comments In the survey that we did of our members some of our members do Up to several thousands or more shipments per year and may have up to A thousand or more line items per entry. So that would be all of those Data elements for each line item and as a as you and I have talked about before It's also tying up the specific certificate To that specific item that's on the Entry customs entry. So it's much more complicated Than just providing simple data because there is no flow on the back end That would be able to tie those two pieces together for Entry information and is there anything about the it infrastructure that exists at least for the seven participants that would facilitate Since there's already a portal to cpsc Any way that would and there's already an infrastructure at those agent at those companies to report to the cpsc for those particular participants Is there some Savings or efficiencies that they can enjoy if we were to require a data element filing? Um, that's a great question and i'm not sure I have a complete answer for you. Um I can tell you that the reports that are filed are often handled by two different departments Because one is gathering to be able to inform The report that's submitted to the cpsc. They gather information internally. So that's monitoring your social media It's monitoring consumer reports, which may be online. It's talking to your customer Service group. It's talking with your merchants. Um, so that's a different Area where they have a source of information where they collect that It would not be at all the same kind of process Of data that they would need to collect for the e filing certificates Okay, thank you for that and we talked earlier about resources and retailer expertise What expertise does rila rely on for the judgment that this is valuable consumer safety information in terms of injury and death prevention? And the reason I ask that is that from our perspective We are guided by our experts our epidemiologists our statisticians and from their perspective This is a useful set of data, but it's not the most useful and it has limited value Relative to other data sources and so that says a lot to us We're getting a very different opinion from rila on the value of this data And so are there epidemiologist statisticians who are trained in injury prevention? What are you relying on that you provide a different judgment on the value of this than we have? Well, I can tell you that rila does not have an epidemiologist on staff Unfortunately, you have me and i'm an attorney so But what I can say is that if you look at the if you just Listening to the panel that appeared before us today A huge complaint about the nice data is that there is no product specific information This data that through the retailer reporting program is absolutely product specific information And you have consumer information and it includes timely reports because they're You know returns or customers views on products are made in a much more timely fashion. It's not necessarily Just Trips to the emergency room and it also includes things like that would cover I had to go to an urgent care because of x you may not collect that in your nice data Well, we may have that because it was reported to the retailer Great, thank you. And as an attorney I can appreciate there's a limitation on how much we can speak on which is why I rely on our epidemiologists and statisticians commissioner adler Thank you very much Ms. Potter, I just wanted to ask one quick question It sounds like you gather a very large amount of data and it's extremely useful And I'm sure we would find it extremely useful just a Random question you you do not make national statistically representative estimates of the information that comes into am I correct? No, we can't it's not a federally funded program. I'll just try to answer briefly The states vary one from the other so dramatically that we couldn't draw national statistics from it There are about seven states that review every death But the rest of the states range in what they review and there's one state For instance, that only reviews child abuse and maltreatment and neglect So we can't draw national statistics from okay I was just curious because listening to mr. Mays talk about the fudge factor and not being overly pristine Have you ever looked at the data that you got get in and tried to look at a data from national center for health statistics? Or even look at nice data and see is there any even impression that you can draw about The representativeness of the death data that you're getting or is that asking too much? No, it's not asking too much except that yet one more attorney could plead ignorance about the data But we do have Have tried with the actually this national center for health statistics to use our data along with their data to see if For instance, they're using the right language on the death certificate to identify the particular In this case maltreatment deaths And The cdc and this may be a little off what you're asking But the cdc it's two projects that it funds for us is trying to trying to create Surveillance with our data by funding their pilot states to To review all the deaths of a particular type in this case sudden unexpected infant death and sudden death in the Particularly cardiac and epilepsy So there there are ways to move towards surveillance, but we're not actually true To true surveillance yet. Thank you very much and mrs. McGeegan. I'm sorry to keep Pestering you but you and commissioner moho rovick and I were on a panel talking about retail reporting And I just wanted to reiterate some of the thoughts that I that I or some of the facts that I'd had and First of all, we get roughly 30,000 reports a year and according to staff for every 10,000 that we get That's $50,000 in expense to the cpsc. So on an annual basis That's costing us roughly 150,000 dollars, which let's be clear would be chump changed almost any other agency But it's not to us It's big bucks and we're always trying to decide given Available resources and needs where should we spend the marginal dollar and I will just reiterate that I'm still Fairly undecided on the issue. It's I think you also know it's a source of Great discussion and debate within the agency about the usefulness of the data um, I can confirm at least what Chairman k's been told and that is if you actually sit down and you try to be utterly cold-blooded about and say tell me the Number of recalls that have stemmed directly from Retail or reporting it turns out to be a very small number But that doesn't necessarily answer the question because sometimes it's confirming data Sometimes it's alerting data so that we look more deeply So part of that is a judgment call within the commission But I guess the other question well the other thought is We're right now getting what we call 500 to 600 15 b reports suddenly moving to 30,000 Is a big jump and i'm not sure any of us is prepared to make that jump Although I would be the first to say that some of the retail reports that come in are Probably would be considered 15 b reports But my question to you is picking up on what mr. Mays was saying Have you ever sat down and tried to do a big data analytic on the data that you're seeing And I'd love to hear mr. Mays if he has any further refinement about that that concept I have not because I don't see the data that our members are reporting to you Through the retailer reporting program. I do think it's I mean it's I do think it's a great topic of discussion And I think it's appropriate for current participants And pending stakeholders along with other stakeholders to have this discussion If there is product specific information that is Potentially available currently is available is being given to the cpse. How could that be made more useful? What tools perhaps could the cpse have could it be in a different format? Are all those data elements needed should they be narrowed? What specifically do you need to help identify Product risk emerging product risk mr. Mays. Do you have any further comment about retail reporting and big data analytics? Reducing the data down to the meaningful data that's important to the cpse to help prevent injury and death Is the thing that I think that you'd really want to do And and when you look at you know, I look back at When we did a first year analysis of saferproducts.gov and we published this in a white paper There's a lot of information in saferproducts.gov that probably doesn't rise to the level of Even my concern and I've got a pretty low threshold And and once you start sorting out The inconsequential information, you know those those reports You're left with far far more rich data in which you can act and that's where I think that in the retailer reporting program You're going to wind up throwing away things like You know people complaining about diaper rash And be more concerned about the people that are complaining about you know broken bones and lacerations not the diaper rash Isn't important. Thank you very much My time has expired. Thank you commissioner out there commissioner robinson. Thank you, mr Chair, um first of all, I'd like to thank miss potter and mr. Wallace for working with my staff this year for our Small but very important changes that we were able to you guys were able to affect in terms of reporting information to us And I know that tom schrader was sitting back there worked with terry covington Miss potter years ago trying to figure out how to get your information But um in a brainstorming session She came up with the hyperlink so that we at least are getting some case specific information Now out of and not just the aggregate information and mr. Wallace I know that you are now reporting what gets reported to you to us and I thank you for that I hope I have time to come back and ask you some questions, but I'm going to start with you mr. Maze um I was just talking to dr. Quinlan during a break and he was talking about the data fields and narratives and so forth Would you think that the narrative section of electronic medical records? Could could could be classified as unstructured data? Yes Absolutely And I was hoping you would say that and so if but but when you talk about the predictive enough when you're doing a predictive analysis Using integrated data sets for structured and unstructured data It frankly sounds to this little agency like it means analyze and analyzing you and utilizing Exponentially more data and I'm just wondering if you could speak a little bit to the resources that would be required for doing That type of analysis in terms of labor in-house contract out. Are we talking millions of dollars or how does one go about effecting that? um Think of it as an automated process an information technology system that can help you Reduce the the amount of time that would normally take to go through that level that amount of data um The cost of that is it all depends upon the amount of data and the depth of the analysis And the reporting out function but million dollars probably not um It would probably be significantly less than that if we were look at let's say the data with the In the retailer report reporting program the 30,000 data entries that you get Uh, it would probably be significantly less than that and where would the where would the integrated data set? Would it be in a cloud or would it be at the cpsc or where would it sit? um Norm normally at least in the class that we've worked with they generally don't like it in the cloud because they're worried about security issues And it's like all of my personal information Exactly Uh, so most people have it in in their own uh in their own servers protected And has industry so far developed any sort of best practices in terms of Examining the accuracy of the unstructured data I'm not sure that there's been you know, the technology is relatively new And so the effectiveness I'd say is still being proven to a certain extent As far as the accuracy you can't look at unstructured data as being perfectly accurate What you can look at is once you you marry that with structured data It can provide the indicators that people need to be able to make better decisions And can can you tell me any more about how using big data analytics works with using Statistically significant data like our nice system Um, maybe you could walk through an example Well, you're actually you'd actually be pulling information from nice Which is the structured data with unstructured data pulling it together and trying to make sense of it Giving you a richer data set and being able to provide perhaps early warning of emerging issues Where you might not have enough data in your nice data set to be able to do that As far as a statistically valid Answer at the very end it really all depends upon the final quality of the data I don't think you'd necessarily want to be You know from a scientific standpoint anyway I think that I would be uncomfortable publishing rigorous statistics associated with it But certainly it can spot trends very very easily and and by by using techniques In common systems like Tableau you can display the information in a way that it makes it so easy to visualize and pick out You know those those blips on the curve that all of a sudden you haven't seen any reports and something pops up You can spot it immediately and is just quickly Is there any other place that you think of other than I mean you've talked about retail reporting and unstructured data And we talked a little bit about the narratives in the electronic medical records But is there any other place you think of that would be a source for the product Product injury related data that we would be looking for So well the sort the sources can be multiple. I mean, you know, depending on I guess they could be facebook or twitter or whatever Exactly. That's exactly right. And and as I made as the comment I made in my testimony You know, sometimes you might find that the The emerging issues are first disclosed in twitter before they get to the cpsc or to the manufacturer Right. Okay. Thank you. Um, miss potter. Let me just follow up quickly with you I've only got a moment But I know that this aggregate data that we get is is of some importance But I've been concerned since I first realized what a lag time there is on our death data Here at the agency about finding ways of making it More current Um, and you obviously have that information, but we don't have access to the case specific information My my I am told by miss covington that in order for us to get case specific information We would need to go to each state and get permission from them. Is that right? That is right. They own the data But as I said earlier, we're willing to help facilitate that conversation. We have relationships with all the states I think that's a wonderful idea and I and I appreciate that and is there any way I know this sounds like a beggar, but is there any way that that work could be included under your grant? Uh, I believe so. Yes, I do. Let's talk, but if you've got any money, that would also be great. Okay. Thank you very much I think we're just going to stop at the yes. Thank you commissioner burke Thank you very much and thanks again to our panelists for being here and for again all of the valuable information You've shared with us I'll start with miss potter. I just have a couple of questions really clarification On the one chart you showed the deaths related to consumer products The breakdown you kind of went through those quickly and you don't have to do it now I can come to you afterwards, but I was just wondering if you could provide us with When you say other 14 fall crush 5% those specific percentages of of deaths if you could Provide us with the breakdown of those other types of within the categories. Absolutely. I'd be glad to provide that Thank you. And then on this other Slide that you showed the deaths related to consumer products Product used now incorrectly it says 45 percent here, but I think on the When you showed the slide it was 54 percent. I don't know I And again, we can clarify that afterwards, but those two things would be really helpful Okay, I'll clarify which is 54 or 45 one could be a typo and I'll figure out which one it was I'm sure one is interesting. They were both a five and a four just right good. Thank you so much. Thank you for being here Mr. Wallace. I just had a couple of Questions for you with regards to your testimony. Thank you very much It says we encourage cpsc to continue its efforts to make safer products that gov is up to date and consumer friendly as possible The second point was to minimize inconsistencies between recall information And what's posted on our gov on our cpsc.gov website? Can you give an example of that? Is that where that's happened? And you don't you know, if you know if you know it right off the top of your head, that's great I don't have an Specific example with me right off the top of my head, but our our consumer safety and sustainability group really frequently relies on Both of those data sets and they've they have noticed inconsistencies Okay, it would be helpful for us to understand what those are and what What constitutes inconsistency in terms of what's on our website and gets more specific information. That would be helpful Thank you so much and then This morning when you testified and you were here talking about priorities Again, again, you brought it up here. I just want to clarify and make sure it's you were talking about the same thing So it says you said this past year we established a system to share broad Data reports with the caveat. It's unverified. Is that what you were referring to this morning? Yes, okay. All right, and so then you've just set up a mechanism with our agency to share what kind of complaints you that's correct Okay, good. Thank you very much for your help And now mr. Maze You've gotten a little bit of flavor about our retailer reporting program And I don't know if it's something you were familiar with before you got here. Yes, okay So you talked to us about the Data analytics or advanced analytics Is that Is that the kind of analysis you can look at this data? And and what do you do you set up parameters or rules with regards to what trigger words or how would that work? trigger not just words, but trigger words and phrases And being able to draw relations between them So, you know, I use the example of a hot product You know If the hot product is associated with a word that means that it's selling well Well, that means one thing But if it means if it's related to something that is burning you it's a whole different thing So how does it relate to other words within that same? Um textual data set Thank you very much. And I look forward to future conversations with you about that issue Um, I do want to just point out a couple of things with regards to retailer reporting Because one was raised by my Colleague and that has to do with the number of recalls that are the result of retailer reporting And I think we haven't really done a good job at measuring those recalls It's not that To the best of our knowledge because we haven't really looked at them the number seems low But we're not really sure if that's actually the number of recalls that's Resulted from retailer reporting. So Ms. McQuinnig if you want to comment on that Yeah, and I think when we talked at our panel at Iqviso, there was anecdotal You had one retailer that stood up and indicated that he that for that particular retailer they had had over 10 Recalls within the last year that they had done. So You know, again, I think there has not been a comprehensive review But I think what part of it is that a retailer may or may not know um And or certainly may not be the initiator of the recall So if a one retailer is has one incident and another retailer has another incident and then other information is coming into the cpsc From other data sources the manufacturer may be engaging in the recall And it's not the retailer engaging in the recall So there are a variety of ways in which that recall can happen, but to my knowledge there has been no comprehensive Um review or analysis of You know the direct correlation between number of recalls and the information reported. Thank you. Um, and just I I see oops I can't even share my last thought with you Thank you. Stay tuned. Thank you commissioner burkle commissioner mohovo Thank you, mr. Chairman I'd like to thank the entire panel and make just one brief comment as it's my intention to yield as much time as possible to commissioner robinson But on retailer reporting in particular I I hope we don't stifle our thinking on retailer reporting in terms of what the program is Right now and how it exists to me. It's one that has fantastic and great potential But it's my firm opinion that unless we open up discussion of retailer reporting to the experts like we solicited interest and opinions from today We won't be able to fully yield that that potential As we're not we're not investigating and exploring it in a very open fashion and I hope we do that I implore the commission to to move us in that direction Yes, we would concur I would just like to note that the the program has been a pilot for 10 years that technology has Transformed significantly over 10 years initially when members were First participating they were sending the information in an excel spreadsheets clearly not a usable format That has changed slightly because there is a portal now in which members provide that information But it has the opportunity to again Change dramatically again with the idea of potentially using Data analysis to cull together information To mine this data to use for your hazard risk analysis And we would welcome again the opportunity to participate in those conversations. Thank you. Ms. McGuigan. I hope you have the opportunity to do so Finally before I yield also to commissioner burkle with apologies Mr. Mays, it's a pleasure to see you are a giant in the consumer product safety community It's a pleasure to know you it's uh Important to see you here today and I look forward to always being able to call on you for advice So thank you for your participation Mr. Chairman I'd like to yield quickly to commissioner burkle just that last thought I do want to say I think and I want to concur with my colleague that The potential with retailer reporting is is vast and making sure we we get the right fields and we are Much of what we talked about in the first panel Those those the information we receive is limited. You can only receive so many death certificates We're getting them all but in this instance. I think there's a wealth of information that should be available to us Thank you. Thank you. Thank you commissioner burkle and in our rov rulemaking We often say that only one Rollover life that is saved through that standard would justify all the costs if we can yield the the vast potential Retailer reporting to be able to surveil for emerging hazards It would certainly um, it would be one of the most beneficial investments in our time and efforts Without a yield to commissioner robinson My suggestion, thank you very much But my suggestion is we move to the next panel just because we have some terrific testimony come Great. Thank you commissioner burkle rovick and commissioner robinson and thank you to the panel I'm just missing McGuigan quickly if I can amend my request from this morning regarding the information on the Cost of data elements if you can do a comparison Please relevant to our discussion We just had earlier as to why it's not a burden on the 64 data elements, but it is on the 10 That would be very helpful. Thank you again to the panelists We're going to now take a two minute break and resume with panel three. Thank you Good afternoon. We're back for our final panel on our afternoon data hearing and thank you to these panelists We've got abhi bender chief technology officer for the u.s. Census Bureau Mr. Tony summerlin senior strategic advisor to the federal communications commission cio We have dr. Philip born associate director of data science at the national institutes of health And joining us again. Thank you. Mr. Dennis mr. Dean tennis and this time on behalf of the northeastern ohio fire prevention association as I mentioned in the beginning Unfortunately, we do have a bit of a hard stop at five o'clock. I completely apologize. I promise you We've read all of your written statements very very closely And so if there's any way that we can move through in a more quick fashion with the oral testimony We could turn to questions. That would be fantastic. Thank you. Mr. Bender Thanks very much. I really appreciate being here. I'm abhi bender the chief technology officer of the u.s. Census Bureau I came here about several hours ago to listen to the other testimonies because I just wanted to get a better understanding of the The nature and scope of the issues that you're dealing with and many of these things sound very familiar to home and Within five minutes sort of a speed dating kind of effort here I will try to get through this as quickly as possible Just a bit of a background the u.s. Census Bureau is the largest statistical organization in the federal government And serves as a leading source of quality data of our nation's people and economy I emphasize quality data I view with respect to open data that the Census Bureau information Provides this foundational socioeconomic demographic layer upon which all other data are matched up and connected to derive insights What do we do we collect we secure we process and then ultimately we disseminate open data information Um, we are a big data organization and have been since 1790 when we conducted the very first So that we're the quintessential big data organization Data is not something that we do as part of a job Data is really within the fabric of what we do for a living And for that reason we have built Governance processes we have talented individuals methods and research capabilities around big data In addition to the decennial data that happens every 10 years We also do the census of governments the u.s. Economic Census A sampling of 250,000 individuals every month through the american community survey very valuable because it provides more granular data We do the current population survey Many federal state and local organization use our data for land use planning demographic understanding demographic characteristics of their communities Uh in planning and transportation systems and roadways Our data are also being used by companies entrepreneurs We're looking to build businesses and need timely access to information to understand the communities in which they want to operate So it's up to us to really find ways to do a better job of disseminating this information We emphasize data quality Data integrity the format of the data the content. These are all important for our consumers Just recently we initiated something called the national day of civic hacking which you may have heard about It's a national event where many communities participate in actually leveraging federal data to build solutions to serve communities And we met with some of these developers and what we heard from them was when we asked them What can we do in terms of a better job of disseminating our data? They said to us You know as small businesses and entrepreneurs We really need to focus on what we know best and that's the business that we're trying to build We cannot spend time translating the data that comes out from the census bureau So we want your data to be much more easier to consume Ultimately any individuals be at a federal agency state Entrepreneur they're looking to search discover consume Analyze derive insights from the data and make decisions from that And it's sort of critical for us to continue to engage with customers around our data For the bureau and I would say for other federal agencies Including potentially this commission For us to be relevant going forward. I think it's really important to understand what business we're in Are we in the business of building infrastructures for data dissemination? Are we in the business of building predictive analytic tools? I think we really need to understand Are we data producers consumers? Aggregators data scientists regulators technologists. What is really our collective role? And I think if we fully understand the ecosystem in which we operate Especially in this interconnected global economy, I think we will have a much better capability To support our mission By staying close to our knitting and understanding what in fact can be outsourced And what in fact can be leveraged in-house So the the problems that I've heard today appear to be insurmountable But these problems can be addressed in fact through coalitions collaborations working with other federal agencies And and having some standards Um data management for us is not an afterthought as I mentioned before data management for us is really a fabric It's part of our dna It's how to go to we it's how we go to market. In fact, we think ourselves as a big data product company We have a distribution channel of data scientists out there and data dissemination specialists We have scientists who are working with this data to create Special products for communities, but at the same time We need to democratize our data. We need to make sure that we publish While retaining privacy and security, we need to publish our data at a certain roller level So that we we do not pre tabulate this information and add complexity So I think thinking through dissemination As a strategy is really very important and every organization Whether they know it or not are a data-centric organization and they're in the data business I mean, that's how you make your decisions. So I think having a strategy in place Understanding the ecosystem in which you operate is really very critical And lastly in my last minute I heard some interesting discussions about structured data unstructured data natural language query I would remind, you know, many of you that you would imagine many of you are lawyers by training And all of that innovation started in the legal industry I mean, how do you do discovery, right? It all started back in the late 70s 80s and early 90s So many of the tools that we speak today about big data actually had their origins in the legal industry And now through rebranding and remarketing these tools are now being sold as With a different umbrella under big data. So, you know, next 10 years we'll have a different terminology to describe that The commissioner Berkel asked something earlier about how do you connect systems? The way you connect systems today are through something called apis application programming interfaces These are merely software tools that allow you to communicate between different technologies And we're moving to an era now where organizations can become curators of apis and by through this curation You have access to other data sets, which will allow you to do mashups of this data information There's still issues with metadata standards and so forth But the bureau has successfully launched recently what we call a city SDK And that's a software development kit It simply takes our api which channels a lot of our data out there And makes it easier for developers and others to consume this information by removing the intermediaries who are the subject matter experts Who typically you need to call upon for the explanation Of the metadata that someone had to mention about having the subject matter expertise No magic bullets out there, but there are ways to use technology in order to improve communication among federal agencies states and localities And then The last thing I would like to add is a recommendation I've been very impressed in listening to to this group In terms of the quality and the caliber of people that you invited to speak here today We truly see at the u.s census bureau that innovation Doesn't happen because of smart technical people innovation occurs because of diversity diversity of thinking And diversity of skills So having data scientist economist health professionals technologies in a room Will help us get through these insurmountable issues And lastly, I would say that in this meeting you can just say so much, but I believe that in this meeting It's a conversation. It's the beginning of a very important conversation and I'll stop at that Thank you so much. Mr. Bender and thank you very much for accommodating the need to try to keep it as short as possible Mr. Summerlin Although I've known avi for over 20 years your microphone, please sir. Thank you Although I've known avi for over 20 years. We come from a different place I spent eight years at lmb and It was our job to make things difficult We uh We did everything we could with oh ira and pra and making sure people couldn't collect data So as a reward for that I got to go to work at the FCC a regulatory agency that Obviously tries to make things difficult for industry and as probably not as In a tough spot as you all are but sometimes So our data is based on the fact that we have 1700 employees We collect the same thing numerous times because we don't know exactly where it is What they used it for so we go back and collect it again Having worked at omb People don't like to give you the same information repeatedly and it really aggravates them Luckily, we just have industry as opposed to citizens. I think if they were citizens, it would be even harder But with 1700 people 700 of them being lawyers no offense to anyone Lawyers don't like to share anything So their data stays on their PCs until I pull it off They share nothing I did a little video of all their offices They have a mandatory cleanup every six months just so they can walk in their offices We have 300 engineers. They're not much better And 200 economists And the new thing of course is data scientists, which to me look like statisticians, but It's always good to rename things so we have We found 104 independent stove pipes of data because we're application-centric rather than data-centric and caused A lot of issues and our latest collection of information Apart from the 4 million nasty comments on net neutrality was we did a special access Requirement which is Exorbitant over 1600 data elements for people like Verizon and AT&T to love us for and What we were going to do with those was Highly in question. So we got to build a real database for it for the first time But independently each one of those data elements would have gone to someone's pc And they would have done whatever they could with a stat pack or some other tool And then the results would end up gosh knows where I went to visit obby who built a much nicer database but As I asked tony scott on a panel a couple weeks ago. I said well It was on big data and I said actually it was last week and I said tony scott's the new cio for the u.s. government Who I've only known about 20 years also so it shows how old I am but I said what's big data? He said I have no idea I mean, I Really have no idea what people are talking about and To confirm to people the FCC that we have nothing that looks like big data I brought jeff joneson who built noro non-obvious relationships software for nsa And they said well, we have a lot of data. You don't understand our problem. He said I'm going to try not to laugh at all of you, but seriously We look at more data in an hour than you have any year So it's it is what it is to whoever is using it And whatever the purpose is my caution always is know why you're collecting it know how you're using it And if you have a predetermined outcome like you're going to tell me that 80% of the arrests are minorities. I don't need data. I mean it it's just don't go get data just to prove your point because it's useless Um, but as far as I'm concerned big data has to do with its velocity It's complexity its relevance And how sticky it is not just its size So When you collect it know why know where you're putting it. Hopefully you can get it back at some point And that's about all I have to say Great. Thank you. Mr. Summerlin. Dr. Born Yes, good afternoon everyone. I guess I'm here representing the 27 institutes and centers of the NIH So you could call those 27 silos or cylinders of excellence whatever you want to call them Where of the view that data could be a catalyst in principle to break down some of those silos And I have a trans-nih role trying to make some of that happen But I just say it's sort of a leading into that as we Begun to analyze this I would say I would say a few things about data in general There is a promise to big data in the way we've just heard But I I would also say we don't know much about the little data we've already got We don't know exactly the usage patterns of that data why it's used that way what innovation comes from it in a really measurable way So I think one of the things we're trying to do now is to get a better handle on those things So that that will better inform what we do with big data However, you cut it good data is expensive There's no getting away from that if you need to retroactively look at data to make it good data It's really expensive to go backwards from our point of view and I'll say just a word about this in a second but That good data begats trust in the community And the community for us is God essentially We need to satisfy those communities and Good data does that so it's high it's high on our thinking The problem is that what we're doing currently good. We're good or bad data is not sustainable If you go out into the community you ask an average academic professor What's what's his or her business model for data? It's essentially to ask the federal government for more money This doesn't scale the way we're going So we've been thinking about these problems and I'll just say that we have a sort of three-legged approach to it A stool with three legs One leg is this community I think a lot of what we do has to be driven by the community with data So their views on standards their views on what's important Must be what drives us forward There's also policies. There's a second leg of the stool And those policies as we see them are both top down and bottom up The top down ones are the ones that come by mandate from the federal government The ones that come bottom up are the ones that the communities themselves believe that we should put in place To facilitate the biomedical research that they do And they sort of meet in the middle And then the third part is the infrastructures we put in place to support this enterprise And we're looking at it as an enterprise and we're looking at it as an ecosystem where Whatever we produce gets fed back into the system And is reused as much as possible To do this we're adopting an approach that's not dissimilar to what a number of other folks are doing Which is really to create the notion of a commons So we're taking what we call in our world Research objects which could be data software narrative Papers whatever it is in the research life cycle. We're putting them into this shared environment Where we identify Each of those research objects with an identifier We're developing tools and processes to essentially be in that in what we call being fair Which is to find access interoperate and reuse that content And we're in the early stages of this and we're running pilots in an agile way to evaluate how well this is working So we're certainly very interested in what other agencies are doing in this regard and How we might certainly learn from them and hopefully impart something ourselves. So i'll stop there Thank you so much dr. Boren and and mr. Dennis Welcome back and again in the interest of time if it's possible, please try to condense it as short as possible That would be fantastic. Thank you. Clearly. Well and once again, thank you very much Um, I want to dedicate this I have a slide Hopefully it shows to uh every on a dale and new york little girl who we all think died needlessly because of the failure bianization smoke alarms I'm here representing the neo fpa. We as you mentioned northeastern. Ohio fire prevention associations It's 200 gentlemen that are fire officials that are dedicated the training and education for other fire officials and they Want to support scientifically based code standards And they are very concerned that the data needed is not being about on this edge of smoke alarms is not being captured um They actually went out and had two buildings that were houses that were going to be torn down over the last couple years period ran their own data testing set up smoldering test and their tests Actually replicated a lot of the information you found this that I shared this morning Significantly did response delayed response of ionization smoke lards and not triggering And the ukulele test had done in january and february There's over 20 minutes time difference in the mayfield village test Surprisingly in the three separate tests and If you look aggressively um And only one test in a ionization alarm even trigger after an hour Smoke conditions were so thick People in toxic the families would have died in that so What nist has given them in their statement from the boss to the boss of city council that they've known not to fail They've known to fail And smoldering fires and completely not sound at all was validated by this group's testing um the reason That I'm bringing this up is not only our injuries and deaths not being captured in any of the data But the significantly day delayed performance of ionization alarms are not being captured to the consumer product safety commission um Another independent fire department barrio vermont upon the death of four children and a mother happened to be relatives of the fire chief of barrio vermont They all died None of the ionization alarms all hardwired went off when they went into day after the fire testing them. They all worked now In 2008, I think it's important to bring up a case. It was called hackett versus first alert this case was appealed all the way to the Second circuit court in the court of appeals And the judge it resided over that ruled in favor of the The plaintiffs and ruled that quote that the smoke detectives failure was a legal cause of deaths of william and christine Hackett reason i'm bringing this up Um at this meeting there was 355 complaints filed. They had to get they had to be deposed Is there a way for the cps to obtain these real world complaints? Um do the manufacturers have to self report this is valuable information that is not being being captured we know from enfers reporting that More than 60 of people that purchased a smoke alarm yet died in a residential fire That smoke alarm was working. It was found to be an effective working smoke alarm But yet 62 percent of the people died. This is a statistic That's annually changes maybe by 1 percent 2 percent But the majority of people that die in house fires Their smoke detectives work If they had them I mean there are significant people that did not have them But why are so many people dying with working smoke alarms? We certainly believe it's the technology In that it's not being captured by ionization technology Lastly one of the biggest problems is the Enfers reporting system I If um, let me see I have this slide. I think it's important to go over this one If you look at this reporting, there's nothing that mentions the types of fire Or the type of technology involved a matter of fact when you get into l4 it talks about detector operation If it operated you go to block l5 If it failed to operate which we would take an ionization. This is the problem you go to l6 and They the reasons that the person has to check is power failure improper installation defective battery missing Discharged or undetermined So there is nothing there that captures anything about technology So as I stated earlier this morning, we have anywhere between 400 to 1,000 people dying Needlessly because of ionization detector failure and we're not capturing it in our data system At on any level Thank you very much Thank you so much. Mr. Dennis and you focused on the one slide in the beginning that I got stuck on in your presentation Which was the a brianna dale story in the fact that there was no No fire damage Yes, she still passed away from the smoke and so that was that was certainly very eye-opening We're going to turn to the questions from the commissioners I'm actually going to yield back my time in the interest of making sure that my fellow colleagues have a chance to ask So commissioner robinson your five minutes are up. Thank you, mr. Chair if I if I could what was since we only have four minutes Are are we indeed stopping at five since commissioner adler's left because if we are Uh now that he's run out of here. I think that we can probably go a little bit longer I'd like to try to accommodate the five minutes for each of us. Okay. Thank you Um, thank you so much for coming in today. I really really appreciate it. I'm sorry. We're running out of time Um, but let me let me ask a question of you three agency fellows who've got census fcc NIH I would really love if you could succinctly tell tell us How your information department or data departments organized you have a cio a cto a cdo How do they interact and um, how do they work with the end users within the agency? I know you could each talk for three hours in response to that But we're just rethinking ours here because we only have one and we have that vacancy that we're filling But um, I'd like to know how yours work. Mr. Bender So we have the director of the u.s. Census Bureau the deputy then we have the cio the cto and a number of other individuals Data is dealt at the highest level of the organization through a data steering committee All decisions are made at the executive level of the u.s. Census Bureau And these decisions include the use of administrative records MOUs with other organizations such as the social security administration internal revenue service Potentially organizations such as yours. So that is done at the highest level. It's called the decept And then underneath that with their disclosure avoidance boards that are set up It's a highly structured governance process because ultimately all employees are sworn for life To protect the data with severe penalties. So the the governance process is extremely robust. Thank you This is someone mine won't take any time at all We have a oh, and thank you for having me here I'm sorry. I didn't thank you first tom. We have a great chairman tom wheeler Superb managing director, uh, john wilkins. He runs lmd office the managing director, which has the cio I'm a special advisor to them. I'm also the cdo. I found out last week and the electronic records manager I do that too and In the modernization head. So that's and we have a Deputy cio for management and a deputy cio for it operations and that's That's it and how do they work with the end users within the fcc in terms of how those people need the data Well, because we have 18 bureaus and offices We set up a structure with uh Intrapreneurs that actually have each have two or three Bureaus and offices and they're supposed to work with them to get their requirements and then build a system Chairman wheeler brought discipline from the standpoint that they used to just buy whatever they wanted and loaded on their machines, which was delightful Until we had a breach that cost us 10 million dollars and then that kind of stopped that but anyway Uh It's uh, it's it is organized very well now from the standpoint of on The intrapreneurs go out and and meet with folks on a regular basis if they like them And uh, then they submit the requests and we put it in the budget Um, but it's it's very it's changed a lot because they all used to have their own It and that's just been in the last year 18 months. Yes They're the one exception. We're not too far behind right the one exception Are the people who run auctions to which we Grateful, he gave back 44 and a half billion dollars to the treasury on the last auction And the next auction, uh, who knows, uh, but it's called the incentive auction Which is a very strange nomenclature if you read about it, but it is an incentive auction And we're hoping to give back a lot of money to the american people on that one as well So the auction group functions Kind of hands off because they Vibrate and create a lot of money Okay, then we don't get to keep dr. Born Well, we certainly don't give any money back But we would like to think we give a bit of extra time on your life, but anyway Um, so let me just describe what we have conceptually rather than detailed organization So we have each of the institutes and centers has its own Cio effectively, but that's overseen by The nyh cio andry norris who reports directly to the nyh director And so that's a sort of layer an infrastructure layer that covers everything and then I come along as a relatively new appointment and the first appointment in this space as I would say the Layout on top of this we now have a data and knowledge layer which interfaces between what we do in the it world of what what our scientists do and so that That layer covers also all of the nyh and that that interface I think is it's tricky in some organizations before I certainly took this job I wanted to make sure I was going to get on very well with the cio And I have to say that's worked out very well, but it's an important interrelationship Excellent. Thank you. I know I'm out of town out of time. I'm going out of town To washington, um, but I do want to say that um, mr. Bender you said it Correctly that I hope this is just the start of a conversation and I'm sorry. We're running out of time Thank you commissioner robbins and commissioner burkle Thank you, mr. Chair, and thank you all very much and again our apologies that this Got down to the wire and we're running short of time um, I just want to and As commissioner robbinson said and and one of you articulated. This is really just the beginning of the conversation because You have so much information to share with us. And so thank you very much. I just have one question and then We'll we'll all get back to you with regards to and this is for mr. Bender You were talking about the apis. Could you just elaborate on those a little bit? All right, so the the apis application programming interfaces are ways to exchange information between Systems that typically do not speak to each other And we have instituted an api policy internally to deal with our internal systems that don't speak to each other And we have an external api Which helps us communicate with our constituents and people who need to use our data The api policy is really critical because when you look at how businesses build business today like amazon and facebook and the rest These are all built in apis because it's very difficult to standardize on data systems And lastly in terms of our data management Most of our data actually resides with different business units. It's very difficult to build a centralized business repository so instead of trying to Collect this data we communicate with these repositories through these apis And the permissioning is done through a business rules engine So that these are examples of how you can use technology to deal with the fact that you know You have these silos and they probably will remain silos Very good. Thank you. Mr. Chair. Thank you commissioner burko commissioner marovic. Thank you, mr. Chairman I just have one line of questioning for mr. Summerlin Part of your testimony and thank you to the entire panel for it Had to do with A recommendation in terms of not just collecting data first and then figuring out what to do with it later You also gave the recommendation of not collecting data just to prove to be outcome determinative Etc that aspect of your of your testimony and i'm wondering about Your the FCC process for data collection How you monetize or are those kind of data collection efforts subject to Benefit cost analysis for instance No, I In many cases. I mean, I've only been there 18 months, but A lawyer may look at something or someone may file a complaint that says that they believe that their billing is not Equal to or better than somebody else's billing and it will trigger an investigation And then it will trigger a data collection that says, okay If there isn't fairness in pricing We need to do a massive data collection to see if that's true or not and So a lot of those are They could be brought by someone in the department in the commission Or most often someone files a complaint On the other half of it a lot of the other data collections are about spectrum We need to know how much spectrum you have who owns it Whether we can combine your spectrum with another spectrum and do narrow banding and therefore free up more spectrum for other purposes So that we can sell it and give a lot of money back to the citizens or repackage it So it it comes from engineering. It comes from pricing. It comes from very many areas But you can ask the same question on all of those Which is why this last data collection? My first enviable job was they handed me a data collection. They've been trying to get through omb for five years But I mean five years and omb kept saying no No No try again and it was because of those issues. They're collecting the same information they saw before the cost benefit There is a benefit to the consumer. You know, we don't really add it up Um, but we did in this case because it was the only way omb would allow it I mean they said you you have to show us something because it's so It's so burdensome on each one of the carriers to do this I mean they add up the hours it takes it was a lot of hours Is that a paper paperwork reduction act requirement? Yes, why you needed the Sorry, yeah, so it's I mean when there are over 1600 data items and you're asking them what the rider Router conditions are in the exact x y locations. This is hard. I mean, we're talking about a telephone company, right? I mean, this is not easy And although they have it they'll tell you that'll take us 6,000 hours at $280 an hour. So you really want this and So the cost benefit is done, but I think the social good of a regulatory agency. I mean, that's that's what you're doing You're saying you you believe that people are being charged unfairly So that could be one of the motions So sorry, it's not more exact, but it could be an engineering exercise could be a pricing exercise That's an excellent answer and thank you for it. Thank you, mr. Chairman Thank you commissioner mohorovic and so thank you very much again to the panel live feeling You're going to be hearing from us again, whether you'd like it or not We certainly look forward to continuing to work with you on these issues Before I close I want to spend 10 seconds talking about something that's unrelated But is important to me from my office I want to acknowledge today is the last day that dr. Christina hadlid Will be attending a public hearing as a member of my staff Dr. Hadlid is a toxicologist who it works in our office in our office of health sciences as part of our Has a reduction hyzer identification reduction Office she's been detailing with my office for nine months And every time and it's happened a lot when somebody has said about me Wow, he's not as clueless as I thought on chemicals It's entirely because of the work of dr. Hadlid. So I'm really grateful. I'm sure for her She's very eager to get back. She's been going to be freed from this assignment and gets to return to her Much more probably enjoyable and predictable life Doing great work on behalf of consumer safety. So thank you so much dr. Hadlid for everything that you've done It's been incredibly valuable to me and it's made a huge difference to the work of my office With that this concludes this public meeting of the united states consumer product safety commission. Thank you