 Thank you for joining us today. My name is Andy Wilson-Thompson and I am a senior policy analyst at New America's Open Technology Institute. OTI works at the intersection of technology and policy to ensure that every community has equitable access to digital technology and its benefits. We promote universal access to communications technologies that are both open and secure using a multi-disciplinary approach that brings together advocates, researchers, organizers, and innovators. The folks at Scoop News Group graciously invited us to hold a community event as part of IT Mod Week. Based on work that OTI has done on the Internet of Things Security and Evaluation, and based on the National Institute of Standards and Technologies' recent work on IoT labeling, we thought this would be a perfect opportunity to pull together an event on the next steps for getting an IoT label onto shelves. As part of his cybersecurity executive order last year, President Biden asked NIST to identify IoT cybersecurity criteria for a consumer labeling program to initiate pilot programs informed by existing consumer product labeling programs and to consider ways to incentivize manufacturers and developers to participate in these programs. For those of us who have worked on issues surrounding consumer privacy and security, especially the risks caused by the proliferation of IoT devices, this is a big opportunity to address the often poor security practices of companies that produce these devices with some kind of nutrition label, trust mark, or evaluation metric that would help consumers have more confidence in the devices that they are purchasing and incentivize manufacturers to implement best practices in consumer IoT security and privacy. Today we are joined by three experts who can help us discuss the next steps in this process, the need for some kind of label in this space, and the ways that civil society and other entities can collaborate to help get a label out into the world. I'd like to introduce today's panelists. I'll be moderating the panel. Then my colleagues Kat Megas is the NIST IoT cybersecurity program manager. She is responsible for coordinating across NIST on work related to advancing the state of cybersecurity at the Internet of Things. Nat Meisenberg is a technologist at New America's Open Technology Institute, focusing on tech security privacy issues, including research into the IoT security landscape and testing standards. And Justin Brookman is the director of technology policy at Consumer Reports, where he focuses on issues of privacy security competition and platform accountability. Kat, could you give us a little bit of a background on the goals of the NIST process, where it stands now and how you envision it being implemented? Great. Thank you. Thank you for having me here. I think before I jump into the work on the executive order, I'd like to just kind of touch on a little bit of the cybersecurity for IoT program at NIST because it is a program that's been around for about five years now. And a lot of the work we did in the previous five years actually were used and were built on in advancing kind of the work we were assigned by President Biden under the current executive order. So going back to kind of the beginning of the program, we started tackling the problem, how can we improve the state of the Internet of Things. And we looked at existing tools like the cybersecurity framework, the risk management framework. What did organizations have today to manage their IoT risks? And one of the things that became evident to us was there seemed to be kind of this disconnect in the world between the organizations that were buying or the people who were using IoT devices and the folks who were producing and building the IoT devices. There was a little bit of what seemed like a mismatch and expectations across the two. And that's when we said, right, I think we could actually provide value in this space where we could build a bridge between the organizations that were building IoT devices and then the users or the consumers or the customers of these IoT devices. Right around that time, Executive Order 13800 happened, and in the report that Department of Commerce and DHS jointly drafted and submitted to the White House, IoT devices were kind of identified as a key nexus of threats for the Internet. So as such, NIST embarked on an effort saying, right, now that we know we need to focus on IoT and IoT is a critical area for the nation, where do we start? So we opted to start with what we call a core baseline, which is listening to stakeholders who've said, you know, fragmentation is not good for the market. Can we first identify where can we all agree is a common baseline across all IoT devices? We understand that there's going to be differences needed for critical infrastructure for federal government for perhaps other applications that are non-governmental and consumer, but where can we agree? So we developed this core baseline that's called 8259. It's been out there for a while and we've actually seen a lot of support for 8259. Coming out of that work, we started working on implementing the IoT Cyber Security Improvement Act of 2020, which took that core baseline and adapted it to the federal government and saying, all right, well, if the federal government is the customer, how do we need to adopt and adapt the core baseline for the federal government as a customer? And then we kind of get to today, which is talking about 14-028 and the executive order there. And again, we said, well, why don't we start with the core baseline again and recognize that the consumers, the customer is different, but how do we need to adapt it because the consumer is probably a different sort of customer than the federal government? So that was kind of the first pivot point and that was our starting point. Thank you for like reiterating what we were tasked with doing under the executive order. I do want to highlight, and we've said this in our blogs and most of our workshops, we do not intend on standing up a program at NIST. What we were actually intending to do is articulate the desirable attributes of a labeling scheme. We are hoping that this is going to be something that is going to be able to take advantage of existing programs out there. So rather than NIST establishing the program, what we're hoping to do is articulate what are the outcomes and then hopefully we'll see the market respond to that. So the ways that we like to talk about the labeling scheme and I'll kind of organize it this way to keep myself organized as well. So let's talk about the criteria. The criteria are kind of the things that we expect the manufacturer to do and what we expect the product to do or the manufacturer to do in support of that product cybersecurity. The second area we kind of focused on was how should the manufacturer demonstrate conformity, right? Is it self declared? Is it something done through some third party? The last part is the label. What should the label look like to convey to the consumer the information about the device and what the cybersecurity of the device is? So going back to kind of the criteria, which is the area that I was very focused on because that was my team's background and kind of my background. There were a couple of pivot points in our thinking. One was when we said, all right, now we are talking about the consumer as the customer. We're not talking about an enterprise customer. We're not talking about the federal government as a customer. How are they going to look at the device? And we also looked at kind of the landscape review of other efforts out there. And we decided we needed to look at the product holistically and not just the device because your average consumer when they buy a device, they're buying a product. They're buying a device on a shelf, but they're actually getting everything that comes with delivering that smart functionality or the smart features, whether it be the cloud, whether it be the back end. Sometimes it's the mobile app that it's on your mobile device that actually controls the device. So we decided to take our core baseline and we decided to adapt that to include the entire product. The other kind of pivot point and what we did as well going from our recommendation for core baseline is we started looking at elaborating requirements. We started looking at what was already out there, what standards already existed. And as we started working through the process, we did, we put out a draft, we invited public comments, we held a workshop, and we were receiving comments and we tried to be very cognizant of the feedback we were getting and articulate requirements. And as we began to articulate requirements, we realized the more specific we became about requirements, the more brittle those requirements became and unlikely to be able to keep up with the rapidly changing, you know, landscape of whether it be threats or whether it be device functionality that was trying to do something across all of IOT, even if it's just consumer consumer IOT devices take so many different flavors from, you know, very sophisticated and very capable down to your very constraint device constrained in terms of power and bandwidth and processing power. So that's when we pivoted to what we call outcome based. And the picture we like to paint when we talk about outcome based criteria is what we tried to do is articulate what are the outcomes for cybersecurity that we want the device to achieve. How that outcome is achieved is likely going to have to be fulfilled by multiple standards because you're trying to fulfill a large landscape of requirements and also we think standards are likely to be able to keep up and evolve more quickly than any sort of brittle requirements that we could have created. So if you if you kind of look at the picture that we like to paint. At the very top, we kind of talk about outcomes in the middle we talk about all the standards and specifications that are likely to kind of backfill and establish a more articulate or more detailed description of how to meet the outcomes. At the very bottom is are the devices that are going to or the device manufacturers or product manufacturers are going to have to either self attest through some sort of self declaration or either use some sort of third party certification to meet those requirements. In addition, when we talk about the label, we didn't go into any sort of specificity about the label other than making a recommendation for it to be layered and that it be binary. This was based on kind of our experts assessment and research that we've done and what was likely for us to be something that a average consumer would be able to understand. The idea behind a layered label would be that the actual device, the first kind of layer of information for the consumer is kind of a yes, no, the device either does it or the device does not. For those more sophisticated individuals that might want more information, whether you're a repair man or whether potentially you are kind of a very tech savvy cybersecurity savvy consumer. You would be able to go to this layer that would be digital and actually find a lot more information about what the device does. So that being said, I'll try to kind of like wrap it up to like kind of where we are right now and we're right in the middle of kind of the next steps. We published in February what we kind of see is our final recommendations based on the white papers based on workshops comments received landscape review. Now that these recommendations are out there. We've invited what we're calling contributions where we are asking, whether it be the market stakeholders to provide us with contributions to help inform this kind of pilot phase where we're piloting the concept of what this labeling scheme could look like, whether it be that you have an existing program, you have existing specifications or standards, how those relate to the outcomes that we've already published, all the way through giving us contributions discussing what are potential incentives that you see would be needed in order for this sort of scheme or the sort of concept that we're piloting and are in our white paper. And give us your feedback, what are the incentives that are needed, you know, who would undertake those incentives where could they come from any sort of information that would be helpful to us in completing kind of this report that's due to the White House in May. So I did not start my timer Andy but I probably talked a bit more than my five minutes. Thanks cat know it's very helpful to get a few of you know how this process is gone, and what the background is I'm not sure that people are necessarily super up on that. And so it's very helpful to sort of inform our discussion. I wanted to move to Nat, you know civil society and security experts have been sort of sounding the alarm about the risks of it devices for years, especially as many manufacturers used to make sort of dumb products start to connect their new products to the internet. Could you talk a little bit about why an effort to label devices is so important right now, and some of the risks of continuing on as we have until this point. Sure. Um, so. So to start with the landscape of IOT security has historically been pretty bad when we talk about IOT let's just step back and think about like the kind of broad range of things we're talking about because it is everything from the lock on your front door to TVs. I've been thinking a lot personally about the kind of close nexus of things that are that have ostensibly been like home use or consumer goods that are inextricably also kind of office goods TVs being smart TVs being a primary example because you can't build a modern conference room without a television without a big TV on the wall, and you can't find a big TV that isn't also a smart TV. So it's a broad range of products and historically the short cycles of these products are short and so fast that the how to secure them has has been an afterthought because the incentives have just been like poorly aligned. So this is led to a situation where you can buy a product like a baby monitor and have no idea exactly like what services it interacts with. And you know there's countless examples of just like hacked IOT devices that come up in the news. You know everything happens from leaking your personal data to the ability to get onto your network and get to other devices or in the cases of TVs, modern TV also has a camera and microphone. We're talking about a lot, a lot of the ability to capture several kinds of very sensitive and personal data. And this is really an issue, both in the home context and in office context, government purchasing context by extension. And then there's a whole range of other like smart devices for the office from the modern voice over IP phones to even things like the smart coffee makers. And so labeling would be and a standardized testing framework behind it would is a useful first step in being able to get to the series the kind and purchasers the kind of information they need about whether the product can meet a minimum baseline and we have these kinds of labels all over you know there's the ubiquitous nutrition label on food, but like we have energy star on other electronics we have lead for buildings. And all of these are certification schemes that just allow people to out of glance kind of have some knowledge that this product meets a baseline set of tested standards. Labeling iot isn't new either. In 2020 Singapore started requiring iot products certain classes iot products to adhere to labeling standards. And in 2021, they said Singapore and Finland, bilaterally agreed to cross honor each other's testing, or in other words, Finland and Singapore is standard and adopted it as its own. And now, if a product gets tested for the finished market, it is considered tested for the Singapore market vice versa. Yeah, and so so also in the consumer tech kind of adjacent to security labeling. France has implemented a repairability index law, which requires tech product manufacturers to similarly identify how they adhere to a set of standards on being able to repair electronics. So, these, so major international iot product vendors are already conforming to these kinds of standards in other markets. And so, these kinds of labels have the potential to kind of push the entire industry in a better direction they just need the wider the adoption and the more the consumer demand and the more ways we can induce that consumer demand, the more we can hope that that that kind of baseline standard will find its way everywhere over the next five 10 years. Justin. So as part of consumer reports efforts to educate consumers about the products that they can purchase. Your digital lab has taken a close look at some verticals of iot products and provided reviews. As a consumer advocate yourself. What issues do you think need to be considered as part of the discussion around a new labeling scheme. Yeah, I think the key issue is it's really difficult to impossible for consumers to make security conscious decisions today you look at the marketplace just don't have any information available to you. In theory, like it will legally everything should have some degree of baseline security. The Federal Trade Commission, which enforces the general purpose consumer protection statute has said that it's prohibition on unfair business practices means companies are legally required to use reasonable security to safeguard data. And the good news is they brought 80 cases against companies for violating that the bad news is they brought any cases and there's like eight billion products out there. So there's obviously been not been enough deterrent effect to get a lot of companies even implement the most rudimentary of schemes. Also about half the states have some sort of data security requirements California has a dedicated cybersecurity law. Again, though, because of lack of resources lack of enforcement lack of technology expertise and regulators. This haven't been able to put enough fear of God and into companies or enough companies to move the marketplace to a place where consumers can reasonably expect that things will operate safely. So I think as Matt was saying, it is a little bit of a wild west if you ever attend the consumer electronic show in Las Vegas and there's all these new iot devices and you go to a booth and ask them about that, you know, data security or how long will this product be supported they look like you're talking a different language. And Amazon right Amazon now a third party marketplace you can buy tons and tons of stuff from tons of tons of different vendors, often, often have no idea who's actually manufacturing it. A lot of this stuff is cheap and manufactured overseas which is great that it's cheap and that's that's a real benefit to consumers, but perhaps safety and security and other issues have not been accounted for there's no testing regime there's even even even even fewer requirements here in some cases that can lead to real danger hoverboards is a kind of famous example on on on Amazon people buy these hoverboards for Christmas. There's no safety protocols in place, and then the house catches fire. It's very similar like on data security of this is done, you know, when you buy something on Amazon, there's no reasonable basis to expect that security is baked into a product. I think even beside me I think the other issue that's interesting is support period right you buy a device, how long is the company going to like provide security updates because security for connected device is not a one time thing it's done like you have to do security updates you have to like dedicate resources to looking for bugs and finding them, and then patching them. And that's that takes money that's something companies don't aren't thrilled about doing. And so there's just not a lot of real clear expectations today for iot products and again because of lack of enforcement, maybe maybe your expectations are that it's not going to be done. So if you think about desktop operating operating systems, like you get updates every month is very well expected operating systems are updated for years and years and years, and people generally don't like Microsoft VISTA was supported until a couple of years ago. And so that's a place where people do have reasonable expectations. So phones is a lot dodgier right I mean some phones like Apple's Apple iPhone tends to get supported for a very long period of time. Android phones and practices are all over the map. I when I was the FTC we did a study of these of these systems and some phones like you know get get, they promise support periods for three years and maybe they'll support them for five others like your expensive flagship phones come out of the box insecure and never a patch. And then I am an IOT is even even worse like you get you buy a smart TV how long do you expect is going to get security updates how long they're going to support the apps that are on there there's just no real meeting of the mind and people don't really know what they're getting they buy something they don't really think about what the downstream cost could be so that's one place I think a label could be really helpful if like you buy a product and it says this will be good for five years. This will this will be received security updates for five years I think a lot of the elements like identify the NIST framework are really hard for people to wrap their heads around like access controls or encryption. And it's hard for people to price that like, how do I how do I price price server to server, you know, and any encryption is really difficult people to assess the tail risk. But like product period like when I will have to replace this this product is good for five years versus three years. That's something that a person can in would make an informed market decision. And so I think that's one area that's that I'm super enthusiastic about that a label could meaningfully convey convey to someone how long they should expect it to last. I've been I've been to see yes and sort of seeing the large areas of the conference room that are like Shenzhen company number. And you know they'll show you the light up backpack that connects your phone and changes colors and if you say does this have a privacy policy they will look at you like you have two heads, but it is a very cool backpack that changes colors supposedly based on my mood so I don't know, but it is definitely hard especially with those smaller companies or you know one off device makers or, you know, inexperienced, it's not hard to connect a product to your phone to connect it to an app to have like fun cool features but the risk posed by that is challenging and something that just isn't necessarily front of mind for a lot of the manufacturers that make them. I wanted to ask you about sort of the layered approach that this is something you've promoted Justin brought up the fact that it is really hard to identify things like encryption or you know features that to those of us who work in tech policy are extremely important encryption authentication those kind of things, but are hard to visually represent for someone who doesn't know about this right the why should you care part is a challenge so could you talk to me a little bit about why you guys should be a layered approach like sort of trust mark layers versus a nutrition label, which at least some of the aspects of the nutrition label are things people understand like fat calories and kinds of things. Why did you think a layer approach was the right way to go. Yeah. So, I don't want to speak for my colleague Julie Haney, who is our human factors. Lead on a lot of this work, but I but I think as a team, you know we were we were very much in consensus. There were kind of two schools of thoughts, but we've also done a couple of workshops we did a consumer IOT workshop in. Oh my gosh this goes back now to November, not this past November the previous November of the year before that. And we heard very strong feedback at that workshop and this was even before the executive order was was kind of on the basis that your average consumer cannot be expected just like you said to kind of like understand what is the importance of encryption, how am I going to use encryption. The idea as well and IOT cybersecurity right there's also this kind of displacement value proposition. My device still continues to work for me. My device may get compromised may get, you know, may get like recruited into a larger kind of botnet attack. I may be bringing down, you know, potentially third parties like then. However, I as the individual. My device still continues to kind of work for me that being said though, we did realize that there are people in the environment that might need to understand whether you're a repairman or whether you're some third party or whether you're a security researcher. There were stakeholders that were actually interested in understanding exactly what the device does. So that's kind of where we decided it's really important for us to establish a baseline. All devices for the most part should be able to and should meet the baseline that you should either do it or you should not do it. But we did agree that there needed to also be the kind of the more information that's put out there. I'm kind of curious I was listening to some of you know the other folks speak and I, you know, I'm not sure if everybody can see my head nodding because there was a lot of it I was agreeing with. You know, one of the very first things that came up. When we kind of kicked off the executive order activities was will a label do we actually think that a label will change consumer behavior. And we received a lot of questions from our stakeholders on that missed has done some research, you know, and we do. We have seen from our research that at least those that do participate. They do say the average consumer does say I do care about cyber security. However, we've also seen though in other areas that sometimes consumers say they do care about something. And yes, they get that that's really cool backpack that has the ability to kind of change life depending depending on my mood. You know who's calling me on my phone I can tell by the colors flashing on my backpack and you know sometimes in the, in the moment to moment when you're standing at the shelf and you're making actually a purchase decision. And the intent to actually, you know, care sometimes gets overshadowed by kind of decisions about all the cool features so, you know, I think there's still a lot of questions remaining open about, you know, what will happen. I also say in our paper that we think any sort of specifics about a label would have to be market tested. We would probably have to get into significant details in this in this nine months and three months kind of pilot. We didn't have time to get into that level of detail so we're actually saying, whoever is going to potentially take ownership of the scheme, whether it be someone from, you know, industry that wants to step up whether there are a couple of organizations in the industry that want to step up and say, you know, this looks like a good approach we're willing to kind of take it on and become a scheme owner. You know, our recommendation would be for the label that they would probably have to be some significant market testing to kind of figure out which one would actually change purchasing behavior and be most useful to the consumer so I'm not proposing that we have the end all be all answer. Yeah, I think there's been a lot of research around consumers positions on privacy Pew has done some research and I think part of the question that we all struggle with is how much do you care. Right, you know, you ask, oh, do you care that your product is encrypted and people say yeah that sounds great. But the question is, do you care enough to buy a product that is encrypted over a product that is not encrypted right, you know, these these ideas that privacy and security are important sort of translate, but does it overcome the cool flashing lights, the fact that this product seems very interesting, the how much is always a question throughout public policy in general but definitely when it comes to privacy and security, sort of on that Justin I actually wanted to ask you, you know, we consumer reports as a project called the digital standard which is a set of metrics around security and privacy that you review connected products on. We've worked with you on that. And often what I was told by companies is the carrot for them for adhering to these best practices was getting a great rating from consumer reports. And so CR being independent from companies and from government has that great sort of market impact like carrot stick approach that has been extremely successful you for you and people in my family and I know you know if you're going to buy a car you don't need consumer reports listing. How have you found that rating companies and rating products can incentivize like adherence to best practices. Yes, we've done for about five or so years now yeah we not to digital standards it's kind of like their criteria is that you don't share data you don't need to you don't click that you don't need to have been evaluating or more I should I should know sorry product lines on those criteria and give them scores about you know this company and again, you know, it's tricky because you can't actually see a lot of it right. So like you know we if there's data sharing going on the background like you know from their servers to someone else's like we can't see that. And so that's very same with security right they're using encryption on their on their servers. That's not observable to us. Another challenge is like you know. Sorry, I lost my train of thought. Sorry, the practice can change sorry. Turn a switch and then. Suddenly the behaviors are wildly different. There's also lots and lots and lots and lots of products like I said like you know on Amazon now is you're not choosing between 10 television choosing between 100 different models from you know 30 different manufacturers. It's really tricky. I think you know we have done some analysis to see whether it doesn't have changed buying decisions. We did a conjoint analysis to see what whether people do change or behaviors based on it and like you know they, they say they do. You know they said they're willing to pay more for premium it's really hard to see in practice, whether they whether they do or not we don't have the visibility into that but I think people we've gotten good feedback that people appreciate the information. I think cat makes a really good point that there are externalities that you know that you don't price right my computer maybe spamming people around the world. And maybe I don't care I don't really bear the cost of that and so I don't really I'm not incentivized to take enough action on it. Also people are really bad at calculating tail risk right that's like the very small chance that's something really bad happening people just don't calculate that which is why we mandate seat belts right it's why you mandate airbags. Don't make me people make an informed choice like I'm going to pay extra the airbags or not no as a society. We have to make the decision to do it and I think for security, a lot of it's going to have to be that we're going to have to mandate certain things, and then some things maybe the margins. Maybe like you know how long product products will be supported or something that they're going to have to, maybe that's the place people can make informed decisions. We all, you know, really want security and privacy to be an important factor for consumers we all spend a lot of time trying to convince people that is important but ultimately. It is hard both to get people to make decisions based on those sort of more amorphous things and also to measure whether that's how they're making decisions. You know, I want to talk about mandates a little bit later but not one of the things that I think is a challenge with labeling is, you know, we live in a country with big tech companies companies that make a lot of these products, that are very powerful when it comes to political lobbying and companies don't love being regulated right like labels often require some sort of push to be popular but I know there are also a lot of barriers to a labeling scheme, not just for large companies but for smaller companies that produce products that are available to consumers, could you talk about some of the sort of more technical or logistical barriers to implementing labels. First of all, like the just doing any kind of product testing is placing more time between when I make a thing and when I can sell it. So, you know, the and, and that's for any, any safety testing any product testing is just going to take some time need people to do that test. So, in the So you're going to need at companies at a certain scale are going to need to do this in house and then smaller companies are going to need to rely on like external provide providers to do these things. And that is time and money. There's a lot of other issues by the that get introduced by the way the ecosystem is just structured. So, I was reading about a vulnerability that last year that that was announced by Cisco in their IP phones so like the phones on people's desks where Cisco did all they could do to patch the vulnerability, but the problem was partially the software on the phone and partially the chip in about eight different model series and until Broadcom, the maker of the chip did something about it, the vulnerability which I couldn't find whether or not Broadcom has even patched this vulnerability that these phones are less vulnerable than they were last summer, but there's still that vulnerability sitting there because of the like downstream effect. And this is Cisco and Broadcom like two of the like largest players in this field, who are having trouble getting this kind of coordination, right. And so there's just like the disincentives to do this about. And like I often say to people in you know my career doing technology, nobody ever asked me to slow it down to make it more secure. There's never an like, and that, that has brought all kinds of problems up and down the ecosystem and, and that is why you get the pushback to this lately because it's kind of one of the things we're trying to do is backfill decades of inattentiveness and like these kinds of reflexive responses built around that inattentiveness. And so there's, yeah, there's a lot of pushback there. But I think that when we're talking about the disincentives, I also like to think about like, well, what are the, what are the ways that we can actually like reduce the pain of doing this, as like the groups of people who might be doing the testing or making the standards and I think one of the ways to do that is kind of linking these kinds of outcomes in testing standards to how technical standards share technical standards get developed. So if you can develop, you know, the kind of IoT analog to, you know, a wide known internet standard like HTTP or something like that. If you can get all of the makers to say, we're going to use this one standard for pushing updates out to IoT devices, then that gives everyone who's testing something to say, is that standard implemented or not. And if it is implemented properly, we can make all of these various assumptions about it security values. And but that doesn't really exist as just next to impossible to find out what's even running on a lot of these chips. So having the testing and labeling standard will also require just a lot more information. The fact that it requires more information cherry just so really important because when you go looking, you know, how do I know what chip is in my random IP camera that I bought on Amazon for $30. Yeah, I think that's one of the things we've struggled with when testing but is a feature of the alternative things in general is there are so many different components that interact for each of these devices, whether it's the operating system they run the chips that they use, shared code. A lot of them have an app or they interact with an app that maybe controls multiple different kinds of devices. They might have a relationship with Amazon or Google, or one, you know, another connect to Alexa. It's so many different pieces that make up sort of the security landscape of an individual device. And that makes it really challenging to test it makes it really challenging to evaluate. But it also introduces a lot of security risks that, you know, aren't the case for something that is like a dumb product for something that has a much more simple structure and wave interacting with other devices and with users. And Nat, did you have something you wanted to circle back on? I just wanted to build on what you were saying there because I think that one of the things I really like about the NIST criteria and guidelines is the that kind of broad ecosystem full functionality definition of what constitutes a device. The device and the server and, you know, the cloud infrastructure, because like one example of why I think it's like so important from the kind of testing we've done is we were testing a baby monitor and I found that the baby monitor was every 15, 20 minutes. Just sending a little data packet to a server in Beijing. And I did a bunch of digging. From what I can tell, the company's server that it was connecting to is a company that runs an IoT control, cloud service, and one of the services it offers IoT makers is a way to like locate a device on the internet. So technically like to think about why you need this technically. If I have my, my phone out in the world, and I want to look at the camera feed from the baby monitor, the app needs some way to find where in the world my home internet connection is, and broker that connection between where in the world my phone is. And so there's, so there does need to be something that's that's saying hey here I am. That's like a completely feasible technical requirement to making a smart baby monitor do the thing that you buy them to do. Without any kind of labeling or disclosure requirements, like, I don't know that that's the data that's being sent to the, to the server in Beijing, that there's no like standard like datagram protocol that I could inspect to say like, oh yeah, all it's doing is sending a serial number to a server. And so there's, you know, all I can tell you is that it was periodically sending a consistent data packet to Beijing, and that's it I can't tell you what, and everything else like this speculation. And I think that, yeah, having having that that kind of information disclosure requirement would make that process easier to test and label and standardize as just one just really concrete example from the world. Yeah, I think that's part of the challenge is sort of the black box nature of a lot of these devices is, unless a company explains to us, you know, what they are doing in each of these cases, it's very hard as an external testing organization, especially when some companies are hostile to external testing to be able to truly understand anything beyond what they tell us and what is immediately visible right like I can look at a privacy policy and they can tell me that they only disclose information in certain cases, but that's all I've got. And that I think is one of the interesting things about when companies have to get involved in their own testing when they might need to self test to a standard, when they might be a collaborative actor as part of this versus an external organization imposing testing upon them. There's some sort of cooperative nature that means you know, they might be asked exactly what information is being sent on that ping. Something innocuous or is it something that consumers might find concerning because we were unable to actually figure that out. Kat, I wanted to talk a little bit so we all have very strong opinions about what this should look like and very strong opinions about iot testing in general and the importance. But I know the way that this has been working on this you have sort of left some open ended components of needing someone to implement this. You know, it stops at a certain point and you need sort of a scheme manager. So where, where do you see sort of civil society external experts collaborating on that kind of next steps like what do you need from those of us with strong opinions in order to help get this on shelves. Well, first of all, I will not get ahead of the White House since really we are on the hook to deliver the report to the White House after that I think it is really going to be up to them to kind of decide what is what is meaningful and what what makes sense for the next I think right now getting those contributions whether it be to say to stand up and say hey, I may not have a program currently but I am interested in standing up and stepping up and maybe being a scheme owner and perhaps you want to be a scheme owner for a certain type of consumer device. Maybe you are interested in saying look, I, I, I think there might likely be multiple scheme owners out there. They could potentially all step in. But one thing we've also made as a recommendation is understanding the consumer. There's multiple schemes and each scheme is is perhaps organized around a product type. There probably might need to be somebody that steps into the role of being, let's say the label governance and saying right. I don't actually run the laboratories myself I don't actually pick and choose which ones are the standards. But I do kind of manage the process to ensure that only products that have satisfied the requirements or that are participating in one of the scheme owners is actually putting this label on and then doing all the market surveillance that goes along with with the labeling so I think right now we are looking to draw on kind of the collective brainstorming of the community to say hey, here is here is either a role we would like to play or here's some ideas we have on how this could work or here are some specific steps that you could take that the government could take to actually help close some of the gaps we see between where the market currently is and where existing conformity assessment programs are and, you know, here's some gaps but here's some ideas about how we can close the gap. So it's a little bit of an open ended question we have for you all right now. But that's because we really want to not constrain the responses that we get. So hopefully we'll hear from everybody I think because we do have kind of an early May deadline and you know we have our own processes. Once we actually draft the report to get it through before we can actually deliver it. We are asking to hear back from folks by mid April. So that is quickly coming up. I'm sorry mid March, mid March. So there is still time there's there's still a couple of weeks out there but hopefully we will hear back. I think generally even early on before this executive order started. I used to get a lot of questions from stakeholders about, you know, what's the government going to do is the government going to stand up a program. Generally, we think the likelihood of something being sustainable in the long term is really when it's driven by market need or by market demand, wherever that might come from and so rather than it be kind of a top down approach we always think that a bottom bottom up is better. Because if there's a demonstrated market need you're obviously going to find a way to continue to sustain the label and the effort whereas if it's really top down driven. That's usually not the best way for things to exist so hopefully we'll have people who will respond over the next couple of weeks and say, yeah, you know, we would be really interested in playing a role and we are going to provide that information and our reports to the White House and we'll leave it up to them to determine what the next steps are. I have questions from the audience but I just want to raise one more question based on what Kat had said and this might be a question for Justin. You know, we talk about market need but also there are things like mandates or regulatory enforcement of standards, you know, although we would very much like things to be voluntary and you know stakeholders, including companies who create these products to be on board, there is discussion about whether some sort of forcing function might be necessary to get this standing up some sort of requirement. And I know you were previously in a regulatory government capacity. I wonder if you have thoughts on the necessity of sort of having a forcing function having some kind of mandate or requirement and if you think that that might help move things forward or at least hold certain, you know, companies accountable because sort of in both Finland and Singapore, there has been, you know, sort of more of a requirement than we're currently looking at in the States. Like I said, there already is a requirement I think the FTC has said you doubt, shout, use reasonable security if you're legally required. It hasn't been super tested in the courts so I think clarifying, you know, Congress clarifying it, the FTC doing a rulemaking saying maybe a little more concreteness, a little more specificity could be valuable. But I think funding is probably the next next biggest piece in the build back better bill there was provisions to give the FTC a ton more money again they have 50 people in the privacy and security group right now to do all of privacy and security which is way way way underfunded. So they need to be a lot bigger. So they can bring cases and also deliver more informal guidance or maybe formal guidance and then you know they need to hire technologists in addition to lawyers. As the FTC we're in a very small wing called the Office of Technology Research and Investigation but we were like a hand like less than 10 fewer than 10 people. We're reporting all across other divisions within the Bureau of Consumer Protection so they just need a ton more technologists in order to kind of, you know, bring these cases or to kind of articulate what the standard should be in order to give you know that clear rules for the road and currently exist. There's always a capacity issue in government and always capacity issue in general but I agree, you know, the role of technologists in sort of these regulatory discussions is really important. You know that kind of expertise. I'm not a lawyer. We have lots of excellent policy lawyers but there is a sort of contribution that is necessary and you know I appreciate when government agencies show a recognition of sort of that role and that contribution being important to these discussions. Now I want to raise one more thing and then I'm going to move into some questions. So, ultimately, in other countries where labeling schemes have been stood up, a lot of the same products that we use in the United States are being used. We've seen that, you know, if the computer that I purchased is being purchased in another country, if these devices are required to have some sort of labeling scheme somewhere else, you know, all of these big companies who would probably not be thrilled with a mandate in America are already held to that kind of requirement. How does that work, you know, do you think that's something where that kind of testing could be a reciprocal arrangement in the same way it has between in that situation. I mean, I think there's lots of examples of labeling schemes and like standards meeting where, where you see market moving. One really great example is the California mission standards in the way the entire auto industry in the US shapes itself around that. And there's weird little pockets like that, where you have places where there's outsize influence on a market force that can shape the contents of products. And I think that, yeah, with IOT there's with tech stuff. I mean, once Dell is, you know, filling out the paperwork for Howard Parable, one of their laptops is in France, they could in theory just start doing it for all of their models. They already have the systems in place to do those things. So certainly like companies that are already meeting the Singapore standards are certainly already well positioned to start meeting new standards. And I think one of the, one of the things that is, is hopeful potential for future work or thing is like moving towards the like, harmonizing of those standards or at least finding the Venn diagram of those standards, so that we can say, Well, one so that I can say if I have a product that has a Singapore label on it, that it at least as a technologist who does this kind of research, I can at least say, well it satisfies this three quarters of the NIST guidelines, or something, you know, I don't know the actual percentages that look, but like, so the idea is so you can use one set of standard the satisfaction one set of standards as a shorthand for the satisfaction of a subset of another. And so, as companies start doing it they can say well we need these 10 things for Finland and half of the US. And then like, but then they're halfway there right by by fulfilling the standard for Finland. If the ratio is one half that then they're already halfway there and then so it's not twice as hard for them to then meet a stricter standard elsewhere. So having these standards in other places. Another example, where the EU has pushed market changes is they started requiring standardized cell phone adapters, and that is what like secretly that is what push every cell phone having its own kind of plug to everything be in usb. I feel like you know they're having a more global approach to some of this makes sense the sort of you know rising tide lifts all boats of certain I live in California, having higher regulatory standards than either other countries or other areas of the same country. I think one of the things that sort of I have noticed both in the questions we're receiving and this conversation is thinking about the relationship between companies and external evaluators so you know sometimes there seems to be a lack of trust around companies evaluating themselves or self reporting their security features you know there have been situations where I think consumers have felt misled, you know, some FTC enforcement where, you know, we have been told that a router, or, you know, communication platform is more secure in certain ways that it turns out not to be. But it seems that like in both Finland and Singapore it is self reporting. You know there isn't an external evaluator. Yeah, that is the case as far as I understand it is just self reporting it that's certainly the case with the French repairability index. But yeah, I think that that that is part of the that so we do need to, we will need to rely on certain amount on self reporting because as just mentioned, we don't know with there's really no good I mean this is a large problem with evaluating any service provider is you don't know what like who their servers are talking to. And who they're really hard to audit data back in data sharing. So there's, you're never going to escape all of the trust questions but I do think that you can move towards a space where manufacturers are certifying that their product does something, and then you have other organizations that can come in and do some of that spot checking, not dissimilar to like auto crash testing or something like that. So, automakers will do their own crash testing, and then consumer reports and other people like that will, and other testing insurance, the national insurance testing group or whatever it's called. Well, I'm sure I got that name on will, you know, randomly buy these cars and crash themselves. I think that that moving towards that kind of an ecosystem, which is this kind of like a mix of self reporting and spot checking, because I don't think, you know, honestly, you know that there's not the ability to just test every revision of every product as it comes off the shelf because there's just too, too many the firehose is too powerful. You just have to sample it. I hope you're catching enough of a representative sample. Well, I appreciate I wanted to give each of you an opportunity to sort of come around it if there's any final thoughts you'd like to share or any next steps you'd really like to draw the eye of listeners toward cat do you want to if there's anything else on this behalf that you think that would really should be aware of or sort of that you would like to share with us. Well, I mean, I think we I think we touched on everything. Again, I encourage everybody to submit a contribution. I do want to say, you know, especially being a cybersecurity professional, we do tend to focus on kind of the problems with cybersecurity. On the other hand, you know what we really want to see is we want to see IOT adoption, kind of as broadly adopted as possible because we recognize the benefits and the immense benefits that IOT can bring and, you know, lately I've been talking a little bit kind of about the strategic vision of IOT and if you think about it, you know, AI is one of the big topics that's kind of on the horizon that it's talked about as well that has some things that need to be worked through. But, you know, IOT is really going to be the mechanism through which a lot of the data and a lot of the information that is going to be used by a lot of these AI systems is going to be needed and then a lot of these AI systems are actually going to leverage IOT in order to actually implement the decisions that the AI kind of produces or the outputs of the AI. So we do have another effort going on at the Department of Commerce. It was in the NDAA of 2021 and the Department of Commerce was directed to stand up an IOT steering committee consisting of non-governmental stakeholders. But the focus of this IOT steering committee is really to look at what are the barriers to adoption across kind of the US? Where are there potential kind of like regulations in place or programs or policies that might actually be inhibiting the growth of IOT? You know, if you think about it, what we really want to do is we want to make sure there are no barriers. So unfortunately the nominations for what we call the IOT FACA, which is the Federal Advisory Committee under which the structure of the steering committee is being established. Unfortunately that closed yesterday so it's a little late to ask for nominations. That being said though, we will be inviting people to come in and actually speak to the IOT FACA, FACA. I anticipate over the next year it is going to be resulting in a report that goes to the federal government to a working group, an interagency working group, and ultimately it's going to result in two reports that are going to be going to Congress with some recommendations. So for those of you who are interested and passionate about IOT, I encourage this as kind of another area for you to follow as well, because there might be opportunities as well there to talk about the role of cybersecurity. I'm actively involved in it because I feel that we really need to address the cybersecurity issue if we really want to recognize the potential of IOT because I think the first time something really, really, really bad is going to happen. The chilling effect on the market is potentially there and we may actually set back adoption significantly. So thanks for the opportunity again to speak and talk to your community. Of course and Justin from your perspective as someone who you know works deeply on consumer issues. As we move forward in this sort of what should a label look like who should implement it process, you know what would do you think we should really, you know, keep an eye on. Yeah, I think I'll follow up on something that said that it doesn't be a combination of, you know, mandate transparency information provided by companies, they should be probably legally required to make more information available than they have today but I think third party testing and third party auditing is going to be essential and there are actually legal impediments to that today sometimes companies will put in their contracts like you're prohibited from publishing tests about us or you have to talk to us first so we get a hearing. And the DMCA the Digital Millennium Copyright Act like you talked about black boxes like it makes it makes it illegal to look in the black boxes. You know sometimes, like, I think back to the NYU ad observatory which is a system that was looking at Facebook and looking at their political ads and try to figure out how they're targeted Facebook didn't like it so they cut off the API. I think we're going to need some more mandates and more clear mandates around data security we're also going to probably need some more mandates around external auditing and transparency and so you know there's something there's some elements of that in Europe with the Digital Services Act. My colleague Laurel layman was testifying this morning on some of the bills in the House Energy and Commerce Committee and transparency from representative Trayhan. We're going to need some some more elements like that too in order to be going we're going to need a little bit of everything the whole this this place the ecosystem more accountable. Yeah, for those who want external testing and you know feel like external organizations and security experts should be part of this holding companies accountable aspect. One of the ways you know something OTI has done significant work on is DMCA reform CFAA reform, you know, there are legal impediments that bar security researchers right now from some of the types of research that we see as extremely valuable and important. And unfortunately that's often because companies are hostile to this type of research there's not, you know, as great a relationship like an interdependence on seeing the value of external evaluation and research. And I do think I agree with Justin it needs to be sort of a multi pronged process with a lot of different actors who come together to make sure that we're doing this right. Nat, did you have any as our, you know, OTI technical expert who's been, you know, mostly involved in this anything you want to close out with. I do want to echo that there is a lot of good policy movement in the correct direction the trade hand bill that's going through the house right now is actually very has a lot of interesting stuff about, you know, more about like larger internet platforms and the information they have to provide, but it's it is a very good piece of legislation in terms of like understanding the research requirements and the public interest requirements of the modern internet age. And then another example of good policy moving in the right direction is the 2021 round of temporary exemptions to the DMCA. So the DMCA has a provision where the library and Congress certifies a set of exemptions to the DMCA. The last several rounds have seen an expanding exemption for security research. And while not perfect. There's a lot more clarity than there used to be around what you can, what parts of the black box you can look into and what you can do once you've tried it open. But again, there's there's work to be done there. One thing I want to come back to is is part of the NIST guidelines towards the notion of schema owners and and how that's in my head plays into the idea of the of like this kind of ecosystem of third party testers and how to incentivizing people to pick up being schema owners, and to think about like, who it is we talked to to get that work to happen like do we like, like, and how do we think of the like overlap of schema owners like, do we prefer, you know, is it more useful to have a schema owner that's thinking about lighting and light bulbs, or is it more useful to have schema owners that are thinking about child safety and children's products. Or is it a both amp, like, do you know, do, do we want, you know, one, one schema owner to think, as they're like primary set of outcomes to be just centered around what do parents want to know. What are we, you know, what does this, you know, what does our child safety IOT label me, like what set of values does it represent. As opposed to just like the, like, does this light bulb do these things or not. Like, does it check the boxes and, you know, both are very useful. And, and again it's very, I, it's helpful because you want to think about like the outcomes and consumers here when we're doing the labels. And so, thinking broadly kind of about schema owners in that sense too it's like, who, you know, what the kinds of labeling scheme that consumer reports might come up with might be different than the kind that like some, you know, building maintenance association comes up with for like, you know, thermostats. Yeah. So thanks everybody. I agree with you. Oh, I'm sorry cat did you have any final thoughts on. I was just going to respond to Justin. Yeah. I completely agree. I think you bring up two very good topics. Please do submit comments, Justin. But the governance around multiple scheme owners is indeed something that we recognize and we touch on. And then profiles is something that we also recognize and I could have talked to the longer during my intro but one of the reasons we decided not to look at tears straight out of the gate, because we felt how can we ask the consumer to make the risk decision and select which one of the tears are appropriate for the risk associated with the device. But we rather would see these profiles kind of emerge from the market and say right for baby monitors, we may have to adapt this profile to be appropriate for baby monitors because the risk of a baby monitor is maybe slightly different than the risk associated with I don't know my, my connected tire inflation pump, you know, that's connected to my phone so wanted to respond to to tempting to jump in there. Thanks Andy. Thanks so much cat. So I wanted to flag that my colleagues at OTI comms. If you go to Twitter at OTI are going to put up the link to the next comment opportunity. If someone has strong opinions and would like to share that helpful expertise before March 15. Also, if you would like to, you know, participate in any other it mod week events, it goes till Friday. The website is it mod week.com. And I know there's a lot of interesting community events, another one that is a new America event but also, you know, on a wide range of things. So, thank you everyone for joining us. We really appreciate, you know, NIST, giving us this opportunity to ask questions and learn a little bit more about where your process stands, but also, Justin and that thank you for joining to provide, you know, those other perspectives and both technical and legislative and regulatory expertise. So thank you everyone. I hope you have a lovely Tuesday and please continue to follow the discussion around IOT security at OTI on Twitter. Thanks everybody.