 Welcome to a very special edition of Security Matters. No need to adjust your screen. Our intrepid host, Andrew Lanning, is taking a well-deserved break this week. My name is Cameron Javdani. I'm the president and co-founder of SoundSecure and with some very big shoes to fill today, guest hosting in Andrew's place. But we have a special guest to help me carry the load, Derek VanderVorst, the owner and founder of Sound Intelligence, here today to talk about a number of different audio issues with analytics, sound processing, and what we can do for security systems. You caught last week's episode, you would have heard about the new SEA Audio Working Group, and I'm thrilled to announce that Derek has joined as a part of it with one of the subcategories of that working group having to do with the analytics. Derek, welcome to Security Matters. Thank you, Cameron. Good morning. Good morning to you. Now, tell us a little bit about your history with audio. You and I have known each other for a few years, so I've heard some of your back story, but tell us about all the years you've spent doing audio projects in security or unrelated to security, and then how you found yourself working with the security industry. Yeah, sure, sure. So my career started with Philips Electronics, the Dutch kind of large electronics company. I started working with them in Japan and then Hong Kong on audio products. So I was actually a product manager for their very first MP3 player back in the 90s. I was then transferred to Silicon Valley and I worked for their speech recognition group. That's kind of where my audio enthusiasm changed over to kind of the recognition of audio, but then specifically for speech recognition. And at the end of that kind of, actually we sold off that business to another company, I was introduced to Sound Intelligence, which was a spin-off out of the University of Groningen in the Netherlands. And the main thoughts behind Sound Intelligence were to see if they could use technology to improve speech recognition, to separate sound sources. But actually by the time that I kind of got on board, we had a few customers asking for very specific security related topics like detecting aggression in people's voices. So that's kind of the history and from there we've developed into supporting many more sound detector. Very interesting. And I think a lot of folks in our industry are familiar with video analytics. That's more commonplace now and we've seen a rise in using video analytics for temperature detection for things like COVID screening in the last few months. But audio tends to work very differently than video will. Talk about, if you would, some of the types of sound analytics that you've developed with Sound Intelligence, some of the products that you offer and how they're different in terms of the application that some people might be familiar with from video. Yeah, sure. Now, like I mentioned, aggression detection is kind of where we started in security. So it was the city of Groningen where we were founded, the Justice Department in the Netherlands and actually the National Rail Company who had a big problem with people getting aggressive against their employees. So they asked us, can you figure out and trigger an alert when there's aggression? And that's where we started. Over the years, we added all our sound detectors to that. So of course, bunch of detection is an important one that we've supported since I think about 2009. We added glass break detection to it, car alarm detection. And over the years, we've actually also looked at another vertical, which is healthcare. There's more about remote patient observation to recognize sounds related to epileptic seizures or sleep apnea or panic attack. And actually what's interesting is that you mentioned some of the COVID applications. Since 2014, we've supported coughing detection in our healthcare products. We basically help nurses at night time be alerted when a patient has a severe coughing situation that needs help. Obviously with COVID, people have asked us, hey, can you detect coughs and can you count coughs and tell us what's the baseline and alert us when the numbers are higher than that? So we have actually done that more recently in the security product as well. That's very interesting because you just mentioned five or six or seven different use cases and different types of industries that are all using sound in different ways. You mentioned the police, you mentioned train stations, you mentioned healthcare, which have wildly different needs when it comes to security systems. But one common element is that they like to use sound to detect these different events. Well, and it's like we as humans, right? Most people can hear and use their ears very effectively if there's an incident or even for other things. I mean, if something falls on the ground, you immediately turn around because you heard it first and then you watch. So traditionally security systems have been deaf or just been relying on the video part of it. But that means that it's very difficult to be very proactive with it. And of course, video analytics can be a great addition to your system. But we believe that audio is as important or maybe even more important for certain behaviors that you wanna detect. So you mentioned when you hear a loud noise, or you hear some kind of event, your head snaps around and you look towards it. And you can probably tell the difference between say someone slamming a dumpster closed and a gunshot because I've heard countless numbers of sounds through all the years of my life. So my brain can tell the difference pretty well. How do you go about developing an analytic based upon those nuances in different types of sounds so that your algorithm can tell the difference between this is just a loud noise and some kind of commotion that we can ignore versus actually being some kind of security risk? Well, and that is the magic, right? That's where we try to continuously develop better and better algorithms. And when we started, it was a bunch of really smart PhDs in acoustics that were listening to sounds and were kind of manually figuring out what are the characteristics of different sounds to filter them out and differentiate them from each other. Over the years, we've grown into using machine learning but then most recently, the last, I would say four or five years, we really moved to deep learning where you use complex, multi-layer neural networks that work on a very large data set of sounds to recognize these sounds. And basically, we don't even know anymore what are the differences in the sound that the algorithms use. It's basically the algorithms figuring it out themselves and that the input for that is like your example, like we as humans build up a sound library over the years that we live, we've over the lifetime of sound intelligence have been able to build up a very large database of sounds of all kinds of environments with which we train the algorithms. And that's really how it works. Now, the question that I always hear, and I know you hear as well, is whenever we deploy some type of audio application in any of the different types of installations you mentioned for healthcare, for law enforcement, for transportation applications, we're always asked the question, well, what about privacy? And how can we be sure that we're not listening into people when they don't want to be listened in on? For sound security purposes on purely the hardware side, we provide ways to give people notification that audio recording is taking place and the way someone might do that will vary based upon the type of facility that they have. Talk about the privacy implications of these analytics. If you're listening for aggression detection and you hear me make a threat towards you or towards someone, is that what the analytic is picking up on? How do you manage those privacy expectations? Yeah, sure, sure. I mean, it's a very important question, of course. We want to make sure that our technology is privacy friendly and is not intruding on anyone's privacy. So if you look at the total system, our system is just part of a piece of the puzzle, right? So you need the hardware, like the microphones, the cameras, et cetera, to pick up on the signals, whether that's video or audio or some other sensors. Then you need the analytics, which is our component. And then once we've detected a sound, it needs to go to an operator, a human to verify what's happening and we integrate with most video management systems or security management systems to do that. And if you look at the total system, the end customer is, of course, responsible for managing the privacy. If you look at our application, it's not interested in what people are saying. It's not listening for words. It doesn't do any speech recognition. Also because that is pretty impossible with the current state of technology, if you look at the way these sensors are being used. Now, I know that people are getting used to having your Amazon Echo with a microphone array that kind of filters out the background noise. But if you look at the systems that we rely on, there are only directional microphones that have difficulty in specifically filtering how certain sounds. Now, if you look at our application, it doesn't record any audio. It just analyzes the audio as a signal in real time. And if it detects a sound that is relevant, like a gunshot or a breaking glass or a screaming, it will send an alert to the security management system. Now that is up to the configuration of the security management system, whether it's important for the operator to hear the audio in case of an incident. And some customers say, well, no, we don't want to present the audio. We will just use the video to verify the incident, which can work fine for many customers. There's other extremes where certain customers actually record audio continuously, which we would not normally recommend. It's something that you want to be very careful about. Technically that's possible, but you want to be extremely careful. We would not recommend that. And it's not necessary for our system to work. But there's something in the middle, which we call the buffer mode, which is where in the individual management system, you set it up and it will buffer a few seconds of audio, which will automatically be deleted if there's no incident. But if there is an incident, it will record that video, sorry, that audio alongside with the video so an operator can verify what's happening. And if you look at the privacy aspects of that, people who start shooting guns or who are starting to shout aggressively, they can have no reasonable expectation of privacy. So recording a buffer of a few seconds before and after a detection is legal in all 50 states of the US. You mentioned a handful of different technologies in there to coincide with the privacy applications, notably Amazon Alexa or EchoDots, or if you have any kind of smartphone technology that uses voice commands to activate the closing your windows or turning on the lights. And those are microphones that are always on. Interestingly though, you mentioned the omnidirectional aspects of microphones that you used and the fact that these microphones don't filter out noise. How important is that for the performance of the application to either have just a specific frequency that you're trying to listen in for or to have a broad spectrum of all the ambient sounds, which the algorithm then goes through and decides whether or not to alarm. Well, you make a very good point. You see that depending on the application, you want to choose the right hardware. So, I mean, of course there are situations where audio analytics is not an added value and maybe you want to use your microphone purely for recording audio in, I don't know, interrogation rooms in police department. For that purpose, the pickup range of the microphone should be, you know, say between 50 hertz and maybe 4,000 hertz because that's where human voices are present. But if you look at the sound of glass break or gunshot, there's actually a lot of information above 4 kilohertz. So for the purpose of audio analytics, you need a microphone that picks up sound also higher than 4,000 hertz. And that's kind of important to choose the right components for the application. That's a, I think a key explanation on the differences between security technologies and what you might find in different consumer electronic devices and why one doesn't necessarily work with the other. We're going to take a quick break here and pay a few bills. Stay tuned and we'll be right back with Derek VanderVorst of Sound Intelligence on this special edition of Security Matters. Welcome back everybody to a special edition of Security Matters. We're here with Derek VanderVorst of Sound Intelligence talking about all things audio analytics, what the industry in security is learning from them and different deployments. Derek, give us a little bit of the history of sound intelligence. You mentioned your work experience with Phillips electronics and then growing into sound intelligence and how some of the different analytics were designed by customer requests from law enforcement or transportation companies or even medical centers. But talk about the growth, if you will, of sound intelligence. It's a company based in the Netherlands. You recently opened up a North American office in Chicago. Tell us a little bit about how the company has evolved over time and what you see in terms of different applications between Europe and North America. Yeah, sure, sure. So, you know, when we started, I think we, looking back, we were very early to the market. We had technology. We thought it was, we found an application for it that would be appealing to customers. And I think, you know, in the end, we were right, but it's taken a while before the market also picked up. So I think the first few years, you know, our focus was to obviously develop the technology, get it to a certain level where you filter out as much as possible the false positives and get a robust system, have a deployable and scalable system, set up the organization to support your customers. And of course, in parallel, build up your reference customers. And I think, you know, over the years, we've gone from a, you know, a technology push, if I'm honest, because nobody had heard about sound detection before to a situation where now we are, you know, we're seeing a lot of demand in the market. So we see a market pull now where, you know, customers ask for this technology in RFPs. We're getting interest from all over the world. And we see that, you know, the growth is, you know, has really the last couple of years has been pretty impressive. So going forward, I think, you know, we're at the point where we are, you know, we're able to really scale the technology, deploy it in thousands of locations. And that fits really well with what we see happening in the market. It's interesting you mentioned that the demand for this technology has been accelerating over the past few years. And for our purposes at SoundSecure, you know, we see the demand rising not just for the security industry, but also being driven by consumer electronics and the different types of applications that those include before the break about the differences between what you offer for sound detection and something like an Amazon product or a speech-to-text product. But the consumer electronics inclusion of audio in all devices, every smartphone, every laptop, I think increases the expectation for security systems that sound will be included. Are you seeing that driving the demand or is there some other reason that you're seeing audio analytics included in RFPs today? I think it's a combination, but I don't know if you have an iPhone, but people who do have an iPhone and that received the iOS 14 release, actually Apple included sound recognition in their latest update. So people who are maybe bad of hearing can be alerted when their dog is barking, their baby is crying, or there's no blind beeping or the doorbell ringing. So a different application very specific to consumer use, but of course, Apple including it in their devices shows that sound detection is becoming mainstream. So absolutely, the developments on the consumer side, I think have a big effect also on us here in the professional side. Now, where do you see the future of this analytics technology going? Is it driven by our industry? Is it driven again by consumer electronics? Are there new types of analytics that you're looking to develop and deploy? If you were to put the crystal ball on the table and gaze five years into the future for sound intelligence, what would you see? Well, I mean, there's a lot of opportunity there. I mean, I think the first thing is indeed scaling this application. There's still so many schools, hospitals, prisons, corporate offices that have issues with workplace violence, with aggression, and of course, the threat of gunshot is very real here in the US specifically, and that having a system that will alert security staff very quickly so you can respond much quicker and potentially save lives, I think that's a key advantage that we can offer. If you look ahead, I think some of the new technologies of course, revolving around AI and deep learning will really increase further the accuracy of the systems. One new technology that we've introduced recently is multi-sensor awareness, where multiple sensors will actually communicate between each other in kind of a mesh network in order to improve accuracy of the systems, eliminate duplicate detections, and also pinpoint the location of the alert better. And I also believe that in five years from now, sensor fusion will really happen. So where it's not just the audio signal that's being analyzed by the neural networks, but it's the combined video and audio and maybe another sensor or multiple sensors being analyzed in parallel to start automating responses. I think where we are now in the industry, human verification is essential because there could be a false point, right? And you wanna make sure that you give the operator all the information he needs to make a decision of what the appropriate responses. And if it was a bunch of kids pulling around that triggered the aggression detector, once you review that, you can say, okay, the correct response is to ignore it. But if it's somebody shouting at it, like a customer shouting at a staff member, obviously you wanna send a security guard. I think in five to 10 years from now, those responses can be more automated where maybe even robots could play a role in that to automatically respond to certain instances and de-escalate situation. So there's a lot of opportunity if you look at the technology and the market for our current solutions. But then there's, I mean, we of course have plans and dreams in other areas. So you see that sound detection can be used for many other applications that we haven't actually explored in detail yet. But you can think of noise pollution and you can think of traffic monitoring and you can think of condition monitoring, listening to machines if there's a defect. And you can think of maybe farming, listening to animals having certain behaviors that big farmers wanna be alerted about. I mean, there's many applications for this. In five to 10 years, I do believe that we will have explored a few of those and proven that technology can also be helpful in other cases. You mentioned the faster response times and let's get in the time machine and go back to the present day from five years out. So today in numbers, different numbers of applications you might have someone or rather a algorithm listening for gunfire or listening for aggressive voices or breaking glass. And you mentioned how that can give you the algorithm can give you a faster response time. What are some of the guidelines or recommendations or best practices that you've shared with customers to make sure, one, that they can get that fast response time, but also two, you mentioned false alarms. How can we be sure that we're not responding to false alarms or calling first responders for something that is in fact not a security incident? Well, and I think our vision is that there's four main actors of death. Well, I guess I can add a fifth one which is kind of a starting point. A lot of people are focused on eliminating false positives. And while of course that is our goal and our engineers back in the Netherlands work on that every day, it's not the only thing to focus on because theoretically you can eliminate false positives by accepting false negatives. But that means you're potentially going to miss an actual event. And that's we believe far worse than having the occasional false positive. So if you look at, when is a false positive a problem? Well, it's when there's too many false positives and it's a real nuisance. Or when you automate responses and you raise anxiety levels because you've just locked down a whole building and played a message over the emergency notification system that there's an active shooter while it's actually a false positive. So we believe that with our technology, the deep learning will help eliminate false positives as much as possible. Then the multi-sensor awareness where you train the late between different sensors will further help. The third part is integration into the existing system. So your existing video management systems are existing processes. And the fourth part is training the operators that they know what to expect and know how to verify an alert and what actions to take. So working with standard operating procedures based on the different applications, I think that is really where, which only needs to come together to have a good system. Well, just like any kind of video technologies using AI or using machine learning, this isn't a plug and play type application. From what I've seen and from what you've described, you wanna make sure that the system is configured properly, that it's configured for the environment you're putting it in. So a corrections facility is gonna have very different acoustics than say an office building will. Yeah, I mean, so if you look at our software, I would say three to five years ago, that was definitely the case. I think now with the latest version of the software, it will actually learn from the environment in the first week of deployment. And based on that, the system can be set to the right settings for that location. So it has become a lot more plug and play. And I think there we will see more advancements in the future as well. Fantastic. Derek, with just one minute left, tell us what you're hoping to accomplish with the CIO working group for audio and how you see the analytics offering that sound intelligence provides fitting in with the overall security landscape. Well, I think it's really to support this initiative and to support people getting more out of their security systems. I think many of our customers have invested thousands of dollars into cameras, video analytics, maybe panic buttons, things like that. But still, if there's an incident, there's no guarantee that they will be used effectively. Well, I think educating people how sound can play a role in that total security system is, I think, part of what we're trying to accomplish. And of course, supporting that process of educating the market. And of course, hopefully selling a lot of sound intelligence licenses in the process. Of course, of course, we'll see what we can do together. We're gonna wrap the show. My grateful, excuse me, my deep appreciation to you, Derek, for being our guest here today on a special episode of Security Matters and my gratitude as well to Andrew Lanning for giving us the opportunity to appear on his show today. Andrew will be back next week, taking the reins back over. But for today, I'm Cameron Jabdani with Derek Vanderhorst. Thank you for tuning in for this week's special edition of Security Matters. Mahalo.