 Hello, hello. Okay, perfect. So, hello, everybody. Good afternoon. My name is Petr Stukyunger. I am an application support engineer from ST Microelectronics. And I have a pleasure to present to you a presentation speaking about industrial motion sensors with embedded processing unit. So, in this short presentation, I will show you several options how to embed programmable logic inside IMU. I will tell you details about one of the products that we have with programmable logic. And finally, I will also show you a couple of examples, what kind of applications, what kind of algorithms can be embedded directly inside the sensors. So, you may have been asking, why do we need processing inside a sensor? And the answer is that we need it to decrease, to minimize power consumption of our systems. So, in recent years, we see more and more people, more and more customers coming and asking, how do I implement my system with minimal current consumption? And what we do, we are implementing sensors with processing cores, with processing logic, which allow to move processing from the sensor, from the MCU inside the sensor. So, from traditional way, which we see on your left, so moving from processing from the microcontroller inside the sensor itself. This way, we are able to decrease overall system power consumption, and we are also able to minimize traffic on the bus between the sensor and the MCU. So, what is happening when we follow the path, we started processing on the edge. So, processing as much sensor data at the source of the sensing, and this has been done in the past, and it's still done, of course, using MCU. So, inside MCU, we process the data and then we push them to the cloud, post-processed with some results. But, recently, we see more and more coming from on the edge to in the edge, which means to really go to the very source of the data, to go inside the sensor and to do all the job there. So, first step was programmable logic or is programmable logic. So, dedicated logic inside the sensor dedicated to process sensor data. This was the first step. Now, what we have is a really complete processing unit processing core, which is capable to run very sophisticated algorithms inside the sensor itself. And this is what I'm going to speak about. So, to compare the three options. Starting with the first one, so running the algorithms inside the MCU. There is a clear advantage of this solution, which is the flexibility inside the MCU. Whatever started from Cortex-M0 to Cortex-M4 or even more capable, you can do what you like as long as you have, of course, powerful enough processor. What are the benefits of using programmable logic? And I will explain in a few minutes what I mean by programmable logic. So, with programmable logic inside sensor, we have not so high flexibility because this logic inside sensor is dedicated to process sensor data. So, you cannot do all the stuff, but you can do a lot of sensing stuff. What is, on the other hand, really great with programmable logic is power consumption. It's dedicated, it's optimized, so processing sensor data with programmable logic is the best you can do to decrease power consumption. And finally, for the communication between the sensor and MCU. This is really, again, very, very good because we don't have to push all the data continuously all the time to the MCU. And finally, the programmable processing unit. Here, we have processing inside the sensor, but with very powerful core, capable to run sophisticated algorithms. I will give you examples, but let's say, for example, sensor fusion, such kind of things, we can now implement inside the sensors and run it directly with optimized current consumption. Not as great as the logic, but still very nice. And we don't have to push the data at all, we just do the computation inside the sensor, no involvement of MCU, only inform the MCU when there is the result available, when we have detected some new situation, for example. So, but let's have a look at the middle column of the preview slide, sensors with programmable logic, I would like to give you a concrete example. And this is an example of ST6 axis IMU, ISM-330 DHCX, where as it is 6 axis IMU, it is combining high performance accelerometer and gyroscope with very high accuracy. But it is adding to this sensing part also programmable logic, which we call machine learning core and finite state machine. And this allows us to implement the sensor in all ways on applications, so applications where you need to process the data all the time, the sensor is running all the time, the algorithm is running all the time. And thanks to the programmable logic, this is done inside the sensor itself. So the MCU or application processor is not involved all the time in the calculations. This is where we save the overall system current consumption. Remove the algorithm inside the sensor to offload the MCU, it can be in some low power mode. In addition in this sensor and in other sensors of ST, we can also combine them with external sensors like pressure or magnetic sensors. And also this sensor data, thanks to the sensor hub feature, we can read from the IMU and we can process inside the IMU. Again, decreasing the need of MCU to communicate and process data of these sensors. So let's have a closer look at these two types of embedded logic. So the machine learning core as the name suggests is logic that implements that is allowing you to implement machine learning inside our sensor. It is used to recognize motion patterns in sensor data. You can imagine, we can classify activities like walking, running or some more industrial like drilling, hammering and so on, using this logic. The logic is based on decision trees depicted here. So the tree is composed of nodes with a pretty simple architecture. Each node has an input, sensor input, sensor data input. The condition that should be examined and two outputs, two passes to continue the algorithm. So true sparse or false sparse depending whether the condition was met or not. We combine in the decision tree the nodes together and this makes up our algorithm. How do we do it? We use machine learning concept. So it's not that a person comes and starts to build the decision tree to connect nodes in some code. This is not the way machine learning is working. We are using the standard machine learning process where we need first the data. We need to collect the data using our sensors in the situations we want to classify. We label the data so we say, okay, this file belongs to walking, this file belongs to running and so on. And then the building of decision tree is done using machine learning tool. It can be VECA tool, it can be MATLAB, Python. There are many different tools that are there for a long time. So any such tool can be used also when you are creating an algorithm for MLC. And as a result, we get the decision tree which we program inside our sensor and it starts processing the data and classifying the motions. Then the second type of programmable logic we have in our sensors is finite state machine. It consists of states where each state can be either condition or a command. Condition is basically checking sensor inputs versus some thresholds or it can be also some timing condition. Command is instruction to do something to generate interrupt to make a jump or so. So it's more closer to traditional programming. What kind of motions you can recognize? So with machine learning core, we can recognize motion patterns, as I was saying, running, walking or drilling and so on. With finite state machine, we are more targeting, let's say, well defined or motions that can be well described by just looking at the data. So this is an example of a motion we want to recognize. It's flip, turning upside down. This we can see from the sensor data very clearly and we can describe by combining several conditions together. Now let's come to the third pillar, third part of the way how to minimize current consumption of your system using processing inside sensors. This one is by using programmable processing unit. So we have MCU but we have sensor with processing tuning which is capable to implement really sophisticated algorithms. So what we need, we need to have really programmable core with ultra low power consumption so that when we move the algorithm there, it will decrease the current consumption of the system. We need to have capabilities to process sensor data. Of course, we are inside sensor. We need to process the data again, the internal or external sensors. And finally, we need to support developers because we want to have open system so we want to support developers in developing algorithms for the processing core. So we need to have two chain and some standard programming language which can be used to implement the algorithms. So what are the challenges when you want to design such a device, such a sensor? You need to have really optimal performance versus power consumption. So you have to justify what you do, how much current it will take. You also have to think about if you are what you squeeze inside the chip, what will fit on your silicon, which architecture you will use to implement what kind of, will it be RISC or some other architecture of processing core. And you have to think, as mentioned already about the developers, about the people who will in the end use your sensor for processing the data. So in ST what we came with, what we have is intelligent sensor processing unit. It's embedded processing core inside our sensors which covers, which is implemented on very small silicon area using only eight kilo, kilo gates in very small package. So the package that we are using is the same one as for our previous generation of six axis IMUs. So I have here a sensor with ISPU and here a six axis IMU. So six axis IMU with ISPU and six axis IMU without ISPU. Can you tell the difference? No, you can see it probably. But it's the same package, same size, same pinout. Only what is inside is different. And this is the logic which is there, which uses, which implements the processing core. So looking inside the sensor, what we find there is of course the sensing part itself, the MEMS element, the mechanical part implemented on silicon. In this case, implemented three axis accelerometer and three axis gyroscope. Then we have the sensor hub. So we are able to process not just the internal sensor data, but we are also able to read external sensors and to process the data. We have there the processing unit itself, ISPU. And all of this can calculate it embedded inside the standard package, which has the size three times two dot five times zero dot 83 millimeters. Having closer look on what is inside. I have mentioned already the accelerometer and gyroscope. We have also embedded temperature sensor. We are connecting external sensors and for the processing part, the ISPU core based on the risk architecture is running at five or 10 megahertz running the instructions. We have interface registers. So to talk to the external world because we still have some microcontroller to talk with the sensor. We have registers to exchange the data and we have memory to store the program and store the data. So for the program, we have 32 kilobytes and for the data, we have additional eight kilobytes of RAM memory, which case fully available for the user application to implement the algorithms that user is targeting. At the moment, we have two sensors with ISPU. One for consumer applications, a lesson six DSO 16 is and one for industrial applications, ISM 330 is. When we look at the sensor parameters, so really what is related to sensing only you see the data that you can be familiar with when you know ST sensors, because we have four accelerometer full scale of 16g for gyroscopes full scales up to 2000 DPS. We have current consumption of just 590 micro amps. We have very low noise for both accelerometer and gyroscope. What we have in addition is the ISPU. One point that I have mentioned, which is important when you are implementing a sensor with processing core is the tool chain for developers for the end users of the processing unit. So here we have two ways which you can take one way is using a C compile. So really the standard C that is there for many, many, many years. This is the programming language that you use with the ISPU. You can choose if you will use a command line or if you prefer to have a graphical ID to develop your program. Eventually you can also use ST's AlgoBuilder GUI for developing sensor applications. The second pass you can take is a pass of using AI. If you wouldn't like to develop AI algorithms, and this is one type of algorithms that was ISPU developed for, then we can offer Nano Edge AI Studio, which is basically a tool, PC tool that generates for you AI based learning models automatically. So without any deep knowledge of AI programming or creation of AI algorithms, you can use this tool on your PC where you will create the library, and then you will deploy it on the sensors with ISPU as many times as you like. And it will be basically running the algorithm, the AI algorithm. It involves learning phase and then the interference. So the final implementation of detection in this case of anomaly detection to be more concrete with Nano Edge AI Studio. So when you are using it, what you need to do after installation of the tool, you need data. So we are speaking about AI, about learning. So you need the data logs from which the tool will analyze the data, will visualize it, and it will provide you the best algorithm that suits your particular case, which will be generated. And then once you have the algorithm on the side, you implemented your programming inside the sensor with ISPU. You run the learning phase where the algorithm will learn the particular motor or particular fan will set up the embedded parameters for it, and will then detect any anomaly if it is happening inside your system. In the last part of my presentation, let me give you a couple of examples of what you can do with programmable logic and what you can do with programmable processing core. So speaking about programmable logic, one type was the finite state machine to recognize gestures or some motions. So we have several algorithms already available. This is why I'm putting here the examples. Nevertheless, don't forget this is open system. You can also define your own algorithms, which is true for finite state machine, MLC or ISPU. So the algorithms that we already have and that you can find on GitHub repository are targeting many different industries. It can be the wearable markets, electronics, industrial, and of course also other markets. In there for FSM, we have different gesture recognition algorithms. We have also algorithms for asset tracking for free for detection, etc. There are many algorithms available and you can always extend with your own algorithm. Then for the machine learning core. Just let me remind this is the programmable logic that allows to implement machine learning inside our sensors based on decision trees. So there we are basically having algorithms that classify motion. They are doing motion classification. Again, we have algorithms for different industries for personal electronics for industrial IoT or even automotive. And the applications are different. Some are targeted to monitor human to detect different activities to detect head gestures, for example, and so on. Some are targeting machines to check the orientation or motion intensity. And also vehicles, cars to detect whether, for example, if the vehicle is stationary or moving or if somebody is trying to tow to steal your car. Finally, for ISPU, the algorithms that are there are listed here. So we have several, let's say categories like sensor calibration. You can run sensor calibration inside the sensor. You can run sensor fusion inside. You can run frequency analysis using FFT or you can check human activities. You can also add CRC to your data when to make sure that the communication is not disrupted between the MCU and your sensor. All of this is really high efficient in terms of computational resources and power consumption. And again, you can customize, you can add your own algorithms as you like. To give you some numbers about current consumption, because I mentioned several times that this is aspect that is very important. This is the aspect why we are having programmable logic or programmable processing unit. So to give you some numbers, we are, as you can see, in the range of micro-amps. Whether it is finite state machine or machine learning core, different kind of algorithms, we are in really micro-amps range. Similarly for ISPU, the algorithms I have shown, we have implemented. We have tested. We have also measured the current consumption. So again, the numbers are in micro-amps. Of course, for some more demanding algorithms like sensor fusion, we are coming to tens and even hundreds of micro-amps. Nevertheless, comparing to power consumption of MCU doing the same job, we are still far, far below. This is the aim of ISPU. So let me summarize, let me conclude, which are the added values of ISPU. So this is the ultra low power consumption, just demonstrated on the previous slide. Ultra low latency because we process the data at the very source. We cannot be closer to the source of the data. It's very easy to use the processing core because it's programmable in C language, the most spread language. You can be sure about data privacy because the data are not sent over bus, processed locally. And the integration, if you remember, the same package for the IMU with ISPU. So thank you for your attention. That's all from my side. If you have any questions, feel free to ask. Okay, so thank you and have a nice day.