 NanoEdge AI Studio is a search engine for machine learning libraries. It provides any developer with a simple way to start embedding smart features into any C code. This tool takes as input minimal amounts of data and as output provides a NanoEdge AI library, which is optimal for a given use case. Today we will get familiar with the studio by creating a new anomaly detection project from scratch. We will use the STWIN board to monitor some vibration patterns on a 3 speed USB fan. Here is the studio's home page where you can create new projects out of the four available types. Below is the list of all available projects. On the right side, the inspiration section lists many use case ideas which link directly to ST's use case explorer. There you will find out how NanoEdge AI Studio performs on these datasets which are all available to download for free. There are four types of NanoEdge AI projects. Anomaly detection which provides a dynamic model that learns patterns incrementally and infer both directly within the target microcontroller. One class classification which is used to detect outliers within data is especially useful when no examples of abnormal behaviors can be provided on the system. N class classification which enables automatic identification of a machine state among many different possible states. And extrapolation which uses mathematical regression models in order to estimate a target value using other known features. Additionally, the data logger feature provides a quick and easy way to configure your STWIN data logger. Simply connect your STWIN and STLINK to your PC as indicated and click load binary. Then click configure data logger and select the sensors that you wish to use for data logging as well as their configuration. Finally, click download configuration to get the deviceconfig.json file which needs to be copied to the root of your STWIN's SD card. You're now ready to log some data onto your STWIN's SD card in datformat.dat. Of course, this is just one way out of many to log some data. For instance, today we will not use this feature but instead a more standard serial data logger procedure. Let's get started with the creation of our smart vibration sensor. Here we have a very simple 3 speed USB fan that will be used as a basic model to simulate an industrial vibrating machine. We also have the STWIN's sensor tile board, the reference is stivalstwinkt1b. This board features many different sensor types but today we'll only use the accelerometer. We will just place the STWIN on top of the fan and secure it using some blue tack. At the moment on the STWIN we have some very basic data logging code, there's no intelligence yet. The logger reads raw accelerometer data as XYZ instantaneous samples and then formats it into a buffer which is nothing more than a collection of many XYZ samples which we also call a signal example or a learning example. And finally it outputs this buffer or signal example directly to the serial port or USB. This serial stream of buffers can be read through the terminal and of course we will use it as an input to provide data to the studio. Here we're using a buffer size of 256 which means that each line is composed of 256 XYZ samples. The sampling frequency used on the accelerometer is 833 Hz. It means that each buffer represents a temporal signal of approximately 300 ms which seems to be sufficient to capture the essence of the fan's vibration patterns. Let's create a new anomaly detection project. In the first step we define the general settings of our project. We'll call it fan vibration. Then we select the STWIN in the list of available boards. The NanoAJI libraries are of course compatible with any STM32 board with a Cortex-M microcontroller. And we choose our sensor, a 3-axis accelerometer. The NanoAJI libraries are completely sensor agnostic meaning that we can use any sensor type or even a combination of several sensors. For instance we could have used a combination of accelerometer, 3-axis, gyroscope, 3-axis and temperature sensor, which is only one variable, for a combined total of 7 axes or 7 variables. For this we would have selected a generic sensor with 7 axes. We could also decide to restrict the maximum amounts of RAM and flash memories that our library will use if we have specific constraints in our application but today it's not the case so we'll just leave it as it is and even increase the maximum amount of RAM to 32 kilobytes because we don't want to limit the search space in any way during benchmark. Then we move on to step number 2 where we will import signals representing the normal behaviors of the fan. To add signals we can import from 3 different sources. From file in text or CSV format. From an SD Win's data logger's SD card in that format as explained briefly earlier in which case we select a sensor, we choose a buffer size and we import the data. Or finally we can choose serial from USB which is what we'll do today. We plug the SD Win, select the right port, the BOD rate and turn the fan on at speed number 1. We'll consider that the normal behavior of the fan is when it is running at speed number 1. Then we can start collecting signals. We'll stop after we've got approximately 30 buffers. Here for the simplicity of the demo we only collect data representing one single mode of operation or regime on our fan which is speed number 1. But of course we could gather data representing all the different states that we'd like to consider as the norm on our machine. It could be for instance signals representing all speeds not only 1 but also number 2 and 3. In that case we would just concatenate all signal examples representing all the normal regimes in one single nominal signal file. Then we click continue and make sure that all data looks fine and click import. The signal screen summarizes the contents of the data we've just logged. On the right hand side we have previews of the imported data with both temporal and frequency plots or FFTs for all sensor axes. So here we have the X, Y and Z accelerometer axes. It is possible to use a manual filter if we'd like to remove some of the unwanted frequency components within our signal. Just click filter settings, activate the filter, then input the sampling frequency used. Here we are at 833 Hz and choose the low and high cutoff frequency values. For example we may want to remove all frequencies above 200 Hz like so and we can see how our changes are reflected on the FFT plots. But today we don't want to filter frequencies so we'll just deactivate the filter. Then we move on to the third step where we import signals representing abnormal behaviors on our system. We follow the exact same procedure as before but this time we'll change the behavior of the fan so that it functions in a way that is unusual. For simplicity we will simulate this abnormal behavior by changing the orientation of the fan by 90° to introduce abnormal vibration patterns. We turn the fan on and collect approximately 30 signal examples. The signal examples we provide at this point are not used to train the library, it is very important. They are only used to provide some context for the automatic search that will happen in the next step because all the learning will happen later on in the microcontroller after the library has been embedded on the target machine. The examples of anomalies provided here don't need to be exhaustive. For a real use case for instance they could be collected on a faulty machine or in conditions that are known to be abnormal. Or depending on the case they could even be created artificially. When testing a library we will see that in fact we are able to detect anomalies that were never seen before because the embedded learning is what makes NanoAGI libraries, especially for anomaly detection, particularly adaptable. Let's import these abnormal signals, check our graphs and proceed. Here we get to the fourth step, benchmark. We click run new benchmark, select our couple of signals, both normal and abnormal, the number of cores we wish to use on the CPU and start. At this point the studio will start working hard to find the best possible library for our use case given the signals that we provided. The NanoAGE library is composed of three main blocks. First there's an optional signal preprocessing algorithm. This could be very simple such as offsetting or normalizing or more complex like FFT or PCA and so on. Second we have the choice of the machine learning model. In the pool of available models we have classics of machine learning such as KNN, SVM, neural networks and so on. But we also have more specific models developed in-house and patented by ST which were designed for microcontrollers and are therefore extremely well optimized. And third we have a set of hyperparameters for the machine learning model so that it can solve the problem at hand as effectively and efficiently as possible. So for each library made of these three blocks, preprocessing, machine learning model and hyperparameters there are many possibilities which results in millions of possible combinations and millions of possible libraries. During benchmark each candidate library is tested using the couples of signals imported previously. We learn the normal signals and then run inference on both the normal and abnormal signals. This whole process happens many times for each library using random subsets of the data provided to minimize any possible bias. The results are displayed here on this graph. On the x-axis there's the signal number and the y-axis is the percentage of similarity which is the raw output of the inference or detection function. Of course we'd like all the blue points which corresponds to the normal signals imported in step 2 to be as close to 100% similarity as possible. And we'd like all the red points, the abnormal signals from step 3 to be close to 0%. The most important performance indicator for the anomaly detection candidate library is the balanced accuracy which translates the ability of the library to correctly attribute the input signals to the correct class, either normal or abnormal. 100% accuracy would mean that all the blue points are above the 90% decision boundary, the gray dashed line here, and all red points are below. The second performance indicator is confidence which translates how much mathematical distance the library is able to put between the normal and abnormal signals. 100% confidence would mean that all the blue points are at exactly 100% similarity while all the red points are exactly at 0. Finally, the amount of RAM and flash memories are also optimized. Previously in project settings we put a constraint on maximum RAM at 32 kilobytes, but here we see that the best library found so far only takes much, much less. The benchmark process can take anywhere from a few minutes to a few hours. Here we've already obtained excellent results so we'll stop the benchmark manually. Now we have our best nano-HI library, which is the only one kept by default, but we can also access the list of the best candidate libraries found during benchmark by clicking here. We may select another one, for example one which has lower memory footprints, despite slightly worse accuracy or confidence. Before testing our library, let's have a quick look at the functions that it provides. For anomaly detection we have the initialization function which needs to be run first, then we have the learn function which is incremental, dynamic and can be called at any time, and finally the inference function detect which outputs a similarity percentage which measures how much any given signal matches with what was learned before. Now we can test our library using the emulator. Each library has its own emulator which allows for easy testing without the need to download, link or compile anything. We first initialize our library and then learn some basic knowledge. We'll use our serial data logger, turn the fan on at speed number 1 and start logging data. We'll take approximately 15 signals. Remember that anomaly detection libraries come untrained out of the benchmark. That's why we need to run an initial learning cycle first. Now that we have our reference knowledge, let's go to detection to see how the library behaves. Again we'll use serial data and here the inference results shows 100% similarity which was expected since our fan is still running normally. Now if we start tapping on the fan, the similarity drops instantly. Same if we change the fan's orientation and back to normal if we leave it alone. Our library is behaving as expected so we can move on to the final step, deploy. Here we have a few developer options available. For the STWIN we only need to check the first one. And then we can click the compile button. We will get a zip file which contains the nano-HAI library which is a static pre-compiled C-language library, here libneai.a as well as the header file nano-HAI.h. We need to import both of these into our favorite IDE. Here I'm using stm32cube IDE and I will recompile my STWIN to transform it into a smart device and we'll be ready to test it. Here's a quick overview of the code that we've prepared for our device. These are just snippets of sudo code. It is not full working code. First we initialize the library. Then we run the initial learning phase with only 15 iterations. Each time we get fresh accelerometer data and pass it to the learn function. After that we have the detect function which runs in an infinite loop as soon as the learning is over. Here we take the inference output, similarity, and if it's higher than 90% we say that everything is normal, otherwise under 90% we say anomaly. Of course this is the simplest possible logic and we could have coded something much more elaborate. Now that the STWIN is flashed and the device ready, let's see how it works. We turn the fan on, reset the board, learning is in progress, and finished. Everything is nominal and if we change the orientation we start getting anomalies. And back to the original states, we're back to nominal. Now let's introduce a more subtle anomaly which was never seen before. We'll simulate that with a piece of paper obstructing the airflow of the fan. When we approach the paper the similarity drops instantly and we get anomalies as expected. Note that here the piece of paper is not even touching the fan. Now the most important aspect of the NWJI library is its adaptability due to the embedded learning. For instance if we suddenly change regimes and decide that this is the new norm, we just have to reset the board and learn this instead as the new norm. And we get a nominal result. If we change back to the normal orientation, which we haven't learned yet, as expected the library returns anomalies. Here we could imagine more complex learning strategies, for example learning multiple regimes or enabling the user to start a complementary learning phase with the press of a button, for example. So hopefully today we've demonstrated how easy it is using NWJI Studio for any developer to start embedding smart features into their devices without any prior knowledge in machine learning or data science. And all that in less than an hour. For any information about the studio or the libraries, please refer to the NWJI documentation which is available on the STWiki.