 So, here we have Giuseppe Brocolo and he's going to talk about data science with Python. I hope you enjoyed it. Can you hear me? Okay. So, yeah. I'm Giuseppe, data engineer in decibel. I work mostly designing data pipelines. Just to introduce the topic of the talk, let me quickly introduce one of the most efficient pipelines present in NATO is the enrollment of a DNHA. It's one of the most efficient process from data pipelining point of view, but if you consider the speed of the enzyme involved in this process, the process will take 20 years. So the secret is that actually the process scales, so actually it takes 20 minutes. So this is a link to the repo presented in this talk. So I invite you to have a look directly to the code if you can, also because generally my code skills are better than my presentation skills, so if you can, please do it. So this is the outline of the talk. Essentially, I will present the requirements needed to deploy a good service in scale. I will present a simple prototype that needs to be deployed as a service in scale. I will present the roadmap, so all the steps needed for this, focusing of course on all the changes needed from Python point of view, and then just conclusions. So there is a document, it's called the 12-factor document, it's essentially a document prepared by Roku guys, where they explain in 12 steps how to essentially deploy a successful service in scale. I'm just summarizing here some points, so essentially just to present the situation, or essentially a good application that works in scale should be a stateless application. It should work asynchronously, this is generally actually something that is good always, not just to work in scale, should ingest input data as more as possible, not like files but more HTTP requests, even better if consuming a streaming like Kafka streaming and so on. We should avoid any persistence in file system, for example, configuration should be passed as environment variable, for example, also temporary artifacts, we should avoid them as more as possible, prefer in memory execution, in case this cannot be avoided, okay, rely on some backing services like, I don't know, some caching service, DB, cloud bucket and so on. Of course, one of the most important point is the portability and the easy deployment of the processing units, so Docker here is a fundamental technology for this. So the scenario, essentially let's consider a case when data science team deployed, sorry, has found a fantastic algorithm just to detect, for example, the anomaly of a Gaussian time series. The idea is to transform this in a responsive in scale service that ingest the time series via HTTP request. I'm reporting here just an example, for example, of HTTP request for this where the time series is provided in the body of the request itself. This is an example of code, it's really simple, I mean, it's just a simple function that rely on a moving window, essentially where it computes the average of the time series and essentially detects the anomalies through the computation of the standard deviation. Just NumPy and pandas are used as dependencies. So now we present the roadmap as promised in the abstract of the talk, five steps. The first step is to define an importable model, so transform the Jupyter notebook, for example, of the prototype, consider if Jupyter has the environment where the prototype has been developed in an importable Python module. So the idea is to create a package, so as you know, so define some init file where to include all the initialization of the package, for example, put the code as it is in a Python module, maybe checking, of course, the syntax because generally when we prototype the code is not always clear, so just taking a bit care of it, some flake eight check could help. And then you set up tools to create the module, put all the dependencies in the requirements file, and you can find all the more details in the link I presented at the beginning if you want. So the idea is this one, you can easily import the module, installing with PIP, there is no need of the source code in this way, you can just easily import the module in the environment essentially. So the next step is to expose a responsive point. This is actually a well-known pattern in Python in order to create microservices, so essentially using nginx and uwsgi with flask to expose the code, there are a lot of references in the web, so I will not go into details in this just, I would like to focus on the flask application, for example, so here you can see the ingestion of the data, so all is made in memory and is passed to a pandas data frame that can be used for the computation of the, for the detection of the anomaly and passed to our module essentially. So the third step is to make it asynchronously, consider that, for example, it can take sometimes for the processing, so for this we are using salary. The idea is to reduce the impact to the request as much as possible so in order to avoid just to send a request and wait for the response, so the idea is okay. As far as I'm able to collect the data, I will send a request and then I will, in a second moment, I will get the response asynchronously. So for this, okay, the idea is to use salary, so salary worker is essentially a scheduler of a synchronous task, the idea is to rely on a backing service, in this case we will use Redis, for this the idea is to submit the task into the Redis broker, the salary worker is able to get the task, process it and put the response in Redis itself again for the, to provide to the client the response essentially. So this is the change needed in the supervisor, I mentioned before supervisor is used to handle all the tool used inside the part of microservices, here there is the configuration of NGNX with WSGI and I added the configuration of the salary worker. Here is the change in the flask application, so we have to configure the salary scheduler and configuring also the Redis cache for the backing service, I use Redis as a secondary cache in order to store also the task ID of the submitted task, sorry for the gain and here is the configuration of the asynchronous wrapper that executed our method, the method that we implemented for the anomaly detection. So here is the change in the endpoint, so the endpoint is doing a sensory to handle both post and get request in the post, essentially it's really similar to the previous one, we ingest the data and we essentially pass to the salary wrapper. We have to pay attention that salary is not able to ingest not adjacent serializable input, this means that we cannot pass the pandas data frame as input, we have to serialize it into a JSON, so this is done with two JSON functions, for example here in pandas and that's all. So from the get point of view, the get has to just read the get, thank you, has to read the Redis cache in order to check if the asynchronous task have ended, have finished, so checking also the status of the task and returning eventually the output in the case it concluded successfully. Of course it has to handle all the secondary cache where the task ID are essentially stored, so the idea is to check for first the oldest task submitted, essentially. Of course it needed a change also from in our module because now we are passing a serialized input, so the idea is that okay, data scientist, we're thinking to work with pandas data frame so just give them back a pandas data frame as well, so the idea is to deserialize the input that is resized in this way. So the first step is to use Docker, do you know Docker, how many of you know Docker? Okay, a lot of people, fantastic. So it's a wonderful tool essentially, yeah. So the idea is to create, Dockerize our service into two containers, I will explain why I dedicated the container for the backing cache and of course define the container of our main application that includes all the nginx and all the Python code and the flask application and the seller as well. This is the Docker file essentially, I'm starting from an image that we have released is public you can use if you want. It's essentially an image that already contains nginx, UWSG installed, flask essentially and all the pandas installed in an alpine environment. So essentially once we have this image prepared, of course this image is inherited from an already existing image, this one. And the idea is to of course put our code and put the flask application here, just use PIP to install the module and that's all, essentially. Okay, if you want to test your Dockerized application, you can just use Docker compose for this, this is the configuration of the Docker compose to test it, essentially it deploy to services. So our main application exposing port 80 for HTTP request and the backing service based on Redis. So the idea is to part of course the Redis cache is configured through an environment as I was mentioning at the beginning, consider the 12 factor document and when inside Docker, the idea is to create a Docker network and within the Docker network the container just resolves through the name of the services you give, essentially. So once you deploy it, essentially you have a service ready to serve listening to port 80, essentially, where you can submit the HTTP request I presented in one of the first slides. So the last step, one you have verified that Dockerized service work is to, okay, just start the deploy in cloud, for example, using Kubernetes, so this is the engine that will be used to create our service in scale. So how many of you know Kubernetes? Okay, okay, a good number, a good number, okay, okay. So what is Kubernetes or there is a lot to say about it, probably I have to study a lot to know it very well, but just to give you some input, it's a way essentially to create dynamic cluster in terms of working units, so the idea you can define a cluster that can auto scale horizontally in order to provide the processing unit ready to ingest your data, your request, and so on, essentially. There are several cloud providers that provide Kubernetes, thank you, five minutes, that provide Kubernetes as a service in Google, in Amazon, and so on. So this is really good because in cloud it's really simple to just add workers, also physical workers to the Kubernetes cluster in order to scale and to deploy further processing units. So just to give some introduction to Kubernetes resources, I'm listing here one of the most important, not the only one, but one of the most important elements in Kubernetes. The most important part is the pod, the so-called pod that is the processing unit itself. Essentially it's an elementary unit that includes one or more containers, essentially the whole service that we, for example, the two containers that we defined before with the Docker compose, for example. Then you have the deployment, the deployment define how the pod are deployed in the cluster, and then you have to define a service so how the deployed pod are exposed to the rest of the world, so how the requests are ingested and so on. So you can find a full example in the link I presented at the beginning of the talk, and then you can define the so-called horizontal auto scaler, so you define how the processing unit really scale based on the request they receive. So here there is, without entering too much into detail, so this is, for example, the configuration of the processing units of the pod, the idea you can define, it's really similar the configuration to the Docker compose one. So you define essentially the two containers that deploy the service, essentially. The idea is to, okay, the image, generally in cloud, you don't build the image, you just put the image in some registry in the cloud itself, so you can put the already built image. You pass, essentially, you are able to resolve the ready sketch just as a local host, because inside the pod, all the containers are defined as a local host request. You can also define a limitation about the amount of resources that a pod can use, both in terms of memory, for example, or CPU, like listed here. In this approach, there is a problem, because, okay, just deploying a single unit where the backing service and the processing unit are together in a single pod can be fine in a static case, but in case of a scalable cluster, this is a problem, because generally pod are destroyed as far as they are not used anymore. It could be that you weren't able to get the asynchronous response in the meantime, so this means that your process has been lost, essentially. So the problem is that the cache in this case is volatile. To avoid this, you have to the couple, essentially, the processing unit and the ready sketch into two distinct services, actually. So in this way, the ready sketch is persisted. The idea, you can define the so-called cluster IP service in Kubernetes that it's a particular kind of service that accepts the request just within the cluster itself, and this is the change of the configuration of the pod of the processing unit. In this case, you have to change how the ready sketch is resolved, so in this case, it's resolved with the name of the service. The pod in Kubernetes for the ready sketch. So essentially, I presented all the five steps here. There are room for improvements. For sure, this is just a stupid example, but just to explain how to play a bit with Python in order to achieve these kind of services. As I mentioned before, the best would be to consume streams and publish in streams as well, the output. Important, we should make the backing service of all the ready sketch more reliable. This means not just creating a single ready container for it, just creating a cluster for some HA. And of course, one important thing also would be to the couple, actually, all the part of the salary worker from the ingestion of data, and make scale just this part, actually, because this is the part that needs to be scaled. Good, so I concluded, exactly, perfect. So I concluded, this is just a summary of the step we made together. Question? So does anybody have any questions? Nothing? One there. Good. More of a comment, really, and thanks for the detailed talk, by the way. I will phrase it as a question. I will phrase it as a question. I don't know how we've been very strict in that format. I apologize. Okay. Have you heard of ASCII, the asynchronous version of whiskey, essentially, and would that be more useful in this context than whiskey, particularly if you want to stream the HTTB response? I think you need to put the microphone closer. Yeah, no, I didn't understand the question. Sorry. Have you heard of ASCII, ASGI, there was a talk about it? Yeah, there was a talk before. Unfortunately, I wasn't there. I didn't know it before. Probably I will have a look for sure to it. I don't know. One thing that I like of Celery is that, of course, here I presented a case where the Celery worker was deployed locally with the Flask application, all the rest of the code within a single processing unit, but the idea is also to create a cluster of workers. So this is something that I like because it's really easy to deploy and it's really easy to deploy. So I will study for sure. I didn't know that kind of framework, so if it allows something like that, perfect. I have found an alternative to what I was knowing. I was citing Celery here before because it's just well known in the Python, of course. But thank you for, yeah, thank you. Any more questions? Anybody? No? Okay. Then thank you, Yosepe, for the talk. Thank you.