 Let's get started. We're beginning this afternoon's cloud and dead-off strike. So please welcome Nebren Padler, who is a student at the Indian Institute of Technology in Milwaukee. Yeah, you got it right. And Nebren has an interesting talk about Firefly. Microservices in Python for doing functions as a service. It's really interesting. Yeah, so hi. This is Nebren. I'll be speaking on building microservices with Firefly. So first, who am I? So I'm a student at Indian Institute of Technology in Durkha. I'm a final year undergraduate right now. And this Firefly framework we built last summer while I was doing my internship at Aurora Data. What Aurora data is? It is a startup which is aiming to build a platform for abstracting out a data scientist workflow, like deploying machine learning models, scaling them, provisioning them, all of that. And part of two major campus groups, one is Team Robocon ITR. We built autonomous robots, and we participate in a competition called AB Robocon, conducted Asia-wide. And I also conduct community events on robotics, like lectures and open projects. With our group, models and robotics section ITR. And yeah, that's some nasty things. So let's first understand what the problem is. How did Firefly came into the picture? What is it primarily meant to? So we have in our team one data scientist. Who knows? Really good machine learning statistics and all, which is the case with most every data scientist. But then when they have built their models, they have their code ready, they have their functional logic ready. Now, to test those out or to show them to someone, they can't just ship them elsewhere. And all the stored weights and all, what they could do is have an API out of that. But then that requires a lot of knowledge about how to make servers, handling requests, modifying the data, because the data can be binary, the data can be normal float integer types, whatever may be. We don't have any bounds to that. So Firefly aim to create a small ecosystem, where like you add minimal code to the existing code. You don't need to do anything and you get a server running out of it. You just have a function and you have a server running out of it using Firefly. Now, why? I like told you a gist of it. So what we can have, right? We have a local environment in our case. We might need to run functions in some other compute resources, better compute resources, like in we might need to use AWS instances or anything on GCP. Now, going on to the next point, we might also want to loose couple the functionalities in our system, like having microservices. Like we might not need to have monolithic architecture, like for a front-end server of a core functionality of an architecture, maybe we need some other authentication server. So such kind of things can be implemented in Firefly. I'll go on to how? Now, that is what I told already, the compute resources thing. Now, let's see what are the use cases. Already spoken, deploy machine learning models. You can easily deploy machine learning models. That also we'll come to it later in an example. You can run functional utilities remotely, authentication servers, model management, anything. Yeah, of course, build microservices. This was not meant to be the primary motive of building Firefly. This came as a side effect and we are using now Firefly to build microservices in our, in RodeRata's ecosystem. Now, let's see what are the challenges faced by a data scientist while building an API. What he needs to know about building APIs and deploying them from his prediction code. He needs to write a web application. He needs to learn something like Flask, Django, Bottle. You have to learn Flask or Bottle. You require a client library. Of course, you can't use requests for everything because you need to put the request in a certain way so that your web server understands and then it processes the request and then gets back to you. Also, you need to authenticate the users using your service or your API. Now, also, how do you validate the data sent by users? Of course, if you send any kind of garbage, your code might return value or attribute or whatever and then the server will exit. No error management. Now, why Firefly? Because Firefly deploying functions in a Firefly in a Firefly environment is very easy. You just need to write maybe three, four lines of code depending on how many functions you are deploying or how many features you want to exploit of Firefly. Then deploying predictive models, you just need to write one CLI command. Nothing else if you have one function. You can access everything in a RESTful API or in a GRPC compliant client. We will see how it really abstracts out things as you will feel that you are using everything in your local environment and not in a remote environment. We can say that it puts your Python functions on steroids because you get more computing resources, you get fast computation and all. Now, how do you install Firefly? Installation, we packaged it to package it and then uploaded it to PyP. You can just install Firefly in Python and get the release version or if you want to develop something, you want to contribute to the codebase of Firefly, you just need to clone it and then do an editable install in your virtual environment or your system, whatever VN or PyN you use. Now, this may be sort of a minimal code like squaring a number and then returning it. This is a very rudimentary example how we can go on to show how Firefly works. How do you run Firefly now? You can invoke Firefly using CLI. You just need to write Firefly space funs.square. Funs is the file name where the function was there and squires is the name of the function. Now, it runs a Whiskiref server. We will also see how you can scale it. So, how do you now access the function? Oh, sorry. You can also invoke multiple functions using multiple arguments like space separator, funs.square, funs.cube, whatever you want funs.predict or you can scale the application using GeoNicon because this is just a Whiskire application. So, you have a Whiskire application Firefly.main app and you can specify the functions using an environment variable funs.square. It would be a comma separated list. So, if you want to have multiple functions, you can have funs.square, funs.cube, funs.predict, whatever you want power. Now, how would you use the functions? You need to have some uniform method of using the functions. So, there are several ways. What we recommend is you use the inbuilt client when you install Firefly, you also get a client with it. So, you just need to specify the URL endpoint. So, the server endpoint or the URL resource of that specific function, you just need to specify to the client what server the Firefly is running on and then you can use it using like you are invoking a normal function. There you would have done square n is equal to 4. Now, you do client dot square n is equal to 4. But, how does it work actually? It is nothing but a simple restful API. You can also use curl to do the same thing, but we have abstracted out everything as a client so that it becomes more easier and more meaningful to you. The client supports any JSON friendly data type or any binary stream. You can even put in binary data to the client and then it will stream it to the server and then you get a request. Also, the response from the server can also be binary. Now, Firefly also supports what you call a config file, we call it Firefly dot it is a ML format. We wanted to call it Firefile, but then it was not that homogenous. So, there you can specify many things. This is the mandatory thing which you need to mention is a functions key where you have a dictionary where the key should be the function. It can be anything, it is just an identifier for your own purpose. And then you specify the path where the function would be hosted. If you do not specify it will take the default function name. For example, if I had not put in path slash square it would have automatically generated slash square from the function name. And invoking using config file is also really easy because you can just tell Firefly that this is my config file, get it and parse it and understand whatever functions are needed to put in whatever end point and then run it. Optionally, you can add two more arguments like name and version to name your API and use a version for your internal use. Version means nothing to Firefly as of now. And in Geonicon you put in environment variable where you tell where the config file is located. Now, as we mentioned there was a requirement of authenticating users. Right now Firefly provides a very naive basic authentication using tokens. When invoking Firefly either using Geonicon or the CLI, using the CLI you can pass in an argument minus minus token 4123 or whatever the token you want. And while using Geonicon you need to use an environment variable called Firefly underscore token. We are trying to add some more features like authentication, LDAP, universal authentication and all. We are getting to it now. Now let's see real life example of how a data scientist will use it. So here you see a simple code where you have the Fisheries dataset trained and then you pickle it and store it somewhere. On invocation it will be loaded once into the memory using JobLib and then this is a very simple function where you use a Cycletland classifier and it is pickled already. Now you unpickle it and then use it. So let's see how you can use it. You start Firefly using Firefly model to predict. You use the client for accessing the functions like this, import Firefly, remote model is equal to Firefly.client wherever the server is running. All the examples are like showing that. Now remote model to predict. Features is equal to, now in Fisheries dataset you have four input features. The sepal length, sepal width, petal length and petal width. And you get back what is the predicted class. So the dataset itself is giving I think 0, 1 and 2. You can have anything on that side. That's all your imagination. Now what is Firefly doing under the hood? That's one thing. What Firefly is doing is when you invoke like remotemodel.predict. So what it does is it takes the arguments and then processes it, packs it into a JSON. If it's a JSON friendly data type, it processes as a text stream. It's just simple post request with a JSON type content type, JSON content type. And then it sends it to the server. Now what server does? First it checks whether the endpoint that is requested is really available. If it's not available it returns a 404 in that case. Now if the endpoint is available what it does is it uses the function signature or the function that is specified by the user to see whether the arguments that are really passed in are the arguments which are needed or not. And then it processes that function and then gets back to you using an appropriate request. Like you can have validation error also, 4 to 2. You can have authentication error also. I think that's 403 I think. I don't remember exactly. Or it maybe will give you any success to 0, 0. Now client invokes the function, processes the inputs. It makes a request to the appropriate endpoint. Server checks whether the endpoint requested is exposed or not. Server processes the request and validates the inputs. And the function result is processed to either a JSON return type or a binary stream of the data. For example you can have a phase detector model also where you send in an image and get back an image, something like that. Or maybe you send in an image and then you use some descriptive model where you get how many people are there or what sort of objects are there like ImageNet or other image based convolutional networks. And then server responds to the client with the appropriate status code so that the client can tell the user that how did the request go. Also we have support for raising custom STTB errors. So what you can do is in your functional logic, if you are processing the data you see that there is something wrong with the data. You raise a Firefly.STTB error and that error gets passed on to the client and then you get an appropriate STTB error. Right now it's not truly REST rule so you can't have all those inbuilt in your code. So you have to use Firefly.STTB error for that purpose. Now being the primary purpose of like simplifying a data scientist workflow in deploying models, it also served as like building all of our micro-services. So you can go to RodeLite.com and see what our architectural system is and then you see that there are like 8, 10 micro-services running and everything is running using Firefly right now. And we built an open source tool called RodeLite. You can check it out at www.rhodet.com slash RodeLite slash RodeLite. So Firefly right now is meant to be like you need to go to the server and then access it, access the server and then invoke Firefly. You need to set up your DNS and all. Using RodeLite you just need to provide RodeLite with the server IP and the server credentials like the username password. And then RodeLite will do everything for you and RodeLite uses Firefly in the back end for running all the services and functions. So now let's see some example micro-services. So you can see that we have a background removal micro-service here running. So if you go to this URL backgroundremovable.rodecloud.io you can see that it's running a Firefly server. So here you have a predict function and it takes parameter name called image which can be a positional or a keyword parameter. We have other services also. And see you can now also you can use Firefly client to you can use the Firefly client or the Rode client to use the API endpoint for processing your image. Now the model that is running on that very URL very endpoint takes in an URL parameter and it downloads the image by itself. But you can also send in the image on your case. We also have a JS library for doing that. It's the same thing basically. Now we have some other micro-services also like face detection and image recognition. You can all of these are deployed using Firefly right now. So there are many features that we think that we should have. Right now the API which is built by Firefly out of the functions is really not a true restful API. For adding true restful API we are supporting other methods. Now every request is a post request. We don't support get right now. The issue with get is you don't have the data type. You can't pass the data. A integer really goes on testing. You need to have some method or some type checking to have a fair idea about the data type that is sent by the user. And we also don't have canonical URLs just like in Flask. You can specify that like restful API is like slash project slash project ID then whatever you want. We don't have support like that but we want to have support. We try to use Python 3 type annotations. We try to have a dynamic type checker. I think now that's possible back seven months back that was not possible. We try to use mypy in dynamic like whenever we are running Firefly we wanted to check how the types are being passed on and wanted to check if it is correct or not. But then we did not find any way to invoke it dynamically. It was a static type checker back then. We want to have support for an empire is. And currently it is a work in progress API browser. We have a work in progress branch in the repo where we have a very basic API showcase where if you go to the slash docs endpoint we reserve the slash docs for our own purpose and you can use a function called docs or have anything on docs where it shows that what functions are being exposed and how you can utilize them. Just like I did show you the API showcase page. So that's handmade right now. We can use Firefly for that also. So that's it. Any questions? Any questions? Just one question. Just wondering about how widely this is being used to have several contributors. So the number of contributors right now sends that active contributors sends that too. And we have like one or two other contributors. Are they other users or another group that actually uses? No, right now Rodeata is being used. Rodeata only uses Firefly. We don't know who uses Firefly because it's an open source library. Many people have forked it. But we don't have any fair idea who will use it. So that's the scenario. Thank you. It's time for questions. Okay. Questions? Thanks. So you can have the resources there. These are all the end points where you can go and have a fair idea about Firefly, Rodeolite and how they are being used.