 Hello everybody, I am Sami and today I will present this tutorial about Beyond Neural Search and hands-on and building multimodal solution with Gini AI First let me introduce myself. I'm Sami. I'm a machine learning engineer at Gini AI And let me introduce Gini AI. So it's my company. We are open-source company funded in during the pandemic 2020 around 15 member distributed across four continents mainly based in Europe, but we have office in China and the US We raised the total of 38 million and we are a top tier AI company according to the three number We have multiple open-source projects And we have like an ecosystem around the multimodal application and Neural Search and today I will mainly talk about Dockery and Gini are two of our open-source projects So in this talk, I will talk about Neural Search Creative AI So multimodal AI in general and I will introduce you the Gini tech stack a little bit Explain you what is Daliflow, a tool that we have been building using our tech stack and we will finish with Tutorials and hands-on when you will if you have a laptop will be able to to create some generative AI application and deploy on the cloud Okay, so let's start with what is Neural Search and Neural Search is basically Search powered by machine learning and especially deep learning. So first example is a traditional search will be Key keyword-based search when you search for a word Let's say in a book and you will search for similar other word With Neural Search you can search with paragraph. You can search with semantic. You can ask a Question and looking for something really precise and it will not only look at the key at the world But try to understand your sentence As a human will do like a human who are already with the book can search for you The book With dinner you can build such as so you shouldn't couple of offline But Neural Search is not only about text search. Actually, it's it's more than text text search is Multimodal search it means that you can search with a different kind of modality for example On the e-commerce shop you can search for products You can search for t-shirt on the e-commerce shop and it will not only look at the keyword or like the tags and the product It will look at the image looks at the description and you can look for a Given a t-shirt with kind of logo on it or Coca-Cola logo for it for example And it will be able to analyze the image and look for Coca-Cola logo in the image And it will use different kind of modality as I said the image the description Same again with our with our open source project. You can create this application in a couple of line Also example of Neural Search will be something like text to image sequence So is there a gif search or video directly video, but for example you can imagine a solution where People are looking for a gif and they just want just give the description for example a man wearing a suit and and it will like Bring you an image of such a man based on the image Also example 3d mesh look like actually we are we have like there is a company Which is using our open source tools to to create an extension for a video game engine when they develop Game developer can like looks for 3d mesh and like look for similar 3d mesh like In in a store like and to increase the productivity so that they can easily find other relevant 3d objects So this is what is Neural Search is deep learning forward information retrieval on multimodal and cross-modal data and just to give you a small Like week when I say multimodal I'm taking about image text, but even audio video and Unstructural data every kind of of data that you can imagine and this is what Neural Search can do is like connects all of the dots You can stay inside the same modality can do you still can do text to take to text to text But you can go beyond and do cross-modal application In Neural Search you have the data, but you don't have the relationship between the data and you infer your Relationship that you are from your for your model Understanding of the world and you're trying to find your data and Then there is another application that you can do is our framework is Dali and diffusion and in general in general like more generative parts. So I bet you have heard about about this, but you can like Give a sentence give a prompt on text prompt and you can and the AI will be able to generate an image for you So this is this has been created by by Dali last year Dali support by OpenAI and this is the first model that has been able to do such a thing and then there is Dali 2 which came out this year stable diffusion which is even a new open source version of Dali 2 which is like not even one month old and she's like making Making all of the news because it's great. It's like really powerful image generation with really good quality So let me give you such an example. So this is a query for a generative art So ocean beach from view in a Van Gogh style and this is generated by the AI So this is not like retrieve of Google image or anything. It's like just generate base and prompt another one Painting of a couple kissing a table up. Well, you can see where the results and in general understanding this Dali and this generative art model is Pretty complex for several reasons first. It's based on a lot of state of the art research and like recent It's based on like break through in NLP In multimodal machine learning when you like can combine text and image On this diffusion idea that I will present right after which is allowed to have a model Which understand things but it can generate things as well But let me try to break it into small an understanding So first to have this to be able to generate art from text you need to have a model which can make them The link between text and image and it's basically trying on a lot of pair of text and image And how he's able to have an understanding and what makes the text what why the text is the image only So you have a model like a clip-based model which which have this link But it's not enough you need a process to generate something out of it, and this is when a diffusion arrived Diffusion is a piece of this is this process. So the diffusion you start from a Beautiful image and you add noise noise noise in the forward process you have noise until you are to a completely noise image and The diffusion backward process will be the other way around you start from the noise and you create something which makes sense So you have those two steps when you go from image to noise noise to image and you train your network You train your AI machine learning model to do this forward and backward process and at the end what you're interesting It's in is this this backward process So you start with noise and you can arrive to an image which makes sense visually at least for humans And you and you guide this diffusion with your your your text query to You convert to an image which makes sense For human visual cortex, but it's makes sense with respect to the text query that you asked for This is an example of how diffusion work in practice So you see that at the beginning it's completely noise and step by step It's converging into something which is It makes sense visually, right um Let's sum up what is Generative AI is deep learning power content generation on multimodal data Why multimodal? Well, you start with text you go with image But actually you can even start with image and go with image. So you have application when you can like sketch You can draw by hand something and you can ask your diffusion model to To make it more realistic or like to make it really beautiful. This is like Text to image you make to text and you can even in the future will be all your base Diffusion when you can mix everything to create new content In creative AI, you don't have the data But you have the relationship you have your text prompt and you use your relationship to make the data to create the data Okay, so I introduced you to concepts never search creative AI and actually um I like to see those two concepts as two phase of the same coin Um, which file under the scope of multimodal application. So at the end both of these application you have Machine learning models that can handle multimodal data text image audio, whatever you want Just a neural search. You don't have the data But you have the data sorry, but you don't have the relationship and you find the data with your with the model understanding Creative AI, you don't have the data, but you do have the relationship and you make the data at the end It's still multimodal application And multimodal is kind of the future of machine learning and AI in general And it's not only me saying that but all of the big tech company and all of the Best Research lab in AI are actually going into working on multimodal open AI deep mind Facebook or meta AI google both of all of them publish some multimodal machine learning model in the last in the last year Um And actually there is a there is a reason which makes that Multi-model is a future and this is a quote for yandikun in the recent blog post for a couple of weeks ago When he says that an artificial intelligence system trained in words and sentence alone Will never approximate human understanding And it's really important to to understand is like what she tried to say here is like you can give All of wikipedia all of the book in the world to an AI It won't be able to be as good in understanding as a human and it's kind of easy to understand why Because when he's going to read a book which is going to describe What her horse looked like in real life. It will have a hard time visualizing everything and understanding Actually what's happening in the book because it will be only world you can describe this world But it will be won't be as precise as with a with a visual cortex that you can have at the same time And so if you really want you Your AI to really understand the book he need to see horse in really he need to have image of horse Just to visualize what which is about and this is this is why if you if we want to create AI Which can approximate human understanding or at least be better than what we have right now We need to go to the multimodal and we need to make image With text and even video to understand how the world what the world is about And at Gina AI we have the same And we have the same belief that multimodal in the future and that's why we're building ML of framework for neural search and more generally multimodal AI and Fun fact actually we started to build this ML framework with neural search But in order to solve building every kind of neural search application We kind of have to go into being able to do any kind of multimodal AI And that's why we ended up with this ML of ML framework Which initially was based on neural search, but right now is way more powerful We can serve any multimodal AI application Let me tell you more in depth about our Tech stack that we will be using later in the hands-on and in the workshop So we have a different layer so to build Application AI application and multimodal Application in general there are like a lot of layers of abstraction, right? And at Gina will decide to expose three concepts three different layer to the user to make them Our full flexibility which is making it easier for them to develop So we have like the data concepts documents that I will talk right after We have the logic concept executor when we basically let user define their own custom logic For create the own application and we have the application here when you kind of like Take together all of the logic to create a final application So as I said we have these document concepts Document we have open source library which is called a doc array For document array. This is our this is the data structure that is that is going to flow To go into each of the application and executor. I just say what is our it's like Just function or classes which will take document as as input and send back documents and then you have flow that we will a flow is just A graph of Executor and document will flow through this flow with flow will go through this flow and and that's how we create an application With gene I will so this is I will go over this small piece of code to explain you the concepts from a python point of view So first you have to of course install Gina, but Once it's installed you can you have to import four objects a document array executor flow and request So first thing that I want to do with an application Is to create the logic what is it going to do this application do and to do that we create An executor it's in python to create an executor. You just need to Create a class which will inherit from this executor which is coming from our package And once it inherits with that Supposedly these classes will not be only by the classes, but it will be A fully microservices that ends that his lifetime will be handled by by gene and Then you need to define the actual logic. So you create a method You decorate it with request to say to the executor that this is what you want to serve And then you add your logic. So here the logic is really simple. You take a document as input and you add A text to the text to the text attack So with this you have Not only class but you better to see it as a microservices which will be able to run in the cloud and serve your application Now that you have your microservices you need to put them together into a consistent application and what we call the flow And with this syntax for example, we are adding two of this executor on the chain So we have this we have a graph with Two of the same executor which follow each other but this This is an easy example, but actually it can be You can represent any kind of application and any kind of Of directed acyclic graph So any kind of graph you can whoever sent acts to create A lot of a different graph to serve any kind of application And then once your flow is ready You need to serve it And using the client to send a document array To send a document array and to send your document you send your data to the logic And this three lines of code it's actually start of An api which Microservices big api Which will serve your executor And put your document to the to the logic So let's see in action how it's working. So I'm starting the And you can see that The flow is ready to serve and we send the the we send the data and we have the results Okay, so This is on the on the right. This is with gena on the left. This is like let's say with traditional python And you can ask me so why do we need to What what is the benefit of using the gena to do such a thing? And what is the benefit of adding this real new line from the python code? Actually, you have a lot of a lot of benefits. First You can you can have replication and charging Each of your executor is a microservices you can skate You have a duplicate streaming. So you do it's not it's not only HTTP But you can talk with GRPC and you can have a duplicate streaming when you can send your data and receive your data You have I think not blocking data processing Our application is polyglots You can if you want to use gfpc or web socket or HTTP or even graphql You can it means that you can send different kind of requests or you can define If you if you want to you have a front end which is HTTP with application You can if you want to use web socket for a video game you can Everything is based on microservices architecture and on the core content Teranarization which wrap is executable and we have a kubernetes seamless integration So it's easy to deploy to the cloud um You cannot observe observability by chromatives and graphonite if you want to do monitoring we have like a lot of default Metrics so that you don't have to add anything new and we have a plug-in ecosystem Who can you reuse executor for other people that are already build it and you just need to pull it and put it into your application Okay, so another day so Once the tutorial will start or like the hands-on We will try to create a small application We are with our tech stack and we want to create an application Which is called deli flow and it will be allow you to generate some piece of art Quick facts about deli flow as I said it's allow you to generate some art And it's a client server architecture It combined deli and diffusion using jina and it was around 300 lines of python code and For online of yamal and it was actually a weekend of work for our ceo Which is kind of do a lot of thing in one weekend, but uh still it was one weekend project um So today we're Directly going to build deli flow, but we are going to build a flow which is a bit Simple also that you can do it live And it will be a flow which concept of two step the diffusion step when you send some texts and you generate an ai Upscaling step Which take your image and put it into high quality image like you can use it for you will what people if you want Uh, I'm not sure about the after that if you're going to have time we will see at the end But we will focus on the first time in the division um So we are about to start the demo But do you have um any question before we open our laptop and start to do some coding? You can think about the question and i'm going to open uh different mind So for the one who have a laptop and he who want to experience the coding in life Please go to this link Actually, I prepare a cure code, but since you can't scan cure code with your laptop It won't be helpful. So you can go to uh my github github.com slash Thanks, I will let you a little bit of time to Connect to it if you need Please ask me if you need the link again I will open it and you go to my repository And this is the first one stable diffusion executor and just click on this one And this is the link that we are going to use to Build everything And we have a google a notebook Hosting on call-up. So you don't need to do anything. Just click on the notebook And you will be able to run it from your browser. So without having anything install Okay, this is a notebook I will let you some time to go over just to open it in the meantime I will just show you the final results that we That we want to have just to give you some You don't need to do this step. It's only me So this is just the final the final thing that we want to do and I just want to give you some insight and some Some idea what is going to look like so you will be eager to continue Excited about building this application Just installing dependency on google call-up should be take a couple of minutes It's installed so No, we are going to so I have already a flow which is running So as I said flow is a fully fledged a microservices beta application And it's already running already deploy it before Starting the workshop. I just want to give you some hint of what what what it looks like Okay, and I will need someone to actually give me a prompt if someone feels inspired so that we can generate some art out of it So if anyone has any idea of something you want to see Feel free to To raise your hand and to try something Well, don't be shy. It's really going to be really funny to see you from going live Like I can give you some hints. You can we can ask for realistic photo of something I don't know something linux based since we are the osse event we seek photo of I don't know Linux tux Playing I don't know the guitar for example and Let's take a look oops Yeah, forget one Okay, so no I am in my google call up But actually as I say the flow is running in the body like on our cloud So Google club does the client here and we connect to the flow and we could connect with I don't know an htp A front end htp application And then connect and let's see the results. I hope it's going to be beautiful This is definitely not uh Oh, I forgot the the the linux tux, but this is uh from stable diffusion what uh tux looks like what linux look like playing the guitar So this is this has been regenerated by the AI. This is not like go on google's image yourself or something Let's let's let's try with tux I didn't try this query before so I don't know what it's going to look like But since we're going to build it in a couple of minutes, we'll be able to have as much time as you want to actually Do okay, this is a tux what it looks like the linux tux playing the guitar I would be happy with yeah, I'm happy with this with this one. So Uh This is kind of cool. I mean you can generate image out of text and let's see how you can actually build it with our solution So for the one who want to follow the life part of the workshop In you have the if you have the notebook open We can go step by step and look at well So I will introduce different concept in jenna and then we're going to actually build the application So um, so before we start Be sure to you to change work runtime to gpu And most of us won't be able to generate anything So know that you have your runtime basic. So I repeat runtime Change runtime and you ask for gpu Okay So first step is to install dependency. So, uh, we are going to install docker and jenna which is coming from us And diffuser and transformer which will help us Actually doing the machine learning part of the diffusion So again, it would take A couple of minutes to install all the dependency If it's complaining about jenna cloud just remove the jenna cloud from the people in starline Yes, go ahead So just I'll repeat the question for the people joining lyre. So the question is is it generating the image or Searching the image using uh neural mesh, right? So no it's on the example that I just show you is generating the image from the text So this image does not exist on in terms It just created like opening like if someone was opening photoshop and actually doing this Okay, so everything is installed And so we are going to start with our first concept in our jenna text stack, which is the document So document is the data without our data structure for instructor data and Distract data structure allow you to work with uh different kind of modality Text audio image, whatever you can think about and kind of have like a common API to to to work with your data And it's fully uh, this data structure is meant to be stream Over the over our flow. So it's like you can send this is like the Request that will be sent to your flow. It's not only data structure But is a thing that will be go over your executor and it's basically if you know document array This is like an entry point to our text stack But it's like all you need to know almost once you know how to work with document you can build any applications that you want So this is an example of how you can um Create documents so you can for example pass a text and you create a document with the text Um, you can as well have documents which contain an image Which contain a history a new So uh, you write or you write so we and and and then you can load This this image from the network through your document array By calling this load you read image tensor, which is a helper function And we have like a lot of helper function with all kind of data to make your life easier And as you see we loaded um numpy and ira into a tensor And you can actually display This image that you put from the internet and I choose uh linux tox image since we are at the oasis So you see that these documents are the tensor objects. It's actually where the image is stored in the document and You can check this shape and um So, uh, you can actually you can add the tax Sorry tags to your data so you can kind of customize it I have some custom tags so here are kind of documents With a halo of text, but the tag is like the event has been done on the oasis and I know what kind of Different documents from incoming from different sources with different tags and my executor my my uh my business my my logic Uh layer we will be able to deal with it different Okay Then you have we have the concept of documentary which is just a List like container for documents And actually are not directly sending documents, but documentary we're using dina and it's just A collection of documents So to create a documentary You just need to pass a list of documents To your to the documentary classes and it will create a document. So let's do that And that's what the documentary looks like. So we have this interface a cli interface or like not cli a ui interface which help you understanding what's happening in the documentary and We for example in in this documentary we have 12 12 documents and They are all of this common attribute which which is a ui and this ui is uh actually Uh the linux text image again So you can like this is kind of the kind of helper function that we have to work with this container like Uh data structure okay, uh, this is all for uh documentary and So our doctor doctorate data structure. So just to sum up it's our data structure And you can send it in the wire to our uh logic and to our to the executor Okay, so now that we understand what's what is that right? Let's go over. What is jenna? So jenna is the name of company that it's first name of open source a project Um, which allow us to create an application. This is our ml of screaming for neural search and And so let's create a really small application Uh, which is going to take some empty documents and add Uh, uh text in this case jenna is fully open source um So to do that again, we create an executor um We we we inherit from From the executor class from jenna. We just have this request to say that This is like basically You can you can see this interface as like a fast api or flash or any kind of web framework When you can like have a logic and have An entry point in each he is going to How to serve this this this logic and in our in our case I want to serve This full function On the on the root level of my micro services So now that we have this Executor we can actually serve it into a flow. So here i'm creating a flow, which is really simple It has only this Uh logic layer in uh in the application And I will send some I will send some documents. So you can see here on this root level. So it's going to go to this one I will say I will send some documents which the text is empty and I will retrieve it in another document All right, so let's take a look at So here you see that before it was like waiting it was waiting for the for the microservices to actually start And instead it sent the microservices. So for a couple of seconds It was actually serving a jerky application On the this part of my machine or like the google collab one And since I only want to do one action one post it's stopped right after but it's not this is not only calling python applications. It's like Serving a microservices and sending A jerky's call to it or it could be an hpl one as well And what do we have here? A document where the text is generous for the open source. So the document was empty at the beginning and went into the executor has been executed It's been processed and We have the text um, so as I said before, uh, we can actually create way more complex applications at just One executor and we have this so we have this python Interface which allow you to create any kind of of of that application All right, so like a graph application So for example here, we can chain two executor one after each other And in this case when you're going to send one Document it will go first one and then go to the second one or you can have like a more complex one When you cover like like two branch and you can specify different end points um to create different logic Okay, and now it's time for us to create The generative AI class of the So an exit so we need to create an executor Which is going to take text as inputs and create some image So I will Show you how we can do such a thing So first we need First we need as I say to create A classes which inherit from executor we need an init function For for the moment, we won't do anything in the init function, but later we will load the model and everything So the generate function it takes a docs as inputs And a parameters when you can specify some things um, so in our case here, we are going to um So here we're going to for each document in the document array We're going to do some things with the text and so, um, so this is the the first basis of our Executor and in now we need to actually, uh, load the stable diffusion model and, um Code the generate function. So let me just look over I already called the executor just going to call me by this line because I tend to forget The model name So we're going to load this diffusion model from the stable diffusion pipeline from the, uh, diffuser It's like a open source library which allows you to work with the generative So first thing that we need to do is to yeah load this diffusion model And put it into our CUDA just so that we can use GPU Um, we might need to have this out token, uh, which is just take, uh Just a text token, uh, which allow me to to download the way When you're going to run it you might need to they will tell you that you need to, um, accept The way, uh, you need to accept, um, a shaft The people that develop the model want you to accept the charts to use the way so that you don't do, um You don't you have to respect what they want you to do with this model. Um Okay, so, uh, let's Actually instantiate This executor I will just copy pass my old token in So it will download all of the weights. So it's around like more than five giga of Matrix weights and in the meantime, let's go over actually coding Actually coding the general generation part. So, uh, let me I will just copy pass this part and I will go over I think it will be quicker. I won't lose your attention by doing this um so, um So I still have my generate uh method which wrap my request, uh end point and for, um For each of the document which are in my document array I'm generating one image from the document and what I'm doing is really simple. I'm taking the doc I'm taking the text and I'm calling this, uh diffusion, uh Model and it's going to generate, um some, uh, some image and then for each generated image I will create documents I will create a document and on the tags. I have the original text I have the name of, uh Like which executor which model it actually creates This is just a system type tags when you when we retrieve the document, you know, where it's coming from And I have a pill image. So like a Python image and I put it into a data that I can send over the wire And in my original document In the match, matches is a nested part of a document which can contain other documents Which are relevant to the original document and I'm adding the generated document which contains image So Let's again I have some indentation And I forgot to import Torch You know, I should be good to go And again, it might take a couple of seconds To retrieve, uh the download weights So if you if you didn't follow exactly the different step You can go into the initial link that I sent to you and you can find this executor.py object Uh, where I already coded this executor and you can see the code source and Reuse it if you want to Okay, go ahead. Sorry Actually knows typing are not important since we are working With python, but I just put it here to make it clear What the What's the thing like what's the executor is awaiting what kind of object is awaiting what kind of object it will tell the doc Initial so your executor would only only take document array and you have this parameter where you can pass like some Keyword based arguments But the input is always document array And but since our we created the data structure since so that we can Easily work with any kind of data And so that's the point of the executor. It's like we have a common interface for all of the data that we send We don't have to like have an image object or text object or an audio object. You only have this Documentary object Which one? Sorry. Yes Yes, um, so This is just for compatibility reason So our um executor take Can take uh When when when the flow is going to instantiate the executor It might be in some case pass more than one than those two executor. That's two parameters. Sorry. So I just put it here for Like let's say safety reason and Exactly exactly. So now that we have our executor working we can actually We can actually start the flow with this executor and uh Send some document and generate an image again So this one will actually be working working from the google colab um So I will start a flow which only have this executor uh for now And I will I will I will do a query on the application which has been created And I will do a query on this on the on the root branch Since I asked here for to be served on the root level And as inputs I will just pass the documents Our document array With document inside Which has just a text with a query. So Let's have a fancy query here um The linux checks Doing something this time With a red hat. Oh amazing And we just retrieve here in this document So let's see how it's going to I'm is going to work. So here Uh, it's going to actually I'll I'll say start this um This executor in like this own Micro services and it's going to send to be sent over the while So here is still local, but I will show you right after how you can actually deploy it On some on some cloud So it's in the process Yet again live. So I hope it's going to work. I didn't make any type or something And Yes Um Yes, I didn't explain it. Sorry to get the old token you need to go to the Hugging face stable diffusion model and you need to If you go to this link, so the link will be on the read me of the first thing I sent you to And you can if you log in and you accept the condition here you will get the token that you can copy past In your Well I think he's not happy about something Uh Okay, don't I I don't want to go into the table. I think my runtime is not working properly Well, I took the risk to do it in life, uh, but No, I don't think so Maybe maybe the GPUs that's google club give me too small for handle the money Anyway, this was This was the life part. Hopefully I did. Um, as I as I showed you before I already Um, I already deployed the application So, uh, now I will show you how you can actually deploy So we created an executor and now I wanted uh, we we we we make it work on the google club Let's try to deploy it to the cloud so, um To to deploy to the cloud I will go just going to create an executor and to push it to our Gina cloud so let's just create An executor folder go into the folder so We just we first need to do to install Gina We just we just first need to install Gina from from from the pip command So pip 3 install Gina once this is done We can call directly the Gina package and okay create hub So here we're going to create a new executor. So this was from the code from the From the python interface now we're going to actually create from the sea lights or to can push it to our cloud So I will call it another table diffusion executor And I will just put my name So that we don't conflict with an existing one I want to create in this folder. Yes um So this this step is actually really important So here I just show you use a python step But here we're going to do more than python. It's going to internalize this application into a docker container container and Who is saying the docker container is saying docker file and we actually have a couple of Template for docker file that you can reuse and especially in your in our case we want a torch a GPU a template Short description a small diffusion what Image so it's image and takes this is just some description so that we can find it on the hub What is real for the repo? I don't want to precise one for now So what you can see here is like we created maybe it's a small We created this for this this folder and which contains the application. So now what we're going to do is just Actually Copy plus the code that we just wrote into the executor so that we can push it to the cloud So I'm just going to do a shape shape full copy pasting And I just need to rename and what would do Okay, so now we do have our executor ready in the required we We in the requirements um, we will need to add diffuser And transformers as required, but we just take it like I didn't do any misspelling Because I'm the king of misspelling Okay, oh good Okay, so now our executor is ready to be pushed to have I need to precise uh the path So jina have pushed. I don't know what I wrote before and as I've done misspelling again So what is going in this step? What is happening is we're pushing up the executor that we just wrote to the jina hub so our hub is just um A place when a lot of people push different executor and you can pull a push Like I'm getting If you are not logged in it might ask you to log in and you can do it by doing jina login But since I'm already logged in with my account So it might take a couple of minutes to actually build The docker container and push it to the hub. So in the meantime, what I'm going to show you Um Is how we can deploy it So today I talked about uh doc array and jina or uh two open source Project, but we are actually have a self project Which allow you to to Yeah, this is our cloud when you can actually send the flow And it and we will deploy it for you and manage the lifetime for you We have a free uh cheer for everybody who wants to try out now today We use the feature to actually Deploy the application of our cloud and at the beginning I show you some image generation on and I told you This is actually already deployed and you can see here. We have this We have this you will help it's actually it's it's and it's pointing to an application that I deployed on our cloud This morning early before joining the The workshop So to do this we are going to use um We are going to use the Um Another way of defining flow in genals that I do not talk about it that much I focused on the python interface, but we actually have a yaml interface Which mimik exactly the python is a python one and it allow you to So you can create gen application like Interface for code so And it's like in kubernetes when you work with yaml And you can as well here work with yaml. This is actually really small yaml It just saying that we have a flow and we're using this diffusion and Exactly what I just push and I just passing my token So that they can actually pull the weight and What I'm going to do is to deploy this flow So gcloud you need to do a pip install gina cloud and it's going to install the Gcloud gc just an alias, but it's actually working for everybody No need to update your bash or anything and gc deploy flow in the channel It's look like a wsc li or gcpc li and you can deploy your Your flow So again, I will start deploying my flow It's taking it usually take around Between between five and ten minutes because I see I have some issues connectivity But It's actually deploying enough kubernetes cluster. It's assigning a new ui high Assigning the certificate that you can use SSL when connecting to the flow and being fully protected and unloading of course all of the weight of the model So I will keep this as a background process and I will take a look again But while I'm pushing the executor and I'm building the flow and in the meantime Let's do some content generation to weight So again, this is like the flow which is already running and in the meantime Let's actually do a more Generation so who has another offensive promise that he wants to try Is there a that one again? Maybe That we didn't have a chance to look at With a red dot so Let's do the same one with a red dot. We'll see how he's going to react Okay, so What was my sentence a real sick photo of linux tube playing the guitar with a red dot Maybe I should have saved A red dot somewhere else. I think he understand. He didn't understand the red part the hot part of the red linux tubes With the red hats playing the guitar We are waiting for it. Okay, let's try kind of scary He only kept one eye of the Of the linux tubes, but but he understands right that More than right at you put a lot of red in the photo. But anyway, um, let's look at So our executor is still in a building process And our floor is still in the deploying process and it's failing again And I do believe I have some connection issue connecting to our API we won't deploy it like and I think that it will mostly be the end of the live workshop since And we and since we can't really deploy the floor Since because of some issue But if you have any question about all of the different steps that I showed you and Feel free to ask Go ahead So I repeat the question so the question is is it mandatory to deploy on the off cloud or can you self host it? Of course you can self host it as I said at the beginning The gina open source Package can have seamless integration with kubernetes. So what you can do actually is to from a floor You can have you can you have a health function would help you to create The kubernetes yaml file and then you can deploy your kubernetes faster Yes, yes, exactly I Because once the summer had, the time also had. And also it makes another strong perception that there is only one time for each summer. Because here you only have one time. I have a host, right? I have a host. But in reality, in practice, there is one summer. It could have been in the summer, 70 or less. It could have been in the summer. So there are all, in introduction, what it is, how they are best, how we do it all. So you kind of have this flow as if you had it in slow motion. All the time, you kind of have it in slow motion. So this is more realistic scenario. So I repeat the question again. So we have a helper function to create the Kubernetes YAML function. And your question is kind of create as well as the Docker image. So the question is yes. And actually when I'm doing GINAR Hub Push, it's pushing the executor and it's like pushing the code. At the same time, we have like a normalizer step, which is going to take the Docker file and view it with the Docker image. And then you can reuse it in your Kubernetes file or local Docker developer. You can also create a lot of package in my server. It can be really useful. You can use it as a state-of-the-art server that was shown today in your local, right? So it can actually be packed in a local version and automatically viewed the image on the mind. And everything happens automatically on the top. So it's not on your mind. So when you push this in, you just push Python and Docker file, the source code to the server. And the server is responsible for building all the sales, you know, resorbing dependencies, resorbing versions, all this Docker server will say, okay, so now I have this kind of like status, right? So this executor status. So now if somebody asking me to retrieve this executor, I will view it on the mind. Doesn't it autoscale? Yes. Yes. The question is autoscaling. And yes, I mean, based on... Yeah, you can add autoscaling. So each of the executor can be replicated in the flow object. So you can say, I want three of this executor. So you will want to replicate if you want, for example, to have a lot of real ability. One machine goes down, and we have two resources that can serve the traffic. And you can activate autoscaling, which allows you to spawn in more replicare of the executor if you have a lot of item. Oh, so I still have to quantify the YAML file to do this. Well, this is an option, right? By default, we do not activate autoscaling. But yes, if you want, you can modify the YAML file to do this. Everybody care about money, but you can set autoscaling to two. By default, yes. However traffic you have, there you have all the scale on demand. But then it will take a lot of money. People usually don't want to set an upper bound. Okay, up to three machines, up to three GPUs. Especially if we're not talking about, since workshop is about multi-modality, multi-modal applications, a lot of multi-modal applications require GPUs. So you don't really want all the scale of GPUs, right? Otherwise, your business will go bankrupt before you're gonna buy the computers. We've got a question from a virtual attendee. What is the difference between display with inter-emphasies from equals tensor and just display parentheses with nothing? Yeah, with question. So just with the question, what is the difference between... Here, what is the difference between dog display without anything and dog display from tensor? This is a good question. In this example, we have actually two... The document is composed of the two things, the tensor and your URI. And I'm just saying that I want to display from tensor and not from same scale. So here it will be like, you didn't know what to do between using URI and tensor and you can actually plot from both, and I just told him to plot from tensor. So we have more questions. Otherwise... Yeah, go ahead. So for the GINA cloud, do you guys have your own infrastructure or are you guys using the party infrastructure? We already have a different group provider, especially for managing of Kubernetes cluster, but we do have our own infrastructure on this cloud and we are at all of the layer that makes it easier for people and we manage our life cycle. And we have, for example, a couple of committees and growth analysis and we automatically generate a link for people to look at the metrics of their deployed application. Yeah, I asked because I want to know where the data will be. For certain applications, all this is required that you know exactly where the data is. And so I wanted to know. Yeah, it works. I want to make a point here. So for now, the G-cloud that we offer is running on our own cloud, but we are working to have this so that you can use the G-cloud self-hosted in your own cloud so that you have control over your data. And this is something that we are working on, water or user defense to find the G-cloud advantages on your own cloud with under your own control. But for now, it goes on our... on our cloud providers. But we are working to make it available so that you have control over it. Sorry, I... Yeah, go ahead. I mean, I'm really happy to take the question. There's a limit for auto scaling, right? So you said that it might not be a good idea to set auto scaling. So let's say you only... you don't auto scale. So you have to limit the amount of resource. If suddenly your service gets a really high amount of request, what will happen? Will the request get dropped? Will it suffer? So we do have a buffer. We don't know in detail too much today, but later we have a lot of talk in detail, technically, how the microservices actually is working, but to give you some details, we have a gateway object. So each time we deploy a flow, we have a microservices for all of the executor, but we have a gateway object which can be replicated as well, which is to work as a buffer. It will work as controlling the request and you start switching rate and it's going to... So if an executor is still in processing some other request, it will throttle and wait. And we're based on the GFPC connection anywhere in the GFPC, so we have this dual streaming interface when we can... It's built in the GFPC so we don't send too much request while it's still processing. But we have this... Yeah, this gateway we can serve as a buffer. So until the gateway dies because it's sent like out of memory because it's a terminal request, it will just wait for the executor to be able to use it. But since you can replicate the gateway, technically you can make it so you would never lose any request. But then again, it's a cost of... I mean a lot of complication. Does the gateway have strong identity and then can you white list a further IP upon the gateway? Or the gateway will respond to every request? So at the moment our philosophy is more first gateway not to reimplement everything that exists outside and we will encourage you to use some other kind of... On top of our gateway some, for example, in our big cloud cluster we are using a comb which is another API gateway which allow you to do this kind of white list or... and doing, for example, if you want to add something else like some authentication layer or things like this and you do it on top. But we are building actually and allowing people to build their custom gateway and their own logic not only at the data level the data processing level but as well as at the hooching level. Thank you. And I think it's all for the question. Thank you all for listening to me and watching me doing the live workshop. If anything, a fail at some time but that's a risk of live stuff. Yeah, thank you very much.