 Good morning. My name is Shweta. I'm a computer science graduate student at Columbia and Today I'll be speaking about harnessing generative AI without making a for par Little bit about me I worked as an SD at Amazon for two years prior to Columbia most recently I had turned that cast in by Veeam for three months over the summer Which is where I came across this problem statement To generate canister blueprints by prompt without exposing proprietary data kind of like a github co-pilot experience Canister blueprints are basically yaml configurations used to define backup restore and delete workflows for multiple databases We all know github co-pilot our handy AI pair programmer, which generates code based on prompt But github co-pilot uploads our code base, which is the IP of a company As context of the github co-pilot service, which it needs for generating recommendations This is why I chose for pilot which is an open source project Which provides the components to build a locally hosted alternative to github co-pilot This uses an nvidia inference server at the back end to accelerate inference and it's required for ml at scale and That in turn uses a custom faster transformer back in which is specific for llms to in provide inference like distributed manner Using something called model parallelism where it basically splits the entire model into multiple different parts and predicts independently Stitching the output together at the end this is the framework I had in mind while trying to build this application and We will see how to build this part by part I just plugged in the client and the server according to my application in My case the client was an IDE because we wanted to see the code generated in front of us by prompt And the first step was to build the server itself, which is what we saw for pilot So some of the considerations I thought about were how and where to host this server the hardware and OS Requirements once I decide where I can host it and Then to connect to the server some of the considerations I thought about were how to ensure and only an authorized client can connect to the server how to ensure efficient utilization of the Server and how to ensure availability of the server for multiple clients connecting to it The next step was to gen gen the set up the client itself and the obvious question is which client to choose so this depends on the end users and the application and In my case, I chose an ID just because the users are developers Another very important question is how to ensure proprietary code is not leaked in the process in my case I downloaded the github co-pilot extension and all this was automated and Configured it to point to my local server Basically the server hosted on the cloud before even installing it into the ID so that it doesn't consume our code base The final step is the model itself and for this one question to ask is what are the existing models available in the marketplace? Which can do this for us and then the next step would be to evaluate which among these models gives us the best accuracy After evaluating if nothing is up to the mark then we can go ahead with fine-tuning the model So fine-tuning is basically the process of adapting a pre-trained model to fit custom data What happens is the last few layers of the model is modified to be able to generate just the custom data that we want And this is much better than actually training a model from scratch Because it takes much less time and data to do so because the pre-trained model itself has a lot of knowledge about language and the domain at large Two important Intererations while fine-tuning the model and these are the steps to do so is in terms of the data a What format should the data be in so that the model will be able to accept and understand it? And for that we need to look at the way the model itself was trained And the second point is do we need data augmentation? And this depends on the number of samples data samples that we have with us of the custom data In my case, I only had 11 sample blueprints, which was way too less for the model to learn So I proceeded with data augmentation to synthetically generate data for me And I did this in these are some of the examples in the way I did it basically generate random typos include random words delete random words and from 11 samples We now had nearly 2,000 samples which was split in at random in an 80% 20% split to create a train And validation data set which was used for training The final results and although we don't have time today to show a real demo was that the model recognized Canister blueprints by prompt and generated results It was able to differentiate between backing various different databases and provided different yaml configurations accordingly I guess the key takeaways I'd want to leave with you is we can apply a similar framework with considerations at every stage to build almost any application and this way We can use generative AI in a secure way without compromising proprietary data I encourage the community to take the plunge harness the power of generative AI and enhance your cloud native applications Without losing competitive advantage here I know that was a lot to take in in five minutes But feel free to reach out to me after this session if you have any questions any suggestions and please scan this QR code for Feedback I'd love to hear feedback on the top. Thank you so much All right. Thank you. I suggest we switch but while we switch maybe we can take one question if there is one for sweater If anyone has one We can try to sneak it in while we switch the laptops Otherwise, thank you. Thank you