 Yeah, I'm gonna start now if that's okay with everybody. Yeah, you want to go ahead by me? Yeah, sure Well, welcome everyone to the hyper ledger meetup. This is the AI FAQ lab we're discussing the New reformed lab how it got created and where it's going in the future Next slide, please So again, this is a hyper ledger foundation meetup. We abide by this anti-trust policy And a code of conduct. So if you take a moment to read that over just to know that everybody is welcome in creating a safe and Welcome environment for the community. So I'll give you a minute to read that Okay, next slide. I just want to give it everybody an idea of what's going to be happening We're gonna talk a little bit about a current community event the mentorship program that is just starting I'm gonna introduce the team and give you a little overview of the AI FAQ project That's really brief and then we're gonna go right into a demonstration and then talk about some technical topics And then open it up for questions and comments. So we could go to the next slide. We're going to Introduce ourselves. I am Bobby mascara. I am Involved in the hyper ledger community for about six years now I started as the chair of the learning materials working group And then I was a member of the technical steering committee the technical oversight committee and I Was the chair of the documentation task force, which is where all this started from So that's why I'm here to discuss this and next slide. I'll let Dan Luca introduce himself Yes, thank you. I'm hello everyone. I'm Gianluca Capuzzi. I'm a software engineer I was a mentee of the last hyper ledger mentorship program in the 2023 edition and also I got PhD in AI and below my LinkedIn link Chappours next, I don't know Hi everyone We're back to the slide shown at the presentation. Yeah, actually I was unmuting my mic That's right Hi everyone, my name is triple Joshi and I'm actually a very big blockchain enthusiast and I worked with I worked as a mentee in hyper ledger documentation task force and after that we got into AI task force and from there we were able to start this new project AI FAQ project and It the journey was really amazing. Bobby is a mentor and Having great developers with us like Dan Luca. So It was a really amazing experience, which is still going on and I hope I will see more people joining us from this meet and If you want anything you want to ask about open source or Anything related to research in blockchain area or in AI, please reach out to me in through my LinkedIn That sounds on my side Great. Thank you So a little bit about the project and where it started from So I had mentioned that I was the chair of the task force for documentation And this is the task force started about a year and a half ago and two things were Coming out of the community one was everybody was talking about the metaverse and everybody was talking about AI So at the documentation task force We were looking into how those two new technologies could enhance the documentation support for the community One of the first ways was the the hyper ledger library, which is a metaverse library where it stores all the current information for all the different projects tools and libraries And then the the next way was AI and how that can help and that obviously could help with Standardizing and templating a lot of the Documentation that comes out of the community making it easier for everyone to create high quality documentation But then the other area that we were looking at where Jan Luca really stepped up was having These chat bots become Almost not privatized, but it's the quality of the information going in And we were looking into how you can guarantee your community information is what you want it to be And I know in the news and all all over the Newspapers and news feeds and youtube channels are people discussing where all of these Like the chat bots got their Information from and was it copyrighted? Did they steal it? Is it is it correct information? I mean what is going into these things and it's really a big unknown Well, Jan Luca and the team came up with a way to solve that problem So again, we're open source everything in the hyper ledger community is open source. So out of The mentorship documentation program that we did in 2023 came this AI task force which has now Applied for and is a hyper ledger brand new Project in the labs So the way things kind of flow in hyper ledger is this particular project a task force started needed help applied for the mentorship program to get New eyes on it. Um mentors came in they worked Through the summer last year And and and looked at that and at the end had quality code to Put up into a hyper ledger lab found a lab stored and now it's in the hyper ledger community as a lab Which means anyone can come help and join and help develop this open source tool for the hyper ledger community or any community So the tool itself Came from the mentorship program and this year We've also applied for another mentorship Program to support this project and help develop it out and it's open to anyone More information like to poor said, please just reach out to any of us at any time And we're glad to fill you in so right now it sits as a hyper ledger lab, which Hopefully will get enough support from the community and enough interesting Developments coming from it that it will go from Lab to incubation Which means that hyper ledger supports it more in its community landscape And eventually a hyper ledger project, which would be awesome for Jan luka and the team So basically that's where the project started and how it's developed. So we have This ai chat bot a quick overview of it. It is support For your frequently asked questions and your documentation So We will show you how it's two parts The first part is the information that you program in And then the second part is the information that you want to to get out of the information you put in So this open source tool lets you Put in your company's Documentation your github repositories your frequently asked questions Stuff from your discord channel you get to put in the information Through this code that jan luka is developed and then you can query it and get your your questions out So moving along trepore is going to give us a demonstration And then jan luka is going to show us a little bit. Oh, I spoke to this slide already So it's basically an open source initiative to help With extensive documentation so that you can have Frequently asked questions answered correctly right away And when you open up your documentation to the community like this it ensures it's going to improve because Trust me if you have any kind of forms on any kind of books you've written people will let you know if you have a mistake So it's constantly improving and innovating And it's friendly and it's easy to handle so that if your company wants a personalized FAQ This is you know, please join us. Help us, you know, let this grow and develop into the project that it could be So with that I will turn it over to trepore Hi everyone so starting from Demonstrating ai FAQ Let me share you We are doing it from the google colab and On jupiter notebook. So this is our github repository. I will share the link in the chat box And so if you want to test this out, you can do it on your own by going to the readme file and then We have mentioned everything then Januka has written a medium blog on it where you can access All the things jupiter notebook and everything that you want to get Where we have explained each and everything about how things are here and here is the google colab notebook link for you if you want to go on it and Want to play around and we have also mentioned in the Readme file how to Like add your link like we are currently using iroha for iroha for you know Example as an example you can insert your own documentation. It's very easy We have shown every each and every step on it and with Images and so if you get stuck we can we also Like if you start running this you will get januka's Email id so you can ask there And so i've already set it up so that i won't waste all of your time So we just have to install Some packages before going further and which i have already done then it will other dependencies importing dependencies Then moving forward Here we have mentioned the read read the docs of iroha and Here we are taking from here. We are taking the documentation And whatever is mentioned in the documentation. We will able to answer it like whatever question we will ask them the ai will answer us and So this is what it's currently looking like we are working on the version three of it so hang tight and Let's start by asking of like a very simple question like what is hyper led So i think you are using The version my version which is working with linux main distribution so you can I think this was the one i was Having some problems here Just a second We i'm starting once again because it's yeah, we if you want i can i can share my screen and you can use the same with With iroha Yeah, for sure. Yeah Yeah, you can use the same with iroha and Okay, let me share the screen Yeah, this is the the same installation Okay, i can start yeah, we can also use link outside the google collab and Yeah, the first this is um i'm using the linux mean read the docs documentation So the first question could be what is linux main distribution okay, and the the system Is an open source community driven linux distribution based on the world blah blah blah and the system And the knowledge base is that one. I started with that link inside the google call a notebook here but the the first the first example the Version which you found on the gdavribo is using iroha hyperledger iroha so we can Try with another question like how to install uh linux Okay, uh, you can follow the step outline installation section this section provide a guide blah blah blah there is Some links which we can use so the The other one the advantage is to have the system which replies to question avoiding or read Lots of documentation so sometimes for user but also developers Can access the documentation? um the Without reading all documents, uh, so we can start to use the system um Asking question to the to the chatbot and so for tripoli if we want to Check the same now I can change I can change using iroha We can use the same link now I will Yeah, I will run that system Okay And you can do the same if you want And I will try to continue with the the presentation Okay, and after that when you are Really, you can also stop me and try another so to to have another demonstration if if you want Okay, thank you. Yeah, and I want everyone here to Suggest some questions that they want to ask from the AI FAQ from the bot and we will Use them for our next demonstration so that you can have a real-time interaction with it So please feel free feel free to comment your questions in the chat box. Yeah Okay, thank you. Thank you tripoli. Thank you, uh, bobby. Thank thank you very much Uh, so let's start talking about use cases the the first one user developer documentation is our use case and so the We we can have a chatbot which replies to our question About documentation But also other companies are investigating This the technologies For example, uh, another use case Can be the customer service? I found um linked in post on that And there are some companies. There. Uh, this uh, a list of of that amazon several other lines Bank of america are using this system are investigating To take advantages Which uh, which uh, which are For example 24 hour per day Availability seven days per week because the system Don't require people so can run Every time and also For immediate accessibility because people Do not wait in long phone, but immediately can access to the the service and sometimes when we Ask something to people on phone it requires to Search something read but in that case If we have an AI system We have the we can have the response in real time or in a very short time So the the system that system can reduce the response the response time And why open source? The the main reason is to preserve data privacies The data privacy and security because we can host on previous If we have sensitive data, but companies and communities organization Always have sensitive data so hosting our system in On premises so in our service servers. We can preserve the the privacy and also for performance because recent research A low open source model Which can run on consumer single gpu pc. We can talk about that Also, I do not avoid to mention chap cpt but chap cpt. It's It's using is running on a very strong server. We know and also costs Because if user the user requests are Are very high So to to save costs We have to install system on cloud or in our server and open source allows also to improve our skills or skills About our collaborator employees because they can ask all the knowledge also the code and open source also produce high quality and flexible system Because the a large community can check test evaluate maintain improve the source code and also we can change The system also in a custom way I also found a very interesting article from Gardner, which said that to preserve privacy and security to assure security It's important to apply DevOps feature in AI models And in this way in the future companies can trust to AI system So it's very important to apply this feature to to AI system and open source can help us to do that Now some technologies used by open source This is a link of research paper which use this technique low rank We know that AI knowledge is Generally lots of metrics Which represents the weights of neural networks On the left On the left we have the Metrics Which is D times D So it's very high metrics, but we can use a trick in this in that paper the authors uses a trick to have the same metrics represented by two smaller metrics a the elements plus b elements is less than the blue one Okay, it's an approximation but good performance And in research paper All the results are using are obtained with benchmark data and public access And okay, but also the representation of numbers because we know that the weights on Neural networks are normalized. So our number between zero to one and generally for decimal decimal data we use in computer science we use 16 bits or 32 bits or 64 but in with small number This is a trick to use only four bits in that case we save a lot of memory So quantization plus low rank save a lot of memory And also the techniques for the training stage The second stage of training is the fine tuning It's as a good performance in in the left solution is like chachi pt which used human feedback But as mathematical results We can avoid to use human which is very very expensive for time but also for money and and and so on so in our system This is the the last slide about the technical result we use R AG retrieval of method generation which means that we have on the bottom right our pre-trained model and for our for our context we use some knowledge sources which can be pdf files websites like read talks documentation but also github issues and poor quests discord conversation and so on So this time um reply using the general AI model with context our context data uh which are the which is the current architecture and which are the current data flows so the system has two steps two main steps the first step is to create the knowledge base the second in second workflow user can ask question to the system so we use the well known long chain open source library which read um documents and split and chunk send that chunk of documents to uh an embedded model which returns vectorized chunks means that can use distance uh search as similarity search and last step about the um ingestion flow uh that data uh will be stored inside a vector database now the system can um provide the user interface for to the user user can ask question question uh will be sent to the the system which uh again uh send text messages to the uh embedded model we return with the same metrics the vectorized messages which uh will be sent to the uh database and returns using similarity search the retriever text chunks now model can work out using the query which is the question from user and the try that the retrieved text chunks and respond uh can can send the response to to the system now the system can show the response to to the user so two data flows the green one the for to build the knowledge base and the blue one to um for the run to uh reply to the question uh from user and for the future if AI if a team what's one to do now we want to build a prototype which has new feature features one is the a modular architecture using container which can be installed on premises or on cloud the same architecture um and which has a standard graphical user interface using uh JavaScript frameworks like react.js and work.js we we are deciding and one activity can be related to the next the hyper ledger mentorship program 2024 edition and clear we want to install decided to install on premises or GPU cloud with new feature preserving data privacy with security and reliable system and also with DevOps and feature like continuous integration delivery on deployment for example if we we want to change the model or improve the model want to add new knowledge to the database the that activity will be transparent to the user about cost this is the first very first study of the cost on the blue line is the cloud solution and the red line is the on-premises solution when if you use a cloud solution we do not buy hardware on the other side if you use on-premises solution we have to buy a strong server we can also talk about the which what does it mean strong server so we can start with a like high costs but after some months you have to pay we have to pay only for the power electricity and so the on-premises is cheaper than the cloud and this slide show an estimation of annual costs after one year on the left cloud solution if we spend $2,000 per month and we we see we saw Google cloud but also AWS cloud this the the cost is similar after one one one year we have $24,000 on the other end on the right we have the on-premises cost we have to buy a server and also pay $200 dollars or $250 per month for the electricity cost and we have more than $10,000 but less than the cloud solution so the question how AI FAQ team can test the system can go ahead there are also cheaper solutions and the cheapest one is to use the on-demand not only Rampod there are several providers which where you can use an on-demand server and it's very cheap because less than $2 per hour and this is a very strong solution 80 GB of GPU memory but we need I think 24 GB so less than $2 and now we can talk about the mentorship program I'd like to pass the I'd like to ask her to Bobby if she wants Hi everybody yeah so the mentorship program again it's a great opportunity and that's how I met Jan Luca and Trupor was last year through the mentorship program they joined the program as non-compensated because there was one person who was compensated who did a fabulous job for us but was more into the documentation and as this morphed into the AI Jan Luca and Trupor came with us and have been working really hard on this and I'm very proud of them we meet every Monday in an open call that can be found on the hyper ledger public calendar on the hyper ledger wiki page which is wiki.hyper ledger and you can find this project under labs and you can find the meeting times under the calendar public meetings so in the next I guess two weeks the TOC will decide which projects are going to be sponsored under the mentorship program and I'm confident ours will pass and make it fingers crossed and then it will be open for people to kind of apply for the mentorship program and again there is a paid component to that but we also do love anyone else who just wants to learn about it and volunteer their time to join us as well along with that we're looking for companies to adopt this technology and help us sponsor this initiative by becoming anyone who is already a member of hyper ledger is more than welcome to join the meetings and and help us with the the next couple steps so if you have any questions about the mentorship program you can just go to the wiki page and type it in the search engine and it'll give you everything for the 2024 mentorship program coming up it is not just our initiative for the AI FAQ but there's other opportunities in the mentorship program available so do not miss an opportunity to learn about the newest technology coming up in the next session of the mentorship program so when we think that's it for that topic did I leave anything out thank you Bob I don't know if people wants to is yeah okay I if you want to share screen yeah oh sorry okay you can okay and I do got some questions from everyone to ask from the bot post we have Bobby and let's see how it will going to respond everyone can see my screen right yes oh yeah you don't take only few seconds I have this could take some time and if till then if anyone has any questions for you and Luca regarding the presentation any doubts they can unmute their mic and ask directly oh here it is so how do I create a bootable DVD and so have you got the answer for it okay what are the what are the system minimum requirements for installing so we didn't specify which system we were talking about if we will again ask it and mention about iora and then it can give us an answer for installing Iroha so it's my first question when I'm installing it what are my minimum requirements yeah yeah I will try to apply the question after the demonstration yeah it takes some time because it you know schemes through through the whole documentation and then it gives you the appropriate answer to it okay we got it so what are the minimum requirements helpful answer according to the official documentation the recommended minimum specifications for a 32-bit systems like this so basically it will answer you and for the specification of 64-bit system argis so this is how we do it and the one we that we asked for is here so moving here everything like all those questions that like frequently got asked like how can I install iora on mac so some specific questions got up and for our question we got answers to install iora you need a 22 gbram and 34-bit cpu and all that again we have one more questions that brad asked that was is the solution base a database solution so let's check that out hopefully the it will answer in much less time like okay 70 seconds so here we got the answer based on the given material it is clear that the knowledge base is not a specific database solution while the material mentioned a knowledge database knowledge base as a potential resource for finding solution no specific database solution is mentioned it is possible that the knowledge base could being implemented using any number of database solution depending on the requirements in the constraints of the particular system being developed therefore it is it would not be appropriate to infer any specific database solution is being used or recommended based on this given material alone so here is the demo simply I answered everyone who asked the question in the chat box so someone mentioned well mentioned lol few is too much yeah it sometimes take time totally depends upon like how fast our computer is working so that's that and moving back to the presentation any questions regarding the slides that will give in here okay if I can try to reply to some question in in chat yeah because there are okay okay is the knowledge base a database solution yeah the yes is a database solution I also use it I tried different solution now I mean we are using chroma db which is a civil light solution but also I see posgre as well solution and other kind of the standard of the base it's a standard of the base but now there are several solution ad hoc for this this application so in the future we will see database dedicated to vector dbms dedicated to vector database and also another question is okay to I think excellent summary on the cloud where some premises cost yes we investigated the cloud solution trying to compare the on premises and cloud also I found several articles on the chat gpt API I didn't mention here because the this is a presentation with focus of open source and the result is if the the number of requests is very high the cost will the system will be very very expensive with chat gpt if we talk only about open source the on premises solution required could buy the server obviously but the guaranteed the privacy and also the monthly cost is will be cheap so will be the best solution is the the solution on on premises in our hardware in my in my opinion and too much yes too much okay I have also an article before if I can share the screen yeah okay thank you for this very very recent research result are in like that one in that case researcher show how to fine tool the system which is very expensive activity using an rtxx gpu with 24 gigabyte of memory is it the cheapest solution it's very the cheapest solution is an article published on on march the this month and so any I think open source community is focusing in this goal to try to have a lightweight models which can be used in consumer hardware and so and also there are some are some other results to use cheaper hardware and I don't know if the any other question about system I guess no more questions so let's move forward with the presentation yes if you want to shell screen yeah okay you can shell screen thank you yeah so moving forward we have how can you connect with us and please join the discord server of hyper ledger foundation there you will find a channel named as AI FAQ and you can message us there and ask conclusions if you come up with anything later on regarding the presentation or how you can contribute in the project and regarding anything the current available version is available on github which I have shared on the chat box also the hyper ledger wiki of AI FAQ the link will be shared in the chat box as well and these are some useful useful links that we have used while creating the presentation and the technical part of it so these will also be will be in the chat box if you want to go through them in the depth and you want to ask any questions can look us link them is also in the chat box and how you can join us is you can join our public meetings on Monday morning so basically if you go to the hyper ledger foundation wiki page there you will find the public calendar and there will be public meeting of AI FAQ every Monday morning and you can join us there and thank you everyone for joining us Bobby I would like to give you the mic here Yes I want to thank you Tripur and Jim Luca for this great presentation again there's so many use cases for this that can come out of developing this and I look forward to seeing everyone on Monday I'll talk to them any other questions please ask them now Okay that's it thank you very much everybody Thank you everybody thank you Bobby Thanks everyone