 Hi everyone, welcome to Hyperledger Foundation's in-depth webinar with Espeo. Please just give us a moment to allow for everyone to join and we will get started shortly. Okay, welcome everyone to another session of Hyperledger Foundation's in-depth member webinars. Today as you can see we have Espeo blockchain. They're going to be talking about the multi-cluster deployment of Hyperledger Fabric Network using the Fabric Operator. We have Marcin who's going to be presenting that information with you today and answer any questions you have as well as his colleague Simone Bidula as well who will be joining as well. I'm just going to go over a couple of quick housekeeping items. One, first of all, to share that all are welcome here at Hyperledger. We do conduct all of our meetings and all of our events under the Hyperledger Code of Conduct and the Linux on Ancient Antitrust Policy, so please look for those on our website or on our wiki if you'd like to learn more about that. This session is being recorded. It will be available on the events page of our website, but also on our YouTube page under the webinar playlist soon after the event. We are also live on YouTube as well. Any slides that are available will be on the event page of our website, the webinar page that we have on the hyperledger.org. These sessions, these webinar sessions are really intended to be a time for you to learn and connect with the speakers, so please feel free to raise your hand, post questions in the chat or the Q&A box. I will be monitoring it throughout the call or throughout the webinar, and Marcin and Simone may take some of your questions throughout the session, and as well we will leave time at the end for questions, so feel free to jump in and speak up. This is your time to really engage and learn with them. I'd like to invite Marcin to let us get started. Perfect. Can you see my screen? Yes. Perfect. All right, so hello everyone and welcome to the webinar. Today we will talk about the multi-clustered deployment of Hyperledger Fabric Network, assisted by Hyperledger Fabric Operator. Let's start with a brief introduction about the presenters in the company, and we'll mention what you will learn today. We will discuss the business problem, so how do we try to find the client's needs, what problems we had to resolve to achieve them, and we will do an introduction to Hyperledger Fabric Operator and show you the proof of concept of our solution. A big part of the presentation will be the demo, where we will actually show you how to deploy the network on two clusters, and after that we will sum it up and proceed to the questions the audience will have any. Also of course during the presentation you can ask them, so I can try and answer them on the go. So after this meeting you will know more or less how things work in our company, how we discover requirements and tools to meet them. You will also know what problems you will face when trying to deploy a similar solution. We are representing SPO Blockchain, which is a company based in Poznan, Poland. We introduce Enterprise Level Blockchain Innovations to Business. We are a Hyperledger Fabric Certified Service Provider as one of 17 certified companies. We have around five years of professional experience and we can help you with Blockchain Consulting and Development. My name is Marcin and I am a recent acquisition of SPO Blockchain. I started working for the company in March this year. Currently I'm a Blockchain and backend developer. I have experience working for industries like telecommunications, e-commerce, fintech and Blockchain. My current technological focus is on Nest, JS and Hyperledger Fabric. To help with the business part of the presentation I invited my colleague from SPO, Simon. Would you like to introduce yourself? Hello, of course. So I am in SPO Blockchain. I am also a Blockchain developer. I am also working on the Node.js site in many projects, but I started my career with a scientific study of human brain. So I was using mostly newer imaging to study the architecture of the brain. And then I moved to the IT sector as a developer and I helped clients with building solutions from the different fields like finance, marketing or HR. I was active on both front and back end and my main language is JavaScript, mainly in the TypeScript formula. And I want to share with you and convey how at SPO Blockchain we are approaching every project. So at the very beginning we are starting with the client. We strongly believe that every successful business needs to put the client at the first place in the heart of the business. So what we have learned when we were discovering and learning about the client is that the client was from the business consulting sector and it had multiple branches that were fairly independent at operational level. That means that technical and organizational levels inside these branches were independent. As a result, there was a strict security policy when it comes to the flow of the information. So every flow of the information of the branch and out of each branch has to be registered. As a result came the problem that the client came to us to resolve. So the client need was the following. We had several independent branches. We had to register the flow of information between them and it has to be done in the audit table form. So each change needs to be registered. The possible solution needs to scale well because there was a lot of these branches and the client has a great appetite for growth. And the client was also interested in extensive monitoring the whole solution. And the client was also very curious if there will be an out-of-the-box administration capabilities for managing the whole solution. So that was the need. That was the client. And based on that, we started thinking about what tools could help clients, the client with this particular need. So we started from the very bottom. And we realized that because there are multiple independent branches, they need to have a consensus of how the data was flowing in and out of them in a secure and trustworthy manner. So as we put all these things on the Blackboard, we realized that the blockchain will be a perfect solution for the client because blockchain is a way of realizing trust between parties that are fairly independent. So we started from that that we need a blockchain. Then we thought about what kind of blockchain will be perfect for our client. And we realized that we needed a blockchain that will be private. It needs to be secure by default. It needs to have extensive administration capabilities. It needs to be easily monitored. And it needs to be developed by active and engaged community. And also, again, as we put all these requirements on the Blackboard, we immediately realized that Hyperledger Fabric Network will be a perfect solution. So we have blockchain. We have a way to implement this blockchain with the Hyperledger Fabric. Now we started to think about how to scale it. And then we realized that the perfect tool for that will be Kubernetes. So Kubernetes helps clients with scaling fairly independent Docker containers. So every Hyperledger Fabric Network could be in a separate container and orchestrating and managing them could be done from the Kubernetes level. Why this is important? I will just show you one stat. So 85% of admin directors or IT managers believe that Kubernetes will be the most important and the crucial part of the stack within the cloud in the future. I am basing this stat on the 2021 State of Enterprise open source report. So one of the important and crucial part of this presentation is to show you that Hyperledger Fabric could work very well with the Kubernetes inside the cloud environment. Okay, so we know that we need to implement blockchain with the Hyperledger Fabric. It could scale well with Kubernetes. But how we evaluate, for example, that Hyperledger Fabric will be great. So as you can see here on this screen, there is a screenshot from the Hyperledger Fabric repository that is currently at GitHub. There is a lot of data. And on the next slide, we are showing you how you could uncover and use this one page to get lots of data about how to evaluate the tool, if it's good for you or not. So we are always checking if there is an open source repository, like the one that we showed you before. If it is, it means that the tool is open source and that's great. Then we are checking the number of the stars. If this number is high, it's fantastic because many people are using it and are thinking highly of the solution. So Hyperledger Fabric ticking also this mark. Then we are checking the license. We are always searching for a license that is permissible and is open to use for commercial purposes. And that's also the case with the Hyperledger Fabric. Then we are checking the documentation if it is extensive and it's covering many cases. Then we are checking if the tool was used in real business cases. Then we are checking how many pull requests and issues are raised. If there are many issues but not too many pull requests, it, for example, could show you that there are many errors in the software but the community is not engaged enough to resolve them. That's not the case with the Hyperledger Fabric. Then we are looking at when was the last commit and how many commits are inside the repository. As you could see, Hyperledger Fabric is actively developed and it has many commits. So it is a fairly extensive and a big software tool. Finally, we are checking if there is some organization that is backing the tool that we wanted to use. With the Hyperledger Fabric, we know that there is a Linux foundation. So we were feeling quite confident that with this help, this community and this tool, we could provide value to the client. Great. Thank you, Siobhan. Now that we have the problems and we have the tools to resolve them, let's actually talk about the solution that we proposed to the client. So the problem number one was the need of server separation. That was a result of independent branches of the company that is spread worldwide. They wanted us to keep a very similar or even the same pipeline for all of the branches and they had a demanding security policy. So to resolve that, we decided to make a separate cluster for each organization or the branch of the organization in this case with an additional CA server. It contains the same network structure on all clusters and each branch will be responsible for keeping the clusters running. The problem number two that we occurred was the complexity of Kubernetes. I believe it's common knowledge that Kubernetes is much more complex than the current compose. In that tool, we just need to specify the right images, ports, volumes and variables. Well, in Kubernetes, we have clusters, nodes, spots, deployments, config maps, secrets, services, and I could go on and on. It's obvious that it's more difficult as Kubernetes offers much more than Docker compose and we did not want to create all the YAML fires from scratch, handle all the network configuration or store the cryptographic material anywhere except on the clusters. So to solve the problem, we found a tool called Hyperledger Fabric Operator. The main benefit is that you don't need to know much about Kubernetes. It eliminates the need to store crypto material anywhere except on the cluster on the administrator machine, which is us, the developers or network administrators. We only have to store one private key, which is an admin access to perform operations on the network. You will see exactly what that means during the demo. It allows for ISSA integration in case the client would like to expose it to the outside world in the future and it generates all necessary configurations for us to operate on it. We just need to install a pod on the cluster and then we need to install kubectl plugin on the administrator machine. Let's talk more about the Hyperledger Fabric Operator. It's an open source solution for deploying the Fabric network on Kubernetes, as I mentioned. Currently, it's not very popular, unfortunately. It has around 150 stars on GitHub. The goal of this presentation is also to bring some lights to the project. From my experience, this tool really makes a life of a developer easier. It is being continuously developed. It has released around once a month or once or two months. It has advanced documentation and it was first created under Kung Fu Software name, which is a Spanish IT company focused on Hyperledger Fabric. But recently, it was moved into Hyperledger Labs GitHub to encourage even more developers to contribute. It is being used in production in a few projects for now, mainly by KF Software, but I hope it will change in the future. The creator of the tool, David Viejo, is very active on the repository, providing developers with a lot of help. It's also a good sign. Before we go into the demo, let's talk a few words about other technologies that we decided to use for our proof-of-concept. You will see them in the demo in just a minute. To play around with the Hyperledger Fabric operator, we had to make a choice for a few technologies for Cloud provider. We decided to go with DigitalOcean and debate with in-between cluster connection. We decided to go with Freedom as public domain provider and DNS management service. Of course, this is not the only possible combination of technologies. It should work with clusters created on any Cloud providers and with any domain provider with DNS management service. It would be even possible to use DigitalOcean's DNS service. You just have to change the name resolution to DNS service on DigitalOcean. If you want to try the development yourself, you can do it for free. DigitalOcean offers a free trial for the domain for free credits and Freedom offers free domains for up to 12 months. Ideally, for production network, we will use a private hostname resolution, but for proof-of-concept purposes, this solution was sufficient. Let's see what we are actually going to achieve in the demo. We decided to go with it. Marcin, I'm just going to jump in here real quick because we got a question from someone and it's related to the last slide that you were just on. Shane Tay was asking, what is the minimum knowledge expectations for someone who is keen to use the Hyperledger Fabric operator assuming that they're not familiar with Kubernetes? Minimal requirements would be basically to watch either this webinar or the webinar from David Viejo. I will provide the link also at the end of the presentation to it. It's a really good material. It's a great introduction to the tool. It provides a lot of detailed information about how the tool exactly works. For Kubernetes knowledge, it's not really necessary. All we need to do is how to use the configuration files of the cluster, which I will also show in a second, and then how to use kubectl, the basic commands. Okay. And then Ramesh Meryala here is asking, what's the difference between HLF operator and Fabric operator? I think that's... Yes, that's the same thing. So a different name, you cannot repeat the same thing over and over. Yeah, HLF is Hyperledger Fabric operator is just using a shorthand for it. Yes, exactly. Thank you. All right. So we decided to go with the following schema. This is a basic Hyperledger Fabric network with three organizations. We have two app organizations, which include 1CA and two peers each. Our order organization contains one order for each app organization. So we have two orders here in place and one CA. This design allows for an easy expand if we move the CA order to another cluster. On the diagram, for clarity purposes, I did not include all of the connections since the peers are able to communicate with each other and with any of the orders. It would create a giant mess on the diagram. All right. Since we do not have much time, we only have like half an hour to deploy the entire network. I might be going for things pretty quickly and I will not explain the very basic concepts of the tool. For that, as I mentioned, I will recommend the David Viejo webinar, which will be linked in the end. With that, let me try to go to the code. Is the VS Code visible? Yes, it is. Perfect. Now it's back to your slides, though. Yep. I wanted to show first how we can actually create the clusters first because I already prepared the environment because it takes a few minutes to spin up things up and assign the public IP addresses. To create a cluster, we need to go to, after we make the account, we need to go to create Kubernetes. There you will have the cluster creation website. We can choose where we want the location of our cluster. For my clusters, one is based in Amsterdam. The other one is in Frankfurt. We use the latest version of Kubernetes. We use all the basic settings. Just change the name to whatever suits your needs. Then just create the cluster. After we create the clusters, we need to give them a few minutes to create. Then we can go back to the Kubernetes panel. We can go click on the three dots next to the cluster and then download config. After we do that, we can go into our code. To use that, we will have to export to config variable to operate on the correct cluster. As you can see, I prepared three columns of nodes and tools to synchronize them together. On the left on the left side, you can see the commands that I will use on cluster one, the middle, the cluster two, and on the right, the information, what we are doing right now. So let's export these variables pretty quick. To see if we are connected to the cluster, we can use the cluster info command. It should return us different IP addresses for the control planes. We are connected to different clusters and everything is running properly. So now that we have that, we will also need the domains. Recently, something was wrong with the freedom website. I will show you the screen shows that I made to not worry about it not working now. This will be also, I will provide a link to this repository with all the commands and all the screenshots later on. So this is the main website. Here you can type in the domain that you want. First, check availability. It should show you if any domain is available. For this, you can add in another one. For now, we will need three domains. One for each organization. So one for organization one, one for organization two, and one for order organization. When you have them all, you can proceed to checkout. Here you can change for up to 12 months for free to use this domain. Then you can continue and finish the purchase of the domains. And then after you logged into your account, you can go to services and my domains. Then open each of the domain settings in another tab. There you can, on each tab, you can go to manage freedom DNS. And this is the screen that you will see. I already added the record so you can see the configuration that I used. I will show you in a second how to get these IP addresses. But this is the configuration that you should use to look very similar if you're trying to deploy the same network as we will do right now. Okay. We can proceed with the installation of the tools. I already installed STO CTL. To do that, we need to use this command. And we also need to install the hyperlitter fabric operator that I mentioned. To do that, we need to add the Helm repository of KF software. Then we can proceed with the installation to check if the things are running. We can type in kubectl getpods command. As you can see, the project fabric operator is running on this cluster as well. To check if the STO is running, you need to change the namespace to STO system. So it's also quickly to that. This here is also running so everything is installed as it should. So to get the IP address that I mentioned just now, we need to type in this command. This will return all the information about our Ingress Gateway. This is the public IP address that our load balancer has assigned to the service. And to check if the DNS is refreshed, we can use any command like peer or dig or probably even tracer it. This command should return the IP address that it resolves to. So you can see this is the same IP address. Let's put it on this on the other terminal. This one should return the other IP address. As you can see, it looks correct. It's also quickly checked. Check the order IPs. It also looks like it's running. So with that, we can proceed to deploying the actual network. I will create a few files to organize the deployment configs. And to deploy anything, we need to know what storage class the master operates on. So to do that, we can use kubectl getStorage class. We will just use the default one. So do-plot-storage. You can see it's being used right here, the deployment command. And we need to provide some basic arguments like the storage class just mentioned, storage capacity, the name of the CA, and ID and password to generate new users on the CA. Hypervisor Fabric Operator offers an option to deploy things right away if we removed these two lines. So just this command would result in a CA being deployed right away. Just with this, we can also inspect what's happening, how the deployment looks. So let's quickly deploy it. Let's do the same thing for the order CA. Let's also run it on the organization too. And with this command, you can wait for all of the CAs to be deployed because currently, they're just started, and we need to wait for them to be operational. Hopefully, it will not take too long. After that, we can see the pods are really running. On the cluster one, it's already done. Both of the CAs are running. So as you can see, the pods are created. And the other one as well. So with CAs in place, we can proceed to deploy peers for both organizations. So for each peer, we need to register peer identity on the CA. Then we can create and apply peer configuration. What differs from single cluster deployment is that for each service that is exposed for communication with network components from other clusters, we need to add the following flags. So hosts, which is the domain of the complement, the URL to access it. Ingress Gateway is the default value for the Ingress Gateway. And they still port this 4.4.3, which is also the default value. So if you install it the same way I did with this command, then everything should be the same. So let's also run these commands. I have peer1 is created, peer0 is created. Let's go with peer1. These commands are very similar. I'm just changing peer identity. I'm creating a new one. I'm using a different name for it. But basically they look almost the same. Let's do the same for organization2. Both created with no problem. So without stopping, we can deploy our orders. The order for organization1 will have no problems deploying as it follows basically the same path as the peer deployment. So we need to create the order identity. We just change the type to order. Here above you can see those type peer. And then just comment or not create. You also need to add the hosts and Istio parameters to it as it will be communicating with the other cluster. So let's also run these commands. And for order2 we will have to make a few adjustments. Function or not create, which we just executed for order1. It's responsible for generating order pod configuration. It scans the current cluster for certificate authority provided by the CA name flag. So it's no problem for order1 since the order CA is running on the same cluster. But for order2 we need to make some changes manually. As you can see, if we try to do it with the order CA, it will throw an error that the CA is not found. And so how do we avoid that? We can use the CA that we deployed before for order2. Just before applying the configuration, we will make the changes manually. So let's first register on the first cluster and the order2 identity. Then let's create the manifest file. And let's open these files that were just created. We will also need to copy some values from order1 configuration since it's used the correct CA. So as you can see here, we will have to change the CA host parameters on this cluster. And it will have to point to order-ca.hlf.dk, which is my domain for the order CA. We will also have to change the port to default port of ingress gateway, which is for code 3. We will have to also copy the entire certificate from order1 and paste it in this place. We need to also change it for the TLS connection. So let's quickly do that. Change the port to default port 3. And let's copy and paste that certificate. And to be able to connect to it, we need to also add it to the host area here. And with that done, we should be ready to go. Let's save it. And let's close the files. And let's proceed to apply the order2 configuration. Now let's run the commands to wait for the deployment of both peers and orders. So as you can see, the peers are already deployed. Here you can see what the order is already also deployed on cluster1. And we will just have to wait for the order on cluster2 since we just applied it. You can also see that both are running. This is the order and the two peers that we created. Okay, perfect. The order is also deployed. This is what we created. Now in order to deploy the chain code and create the channel, we need to generate admin connection files. To do that, we have a few comments to run. First, inspecting the organization. This generates a simple YAM file which contains all important information about the organization, along with a few certificates to authorize the connection. Second, we have to register and enroll the admin user. Register command was used before with the order and peer creation. Now we also have to enroll it to generate the private certificate on our file system. Then we have a quick few tool that we need to use in order to add the admin file to the organization file. So let's execute this. Perfect. As you can see, the admin contains a public and private key. I'm not afraid to show it since the network will basically be removed after the webinar. And then we can see the organization one config, which also contains the important information, like what peers do we have and what orders are connected. And if you also look for it, you can find the admin user that we just created. Let's do the same for the order organization on cluster one. Let's do the same for organization two. And this file won't be needed for the demo, but just have to so you can guys have a look. If we need it for order on the cluster two, we need to generate the admin on the first cluster. And then we can just use that to create a configuration on the other cluster. And it also should be working. Okay. Now that we have admin rights to operate on the network, let's deploy some chain codes. I prepared just a basic favcarn check code written in JavaScript. It's stolen straight from official fabric samples. It takes a while, so let's run it in a separate terminal. We need to provide it with a path. The chain code sits right here. We need to provide it the config that we just created, language, label, the admin user username, because you can add multiple users to the configuration and then appear that you want to deploy it on. So let's run these commands. No quick error, so just a good sign. Okay. So let it run in the background. And while this is installing, we can create a channel. So let's start with creating a genesis book, which is a simple command. We need to provide it an output where we want to write it to channel name and then organizations that we want to include. And because the org2 is not visible on the cluster that we will be creating it, even if I include it in the command, it will only read the peers from organization one. So we will have to add the organization to manually later on. So to join the order to the channel, we need to enroll a TLS configuration because by default we are using a craft consensus mechanism. We will have to enroll it using TLSCA. And we already registered the user. We can use the same credentials to just generate a slightly different certificate. Now we can join the channel. Status code 201, which means everything succeeded. Now we can proceed to join the peers to the channel. We also need to provide just the name of the channel and then the config username and peer that we want to join to the channel. Channel join. So let's do the same thing for the other peer. And the first thing that we should do after we join the peers to the channel is to set an encore peer. So the other organizations can discover what other peers are in the network. Okay. And now we can try to inspect the channel on this cluster. We can see that both of the peers already joined and it's two transactions were consumed already. To add organization two, we need to inspect the organization on the other cluster. I will also move it right away to the resources folder that we created at the beginning, since it will become a big cluster clatter in a second as we will be updating the channel configuration. With that we need to execute this command, which automatically for us adds the organization two to the cluster, to the channel, excuse me. Channel updated. It should update. Yes, there you go. One peer already registered the change. The other one is synced already too. So now we need to do the same thing as we did on the order one. We need to create the admin TLS configuration and then we can join the other order to the network. So now the order two is in the channel, but it still can't do much. Let's say we want to add it as concenter instead of order one being the loan leader. The Qubectl HLF plugin offers add the concenter function, but unfortunately it doesn't work when the order is deployed on a different cluster. So we will have to make the change manually. We will use config.exlator from fabric binaries, which I also installed in this folder. So let's export the path. First, let's fetch the current channel configuration and encode it as we will need it for later. Then we can make a copy of the JSON configuration and we can make changes in the file. So the quickest way to navigate to add the concenter is to find the existing order. Just import one that HLF in my case and we need to add our order two to this admin array. So let's quickly do that and we will have to add the order two into this concenters array. So as you can see here, there are two certificates, hosts and ports. You can copy the entire thing and make a second element in this array. We can remove the certificates as we will have to fetch one ourselves. We can change the post to order two. The port stays default. Now to get the certificates, we need to first, we need to fetch it from the pod file system. So let's get the name of the pod that we will want to get it from. So now we're ordering node and let's type in kubectl xx-at, paste the pod name and then let's paste the rest of this command. This actually gave me a pretty big headache because I couldn't figure out what's wrong and this line was life changing because when Bash was adding carry returns, which are invisible signs that were changing the base 64 encoding of the certificate and after removing that, finally everything started to work properly. So let's remove, let's copy the certificate. Let's paste it in the correct place just here and here. Let's save it. Now we can close it and proceed with the standard hyperlitter fabric channel updates. These commands are also basically taken from the documentation. So I will not go into details on what they are doing. Basically, we are creating a transaction that will update the channel. Before we can apply it, we need to sign the update by the current authority, which is the order one in this case. But as you can see, unfortunately, there is a small bug in the program that I did not have a chance to fix yet. Need to go to order one config and not the very bottom. The tool expects a map instead of an error. So we need to change that. And now if we execute it again, the command runs successfully. So now we can update the channel. The transaction is committed. So it should also show here, perfect. The peers are still synchronizing. And now we can join the peers from order, from organization two to the channel. It might throw an error because the channel updates makes the order unavailable for a second. But thankfully, it worked already. You can join the other one. And you can set the anchor peer, just like with the organization one. Okay, let's see if our chinko is installed. Looks like everything went smoothly. With the status 200, here we will need to use this package ID. But we can also just get it by using query installed command. And this is the same result that we saw in the other terminal. And let's export cc underscore package underscore ID. So lazy developers like me can just copy paste the rest of the commands easily. We can query all approved chain codes. For now, we shouldn't have any since we didn't approve any on the other cluster as well. Let's not forget to export this on the other terminal as well. Now we can proceed to approve it. We also need to provide it with the configuration, the user peer, which will be used to approve it, then the package ID and the version and sequence. Since this is the first one we are deploying, we will just use version and sequence one. Name is Fobcar and we will use ant policy so that both of the organizations need to approve it before committing. Yes, do we have any questions? Yeah, just someone, Ramesh, jumping in here again with a question about whether the container registry is supported for chain code installation. I don't get the question. Could you maybe rephrase it? Let's see. Ramesh, do you want to come off mute and ask the question and clarify? Let me find him in the attendee list real quick. Ramesh, would you like to come off mute and ask your question or please type it again into the Q&A box with a little bit more language around it and hopefully Marchion can answer it. Okay, maybe while we wait, I can go a little bit forward since we are almost running out of time. Yep, go ahead. Okay, so let's run the approve command and let's run approve on the other cluster also. If we execute query approved again, it should hopefully show that we indeed approve this chain code on this cluster. Should also perfect and now we can commit it so we can actually use it. After that, we can invoke the initialization of the chain code and after that we'll be able to execute the chain code commands to chain code functions. Okay, it went through successfully. We are already on block nine and we can invoke create car chain code function. We can create cars on both of the clusters. The transaction is successful here and here as well. Now you can query the cars to show everything is running properly and as you can see it's working just fine and the two functions that we executed were created a new transaction. The peers are all synchronized and that's basically all from the demo that I prepared for you. I just some quick cleanup functions. I will not execute them right now in case there are some questions afterwards. Simone, would you like to continue with the summary? Yeah, sure. We will now move to the take home messages. So probably one of the most important things that we have learned because Marcin showed that there is a lot to digest on is that hyperledger could be used in a very complex scenarios. These complex scenarios could be used even in the cloud environment. We have also learned during this session that there is a lot of tools that could help you with implementing even the most complex scenarios that we have faced on. So there is a large community inside the IJLF and it could help you with deploying your blockchain network to the cloud and manage it with the Kubernetes. So I believe that these are the most important take home messages and now we could move to the next slide and one more. So we have good news. So if you are excited about hyperledger and would like to join us, we are hiring. We have open positions for senior developers. So if you already have experience in blockchain, then you could join us. But if you are not an experienced developer in blockchain, we also have a program for starting with that, starting with hyperledger. So there is a junior academy, junior blockchain academy that you could join, and we will teach you and help you to become a blockchain developer at the spell. So that are the great news if you are interested. And if you after even this webinar would like to have in touch with us, then please write to us or write to Swavomil. He's a great resource and definitely will help you with any problem that you have. So that's all on our side. Thank you. And if you have questions, we are happy to answer them. Yes. So Ramesh, maybe you would like to open with your question. Hi, Marcin. I got booted off. Could you make me host please? Okay. Thank you. Okay. So Ramesh, would you like to come off mute or Jeff also had a question as well? If you'd like to come off mute and ask your questions, we have enough time to address those live. Yeah, definitely. Hi, everyone. So my query is, so every day we'll be doing the chain code development that we are pushing it to the Docker container registry. So is that supported in the HLF operator? Rather than giving the source code path, I just mentioned the Docker image path so that I would like to install the chain code on the peer section. That's my query. I'm not exactly sure since I'm not that much proficient with the tool, but in general, the tool is very flexible. So it should be able to, if not with the tool itself, then it should be possible to make some manual adjustments. And one more thing is the Fabric Gateway also supported on the HLF operator. From the 2.4 onwards, Fabric Gateway has been produced. Oh, I didn't have a look at that yet. So I need to catch up. And how are the monitoring of the blockchain, something like Romotis and Grafana? That is also another step that we are considering. Because for now it was just the proof of concept to see how difficult it is and how much time we can spend on it. So this is the next step that we will try to implement. So because I'm already using the Fabric operator, that's where I'm trying to bring up all the things. But I have some challenges actually. So that's the reason if I can switch to the HLF operator, so if we can address all these things, then it would be very easy to integrate the HLF operator and the Kubernetes to bring up the blockchain network. For now, the tool provides most of the basic operations that we do on the Fabric network. But if you want to do something more complex, then usually it requires some more work or working around some issues. Yeah. Thank you. Thank you, Ramesh, for your question. Jeff, did you want to come off mute and ask your question or we can also just... Sure, I'm actually off mute, I think. Hi, Jeff. Yeah, I was just curious. Everything that you showed was that part of the operator framework. And if so, if I already had an existing configuration, is it possible to import that into operator to make it possible to use operator going forward to manage it? Okay, as far as I'm aware, moving the current configuration is not possible for now. The network needs to be created basically from scratch. But all of the things that I showed during the demo, except modifying the channel manually, of course, which is just a part of the framework, everything was executed using the HLF operator plugin. Okay, thanks. Any other questions? Let us know in the Q&A box or raise your hand and I can unmute you. I will do one thing. I will integrate the step in the front end device so that it will be like a complete example. So I already have that set up with the Fabric operator. So what I will do is I'll create the pull request and I'll just add it in the current code, whatever you hosted on it. So that it will be like end-to-end pull stack up. Nice. Yeah, the Fabric operator really needs some attention and some contributors. So that's nice that you're helping in that topic. Yeah, because I'm already started in the Fabric operator. So I think this is the same similar kind of work I think I can do it for the Fabric operator, the HLF operator. It's perfect. Thank you, Ramesh. That's wonderful. Okay. I don't see any other questions or hands raised. So I want to take the opportunity to thank Marcin and Simone for their presentation, for the demo. It was really informative and also really demonstrated clearly the process that you all took and the lessons learned and take away. I really like that you included that part at the end as well. So thank you all for joining today. Feel free to reach out to our team or the Espeo team if you want to understand more about what they're doing or understand more about the Hyperledger Fabric operator. We do have another member webinar coming up next week. So please join us again for Indicio. They're going to be talking about scaling verifiable digital credentials. And you can sign up for that on the events page on our website, just like you did for this one. The recording for this will be available shortly on our webinar page on our website at hyperledger.org and also on our YouTube page because we were YouTube live. So that'll be under the webinar playlist there. But join us next week for Indicio. If you'd like to continue the conversation, get involved in our community, please join our Discord. We have chats for chat channels for all of our projects, all of our sigs and communities, including our labs as well. So please engage with our community there. It's a great place to get your questions answered. And finally, thank you again to Espeo and thank you all for attending and joining us today. We look forward to catching you on our next member webinar. Yes. Thank you very much for listening. Thank you. Have a great day, everybody.