 Okay, boy. Hey guys, how are you? Thank you for coming. Thank you for attending. I know everyone has traveled Quite a way to get here, right? And I thought maybe before we even start talking about anything like Kubernetes or operators or the like Well, first let me set some context. I'm Josh Neubel. I work at IBM in the research. I'm a technical staff member here I've Actually a fairly newcomer to fabric. I'm about one year into working with the product And I think this is a really good opportunity to provide some outside Observations without any of the backstory or what happened what worked what didn't work and just provide a perspective of What's it like for a newcomer who has you know, pretty good technical background coming in and trying to work with the product What works what doesn't work? But maybe to set some context. I just I love this picture because Even though my my journey into blockchain is one year old. What is this? What is this map anyone from the audience? Please tell me what this is Right. It's it's 500 years ago, right our blockchain journey of Commerce across the globe did not start five or six or seven years ago This is the story of we are still in the age of discovery, right? This is Magellan's route as he and his crew Circumnavigated the world looking for a commerce route across the entire planet All right, a couple things to think about like look at if you can see like even the winds of fate from distance are Infants, right? They're blowing these winds. They don't really know where they're going. I love this map because if you look at it What's what's going on? What's going on? We don't really know what the world looks like We know we're interested in finding it out Think about the notion of the projection of the map that's there. It's not necessarily accurate mathematically But it has a pretty good idea of like keeping keeping track of where you are in the position But like what happened to Antarctica, right? So my point is It still feels very much like we are still in the age of discovery for blockchain And despite the ups and the downs of working with blockchain, you should put it in perspective, right? Magellan's route around the world was three years long, right? He started off on a voyage with 300 people ended up after three years He didn't actually even survive the adventure but died somewhere in the Philippines, right? There were stories of scurvy and starvation Newtonies so a newcomer Josh comes along and hello fabric. What's going on? um If you look at the open source documentation for getting started with fabric the word cloud on the right Is a representation of when we looked at the fabric labs only the labs in the hyper ledger labs projects we found about 35 different projects which had some crossover with operational impact of how to build Fabric networks how to work with them how to deploy them how to start them how to manage them and I have a hunch that the number of projects that made it into the labs We counted them as 35 and this is not just like the commercial blockchain as a platform x y or z Or any of the number of companies that built a business strategy that decided to go it on their own with their own deployment There are probably It's either log in Or square root event. I mean there's like maybe a thousand other projects That have also tried to build like people have sat down and said I want to work with fabric But I can't we only saw the top 35 that people were willing to put out and in the open source domain so my first Like gut check about fabric is You know, what do I do before this? I built um machine learning platforms and deployed them on cloud and tried to make them simple and I'd say my My chase on this is I've noticed The realities of what it means to deploy an application in the cloud Maybe starting around 2012 For the past 10 years or so have changed every 18 months like clockwork The definition of what it means to be on a cloud changes on a regular cadence and When you sit down and work with fabric and you don't have the solidity But it feels like you're stepping on quicksand what this does for the like the innovation lifecycle of I have this idea that I'd like to try out Right. I just want to I want to try something new And if the only mechanism you have to get from you know, the innovation lifecycle from I have a great idea I want to try it out and then go and say based on the outcome of my experiment. What do I do next right? It's like a continuous innovation loop My feeling of fabric is like I can't I can't just try it out. Right. I have to go and invest Years or millions of dollars. I have to go find technology partners and vendors I just wanted to try something really simple Which is to sit down and work with hyper ledger fabric set up a channel set up some chain code I just want to sit down and write some chain code and the feeling of being blocked Like I think it's important to think about The human aspect of this and it's odd for me to talk about how it feels to work with technology, but Does anyone here deployed a fabric network raise your hand if you have not raised or if you have not deployed a hyper ledger fabric raise your hand It hurts right and it's hard to put a finger on like why Why do I feel uncomfortable when I'm working with fabric? Like has anyone felt that unease if it's like it's really complicated and I don't understand the mechanics What's going on here and the best analogy that I can think of it's like There's supposed to be some interconnected lines here. Does anyone recognize the constellation? Yeah, it's Polaris. I I know where I'm going right if I can find that north star then I can orient everything It's hard to put a finger on the human aspect of what it's it's like to sit down and work with technology when you can't make it work And we're all very creative. We're all very diligent. We're all very disciplined We keep working working when you can't make it work. You're like something's not right. So my My theory is like, oh, let's do something different right pulling on the navigational theme or the nautical theme what really need is An executive officer, right? We need someone to say I just want to step back and think about strategic objectives of where I want to go I don't want to worry about the mechanics of setting the trim I don't want to worry about the mechanics of aiming exactly how to turn the rudder I don't want to have to worry about all the nuances of operating a sailboat I just want to think I want to get from here to there. We need like an executive officer if you will so I'm very Oh first What's different between this picture and the one we saw before? anyone What Yeah What happened in 500 years of our chain at some point someone said a couple things first So the canals are obviously the striking difference. Someone said wait something is different. I don't need to sail These are some pretty stormy waters down here. I don't want to go anywhere near that right someone said Can we just please cut a Canal between continents and take a shortcut Wouldn't it be cool if we had a shortcut for building hyper ledger fabric networks where we could just say The world has advanced technology has advanced even look at the map Right the map here is a modern rendition and there's a couple things that stick out for me not just the Not just the shortcuts Look at the projection the mathematics of the projection right, we're better mathematically look at the The intricate details of the lake region in canada and the upper upper territories There's a satellite here, right? So our expectation that circumnavigating the world used to be three years of work Has been fixed because modern satellites do it every 90 minutes, right? We have a much better understanding. We're not necessarily in the age of discovery so kind of setting the context i'm I'm just i'm absolutely pleased. I'm just really really pleased. I have tremendous aspirations, but realistic expectations and the the change the thing that's different as of today or a couple months ago is we have released a Open source fully open source fully community Golang based kubernetes operator for hyper ledger fabric. It's the whole thing. It's production ready. It's production grade It's very cloud ready. It's cloud native Most uh Feature comparison charts will say it does this and it does that. You know, does it really fold your laundry? Anyone no, but it might i'm sorry to say give you a little more time to fold your laundry right So it runs fabric on kubernetes, but So is anyone here familiar with the cloud native compute notion of the operator pattern and what crd's mean? What is kubernetes? Quick cut check. What do people think of when they think kubernetes anyone? Yeah, good. Is it worse or better than fabric? I don't know Right, so there's a there's a trade-off there right kubernetes is absolutely complex. There's new concerns. There's new patterns in the previous slide the world of Cloud native computing has evolved It is fundamentally different now the notion of kubernetes starting off like a lamp stack where if someone says I have a web server I have a database. I have maybe some ingress controllers. I want to set up some networking constructs It's different and and what the real benefit of kubernetes When it happened we said Let's just teach the kubernetes api controller How to conduct fabric for us, right? It's a very different perspective and saying I have some scripts that I used to go and drive some things some imperative control script Maybe some bash scripts. They go and write them and they do some things. They have some side effect And the thing that's different about kubernetes as a platform is it's almost like You know in the j2e Heyday, you have an application server and it's used to deploy servlets Think of kubernetes like the application server for you to deploy things to the cloud one of the one of the challenges that i've seen in fabric are The preponderance of service SDKs or client-side SDKs are kind of a mess, right? Fabric tries to support multi language multi version multi sdks multi runtimes and the like and it's very very complicated So what the operator pattern does is it says, you know, stop trying to directly control all the parts of your system with multiple languages multiple runtimes multiple versions of golang and java and sdk and note all this stuff instead What would just happen if kubernetes api controller Didn't just know how to work with pods and deployments and services But were things like peers and orders and maybe even channels Right or organizations So you can upload a new resource into kubernetes And kubernetes says, ah, something is different. You're telling me a new thing is available And that's a trigger to a callback and some golang code that actually does the rest of this Like the hard work for you, right? I'm not saying fabric is not difficult It's appropriate for automation with computer programs and the cloud native the kubernetes compute pattern fits really well, right? We're teaching kubernetes how to operate fabric What that means is any programming language any client sdk that's supported by the kubernetes community Can be used to operate fabric networks every single one of them Right, that's a pretty big advantage. So I understand it has some complexity to it and there's some trade off to it my finding Kind of mentally is the short circuit cycle of innovation before was this big red x, right? Now even though it's a little bit more complex. There's some magic there and I can say, you know what? Here's the resource specification from my network. Just bring it up Do your magic and now I can step back and think about is this where I wanted to go or maybe do I need to change course a little bit so Fabric operator kind of sits in the middle. I think I talked about this a little bit, but there's kind of three It does very very simple things, right? What we've done is we've taught kubernetes the kube stks the kube apis it now knows about peers and orders and ca and The normal entry point into that is through I'm going to I'm going to call it kube cuddle here But it could be any kubernetes compliant stk language or library So if you have terraform scripts, or if you have ansible scripts, or if you have kube cuddle scripts, or you have custom go line code that works directly with the kube api controller, then this is the same route The other opportunities we have are in many environments. It's Boy, it's really nice to have a user interface, right? In some cases It makes really good sense to see the side effects kube just does its thing. It's kind of invisible. So we have another um, we have another route through a web browser or through ansible scripts That are actually hitting programmable stks in the background against our operations console and There's a little g rpc web bridge in there. So some hidden magic in there But it makes it very easy to have a web user interface if you want If you prefer the ansible target, we can also set up networks with ansible all the way from channel construction From organization and consortium building chain code installation and the like, okay? Um the other part here that this is super important The notion of this bridge right here between You know, what's what is fabric as a network? It's a set of service URLs That's my claim And to be truly cloud native it does not matter where your code is running The only thing that matters is how you find it So the opinionate one of the opinionated perspectives of the fabric operator Is that all of the services are exposed via consistent ingress? All right, so there's a nice dns wildcard naming scheme So everything is exposed within your sub network behind one common entry point So fabric can just look like a set of services that are running So I'm going to call this fabric cli, but this is really like peer binaries order of osn admin binaries of the like If you have built a grpc client stk that speaks an admin api or something think of that as this is speaking grpcs In between here and there so it doesn't have to be the fabric cli, but once you set up the ingress it's ready to go Can anyone read this? I can't So here's my here's my cycle of innovation. Let me let me kick it off um in the true spirit of you know Magellan had a really hard time. There were a lot of risks. I'm going to do something live here and try to actually make a network So on the top hand what I have is It doesn't look very good I have a console on the bottom for a command line client And on the top there is a kubernetes View it's an ascii client that will show you the pods and services and deployments that are actually running in a namespace so How do we get started? How do we get started building the equivalent of the test network or a network that's like it just Oops josh Just operator I have just taught kubernetes everything that's necessary to run hyperlegic fabric networks. That's it I installed a api extensions or resource types which are registered into kubernetes And we should see in the top that okay in the background I've done it the equivalent of kube cuddle apply make a deployment that spins up the operator So now the operator is running and listening. It's doing its thing. It's waiting for stuff Africa is now running in our kubernetes cluster. By the way, where is the kube running? Anyone it's cloud native. Where is this kubernetes running? Yes, oh leg I'm picking on you. Where is it running? No, where? Yeah, it's on my laptop right before this I ran just kind and now I have a kubernetes cluster That's running virtually pinned to a specific revision of kubernetes. It's running on my laptop And then I installed the operator. So let's start Uh Someone tell me what to type next I'm not to just Install chain code yet. What would you do? Just start? fabric Please just start fabric. I just want to write some chain code. What's going on? So what I'm doing here in the background Is we have a series of kubernetes resources to represent peers orders and certificate authorities Um Aaron, I know that you have set up the ca's and just why I saw oh my goodness. What a tremendous chunk of work. It's necessary The three rows on the top that are invisible are bringing up the ca's After those are available I have a look. There's a little script in here running just start fabric There's a little nuance that before we can bring up peers and orders in our network We have to wait for the tls certificates to be provisioned by enrollments for the ca's I'm sorry that I have to understand what those words meant. It's too much right fabric is too hard to work with Point is we have to wait for the ca's to start out before we can go and get some tls certificates for the peers and orders So we're waiting ca's are up Now we're starting our orders We're starting our peers In the background. They're just going to kind of come up That's it. We have our network. It'll take a little bit a minute or so Which ones? So let's what's operator doing? Is he doing anything or is she doing anything? Oh the highlight. Yeah, okay. So we have an org. See it. Yeah. Thank you. Thank you, Dave So there's an org ca. There's an order. There's if we have a three three orderer Ordering group We have an org one which has a ca. There's an or there's a peer one and a peer two in one org There's another org in here org two as a ca and a peer one and a peer two And it'll just take a little bit of time. I mean, it's not I guess There are ways we can make this process faster But what I appreciated the process is that it is 100 percent predictable Right, let's give it a second. We're almost there I guarantee you This did not take three weeks of time to try to figure out how to do the bootstrap ca enrollments and the like We've already figured it out, right? It does a very careful nuance of things that have to happen in some order And the nice thing about having go lang code is it's code Right. It's a it's a mechanical process that can be implemented and tested and validated software Not in the wet where that we all have upstairs. Why do we need to keep track of that? Right? Okay, we're up. The network is ready. What happens after we create a network? Just someone told me what to type create channel um We're using the channel administration apis for this There are a couple is the I guess it would is it's called the channelless config option where we're using the osn admin There's that there's a term for this that involves we're not necessarily using the channel the bootstrap system channel To put in channel config structures and build it up inductively There have been some advancements in recent versions of fabric One of the tools here is oh, it's already done. We have our channel There's a restful api that's exposed by the orders and peers to do the equivalent of join a channel Right. So behind the case in this case. We're using the admin apis Hitting the ingress controller and we've just joined all of our peers to a channel which was just created The alarm didn't turn red. It didn't flash Nothing went wrong. It actually kind of worked, right? We have a fabric network running on a laptop running on kubernetes in three minutes or so, right? Again the cycle here and again, there are other options here If your business case or if your use case says I want to try ansible or I want to try the user interface That's fine. But what we just saw here was to skip Directly to the lowest common denominator of working with kubernetes through the kube api controller. Okay So if that wasn't enough who here has worked with the fabric. Um, has anyone tried chain code as a service yet? One out of 53 you guys don't count right you wrote it or well Right, so um on the complication thread earlier, right kubernetes is complicated Fabric is complicated Fabric's notion of chain code inherently is complicated. There's a very careful sequence of I need to install a chain code package and swallow the peers and then I need to Approve a chain code definition for each of the organizations in my channel and after that I have to commit a transaction Which says okay, we're ready to go and deploy this right? It's really complicated one of the problems with chain code as a service is Oh, now I have to get the startup of the process to receive the hand the handshake for the chain code in there And now you're back into kubernetes, right? So the chain code builder I'm actually as excited if not more excited by this innovation than I am by the operator And if you guys have heard me talk about the operator, I get very excited about it But so um, so fabric builder kubernetes or the fabric chain code builder was an innovation by our colleague James Taylor from IBM And james said okay josh, okay I see where you're going with this kubernetes stuff. We've got this external chain code builder It's just not coming together, right? Please for once and for all Can we please do the right thing from the perspective of the fabric chain code life cycle and just use it as it was supposed to be The normal integration for the original vision for Chain code was the peer client Binary would instruct something about the peer service endpoint by uploading some code, right? And then what happens? The peer has some pre installed notion of I support goaling or I support typescript or I support java At some specific revision and then it has to go and compile the chain code has to build a docker image It has to control the life cycle or in some cases it might just do a fork An exact of a chain code process in the scope tightly associated with the peer So james comes up one day and says no, why are we making this complicated? Why can't we just build a docker image externally? Push it up to a chain code image Excuse me a docker container registry or an oci image repository and then just go through the normal chain code life cycle Right. I want my chain code right now. This is like the This is what I wanted from the onset. I just want to write chain code What's stopping that so the parts here are No, I just want to build my chain code I just want to install my chain code and then I just want to work with my chain code Why is chain code any more complicated than that? So the parts here are again The workflow is I've got I want to apologize. This is the official OCI image container registry icon. This means it's a docker registry, right? Maybe you don't want to upload the work in progress for the chain code that you're working with to Docker hub, right? I'm working on it So when we ran our kubernetes cluster One of the other companion services that we have also started is a local insecure docker registry So I can build my chain code process For my chain code image I can upload it to a chain code repository And then I can instruct kubernetes with the kubernetes builder. Would you please start chain code? some very particular image digest and the digest is important because So who here is docker? Everyone's familiar with docker, right? Docker labels What happens if you commit a transaction to the channel with a docker label on it? And then someone applies a new label to your Repository, right? What would happen if You install chain code that's associated with a label And then at the docker container registry started changing the definition of your contract by applying new labels. That's kind of a problem, right? So with the chain code builder, we're using the immutable Image layer hash So exactly the image that you just built exactly the image layer that you uploaded is exactly what will run because that is exactly what you committed To the channel, right? So immutability is fully supported here, right? um Dangerous scenario number two, right? Let's install some chain code Anyone want to shout from the peanut gallery? What do I need to type just? build Has anyone here used makefile before? Right, it's a dependency system. So when I'm typing just Just is an open source equivalent of make That's more amenable to general purpose scripts or other binaries I happen to like it because I like making dumb puns and the use of just is an endless source of them, right? So I have made my chain code. I just built it in the background. I um I had pushed it up to the local container registry So the next step in the in the lifecycle is to We'll just package the chain code You couldn't see it here. There wasn't really any console output, but we should have a An asset transfer archive And if I unpack it Somewhere in here if you cannot read this what we're doing is we're saying When I install this chain code package I want you to use the kubernetes external builder to install this very particular Image layer at some digest tag into the cluster. So let's go and install it just Just yeah, I didn't okay, and then the top what we should see is chain code builder Detects in the scope of a peer That a new chain code contract has been Instantiated and it started the pod and chain code is running. That's it. There's no 10 minute life cycle. There's no waiting. It's just Run chain code right now, right? So Let's go and just to see it kind of work. I'm I'm inquiring for the metadata about a chain code contract With the peer cli binary It's routing into the cluster via ingress and it's accessing these two chain code pods Which I have installed on two peers in one or okay So what happens when my chain code has bug? Anyone just build chain code right I'm going to increment my sequence counter. This says please make the sequence at which I am installing the chain code up by one And then we'll go and just install it again okay We have a new chain code built imagine that in between those steps I had edited some code we could have added a console out debug log or something like that to see it was different but Our new chain code contract is running Any questions or I think we're actually Getting pretty close so Let me just back up We just went through the act of creating a smart contract installing it on a channel Creating a test network running it on a cloud native compute running on a virtual kind cluster running on my laptop I have a very very little tiny like a like a dingy sitting there in the ocean by itself right and One of the big lessons that I learned Mark, thank you for the the panel breakout yesterday. This is not a technical issue right there are technical challenges, but What good does it do to build a blockchain? That sits in isolation from anything else There are obviously challenges around building consortiums around working with other companies Does it do any good to have An operator run a fabric network just on one kubernetes cluster? Or do we need to start thinking about how can we make it easier for different organizations to work together? Even though they may be deployed on different cloud stacks so Quick question. What do all of these have in common? There's two parts. They all start with or end with kubernetes service right? The other thing that they all have in common Is they all equivalently run exactly what I just ran So the only difference between me pointing out a digital ocean cloud or a rancher stack or google cloud or xyz kubernetes service Is the address of where I am pointing to the kubernetes api controller The exact same scenario that I just ran here will deploy a full stack hyperlegia fabric on any one of these clouds equivalently Right, so at some point It's also worth noticing. What's another thing that these companies all have in common or indifferent, maybe Right. They are fierce competitors in the cloud space How do you get all of these companies to agree on anything? I told everyone i'm not going to say i'm a kubernetes salesman But I am right. What did the cloud native compute foundation do correctly to get the big heavy hitters in cloud deployments to agree on anything? How is it even possible technically that I could run a piece of software like this with containers equivalently on all these stacks? I think they did two two things right One was they're just Better right the the kubernetes pattern of controller based and operator patterns really is something special It's new it has some complexity But there's something very special that said we can all agree that this is the best outcome We should stop go writing the equivalent of xyz cloud. It's a a proprietary version They also had to agree on some format interoperability protocols I think Yeah, maybe the trough of disillusionment. It sounds like one of those giant It's not necessarily a lull in in fabrics heartbeat, but think of it Remember that first picture of what did the world look like 500 years ago the trough of disillusionment looks like You know some giant leviathan is going to reach out of the sea and break everything right so so maybe the way forward is to Cast the frame of moving forward with fabric as an opportunity to work at interoperability And alignment in a cloud native fashion that allows us to deploy on any stack anywhere And let the platform wars go right. They're not adding value to the overall mission that we're here for So come this is the last point This is the most important point absolutely the most important point work together Right, if we keep bickering and working and challenging each other building all these platforms, they're not going to work together. So This is a community contribution. It's open source. It's free. It is a true community contribution. It's been I would say I want to say thanks to the many hundreds of thousands of hours that the team had put it together But it is a real asset. Please use it abuse it tell us what works if it doesn't work for you then carry it forward All right, thank you very much. If you guys have some questions ask them now If not, let's figure out a way to string the parts together So we're working together on this voyage around the world. It really is a 500-year mission. Okay. Thank you very much Yeah Yeah, I'm happy to see so um, so it's a great point. So what I would like to do So again, this is the satellite orbit pass in 30 minutes. That's three times the orbit rate So we have um, what next how can I learn more? There's a couple of resources one is Pleased to learn more about it. We have it's an actual hyper ledger lab There are reference examples of the crd resource types for peers and orders and the like They have their own unique dialect if you will but they're kind of boring There's nothing really eventful enough. They say my org name is this my My storage class is this There's a couple nuances in there around how do peers and orders find their certificate authorities that the loose end I'd like to cut it out Please follow the trail for this if you'd like you can also trap me afterwards I can walk you through some of the scenarios through it If you are available tomorrow, we have an interactive workshop Um, you got to see it today, but you can experience it hands-on tomorrow And I think that would be actually a really good opportunity to dig into some of the details right Yes Yeah, yeah Yeah, it is. Yeah, so so here's what's interest. Here's one of the findings that I had of this is it It's too complicated. You have to be able to divide. What is kubernetes configuration versus what is fabric configuration? I think this is the right line There's a cube part set up the operator set up peers and orders And that's distinct from organizing a channel or a consortium on top of it right and once you have the viewpoint that Fabric service endpoints are network URLs It's absolutely possible to string them together where you could be running some on one cloud and someone the other and because they're traversing via ingress It's just how do you specify the location of where to find the remote peer? It might be in the same cluster It might be transiting on a private network within the cube across pods It might be traversing the internet between dubai and los angeles, right So, yes, it is possible. Um, yeah, good one more time Yeah, yeah Yeah, it is so this was a usage scenario that came on from Use cases that were really hammered out with ibp's ibm blockchain platform and I was absolutely a scenario of multi Yeah, yes Yeah And perhaps I didn't say this enough we it is an open source operator, but I'd like to reiterate it has served through the Trials of fire and the tests of time it's seen maybe not all of the different scenarios, but it is absolutely production grade All right, and it's free. It's open. It's a community contribution Any other questions if not, thank you very much for your time. You can find me outside the door I will talk the ears off of an elephant if given half the opportunity. So please take the opportunity I'm happy to talk about it. All right. Thank you