 Okay, well, it's 11.15 in a bit. So we'll get started, shall we? So this session is called developing enterprise blockchain applications with hyperledger fabric My name is Andrew Coleman. So I work for IBM and I've been working on hyperledger projects for the last five years or so notably hyperledger composer in the early days and and now hyperledger fabric Which I'm currently a maintainer So this session is very much developer focused. It is literally teaching you how to write client applications For the latest version of fabric, which is version 2.4 Just out of interest, are you familiar with fabric? Are any of you currently fabric developers? Yeah, and are you familiar with the new programming model that came in in version 1.4? Yeah, good. That's good. That's fine So this is going to make no assumptions that you're familiar with that But if you are then what I'm going to do is highlight the differences The simplifications and improvements that we've made in the latest version in 2.4 And really it's all around, you know, the client applications that I'm not really going to talk much about developing smart contracts But you know as a client application developer, you're going to be interested in submitting transactions in boaking the smart contracts that are on the ledger And then we'll you know, we'll talk about the basics and we'll go on to some some more advanced stuff And I'm going to pepper this presentation with with snippets of code In the various languages that we support and I don't favor anyone over and over the other. They're all equally capable So to start off you probably all know this. What is hyper ledger fabric? It's it's one of the older hyper ledger blockchain projects It's been around since 2016 was the first release and it's had a lot of contributors from a lot of organizations It was very much designed as a modular framework for building blockchain applications It it implements this execute or the validate model Which is unique to fabric and that's very much reflected in how you submit transactions It has pluggable consensus protocols So, you know the ordering node can has evolved over the years it currently the default is raft based orderer Which is crash fault tolerant, but we are actually actively working now on a Byzantine fault tolerant ordering service And we plan to deliver that in version three Now notably smart contracts can be authored in a general purpose programming language and we support three out at Box, but you know it's extendable. You can implement your own if you if you feel the desire Rather than to have You know a language like Solidity or something prescribed for you that's written, you know intentionally for smart. So that that widens its appeal to general purpose programmers It's a permissioned blockchain so the participants are known to each other and that You know that has a bearing on the the consensus protocols that we use We don't have to have proof of work and things like that And it was designed from the outset support a broad range of industry use cases Finance Insurance, you know, we've been hearing in the keynote this morning a variety of industries that are successfully put in fabric into production So what I'm going to do now is sort of start focusing on them on the topic of this talk And I just want to give you a picture of One component of fabric which is the peer now the peer and You know a blockchain network will run multiple peers across multiple organizations But inside the peer is the ledger itself and the ledger is made of two components There's the there's the blockchain which is the you know the the ordered set of transactions Where the transaction, you know marks where say, you know, if you look at your bank account You've got lion saying well something was paid in something was paid out They're the transactions and then you've got the running total the current what we call the world state Which is if if you run all of those through this database you end up with this world state like your current bank balance And so all valid transactions will affect that world states when it's finally committed to the ledger Now Hosted by the peer although in a separate process is a smart contract. So all transactions are Encoded into the smart contract. So When a client application Invokes a transaction submits or evaluates a transaction. It runs The smart contract in this chain code container. We call it chain code in fabric And that will interact with the ledger and it can also emit events so you can have contract events and also when blocks are committed You get these block events Now At the top there are the other blockchain developers us Okay, and I I split it into two because we've got developers who focus on smart contracts And we've got developers that focus on the client application and I Don't believe that they are necessarily the same person. So You know that the developer that develops a smart contract will have deep knowledge of how that works The developer who develops the client application is probably also developing You know interacting with other systems database system messaging middleware integration as we heard earlier that these blockchain developers and not just focusing on blockchain as a whole raft of other things They have to worry about in their day job. And so probably the skill sets of These two developers are different So certainly the the client application developer will need to know what that does but not necessarily how it does it And that's very much driven our design of the programming model So for the purposes of this presentation We are focusing on the developer at the client application So many of you raised your hands when you said you were you already developing client applications using the the new programming model as we called it back in 1.4 when we first developed it So we set out to create a set of of high-level APIs That expose the more sort of business type abstractions that the sort of things that the typical client application developer would be doing and As I say we first delivered that into into fabric 1.4 And we built it as a as a higher layer of Abstraction on all of the existing SDK software development kits So up until that point that the SDKs were very low level in order to submit this Transaction and and don't worry about this because we built the high-level applications so you Program model so you don't have to worry about this But you know back in the old days you have to use the right hundreds of lines of code just to submit a transaction on to the ledger and so we we felt the need to Raise the layer of abstraction on that and we also We're very careful with the naming of the Classes and the methods in the new programming model We wanted to speak the language of the target audience of people that were familiar with networks and contracts and transactions rather than you know fabric terms like you know Chain code and channels and things like that. Although we've kind of mixed and matched a little bit and we came up with it with the idea of The of the gates way which was very much a sort of a virtual object used in this Programming model and and we built that in all of the three languages that we supported node Java script type script Java and go So towards the end of last year we developed we we released fabric version 2.4 And one of the major new components in 2.4 was with the gateway. So the gateway is An implementation of this high-level programming model, but it's now in the peer itself rather than as a layer in the in the SDK and So we we now expose that as a as a service from the peer and so we went back and we rewrote the three SDKs the client API is into these lighter weights ones which are easier to use much lighter weight dependencies and selfishly they're much easier to maintain as well and So when this finally gets into an LTS release version 2.5 We do actually plan to start deprecating the older SDKs And You know we've built up a fair bit of experience now of usage of the programming model and In certain aspects we've made it simpler to use And at the same time we've also enhanced some of its capabilities So made it slightly more flexible based on on patterns of usage that we were seeing and and we're being asked for things like You know asynchronous submission and support for offline offline signing and things like that And One of the benefits now of the gateway being in the peer itself is that the client application Only needs to make one single gRPC connection into that peer the gateway peer the chosen peer appear that you trust okay to Run this life cycle transaction life cycle on your behalf and then the peer itself will Make connections to all of the other peers in the network And also all of the ordering service on your behalf. So you only have to now make One connection and a consequence of that is that you no longer need these these Network config files or CCP files depending on the SDK that you used it was called different things But it was a it was basically this yaml file format that defined all of the endpoints that you might want to connect to And it also had other things like gRPC options timeouts and stuff like that But what we have actually done now is we say actually the client application can make that one single gRPC connection supply that and We no longer need that ccc profile Commit strategies So that was something where you know What defines when a transaction is committed to the ledger? They put out these block events, but how many block events that we listen to well actually with by embedding the gateway in the peer We've now Automated all of that and we actually have an ideal strategy that I'll explain to you later. So you don't have to worry about that either We've done away with wallets for storing identities These are all things that weren't really central to interacting with fabric And in fact, you know generally Developers who are developing these these applications will have their own mechanisms for storing their Private keys and things like that okay, so This isn't about smart contracts, but it's useful to know what a smart contract does and and what it looks like An important thing to note is that all transactions have to go through a smart contract all interactions with the ledger have to go through a smart contract and This is really sort of the the framework of what it looks like. You'll have a class. This is this is JavaScript You've got a class extends contract and then you've got methods now each of those methods is a transaction function So we've got three we've got four transactions issue transfer redeem description okay, and so When you submit a transaction from your client application, and you have to give it a name that will be the name and And Although we're not worrying about what code is in there Fundamentally the code in here will be implementing the smart contract logic And it will be interacting with the ledger and it interacts with the ledger via these put states and get states Okay reads and write operations on the on the you know the world state And what that will do is will form a read write set. So it's like a Simulation We'll get on to what what endorsement is a little bit later So if you're already using the new programming model, you'll be familiar with this basic hierarchy of objects at the top level We've got a gateway which represents you know in the old days a virtual connection into the network now It's that gateway actually is in the pier and so You can then get a connection to each of the networks, and that's a that's a channel in fabric terms And once you're targeting a channel you can get an object representing a Smart contract instance on that channel and there sort of that at the highest level of abstraction you can evaluate a transaction Which will invoke a transaction a transaction function to do a read operation on the ledger or submit Transaction if you want to modify the ledger and under the covers. They're doing very different things And you can also hook into events So your client application can listen to events with their chain code events or block events as they're being omitted So this is now the general structure of a client application Okay, we've got some work there to specify our identity because you know it's a permissioned network. That's the identity that Your transactions are done under You'll need some sort of signer implementation a function that signs the proposals and the the transactions So that's a function that just takes a digest and returns a signature And then you'll make your gRPC connection into your favorite peer And given that you'll then create a gateway instance using these three things Access your network your contract and then you can start submitting transactions So we've got two transaction submissions one after another and then When your application ends there's some there's some cleanup code. That's that's the general structure of a very simplified application and That's taken from real code barring some imports. This is Java You know barring some imports at the top that will work By the way, do you feel free to ask questions as we go along as long as they're relevant to this Yeah, so these slides are all uploaded onto the onto the website. Yeah Yeah, so we'll come on to that question was there's a submission of Transactions wait for the commit. Yes, it does, but if you don't want to then we can support that as well So we'll come on to that. So, yes Yeah, no, so what what some so the question was the peer that you connected Does it have to be a member of the channel that you want to transact on? And the answer is not necessarily because the gateway service itself will When it comes to we'll come on to this in more detail. So let's start with a quick answer Which is no and then I'll explain why later Yeah All right, so we support three languages So we have three client API's SDKs then yeah Because these these these are now the lighter weight ones that replace the older ones and They all do support the same thing, but they Look different. They do it in different ways and the reason is we've really tried to you know adopt the The idioms the patterns that are meaningful to that particular language. So so for example in Java We we've adopted this sort of fluent style Builder patterns. So, you know, you can chain your functions together Obviously Java supports exception handling. So, you know, you surround it with a big try catch block And that you know that makes makes for a Java style With with node Type script or JavaScript Then you typically pass, you know JavaScript objects JSON like objects to pass in your your options and with go We've adopted this this idea of optional function functional arguments, which is which is very popular in in in go API's But they all do the same thing So let's step through each each Part of the process. So we've got a connect to the gateway and a gateway object represents The connection to the the fabric network with a given identity So, you know, if you want to interact with the network using two or more identities, then you can have two or more gateways All we ask is that you share the same GRPC connection. I'll tell you why in a minute And so to create a gateway you need a GRPC connection You need your client identity and you need a signing implementation So, you know, I as I say, I'm going to give snippets of code in In random languages as we go along just to not show any favoritism Obviously when you're dealing with go You'd want to check the error As it comes back and do error handling You know, as you go along in the code in node or Java, you would you would use exception handling mechanisms So that's creating your gateway object And so before you do that, you need a GRPC connection Now we've got out of the business of making those GRPC connections on your behalf You only need one now rather than one to each peer you're Connecting to and actually each language has a perfectly capable GRPC Library itself And so rather than you know us sort of wrapper that and then you know every time a new option or something different is required You know you come to us and say okay, can you support this and then we you know, we used to have to Add that support in there and make sure that we then extended the CCP file Format to support that and things like that and we just you know, it's not necessary So you would use a GRPC language library of your Chosen language and then you would pass that into the gateway when you've got it and Very similar with identity and signing You know rather than you know with the old API as we said well give us your private key and trust us We you know, we won't do anything nasty with it. We'll just use it for signing What you now actually do is you provide a function that does signing And we will invoke it at times when the transaction needs to be signed Now if you have a private key, then we provide a helper function to create a signing function for you but you know Be reassured that your private key never goes across to the gateway. It always stays in your client application. It always did But you know this this API emphasizes the fact that you know, no keys get passed across the network into the server So we always call back again into your client application when we need anything signed right, so We'll come on to submitting transactions so Any transaction that modifies the ledger has to go through a process of endorsement and Depending on the endorsement policy, which is associated with the smart contract that you're invoking this Transaction might need to be invoked or simulated on multiple peers across multiple organizations And you know as we saw that smart contract that was Installed on on a peer we saw that it gets invoked and I said that it generates a read-write set It doesn't modify the ledger at that point what it does is it runs against that peers copy of a ledger And it and the read-write set represents what it would do to its ledger Okay, so it that's that's an endorsement the peer then signs it and sends it back to the requester so to Submit a transaction you've got to go off to various peers and gather all the endorsements and they must all agree Okay, both the read and the write sets have to agree across all of them And when they do you gather those together it gets signed again on the client side And then it gets sent off to the ordering service Okay, and then that that orders the transactions using its consensus mechanism and it broadcasts it out to the to the network and Commits it And that's quite a complex bunch of stuff and that all gets done in one function called submit transaction So just to step through that In a diagram. Yeah We've got our client calls submit transaction. Okay one g rpc connection into the whole network Okay, we decided that this is our gateway peer So what that then does on your behalf is it gathers the endorsements From itself and from any other organization that's required in order to satisfy the endorsement policy And then once it's gathered all that it will send it to the ordering service And the ordering service will batch up transactions order them and broadcast them to the to every peer Okay for committing to the ledger And then only at that point that they emit a block event and The gateway will return control from that submit transaction It's a query in the ledger doing a read from the ledger So just to emphasise the all interactions with the ledger are done through the smart contract So if you don't have any transaction functions that Read from the ledger and return values then you can't query the ledger. It has to be done through the smart contract It still has to be invoked but if you're not modifying the ledger it doesn't have to go through this endorsement an ordering process you can simply just invoke one peer and Typically it will be the gateway peer or appear within the gateways organization in order to Run that transaction function and return to the user or whatever was returned from that smart contract function And we use the evaluate transaction function for this and so the gateway peer will always try and choose A peer of the highest block height in order to assure you you're getting the most up-to-date value And it's actually using this mechanism that we've we've been able to do away with the need for These commit strategies Because when you go to the gateway That the commit strategies will always about well I've submitted a transaction when I follow it wouldn't evaluate transaction query How do I know I've got the most up-to-date value and you know You used to wait on these various commit events before you were sure that you know you got that but now The gateways managing that all before you so it knows after it submits the transaction It should look when it's doing a query for the highest block before it returns a result to you So slightly more advanced when we're dealing with with private data So private data collections are used to store sensitive data and it's stored off-chain Only the hash of that data gets written into the into the shared ledger Now the usual pattern for an application is that it's when it's dealing with sensitive data It sends it using the transient data field in the proposal and Again, that doesn't end up on the ledger and so the gateway is going to be aware of Data that comes in the transient and it's going to be more cautious in how it requests endorsements from other organizations because you know It doesn't want to get into a state where it's it's it's sharing your sensitive data with other organizations unless it's pretty sure That it's okay and I'll explain on the next slide what the endorsement logic is but You know we we're transient data it does take a more guarded approach and because of that You could result in the gateway not collecting enough endorsements to satisfy the various policies And so that's that's the one situation that I can think of anywhere Whether the application developer would actually want to specify who the endorsing org should be and that's probably okay with with Private data because you know typically you're doing things like well, you know, I want to Sell my asset to that organization there and you know who the buyer and the seller are and It's a good chance of they will be the two walls that need to endorse the transaction so You'll certainly need to be aware that Who you're using as your as your gateway peer and I said at the beginning that's your trusted peer And if you're sending sensitive data across then, you know, you really are trusting that peer To to handle that or to even see that private data and If you're an organization that doesn't run a peer then you'll need to send it to a peer that you you do trust in another organization so a little word on on endorsement because we We put a lot of work into this so each each Yes Can Okay, so so there was a there was a timing issue where the private data wasn't available Yeah, so I mean it's this does have a retry mechanism which which I'll talk about towards the end but some Yeah, I would I would need to look at that. I don't know if you've gone though Further Okay, thank you So each smart contract is deployed an endorsement policy So for example You know that that top one is a majority you've got three orgs. I need two of them. That's the full policy The second one is a bit more specific. It says well, actually I want all one to sign it And then either all three or all four So all one might be a you know an auditor or you know regulator someone who absolutely has to sign every every transaction But it gets more complicated when we're dealing with with private data collections and multiple chain code so for example you know a Transaction proposal might have to satisfy Multiple endorsement policies at the same time So if we're doing chain code to chain code if one chain code is deployed using that one and the other one is with the second one Actually, both of those at the same time have to be satisfied And the only way to satisfy both of those at the same time is to have all gone and All three sign it So we're now narrowing down We're forming a sort of a derived endorsement policy that takes him into account both of those And it gets worse with with private data collections because they have their own signature policies And so we need to bring those in as well And if you're doing a read from a private data collection, then you've got the ownership policy as well Which might you know limit you to hook, you know, whoever has copies of the private data collection And then it really gets worse with with with state-based endorsement policies because you know each individual state within the metadata and the ledger can have an endorsement policy that's just just for that state itself and They can change so a transaction can change those so they're dynamic. So Whenever you're submitting a transaction all of those have to be satisfied and So how does the gateway do that so? Prior to 2.4 the client application had to understand all of the things that were being touched by The smart contract, okay, it really needs to understand How the smart contract work rather than just what it did it had to understand how it did it in order to be able to Invoke, you know, the right organizations to to endorse The gateway takes that all away from you and the way it does it now is so well first of all it simulates it runs Induces on its local peer either the gateway peer or one of its local log peers and We've we've we've augmented the ledger with it gathers info on which chain codes which private data collections which state-based endorsement policies get touched as a result of that Transaction invocation on the gateway peer And when it's gathered all that information it then works internally with the discovery service to say well, okay What what you know, what's the overall endorsement policy? Who else do I need to get to endorse that transaction? And so it will then Invoke those those other peers and gather the endorsements and it's got a pretty good chance that it's going to succeed So question So is there any performance overhead? No, I'd say the other way round It's you know because because we now know exactly who to target We don't have to go a lot of applications the way they used to handle this was just get everyone to endorse it because the problem Was too difficult to solve if you get everyone to endorse it Build up this huge transaction and pass it through and let the but like let the final validation phase decide whether that worked or not But now you know up front we've got a pretty good idea of who needs to endorse that transaction proposal So it's so it's two phases first it simulates it on on the local peer in order to then Understand what the endorsement policy will be and then It's got a you know a set of endorsers that it needs obviously it's already done it itself So whatever's left it will then in parallel go out and gather all the others So it's a two-step. So there's no No, it just doesn't yeah, it's just one simulation on each Of the peers that needs to simulate. Yeah, okay. So so we've talked about submit transaction and You know you just give it the the transaction name and the arguments, but if we want to now start doing some, you know You know give it some more options So for example the you know passing transit data Targeting specific organizations if you you know for endorsement if you if you need to do that Then there's another form which is submit and evaluate, okay? So they're slightly lower level than the submit and then There's go and node examples, okay using Functional arguments or or a JavaScript object But that just allows you to you know slightly finer control or We can get even More control if we want so well, okay the submit transaction is you know, it's going out It's getting multiple endorsements It's gathering the responses You know it's submitting to the order. It's a wait and a commit notification We can break that down now into individual things. So so so in Java We're using the you know the the star where you just change them together so we can create a new proposal We can put transient data in there and and then we build an endorse and before we submit we can you know We can pick out intermediate results. We can get the transaction ID or the result And so you know and we can implement you know different flow logic We might want to you know do some different retry options or something like that in between each step So if you really want you know The high-level one is great But if you want finer control you want to do your own logic in there then then we can break out these steps And you can have more control over them So if a peer doesn't have the chain code Installed then the gateway will not try and get it to endorse a transaction so that so the gateway peer You know the service discovery and the gateway are working very closely together Yeah, yeah, so if there's a transaction proposal on on on a channel So come back to a question earlier about channels. What if it's not in the channel? Well, that's fine because it will work with a discovery service to to find a peer that is a member of that channel Cross organizations. Yeah, and the site the same with Chank it unless you're passing transient data in which case It won't just go off to another organization willy-nilly But yeah apart from that You know, you've got you've got a channel And a chain code ID it will find the peers and the organizations that can run that and it will it will ask them Yeah, all right, so Some some other logic and this comes back to your question earlier about well Does does submit transaction wait for the commit as it does? But if you don't want to wait for example if you if you want something a bit more responsive if you're writing a Arrest application where you know you want you want to get a result back fairly quickly or you're updating the UI or something like that Then you don't want to sit there waiting for the you know The order to batch your transactions and to validate and commit You'll still need to eventually before you know that transaction has been committed But in the meantime you can you can do some other logic And so you can do a an async submit submit async And or in and then it will return control to you as soon as it's successfully sent to the ordering service And it will return an object along with that that you can end that you know invoke a wait So you you can block to wait for the results eventually. Yes Yeah, so so the so the commit will actually wait for the commit on the gateway itself So it's only going to wait for that one one commit Yeah, that's right because it yeah, that's one and it only needs to because if it's followed up by a query and evaluate transaction That gateway knows it can find It knows it can find a peer that has That one because it's not gonna wait. It's not gonna wait for all of them No, it's it's only gonna wait for what it really needs to bear any mind You know, it only needs to wait for that because if it's followed up by a query, it knows it can find the latest value so So you can handle block chain events yourself So you can you can wait on block events. So you might have some application that you know wants to Receive the block events and do something like, you know store data on a you know an off-chain database or something like that and you can wait on these chain code events or contract events typically, you know for invoking business logic or you know if What it whatever the chain code is sending out in its events you can consume that We support so in the in the in the older SDKs we we had the idea of Event playback and event checkpoint in so we've implemented that all as well So we've taken nothing away. We've we've only added error handling so So that errors are you know potentially coming back across your GRPC channel and we've very much adopted the the GRPC best practices for error handling and we you know, we support what it calls the the richer error model so there's a link you want to read about that and You know, there's an error type says hierarchy of errors So, you know, we can indicate whereabouts in the transaction flow that error occurred whether it was in endorsements Or you know submit into the order or something like that A GRPC error has a status code and You know Google define a you know a well-defined set of status codes and each one has a different meaning And as far as you know should you try and retry is it a transient error or is it more of a an application coding error and things like that So we we've tried to you know keep all of our errors within The framework that it sets out there. So for example, you know, if the if the read write set mismatches Then that should be with you know retry because it's a transient issue But you know if you've invoked transaction with the wrong number of arguments then that you know You could have all to your application at that point retrying that's never it's never going to work And also these GRPC errors have Extra details that can be added to them and what we've done is that You know if any of the when the gateway fans out to other peers and ordering nodes if they if they return errors GRPC errors then what we do is we bundle them together under the this detail section And if you look at the merry the error message you can and dig down into the details You know, you can see which which peer you know actually through the error and why So we've got a lot more detail on the error messages there and there's just some snippets of code that show how you would dig down into the details So we've made a much better job of you know higher availability with with this new gateway model. So for a start High availability so you you know, you're now in charge of managing your GRPC connection and typically in a cloud deployment You'll go through a You know an ingress controller a load balancer Okay, and that you know, you've got you've got a single connection endpoint for your client and that will fan out So it's a multiple gateways by the way every peer is a gateway peer unless you turn it off so by default every peer can can work as a gateway and We can do that fairly easily because the the gateway is stateless, you know, there's no there's no peer affinity That's needed as you're going through the the process And there's you know, there's a fair bit of retry logic that we've we've put into the gateway as well alright, so we've We've also done a lot of work on documentation. We've we've rewritten the You know the developing applications section And so and you know, there's a large section on how the gateway works internally if you're interested in the internals And we've built a a lot of new samples So we've got these asset transfer samples and there's quite a few of them We go from the really easy through to the really quite advanced So we demonstrate all you know how private data and state-based endorsement So they're all in there in the fabric samples We've got you know, the Java doc JS doc go doc API reference. Okay, so that's all index from that URL there Getting help of course where there's the fabric mailing list Discord whenever conversation and of course stack overflow And if you find bugs then by all means raise an issue on the on the fabric gateway repo But that's that's not for questions. That's for that's for bug reports. So just to summarize, you know You know this work on the on the program model started You know way back in in version 1.4 and this is the natural progression where we've you know we put all of that logic into the peer itself and Really I encourage you if you're if you're around Wednesday morning to try this out because we've built a a workshop that will take you from nothing through to deploying a fabric network on on Kubernetes and You know building some client applications and and running them That will run Wednesday morning in the hotel next door. I believe so Thank you. Any further questions? All right, thank you. Yeah So so so the question is what about calling the external Service off-chain. So are we talking about from a client application or from a smart contract? Okay, wait, that that's a conversation over a beer tonight But this was more about client applications, but generally speaking, I would say that's a bad idea because You know that that external thing could change and it won't have been endorsed by the multiple endorses that have to sign off a smart contract But we can we can talk about that All right, thank you