 Hi, welcome everyone. My name is Netaji Thumuru and today I have my colleague Michael Kunsman and Mario Ponce. And for the next half hour, what we're going to be doing is sort of taking you, giving you an overview of the solution and also taking you to the journey of how we went about building this application. Right from building everything on AWS and vermin containers by ourselves to where we are currently with sort of more hybrid approach of having most of it running on Cloud Foundry and bits of this of running on AWS and the end state of where we want to be with everything sort of running on Cloud Foundry. So what we will not be covering in the session today is sort of explaining the intricacies of blockchain and how it sort of works. But you know, most of you probably have heard about blockchain and you heard about blockchain being this, you know, global distributed database and sort of you heard about it being the network of computers that you can transact. But you know, for the most people sort of think about blockchain as this global immutable distributed database that sort of keeps a record of growing transaction and that sort of opens up a whole new paradigm of application that could be built. And one such application that we kind of built on this is called TrueRec and I'll sort of explain more on what it is. And the World Economic Forum predicted that 10% of the global GDP by year 2027 would be stored on blockchain. So we all use internet. We sort of that's kind of what we, you know, pretty much are on the most part. So think about if internet is information is what is being stored on internet. Blockchain is internet of value. That's where all the commerce is going to happen. That's where a lot of transactions is going to be performed. So what is TrueRec? So TrueRec is this a secure digital wallet for identity economic and employment credentials, all being built on blockchain. So why does it matter? So I'm sure, you know, most of the people that we are here have gone through the process of applying for a job or even hired someone. You know, how do you make sure that the details that were furnished and the self-declared profile such as resume or social profile are valid? You know, it sort of takes an enormous amount of time and effort to verify someone's credentials, but also sort of prove anyone's credentials. And when we mean by credentials, we're sort of talking about someone's identity to, you know, academic, so where he kind of studied to where he kind of worked. And it's a tedious effort. And, you know, once a person is hired, that's when the actual bureaucracy starts because companies want to be compliant, at least in U.S. So they kind of work with third-party agencies to do this whole verification, and that takes forever. And in some cases, putting a lot of pressure on the people that were involved in the process. So why does it take so long to verify anyone's credentials? So this is kind of a very high-level view about current process on how the hiring works. I just want to focus on the three main actors, which is the job applicant, the company, and the third-party agency. So once a person is hired, companies tend to use, you know, third-party agencies to actually do the verification. And it's not that they have all the details, so they need to go and work with, you know, whole sorts of institutions like universities, the garments, and, you know, former employers. And oftentimes, this is pretty manual and tedious process, and that kind of adds up to all the delays that's being caused. So how about, like, wouldn't it be cool if we empower individuals to carry all his credentials in one wallet where he'll be able to independently prove the validity of those without having to, you know, reach out to all these institutions, like proving my drivers, carrying my digital driver's license to carrying all my employment records and education records or micro-certificates in one central place that I can prove, you know, without having to, my employees calling my past institutions. Wouldn't it be cool if, you know, the recruiter sort of, you know, gets this resume and then can verify instantaneously if all the details that are furnished in the resume are actually valid or not? Can the records in the resume be, you know, verified instantaneously? Can they be notarized? And can they be verified with a bulletproof confidence that they were not actually, you know, modified? And that's kind of what all TRUREC is supposed to provide, and that's kind of what we kind of aiming this application would do. And this is all built with blockchain machine learning and built on top of cloud foundry. So let's quickly go through a solution and how it sort of looks. So essentially TRUREC is this decentralized credential issuing and verification network that guarantees authenticity and significantly simplifies the process of issuing and verifying credentials. So let me walk through the process on how it works. So every institution that's part of the network would be able to issue credentials and it could be anything from the diplomas to, you know, the digital identities to your employment records or it could be, like, even micro-certificates and pay-to-pay records. So all of this could be issued on the network. And all of this would be issued directly to the individual. So this is not stored in a central database where that's how the traditional applicants are built. So everything will be owned by the candidate. And the beauty of this is only the fingerprints or the hash values of those certificates would be stored on blockchain. And why? Because you can, we just spoke because blockchain is this global database and we don't want to expose that to everyone in the world to see all my information. But at the same time, we want to make sure that we can go back and prove those records existed at a point in time and validate if those are not actually tampered. And that's why we only store a fingerprint of those on the blockchain. So similarly, you know, any credentials could be issued and all of this would be stored in the individual's digital wallet and all the fingerprints would be stored on the blockchain. At any point when the candidate want to prove his credentials to any institutions, he can select the details that were relevant in that context and then pass it on to a potential verifier. So the verifier gets a link that the candidate has shared. And once he opens the link, the re-competition of hash has happened and then they get compared against blockchain. So that way we can make sure that those credentials that were issued were not tampered. We know who it was issued from and whom it was issued to and we also know that at what point it was issued. And because there's no one, because blockchain is immutable, no one or no central institution controls it, there's no way that someone could modify it. So that kind of provides, so in essence we're using blockchain as an anchor proof to make sure nothing is actually modified here. So quickly walking through how the solution could potentially look. So on the left side, you imagine we have this hypothetical situation where we have a university that's issuing credentials. So a student could log into the student portal, request for his credentials to be issued. The document would be sent to the candidate. The fingerprints would be registered to the blockchain. A student would open his mobile wallet and then he goes through this whole identity validation and we apply computer vision and machine learning techniques to make sure the person is who he's claiming to be. And once he kind of establishes his identity, he gets into the app, he imports the file that was issued by the institution and that's the central place. So he wants all the records in one central place and that sort of becomes his repository of all his credentials from his identity to employment records to education. And at any point he wants to prove those details, he can then select the details that he wants to send to a potential verifier and then pass it on and the verifier would get a link. That's what you see on the right side there and once the verifier opens it up, then the re-computation of hashes happen and then the validity to make sure those credentials are valid or not would happen behind the scenes. So quickly wrapping up just kind of how the data flow happens. So I mentioned we have three main actors in this process which is one is the issuing institution, the student itself and the potential verifier. When the records are being issued, the actual data which is the off-chain data which is the actual credentials, the clear text document is sent to the candidate itself in a special file called true file and the fingerprints of those documents which is the on-chain data which actually gets stored on blockchain is written to the blockchain using Ethereum gateway which is the component that we sort of the wrapper that we built on top of Ethereum. And all this data would be stored on the mobile wallet and when the candidate actually sends it, he generates a link which kind of creates temporary storage for the data that's stored on the mobile phone and the link gets sent to a verifier and when the verifier opens a link, the re-computation of hashes happen and then they go back and check against the blockchain to make sure none of things have been changed. So this is kind of the high-level flow on how it works. I'll now pass it on to my colleagues. They'll kind of dive a little bit more deeper into the technical architecture and sort of talking to the journey on where we started and sort of where we are right now and kind of explain more details how the application works. Hi. Hello. How many of you are developers here? Awesome. So my name is Mario and my colleague Mitch here. We're going to deep dive a little bit on the architecture and the components that we actually used to build this platform. So I will start describing the journey. So when we actually started to develop this application, we were exploring how the blockchain works and how we will actually offer the service and how we'll integrate the pieces. So we are starting with a lot of Docker containers and putting some pieces in each container. And at some point when we were developing, we realized that we were investing a lot of time actually containerizing each application and the most of those components were actually like Node.js applications or Python, Ruby on Rails. And we needed some basic support for databases like MongoDB to store metadata. And we said, okay, maybe we need just a platform where we just pushed applications instead of investing time and creating those containers. And we will use containers only for the parts that are strictly needed. So from the slide here, you have on the left side that through reclined, there is a small Node.js piece of software that the issuer has the private key. Then let's say that this is a university. So the university will have the private key. They will encrypt the certificates and then they will submit that through our platform. And then we will pass it on to the Ethereum blockchain. So I want to mention that we are only like a proxy application. We are not storing the documents at any point. So we are just passing it on to the blockchain. And then in return, we send the candidate, in this case, the beneficiary of this certificate, a small file that he will share with any potential verifier. Okay. So then, as I said, from the containers, we decided that we needed platforms and that's where Cloud Fundry was a natural choice for us. And we started pushing applications inside Cloud Fundry. So here, again, we have the through reclined component on the left. And the first trip for the transaction is that we are receiving it and sending it to an issuer API that runs on Node.js. That's an application in Cloud Fundry. And this actually gave us the opportunity to scale up because we introduce components like RabbitMQ there. So we have worker queues and able to distribute the workload. And we left the Ethereum Go client on the AWS because that was the part that we still needed to maintain it for that part. So our next step is actually we're going to move that to Cloud Fundry. But for now, that's still running on AWS. And then the second part where the candidate wants to share that with the verifier, then we also have an API there and we match with some metadata that we previously store on MongoDB. And that is sent to the verifier. So what this gave us is, as developer, the flexibility of just taking our apps and pushing to the platform. So that gives us more time to focus on development and not working too much on containers. And my colleague Mish here will deep dive on other pieces of the architecture. Thank you, Mario. All right. So I will dive a little bit more into detail and also like why we made certain design decisions. So traditionally, if you develop a blockchain application, especially like a distributed application on top of Ethereum, like the typical approach is that you would typically run your own blockchain node on your local device basically and hook a web application up to the RPC API that that blockchain nodes, the Ethereum node would expose. However, like you run into the same old on premise problems again, because you need to operate that actually. So that means like the issue would need to operate this blockchain node. And especially in our project, we experienced that there's like a lot more like complexity of running a blockchain node than just running like or like running a let's say a web server. One big problem was storage because basically whenever you join a blockchain network, usually you need to download the whole blockchain. And I don't know how it is currently with Bitcoin, for instance, like there's I think 70 or 80 gigabytes of the blockchain. So basically if you want to participate in the blockchain network, you need to first plan for two days to like sync up with the whole network to download a whole distributed history of transactions. And so we are so we're currently doing that project with one of our online learning platforms open SAP to deliver the first solution in a small circle of users. And basically for them, they said like, yeah, it's like for us like this too much of an operational burden. And also what we have experiences like whatever what actually also happens for some time that sometimes if let's say there's a data corruption in our local in our local nodes, then we had to kind of recover from that and that would typically mean, okay, we need to re sync on the whole blockchain again. So basically, our task was basically to kind of provide like, like to take a lot of operational burden from like issuers like credential issuers, and kind of take the storage part out basically, and still but still maintaining the the attributes of the blockchain being a distributed, distributed network, where different institutions organizations have an identity established and can issue and sign transactions with their private key. So that's why we build kind of our own little light blockchain node, which is called the true client. And all it does and all it can do is nothing else than just signing transactions. And basically what we do all these signed transactions, it's basically, basically just a just a hex string that we just push into our platform and we just use our central blockchain node to broadcast the sign transaction out into the network. So therefore, still, the transaction cannot still be tampered because it's signed by the by the issuer. And still like, the whole storage problem is solved because we we kind of manage the whole operations for the for the blockchain we have redundancy basically in our platform. Yeah. So another thing interesting thing that we had to solve is like the whole deployment aspect. So traditionally, like if you deploy like, if you follow like all these Hello World examples, and you do nothing else than a CF push basically on your command line and the application runs. However, whenever you deploy an application, you have a short downtime. So especially for those issuer API, so the restful API set exposed to the to the issuer client, they need to be highly available. So whenever we need we need to fix a buck or something that API needs to be highly available. So for that, we introduced a zero downtime, zero downtime deployment. And for that, we're using blue green deployment. So I'm not sure how many people are familiar with blue green deployment. Basically, the idea is that for every app on Cloud Foundry, you have a instance that you one tech blue, and another you take it green. And basically, all you do like on an update, you switch between them, you switch the routes between those applications. So that's what we're using in our deployment. So for that, because the Cloud Foundry client does not support that out of the box. So we have to write our own shell scripts for that, that we just executing with our on premise Travis CI. Exactly. So another interesting thing is also that we rely heavily on rapid MQ. So especially like when we so we have you see like a lot of different services, you will see the issue API, you see the sharing API, we have an email microservice. And basically, we use rapid MQ to hook everything like everything is happening platform, we basically do that heavily message based. So that means like whenever we broadcast a transaction into the platform. As you can see here, we kind of put it into a Q basically, and have kind of a transaction monitor picking up that transaction, put casting it into the platform, and monitoring that transaction. So that means like if something goes wrong with a transaction, we can notify back and say, Hey, there's something wrong with a transaction, please check on that. Or once the transaction has been confirmed, which actually can take up to one minute for the transaction to be to be confirmed. The monitor will watch for that and will then broadcast back to the Red Room Q. Hey, we need to send the document to the application to the to the actual credential owner and we use basically, in this case, MSN and SES for that. Also another thing, another issue that we had, especially with with cloud phone, it was God found out of the box doesn't support something like API rate limits and stuff. So for that, we still had to use AWS API gateway. And also for CDN, we still had to use MSN cloud front. So this is currently where we're at right now. But there's also a lot of things that we plan to do. I mean, track still like very, so it's a pilot project, we're still learning a lot of things we're, we have a lot of plans on what we want to change. So still, it's not really nice that the whole Ethereum stuff has to run. So currently we run it on Amazon ECS, Amazon Container Services. And there's like a lot of gap between, because like we need to make sure that everything's secure, like a whole communication between cloud phone, SAP Cloud Platform and Amazon Container, Amazon Container Services. So we had to introduce like a lot of security mechanisms in between. So our idea is like, like in the end, everything should kind of run in cloud phone. We so currently you can still run like binaries in like and containerize them in cloud phone apps. But still there's like lots of storage issues because blockchain nodes take a lot of like file system storage. And so that's why we kind of just used ECS for that. But we are planning on transitioning that into into cloud phone and use SAP services for that. And also what we also plan to use in the future is SAP API gateway for that. Yes. All right. So do we have any questions so far? Okay, there's one question. Yes. So basically, basically what would happen like so initially, if you start using if you start using the system, you would typically download this Node.js client actually. So it's, it's actually a small web service that people can integrate into their platform. So in our case, OpenSAP, they use this client, they integrate that into their existing Ruby and Rails platform. And with that client has capabilities to generate that key, basically. And for that, they would establish the identity on the Ethereum network. So whenever basically a user basically on the platform clicks, hey, I want this certificate of this course that I completed, I want this to be on blockchain, they click on a button. And in the background, the microservice would collect the document would hash it would bring it would create like our so we have to find our JSON structure for that which actually follows an open open standard we actually use Mozilla open badges for that. And basically, that data gets basically hashed, it gets written into a transaction and this transaction will actually become signed by the issuer by their private key. And from there, it is always can always be associated and verified. Okay, this transaction has been signed by this person and we broadcast it into the blockchain network. And the blockchain network will do a lot of checks, it will confirm. There's a lot of like blockchain specific stuff that need to be confirmed like gas price and like, okay, do you actually provide like a high enough transaction fee and stuff like that. It will also validate the signature and stuff like that. And eventually, this, this transaction will be up but this goes a little bit more into blockchain. So I think for that, for people who are not like familiar, how like the whole trust system with blockchain itself works, I would recommend looking further into that. But basically, the transaction at some point by the network gets written into a block, which then at some point it's confirmed by all those nodes. So basically all those nodes in the network, they use a so called consensus mechanism to agree on, okay, like, okay, so this is actually what we're going to write now. So, and basically, once it is written there, it will stay there forever. However, for with Ethereum, we're using so called smart contract functionality. So that means we also actually don't just allow issuers to write things to the blockchain. Because if we just write things to the blockchain, then why don't we just use like something like PDF signed PDFs or something like that. So actually, the whole strength of blockchain is that you have a history of transactions. So what we imagine in the future is and actually what we currently implemented is that you can also revoke a transaction. And the smart contract, which is kind of like a object containing the whole all the certificates basically on the blockchain will verify that only or make sure that only the issuer can actually revoke a transaction later on. Yeah. And in the future, we also plan stuff like okay, like expiring certificates and certificates that can be renewed and stuff like that. And all that stuff is currently like hard to handle if you want to do it like in a central or traditional way. Yeah, this is a service. Yeah. So so currently it's available like part of the I think it's the Imagine IoT course on SAP. So actually, it's not available yet. It will be available soon. We are still working very hard on getting this out. Basically, it's a service that we provide to SAP customers, basically, or actually, any people who will participate in that course. And then for like issuers, we provide kind of a library. It's actually the MPM MPM module that you can install globally. So and you can just start it with from the command line and just run your platform. Yeah, exactly. It's a parallel application. Yeah, it's still kind of in pilot and labs environment. So just doing some testing. As we said, we're still in the pilot phase, we're going to look at the scalability aspects and high availability. I mean, the blocks in itself is very much though it's very promising, it's still very nascent. So we want to make sure it's stable. Anything that comes out of SAP has to be highly reliable. So we going to make sure of that and then probably look into the commercialization aspects. Touch IOT. Touch IOT. Touch IOT. Yeah, it's an open SAP MOOC course. Exactly. Yeah, I mean, there's also there's a lot of commercial or possible commercialization aspects that we have looked into. But currently, we just want to focus on this small thing, making the platform stable. And then from there, see, okay, like, how can we scale from there? So there was also question over there. That's a very good question. And that's a that's a problem that we are currently facing at the moment. So right now, what we do, we have a limited set of issues. So basic currently, we pay them. So we provide them with the ether or with the cryptocurrency to be able to issue those transactions. So there's a lot of things that need to be figured out still. But part in this pilot, I think this is like a currently available way for us to do it. I mean, we're still looking into the business model aspect of it. But technically, when we kind of evaluate this project, in US alone, there's like 60 million checks that happen every year. And it on average, it costs for like 3040 bucks per check. So there's there's definitely a lot of money in it. But how do we commercially is who pays for it? It's still something in working progress. We want to make sure the platform itself in scale, and then sort of put the hooks into monetization aspects. Yeah. Yes, correct. Yeah, there's also like a lot of like I mean, there's also like private blockchains and stuff. But we really see like for this project, this has to be on a public blockchain, because the more nodes on the network participate, participate, the more trustworthy and stable the network is. So that's so we evaluate a lot of options. There was like, there was Bitcoin, we have a we have a Bitcoin Ethereum, things like multi chain and stuff. And like, the only things where we see like, what is really like truly blockchain, in our opinion, is something like Bitcoin or Ethereum, where it's like really publicly and transparent network, basically, where a lot of different use cases are implemented and a lot of different people have interest in the platform to be running. So that's why we, at the moment, have committed to Ethereum. And for us, as of right now, we had, we have made really good experience because Ethereum itself also exposes like really sophisticated APIs, especially like with the whole smart contract functionality. So this is very, very beneficial for us. So with Hyperledger, basically, the idea is that you would create your own, like blockchain network. So that would happen typically, like if you run a business network of like, for example, for instance, like we have a supply chain, stuff like that. And they're like the typical use case would be like you want to create your own network. And also Hyperledger solves a lot of more, more problems. For instance, like encryption of the actual data. So the data is actually not visible to all the nodes. So we are also kind of related to the team who works with Hyperledger. For now, we have invalidated Ethereum. And we have made a really good experience with Ethereum so far. And kind of the reason why not Hyperledger is there is no like, like the one Hyperledger network, right? So and also especially like if you if we want to develop like a product at this point with like higher trustworthiness, I think that's why we decided to go for a public one instance network basically. Yes. So yeah, so only in the so only at two stages, the first stage would be doing the issuing process, because like, the transaction needs some time to be confirmed on a blockchain. And for that, we cashed the certificate in the MongoDB and after we send out the email we deleted from the from the database basically. Then in the second stage, when the user actually decides by himself to share the certificate, like and store it on our platform and hence generate a link. So this would be the second time. What we also were implementing right now is kind of a self destruction mechanism. So basically the certificate or the or basically the user can say, Okay, I only want this link available to be available for let's say 48 hours. And after that this is gone. So our aim is truly like the like the user kind of on the data basically. Thank you all. I think we started running out of time here with more than happy to take any questions offline. We are hiring by the way.