 Thank you. Thank you for joining us. I am Bruno Carrasco. I am here with Akshay from Oracle and we are going to talk about how to build a cloud native image recognition solution using Oracle Cloud infrastructure and services. I am from Mexico and this is my first time presenting in KubeCon. What about you Akshay? Me too. First time at KubeCon. Very excited to be talking to you all. Okay, so let's get started with our own introductions. I'm going to do it first and then Akshay is doing himself. So I am from Mexico as I already mentioned, I started my own company like 10 years ago. Sorry, it is an MSPS. I used to work for Oracle about 2002 or something. And then 2010 I just quit and started my own company. I've been working with Oracle for several years, Oracle technology I mean, and that is why I've been joining a couple of advocacy programs, Oracle ground breaker program and Oracle laser director. And my background is with distributed systems. That's what I've been doing for a while since the old days of enterprise architecture integration and web services as way. And now with these microservices, cloud native architectures and principles. My contacts are there with Twitter. My Twitter handler is in the slide also my LinkedIn. So you can contact me with those through those channels. So I think Akshay you can move forward. Thank you. It's my pleasure to be talking to you all with Rolanda today. I am the principal director for Oracle cloud native DevOps and observability. I have about 12 years of experience first with on-prem technologies and later with cloud infrastructure across various organizations, Dell, Amazon Web Services, NetApp and startups after that. So very excited to be talking to you all if you want to reach out to me my social handles are provided here. So let's talk a little bit about the story itself. Right. So this is a presentation that has a lot of practical implications. The seeing that we are in cube con this story is about using cloud native principles for this particular very practical problem. And we want to share that experience with you how we created these APIs on top of cloud native technology. Now the use case itself is about the use of scanned documents IDs contracts and so on. So traditionally, there are a lot of organizations think about loan processing think about bank banks and those kinds of industries. Anything that requires a processing of documents is usually employed. Usually employees, you know, people to manually look through those documents, populate those fields, maybe name your bank account number, you know, other identification data manually. And as people do it manually, it can be slow, not to mention it can also be error prone and it could honestly be a waste of human resources. People are, you know, made to do much more complicated tasks. So that's the problem we want to solve using cloud native and AI technology. So the context for this is that, you know, you could have multiple channels in which these documents are provided right maybe they're provided directly, maybe they're provided to a bank teller in person by the customer. Maybe they're provided by web or mobile, somebody just, you know, uses the website or their mobile app to send the documents over could be sent by email as well. And there are many other ways, right. And all of these documents usually have to sometimes be validated against some kind of enterprise system, a financial institution like a loan processing company might want to validate validate the account number of the person, the name, the address and so on. They also might need to be validated against some kind of government registry, maybe like if you provide an ID, you want to make sure that that ID is genuine. It's not a false ID. At the end of the day, you know, SPS undertook a project with a customer using Oracle technologies to solve this particular problem. The problem was around automotive loans and credits, and the loan approval is actually done only after a number of checks like credit check ID check for fake IDs and so on. And there are 15 types of documents you know that need to be reviewed across addresses loan payments credit scores, and many other things right. And you want to do it in a cloud native fashion. Now Rolando is going to tell us a bit more. We work with this customer who has a specific challenges, and they were looking to streamline the time that they take to approve or to reject a loan credit or right or a loan or a credit for automobile. They were reviewing already these images and these documents but sometimes they had already a scan, but the quality of the document was not good. For example, sometimes their documents were scanned in T format, so it was very low resolution. Also, another thing that they had or another problem that they had is that sometimes the image was not in the correct position. Okay, so they needed to rotate it in order to get the information and sometimes while they were doing the rotation some of the elements were were lost. So this process it is very relevant for them because they use they actually use the documents in order to approve or to or to reject the loan evaluation. All right, so they use those documents to get it very specific and valuable information for them in order to say, okay, this is a good person in terms of their score and whatever he's showing to us, and we are going to approve or reject the loan. Okay, so this was kind of the scenario, tons of information documents and so on, and with that trying to approve or reject. So our main goal, it's been always to offer something to our customers and to this specific is a specific customer as well. Obviously, which is to expose a single API that can receive the document already scan, not only to retrieve the information or to get information from it, but to classify it to manage the the information and actually insert if you will the information into the back end, transform the format we needed to do it to rate it for example if it was rotated or some of the elements that we retrieve from the document to be classified and tagged properly. And with that return a level of confidence so so so the customer can can realize if the information that is being retrieved from the document is with a high level of confidence or with a low level of confidence. So every time that we increase that level of confidence then the customer is feeling much more comfortable in order to move forward with their processes alright. So this is our main goal so we are going to be talking about that we created an API for them and actually an API that we offer to many other customers so so you can get the idea about many customers using the same API and then all these cloud native principles are coming together in order to to make a good offering. So the first step is to get the image recognition solution which is via the API that I just mentioned in the previous slide, which is supported by many Oracle cloud infrastructure services plus a couple of Google services alright that we are about to talk about and then for this specific customer we also incorporated an RPA but which is actually reviewing the information that we get from the first step and then making the decisions either to approve or to reject the loan. Alright, so those two things for this specific customer were created to solve the problem, but for us SPS we created the API and a couple of things inside that API in order to offer it to many other customers with cloud native solutions alright so that's that was the consequence of dealing with this project and creating this API. Okay, right. So, you know, we have a number of services from Oracle for cloud native and DevOps, you know, but as I talked through these services, the ones that are marked with the checkmark are the ones we use, either services or projects that are relevant to this particular use case. But before we go into that one thing I wanted to point out is that Oracle cloud is committed to its openness of its cloud platform right so resource manager for example is an infrastructure as a service offering that is compatible with open source Terraform. API Gateway builds off of open standards like open API 3.0 and API blueprints and so on. Functions is based off of an open source project called FN project that is led by Oracle container engine is 100% open source based off of Kubernetes upstream Kubernetes. The streaming services compatible with open source Apache Kafka. We also leverage the cloud events standard for our event service, you know, you can specify targets using fluency agents for the logging service. Grafana for monitoring as well. So the first one we want to take a look at is API management and API Gateway. So when it comes to API management, you know, API management could be about design of API is the deployment of API is and then promoting and consuming the API is right. The one that's particularly important in this case is the deployment of API is and by deployment of API is, you know what I mean is you should be able to receive API calls, you know, as is the case with the application to receive API calls at scale. And also receive them and send them to a back end service that could be load balancers compute Kubernetes serverless functions. So serverless functions is what we use in this particular case. And you also want to provide the necessary security and rate limiting for these API API calls to. So, Rolanda will tell us a bit more about the use of API Gateway. Yeah, sure. So there are two main things or two relevant things that we can mention about the usage of Oracle Cloud Infrastructure API Gateway for this regard for this specific project, which is the first one is the customer was looking to have a solution where they didn't need to use elements in the on premise environment, even within the cloud environments that they already had they wanted to have something like can be used as a service so so we needed to to have a place to deploy these services. And we realized that Oracle had this API gateway that we can be using in order to expose the API is that we deliver to to our customer so that was one thing. And second thing was that we also didn't want it to to manage any servers or any anything related with infrastructure we just wanted to use the service and expose the API. And it happens that this gateway is just like that it is a it is a server less offering from Oracle that we just provision it and then just use it so it was it was very straightforward to do that. And for us, it was fulfilling those two things that the one the customer didn't want to provision anything other end. And second thing that us SPS we didn't want to also lose time in order to provision anything related with this so so to where the two main reasons that we that we use for that and that there is an extra that we are about to mention, but it is the relationship that it has with the Oracle functions which is what actually is going to be mentioned now. Yeah, Oracle functions is a serverless platform. So functions can integrate with Oracle cloud infrastructure services of course, it's a part of Oracle cloud infrastructure that can also integrate with platform services, and the sass applications from Oracle, right functions is based off of an open source project called fn project that Oracle leads and, you know, being based off of open source. It allows applications to be easily ported over from other cloud and on premises environments. Rolando, could you tell us a bit more about the use case. Yes, sure. So I just mentioned in the previous slide, the reason or the reasoning about using the API gateway, correct. But then we needed to implement those API so so we also wanted something to be pretty straightforward to use to deploy to manage and that we had flexibility in terms of the programming language that we were using. And also because of the model that we were looking for with this specific customer, which was to try to charge them by the usage of our system of our API. Then the usage of functions was a very, very technical appealing product for us to use. And the second thing is that the serverless model and the commercial model that functions has is also very good for us because we are just paying for the usage for this. So we wanted something very flexible for the programming language again that I just mentioned. We were looking to use Java functions and also node JS functions. And we just realized how integrated it is with your API gateway. Great. So, of course, we used the Oracle Container Engine for Kubernetes, which is also called OKE containers as you know, easily package and move apps. And they package just the code and the dependencies without the operating system so they have advantages of being able to be spun up and spun down quickly. Container Engine for Kubernetes provides orchestration at scale. It's available globally in a number of regions of Oracle cloud. It's based off of unmodified Kubernetes and it's a managed service, of course. It's integrated with Oracle's registry, it's tools for CI CD like the developer cloud service and other services to Rolando could tell us a bit more about how he's using it. So we just mentioned those API gateway. That was the first step for us. Then we used functions as well. And for our first phase of delivering this API, it was good enough to have that because we were able to fulfill the customer requirements. We were processing their documents and recognizing the information within those documents. But we are already working on a phase two for this project. And for our offering for many other customers. And then we have these alternatives and there is where OKE came into the picture. One is that sometimes customers are looking to deploy the solution in their own cloud environments and then normally are using a Kubernetes based cloud environment for deploying applications. We are able to deploy our solution on top of Kubernetes and of course on top of OKE. So that is one thing we just realized that customers are looking to deploy it at their end. Why? Because of the documents are sometimes are very, or for security reasons, that's what I'm trying to say. And they want to have everything managed at their end. So that's one thing. And it is kind of normal to have Kubernetes as the platform where we deploy this. And the second thing is that we are improving our solution and not just offering a single API, but we are actually offering a set of web applications where the customer can keep track about the verification of the file or the document or we can provide to them analytics for the usage of the platform and many other things that can be used through a web UI. And we are actually deploying that web those web applications on top of Kubernetes. So for our from our end, it is pretty easy to have. Well, we just mentioned in terms of functions and OCI API gateway plus OKE because we can mix those things together and and deliver a very good solution for our customers. So, so those are the two things that we are looking for our phase number two, actually. Great. Object storage is an internet scale storage platform for any type of data. So you can securely store and access any type of data format. The reference architecture over here just shows an example of how you can use object storage you upload an image, and you use the Oracle functions to get the metadata and store it back into object storage along with the image. Rolando would tell us a bit more about this more complicated use case. Yes, and these subjects of storage it was very straightforward for us to incorporate it because we are archiving and we are first we are saving documents and managing them through the object storage so every time that we receive a document we are restoring it in the object storage why because for later purposes, for example, the tracking of the document, or if we need to retrieve the document to for any other type of analysis, then we have it there. And secondly, because we can archive it. And this is just out of the box. We just generate the rules for the archiving and the information is being archived automatically. So those two reasons where that were our reasoning in order to use it one just to store every single document that we are using and secondly, because of the archiving capabilities, and it happens that between functions and object storage is pretty simple to to mix them so technically also it was very straightforward to do it. Okay, so now, before we go to the demo, we just wanted to highlight how we are connecting the dots and connecting the dots means that we have the use case. We already explained to you the use case, then we explain a set of Oracle services, and then we are going to connect what we had as a use case and as a need, and how with Oracle technology we are able to fulfill that need. Okay, so we had now all the information in order to to deploy our application and fulfill the needs that that we just mentioned. So in the next slide we are going to to check exactly how how we connect those two to sites of the story. All right, the needs and the, and the Oracle technology because this is the process, the solution of our process that we, after we realize how to use the, the different Oracle elements. This was the set of seven steps that we were able to deploy and and to make them together in order to deliver the solution so there is one first step to transform the documents then how to upload them, then how we classify that document. If it is an ID or if it is a balance or whatever, and then how we curate the document the document remember that sometimes images are rotated or it has some elements that are for, for example, low resolution. Then we go to the Google API is and retrieve all the information and then we tag every single element so we can identify it properly. We also have these machine learnings in order to identify the documents and obviously the recognition about the elements from within the document so so these are the seven steps that we were following and those seven steps are going into this the elements from Oracle or components from Oracle so we're going to just highlight two or three of them. So for example for the documents upload we are using again OCI API gateway which is the first contact from our customers with us in order to receive the document the object storage where we are storing the information also the classification curation and tagging we are using the Oracle functions we have a specific Oracle functions that are doing that. And then we have other other elements that we already mentioned like Google for the OCR and also for the machine learning. Okay, so now we are just going through a very fast to review this flow, which is a sequential diagram or sequence diagram sorry where we can see how these elements are being played together so whatever we just mentioned in previous slides about the OCI API gateway, the fm project Oracle functions, Google, and the object storage is it is right here so you can see these steps that we are taking and how the information is moving and it is hitting every single product that we just mentioned and how we are using it. And again, every single layer that we are using is is is a serverless offering either from from Oracle so so it was pretty, pretty easy to deploy it all right. So, this is the architecture that we use and I mean this is very, very simple diagram. But we are going to use it. This is the things that we use so functions getting related with the object storage as we mentioned the OCI API gateway presenting those functions into the external world for our third parties and being the first document that we received the document. And we are also supported by the Google API is in order to retrieve the information from the document and also for the machine learning itself. So these are our environment this is our ecosystem if you will. Okay, and then we are going to to the demonstration in order to show you whatever we just mentioned in previous slides. All right. Okay, so let's go for today. Okay, so this is the web application that we built here in SPS. This is a work from Pliny Arvisu and Arturo Gonzalez from my team all credits to them because they were the ones who created this. I am showcasing it now but they are the ones who created this from from scratch. So this is the web application so this is basically what an end user could be using but behind the scenes it is happening. What I just explained so the API gateway functions. The Google APIs, object storage and all and all what I just mentioned in previous slides. All right, so we are going to do two things we're going to scan or not to scan we're going to upload two documents one it is a high level or high confidence level sorry document which is in a very good shape if you will that we are going to evaluate the score for that document which is a telephone line balance for Arturo actually and secondly we are going to upload balance but for the internet subscription for our company SPS and you're going to get the difference from those two the first one which is this which is a tell makes balance. This is a document that we have the scan for that and the level of confidence for this is very high that just keep in mind something that if you are just thinking well this is basically they are uploading the the document into the Google API and just retrieving whatever Google is it is getting back to them. That's not the case because in the normal process what we have is these rastering process which goes from left to the right and from up and down and it is basically taking the elements of the document as I mentioned from left to right so if you can imagine, for example in line which is CLL Porfirio Diaz 8 and then Sue Stado de Cuenta Pueze. It has nothing to do the one in the right the left sorry is the address. Okay, and the one of the right. It is just a paragraph it is informative things or it is just informational section of the document. Alright, so if we go just left to right and get the information from it is not going to be very valuable it is valuable itself but it is not relevant for the for the business process itself so if we go down and see the result for the API this is the actual result of the API we have different sections in the result. The first section is this which is what we call the quality gate so as you remember in the sequence diagram that I show in previous slides. We had the first step or one of the first steps was to get through the quality gate and the quality gate is returning the general score for the document and the general score of the document. We are trying or we are targeting to have it above the 70% so it is 0.7 if we have a general score above above 0.7 then it is a document that we trust that the information that easy that this API is retrieving is accurate. Okay, if we get an score below that 0.7 then you can definitely use the information but the level of confidence is not going to be high. And the second thing that this thing is recognizing is the language of the document in this case is this is Spanish so it is it is getting the information in the right language in this case again is Spanish. And then we have whatever the OCR returned to us. Okay. And this is basically all the information that was able to retrieve. It has now a little order itself. Now we have a structure it in such a way that it is not the left to the right and up and down that I just mentioned for the rastering process. We have very clear vision about addresses, names, numbers and so on. But if you take a look into the lower part of the application which is the NLP detected labels. Now we are not just returning the information from the document but we are able to classify it so we can see that this has an expiration date, which is Regencia, which is the name in Spanish. And it's clearly Gonzalez Romero Arturo we have the address for for for Arturo and the action it is the name. Or that's the word that we use in Spanish direction address in English, and now we can get or return the information but in such a way that it is, it is understandable what it is, we are not just returning the information and that's it and okay use it to classify it and give you a level of confidence in this case, pretty much the name, the expiration date, and the address you have 100% level of confidence that this is the right information so you, you can manage this information at your will so this is, this is a very high level of confidence actually the top level of confidence. But in the other hand, if we upload the other document which is internet subscription balance by this company by the name of steel. And this is for our own company services and process solutions, or SPS. This is probably you are thinking well this is a good document it is clear. The image is not that bad. But there are some elements that are not going to allow us to read to return a high level of confidence. One of those things is that the size of the font it is, it is kind of a small so as you can see there are sections of the document that we see that the size of the font it is not that good it is actually pretty small so it's going to be difficult to, to retrieve it and actually Google itself is, or has a minimum regarding the size of the font so that's one thing the second thing is that some of the elements that we can get from the document are elements that can be confused in what way that probably instead of a zero we are recognizing it as a no for for example so there are elements that may be complicated to, to recognize and the third thing is for example in this square that we are putting here in the screen. It is our long strings of information that in this case represent something related with the IRS. And as you can see those strings are, first of all they are kind of long. So in order to identify what they are, it's going to be difficult because as you can see they have nonsense they're just pretty much a sweet sequence of characters so if we go below to the quality gate score. Then we may see that they this document in a specific has a very low score point 53. So the quality gets says no this is this probably needs a manual revision. It doesn't mean that you will not be able to use information actually the information is here in the section of the OCR structure takes we can see the information by itself. And in the NLP detected labels we we see that this level of a score is not that high as the previous one. Okay. So we did the whole process, we are able to to process the whole document and retrieve from from the API is the value or the level of confidence and the labels, and also we are able to group the information but this is not so so high in terms of the score. And just to finalize the explanation of the demo, as I was mentioned in the rastering process. That's something that we use, and we actually program in Oracle functions, because we know that because of the position of the elements in the document we are able to recognize what it is and to identify that that is that the name is written actually in the name section. This is written in the address section because of the position in the document so if we make some some mathematical calculation, we are able to to to to identify and group the information that we get from the documents but it is basically mathematical calculation, because at the end the rastering process, it is at the same time is giving us a matrix dot matrix so we are able to to get the information. Doing some mathematical calculations. Alright, so this is what we wanted to show you, we show two different documents to different levels of confidence score, we explain how those different elements are in the mix. And with this probably you are, you are having a different idea of what we are what we are doing and what we are achieving. Okay, so thank you for being with us in the demonstration. This is the list of things that we learn for deploying this. We have benefits for the business and also technical benefits, the benefits for the business and we are, we are conclude we are giving you these conclusions because cloud native is about worrying for the business needs. And we are using the technology in order to deliver a solution for that. So it is very relevant for us to highlight the benefits for the business and what we learned with this. And the first thing is that we streamline the credit and loans approval and delivery. So it was, it was very clear how this reduce the amount of time that they were using to review the documents so it was maybe a 90% of the time was reduced. Also, we are allowing customers to upload information directly, they don't need to go to the branches, or have an in person meeting in order to deliver the documents, or scan those documents they can do it through different channels. The institution it is actually now presenting API for third parties. So this is also very useful for them. So now they are offering these services for third parties. So now they can have external organization using their platform in order to review documents and and be part of this chain of activities. And also for the in terms of manual intervention we reduce that because now we don't have this large or very large teams reviewing those documents. And for the technical side of the house. We streamline the deployment obviously it is very easy to deploy this. We can actually do it at the customer infrastructure if they want that we are also having constant changes and we have the ability with cloud native in order to incorporate those changes. And also in terms of a scalability, we have this, this way to scale and pretty easily with with all these Oracle cloud infrastructure components. So, so this is kind of our summary. Hopefully you found that interesting. And we are just finalizing sharing with you, our contacts again, our social network contacts. And so you can check out the GitHub repositories that catacoda catacoda scenarios from Rolando. And also, you know, we have some product web pages for Oracle cloud infrastructure if you're so interested in giving a look, or even trying the cloud. So, please feel free to reach us on Twitter on and LinkedIn, our handles are provided there. They're now ready to take any questions you may have.