 Hello, and welcome to the Virtual AI Summit in New York. My name is Marius Bogovici. I'm a Chief Solutions Architect of Financial Services team at Red Hat. And in this session, I will share Red Hat's experience of helping financial institutions build applications, and in particular, discuss several strategies to help you maximize the return on your investment in the artificial intelligence and machine learning. It is important to understand the driver for these changes because financial institutions have been early adopters of AI, and it's not a new topic for them. Existing data has been used for a long time to improve existing products and to devise algorithms that power specific bank functions, especially on the back end, especially in the capital markets area or risk assessment or even fraud. What is really changing is that banks in financial institutions find new ways to increase the efficiency of their operations. Aspects like just-in-time landing, automated compliance and reporting, improved loan default prediction, just make the bank operations faster, quicker, better, and they help their customers with the same products but working so much better. At the other end of the spectrum, there's experimentation. Financial institutions create new business models based on intelligent decision-making and insights derived from data, personalized incentives and rewards, client advisories, personalized insights for investment. So this way, they can create new value on a new business and serve their customers better through new products. As a combination of the two, AI becomes gradually interwoven in all the areas of operation and becomes an integral part of financial institutions function in areas like payments or omni-channel. So as it becomes a core functionality, functions related to AI start being shared across or outside the organization. Thinking about prediction as a service, thinking about sharing the expertise of building AI systems as a center of excellence. So this UB security of AI is also something that AI leaders have to take into account. All this is supported by a number of emerging technological trends that help support implementing these AI solutions. Commodity hardware provides simpler and more cost-efficient ways to get the compute power that you need. You don't need a larger, bigger computer when you can get a bunch of smaller, cheaper ones. Specialized hardware for AI ML also helps accelerate innovation. And usually the solutions that we see built are a combination of the two commodity and specialized. Cloud platform, on the other hand, provide the means for elastically scaling and allocating resources on demand for resource-intensive workloads without the need for the massive investment, for example, for building a larger data center that is largely un-utilized. And open source offers companies freedom to innovate and to tap into the power of communities to create new and better tools to implement an analytics and machine learning more close. And in fact, the majority of these new tools, the majority of the emerging software in the machine learning space that's powering AI initiatives comes from the open source environment and allows a great degree of flexibility to pick and choose what they need to implement what they need. So armed with these new capabilities, they can implement solutions that meet certain business viability objectives like agility, solutions but must allow for easy and rapid change to deal with changes in market conditions or consumer preferences or just try out new things. They must be deployed to scale. As I mentioned earlier, these solutions tend to become ubiquitous. So once they're deployed in environments that, for example, are part of interaction channels like web and mobile and deal with an increased number of users, they also need to be able to scale with increased usage. And when these functions, for example, become supporting functions for the entire enterprise, again, they're facing this increase in usage which demands this scalability. And finally, companies must balance the value that they create with the cost of implementing and operating these new solutions. So cost effectiveness is a big part of the story. Now, financial institutions also must meet specific standards and obligations which are kind of the fourth driving factor when we discuss these solutions. They must have the, they must be solutions must be secure. And it is important to notice that machine learning models and training data become part of the attack surface of the solution, compromising a fraud detection model, for example, can create a great loss at the bank, especially if it goes undetected. Reliability is important. Being able to run these large scale workloads is a critical part of actually building machine learning models, but also once they're deployed, these models must be highly available as they become critical business functions. And deploying your versions to production must be performed with minimal downtime, ideally no downtime. Finally, AIML provides solutions to complex regulatory requirements and are subject to regulatory scrutiny. So they must comply with, they must meet certain security criteria as we've seen, but also comply with data access and privacy regulations and meet the transparency criteria set by regulatory. What we'll talk about next is how Red Hat sees and how Red Hat experience of helping financial institutions deal with these four driving forces and implement platforms that allow them to take advantage of the business opportunities while leveraging technical advances in an environment that means the viability criteria and also is compatible with the regulatory requirements in which they have to function. So in order to move forward, it is often noticed that implementing AI solutions in financial institutions faces a number of challenges which create and always create a, which make put into question the return of investment of such solutions, because they're typically seen as requiring a lot of resources, requiring a lot of investment and very often not delivering on the promise. And in implementing such solutions, the AI leaders and their teams must clear a number of hurdles. The first one is data access. Finding and preparing quality data is difficult, especially when regulatory constraints restrict data access. So this is something that companies have to consider a worker app, but also finding talent and using efficiently that talent, especially when teams do not collaborate well with each other is another problem. So solutions are very often hard to implement due to slow manual and side load operations. And this leads to models, for example, becoming outdated by the time they reach production. So they do not deliver on their business value, accelerating that process is a critical part here. This happens very often due to either the unavailability of infrastructure of software for data scientists, for example, to conduct their experiments or to find the resources that developers need to incorporate these models into production. And ultimately making sure that all these processes are transparent and easy to assess by regulators and address ethical concerns such as bias. The observation here, and this is a diagram from a well-known paper on machine learning and its impact in building applications is that the building the model and training the model is just a small part of building an intelligent applications. Usually there is a much larger, a model is part of a much larger distributed solution that incorporates a number of supporting activities and tasks. And this can be easily seen in this diagram that illustrates a solution for building an anti-money laundering architecture. It is RedHats architecture for building for anti-money laundering that illustrates how artificial intelligence and machine learning can be combined with other functionalities in a bank in a modular event-driven fashion that leverages data from a variety of data sources to produce, to build a cohesive holistic solution as opposed to, let's say, a monolithic Antoine one-size-fits-all type of approach. But what's important here is that while the anti-money laundering that the machine learning capabilities are a critical part of supporting functionality such as know your customer or transaction monitoring, it needs to be integrated in a larger landscape where for other aspects like data integration, case management or analytics exist. So that is essentially a requirement, translates into a requirement for collaboration between different teams that are part of this process. Data engineers, data scientists, and application developers, data engineers essentially are more focused on collecting the data, transforming and preparing it. For data scientists who analyze, extract the features, build models and monitor them. And then application developers are using these models to build intelligent applications, to build the business solution around them, interfaces, APIs, management tools. So all these teams must work with each other seamlessly. And what we see in typical machine learning workflows, especially when it comes down to getting the work of data scientists up to production, this is a classic workflow for preferring machine learning tasks. We typically see two big bottlenecks in this process. One deals with building discovery environments. Simply giving data scientists access to tools, to the tools that they need, to the libraries that they need, is without the need to file IT tickets is usually a problem. So moving from the mindset of everyone does their work on their own laptop, and then somehow shares their results to a mindset where they have a platform which they collaborate is a big change here. And the other one, of course, me is concerned with moving these models into production. Just kind of throwing notebooks over the wall is not a recipe for success. So integrated build pipelines that take the results of the earlier group and translate it into seamlessly working services, I think is an important part. And again, these kind of bottlenecks, the solutions here share a number of common traits. There is a need to have reproducible environments. What worked on my machine needs to work in the cloud, needs to work in the public cloud as well, or in my data center, so a different environment. All of them need access to specialized hardware. In the case, for example, of production deployments, they're also a requirement to monitor against degradation and drift. So what's the solution for addressing these pain points? How do we see this happening and how financial institutions have been successfully implementing such workflows? Essentially building a single workflow that unites the multiple personas, recognizes the needs, their different needs, and provides self-service access to tools and data, and minimizes the burden of the IT operations, people that have to support it. And the way this workflow takes place is through this conceptual architecture for operationalizing AI that we are proposing. And we have used for successfully helping customers build and accelerate the AI initiatives, as you will see later. At the top, essentially at the top of this architecture, you see the project lifecycle with a business goals, like going from the setting of the business goals to developing the machine learning model, to implementing applications and inference, and monitoring and retraining models as needed. In order to execute this process, in order to support it, you need a machine learning software tool chain that consists of specialized libraries, tools like TensorFlow and Spark, and Jupyter Notebooks, Python, Selden, a few other. And you can recognize that all of them are essentially open source tools. So this is kind of coming back to the conversation on the power of open source as a catalyst for these types of environments. So how do you get access to them and use it in a seamless fashion is the key question. Now, also there is a need to have these access to a variety of data sources, databases, SQL, NoSQL, data lakes, data pipelines that provide the data for the data scientists to transform and later on, provide the access to data for inference. So all of them, there is important, like having a single platform on which these two, these components can run side by side is critical. So the platform here is essentially the key component that enables the seamless operation of these data pipelines and tools is the hybrid cloud platform, is a hybrid cloud platform with social service capabilities based on containers, Kubernetes, and with embedded DevOps practices. I should also add security here, which is like baking in security as part of the platform is a key way to address the regulatory constraints that financial institutions require. Now, this hybrid cloud platform also provides an abstraction layer for simplifying access to hardware accelerators, right? Like GPUs that help speed up model development and inferencing tasks. And the role of the platform is to make this access to these resources transparent, just as it is to make access to infrastructure as transparent. So offering the consistent experience across on premises, public clouds, edge locations, and can be efficiently managed by the operations provides a seamless experience and provides this flexibility goal of being able to deploy your models, being able to deploy your workflows whenever you need it in the way that you need it. As I mentioned earlier, one of the foundational blocks of this hybrid cloud platforms is containers that provide the agility, flexibility, portability, and scalability for data scientists to build and develop models and for developers to code software applications powered by these models. So of course, deploying and creating, deploying new applications and getting access for resources requires them to be agile. And not waiting, for example, for a ticket to IT in order to deploy the task. Also containers means, containerization means that applications with different profiles of a different nature, like your job application that access is a model and for example, the Python application that embeds the model, run the same way on the platform. Those will provide portability. So the ability to share and deploy models consistently across in the different target environments, the ability to provision environments as needed and when needed. And finally, being able to aspects like scalability and self-healing and high availability means that this Kubernetes provides this natively for containerized applications. So, and we can easily scale your applications in a declarative fashion as managed by Kubernetes. So this is essentially captured in the, in Red Hat's Kubernetes powered Red Hat OpenShift container platform that is the foundational tool that helps accelerating the ML lifecycle. So OpenShift, of course, provides, simplifies the deployment, scaling and lifecycle management of containerized ML tools. And that, like HDOAI, Starburst Presto, Seldin and ensures high availability and faster time to value and also provides access, as I mentioned earlier, to hardware accelerators like NVIDIA GPUs. So it means that it acts as the glue between this specialized software and the underlying capabilities. It also provides consistency and portability, offering consistency of day one and day two operations across the data center, edge and public clouds. So machine learning and application development workflows are portable across different environments. And also it extends the value of DevOps to the entire machine learning lifecycle, enabling collaboration between different teams. Finally, it is a hybrid, it's a fully integrated hybrid cloud platform that includes key capabilities like monitoring, automation, DevOps dual chain, completely built on open source projects. This helps drive innovation without vendor walk-in. Now, to illustrate this and to illustrate how this happens in practice, I'm using a case study based on a successful implementation of any platform at the Royal Bank of Canada. The Royal Bank of Canada is a top global bank. It handles massive amounts of data, easily 10 billion clients in directions per month. It has identified artificial intelligence and using machine learning as a critical business capability, going so far as to create its own research center for AIML called Borealis AI. The challenge that the Royal Bank of Canada had was that AIML projects to up to two months to get off the ground. So they couldn't really see the result of the investment because the getting them into production took too long. So, there were a number of symptoms of that, workflow orchestration and social service access to GPU was slow. It was hard to build platforms, the actual applications and this of course security and compliance requirements were hard to meet. The goal in the end was to build an AI platform that could serve its more than 100 AI developers and engineers helping complete projects faster, but most importantly, adopted DevOps culture into Leverage Cloud to improve efficiency and application through these cycles. And the solution essentially was to use Red Hat OpenShift to build this platform in collaboration with NVIDIA. So this is another great story of how Red Hat in collaboration with its partners, helps deliver value for customers and help them accelerate their processes. It relied on a few things that we kind of spoke earlier, continuation of machine learning applications and services, building a private platform for banks, AIML center, a unified platform to which the different teams that are part of the bank can collaborate. And of course, it simplifies access to hardware for the NVIDIA through the NVIDIA operator. The results speak for themselves, more than a thousand models have been deployed, experiments run 10 times as fast, models take days to build instead of months and the performance of the system is great. It can process 13 million records in less than 20 minutes. So this not only kind of helped RBC to meet its goals, but it also set RBC on a path that where it's able to, when it's thinking to leverage AI in customer facing applications as well. So this story has not only helped RBC meet its goals, but actually it sets the path for future success. In summary, what we talked about here today is about how financial institutions that have been have a long-standing experience of using data-driven processes are launching new initiatives. And in order to be successful with these initiatives, in order to leverage, like to meet the business opportunities to take advantage of the technological advantage and to meet the business objectives and regulatory obligations, they need to build a platform that allows self-service access to tools and data and reduces the operational IP burden. Such a platform is provided by Red Hat OpenShift that provides container and hybrid cloud-based capabilities that provide the flexibility, reliability and security required by financial services environments. And as an example, you've seen how top financial institutions like RBC have accelerated the adoption of AI and can easily launch new initiatives, creating value for the business. I will end with a number of resources, a number of hybrid cloud success stories from Red Hat, a bunch of reference architectures for the use of artificial intelligence in a financial institution setting, like the architecture for money laundering, which I've spoke about earlier, and also an e-book for an example of operationalizing AI into production. With that, I would like to thank you for attending this presentation and to wish you a, to enjoy the conference.