 Good day. This is the session on what, why, and how of federated machine learning. I'm Neeraj Arora and I'm joined by my colleague, Lane Pung. And we are glad to have this opportunity to have you here with us. Both Lane and I work at VMware and are involved in machine learning and artificial intelligence initiatives within the organization. We are also involved, Lane, more deeply than I, in FATE, which is one of the many ways you can realize federated AI. FATE is an open source project and is discussed in this presentation. In this session, we'll cover federated machine learning at a conceptual level before looking at FATE broadly followed by a demo. We have been working on the FML site portal and Lane will take you through a demonstration of that also. This is an introductory session and quite a bit of information for one session. And so we'll leave you with references that you can use to learn more. Of course, we are always available to help you. So please reach out if you need to. Let's get started. AI or ML is realized using three things. Data, algorithms you can apply to model the data and enough computing power to achieve this. Most algorithms you'd want to use are available by libraries. Computing power has also been getting cheaper and readily accessible. The barrier might be cost, but for the most part it is readily available commercially. Data, however, is a different story. Due to regulations or sheer logistics or the foresight required to model a problem and the cost associated with collecting a large quantity of data, getting the right and enough data for model building remains time. As a community and industry, we have tried to solve this using public open data or methods such as transfer learning whereby compute intensive training on a really large dataset is performed once by a capable party and then the model refined for specific uses. Even if an organization has the capability to collect and store data, it may not all be available where the compute power is available to utilize it. Or there may be regulatory issues or privacy consideration preventing relocation of the data. These are the type of problems that federated learning can help with. There are a couple of excellent resources by others specifically Google that can help explain the idea in detail and I'll try and introduce it simply. In what we might call traditional machine learning, we bring all the data together in one central location where one or many compute resources as might be the case with distributed clustering systems train a model using the entirety of the data. In a federated learning system, we don't collect the data in one central location and the data lives on remote devices or edges or nodes. These remote devices or edges train a model with only their data and submit updates to a central aggregator which aggregates all such updates from all remote devices. The resulting model is then pushed back to the remote devices where training continues to refine with the model using data available to them. It is an iterative process and as you can imagine, it allows for some imaginative distribution of work especially when it comes to the train validate test process for model building. We'll see that in a following slide. Significant amount of work has been done and is ongoing since federated learning was first proposed. This is an excellent survey paper if you're interested. As you read through it, you'll see that federated learning provides some very novel answers to the problems with data science. It is not difficult to imagine how federated learning may affect and change the world of IoT and industry language. I'll try and illustrate the essential concept of federated learning graphically and we use federated learning and federated machine learning interchangeably in this presentation. So let's assume we have a few mobile devices with their own local storage as participants and one initiator and aggregated device. It's shown as a mobile device but realistically it will be a heftier system. On the screen, the participants or nodes are shown on the top row named P1, P2 and P3. They're connected via some networking technology be it 5G, LTE, Wi-Fi or Ethernet or any of the other million choices. The initiator initializes a model and sends it to the various nodes. The nodes train the model they receive using their local data and send the updated model back to the aggregator which combines them into an updated model. Surely I have simplified the process for explanation instead of the entire model being exchanged. For example, only the Delta weights need to be exchanged but the concept is the same if you will. Let's look at the process again but this time let's extend our imagination to the three phases employed when model building namely training, validation and testing. Not all nodes always participate in training or validation of a federated model. The aggregator or the server decides which nodes participate in a run or an epoch. By excluding participants and consequently the data those participants hold locally we are dividing the data set into subsets just like we divide data in a data set into training sets, validation sets and testing sets. Notes that participate in training in an epoch will be excluded for validation and testing in that epoch. I hope that it helps explain federated machine learning broadly. Overall, there are three types of federated machine learning at the moment. There's horizontal federated learning, vertical federated learning and federated transfer learning. Horizontal learning is where the same fields exist with multiple nodes. For example, branches of a bank that use the same software may store the same fields about completely different users. If this data is stored locally with each branch then this is a good candidate for horizontal federated learning. Other examples would be network of hospitals, shipping companies or manufacturers with similar assembly lines. Vertical learning is where different fields exist with multiple nodes regardless of the entries in the data having any relationship or not. A good example of this would be different merchants or businesses that service the same population such as a credit card issuer, like a bank that issues credit cards which customers use to pay utilities. A good use case would be a bank geotracking credit card usage and determining either lots of customers not local to an area or descending as tourists or for events with the utility companies in that area or public transport using that information to tailor capacity for changing demands. Federated transfer learning is applicable similar to regular transfer learning where generalized models with aggregators can be specialized for local use by node. For example, infectious disease detection at hospitals, fraud detection across banks operating in different geographies and fault and accident prevention by mechanical failure across an entire industry that uses similar equipment. Summarizing simply, federated learning attempts to address some of the issues that we are seeing more and more, namely the issue of privacy, trust enforced either by user choice or by regulations, the issue of aggregating lots of data in one location, the lack of data itself possibly driven by the cost of bringing significant amount of data together. Depending on the kind of nodes you deploy, it may help with sourcing compute by conquering a large problem by not only dividing the data but also the computational load and depending on your implementation it can be highly scalable. Although federated learning brings in these benefits, it also presents some challenges. Being a heavily distributed system, it can get tougher to secure data and apply security policies. This could compromise data confidentiality with emergent attack vectors. Being distributed will also affect availability of both compute and data. There's also the issue, and this depends on your use case, of the integrity of data that remote nodes are presenting. At a basic level infrastructure events that are relatively easily solved in a data center environment become complex in a distributed network and more so in a widely distributed network. These are some things you want to keep in mind as you consider federated learning. We saw earlier the three pillars that support AI and machine learning. I'd like to add a fourth pillar here, ops. It is one thing to build the model but entirely another to include it in a product. Models are also often refined with new data that is constantly collected. Federated AI technology enabler or FET is a suite of libraries and services you can deploy in various forms on various form factors to enable your solutions with federated learning. FET brings together algorithms on distributed computing power whether it be servers in data center or compute power and edges with data that is local to those nodes. FET also adds this additional pillar with its various sub projects as you'll see later in the presentation and in the demo. FET is also open source and open to community participation and we invite you to participate. The ops pillar is critical for success with any ML or AI initiative in production. Our work on FET adds in pluggable compute libraries, data sources, the ability to scale across clusters and nodes, full lifecycle model management of FML-generated models with resiliency against infrastructure events being built in. With the right combination of third-party products you can also extend the capability of FET. And now I'll pass it to Lane to take you through some parts of FET in a bit more detail. Over to you, Lane. Thank you, Lila. I'm Lane Tan. I will cover the projects we've contributed and working on to bring federated learning into production easier and then we'll demo a full flow of federated learning tests. We know besides the capacity of normal machine learning, federal learning plans, actual operational changes to cluster management to help easier to manage the federal learning cluster with initial and open source project, QFET which is part of the FET community to manage the federal learning cluster in one site. The QFET support both Dock Compose and Kubernetes environment. The Dock Compose is decided to equip staff and for verification and development cases. It can be very quick to provision a two-site federal learning environment in your laptop or machine and may you try the learning develop and the learning operation in minutes. If we consider the production environment, we suggest the Kubernetes deployment because in this case, we provide extra features to show professional changes such as unified control for multiple FET cluster which may happen when an organization collaborate with different pilots and how to keep the network device isolated from each other. Declarative resource management which is proved a good practice we manage a FET cluster and QFET also provides upgrade migration, log creation and monitor all these very useful professional features. Both Dock Compose and Kubernetes is based on container so it can run on premise their centers or public or private cloud. And in FET, we have embedded Jupyter notebook for their scientists to program their machine learning applications. However, as introduced above, federal learning is composed, terrorized and include did not only machine learning but also critical knowledge critical and infrastructure technologies. When we meet a real user from traditional industry, there is a strong requirement to lower the bar to use federal learning culturally. FET is designed an internal pipeline and with a lot of our boss federal learning operations. We can program our application as composing a pipeline. Based on that, we decide the FML side portal which provides a local federal learning experience. The FML side portal provides federal side management, data management, model management and project management features in a web GUI style. With it, we can upload and into the data manage data contribute to the federal learning. Take our boss algorithm in FET such as homogeneous logistic regression, homogeneous SD-boost, et cetera and rating the result all these step by step and put them into the pipeline and kickstart the federal learning job all by creating the mouse. For easier to get what I introduced, the FET to FET FML side portal, let me show a quick demo. In this demo, we will deploy a FET cluster on scratch on Kubernetes with QFET and then we add this new deployed FET cluster to our federation and do various learning with pre-deployed FET. We use Kaggle case as the demo. We firstly call the application in a Jupyter notebook and then we will show the GUI introduction of the FML side portal. In this demo, we use open source Kaggle data to create a cloudflow detection as a data set. As we prepare a small Kubernetes cluster to simulate various management side and other two Kubernetes cluster, the first one we have pre-installed FML stack include FET, FET side portal, Jupyter notebook and other components. In the second Kubernetes cluster, we only pre-installed to FET which is an open source project hosting in the foundation to provision and manage FML cluster of a party. We can provision the FML cluster FET with command line. 10,000 is the ID we used for a FML cluster. We are working on a project called life cycle manager which can provision and manage FML cluster from a GUI app GUI. Now we got an FML cluster installed in party 10,000. We can use it to demo our federally learning job, credit card for our detection. As said before, we downloaded the open data set from Kaggle and 3D, the data into two harvest. Each party keeps only half of the data to simulate its own private data. Then we use the Jupyter notebook which is included in FET as a user interface for the entities. Similar with traditional machine learning, we can write a training code using passion in notebook such as preparing data and setting up the training job. The algorithm we used is called horizontal logistic regression. It is also called homo logistic regression. It is a little complicated to implement this algorithm. FET provides a lot common federally learning algorithm which we can directly use for our job and made the training as a composite pipeline. We run it step by step. Here it goes. After training job is submitted, we can see the training status in FET board, the FET component to visualize the training process. After training, we got our trained federated model. The model contains the knowledge from both parties and the performance is similar to that of an essential training model. But we got it totally without data piracy with leakage in FML. The process of federated learning only is changed in creative gradients. One more, the trained federated model can be converted into other model format such as on this PyTorch and 10th row. Next, we can publish it to our cave serving service and serve it from there. Maybe you found we use Jupyter notebook is still a little complex. For easier confederate learning modeling, we are working on a project called FML side portal which can replace the world we just demoed in Jupyter notebook. In FML side portal, we can manage local data contribute to the project. The model we obtained, the collaboration training project. Our project is a concept to manage the cooperation between different parties. It consists of participants, data and jobs. We can manage different federated learning project in this unified interface. Our job is the FML task we want to complete it. It is a pipeline with different outboards federated learning components. For example, the training we did in the demo with Jupyter notebook is our job. We can have other different jobs such as PSI, predation, etc. I believe if the project we are working on, there will be more FML applications and will appear. Okay, the buffer is the demo. How we can make more clear the project then I will pass back to you Milak. All right. Thank you, Lane. Much appreciated. The demo was awesome. So closing out this presentation. In this session, we went over what federated learning or federated machine learning is at a very high level. And again, at an introductory high level, we saw some of the challenges that arise in high-scale federated learning operations. We saw some information on how to get started with FATE using KubeFate. And Lane presented us with a demo of the system using KubeFate and the FML site portal. And with this, we come to some resources that you can use to get reintroduced after this session acquainted with FATE and federated learning in general. FATE is an open-source project and we are inviting community participation. We would love to work with you on FATE. Love to have you as part of it and to see how it can change and adapt to your needs in your ecosystem and with your organization. You can always reach out to us. I'm your host, Neeta Jarora. And I was here with, I am here with Lane Pung, my colleague. We are both employees at VMware and we are actively involved in AI and ML projects in the organization and outside. So that's it. Hopefully, we get to see you in person sometime soon. Thank you.