 Hi everyone, so my name is Benjamin Feng, I'm an Associate Solutions Architect at AWS and I'm part of the AI-ML technical field committee. So I know what some of you guys might be thinking like that guy's the technical guy from AWS, let's get him, let's ask him all the questions. I'm not here for that today, but definitely contact your respective essays or account managers that are in charge of your AWS account. So also for some context, I was told that this was going to be a 10 to 20 people AI-ML meetup interest group. So you can imagine my shock when I saw like 100 of you guys, so while you're young, just try everything, just make mistakes, so that's what I'm doing here today. So of course some of you might also be thinking there's a young guy here recently joined AWS not more than a year, what can he tell a room full of professionals that have been doing this for years, yeah. And actually I'm not here to tell you how to run your business, but I'm here to introduce a new topic that you might be able to take advantage of in your business. So I'm really passionate about this topic and I've been researching this topic for about one and a half years now and collaborating also with some of my AWS peers in Seattle. And I think it's really exciting this new technology that can unlock a lot of possibilities for your business. And this will be some sort of surface level technical introduction to federated machine learning. And I feel it's quite nascent, so for some of the questions that you might have, actually maybe nobody has an answer for it, so please do be patient if possible. And of course this will be slightly technical on the machine learning side. So I hope everyone here at least has some experience with ML. And the presentation was also built on top of the research of some of my colleagues overseas, Kristoff and Chong Zhang. So thank you for that. So without further ado, let me just get into it. The first one will be the agenda. So what is federated learning? Why is FL growing in importance? And of course how to build it on AWS. So first of all, what is federated learning? So federated learning is a distributed ML approach that trains ML models on distributed data sets. And it's also commonly known in the field as privacy-preserving machine learning. So why I was personally very interested in this topic, I'm sorry my slides has been switching. Okay, so why I was personally very interested in this topic was the possibility of it solving a lot of healthcare problems. And yeah, so let me just give an introduction of what that might look like, what federated machine learning might look like for you. So in healthcare data is quite really, really sensitive. And you can technically reconstruct someone's entire face from a couple of MRI or X-ray scans. And I mean, there's no wonder why hospitals are so uptight on data privacy. And federated learning actually has been touted as a way to cross these barriers. And this is a demo on how. So over here we have long MRI scans, I believe. Yeah, so it starts with a global model. Let's just say your company is the server with a global model. And there's a small clinic, there's a hospital and a research center. As you can see, these different clinics and hospitals and research centers have different varying amounts of data. And the research center obviously a petabyte scale of data and they've been using it to train their own models. And so with this federated learning, the next thing would be to distribute this global model that you have. It might be a model that's already been trained on detecting MRI scans, cancer and MRI scans. And what happens is that you distribute this to everybody. So everybody has a machine learning model to train on and to utilize their data for. And then the next one is that each endpoint will incrementally train a global model based on their local training data. So each of the models that we send will be trained on the small clinic, hospital and research center. And these models will be adapted to the data in their respective areas. So afterwards, we upload actually not the data, but actually we upload the model, the weights and other artifacts that the model might have to the centralized model which is your company. So what the company actually does with this model is that we aggregate it. So aggregating the sense we combine the weights, there's several ways of doing it. There's a lot of research coming out on how best to aggregate your models. But what happens is that this actually gives a chance for a model to have seen all the different types of data. So not just the gigabytes and the clinics, the terabytes in hospitals, but actually the petabytes of data available everywhere. And it's possible that this small clinic is somewhere in another country. And for example, dermatology data, people with different skin colors. This actually will help your model adapt and generalize better to these big different data sets. So then we send these models to the small clinics, hospitals and research centers and continue training it on new data or just to improve the accuracy until we see an improvement in test accuracy. So that is a gist of what federated learning is and how it works. And of course, like I'm mentioning, for example, a hospital might have an image diagnosis for dermatology on skin types that are mostly on the light side. You might not have the same accuracy if you had darker skin and going and you went to this specific hospital. The model might not be able to generalize to your skin type. So this is one of the problems that federated learning can actually help with your model to generalize. So the power of federated learning is beyond just privacy. I think that federated learning has the ability to unlock hard to reach data for machine learning training. And of course, in hospitals, some of you might be wondering, why do I have to wait seven days for my heart scan to return any result? Is it because of shortage of doctors or lack of technology? And I believe that we already have the tools to equip doctors with such assistive tools and increase efficiency in healthcare. But these efforts are actually unfortunately covered in a lot of red tape. And in fact, it should be covered in red tape, right? After all, there are lives at stake in hospitals and whatnot. So federated learning presents that chance to cross that red tape and be compliant with laws in order to build models that are at the level of the biggest companies. Why do all the big companies with the richest companies have the money to buy data sets, right? Why not everybody else build and chip in in order to build something like that? So I'm going to go a little bit more in depth with what that might look like for the technical folks. So there's different types of flavors of federated learning. So you might be asking, you know, not all data sets are the same. This data set in the clinic might be different from data set in a research center. So there's two types of federated learning as well. There's being research as of now. There's horizontal federated learning, which is where the data is the same. We only take the columns, sorry, yeah, the features that are the same across different data sets. Bank 1, Bank 2 has the same type of data set, sorry, features for the data. And of course, there's also vertical federated learning, which is where the data sets are not similar. So instead of going just by features, they go by the person. So I'm sure a lot of us have data set, sorry, more than one bank account in UOB and DBS. So what is, what federated learning, vertical federated learning can do is identify the person instead of the features of that person. So it can actually expand the data set. Maybe your first bank has only three features and the other bank has five. Federated learning can extend this to five features for you specifically and of course, have more insights to perform predictions or whatnot, okay? So this is really, really new. So there's a lot of research online that's being done. So feel free to Google and reach out if you need to. And of course, there's other ways of federated learning where the server can be positioned. So quite commonly, the federated learning server is at the center and federated learning clients will send the weights to this centralized server. But it's also a peer-to-peer. I'm sure a lot of you get very excited when you hear that word, peer-to-peer. This is like a mesh connected clients where we can keep updating our models asynchronously or synchronously based on the people that are around us. And this sort of democratizes that in a sense. But of course, there's other ways that we can, other benefits of this as well. So lastly, when do we aggregate a federated learning server? There's synchronous and asynchronous. Definitely, you can aggregate a set of local models into a global model. Or you can do it asynchronously. This is where you update the global model once a local model is received. So I think this hits home a little bit. I took this from Apple. I think that's allowed. So actually, I think your phone is already performing federated learning in a sense. Like Apple has this face detection and of course speech recognition services that they provide. And what they do is that they probably, I'm a disclaimer, I don't know, but they probably take the data from your phone, train it on their local models and then just send these weights and parameters to their centralized place where they aggregate it. And then of course, that's why everybody can have face ID. Everybody can have a facial detection. Everybody now you can separate the face from your photo and stuff like that. I believe that's what a lot of phones actually do. So you may never know federated learning is closer than you think. Okay, so why is federated learning important? I'm going to go through really quickly. So as you can see with the example of mobile phones, we enable the possibility of ML cross devices and of course cross silos. So being able to access silos of data sets and crossing different organizations as well. So of course, there's a possibility of reducing costs instead of buying data. Now there's a lower barrier of entry to train really, really great models. And of course, there's also a flexibility in the form of being able to perform machine learning operations on-prem or on-cloud, adapting to the multitudes of data sets with a vertical federated learning. So last one I want to mention is why I'm so passionate about it, which is I hope that federated learning can actually promote healthcare equity. Where the smallest players and smallest clinics can benefit from the capabilities and data sets that these big research centers and organizations might have. So rural clinic somewhere in a certain country can benefit and have the same type of accuracy as the model that are possessed by these big research centers. Okay, so another example of course is the bank example that I gave. There's banks with different type of data sets and of course they can work together to have a fraud model that can really detect across banks. If a bad actor were to port from bank one to bank five, bank five would also be able to detect it because it has seen those data sets and these type of examples from bank one without even needing to see their data. So there's a lot of places where you can use federated learning. This is just a small subset of it. And of course with every new technology there's a lot of challenges and there's few uncovered but not fully solved challenges. With this seemingly, sorry, I would like to add also that I mentioned how big a part of FL is. It's also about aggregation of weights. I think there's a lot of research coming out with aggregation methods. Concatenating the weights most likely won't work. Averaging out might work a little better. And I recently saw research on layer by layer aggregation, which is really complicated, a lot of math involved. So if you're interested just find it online. And I might not be able to answer your questions really well on this. So a lot of problems feel free to maybe solve some of them with you and your companies. So also another problem is that who is going to be the host? I think this is a big problem when governments are asking like, why should you be the one with all the power? Why not me? Why not them? So a lot of problems and a lot of related that you still have to work through in order to implement something like that. So really, really quick demo how to implement this on AWS. So let me just press play. So as you can see here, there's a top notebook and a bottom notebook. The top one I believe is on-prem. So this is where they hold some of their machine learning models. And the other one is on the cloud. The one below is on the cloud. So this also holds another machine learning model. So the one on top on-prem has different data sets. And the one below has different data sets as well. So what it's doing here is actually performing federated learning, like I mentioned. What we're going to do on the cloud is that we're going to give a global model to these two notebooks in order to perform training on their respective data sets. Let me just skip ahead a little bit if you don't mind. Yeah, so when you start AWS Step Functions, I personally love AWS Step Functions. It automatically performs this training. It sends a signal to these notebooks. Here's the model start training. And then the model will train on their data sets. And of course, it also goes into a loop as you can see somewhere over here. Yeah, it goes into a loop. It keeps going into a loop because if a model trains on a data set and it's not improving on test accuracy, why bother at all, right? So this Step Functions actually helps you automate this. You automate a lot of things, but it automates this and it checks whether the test accuracy improves at all. So as you can see, the model is being trained above. It's checking whether the accuracy is good, whether it's bad, whether it's improving at all. So with AWS, it's definitely possible. These two are actually on SageMaker itself. So that's where the AWS ML is involved. Okay, sorry. So yeah, let me just skip ahead. The gist is that the training will finish. It says that, okay, yeah, the accuracy has indeed improved and research has shown that federated learning actually improves accuracy quite often. That's a lot of proven results. And of course, yeah, this is something that you can do on AWS with your notebooks and you can go really in depth on how your weights are looking and what actually went into that training. Okay, and it's a little bit complicated, but how to build it on AWS. So like I said, there's an on-premise one and there's an AWS one. Whoops. Yeah, there's an AWS one. And of course DynamoDB will store some of the metadata that you need and the S3 buckets will hold the ML models. I'm not sure if someone's clicking this for me, but I'm just not sure what's going on. But anyway, yeah, so the S3 bucket will store the weights and the models and what not. Tech nowadays, yeah. So AWS Lambda will start the sort of start the training process and start the step functions. And of course, Amazon SNS and SQS will let the on-premises servers know to better to start retraining, whether to signal a lot of different steps that you want to do and maybe even aggregate a model in a sense. Yeah. So yeah, despite the challenges in federated learning, we do see tremendous potential. We being the people involved in AWS in this community that's trying to build up something interesting for healthcare. And of course, there's a lot of places that you can use it. And there's a lot of regulatory pressure on ML, ethical ML and stuff like that. So federated learning could be a possibility where we can go beyond these regulations. And of course, there's a lot of promising signals that are being showed with federated learning. So I hope today this introduction, everyone has actually a better idea of what it might be. Maybe some of you have never heard it before. Great. I hope you guys understand federated learning now. But if you're an expert already, yeah, that's great. Yeah, I hope we can solve more problems in the future. So yeah, thank you so much for that. I'm actually here to also talk about something else. So I hope you can hold up the questions for now. Feel free to talk to me afterwards. I'm already about 16 minutes. So I'm going to really quickly go to my other presentation that I am obliged to give. So this thing called AI ML Reactor that AWS is hosting. It starts on 10 October and ends in about a one month program, I believe. So what it is is that it's a five-week hands-on practical program enabling startups to implement and deploy their ML solution. And of course, we had it in Pakistan already and we're trying to take it across ASEAN. And of course, there's a lot of exclusive workshops, there's mentorship with AI ML specialists from AWS, cash prices, AWS credits, and exclusive SWACs that you can get from there. But of course, that's the least of it. And of course, the target audience is for people that are AI ML first startups. And startups has also reached a stage where AI ML is in their roadmap to drive sales conversions and their business goals. And of course, bring in net new startups that are not on our view as of now. So the five-week program will provide you with these benefits as hands-on guidance as tech mentorship, business mentorship, demo days, cash prices, visibility, stuff like that. And of course, this is a more detailed description of what the timeline might look like. Feel free to take a picture. There's a validation workshops, they'll teach you about all the AI ML solutions on AWS itself. And of course, you can build your MVP together with hands-on guidance from our AI ML experts, one-to-one tech mentorships, one-to-one business mentorships, and of course, a final demo day. So this is done in collaboration with Epiphany, which is, I believe, the business side of things. Okay. And of course, we have a lot of AWS startup ecosystem partners and things like that. There's more people that's coming on board. And of course, we have our own people like Udo, which is a really, really... Oh, sorry, I thought my time was up. So Udo is an AI ML expert in AWS. So there's definitely really, really great people that you'll meet in this bootcamp. So applications are actually open now. Please apply by 5th October. I'm going to leave this QR code on the screen just a little bit for everybody to take out their phones and just take a look at the program if you want to. And of course, thank you. That's my presentation. And maybe I can get one or two questions. Yeah, thank you. Oh, no questions? Oh, please. Can I leave this here? No, no, one guy here. Oh, yeah, yeah. Sure. In the demo that you showed for a step-up response to world property, the client is sitting to be on at the same time or the server is going to be on at the same time? Which level is that you're talking about? So earlier you showed the step function. Yes. And for that to perform flawlessly, do you need to have those two servers on at the same time, the servers that aren't connected to the servers? Yeah. The reason being is that those step functions actually, oh, you could actually have a function that will activate these two servers and maybe switch on the servers when you're not using them and only switch it on and spin up those instances to train those models when you need to. Otherwise, I don't believe that you need to switch on those servers before the step function can work properly. So you need both of them. You cannot check. Oh, yeah, yeah, good question. Yeah, there's asynchronous and synchronous. So you can train one first and of course asynchronously train the other one and propagate the weights and whatnot. So both are possible, but the demo that you showed was the synchronous? Yeah, synchronous, yeah. The demo that I showed was synchronous, yeah. That's a good question, yeah. Thanks for asking that. Any one more question? Yeah, over there? So, you said that you can use multiple? The global model. So it seems that if you just download an open source model online, you're going to have to trade it to make something from scratch. You can just use federated learning and new model. I think you know how the global model that you can download for free is quite good already. Federated learning is a chance to expand that. And for some startups that might not have the resources to reach a lot of data sets, federated learning will post an advantage in that sense to be able to train on new data sets. Because these models might have been like CIFAR 10, like many years ago, not even CIFAR 10, like a Cocoa data set. It's so long ago. It's many years ago that there are new data sets, new types of cars and whatnot. So instead of buying those data sets, you can possibly lease those data sets by pushing a model and just getting back the weights from whoever you're leasing it from. So that is the opportunity in that sense for that side, yeah. Okay, thanks so much. Yeah, have a good day, guys.