 Today, I want to talk to you about serverless architecture, or general. So I had three examples of serverless architecture. The first one was GTScan, which I showed yesterday, which really is a serverless architecture that shows that even with lambda functions, you can put together a architecture that really has compute crunch behind it. The second one was around a use case that requires a continuously learning machine learning applications. So therefore, it is semi, the actual web application is serverless, but the background is server-based. But still, it's in AWS. So I thought this use case would be quite interesting, because it really needs a continuously learning and self-deploying mechanism. And the third use case was around showcasing that with lambda, you can not only have compute-intensive task, but also data-intensive tasks. And also, I wanted to introduce CSRO, which is the organization that I work for. So for the purposes of being agile, show of hands, who has been there with the presentation that I gave yesterday. So minimizing redundancy here. So with CSRO, CSRO is Australia's government research agency. And we have 5,000 staff, which is distributed all over Australia. And the unique thing about CSRO is that we are passionate about translating research into products that people can use in their everyday lives. And the most famous product is Wi-Fi, modern Wi-Fi that CSRO invented. But another interesting application is that they invented the first odorless insect repellent called aerosol. And the way they launched it, I found quite intriguing in that they were planning a big media splash by secretly spraying Queen Elizabeth with their insect control. And then when they walked through the gardens and the journalists noticed that there are no flies around her, they released, they was Aragard. And this is how it became a household name in Australia. True story. So with that, back to the slides. Fantastic. Which is perfect timing. So I'm going to get rid of the keyboard. Yep, sounds good. So it's not my fault if this looks horrible. Fantastic, thank you. So quick recap. CSRO already introduced the GT scan as the application that has the compute intensive task behind it. Crypto Breeder, which I should point out is not a CSRO project, is the one that needed continuous deployment of machine learning. And the FENGEN database has the big data requirements. OK, so skipping ahead. So this is, well, you've heard a couple of descriptions of what serverless architecture is. This is sort of how I think about it. Coming from the desktop compute, which has a singular CPU on it. The next iteration was high performance compute, which has many of these singular CPUs on it. Hadoop Spark, which is a unique thing about it in my mind, that it resolves the boundary between CPUs. So you can paralyze in a standardized fashion, and therefore it's truly data intensive task that you can crunch there. And serverless, I think, in my mind, is sort of the agile way of processing data. Because what it does is recruiting instantaneously the amount of CPUs that you actually need. So the beauty here is that you only pay for what you actually need. And this is exactly the use case. Oh, dear. So this is the exact use case of GT scan that we really needed. In that, we wanted to build an infrastructure, a tool, a bioinformatics tool that is on the web page, that is there persistently. We can, a researcher on the other side of the world can log in and use it, and it's always there. At the same time, it really needed a lot of compute behind it, so therefore the alternative would be to deploy instances that are quite beefy and therefore hugely expensive. So having them always on would have blown our budget way out of the water. So therefore, this was the only way of doing this. So the prerequisite for this was that we needed to chop it down into small tasks that needed to be embarrassingly parallel, so that are independent of each other and can be computed on little lambda, in our case, AWS lambda functions. The other thing is that the application case was unpredictable burstable, where there might be one researcher or there might be 100,000 researchers that access the web page at the same time. And we needed to cater for a scenario like that. And again, agility and scalability, this is where lambda actually shines. So the application case that I want to talk about is in genome engineering. So if you have not heard about genome engineering, it sort of, it really rings in the golden age of medicine in my mind, because with genome engineering, you can edit the genome of living organisms. And this has the potential to really make genetic diseases manageable. And how close we actually are to that was demonstrated by a publication that came out last year, which showed that you can edit out a genetic disease, hydrotropic cardiomyopathy in seven out of 10 embryos, which is fantastic news, because this disease you really don't want to have. And it's a single gene disease or a single mutation disease. So editing out sounds like a good idea. But the problem was that it only works in seven out of 10. And if you think of this being your unborn child, then three out of 10 failure rate is just unacceptably high. And we came in and wanted to resolve that issue by making it work the first time every time, especially critical for clinical implications like that. So with that, we were actually the first research workflow that used serverless architecture to really demonstrate that even though alamptas were originally designed for speech to text recognition, using it in a large scale workflow is actually producing a workflow that is worthy of research applications. So as such, we got a lot of attentions and GT scan is now used by two of the foremost research institutions in Australia. And sorry, what I'm referring to here is this is my architecture video. So if you want to know more about the in and outs of it, I encourage you to watch it. So again, GT scan, we think of as the search engine of the genome where researchers can type in what kind of gene they want to edit. And GT scan comes up with a ranked list of all the options of how the transcription mechanism can actually interact with the DNA. And this is exactly what I'm showing here. Every bar here is a location in the genome where the editing mechanism can bind to the DNA and edit the DNA. And in green are the ones that are highlighted that are actually high activity and in black next to it are the ones that do not have high activity. So in terms of research, you don't want to waste your precious material on sites that are not going to work. And them being so close next to each other, researchers don't really, if I just eyeballing it, don't know which ones are good and which ones are bad. And this is where machine learning has come in. So this is the architecture. This is a technical talk so I now get to talk about the actual architecture. Perfect. So this up here is the webpage that I just showed you where researchers can type in, I want to target this particular gene. This request then goes to an API gateway. And this is obviously very AWS centric. So API gateway. From there, a request is triggered to the DynamoDB database where it says, well, there was a researcher that I wanted to know about this particular gene. And with lambda functions being event-driven, they only can start from an event. And in this particular case, it's the event of depositing the search request into DynamoDB database. From there, we trigger another lambda function which then goes out to this particular area and searches for all the putative target sites that the machinery can edit with. So this you can just think of a regular expression searching a string in a text, in the longer text. So all those putative target sites get then deposited again into DynamoDB database, a new one. And from there, we trigger three other tasks. So one is the off-target search and the other ones are the on-target sensitivity prediction. So as I said, not every target site is created equal. Some of them can attract the machinery much better than others, which is the on-target prediction. Others have the problem that they're not unique in the genome. So with the genome being a three billion-letter-long string of just four letters, little motifs or little words can repeat itself at random. Therefore, you want to search a target site that is truly unique. So you add it only this gene and leave the one that is crucial for life over here unedited. So therefore, this is the off-target search where you need to search the whole genome for sites whether they occur uniquely or not. Now, as you've seen before, there are a lot of sites that get deposited into the DynamoDB database, and for every one of those, we need to search the whole genome. This is a truly beefy task that you needed to do. The other complication here is that this off-target search, searching the whole genome, the genome is large, as I said, so it's physically large as well in terms of file sizes. But Lambda has a restriction on the amount of runtime that it can compute and the amount of space or memory that it allows tasks to be. Therefore, we can't just trigger a whole genome search with one Lambda function that would have been too big. Therefore, we needed to break that task down into smaller sub-tasks in order to trigger many Lambda functions that would search their dedicated part of the genome. So while this seemed to be a huge headache at the beginning, it turned out to be a real blessing in the skies because obviously we had to think a little bit cleverly about our architecture to make it more parallel, which means faster. So the results from the off-target search and the on-target search goes into a third DynamoDB database, which then gets pulled by the API gateway and fed back to the user. So with that, with the API gateway, it basically provides a free interoperable way of pulling the data out or having a third application query the data. So in this particular case, it's the Jupyter Notebook that is querying the data. So I'm going to skip that demo because it's a different computer. But basically what it is probably shown here is that you can in a Jupyter Notebook, which obviously is for data science, the way to go, you know what, to quickly interact with your data, visualize your data and computers. So therefore you construct your query in Jupyter Notebook and then through a single API call that is triggered from within the Notebook, you can retrieve the data back from GT Scan. So this obviously is interoperable. So the results from GT Scan can be ported into the Jupyter Notebook where we can then visualize it or compute, do more statistics on it. So with GT Scan, we were very pleased with it, but obviously the whole purpose of having GT Scan as serverless application is that we save money. We make it as efficient as possible. Therefore finding the bottlenecks in the pipeline that is blowing up the runtime, which is directly out of our pockets, we wanted to find how to make it more efficient, how to reduce the cost, and I'm sure that resonates with a lot of people here. So the way to do that was to use Xray. So AWS Xray was released a couple of months ago, and basically what it is, is that you can enable this, you know, high level umbrella surveying system to record the actual runtime of each of your Lambda functions. And this is what we've done here. So these are all the Lambda functions that are involved in GT Scan too, and we record the runtime. So in the gray one, I show the housekeeping ones, the ones that are important in terms of the background. And in blue, I'm showing the on target sensitivity prediction, which obviously had massive issues in that it would take twice as long, three times as long as the other Lambda function that we had in there. So we wanted to kick this out. So we came up with a new architecture that rather than having, sorry, so this part is a blown up version of this one. So here we had two Lambda functions that were academic tools, a Perl script, I'm sure you know my pain. So Perl script at another function, and we were replacing that with a, of course, machine learning application in Python that is much faster. So these two Lambda functions, we replaced with this Lambda functions and then reduced the time frame that it took by 80%. So it's four times faster. We brought it down from two minutes to 30 seconds. And all of this we were able to record and document using X-ray. So the problem with X-ray though is quite, that is quite manual. You have to, yeah, sure. Yeah, so we rewrote the algorithm. I look, I don't want to diss any academics in bioinformatics, but bioinformatics has the objective of producing results, research results rather than writing beautiful code. So therefore the speed and the elegance of the algorithms, it falls to the wayside. So therefore us taking a look at what actually we needed to do and designing something that is more efficient with the same outcome, it was relatively easy to address that problem. Thanks, good question. So with X-ray, as nice as it is, it was a very manual task as we had to record every lambda function, we had to switch on X-ray for every lambda function. And then we had to manually record the runtime of every lambda function, change the architecture and do the same thing again. So while it was the only way of doing it, it was very labor intensive. And certainly you don't want to do that as part of a regular iteration of updating and upgrading your workflow. So this is where I was introduced to the hypothesis driven architecture by James Lewis. He's going to talk later today at four o'clock. So I heard his talk in Brisbane and I was really impressed by the way that he suggested we should think about architecture as sort of an evolutionary process of making little small changes to it and recording the outcome objectively and keep on iterating until we have a better outcome. So therefore this together with Keith's talk today, which was around architecture as text, I thought is the perfect way of actually evolving architecture. So you start off with your cloud-based architecture as text. In our case, there was a SAM template, cloud formation template. You evolve by, for example, replacing two lambda functions with one lambda function and then you record. So as I said before, we were doing x-ray manually, labor intensive. So here we partnered with a startup in Israel called Epsilon who specialized on measuring serverless architectures. So what they've done, they figured out a way of how to inject something into the lambda function to automatically record the architecture that you have as well as the performance of that architecture. So therefore this evolving and recording the performance is now automated, which of course means we can evaluate many more steps and don't have to listen to our God to our hunters of how to evolve the architecture. We can actually test a couple of them seamlessly. So therefore I now think of this as sort of a loop that where everything is sort of automated except you need to come up with the way of evolving your code for yourself, obviously. But everything else is sort of taken care of and I'm really, really excited to see where we can take this. Oh dear. Okay, crypto breeder. Oh, and the atmosphere is good for that. So crypto breeder is again a task that is not sponsored and is not supported by CSRO. This is my own hobby project. So a little bit of background. How many of you are familiar with cryptocurrency? Yeah, Ethereum sort of smart contracts on cryptocurrency. This is a game, a toy game called CryptoKitties where an Ethereum contract, a token is a kitty and you can buy it and also you can breed it. So this is what I'm showing here. So two CryptoKitties produce offspring that have certain attributes, which they call catributes. So certain catributes are more valuable than others. And when you want to breed and sell your cats, you want to optimize that. And the way these catributes are determined is by the genome of the cat and here you see probably the link. So the genome of the cat will produce a phenotype or the catributes which then determines the value of your cat. So therefore I thought this is a perfect Christmas project for me to put together a serverless architecture on predicting the catributes from the genome of the kitties that they have. And this is the resulting webpage of CryptoBreader where you can type in two kitty IDs and it will return the likely catributes with the probability associated with it. But the problem is that these catributes there, now they're throwing out catributes on a regular basis. So they're new emerging catributes every day. Therefore if I train my algorithm to predict one catribute today, it will be outdated tomorrow. So therefore I needed to have something that automatically gets the newest catribute, learns how to predict those catributes, deploys that back to the webpage so the user can stay on top of that process. So therefore the solution was that I needed to have a continuously learning model. And this is the architecture. Yay. So again it's a webpage. Again it has an API gateway because there's sort of how serverless architectures with a web service work. From here as I said, there will be two catributes or two kitty IDs that are provided from the user. And in the first lambda function, I'm sucking down the actual genomes of those kitties which get then deposited into a DynamoDB database which triggers the next lambda function that predicts from those genomes what the offspring will likely have as catributes which then gets deposited into a DynamoDB database which then gets queried by the API gateway. So pretty straightforward. The only problem is, as I said, this needs to be updated on a regular basis and this is what I'm doing here in this maintenance layer. So with the maintenance layer, I have a tiny EC2 instance. So it's a T2 micro, just virtually free. And it every day sucks down the latest kitties that are produced and gets their genomes as well. Which then gets deposited into a flat file on S3 from where I have a much beefier EC2 instance sucked down that information, trained the machine learning model, pickled the machine learning model and then deposited back to an S3 instance, sorry, S3 bucket which then feeds the lambda function. Now, as I said, this is a beefier EC2 instance that I don't want to have continuously running. So this is another layer here that I have coldwatch which periodically starts this EC2 instance. Upon start, it will suck down the latest information from S3, would train the model and does the whole pickling and deploying. And then after an hour, it will stop that instance. So with that, all the startups here in the room, how much do you think this has cost me so far? Like developing it and deploying it. It's now active for two months. How much do you think it amounts to? And the $10? Okay, I also had to buy the main. So this blew up my budget. So so far I paid $24. So the uptake is not phenomenal. I mean, it's a toy project, but it's probably roughly on par with an early startup. Is there only 160 cold start for our model? That's right, that's right. And we actually have the same problem with GT scan because it's not continuously used. But the cold start time is actually not that horrible. Yes, it's a couple of seconds and probably for other application that is way too long. But in this particular case, in a startup environment, I think that's acceptable. And if we get to a point where we don't have a cold start anymore, then that's a success. Yeah. Yeah, exactly, exactly. So the actual lambda functions, they are really cheap. Like here you see there's lambda listed here and the cost is $0. So even if I do that then every hour in order to keep it alive, I think that would not cost too much. That's right. So I think the big take home message from here is that this is really cheap. Like it's really, really cheap in terms of a startup, getting something started, getting a minimal viable product out there, it costs you nothing. And it's easy to talk about that if you have a tangible thing that you can demonstrate. So with that, let's go to the real world, back to the real world, which is the Denfen database. So here the task was, or the constraint or what I wanted to show you here is that lambda can be used on massive amounts of data. So as I said yesterday, genomics is coming at a fast pace. By 2025, 50% of the world's population will have been sequenced, which means the amount of data that will be produced per year is going to skyrocket. Actually, so it will be larger than astronomy, Twitter and YouTube combined, the amount of data that will be produced in genomics alone. So therefore, this will be around 20 exabytes per year, which is mind blowing. So with that, what we want to build is a phenotype, genotype phenotype database where a patient comes into the doctor and says, this is my genome, this is my medical record, what will be the recommendations for me? What should I do, what lifestyle changes should I make in order to stay healthy? Which means you need to query the phenome database in order to find out, for your heart rates of, for example, these are the recommendations, which is a tiny data set, but you also have to query the recommendations for your genome, which is massive. So the problem here is the data is so large and the solution is that we did not want to have a server-based solution or a server-based database that holds all that genomic information. We did not want to have Redshift because Redshift would have been very pricey for those amounts of data. So therefore our solution was an Athena-based query engine. So this is the architecture. Again, webpage, API gateway. This is the Lambda function that queries the genome. This is the Lambda function that queries the phenome. So the phenome is an SQL-based server-based results. We're going to address that eventually, but this is what it is for the time being. The important thing here or what I wanted to show you is the one up here, where we prepare the SQL query for Athena, and then Athena goes into S3 buckets and queries this massive flat file, if you want, of genomic data. So the data is encoded in parquet format, which is a column-based indexing, if you want, of basically a flat file. And the beauty here is that with research organization, not necessarily wanting to share their data, but they do want to share results, this is the way to do it, because Athena can query two silos, if you want, independently and join then only the higher level data to preserve privacy and preserve research sovereignty, if you want. So the three things to remember from this talk is that distributed architecture, serverless architecture, can cater for a really wide range of applications. So I talked about a compute-intensive task, which was GT scan. I talked about a continuously learning infrastructure, which was the crypto breeder example, which is not funded by CSR, and I talked about data-intensive task, which is a phenotype database. The nice thing about this is that interobility, it comes free because of the API gateway that you need to implement anyway. So therefore, this really supports evidence-based decision-making through a Jupyter notebook, for example, where you can interact with the data and really quantify the results. And optimization is currently still complex, like this x-ray, which is nice and it's a good start, but it's still manual or labor-intensive to actually get the results down and quantify it. But there are many startups that are actually addressing this issue, so definitely watch this space. With that, thank you very much for listening.