 Shellyguide started. All right. All right. Good morning. I hope you are all excited for day three of DrupalCon. And I hope that I can answer this question for you today. Should you go serverless? So some topics. First, I'll introduce myself. I will talk about what is serverless. It's something, it's weird. Sounds weird, right? I'll show you some example architectures. And I will explain, like, can you use Drupal with serverless? And can you run Drupal serverless? And how you can get started yourself. So first I'll introduce myself. I'm Robert. I'm one of the technical directors in the Hilversum office. That's close to Amsterdam in the Netherlands. I'm a serverless enthusiast, which sometimes annoys my colleagues. But we do a lot of projects with it nowadays. For those who don't know MediaMonks, we're a global digital production agency. We're eight and a half thousand monks, as we call ourselves. And we have offices all over the world. And we are happy to work for all the biggest brands in the world as well. So what is serverless? Does anyone know? Is anyone using it already? No one. All right. Then you're in for a treat. So serverless is a weird name, right? And the creators of serverless framework, they say like, okay, at home you're using Wi-Fi. And it's basically the same with serverless. So obviously you're connected to Wi-Fi, but eventually there is a wire somewhere. So just like serverless, of course there is a server because stuff needs to be hosted. But what is it? Well, it's multiple things. The first, which is really important is there's no server for you to manage as a developer. So you don't have to worry about provisioning servers or patching servers. That's all taken care of. Also second, adaptive scaling. So if it's not in use, you're not paying for it. So that's the paper use, which is great. So it can scale down to zero. But if you get a lot of traffic suddenly, it will automatically scale up and then serve your customer's needs. So you don't pay for idle servers, which happens a lot. So your servers are constantly running. Maybe there's one visitor an hour. Maybe there's no visitor and you're still paying for it. And that's something that serverless can solve for you. For me as a developer, it means I can focus on code. So I don't have to worry about setting up all the infrastructure. I just upload some code and done. There's a lot of things you can have. Nowadays, serverless as a service. So you have storage. I think most of you probably use something like S3, which is also kind of a serverless tool since you don't have to worry about how it's stored. You just upload a file and you can just assume that it's taken care of. You can have the same with queues and databases, but there's also a thing called functions as a service. They call this fast and it's super interesting. This would be like the most basic example if you do this in Node.js on Amazon. I'm not sure if there are any Node.js developers around here. A few, all right, cool. The examples are very easy. So I assume that everyone with the PHP background can understand this. So this is basically, this is your blueprint for a serverless function. So there's no framework or such around it like Drupal. This is it. And you can invoke those functions synchronously like for instance an HTTP call. So in this example, this would be an API endpoint that you see very simple, very basic. You return a 200 okay. You can set some headers and you can set a body. This is the most basic like hello world example I can think of. There's also something which you can do is asynchronous invocation. And this is really interesting because for instance if you write a file to an S3 bucket, could be an image, could be a video, you could maybe do processing on it. So when the file is written, you trigger this function and you can figure out like what file is this, where's the bucket? And then you can decide like okay, this is a file I need to create thumbnails for. For instance. Then I think a question I usually get is what is the difference between containers and functions? And I also took VMs into account which is what I think most developers are using. So with a VM, a virtual machine, you have full control. You decide everything. You decide the OS, you decide your runtime. So if it's PHP or Node, you decide the framework, everything. It can do any type of workload since you manage it. And it's suitable for long running tasks because there's no time limit or anything. The downside of I think a VM is that you need to maintain it. You need to make sure that you have your patches of your OS and of your runtime. So if there's a new version of PHP or Node, you need to upgrade it. Also VMs are usually not very portable, which I think is a major downside. And you usually pay for idle. So if you have it running, you're constantly paying for 24-7. Containers is basically the same as a VM, but it's more portable. So it's something you can develop locally and you can use the same container if you want also on production. And obviously this is portable, so that's great. It's nice how you can use your container basically everywhere on every cloud provider or your local machine. The downside is still the maintenance. So if there's a PHP version upgrade, you'll need to do it yourself. And pay for idle, this applies most of the time. There are solutions where also containers are run only when there's traffic, but most of the time you still pay for idle. And then on the right, there's functions. They don't require maintenance because the cloud provider will take care of all the upgrades of the language runtime. So if there's a new version of PHP, your node, they will handle it for you. And you don't have to worry about it. And the beauty is you only pay for use, so you can with hobby projects like side projects. All right, that's quite a few. I have a lot of those too. And sometimes a couple of years ago it would, I would just not do hobby projects because, oh yeah, I need to pay another 10 bucks a month for this project. I would forget about it. And suddenly you have like five projects running, all costing 10 bucks a month for things that probably you don't even use. And with serverless, I can just create a function and if I forget about it, it doesn't cost me anything. And obviously out of scaling, it's great. It can scale, like I said, from zero to thousands in seconds. And I only have to worry about the code, which is the thing I enjoy most. There are a few downsides. Functions are usually not great for long running functions, or long running tasks. So most cloud providers have like a time limit of 15 minutes for a process. Which usually for, this is quite enough because you pay for use. So if you are doing 15 minutes of functions, then it's getting quite expensive. Also, I think a big downside, but you can decide it for yourself, is vendor locking. If you build something with serverless functions and you use services provided by a certain cloud provider, then it's hard to move away from it. So you're locked into their ecosystem. Similar to as you are probably locked into either Android or iOS, you've invested in what they offer and you hopefully like it, it's hard to move away. To me, cloud is not just someone else's computer. I hear that a lot. I don't think this is valid, because specifically with all those services and functions, you don't have to worry about how it runs or on how many machine it runs, or if maybe 10 machines are broken at once and you don't even know about it. They just take care of everything. So some examples, what can you actually do with this? I'm a big fan of serverless framework. So all the applications that I built are built with this tool. It has a free open source version, which you can download and use. It works with a lot of the large cloud providers. Personally, I have most experience with AWS. So the most examples you will see are specific to AWS. But the general concepts can be applied basically anywhere. So the most basic example would be a static site. So you generate a site locally on a build pipeline or on your machine and you just upload static HTML and static assets to a storage bucket, like S3 for instance. And in front of that, you have a CDN. Super cheap, you only again pay for use. There's no functions involved here. Super fast, super secure. Like you cannot hack a bunch of static HTML files. But if you want to introduce something like a search, you can either do it all yourself, but you can also use a service that provides this, like Algolia, which is a great service for search. I'm not sure if anyone used this. Oh, that's quite a few, nice. I'm a big fan of it and it scales well. It's super cheap. So it's a perfect addition to a static site. If you want to do something with e-commerce, you can also easily integrate something like Shopify, which has great JavaScript SDKs. If, for instance, there is something you have that requires backend functionality, like maybe a contact form or signup page or anything. There's on most cloud providers, also Cloudflare and also Netlify, they offer edge functions. So you can have that tiny function and you just upload it to their server and you can have a contact form. And all the rest is still static. So this is very great and cheap solution to host websites. This is a tool, an audio generator, tool that we built for a client a couple of years ago. And the flow was basically that, as a user, you would go through a journey and you would fill in all kinds of questions and at the end of the flow, we would generate your own customized audio file. So that would be an MP3 file. And what we did there, because this was going to be promoted by a top artist, I don't remember which one, but they would post it on Twitter so we could have like hundreds of thousands visitors a day. We weren't really sure. So we built this also the serverless way and what happened is that whenever we trigger the form that contains all your answers, we would send that to a Lambda function that would trigger the creation of an MP3 file. We measured how long that usually takes. So let's say that took like 20 seconds and then in the website on the front end, we would keep you busy for 20 seconds and afterwards we would just check like, hey, does this file exist now on the bucket? And if it didn't, we would just have like a load or a spinner and then we would try again after five seconds. And if the files there, we would just show you your download and you could download your own customized MP3 file. And we were easily able to do hundreds of thousands of visitors per day with this and we didn't have to worry about scaling at all because it was all taken care of. And also it was relatively cheap compared to doing this with VMs. A small tool I built for at home. I have a consumer line like everyone else I think. So that means that I don't have a static IP but I don't like to remember my IP. So I prefer to remember a VPN.myname.nl and I would like to connect to that VPN through the host name at all times instead of the IP. But what if my IP changes? I need to update it manually and usually those things happen when you're abroad or somewhere and you cannot figure out your home IP anymore. So I created a really tiny script that's running on my NAS. So local machine for storage and that triggers a function or an HTTP endpoint every five minutes and the Lambda function then has my home IP. I verify that against the DNS API and if it changed, I update it and then in five minutes my DNS records are updated and I can use my VPN again. So this is really tiny use case what you can do with Fervolus and this calls while I can even use a free tier for this but if I would actually pay for this this would cost like three cents a month which is super cheap for a nice tool that you build yourself. A little bit more complex. Not sure if anyone knows ID&T. They're event organizers. They organize events like DEF CON 1 sensation and ThunderDome. We did a project for them. This is super complex. I'm not going to explain everything here but basically they had an identity provider as a service and that provider was bought by one of the big tech companies and they got a message like, oh yeah from next year on we will triple your bill. So they would need to pay 200,000 euros a month or a year for having that service. So they came to us like, okay can we not build our own identity provider and make it cheaper and more flexible? That's what we did. It was their requirement finally enough to have everything Fervolus because they didn't want to have maintenance and running servers. So they came to me asking like, how can we build this? So I'm not going over all the details but you can see in the middle there's Lambda so that's the functions but all the other elements on this page are also Fervolus so it doesn't require any maintenance at all. What's nice is that this project now contains the credentials of 1.4 million people. So that's everyone going to their parties which is a lot of people and the most interesting is that instead of paying 200,000 euros a month a year they are now paying 3K a year. So that's 98% cost reduction and that's I think quite exciting that you with an architecture like this you can save yourself a lot of money and also have the benefit of scaling. For this client on the background you see the main stage of one of their events and when sales, ticket sales happening there are sometimes 100,000 people in the queue waiting to buy tickets. So then the system needs to scale up from zero to tons of invocations just in a few minutes and that's all arranged automatically. Something we also built two years ago for obviously for the COVID pandemic there was suddenly a need for our clients to have virtual events. So we created a platform that can manage this. It has all kind of interactive functionality like live polling which you see on the left and chats and live Q&A and such with video streaming. And obviously since I was involved in the project we went serverless. Again I'm not going over all the details if you are interested please find me afterwards. But something I wanted to highlight is that there's actually, I was actually cheating here since that part is not serverless and that part is not serverless because we were running an admin tool. It's called Sharnata, it's a bundle for Symphony Framework. I don't think, is anyone familiar with Sharnata admin? It's, oh there's a view view, nice. It's a pretty niche admin tool I would say for Symphony. It's not as good looking as Drupal but it's very flexible so we do like to work with it. And as I said that part is actually not serverless but what we were doing is we were using the CMS as a tool to control all the content, have workflows and such and then when you want to synchronize everything to production we would send everything to ElastiCache Redis on the, you can see that on the right side. So that's where all the APIs would get their content from. But what about Drupal, right? Where are Drupal gone? So I said that this is not serverless this part but that part could be Drupal. So that's one way how you could combine serverless with Drupal. You could use the power of Drupal to manage all the content, send it to another database that's optimized for use for scaling. Like I wouldn't recommend using MySQL directly with cloud functions but in this case we send everything to Redis and Redis is blazing fast so. But that also brings up the question like can it be done? Can you build PHP apps serverless? And it turns out, yes you can. There's a great tool called breath. For more info you can go to breath.sh and they are using a feature in AWS Lambda that where you can bring your own runtime. So what they built is they built a small layer around Lambda with PHP. So that means you can run PHP functions in AWS Lambda which is really nice. So you don't have to learn per se, you don't have to learn per se in OJS or some of the other languages that they natively support. But you can just use PHP and this is something I also use quite often myself for the hobby projects since I like JavaScript a lot but PHP is kind of my first love so I try to stick with that. And as you can see the example for if there's a file created on S3 you can also do that easily in PHP as well. But then the question like, can you do serverless Drupal? I spent a few days or I think maybe a full day on this and yes it is actually possible. I got the umami installation profile and I must say I'm very impressed by it. It's a lot of effort went into this. I'm not sure if anyone is here that wasn't involved in it but beautiful. But this is actually running fully serverless so it's running on AWS Lambda and it has cloud from CDN before it and also all the assets are stored on S3. So this doesn't cost anything if there's no visitors but you can see everything is working. I can use the search and I was looking at this example a lot of times. I'm not sure if anyone's getting hungry but I sure am. So I think that's pretty cool that you can actually run Drupal fully serverless. If it's a good idea, I don't think so. It's not really optimized for something like this and I don't think it has much to do with Drupal but more about PHP running in Lambda. And so while this was for me a nice experiment you can experiment with it yourself and you might find a good use case for it maybe for our project but I don't think it was a very good fit. Also fun fact is that Aurora on the right side that is a MySQL or Postgres compatible you can choose database and they also provide a serverless option. And so that means that the full stack including the database server is fully serverless. So you can configure after how much time of inactivity it goes down so skills to zero and I think that is maybe the biggest problem with this solution is that your first hit will take a very long time because it needs to spin up that database in the background and sometimes on your first hit it will not be able to connect because the server is not running yet. So if you really want to do this and you expect visitors like other visitors than yourself then you would need to kind of configure that you don't fully scale down to zero which I think defeats the purpose of serverless. So what is a better solution I think if you want to use Drupal? It could be using static site generator and a Drupal specific one is Tome. I'm not sure if I pronounced it correctly but this is a tool you can use to build static sites with Drupal and then the end result is again HTML files and static assets which you can put on a bucket or storage bucket or you can obviously use it with any of the other major static site generators. If you want to get started yourself that's not very difficult. Like I said, there's serverless framework which you can go download at serverless.com and it requires a note to work but this tool can deploy to you all the big cloud providers. I have a preference for AWS myself but also Google Cloud is a great option. They support PHP natively, while AWS does not. I think Azure doesn't support PHP natively as well but similar to Breath on AWS they have their own wrappers and Cloudflare the same. So they also created a wrapper that you can use PHP on their system. So like I said, Breath, it's super easy to get started. They provide all kind of examples. They also provide some examples on how to use Breath with Larevel and Symphony, a Symphony framework and obviously as Drupal uses a lot of the components that Symphony framework uses which is the HTTP foundation you can also use their tutorials for Symphony framework with Drupal. So that's something that I always try to explain others that since Drupal uses a lot of the components of Symphony framework or uses a lot of specific Symphony components that means that a lot of these tutorials also apply to Drupal. Sometimes you need to dig a little bit deeper but in the end it's the same libraries. Obviously there's mistakes you can make going serverless. I think the biggest mistake is that you are going into a console of Google Cloud or AWS and you manually start clicking things together. Like, oh, I want to have a database click. I want to have this click. It's easy to forget. It's you usually don't have the best security, best practice when you manually do things. It's hard to clean up and it's hard to duplicate your environment. So I highly recommend using infrastructure as code which is something that serverless framework does for you because the easy thing is you can just run serverless deploy and if you added something the framework will take care of it and if you want to remove your stack you just do serverless remove and then it will remove your stack. So that makes it very easy to spin up a dev environment or if you have a specific feature you want to test you can just easily spin up that feature, test it and then bring it down again. So it also keeps your AWS account clean for instance. Like I said, avoid connection-based services so tools like or databases like MySQL or Postgres those are awesome tools but usually it's not the best fit for serverless and the reason is that they use connections so they keep an open connection and if you make a mistake in not closing the connection properly the connection stays open on the database side and that can cause your database server to run out of connections and then it will just give you an error and say like hey there's no connections available and your site will be broken. So I would recommend going with more cloud native databases like on AWS like DynamoDB or use something like Redis. Be aware of the snowball effect. This is a specific serverless thing like I showed you the example that you can trigger function if you have a file uploaded. So I was playing around with that and I added the picture in my bucket. I created a resize version of it and it triggered again and again and again and again and it was instantly going into thousands of functions in parallel creating thumbnails of thumbnails. Luckily, they usually detect these kinds of things but of course I had to clean the mess and it probably caused me some money, I don't know. But be aware of this, that something you do asynchronously can trigger either the same function or something else. So make sure you take care of using the right parts as a filter and focus on asynchronous. So like of course an API is something synchronous but if you for instance are going to build a resizer tool, do that kind of thing in the background. So don't do it on demand, just do it in the background. When something is created, create the thumbnail. So to answer this question, should you go serverless? I think serverless is awesome but it's just not for every kind of project and for Drupal projects specifically, I'm not too sure. Like I said, I think it's a great combination if you use Drupal as a static site generator but I don't think that hosting Drupal itself on Lambda is a great idea. There's too many downside, it's not optimized for it. That was a lot of information, right? I hope I answered the question for you if you should go serverless. Are there any questions? One question in the back. Hey. So apologies if I missed this but you talk a bit more about how you manage the infrastructure in code. Is that entirely using serverless and you keep it all checked in and deployed using version control? Yeah, that is indeed all done for instance by serverless framework. So that can take care of all that. You can also write your own infrastructure that could be either in cloud formation specifically for AWS or something like Terraform which is more widely supported but that's indeed something you put in version control so that allows you to add new things to your stack in branches or yeah. Thank you. More questions please. I can ask one myself. Yeah, of course. So you run all this in AWS or other services, is there a possibility also to host serverless server? Very good question. Yes, you can also, I didn't mention that but you can also test this locally. There's a lot of tooling that you can test the invocations of these Lambda locally so you don't have to per se deploy it to AWS or Google. If you want to do something like this yourself there are tools for it, I believe it's called OpenStack and it allows you to do these things on your own server but personally I'm not a big fan of it since then you're again managing your own server which is for me defeating the purpose of going serverless because I don't want to deal with managing servers. Okay, thanks. This is not a question but I think as some CLE is one of the options when you run on AVS you can invoke in the local environment and then when you are ready in the local environment then you can deploy with some CLE to AVS. How did you say that was called? Some CLE, CLE. Oh, right, yeah. You mentioned that we have to pay for usage, right? But then there was an example that where there are hundreds and thousands of users coming in and logging in for the event site and that time it scales up. So how much of the cost difference it then creates saying that you pay for the usage but at that time let's say there are events coming up every month and that one time there are 100,000 users waiting to check in and it scales up. So how much are we saving on that then? It's really hard to say, it's a good question but overall using those peak moments is usually like an hour. So you have one hour of increased traffic and also I didn't want to go into too much detail but those Lambda functions are also reused. So when you have cold starts and warm starts and if the function isn't used in a while it's called a cold start similar to your PC if it's turned off you need to boot it. But once your PC or your Lambda is booted then it will process a lot quicker and that means that multiple requests are going through the same Lambda. So that's really speed optimized. So yes it might cost, I'm just saying numbers but it might cost you a dollar to have that peak sale but running the entire server for full day is already more expensive than that. Especially in a low balanced environment because it will detect like, hey there's multiple requests coming in, we need to scale up that takes a while and then you're also paying for more servers during that time frame. So eventually like I said, they were, they're only paying now three K a year for this solution which was measured after a year since we are running this in production for almost two years now. So it was a huge cost saver. So they do load testing as well at the same time? Yeah, we did a lot of load testing before. And also security testing, pen testing. Since we are storing a lot of personal information we need to make sure that everything is safe. So these are runs on servers and servers sometimes has big load because there might be other projects running. So do you see a big difference between how fast the Lambda function is? So let's say sometimes it takes two milliseconds and later it takes 20 milliseconds or are they quite consistent in speed? It's quite consistent. And you can, again, I didn't want to go too much detail but you can configure by usage of memory. It's like you can configure how much memory each function should use maximum and that scales with the number of CPUs. So if you have something really heavy you can also play around with those numbers. Like if I give it one gigabyte of memory it might take, let's say it might take two seconds but if I set it to two gigabyte of memory it might take 200 millisecond. And then while you pay more for the increased memory it's actually a lot faster. So overall you get a faster experience and while paying less that is like benchmarking you can do. But there's no interference between other clients or such or other users on the servers. And that's the beauty, they manage all of that. You don't have to worry about it. Okay, I'm gonna go with one of the online questions. Serverless functions can be cheap and scalable but have you found costs for adding functionality like Algolia and Redis also adds up? Yeah, that's a choice you need to make. Like you can host your own elastic search cluster which is super expensive. While you can also use something like Algolia which is especially for smaller sites a lot cheaper. We're using for the same client as the event. We're also using Algolia in an app and we have several thousand searches a month so that's not too much and I believe they pay two cents a month for it. So for two cents a month you cannot start your own cluster. Okay, you had a question? Yeah. Hey yeah, I was just wondering how are you dealing with error logging and stuff with all these little functions running around? Is that centralized logging or anything? Yeah, you can log everything to CloudWatch. So you can just do console log for instance in node and it would end up in CloudWatch logs with serverless framework. Also you can monitor logs of a specific function. So you can monitor that on your local machine and it would just synchronize with the actual logs. But also in CloudWatch you can search in logs or look into metrics and such. Oh nice, cool, thank you. There's another in the back. I mean, interested in the limitations of running Drupal in a serverless environment, we have a number of projects where Drupal and Tome is a really good fit but at the same time we need to be able to use the Drupal admin section on a relatively rare basis maybe once every six months or something like that. So the idea that you could use Drupal in a serverless export to Tome use CloudFront or Cloudflare, that seems like a really good fit. What kind of limitations did you hit and are there ways around them? I didn't actually think of that, but I like the idea that you can maybe only run your admin in a serverless way and then trigger a build. As far as I know, like it's more for smaller type of projects. So you would do that locally, like for blogs and such. I think that would be a good solution to do locally your admin. So it's not connected to the internet and then trigger your build and update it. But if you are using this for a client, let's say, I think it could actually be an option. I like it. We have five more minutes, yeah, gotta go around. Yeah, hi. Just follow up for your question that you can spin up the easy two instance, deploy the Drupal there and then when the admin stuff is done, shut down the easy two. Yeah, or then use platform message. Yeah, that's also a good option to, yeah. All right. If you want to help building Drupal, go to the contribution rooms and if you want to rate my session, please do. I value honest feedback and I hopefully can make improvements in the future and I want to thank you all for your attention. And if you have any questions or specific projects, find me after this.