 All right, let's get going. It's quarter past 10. Welcome, and thank you for coming to my talk here about building your own MailChimp in AWS with simple email service. My name is Dewa DeBoer, and I am a Drupal developer working at the Sparks Interactive, working at Sparks Interactive in our Auckland office, also one of the maintainers of Sector, which is the distribution that we build and support here in New Zealand. You can also find me in the Drupal.org issue queue and in the Drupal Slack channel under my real name. Nice and easy. 10 years ago, I was at my first Drupal event. That was DrupalCon Sydney in 2013. It's almost exactly a year ago. I'm right in the middle there in focus, listening to Dries deliver his keynote. My dad's there one seat closer to the camera. He was one of the early adopters of Drupal in New Zealand. He's not here today, though. He's climbing a mountain somewhere because it's beautiful weather. I've been doing Drupal for over 12 years, and I probably have what is probably a rare privilege of being a second generation Drupal developer. And as a teenager, learning Drupal 6, it's so great to still be here working with this awesome software after nearly half my life. Then in the crowd, and now I'm on the stage. And I also want to give credit to the photographer there, Tim Miller, whose Flickr account is still running after 10 years. So you can find the photos there. And the Flickr website is still up and running too. Props to their infrastructure team. So let's talk about building your own MailChimp in Amazon SES. But you probably came to this talk with one burning question in your mind. Why on earth would anyone want to build their own MailChimp? Why would you not use existing third party tools, like MailChimp? And why would you opt to build something new? That's also the same question that I started this project with. And to answer this question, we'll have to talk about the client who asked for it and why. So the client is Toi Tu Tefenua, Land Information New Zealand, or linds.gov.nz. They had a Drupal 7 website, and they wanted a Drupal 9 rebuild, had a mailing list, and also a statutory obligation to send out an email to that list every fortnight on a Friday. And they also have very hefty security and privacy requirements to make sure that everything could run on their AWS stack, and they wanted to avoid as many third party integrations as possible. I told my producer that we could probably build the MailChimp integration in maybe a day, which is a bit of an under-exaggeration, and this Amazon simple email service might take like two weeks in comparison. Also a bit of an under-exaggeration, but I did really think it would be, take 10 times as long, and we developers are notorious for underestimating anything, and of course it was a stab in the dark. But the client came back and they definitely said they wanted to build our own MailChimp as I put it to them. So we started to look at their existing process. They send out these notices to mariners to inform them of changes to nautical charts that might be relevant to them. As you imagine, this is very important information because bad things can happen if things go wrong. Now, I found out in early on in this process that a staff member had to set through a Drupal 7 batch process to send out these emails. What follows here is an artist's impression of what MailOut Day would have looked like in the toy two to Fenwell office, which is exactly what went through my head when this process was explained to me. You're sitting there at home, you're waiting for the batch process to finish, you're paying careful attention so you don't accidentally close the tab. You hope there's no technical issues, no power cuts. Suddenly your house catches fire, but you got to stay at your desk because you don't want to go to jail because those emails have to go out. And then eventually your router burns out as well, literally, you're now definitely going to jail and you've lost your house, so. It was very sad, it's always sad when your Drupal developer to come across people who hate their Drupal websites, but it was quite a new thing for me to come across a client who hated their website because it might send them to jail. So we needed a solution that would make these people love Drupal again. So it was a simple plan here, staying out of jail, a more robust process from the existing process that you can see on the left here and what we came up with, what we wanted to do, make sure it looked like after. So the emails were sent out by Drupal, they were generated by Drupal based on a MailChimp list. So everything was filtered through the Drupal 7 Batch process based on a complex set of nodes, changes to nodes and files on the website and their requirement was everything had to be on AWS. All right, that's fine, I said we can look into that. Secondly, privacy and security requirements had changed massively since the days of the Drupal 7 Carboy websites. You don't want to be in the newspaper, front page news, six o'clock news as the next government data breach. So they said, nothing can be stored on this Drupal website that doesn't have to be there. That was one of their requirements. The Drupal website as much as possible shouldn't know what anything about these people were sending emails to. So I proposed a solution that would only require the human being to push one button, the big red button, and then everything else would run smoothly within AWS's uptime guarantees. So it's much better way for the staff to avoid jail time too because you could blame the internal infrastructure team or the AWS employees. And so as long as the AWS data centers keep running, the emails will go out. And even if the data centers go down, everything's properly queued within Drupal, so when they come back up, they continue. As everyone knows, if a Drupal 7 batch queue process stops halfway through, you can't pick that up again and now you don't know who you sent your emails to. So when everything's queued, we know that everyone is definitely gonna get an email and they're only gonna get it once. And they also had this person sitting there for, I was told over an hour to send out these emails. I promised we could do it very quickly. We did not as quick, in the end it wasn't as quick as I had hoped because of API rate limit issues, but we were down to about 10 minutes to 20 minutes or so, which they're very happy with. So if we've got time to go over eight points very, very quickly today, I'm gonna go through all of these moving parts that had to come together for this to work. And of course, we wanna look at what SES is and isn't, what we get out of Drupal Contrib already. There is a module, how we capture subscribers and why we use decoupled web forms, how we actually send them mail outs and use workflows to manage those and then queue them, how we manage security in the cloud and some limitations that I ran into, some gripes that I have with Amazon and of course the ultimate question for you, should you do this yourself? And I'll talk a bit about Drupal Contributions throughout the talk and specifically at the end because being a government client, they thought it would be ideal to do some contribution back to make sure that pieces of this could be reused for other departments or other clients. So first, what is SES and what isn't it? So basically it's an SMTP mail server. If you look at the Amazon website, they help you maintain high delivery, help you maintain a good reputation and important for them, it plugs into the all-consuming AWS ecosystem. So if you are running your infrastructure on AWS and your site is sending emails, I highly recommend just using SES as an SMTP server because email delivery is very, very hard these days. Anyone who's tried to have a Drupal website just send emails half the time, they don't get there anymore. Gmail especially very, very strict with email deliverability and of course on the AWS free tier, you also get 62,000 out-bar messengers per month for free, free in AWS terms because obviously you're paying for your EC2 instance. If you look at this page here, you look at their promotional pages, their list of features, there's actually nothing in here about list and subscriber management. So we'll carry on. We'll look at the UI in the back end of Amazon. Here's an example of a demo account here. UI is very, very limited. You'll see that there's nothing in the sidebar there if you can read it to do with lists and subscribers. There's a lot of talk in there about managing bounces, managing suppression lists. Lots of very powerful options available for maintaining deliverability, but there's no UI there to manage templates even. There's no UI to manage subscribers. You can't even see if the list exists or who subscribed to it. So you're gonna have to use the API to do that. If you have a sandbox account, you can only send 100 per day, which is okay if you're doing some simple testing, but once you've got a production account, which is easy to apply for, they just wanna make sure that people aren't abusing the sandbox accounts out of the box. You gotta go through a little bit of a process to get 50,000 emails per day, and it says here that we can send 14 emails per second, but due to, again, API rate limit issues that's kind of unrealistic unless you're bulk sending the same type of email, but if you're customizing the variables, you'll have to be sending a little bit more slowly than that. Now, how far can we get with Contra? Obviously, we need to plug this into Drupal, and there is a module for that. You can see if you can read the advantages in the list here, basically it says you don't need to set up your own SMTP server. There's very little configuration, you just plug in your API, your AWS credentials, and then the module is gonna send all of your Drupal emails via SES. Again, there's no mention here of list, list, and subscriber management, and this is because these are actually new features that were only available from late 2021, and only available via the SES client V2 in their PHP SDK. So we talked with the developer of the module, we wrote the patches and tests to upgrade the module to SES V2, so that we could actually call the functions that we needed without having to reimplement the API ourselves. So that latest version you see at the bottom, 3.0.1, ships with all of the changes that we helped to contribute there, so thanks to the maintainer, Davis Ben, who let us rewrite his module, basically. So configuring the module is super easy, set up your API keys in AWS, you're ready to go, and that's the simple settings there. You've got verified identities that you need to set at the top as well, so you have to verify the identity of the email address that's sending emails from the Drupal website, so you could see it back here, sending it from dwordspikesinteractive.co.nz, I need to make sure that's in the verified list there, and if you are in sandbox mode, you also need to verify your recipients as well, so you can see I've got my producer Jack in there, so we can spam him with emails if we need to. If you're on Route 53 as well, you can also order verified domains, so you can send from any email address out of domain, you can see in the corner there, it says not verified for the spikesinteractive.co.nz domain, again, super easy if you're on Route 53, if you're not running Route 53, things get more difficult, so all of this is really built to tie into the AWS ecosystem, and of course, up the top there, you can see statistics tab and test sending tab, self-explanatory, and the subscribers tab at the end there, that doesn't ship with the module, that's something we're gonna look at in a minute, and something that we had to add in. So there's no UI to get subscribers into SES, so we're gonna need some web forms, and Amazon does give you automated links and forms for updating subscriptions and unsubscribing, that don't give you any help actually getting a subscriber into your Amazon list, so we use Core's REST module, and some custom API endpoints with web forms, and we'll talk about why has he used these decoupled web forms in a minute, but first, the web form handler on the config screen of the web form, so people fill out the form, hit save, we wanna make sure that submission gets sent into AWS, so you, very simple web form handler, you select the list that you want to send things to, you can choose what submission data you want to go into, the attribute data of the subscriber, and you're basically good to go in the background, I'll give you a quick code example of what that looks like inside the web form handler, basically there's a simple service that's provided by the SES module, and you call the create contact function, which is just wrapping the PHP SDK, and you pass through the parameters that you want to store, and you're done. So this is a screenshot of part of the React web form, the decoupled web form that talks to Drupal via the REST API, and we built similar forms to handle updating and unsubscribing, you can see that there's some complex stuff here that users have to select a massive list of entities, these are actual entities in the Drupal website, and the notices will get put up against these charts, and so you have to say, well, I'm interested in this particular area, and if anything changes with that chart, I need to know about it. If you wanna know more about the website, these forms, how we did all these different progressively decoupled elements, Jack is doing a lin showcase at 3 p.m. today. So, back behind the scenes of the SES module here, the subscriber tab that we've added in here, can see if we can load in here everyone who subscribed to the module. So these are loaded live from the SES 2 client through the AWS PHP SDK, when you visit the page, we run a quick API call in the background to load up who subscribed to the list. In testing, we made good use of the plus schema in emails, so if I wanted to send 1,000 emails to Jack, I could go Jack plus one, Jack plus two, Jack plus three, it spikes interactive, and then I could spam him 1,000 times. During that process, I learned that Gmail actually throttles incoming emails in this process. You can only receive like 100 a minute in your inbox or something, and so we'd send out the emails very quickly and then they would take hours to arrive, but it was all working very well. Just a very simplified code example here again for listing for showing that subscriber data you saw. Basically we loop over, we can load 1,000 at a time and we call the list contacts method on the API and we can drop that out into a table select form. We paginate it and do a lot of other things with it. It's just a very simple example of one simple method that we need to call to make this happen. So how did we build the mail outs, the workflows and use the queues? So the core part of the mail out is just a content entity and it has a workflow attached to it. So here's an example of what the site builders had gone through and set up part of the site already by the time that I came along to plug this into SES. So I didn't use a custom content entity, we just used basically a node for the mail out, but we could easily use a custom entity and would probably be easier in the long run, but as I look into how we contrived this back, how we reuse stuff, we'll work through those issues. So we've got a whole bunch of different states, workflow states in here and the key part of this is that when you use workflow, you can lock down who can access the different states. You've got the entity revision to look at as to when things were changed into a different state and we can hook into the workflow changes to make things happen. So we can test send when things get approved for testing, we can do the real send out when things have to be sent out and we can even change the state at the end to say everything's finished when it's all wrapped up and they could then lock that down so only authorized users could actually push the big red button. So setting up the mail out is easy, you create the mail out entity, you fill out all of the different fields that need to be in there, you need to link all the different entities, all the different charts that might have changed and those get generated to send out and after that they all get loaded into the queue. So I'll show you the code example for that. I was using entity hook entity presave, probably be nice to use an event subscriber there but I couldn't find anything at the time so it felt a little bit archaic to do that, to still be using the hook system for this but we could check based on the moderation state, if we hit the right moderation state, you can see in yellow there, say we've already preloaded the contacts via that API call we made previously, we can go through and create items in the Drupal queue and those would then get individually processed as the queue runs at its leisure. The queue worker then gets to process the email and you can see that there's an interesting problem here. I have to call the get contact method on the API with the email address that I've already got and this is because when you bulk load the contacts from the Amazon API, you don't actually get the attribute data that you stored against that contact. So this is a big problem in my opinion with the AWS API, if you wanna, you can bulk load 1,000 email addresses to send to but if you want to make some changes or run some calculations based on their preferences, you gotta go through and call the API 1,000 times and you can't batch that, you can't just tell it give me all of the attribute data. There's no method for that in the AWS API and then of course sending the email is easy. We pass it a whole bunch of different options, we can pass through, tell it to use different config options that we've set up in Amazon, different suppression lists that you may have set up different API, different IPs that you might want to send from all of those are highly configurable in Amazon and in that orange and the yellow there of the data with extras again, we go through and do some complex calculations to pass stuff through to that template and the template itself is basically an HTML, template an HTML file, you can get your front enders to make the email look nice and it uses double angel bracket notation to do a replacement based on the, based on the JSON variables that you've passed through. So here's an example email that you would get based on the template we've got, the tables there change based on the charts affected. So in the nodes, the mail out nodes that are sent, different charts will be selected, different notices will be attached to those and then the table up above it there, this one here says, you haven't subscribed, none of the changes, none of the charts you subscribe to have any changes. If you, based on the selections people have made in that decoupled web form, they would get different data showing up there so that they would know, these are the charts that I really needed to know about and the important thing with the emails, like I said, is the, we have the double angle bracket replacements for different variables like my name at the top. So you don't run this through the TIG, twig template processing in Drupal because you're gonna get all of those Amazon variables replaced beforehand. So we just process things out, well the front enders process things out with CSS replacement libraries and all of that but I've specifically told them, don't use twig here. Now, and of course the important thing with us sending these out is that Drupal doesn't know about all of these variables in advance so we have to load them up in every single one that we send through because Drupal isn't aware of any of the information that's attached to any of these people. You can see this unsubscribe link at the bottom and the update your detail links. So Amazon does give you tokens for that but again, like I said, you're able to swap those out with your own custom forms if you need to. So how do we deal with security in the cloud? The nice thing about SES is you get full integration with AWS's identity and access management services. So the IAM, the three letters that are the scariest at least for someone who doesn't use AWS all that much when you have to deal with the infrastructure teams they're the ones that set up all of these permission sets and all of that plugs right into AWS and everything that's available in the API. So when these are configured right, the nice thing is that the API keys are completely useless unless you're calling them from the correct environment within EC2. So the production environment keys only work in the production environment. The pre-prod keys only work in the pre-prod environment and so same goes for every environment that's set up. So you can't accidentally send emails to your production subscriber list if you have correctly configured your IAM services. Which is really nice because everyone's made that mistake before where you copy databases across, push the wrong button and the emails get sent to the wrong people. And also every single method on the API can be disabled per environment. So for instance, we can make sure that the magical delete subscriber list command which deletes everything can never be called by the production user. We can actually just disable that for each user. So these are all very clear advantages over many other solutions like even Mailgun or Mailchimp where it's at least I found it's they're not really set up to work that way. And this is plugged in with all of AWS' security requirements. And if you're familiar with those then it's very easy to manage what's happening with SES. But this did lead to issues with initially getting this working because well, you know, we tested everything, we built it, we deployed it up into their environments, the black box and then nothing worked. So felt a bit like the old man yelling at the cloud because it's a black box you don't have access to all of their systems. And I had some of these moments working on this project. The security aspects felt a bit frustrating because you don't know what's going on. But there were lots of cross-team meetings to figure these things out. We had lots and lots of calls, lots of screen share, Zoom calls where they showed me all the teams, Microsoft teams, and they showed me, you know, okay, well, here's the setting screen, you know, which extra permissions do you need? And I had to make sure we went through all of the code, identified every single possible API method we might need and that all of that would work properly. And in the end, everything worked out surprisingly well and everyone is very, very happy with the result. There are also a bunch of frustrating limitations with what Amazon lets you do, things that are maybe not necessarily documented. So we'll get to the gripe session of the talk. The first thing is that there's only one list allowed per AWS account. So they talk about list management and lists in a lot of places, like they're in plural form. When you want to load your lists from the API, there's actually a method to load lists and they send you an array of responses, but there can only ever be one. It's very clearly stated in the documentation when you look carefully that they only ever allow one list per account. In our case, that wasn't a big deal because we only had one list to need it to work with. You can also set 12 topics or segments on lists. We didn't end up using these because we had more than 20 options of what people could send out. There could be a hundred, could be a thousand, different combinations of things of in terms of what people would need to receive as an email. You can actually filter, obviously, when you're calling the API to send emails, you can actually say, just send this particular email that I've generated out to this particular topic. But we didn't necessarily know that in advance. In hindsight, however, probably could have used those to tag certain people who'd always received the same email. So there were some people who would say, always be interested in everything. Well, we could have probably used a topic to tag those people and then we know that we're always going to send those people the same email that would potentially cut down on API calls and time. I only realized that in hindsight when I was preparing this talk. So there's improvements that can be made in the future if it's needed. Attributes also. Attribute data is limited to a thousand characters. We also hit that by accident when the front end was just sending all the submission data, everything into an adjacent blob. And we started getting errors and figuring out, oh, well, you're only allowed to have a thousand characters. So we made some optimizations there, but there's a possibility if you're storing a lot of data or trying to store a lot of data in your subscribers, that you may run out of space. And of course, you could store that in Drupal, but hey, we're not allowed to store that in Drupal. So it's gonna depend on your use case, whether or not this is going to be a really big problem for your project. Also, you can't search by attributes as I alluded to before, and you have to load them separately. So you can't say, I wanna get everyone who's got this particular value set in their attributes. You gotta go through and load every single one every time. And of course, can't batch load it. And that meant that of course you're gonna hit rate limits. If you call the API more than a few times per second, and we were able to call the API 100 times per second, I worked out that we could probably send all the emails in like less than two minutes if we could call the API an unlimited amount of times. So there's also a rate limit exception that you hit when that happens. So I thought, well, that's great. You hit the rate limit exception, you then wait for a little bit and try again. So that way you can sort of maximize how quickly you can chain things together. But that was very unreliable. It seemed to me like they sort of punish you for hitting the rate limit. So we've just built in some hard coded wait times that felt reasonable and that don't cause any issues. So you know for sure you're just not gonna hit the rate limit. So the big question, does your project need Amazon simple email service? It should be clear by now this is not exactly a plug and play replacement for MailChimp although it is something like a mail gun out of the box. The API and tools that Amazon gives you are robust enough to replace MailChimp if you're able to build the components you need in Drupal. So if you're hosting on AWS infrastructure, your site sends emails, absolutely just use SES out of the box. If you're running a simple mailing list or you have situations where Drupal's content needs to inform what goes into your emails and then absolutely I'm working on a contra module called Amazon SES list and that'll have some of the components that we've gone over today in and out of the box and so you can use that when it's ready but you probably will need to do some custom coding to actually build out your own templates and do your own calculations on what content goes into the emails. So it's very powerful if you have a complex use case. If you're running multiple mailing lists or you're expecting a working UI out of the box probably not really the solution for you. Ultimately I was very happy with what SES was able to do. The client is also very, very happy. It didn't actually take 10 times as long as it would have with MailChimp. I think we would have run into different issues with MailChimp and I think the result with SES was actually better in the end. So if I had to do this the same way again, absolutely I would do it the same way again. Lastly contributing back to Drupal, I've got the module page set up. Some of the sample code that I showed you today is from these files that are easy enough to open source we made sure we designed them to be generic enough so that people could use them. However the actually building and sending the email task is incredibly bespoke. So we're still thinking about how we can have a generic framework that people can easily plug into and get started with this a lot quicker. I was hoping to have all of that ready to go back into the module, back into Contrubb. But unfortunately, sorry before the conference, but unfortunately I just didn't have time to finish that up. So that brings me to the code sprint. I'll be at the code sprint on Friday and I'm gonna take the opportunity to make those commits to the Amazon SES list module that I've outlined in this session. And I'm also part of the sector team. I've mentioned up on the screen here which is our dog food at Sparks Interactive. Everything we do and build with Drupal, we do that with sector as a starting point and we've been doing that since Drupal 7 and all of our learnings go back into that. It's a very opinionated way of doing Drupal that we found to be best practice and it's influenced by all of the different projects and clients that we work on because we found that if you're building a new website and starting from scratch, you're doing a lot of the same things over and over again and if people start fresh with a Drupal website, they often tend to do things maybe the wrong way if they're not sure how they should be done. So we're able to provide a foundational starting point that we used on this project and that we use in all of them and our team is working to get that Drupal 10 ready as well and we'll have either a birds of a feather session or we'll have a meeting at the sprint to talk about that. So there you go, going back 10 years ago again, there I am at the table on the right in the white shirt there at my first code sprint in Drupal-Con Sydney in 2013 and we've all come a very long way since then so see you all there. And any questions? I've got a few minutes for questions and otherwise just come and talk to me anytime during the break or in the next few days. Happy to talk to you and I hope you enjoyed the talk. And morning tears should be available in the next 60 seconds. So if you really want to ask me a question put your hand up right now and otherwise run for the exits. Do have a question there? Yes, I may have gone over that a little bit too quickly early on but SES does its own bounce management and we just let it do its own bounce management at the moment. If you're using lists they'll actually automatically unsubscribe people who are no longer have a good enough rating for Amazon to be able to deliver emails to them. So you don't have to manage that within Drupal and they'll simply get dropped off the mailing list if they fail. And like in this case because Drupal doesn't really know or care who subscribed to the list that's not a problem for us at all. Yes, so there is an attributes data parameter on the contact in AWS and you can store up to a thousand characters of data in there that gets stored on AWS's systems. So the subscribe call that I made so when I create the contact so you call the create contact method on the API you can pass through that data and JSON encode it when you pass it through. And it goes straight to AWS, yeah. And so we use web form as well left this out a little bit for time's sake but we actually have submission saving disabled on our web forms. So web form just gets used to let us set up what components we wanna capture. And of course in the case of the website we have entity references basically content references that we need that they get generated out by web form for us and we can then display those on the front end and we make the API call in the background to just send all the data off the people are notified as well that they've been subscribed by web form and then web form just forgets about everything that it. But that's basically it that the client specifically wanted to manage stuff within AWS. So if the Drupal website something is compromised on the Drupal website it doesn't know anything about any of these subscribers or any of their personal information. Obviously if their AWS systems fail that's a different problem but they'll have a much bigger problem if someone's got root access to the EC2 instance to be able to call those API functions then they're in big trouble. It's nice and easy for their team to man they've got a security team that manages all of their internal security and they only have one there's one less thing for them to worry about. So I think based on even just the 20 topics I could work out what the most like what's a common like the 20 most common combinations of options that we need to send out and just tag those people with those topics. So if they had a hundred of them we could do that a hundred times and so say if we're sending out a thousand emails and we're processing a thousand of them but maybe like 700 people actually need to receive the same email and we kind of know that when we've processed it that they've picked that particular option we could tag those them with that topic and then just process out the remaining ones manually at the end. So I only thought about that when I was preparing the talk but we could do that and yet the more options they give you in terms of topics the easier that would become. Yes, so you could when you send the email you could provide the topic name instead of providing it with a recipient you could provide it with a topic name and AWS knows okay cool well all these people who are subscribed to this topic I'm gonna send them all that email and it'll use the replacement patterns that are stored inside their attribute data which is fine normally but like I said in this case we have to send a customized table based on what's stored in this we can't just print out the numbers that are there on the data. Yes, so you could store identity data like in Drupal or you could kind of maybe encode it even or encrypt it in some way and just store the key on the attribute data I thought about that we didn't wanna go there and then you could run a bunch you could then you'd have the data with in Drupal that you could run some calculations on if you're well beyond the attribute the thousand characters so you would have to store some data somewhere else or figure out hey maybe I don't need that data but that is a hard limitation it's a thousand characters on a subscriber within AWS so you would need some other system and the preferences page as well so the topic so if you I don't know if it didn't have a screenshot on my slides but basically if you would go and click on the default preferences link you would get your email address there's no name that's stored there so you would just get your topic list so say we've created a list with 20 topics like I'm interested in skiing and snowboarding and whatever your lists were you would see those checkboxes on the page and you could save those checkbox so if we had say just 20 chart options you'd say well I'm interested in charts from Auckland I'm interested in charts from Wellington I'm interested in charts from Christchurch or just 20 regions maybe you could then pick those regions out of the box you wouldn't have to build your own form for those and same thing with the unsubscribe form by default it would just switch it would keep the subscriber there that was the reason why we did a custom unsubscribe form they'd keep the subscriber in there and just flag them as unsubscribed but the client specifically said if they unsubscribe we want to wipe their data out completely so we built a custom form so we could wipe their data alright well I've kept you already into morning tea so please go and have your morning tea and come and talk to me if you have any more questions