 Thanks for coming everybody. My name is Lee and I'm here to introduce the rest of the previous next guys Who've done some brilliant work on the Nova FM website, which I believe is a responsive-based site I actually did the design for it, but I can't take any credit because I did the easy bit and these guys made it come to life So I've got Jack Taranto Nick, I'm sorry. I don't know your surname. I can't She and Christian Biggins, so please take it away, Jack. Thanks very much, Lee Cool. So as Lee just mentioned my name is Jack Taranto. I'm a front-end developer and themeer and I've been working with previous snakes for about three years might even be coming up in four years now I've worked on a huge variety of projects doing theming for heaps of different stuff small sites and really big sites like Nova FM and Christian Biggins and Nick Xu will also be speaking. So so we'll be covering a largely technical overview of Nova's site It says on the session that it's a beginner, but it's probably actually more like an intermediate So hopefully that doesn't scare any people away, but we'll be showing you lots of different code examples and things like that Focusing specifically on the responsive design and Christian will be taking you through the enormous data migration task and Nick will be focusing on hosting and GOIP So Nova came to us and They wanted a complete I've got a spelling mistake there. That's all right They wanted a complete redesign of their entire website. So it was based on a totally custom CMS which had Evolved organically over quite a few years and it had like tons of different stuff added into it. It didn't really have any Like that this the original structure was completely lost So yeah, they came to us. We had to redesign that we had to migrate all of the data and We had to build it and make it responsive as well so It was a pretty immense task the total Development time I think it was maybe around three months. I'm not exactly sure There was about five of us working on it. I think at the most maybe six So there were Hundreds literally hundreds of thousands of nodes that we had to migrate There were there was an immense amount of advertising the whole site's advertising based and every page has got ads all over it and it's mainly Thanks to it's some it's a really media heavy site as well. So there's images everywhere. There's video everywhere and there's sound files and everything and Also, there's five stations one for Sydney Brisbane Adelaide Melbourne and Perth and they've all there are all the contents served from the same site and Yeah, the whole thing's responsive So a look at the stats Before we released the site. They had one and a half million unique visitors per month They had three million three million page views and two and a half minutes was the average time spent on the site So that jumped up to 32 percent more in unique visitors and Yeah, 64 percent increase in page views and the time on site Almost doubled to four minutes. So I'll be taking you through a few things around the responsive design I'll be focusing on the base theme and what we chose and some of the custom stuff that went into that I'll be Going into a lot of detail about responsive advertising and the media choices as well that we made so getting started we had a few Obstacles in picking a theme to choose so We had not very much time to choose something or not very much time to implement something We needed something that was going to be reliable and it also needed to be based on 960 grid system, which is what Lee's designs use so We could have possibly built a custom Theme with a custom grid system and implemented, you know, all sorts of different stuff to get that happening But it kind of made sense to pick a Drupal base theme that was solid and had You know all the work all the hard work done for us basically So we used a mega to do that It's it's it's probably one of the biggest I think it might even have the biggest user base of Drupal's responsive themes So that's a pretty big plus going for it at the time. It was a version 3.0. I think so that was another good thing It's got an awesome grid system which uses Collapsing column widths, so it's got different grids that you can choose 1224 column grids and the grid widths shrink down as the page width collapse collapses, so and it's also got some other good stuff like JavaScript Body classes so the device width adds a body class to the page and can then use that to manipulate stuff later on Yeah, so the grid system you can see An example there. I've got the desktop version and then kind of the shrunk down version for a tablet You can see how the various regions in the header Resize the navigations resize the logos resize So there's a little bit of custom stuff that went into that but the Amiga like region grid layout system Helped heaps in putting that together. We're literally able to lay out the bulk of the site Like the base theme of the site the header and the footer and all that kind of stuff in maybe a day or maybe two days something like that Yeah, so some other stuff went into Making the responsive site really stand out the grid system It's probably one of the most obvious things it uses these tiles which are based on The medium rec size so medium rec is like an advertising format size. It's 300 pixels by 250 pixels So this these pages that you see here are like listing standing listing listing pages So all the content is arranged in these thumbnails And there's actually advertising actually inserted You can see there at the top And the bottom so these grids collapse based on the device width. So you've got Yeah, three columns for a desktop two columns for a tablet and then the mobile version they kind of shrink down become much smaller So It's yeah, the once it once it gets down to a mobile Size it becomes fluid. Yeah, so it stretches out to fill the fill the page. Yeah We also use CSS media queries to hide certain regions on the page So we we had the example I've got here. There's a There's two regions at the top there are mobile only navigation regions So they're actually in the same HTML that the desktop version gets but they just use a CSS media query to set display none So they get hidden from the user and only displayed on the mobile version So It was all this kind of stuff that that went into it to get the responsive site happening It was the first responsive site. I'd actually built at the time So it's kind of the whole thing was a learning experience, but yeah, we got there in the end Yeah, so the next Probably that probably the hardest thing about implementing the whole responsive side was the advertising So we have tons of different sizes and formats we've got different device widths and The ads need to be specific for each device. So you can see an example here. I've got The desktop version in the top. It's got like a full width ad skin kind of thing Then the tablet version on the side there uses leaderboards extensively Mobile version only shows it mrex so each Each ad each different, you know device sees different ads basically and It's so difficult because you've actually got one because the sites are responsive You've only got one version of the HTML. So you're not serving different HTML for different devices and All the ad tags are in the HTML So, I mean you could just hide the ad Like the particular divs that the ad renders You could just hide that with a media query and say don't show up for mobiles But it will like the ad engine will still record impressions for that for that ad so you have to figure out some way of Only putting the ad tag in the page for certain for certain devices and It's all Yeah, it's kind of unnecessarily difficult. I think if the ad Companies we were using double-click if the ad companies Were on board with it. It's probably quite a simple thing to do There might actually be some some guys that out there that do it But I don't think the major guys like double-click have any real way You're able to embed like a JPEG version of the ad or something like that But Yeah, you can't actually say this ad is only meant to show up on this screen size so Java script was the obvious choice there to embed the ad tag. So we wrote a custom JQuery plug-in which was about It's probably it was probably only about maybe 40 lines of code or something like that. It was quite quite simple really and it took advantage of The modernize a media query so modernize a media query is exactly the same as a CSS media query It's just in JavaScript so it Talks the browser gets the device width and it either returns true or false You know based on your media query so you can put CSS media queries into Into JavaScript and then take advantage of them. So then it was just a matter of writing a couple of lines which Took like the actual Google ad code and then put that into the page So instead of embedding the actual ad tag. We just use this little call to the plug-in. It just tells the Tells the jQuery plug-in which div to render the ad in and then you can actually just specify the screen size Mobile tablet desktop and then the ad will show for the appropriate size so There were kind of a few limitations in that Way of working. We still had to use CSS to hide the ad Block from users in case You know, you might have a user on a desktop which is resizing their browser window or something like that So if they shrink it down, you want to hide the ad and not break it But that was a problem then you wouldn't have ads You wouldn't have Ads reloading in that case. So if people are shrinking the size down the stuff the ads might not necessarily appear But for the bulk of cases it kind of works pretty well really So now on to media It's just a small splattering of some of the stuff they've got going on virtually every article or node on the site has some form of image or video or Gallery or something like that. You can see we've got like YouTube we've got YouTube video players. We've got bright code video players. We've got Views slideshow galleries flow player audio and The biggest difficulty with it all is that the actual developers hadn't really implemented very much responsive stuff at all so even YouTube and things like that aren't really straightforward to put into a responsive site Which is probably might have come a long way since then this is kind of the end of 2011 that we were doing this sort of stuff So it might be a little bit different now so flow player gives us the audio player which is used for podcasts and Yeah songs and things like that It's Relatively simple. They've got a kind of a beta Version plug-in So you just it's just a matter of embedding that JavaScript and then you're just able to add iPad to the end and that gives you a HTML Audio tag or video tag So it was relatively simple Yeah, nothing nothing too tricky there. It doesn't quite work as well as an actual Embedded audio player which gives you like a play button inside your Mobile, but yeah It does seem to work, okay YouTube requires some pretty funky CSS you have to create a div wrapper and then Yeah, you have to set it as Position relative haven't actually got it in there, but yeah, you said it's position relative and then the iframe Google's iframe gets absolute and then you said it's with based on its parent So it's kind of totally hacky really if you look at the code It's setting weird pannings and stuff like that There's a few different ways to do this as well different people have come up with different stuff and If you search the net you can you can find all sorts of stuff like that The iframe YouTube's iframe embed is actually good because it's got they've got two versions of the play They've got a html5 player and a flash player and it'll It'll show like the html5 player for a phone or whatever and the flash player for desktop. So that is something good And yeah, bright cove was kind of its own thing. I think I don't really know Yeah, if that many people actually use it But it's Nova has like tons of their videos on bright cove and they use it to put ads inside videos and all that kind of stuff So it's really important for them But it's it's kind of nasty views. It's putting width and height values into Like deep inside its video tags So it's not adding it to the object tag, which would be kind of workable. It's adding it to the param element, which is a little bit annoying. So You have to actually let that code there is just checking Checking the body class And then it's adding like it's manually updating the width and the height values So that works for a mobile and then the tablet version is separate So you have to manually kind of like hard code the width and stuff into your JavaScript, which is pretty Pretty difficult to work with But the solution actually worked if you check out bright cove on the site you can resize it and stuff like that and it does work Yeah, so Amiga was great to work with it was the first site I built with Amiga. We've gone on to build heaps more after that Yeah, it was really handy in all the kind of stuff that it's got set up already It's not really it doesn't really work the way Drupal works like it's kind of an anti Drupal theme in a lot of ways It's kind of implements its own Like pre-process files and stuff like that and it's got its own way of getting like standard Drupal variables into templates And it's got heaps of extra templates, which are just events and so It's a little bit of a challenge working with it for newcomers It might be a little like if you're getting into the dirty kind of nitty gritty of it It's like it's a little bit tricky, but it is a great theme and One knows a media queries Kick-ass I think there might be another JavaScript that does I can't remember what it is, but they're really handy they give you CSS media queries in JavaScript, so that's awesome and Yeah, like a big slap on the on the chops for the media guys for not not coming to the party They could um, it could be it could be a lot better. I mean things will improve over time I think responsive sites are becoming the norm. So yeah All right, I'll pass on to My colleague mr. Christian big ins and he'll take you through content migration. Thanks Hi everyone I'm Christian. I've been with previous next for a couple of years. I have a Febita background in in data and migrations in general from pre dribble days So I was brought in to handle the the content migration for Nova slides and robot a check Anywho so the considerations When when I was brought into this we had the the existing Nova site Was a bespoke application. It had no no structured schema at all It was on a Windows system with a mssq or database There was lots and lots of content like like hundreds of thousands of rows and there was there was Relationships literally between each row Everything was connected one way or another The content and media was stored in different locations so some some Content was stored in a in a table specific to that content. Whereas others were stored in different tables so for an example of that was that a Video might be stored in its own video table, but it also could be stored in the articles table Using an embed code. So it made it pretty difficult to make sure that you're pulling videos from the right places Challenges there was there was duplicate rows for the multiple sites Jack touched on how there was there's localized sites for the station There's five that we migrated Each Site would have its own content as its own row So if one article was to be published across the five sites, there would be five versions of that article Because of that there was lots and lots of lots of edge cases Scenarios where code had been changed and the schema had been modified to To handle a single scenario Also that not all the content was to be imported because we had That we were only handling five of the sites But there were there were many other sites that we weren't actually migrating they were going to stick with their with their CMS an Example of the relationships that we had So if there was an album on the site the album would have been created by user The album would have Tags associated to it It would have an artist and an image and these were pretty much equivalent of having entities But it would also have lots of manually entered related content, so there would be The manually configured Related content block on the side and everything would have one of those So we had to make sure that all of these relationships were maintained across the migration We had lots and lots of potential issues the biggest problem that we were facing was The chicken and egg scenario and that is what do we what do we migrate first? How do we ensure that the user record is there when it's needed to be when we're trying to provide the the author attribute on the album Is there an order that we need to perform them in so that we get that how do we ensure that we don't get any Duplicates considering there are so many duplicate data already in the in the tables So options for handling this was custom code Which was pretty quickly discarded only we had pretty strong time restraints and And it was going to be really really hard to make sure that our relationships were maintained after the migration The feeds module was another another option at the time there was very little documentation on it and It wasn't going to work very well with the the unstructured schema that we were facing So that pretty much left us with the migrate module So the migrate module gave us lots of things lots of ways that we could work with what was coming in the mapping of columns and fields was was just so easy and and it gave us a methods for pretty much every stage of the migration from Pulling the query out and altering the data To before the entity is saved or after the entity is saved if you needed that And stubs which I'm going to get on to in a minute which would allow us to create Entities or placeholders for data that's not yet migrated to assist us in maintaining the relationships Lastly migrations would occur in batches so super easy if you if you're doing it through the UI and and and you know The connection drops out you can just take off from where you were You can use drush to kick them off So the migration mapping is literally that easy You would have a query above just just a typical normal SQL query and you would provide The column that you're migrating from and the field name that you're migrating to there are other options that you can set such as the Whether or not it's going to be HTML or plain text or anything the bottom the bottom Mapping there sets a source migration So this is in the case that you might want to create a stub for data that is not yet imported It's just a example of using the complete method which is performed after the entity has been created And saved and you can make any changes to that entity. So in that example if you wanted to Ensure that the user's avatar had been downloaded properly and saved you could do that or you could go and get it if you Wanted to do it after the migration Because the entity is already existing Prepare method runs after the entity is being created, but before it's been saved So you can handle any any manipulation you needed to there I'm simply adding in a roll to the entity before it's sent off to get saved And prepare row is is after the rows been pulled out of the database But before the entity's been created. This is just another step earlier And here all I'm doing is setting creating a URI foe column if you will For where the actual full path of the file So a little introduction to the stubs which I mentioned scenario we have is if we migrated an album and That album had an author attributed to it with an ID legacy ID of 999 Migrate module knows that that user doesn't actually exist yet. We haven't imported it yet It will go off and create a stub user So it will go off and create a an entity that's full of placeholder fields. You you specify you define those and then it creates the With the with the fields that you've specified it will create a Drupal user Or it could be an ht. It could be a node anything you wanted So it's created as a Drupal user with the user ID of a hundred It will then go and update the album record that we've just imported With that user ID so we've got that maintain that relationship maintained Now what will happen is when the actual user record is migrated to its own migration? Migrate will identify that that user has a record already that's been created through a stub And it will update the stub with the data that you specify in your mapping so In this in this scenario what we've done is created an album Without a user record and then updated the user as the information has become available The stubs really easy to create and you basically have this in your method in your migrate class So in the user migration class you would literally have this this method Which just creates a stub with the minimum required fields input Word of warning is that users actually need unique email addresses So what I'm not showing here is just a Function that you have to write to make sure that the email address is unique so that it will actually save after the migration I've got my introduction slide next Jack. Thanks After the migration we were just running the batch module just had a few the batches that we Set up to make sure that all nodes had the files associated and we're on the file system We were able to perform a whole bunch of different tests to ensure that the structure was maintained in relationships especially Also, I would just make sure that the content Could be accessed from the same URI the same path so Just to ensure that when the site actually went live that all the URLs remain the same or at least the redirections worked And there's my introduction slide I'm gonna pass over to Nick who's going to just run through the hosting. Thanks Christian Okay, so With Nova FM with the consolidation of these five websites Coming into the new new Drupal sites We wanted to set up a new a new piece of infrastructure that was sort of Drupal centric And It was scalable easy to use we could inflexible By the way, my name is Nick Completely skipped to the thing. Yep. I'm a developer at previous next. I've been here for about Six to nine months now and I've taken over some sysadmin duties, too So now I'll start talking about the host infrastructure Okay, so the hosting infrastructure that we went through I'll go through the first and foremost host That's the load balancer And it's an f5 firewall load balancer that does round robin balancing Round robin balancing is actually pretty simple in concept As seen in the diagram five users are hitting the site. Those are what those numbers under the hosts are User one comes in hits the first app server Two three four and then five comes back around to the first again, and it'll just keep working its way down The next class of apps the next class is app servers application servers These come in two flavors. We have Active hosts, so those are the ones that are currently running right now Four of we have four of those running in a in a in a cluster set up And we also have inactive hot spares on the side ready to be added into the load balancer if needed All powered on and then added into the load balancer All these hosts come with varnish Apache PHP and memcache I'll just quickly go through those varnish is a is a caching caching service that you can put in front of your web Host so it'll case up the HTML files, etc. So it's really snappy every time once primed Apaches your web server and php and Memcached is what we're using to cache all the All the we're using it on our Drupal sites the memcached module and passing off All the case started to memcached and stored in memory on the host All this configuration is puppet managed and What puppet is is it's a it's a manifest kept in code and it Provides install and update instructions So if we were to spin up a new host we can install puppet give it a blueprint for what the site's going to be and it'll spin it up And we can start adding more and more if we need to if there's a if there's a case for it If anyone wants to talk about any of these like varnish puppet, it definitely pop around after and we'll have a chat And lastly the the code base gets deployed via Drupal Capistrano And what this means is with one command we can and get get control We can push our code base up to every single app server So everyone gets the same experience across app one two three four and so on It also eliminates any any risk of Things being missed so Very very handy These app servers connect to some storage. So in this case it's We're using MySQL a DB and solar These are replicated across two hosts. I actually have a diagram in a future slide But I can sort of show you through but these operate in a master slave Kind of setup So if one host was to go down the other one can take over and keep going We also have a some NAS file storage across all the app servers This allows for all static assets like images Podcasts I know they have a lot of podcasts up there To be served out across all app servers because if someone uploads an image on the first app server You want it across two three four and etc It also handles For Drupal the settings dot PHP so every app server knows how to connect to the database host And if you make a change every host has it mounted up so it's consistent that is also Puppet managed in the manifest so we can push that out to all those storage related hosts For the website, we also use CDN. We use the Drupal module CDN and what this does is We provide a URL of CDN.noverfm.com.au It rewrites it to in this case edgecast and Edgecast stores the CSS JavaScript images and that means as you can see on this map They have a lot of of stores and if I was in Another country it'll load nice fast snappy Okay, so what we have in front of here is a Is a diagram of the hosting infrastructure. So at the top, we got our load balancer F5 load balancer below is our app server cluster To the right we have our spares These all undergo the same patching and code updates as their active counterparts We also have the NAS on the left to the file storage which connects to all these hosts or these hosts connect to it And then we've got the the storage for solar and my SQL so as you can see Master of solar doesn't also have master of my SQL on the same host and They swap so if one host were to go down it would move across Okay, so with all this in mind With Nova FM. There's definitely content that goes viral I think when I came on to Nova FM Nova FM was already up and they they were doing competitions and I Learned about they called it the one direction effect so And What that was was and I'll tell you what it is in this case. We've got 17. That's that's in one hour That's 17 18,000 hits So and I guess the what you have what has to be considered is what type of trap like what type of page is it that they're hitting like what what's going on and Is it cached is it not cached and I'll go into that a little bit more but In the case of being cached. That's that's fine. It's It's not really that noticeable if we're not we're we're in trouble So scaling and large traffic So sometimes content goes viral one direction and one of the options. So With all these things in mind We've learned from our mistakes Maybe when we push up some content that we know may go viral or there's a competition coming that we may Know that goes viral. We can provision in advance. We can maybe get some hosts into the Pool beforehand get it added into the load balancer. So it's less less work at the time and less worry But I guess the thing that we have to consider is at that stage Is it going to be a video where it's cached the page is cached? And it's just going to maybe a YouTube clip or something or in the case of nova fm The competitions require a login the user has to be logged into the site Which means that they're not anonymous anymore meaning that That the site's not cached This means that whoever wins the competition. They know who it is and they're legit across the site And if this is the case, we definitely have to consider like adding hosts increasing resourcing on existing hosts, so beef up what we've already got and Sometimes you don't have get The chance to provision in advance and at a very last call right it right it out. Yeah Yeah, okay So next we have our geo location so um as my my colleagues have said we had five websites So the case was yep. Nova FM has five websites Adelaide Brisbane Melbourne Perth Sydney and And they were all coming into the one Drupal site and we had to ensure that That user base were actually getting directed to the correct content. So if I was in Brisbane, I want to see Brisbane content I don't want to See that of stations on right now or programs on right now and click to the radio and it's not Because that's that's just annoying So yeah, we need to make it as easy as possible So enter Apache mod geo IP to So a quick quick little overview is geo IP offer a city and a light version In this case we went with the light version as city provides is a paid service and It gives you a database that you can update and or it's more frequently updated because it's very granular In this case the major cities aren't going anywhere But if someone in their app were to purchase the paid version They would do it because they want to know street numbers and street addresses and that that kind of granularity that really low granularity So we went with a static the static copy of the free geo IP light database So the benefits of that were The configuration was within the Apache v-host com Configuration so we didn't have to bootstrap Drupal So if we were to put this kind of logic in Drupal, we would be bootstrapping Drupal like firing up the site Checking where they were and then redirecting to the appropriate URL for the station So that's two bootstraps for the same same thing that we can do right there in Apache and Yeah, that the redirect is is cached to Okay, so with all this in mind after it was all implemented. We ran into a few issues So each each station gets a gets a header set for it So once they're redirected to their self I was redirected to Brisbane I would get a header of that station that means that when I go to a competitions page all the all the content or all the related content Is Brisbane esk Brisbane related? I'm not seeing anything Sydney related I'm not sort of getting pushed to other stations content And The issue that we ran into From doing this is Drupal cases its page and uses the URL as the key So in this case you can see Competitions it would have the page cached with the key as competitions But we don't want that because if I'm if I'm in Brisbane and someone's already primed the cache and Created that case in Sydney. I'll be seeing Sydney content So how do we get around this? Varnish so we use varnish to interpret the header and As you can see in this diagram. I'm I see slash competitions varnish rewrites the URL To add this Nover FM station key on the end and that's what Drupal that's what Drupal caches and I don't see that that's that's deeper That's deeper level stuff. I don't see that in the browser. I never never see that and What we now have is if all that all the stations are If five people from different areas of viewing that one competitions page That's five five versions of the page being cached now by Drupal And before this myself and Meg had a bit of a chat And how could how could this be improved? How could we improve what we have already? Drupal setting the header Apache is doing the rewrite varnish is doing this this really nice rewrite of the URL that that the end user doesn't know about And it turns out that varnish now in version 3 there's a v-mod for G.O.I.P. and What this would mean is we could We could set the cookie all in varnish and we could We could set that URL and varnish and we could do the redirects This would simplify the approach down This would mean that yeah We don't have like Apache doing a little bit Drupal doing a little bit and and varnish doing a little bit bring It in and it means that we're not locked into Apache no more way We could go nginx or something along those lines moving forward adds adds more flexibility So in conclusion We now moving forward have a have an environment with more options of scale and With with everything going on we now we now know that we need We need some some forewarning of contents going to go viral That's definitely a lesson that we've learned moving forward and I believe The relationship between us and the client like the client knows when something's going to go viral and the conversation starts most of the time sometimes you just don't know and The last one is yet. G.O.I.P. is a great solution. There's definitely many ways to implement it So across Apache or varnish So The question was could we use the user agent to sniff out the browser? Yeah instead of a media query Yeah, I guess the problem with that approach is you're not so much sniffing out Devices or browsers you actually need the device width Yeah, I don't know if you can I don't know if you can actually get that in the user agent or not you can Yeah, yeah, that could work too. Yeah. Yeah, there'd be tons of different ways of doing it There wasn't too there wasn't too much additional JavaScript in the page. There was kind of like for each ad tag There was just that one line that you saw Yeah, yeah, so that that adds the ad tag in yeah Yeah, I basically just chose that way because it seemed like the simplest way for me to be able to implement it. Yeah Cool Yeah, the question was the CSS. Did we use a CSS preprocessor? Yeah, we used our SAS Yeah, so to add to that. How did we compile SAS and integrate it with Amiga? Amiga's got a bunch of different CSS files that it spits out for Layout, it's got different Different CSS files for each layout version So we literally just had a SAS file in our theme a SAS Folder in our theme and it had like SAS versions of all the Amiga Style sheets and then we just ran the compiler locally And then check the compiled CSS into into github. Yeah Yeah, we did. Is that is that a Drupal module? Yeah, there's also sassy or Susie or something. No, I think it's called sassy. Yeah Yeah, we didn't use compass. We kind of had our own library of mix-ins and Things like that. We had a variables file and like a base partials file with mix-ins and stuff like that But we we didn't really this this was kind of back in 2011. I've only really started using compass recently. So Yeah, it was probably a bit before that time. Yeah At the back. Okay, so So the question was how do you manage content across all for example all for app servers? So the database is external to the app servers So all four hosts are connecting to that database for that for that record And in terms of files because that's mounted up on storage each host has its files directory mounted up in storage The files available on on all four Yeah, yep, so so you're Okay, no, so the question was when content when creating content is there some kind of automation to Yeah, yep. Yep. Yep. Oh, can they do that? So, oh, okay So, okay, so do Nova FM have the have the means to spin up extra extra app servers Or automatically No, that's that's all we have a pretty pretty tight relationship between Nova FM and the and the hosting provider So it's it's really all discussions between between three of us Oh, so sorry, do you? At at the time. No, it's it's a methodology that we've recently Started kicking on and it's definitely something that has is that something that you've you've taken on? Continuous integration because I'd love to talk with people and see what they they do to solve it Absolutely, well, yeah, well, I'd love to I'd love to chat afterwards That's that's for sure because it's definitely top of mind. It has been for the past Not not on this project. No, not at that stage So the question was how do we classify content to stations? It was it was all taxonomy based there was just one taxonomy with five terms and When content editors are creating content they can just choose one or more of the stations Yeah, yeah, they're they're all VPS is hosted by another nothing so the question was why didn't import the user first The user was the easiest way for me to to explain how Stubbs worked The relationships went across everything So an album would have an artist record and that artist record also has a user That assets your record also has tags, etc. And this was across every every piece of content articles artists albums Videos songs audio you name it. So There was there was no starting point, you know, there was no where you could start yet users was probably the only thing we could have imported On its own initially, but everything else. There was this far too many connections involved I'm not sure. I actually wasn't across the project at that stage. I was post Me yeah, absolutely me nothing nothing's there's no sorry the question was Does the scaling of the hosts reflect nodes or users? No, no it doesn't it's more of a it's Very static, but it's sort of monitoring performance and in How much resources are taken Over a period of time and should we upgrade moving forward the question the question was was Automation considered in the in the beginning of the project Consideration is that there was no At least major Australian based cloud providers Towards the end of 2011. So there was there was simply no option for an automated system at all Yeah, well, there's lots of options now. Absolutely. If you were to do it now Things to consider So the question was how did we clean the data the We tried lots and lots of ways to try to automate the data cleaning At the end in the end because some content was plain text and like I said lots of things were embedded files In the end it was just it was all tags were stripped out There was lots of JavaScript embedded in a lot of the content So it was literally as if someone had saved the page as it sat and then whacked in the database So there was lots of importing of scripts There was there was no way to safely ensure that What we would end up with was was valid safe HTML. So We there was lots of things that we were able to keep Strong tags and things like that, but the majority of tags were just stripped out on the way through So the question is how do we test the migration? We sampled we just grabbed samples of certain Certain content of each different type and then matched it against the database that we imported From the legacy system So there was no We weren't we weren't comparing with a live site We were comparing with the database and then writing queries to match as much as well as we could There were the the Cleaning scripts and the and the testing scripts were rewritten Dozens and dozens of times because we would get 90% through and there'd be one simple one single case Where it wasn't the same and so that would have to be included So yeah absolutely and that's where I sing the praise of the batch module. It's Makes that stuff very easy. It's all accessible. It's all accessible through solar. So solar does all the indexing for us As far as I know nothing was actually archived from what we brought over from the previous system So it's it's it will be it's it's massive. We're talking hundreds of thousands of nodes We did we did lots of that towards the end. We had our whole team That would just be clicking through randomly just following links and and just site checking samples of code as there's no way We found issues with missing files and that came down to some files being available some places and not others So there were a lot of cleanup scripts that we wrote afterwards to read download files that we're missing Yeah, pulling out the source downloading the file to the file system and then and then importing it as a media Object a media entity it would it would take a while I think to pull it out so that it's so that's generic enough to be used We definitely considered doing that It was very specialized what what we were doing. It was very very specific to our needs at the time I Don't know many use cases where where what I wrote would be useful Sorry Yeah No, that's right. So in those showed that the complete and the prepare methods So in both of those in the prepare you can clean it and in the complete you can confirm that it was that it worked So the entity saved entity matches the row without Anything additional?