 The word for this game is on the main starting solution board, which is your speaker is Kim Kizerlake. The developer working for users perspective, Kim, has a very career delivery and creation and content of all types of mobile audience. Her first word was written on punch tape. Does anybody know what punch tape is? Is that so? Yeah. Kim, she has to stop learning new programming languages throughout her career with engineering and information technology. She has a B.S. in chemical engineering and a B.S. in chemistry for Purdue University, which qualifies for the word for WordPress apparently. And she's worked for several courses in five hundred companies. She's been creating websites for all of them in the internet. And now we're to respond to B.S. companies using a lot of WordPress integration, so please welcome Kim Kizerlake. Yeah, that I will comment on. I guess you have to be from a certain point of your head when you're calling. So I'm going to talk to you today about an application that I've worked on for a company called Channing Celebrations. Channing Celebrations is based in New Jersey and they are a company that places videographers and photographers and DJs and officiants out legs. And although there's only about five permanent staff people in Channing Celebrations, they put their contractors in hundreds of weddings every year, primarily during the summer months out in the New Jersey area. So when you get to talk like this, you always want to tell a story. And so I have to ask myself, am I going to tell the story from the point of view of the developer? Or am I going to tell the story from the point of view of the user or the people who are paying for the program? But then I'm going to try to mix it up and show you a little bit about how you come in as a developer and solve problems. Some of my training before I became a WordPress person, I looked at that chemical and I was trained in and taught consulting skills. And consulting skills teaches you how to go in and be a useful consultant, a consultant that people listen to and come back for. And a lot of times when you get involved in a project, especially if you're hiring as a freelancer, you're going to see what they call the presenting problem. So people are going to come to you and say, this doesn't work. And it's very easy to get focused on trying to solve that problem and not be in a position where you can open it up to really understand what's the real, what's really going on here. Are we really solving the right problem? At first it's just, you know, solving the problem that people see on the outside. So I'm going to talk about this application that they have in a chain of collaborations called the event card. And event cards are a custom post type. And what they do is they provide a potential posting of the thousands of images that are generated when you have a wedding. Now it used to be that you took a photographic film, you would take hundreds of pictures. But when the digital revolution came around, when digital people go and film weddings now, they're generating thousands of pictures. And there are places that will post pictures from events that they typically will take down 30 days or 60 days. And in chain of collaborations realize that people like to come back like a year later and look at their images. And they like to share them as well. We use this as a marketing tool because as you'll see, each event card is linked to a location where a wedding can happen. So when the marketing people are talking to someone who's considering to hire a photographer, they'll say, well, where are you getting married? They don't say, oh, we're getting married at Reggela's in a certain city in New Jersey. And they can go into their event cards, boom, and come up and say, well, here we go. And we're glad that we did it for Reggela's. And here's all the great pictures that we took of the wedding right there at Reggela's. And in addition to that, these images that we post have all sorts of links to social sharing. You can share pictures from your wedding on your Facebook or whatever. The link that you have will bring you back into the application and to the light box that opens to show you the image. So when you have thousands of images, just giving a link back to a page that has thousands of images isn't very useful. So it's that deep linking and some of the functionality that we have in the event cards that became a complication for how they worked. Let's take a look at an event card, Kay and James wedding. This works really well for SEO for these folks. Even if they do hundreds of weddings a year, in every single wedding that they shoot, they put a post out that has any information. So it talks about the event, it talks about where it happened in location. So they get a lot of SEO out of this. It's not something that they have to think of. What am I going to write about or walk those? It's a post for every single event and you can scroll down here. You'll see that we get into this section where you have like a print sheet that you can see when you come after the card. And each one of these is below the photographs. You click on the light box over there is the food box on the leg. Like I say, I really like it because the links, the two point of social media will take you back exactly to this view of this image on this page. And that is something that is much more user friendly than just jamming it back to a page with thousands of images. You can see a little bit of the work that's kind of happened along the way here. Each one of these sections here is for different points in the process here. We even have group bits. So pictures of the group, the pictures that you want to share on your social media site. So one of the things that we did initially was to try to divide it out. That's a lot of images to try to download at once. Let's divide it into different sections. That didn't seem like enough. So let's divide it into two sections. We'll have Bride 1 and Bride 2. But when I came to work on this project, it really had an issue with the performance. And it just really didn't work very well. Even though we're using masonry, the jobs for masonry plugins, loading images as you scroll out the page. The reality is that in 2015, the event cards, we loaded up 2 million images into those cards. In 2016, it was 10 million images. 2017 is 23 million images. So we're talking about some really intense image uploading and presentation. This isn't an application where people come back again and again and again and look at the same images. And frankly, you're probably not going to look at these images multiple times when you come and you look at it. You've got your brand on her guidebook to a computer that she's got. And you've got your hands. And so the range of users that we had was large. And we did have to support people who had some older equipment. Because some of the older people in our user group just don't have real screen and fast equipment. So I'm going to talk today about these three techniques that I use to make this application actually work. Domain sharding, dynamic image coding. And then how we put it all together with the Amazon services. And it's a lot more than just the S3 that we're using. So domain sharding. Now this is a technique that some people are familiar with. Considered to be old school with HTTP2. It's not as effective. But domain sharding is a situation where instead of downloading all of your assets from one domain, you can download them from multiple domains. Because browsers typically will give six simultaneous connections. And so if you have 4,000 images and you're taking six images at a time, essentially the browser finds out. So this only shows two images at a time. Two and two and two and two. The reality is for most modern browsers it's about six, six, six. With domain sharding, what you do is you set up C names to alias your domain. Because browsers don't care about the IP address. They're only looking at the fully qualified domain name. So shard1.inchandancecelebrations.com and shard2.inchandancecelebrations.com. So on the way back to the same folder, I can serve images up. So instead of serving six at a time, I can serve multiple batches of images at a time. Like I said, a lot of people say this is old school. Just an idea of what this looks like. Here are three of the image lengths. And I highlighted in the blue the fact that they all are enchanted celebrations. But we use some words that help make a bride's photo, those fun photos, embrace, extraordinary. Some of the words that they use. So that is essentially what a sharded link, a shard system looks like. Your links have different subdomains. They all resolve to the same location. So they're all able to let you reach out for any one of these main domains. The pros and cons of domain sharding is number one in the cons. There's time required for the DNS looker. So it's not a one-to-one correspondence. If I have two domains, I don't get it twice as fast. If I have three domains, I'll get three times as fast. Because if that time required for the TCP, for the domain looker, it does degrade it. HTTP2, Chrome calls, secure port calls. They are replacing domain sharding as a way to speed things up because they don't make the connections in the same way. They don't have the same kind of browsing limitations. And number three is website modification. So if you want to use this technique, you do have to use some editing to make this code that you have. But on the left side, it still works. I'll show you some data. This still works. We saw 40% increase in page loading time with sharding our resources from our content delivery network. And for us, for the thousands of small images, this is ideally suited. We're not downloading large images. This is not the production images that you're going to eventually buy in large format. We do deliver that digitally. But these are for these thousands of images to show on these pages. And the internet sharding is out there, and it still looks like a good option. So here's some actual waterfall data from a test that I did. So I created a page that had 1,092 images on it. And I looked at the waterfall. And you can see in this first waterfall, this is the not sharded example. And it's downloading all of these images. You can see six, six, six, six, six. So you can really see that simultaneous number of downloads that it's your browser that moments this. Not the seminar, you know, to keep one domain from coming in and just, you know, monopolizing your browser. This is a test showing a situation where we had three domain shards. And you can see in here that we increased. And it turned out it's about 3%. This is, I think, that counted it up, but it's almost exactly percentage-wise what we saw. So you really can see it right there in the markup. This is actually working. And I did more than four tests, but this is a compilation of that data and showing some averages. And with no shard, our 1,092-inch page loaded up between 60 and 80 seconds. So this is not a, this is not loaded up instantly. There's nothing about this, this instant. And people understand that when they come to look at them. Two domains, a little better. We found that for us, three domains to shard these images from three subdomains gave us our best performance. Typically, we're under 40 seconds to load that page. As you go up and try over five domains, now you get into this situation where the time it takes to look up the DNS is degrading it and you're not really getting any magnification up there. So the good news was sharding still works. It seems to work really well for this particular application. So great. So we sharded up some domains and we declared victory. And then the photo editor started creating event cards. And our photo editor group is contracted as well. And I find that in my talk, you really don't have a lot of ability for rules. I'm giving people a creative content. That's like the tail trying to wag the bar. And in most cases, that doesn't work really well. So let me show you this code here. Yes. Now I go back. So this is an I haven't get a page. You might see some code. This is just a really simple example of how you can shard code. And I'm showing over here. If I have a string, that's just a variable that contains that big long code with all these links. I'm just doing a simple explode. I'm going to explode that long string with all those links in it at the www. And then I'm going to go back and I'm going to put that string back together with my long domain names. And in this case, it's three. So I just go through and iteratively go through and I use this. If it's divisible by three, it's a prize. If it's divisible by two, then I give photos. And if it's not divisible by either of those, I give it fun photos. So I'm able to take my content and shard it up. And so that's what we did originally with the content. Where can you put this in? It's best if you do this when you save your code because you don't want to create multiple sub-domain versions of these images for casual reasons. So you could shard it at the time when content is loaded. Put a filter in and shard the code right when it loads to the page. We've done it the better at shard the code as we were saving it to the custom post. So it looks great. Shard it up. This is what the Metcarn creation looks like. You come to the custom post. You add some text. You've got some custom fields. And then the old method was they actually took a folder and slid it up into this content box that we created. We have content boxes with top picks and brides and groomers showing in the top. I didn't even know you could do this. It should be a little bit fuller on your website and grab and slide it right up into that content area. Sounds great. But you slide it up and then you go get a copy because the thousands of images take a long time. And then when you come back from your coffee, you see all these various, depending on what day you're in, it's scripted out, it's timed out. So although the front end of being able to show the pictures, we're ready to declare victory, and in reality was they could only create two of these cards a day. In the busy wedding season, they have dozens of events every weekend. So if we were in a situation where we just simply could not create these, could not run the images through the editor in this way. So that brought me to the second part of this, which was instead of uploading the images that way, we're just going to create the links that I am. We know what they need to look like. We know where they're pointed to. Instead of bringing them into that way, we're just going to go in and write a code to just generate them out. I wasn't able to control how many images the photo editors could put in, but I wasn't able to get them to sequentially number them and to give me the starting and the ending number for each one of those sections. So I'll write pictures of image 200 to 400, the group pictures are 401 to 500, and to do that, for the photo editors to generate a folder with all the images sequentially ordered, then they also each have a file name that starts with four letters, and then the acronym here is a depth of file and then an acronym for the images. So with very few pieces of information, now when they go to this event card, they put in their content and they come over here and put in the two pieces of information that identify where this file is going to be found, eventually a folder, or in our case we'll show it with a bucket name, and then an individual acronym for each type of a wedding, and then over here they can come in and fill in the start and then for each one of these sections of pictures. So we went from taking hours to the one event card to a situation now where they can actually put all the information and upload the images, and by the time they're done it's ready to go. So, as you saw, our strategy to upload those images through the WordPress system was not feasible because of the number of images, the fact that it would completely bog down their inter-office internet system. They actually had con-cats come out and put in a bigger single user pipeline wired to their office to try to make that work. At the end of the day, the crunching that WordPress does, the number of images it generates in the background will upload images through the content editor, which is too much for this application. So what I did was I went over to Amazon Services. And so we had already used an offload plugin to upload images and take them up into S3. And the way that plugin works is you actually put your images into WordPress in a normal way. And then there is a script that runs in the background with a third card who provides the plugin and they actually move your images up into Amazon. They replace the links in your media library and they make that work. So we had already moved into using Amazon as the repository for these images. And in addition to that, when I created this shard, I'm not going to talk too much about Cloud, but Cloud's front is the front for showing your images that are on Amazon. So if you store your images in an Amazon parking lot, that bucket actually has a URL address. And then so you can show your images directly from the bucket through Cloud Run with that address. But the third thing I want to bring into this is how we use Lambda, which is another Amazon service, to handle image manipulation. Amazon Lambda, if you're not familiar with it, is a service kind of like a Heroku, which basically, if you have code that you want run, which you don't have to have your own server, you don't have to manage any of the back end to make that piece of code run. You can run on Lambda. It'll bring up the resource you need to run the code and you're done. And so I said, how are we going to replace the image manipulation that happens in WordPress and take that offline and do that somewhere else? Before we looked at Amazon Lambda, we looked at doing it on our own server. We were on a VPS or an WordPress engine now. The problem with a VPS is that there are too many people doing the uploading. We couldn't go into all different machines that we're using to upload images and get everything resolved and set up so that every machine that we had was doing the image manipulation because, of course, there's a lot of image management that should be done on a server that can do a lot of image work. But our problem was we had too many machines and not enough people with the training to get that set up and run it. So I started doing what any developer does and managed it really well. How do I do this? How do I do this? How do I do this? And one would be cool. Thank you to Jean-Marie Steeves and Lavalat. There it was. The image conversion using Amazon Lambda and S3 and Node.js. So it was just, oh my God. And what Jean describes is something that's not new. There are other people who describe parts of pieces of this. Amazon itself and their web services even talks about how to do this. But Jean provided me the starting point of a Node.js file that would do the image conversion. I can take my images and upload the large size and the Node.js script will create a center optimized phone mail. It will also create a optimized larger size image that you see when you click. And you get that white box. So a little bit about what it looks like in Amazon and the Lambda. So I have this script. And I'll show you just a little bit of it. The full script that I'm using is included in my GitHub page. So if you want to go on and look at it, that's great. So I've uploaded the script and then I set a trigger. And you can see over here that you can add lots of triggers. And one of the triggers you can add is if something goes into a file. So I have a situation now where whenever content is loaded into the watching, the bucket, easy weddings, this trigger immediately takes those images, does the work, thumbs them, full size them, puts them in another bucket on Amazon that I sort my images out of. And so people, my photo editors can now go in and just simply upload the entire image folder and by the time they get over to the page to look to get that part, it's there. So we just went from hours and hours and hours to literally seconds. And this is with, with, with, with, and with Lambda, looking at that body and automatic movement. So just a little bit of what this Note.js file is. I've never done anything with Node and do a lot of the JavaScript. I've never done anything with Node. It's pretty easy. The, the first notice that I have on GitHub has all of the, the, the logging. So there's a lot of logging in this script to make sure things did what they were supposed to do. See the folder, find a lot of file name. So it really went down and I used that to develop this because I'm a big trial and error person, a big trial and error developer. So with a lot of notes and everything to pretty, pretty easily get this working, it uses image magic. And at the beginning of your code, just pull in a couple of those libraries. I get the name of the folder that was uploaded. And so that information that I eventually have to put in, I don't have to put anything in here. It knows the folder name I uploaded. It reads that acronym for each one of the images. And then I go in and I set two different images size, image sizes with it. It streams those images back out to the output bucket into two folders with the wedding name, thumbs, wedding name, large. And so there I have images that are ready to go. The final bit of this is to make that bucket on Amazon available on the internet with a web address. And in our case, this is where we did our C names. So I have what you do over here, but I have that distribution here that points to a bucket. And way over on the right, second one down here, you can start and see the word Bride. And I'll show you in more detail that I created a distribution. I told it what bucket to look in. And then I gave these awesome alternate C names. So you can, all from Amazon right here, set it up for the C name that you want to use. It's looking at that output bucket right there. And then I won't show it, but I went over to my DNS and then I created these three C names for the channel celebrations.com domain. So using three of the Amazon web services, the Lambda and the S3 and CloudCon, we were able to really take the image manipulation completely out of the WordPress loop and yet still bring it back in so that it really looks seamless. There's no, if you do know what was going on and you just looked at the source code on the page, you would never know. There's no extra scripting involved on the page unless you're pulling the library when that page downloads, it all happens behind the scenes. So I'm putting it all together. This is the process now of our photo editors. I'm using this product here, CloudBerry Explorer, which is a really nice tool for being able to FTP upload into Amazon markets. So I open up their Explorer, put that folder there and just slide it over and that's it. They are not. Once it gets into the watch bucket, Amazon Lambda, little script takes over, comes back, puts it in another bucket. I have CloudFront, which is serving up those images across a number of C names and then finally it generates the event card there and it's loaded. And I'm done. And I'd like to acknowledge Alex Harris, DJ, and Chanda Celebration, this is the company. I'd like to acknowledge Edward Jenkins, who I don't know, but who wrote the original event card again. Jean-Christine, Jean-Christophe, who's landed code and not that really helped me to see what's in the result of each. And then finally, I have a GitHub here where I put up some of the coding for the event cards and as well as that node.js script that will come. Now I'm ready for the next one. I asked you why you didn't just use a CVN, but that doesn't solve upload issues. So I'm not a questioner. Well, we had the CVN. So we had implemented, you know, an upload procedure where all of the images were being served off the CVN and then we even, you know, put a chart in place so it was coming from the CVN with different names. But that still wasn't, it still didn't solve the problem, because this is where this was never brought to my attention as it developed when I started this little project. It was all about looking at a picture or they can't look at their picture because it's timing down, script errors are calling me on the phone. I mean, let's face it. A lot of clients love a different show called Lawyer's Zillers. There's no television show called Dr. Zillers, but there's a television show called Bright Zillers and there's no reason for it. I'll tell you that. I used to build chemical plants when I was living and working in the bridal industry because it was way more high-stakes. They built the chemical plants. So, yes, that was the thing. We had a presenting problem once the bride and the users could use this, but when we expanded it and started to gather more data from the photo editors who were actually using it, that's when they brought it in to do that. Okay, we're starting to talk about all those error messages that the users are seeing, we're seeing them too when we try to create them. So... That question is because I'm very impressed by how hard you worked through and what you achieved with getting over this very challenging, this challenge that somebody put in front of you and they were one kind of yours. I wondered whether, first of all, whether you like doing, you know, work with lots of, you know, I don't know, objects, photos, whatever, and whether you've seen this opportunity to sort of be known for doing this and specialized or whether you just look at this as just yet another client challenge and this is how it always is and it's always been a challenge. Well, I know that there is, you know, I mean, for some of these things, and it bothers me to read, you know, people, you know, point about some of these ideas and things that still work very well. I'm not personally into the many-image industry, but I can see that, you know, with digital photography, it's just exploding. You know, so it used to be if you were selling products online, you pay somebody a lot of money to produce a few photographs of your product actually go to the studio in the right lighting and you're just the right, you know, and if you were lucky, all of your products were shot similarly, some of your websites, they all looked consistent and so that, you know, the time and the energy required to get images was just huge and that's just one of the biggest paradigm that I've seen is the explosion of images in this kind of content because you can in this case. So, yeah, I know that Alice has talked about this technology even in Jersey, so I wouldn't be, you know, making it like a part person who would want to do this technology in Michigan. There's not going to be any competition. The business where you have to hook up with the Amazon services, I have to be a little difficult, but really I'm really, really happy with the quality of the Amazon help. If you have a channel with Amazon console and you go into their help documents, they know that and so the help that they give you is actually for you. So it'll say, I won't just say I'll go here and put it in your whatever, it'll say go here and put this in because it's your login, it's the name of your, you know, yeah, yeah, yeah, yeah, but all the Amazon things that can be very confusing. There's a description of how to do this pretty much exactly in the Amazon descriptions there. I'm very much trialier, so I saw that John had this article that said you could do it and so without that I have to say I'm not sure that I could not solve this for them because and that's what you don't want to solve it for what you're trying to do is impossible. Well, no, it's not impossible, but for them, is it practical, is it cost effective? They pay $7 a month of Amazon in Amazon costs in Amazon costs it's $7 a month which is now they're not full you know, and they're not 40 megabyte, it doesn't keep any of the original Amazon news? Well, it doesn't have that one bucket still. Well, because these are images that are taken to build the albums and things like that. Yeah, so they don't need to be resized for a web and we're just putting them in there and then they're, we're optimizing them again. Well, no, they're being resized for the web because this, you know, this is the scale we're talking about. You know, one of the first questions when we started this was we're looking at WordPress as an image manipulator, I need you to give me some thumbnail I want you to give me a full size and some version that's centered because we are and I'm starting to say that photo editors couldn't do that even though they were photo editors and that's kind of what they did you know, to go back to them and there's a lot of them. That's the limit then how many people are you going to try to tell what you need to be doing your job this way and so finding the tool that will completely eliminate the need for them to do this and it's also not, not that you know, they're processing images to get them out make the wedding albums to make the videos and things. This is a side deal for marketing for purposes like that and so our ability to completely separate that out was really fun. Otherwise I really don't know what I think we would have just ended up going more and more tabs or locking it down. In fact, we had some ability here to lock it down in the kitchen on low bandwidth connection lock it down show them off because originally it was like well everybody puts about 200 images in each tab that would be fine so you know, two weeks after we get the solution you know, somebody has put 4,000 images in one tab and so you know, that's where unless we're going to get in to try to control what people are doing to make it easier for the developer which really isn't the way we need to find solutions that make it easy for the people who are photo editors in their job. So there's people coming over with their cameras and stuff like that stuff you talk a little bit about you mentioned that they have numbers of ranges like they know like these are the drives one and two so are they pretty much organizing all the photos at a time bringing those in or the first question that came to mind was like so they get all the photos and that people can find cool everything is uploaded good, is there notification like your trigger ran there's something wrong with it that upload all their photos again or they can easily go well that wouldn't well in this case there's something they've done really wrong it doesn't mean wrong because nothing bad really happens with all the answers if it's not there for some reason they've got the wrong numbers then they just go back and the original images that come from the photographers and the photographers we actually are using a different system for that they upload using smart files some smart files is a place that will post not post but a content storage smart file and we have a smart file now a few hundred aerobytes of images so what happens is the photographers shoot the wedding they upload their content to the smart file and then the photo editors they need to work on each job they download to their working workstations each job and then their job to look through the images and select the ones that would be in the event cards so that's what they do they select that group of images and then put them in a folder and sequentially upload them like we're using Amazon because they initially said we came out of our or was there a comparison between them they were using their own server they were doing all this in Amazon you can see that that was that wasn't going to work for even a few more months because of the volume inside so we didn't look at anything other than Amazon because this company had already worked with some other companies that did this and those companies wouldn't keep them up their perfection today we're working with some third party companies who would put them up for 30 days or 60 days or 90 days but in a fairly short time they were gone and so they wanted to change that would be there for a year or two years or whatever you wanted to come back to look at them and then like I said we've used them at the marketing tool to be able to show the bribes beautiful images that you have and your wedding at the Bluehand Golf Course is all the event places are catered for weddings and so they have nice places you know here's the the B.C. Bowl so here's the B.C. Bowl and things like that so they're really able to go right in and they're not responsible for which event they need to select so they don't have a dog they're not in that game but for some who goes to that location they want to be able to show exactly what it would look like a movie or a movie or your wedding right there it really does the trick like I said 20 million images and you know what I mean it's not a new trick starting I can tell you that people don't know what it is don't want to talk about and that was kind of like 20 million because you know we're going to do a start like what? not that word thank you very much