 All right, so welcome everyone to our business showcase. We're gonna be discussing personalization and Drupal, which is actually a topic that I think will appeal to almost any brand, any business, any industry. And so what we try to do is assemble a panel of experts that are kind of on the forefront of using personalization in a Drupal context. We have Lee Hammond from Interscope Records, Aaron Peterson from Time, Inc. and Mike Lamb from Pfizer. So sort of representing a good cross-section of B2C, B2B, and MEP media entertainment business. My name's David Many. I'm part of the Aquia Lift product team at Aquia, which is our personalization offering. What I thought I'd do is kind of kick off today a little bit around just the fundamentals of personalization, some things to keep in mind. Especially why Drupal is particularly well suited, sort of a more open approach to bringing data together and using that to more effectively target consumers, customers, and get people to write content and information. And then what we'll do is sort of move into a panel format. So in order to sort of tap into the brain trust we have up here, we'll have a couple of seed questions to kick things off, but definitely want to keep things interactive. So if there are questions that come up as we go through, especially in the panel section, yeah, feel free to use the mic that's in the center here and we'd love to have you ask anything that's on your mind. All right, so we're gonna start off with sort of a what is personalization, why it's important, and why an open approach really matters. And then we'll sort of dig into each one of the customers we have here and sort of get a sense of how they're using Drupal today, how they're using personalization, what kinds of things they're thinking about doing now and in the future. And then we'll open it up after that for questions from you guys. So to sort of starting off on a what is personalization, I think most industries, most teams across a variety of different businesses have fairly common challenges, albeit their uses of personalization may be completely different. But the fact is it's all around getting a quality volume of content produced and in front of the right type of user. As we know, people evaluating products on a typical website built in Drupal, there are technical evaluators looking for more technical content. There might be marketing users looking for the business value or the impact of a particular product towards their business. Serving them the right content and at the right time in their buying journey is one of the key reasons why personalization can really help. So obviously about delivering a great digital experience, Drupal's phenomenally well suited to cutting edge digital experiences. But it's also more than just the web, right? And this is another area where Drupal can really help take some of the insights gathered from someone interacting with a webpage and then be able to serve them content across different channels. So maybe it's a targeted email campaign, maybe it's interconnection through a social metaphor. It could be even shifting context to something like a call center. So when someone's searching for content on a website that they aren't finding, how do you be more proactive in terms of shifting the context from the web conversation maybe to a call center conversation? Because you know that person may call in for some additional detail. Segmentation is also an incredibly important starting point and an incredible important piece of success around a personalization story. So really helping understand what your audience is doing when they come to the site. It's one thing to say, people are logging into the site and we know who they are because they purchased products or things of that nature. But what about anonymous users, right? And most of the traffic coming to a typical website, these people aren't logging in, they're not identifying themselves, they're not filling out forms. But they may be doing research, they may be looking at products, they may be trying to educate themselves, they could even be constituents coming to a local government site looking for information. So how do we make sure that we can create an experience that's compelling to them, that's engaging for them, that serves them the right information? That's really one of the goals of personalization as well. So when I really boil it down, it's these three things and we feel sort of in this order that understanding your audience and sort of building up that unified customer profile about what your audience is doing on the site, what content they're consuming, how they got there, and anything including the real-time data that's available, for example, what time of day it is, what geolocation they're in, what kind of device they're coming to the site with. So if that's a smartphone, a tablet, they're coming through a laptop, all this stuff can be really critical in deciding what content to present them. The segmentation that I mentioned is key. Anyone who is looking at segmenting an audience typically looks at a known audience. So for people that have responded to an email campaign or have bought a product from us before, that's the easy group of people to segment. The anonymous traffic is even more important to segment. So really understanding who's coming to the site, who's your highest-value customers, what kind of content are you gonna present them? And once you understand the audience, you understand the segments, then you can actually start optimizing that site. So whether that's through more behavioral targeting, understanding what a user is doing, sort of anticipating the journey that they're on and then presenting content to them at the right point in that journey, or doing something like running an AV test or a multivariate test to sort of test content and messaging and to see what resonates with that audience. So those are kind of the three pillars of personalization that we see. So you guys have probably seen the Ice Range chart, Scott Brinker's famous for the Chief Martek Technology landscape. Bottom line, there's an absolute ton of marketing technology that's available, roughly 1,800 different systems and platforms that are available today. And Drupal is incredibly well-suited at connecting with these systems out of the box, right? Because there's generally more modules, more agility in the Drupal community to connect to these systems and the series of modules available to connect to these systems is freely available. But then when you go into an omnichannel world, again, looking at the website and personalizing the website, but then looking at other channels as well. So while Drupal is incredibly well-suited and you can do a lot at the JavaScript layer, interconnecting different data sources to make choices on the website, it's also really important to have rich APIs and the ability to do sort of system-to-system connections, things like marketing automation tools, where a lot of demand-gen and lead-gen marketers live, CRM systems, which are important to the sales force. It's your record of your customer base and bringing the data from these systems together to build an even richer profile of that customer. So starting off with basically anonymous traffic, moving through to a known user, and then augmenting the data from existing systems that you have. That really sort of builds the richest possible profile. And the advantage is that all of the teams within your business can rally around that single integrated view. I think digital marketers traditionally are closest to the website and having a good understanding of what content is consumed on that site is important, but there's lots of marketing teams. There's lots of people that have a shared interest in what that customer is doing, including your sales force. How do you get a shared view? It's by federating that data together in a single spot. So just sort of looking at the journey, right? Typically, customers coming to a website are gonna be clicking on a content, coming there via keywords, coming there through search engines, maybe responding to an email campaign that you send out, but they're still relatively anonymous to you, right? At some point, they're gonna sign up for a newsletter, they're gonna add their email address to a form for more information, they might buy a product. When you have some unique identifier, they kind of move into that known world, and you can kind of merge the anonymous tracking that you've done of that user with their known identifier, which you can then use as a key into the other data sources that you might have in the enterprise, right? And using Drupal to bring that information together. So it's all around building that unified customer profile, and as we go down from your most traffic being anonymous users to the more detailed traffic at the bottom, it's basically that continuum that personalization can really help with. As I mentioned, it's not just the website, and it is really, really important to make sure that you're grabbing the insight from all the channels and the touch points that your customers are interacting with you around. Sure, the web is probably where most people are doing their primary research, but again, are they coming through mobile devices primarily, is there a certain tendency or a time of day or a certain point in their journey that they use a different kind of device or interact with you in a different way, and how do you optimize the experience to that? So if you look at a typical journey map, I mean, these things are pretty complex, right? They're multi-channel. There's a bunch of different touch points that a customer will have with your particular site and with your particular brand at any given point, and bridging that anonymous visitor traffic with the known user systems that you have is really sort of the goal that personalization has and why Drupal's really well suited to sort of bringing and bridging those worlds together and those channels together. So the unified customer profile was the first pillar. The second piece is really around segmentation. So this is really understanding what your audience is doing through the behavior that they exhibit, because again, if you know who they are, that's one thing, if you don't, because they're not telling you who they are, it's only through behavior that you're gonna understand what they're really up to. So being able to track some basic situational data, where they are, what time of day it is, things like that, behavioral data in terms of how often they come back to the site, when they come back, how long do they spend, what kind of content they look at, what kind of interest do they have, and bridging that on to more of a multi-channel profile in terms of what else are they doing on social channels, what do we know about them from our CRM system, had they contacted our call center for help previously, things like that. Then the third pillar is optimization. So once you have the data collected, once you have an understanding of what your audience is doing and what sort of segments they fall into, how do you use that data? How do you make it actionable? So it's all around surfacing that information in the analytics tools that most of your teams are probably already using. So that could be something like GA, Google Analytics, that could be integrative reporting within the Drupal environment, or it could be direct access to the data that's being collected. I think that's a really important tenet, is keeping that data open and making sure that you can access it in a unified way is also really important because otherwise, while we're sort of breaking down the data silos of where all this customer information is, you don't want to have reporting silos on top of that. Because if you have to look at 15 different places to figure out what your customer is doing, it doesn't make you very agile and doesn't make you sort of able to take advantage of the best opportunity. So there are different reporting and analytics strategies, but these are the ones that we definitely see, we see most often. And then of course, if you understand your audience and what they're up to, then running a directed test, like an A-B test or a multivariate test, definitely something that a lot of people are looking to do in a personalization context. But running a test to a general audience, generally not that effective. Like asking the world, do you like this better or this better? Generally doesn't give you a statistically significant result. But once you understand your audience segments, if you run tests against those segments, then you're generally getting a very relevant result and usually much quicker as well. So that's kind of the process that we see most of our customers take when they're tackling personalization, especially within their Drupal site. So I wanted to pause there in case there are any questions just on the fundamentals. But what I wanted to do is sort of shift gears to the panel, introduce our speakers and sort of dig in more around how they're using Drupal today. And what they sort of see the promise of personalization within their respective companies. So if that's okay, any questions at this point? Nope, great. All right, Lee, I'd love to hear who you are and what you're up to. Can you hear me okay? This is perfect, thank you. So I'm Lee Hammond, I work at Interscope Records. A lot of artists I'm sure you know, Lady Gaga, Eminem, Mermin V, Imagine Dragons. So I manage the official properties for those artists. Again, Eminem.com. These are umbrella marketing sites and more and more becoming direct e-commerce sites as well. That's something we're adding to the mix. To sort of speak to, one of the problems that we wanted to solve at Drupal was the managing a big portfolio of sites at scale. And this is a high touch, high brand business. So each of my artists wants a snowflake and I need to make it appear like I have to have a machine behind it that makes snowflakes. So I do feel like Drupal, multi-site Drupal and some of the products we use from Acria make that a possibility in reality. And so now that's sort of the framework as Acria rolled out personalization, that landed with a couple of other things that we've been spending our time on. I mean, honestly, the music experience for fans is so fragmented that the artist side is sort of a catch-all which should be a good thing, but oftentimes we don't really know why you're here. Are you here for the tour? Because again, and in fact, not only is it a catch-all, I work at a recording label, but I'm one of the business partners to an artist. We sell her Lady Gaga's recordings, but she has a separate merch company and a separate booking agent and endorsements. And yet we are the emissaries for her brand and fans want to come find out a lot of different things there. So in some ways I'm using Lyft to just ask a basic question, why are you here? Do you want to buy something? Do you want to... And also I want to... Do you want to buy a t-shirt? Do you want to buy tickets? Do you want to look at some videos? Do you want to look at some photos? The learning's from that. Then combine that with, I also want to tell marketing, what you want them to do is maybe not what they want to do. And this has allowed us to say, we all know that one of our revenue streams is when they buy the new Alameda iTunes or context for these users is they're coming to an artist site, they probably, what I've learned is they probably know where iTunes is, but they're coming to the site for a different experience. But I have to tell, even internally, and I think this is going to be just in a little bit of conversations here with Lyft cohorts is that it's as much about personalizing experience for the user coming to the site as it is teaching our internal marketing teams. You've got new tools here. You have to think differently. You have to, you know what, and so for me, I just want to make sure, but I gave you for slides is what I'm talking about. It's good so far. It's good so far. Yeah. So those are the kind of my learnings. And then, and we're starting with very lightweight cases. Like I said, why are you here and doing multivariate testing? Is this working? And I'll give you a couple ad hoc examples where we took a carousel of hero banners that were loading up and turned them into persistent randomized testing. And the one that we had thought was going to be the winner of the iTunes banner on street date. Nobody cared about that. The video that was two months old was still the winner. And that sort of surprised me. I guess it's media, but we thought, well, that video's played out. So I like to say to marketing, this is sort of having a dance before I want to come into the kind of, they'll buy, and again, we will all make money when we have more fans engaged with the brand. We don't have to draw a direct line and quickly have them leave the site. So I need more data like that. I think we're still in early days here. And even in the case where the number one thing is driving something, I then want to say, well, let's try three different creative. Let's figure out a different way to work with that. And then when it comes down to the next level, personalization is great. I know where you live and I know this artist is coming there the next two weeks. Here's a concert, highly relevant. The more we can extract the name of the local venue, even better. And then because we know a device, we know that an iPhone user's gonna be an iTunes user and an Android user. Well, most likely it could be a Google Play user, but we can customize in that regard. And then as we get that anonymous to know, we know you're a male or female. That's where we could have some, this is definitely like a walk, run, and I feel like this is at the run. Okay, we're gonna get creative for male or female. We're gonna do different things because we know that. We are also doing something that I'm not gonna spend a lot of time on, but when we do register users, we are registering them with social profiles. And that can be Facebook and Twitter. And then we are expanding on that insight about that user and asking their permission to say, can we look at other artists you like on Facebook, on Twitter, and that's giving us a really 360 view of you as a music consumer. I know you came to this artist's site as an Eminem fan and I could maybe expand genres, but if I look at your social profile, I'll know exactly which artist and it will definitely go beyond hip-hop. And so I have a one-to-one you like these artists and then Interscope as a music company and Universal as a big parent company, we are using that data to talk to music consumers and make sure that we're putting in our music and our artists and staying relevant throughout the cycle. So that's, I would definitely say that's the most aspirational part of that, that I did find some affinity with one of your guys in that we are working with very narrow artists sites. So there is limited age, I may know you love Katy Perry, but I can't really use it in this context. I'm sure you have a lot of drug brands but you can't cross-promote them. Right. But anyhow, these are all sort of, these are nice problems to have and they're down the road here on the piece. That's probably the low-hanging fruit for us on that. Yeah, we can dig in a little more when I give us a little bit of an idea what Pfizer's up to. Sure, so on the, actually I'll do my Drupal intro. So Pfizer have been using Drupal as our standard technology for building web applications for about three years now, at the beginning of 2012, we said this is just our standard now and at that point we had about 60 systems that we consolidated. So we spent a lot of time just consolidating from different continental systems getting to one platform. Again, just like Lee, people want snowflakes. We built about 1200 assets over the last, actually the last two of those three years and we tried not to build 1200 snowflakes again, right? So a lot of efforts have been going in to try and consolidate this and convince brand managers that the thing that you want is actually not completely different from the thing somebody else wants so you can put these newer capabilities in place and then use them. So on the platform itself we've benchmarked to be 40% faster and 60% cheaper than those previous approaches. We consolidated those platforms and now we have 86 brands and 40 markets on Drupal. All right, so this is a really good foundation for us to say we have this capability. We saved a bunch of money by consolidating it. We can build websites really fast. Let's take it to the next level and build some new capability and of course personalization is top of that. What I'd say is kind of similar to what you were talking about there is we built 1200 properties, right? This isn't Pfizer.com one big property that's on Drupal and that's it. That's 1200 individual properties and then a lot of content within them. So if you imagine if you're a customer of Pfizer we have lots of different types of customer trying to navigate that to get the right content for you it can be challenging. So that's what we're trying to address here is if you want to come to us or if we're marketing to you at least make sure that we're getting into the right place and we're getting the- I was hoping you covered that one. So focus on far greater relevance of content for the user. So an example would be if we're doing email marketing, right? So if we're marketing content to a user we want to of course focus on making that relevant and personalizing it and that's not a DMI rather than whatever it is. It's really use the data we have on that person to make it relevant. If we're sending someone an email we probably have some data on them already. We have some things we can use if you have that there's a whole bunch more you can do than if you of course we're talking about just into the first stage of what we're trying to do is simple things that users are going to expect. If you receive an email from Pfizer and we've managed to do a good job and personalize it for you if you click on one of the links in that email unless you're talking about this different systems you're crossing across here don't let them click on that link and then see something completely irrelevant for them, right? That's not going to give somebody a good user experience and sometimes crossing across the different systems can be challenging but using some of the technology we're talking about here you can make that a lot easier so figure out that get the content to them and then so multi-channel lots of different channels we can focus on. Right now we're focusing on web with Drupal because we have a lot of Drupal and we were doing email marketing so these are two focus areas. Pfizer have lots of different customers we have healthcare professionals who might be prescribing our products. We have patients who might be taking those products we've got caregivers of people who might be taking one of our products we have chapstick and audiences and different types of users we have chapstick being at the one end of the scale someone might turn up at chapstick.com from the Facebook page we don't know who they are to visit our web properties and pharmaceuticals, you have to so that actually gives us it's always been a challenge it's been a huge barrier so if somebody wants to visit one of our websites in that situation they've got to really invest in registering improving I am a professional healthcare professional before seeing the content so hopefully now we can take that and say okay you've invested you've given us some data we can actually use that to say once you're in we can actually give something that's relevant to you rather than you're in and now you get to hope that's the right property that you're after and I guess just to round out things here Erin what is your perspective from timing what are you guys so at timing we are roughly where Pfizer was 2013 we started just over a year ago moving all of our worldwide sites to Drupal we also have 86 brands worldwide the 86 brand club of your the 86 brand's library teams are four different locations and for us moving to Drupal means that we have a consistent framework with which we can which we can approach the experiences we're building for our audiences our editorial teams we believe very strongly that our editorial personalization at time we'll go over to Sunset and Rugby World and this audience most people in this room like raise your hand did you read Time Magazine when you were in elementary school experience that people are used to coming to major news weeklies to get Sunset user and you're going to come and look on Sunset content it's very very useful for you if we know the temperature zone that you're living in or if we have some of the basic weather information so that we can serve up content that is incrementally more relevant for you and for us personalization takes the form not necessarily of how can we sell you more stuff but how do we craft an engaging content experience that really to engage more in discovery and less engagement in searching for things and wants of their fingertips and put that right in their CMS makes it easier for our editorial team to create experiences and curate experiences whether those are content whether those are video or whatever files and images that they're creating and get those penances who really appreciate and personalization today audiences and that helps us paint a picture and understanding of who our audiences are both as individuals and as groups and the challenge for us now is how do we take that understanding of who our audiences are and then apply that word giving them that content experience and that very personal and very rich way Yeah, that's great. So we're all at different we're all at different points of entry kind of to the personalization story so I wanted to sort of kick off the panel with some of the results or early results or at least benefits or impacts that you've seen personalization having so we've talked about some of the things that can help sort of stitch data together and things like that but what's what have you seen has helped you guys move the needle in your respective businesses or what do you anticipate it'll help most with and anyone can jump in and sorry Lee So I'd jump in and say excuse me when we started out with personalization we were very excited when these products started to mature especially when they were built into Drupal we were very excited about the potential and when it spoke to lots of people about what could happen with this and we read a lot of research and there was one I hope I can quote here right a piece of Juniper research that said an email that is personalized again is 19 times or drives 19 times more engagement than the one that's just the last email right so we were quoting some of these statistics in the case and then we tried a few pilots and there was this bit in the middle we were very nervous of like okay we've quoted that this is what could happen and what really does happen and we it was representative of those numbers a lot of interest in continuing this it is hard to do like we were talking about the market is and their abilities to understand and implement this technology so it's hard to do on a large scale but the numbers that we've seen are definitely in line with published research on some very impressive Fantastic actually on that front I don't know if you guys have anything else to interject around results but one of the things I want to dig into too is just around the sort of the culture of optimization or getting teams to kind of think about how to do this on an ongoing basis because I know if it's just to set it and forget it it's probably not going to drive any meaning from results over time so it's really doing it in a prescribed iterative way so I'd love to hear more about mechanisms to allow teams to engage in personalization it's not a matter of building a feature and just shipping a feature to you for example editorial teams a lot of times editorial teams here personalization and what they interpret that to be is you're going to have machines start to figure out what content goes out in front of audiences I think we've all been in a position where we've had an algorithmically generated assessment of what content we want to be in front of us at any given moment and it isn't as powerful as the assessment of an editor in terms of this is something that would be really engaging for this audience or this swath of audience today I'm just going to make a joke because I know how much I would imagine if I'm at timing these really respected news brands and you hear personalization you're going to go to BuzzFeed and Buffy to post which means that are you just going to ask? Oh sorry, speak closer to Mike Sorry, I was just saying that if I was a journalist from timing and I hear personalization you're going to take us down the BuzzFeed and Huffy to the post where we're just going to append side boob shot to every headline and massive click through I'm not really known for that but we can run through it No, no, no, I know you're not I'm just, I'm sorry if that's an off-color joke I just mean Huffy to post started as a legitimate news publication and then they just followed the consumer to their lowest common denominator and yet there was something right in what they were doing Yeah, absolutely It's just that there is a line here about where we can't ruin the brand with chasing that Election results that are personalized in the sense that they're derivative of my local not necessarily somebody shoveling content a personalized feedout This is stuff that's going on righty-faster on the draw I'm getting the boys in the chat Side boob Let's be correct No more gender gender Read stuff from other humans That's the point of communicating So I would completely agree with that Yeah, and it really, but it really does speak to the capabilities here our headline, personalized headlines, headline testing and if the criteria is clicked through And again, I'll give you a different example from my world The, and I'm curious how social media and Facebook and Twitter and those teams work with this because in my world, they're all the same team In music, there is no, the digital marketing person is wearing so many hats the digital ad spend, the owned media, the social media And what's often challenging is that they want the workflow to be as easy to publish on Twitter as it is on the website And so Drupal, you mean I have to do A.B. testing and this and that So I'm starting to think about two ways of handling that where we do have analysts and some people who do some additional analysts work and bringing them into augment the digital marketing team that's overtaxed as it is and say you wanna run a test let us run a test for you give us the content you were gonna post on here and we're gonna try some variations for you and report back because I'm adding basically a human service even though they could do it themselves I think this is the crawl phase to get them to go we can do things differently here Have the patience for this answer So a great example you brought up is Facebook So not only are audiences fatigued about having months or so and making sure that the content that users see in front of them lies in sensible and reasonable content from the perspective of time we generate and do see circulate through Facebook and so it is very valuable us that they're able to create a piece of content in CMS push it out through any distribution mechanism and do so in the way that it's effective Is that part, you are, well I gotta ask is that through some of the lift products or is that just sort of your workflow but that's really great because to me these are separate these are repetitive tasks log into Facebook and publish here log into this, log into that We're using tools that are easy to de-fection is that just as our needs are maturing the technical capabilities of... So and I don't want to... You reminded me where I was going with this which was simply that... Sorry, no, I'm sorry to... I hope this is interesting, this is... If it's not, we'll buy you all... Yeah So it... There is a part of me that says you know what Facebook does is all our marketing people put in their Facebook post and they pray or they buy audience to see it and Facebook determines whether it's relevant or not based on any attention they see and there's a part of me that just like we're doing with those hero banners we're saying let's load them all up and see which one is the winner when it comes to the other content the news items, tour dates, the videos I kind of want to say load it all up in the same with email and we'll let the algorithm figure it out which takes the onus out off of them of the marketing person thinking about two different headlines multiple creative because it's making their head explode and I don't know, I think that's underutilizing the capabilities of headline testing but in a resource taxed environment there's an appeal to me of following down that trend that Facebook is doing which is, and even unless it's not just Facebook you buy search advertising on Google but they'll, if you're not getting clicked you're coming, they won't take your money so that goes where I was going with that. It's sort of an information retrieval problem at that point, right? So that's less about content and that's more about information retrieval if you guys go into your mapping so there's some degree of... So it looks like we've had an audience member patiently waiting so if you've got a question for us we'd love to hear. I work in healthcare so I have a question for the gentleman from Pfizer. Our user testing has told us that most of the people coming to our site and looking at content are looking for a family member a parent, a child so it's not necessarily for them. How do we personalize? How do you personalize for caregivers or parents or, you know... So we have entire sites where a parent or a caregiver is the audience especially if you have a pediatric vaccine for example the entire audience is either gonna be a healthcare professional or gonna be a parent and I would say it's not really different apart from many of those websites because of the type of content that we have is so focused depending on the use cases where we've kind of done personalization in all technologies where you turn up and the first page is who are you? Are you a healthcare professional? Are you a caregiver? Are you gonna be taking this product yourself? Because of content in the other areas it's just completely irrelevant. What I'd say is we can completely change our approach to how we build those kind of sites so rather than having just an information architecture that drives you one of these three ways we can cross things across those three or those multiple categories using this kind of... It's a good example where you have very, very different types of content depending on the user. So personalization is critical because you show them something that's just completely irrelevant and sometimes completely inappropriate for that person. And you start with email to get them there or? It depends. So if we know the user then obviously email is a great method. And maybe we even know if we have their email address maybe we already have some data. For these kind of websites it's really people just turning up so we might have an idea based on what they're searching for, for example. It's really gonna be when they've heard about a product that their child's gonna have maybe some immunization schedule or something like that or it's a condition that they're searching for for a friend. You can get some idea of that through search results. Any other questions? Do you want to tackle this one? There's one there. Not had issues asking for those permissions. And again it's, in general that's not been an issue. Well for a while in fact when we first started this capability we saw the value in that social signal that we almost slipped too far we didn't offer it alternative way to create an account and then we added traditional login. Meaning you want an email password combination and no, so that's depending on the artist in the site it can vary and let's just say it's about a one in five prefer to go down that path. So that's and then as far as storage we are federating that across all of these across universal and that allows us to do central identification across not only the inner scope portfolio sites but other technologies, CMSs, email service providers and what we're doing with that is basically creating a LeHavn profile his top five artists that can go to the marketing system that can use it. So exact target can then say and we can create dynamic custom email there and I think that in terms of a walk, run or a crawl walk, email is really well suited for personalization in some ways. A lot of the context is easier to talk about multiple artists that don't make sense on Lady Gaga.com and there's no runtime challenge of I'm gonna go through this big I think you guys are solving the runtime challenges of lots of decisioning to draw that page but a year or two ago it was like you nobody has to wait for that email to be composed. Did I answer the two questions? Yeah, go ahead. This is kind of related it's more an ethical question than anything else. I guess given social login the information you have access to with social login I'm not really sure how much of opting in anymore is really informed consent. So I guess my question is really how well do you draw the line at some point between what's kind of creepy and not creepy in terms of the information you gather about your visitors and where do you draw that line if that does come up along the way? Dris actually did a presentation on this in his keynote at Triple Con Latin America right and I think he had a slide which was like the line is don't be creepy and I know that doesn't like to define the line but it's a major consideration here as we've got more into this kind of technology and understanding this kind of technology it's kind of frightening what you can do with it so it's a massive consideration as you implement this kind of technology and being very respectful of the data that you have and making sure that you're using it in a way that's really gonna benefit the customer rather than- This is one of the places where I'd help system normally enormously to be a global team about a third of my engineering team comes out of London, comes out of privacy and Europe is very very different than the information they're looking for whether it's sports scores or ships so getting to a point of age for social login often the opportunity if they want to build that relationship with you in its first perspective I was gonna say that I had just an anecdotal conversation when we were illustrating our technology to one of our product managers and there's a, I should preface this there's a company out there called Songkick and if you've used it it's you provide a social signal and it delivers concert alerts to you based on when those artists and looks at your Facebook likes or your music plays and it's a pure play around concert information and I basically without naming them I said we can do that with albums and merchandise and we could take the signal coming from Facebook or this and the product manager goes that's creepy and I said yeah that's creepy but what do you think about Songkick and she said oh I love Songkick so you know I do think it's personalization is a service it does involve some intrusion and I think you are trying to you have to present it as a service and while we don't have all these probably not as much polling and resources at this time every time we do a personalized target email open rates go up and more importantly opt out rates pretty much go to zero nobody feels like that message is not relevant to them or stop telling me about this I can tell you the anonymous emails where you're on a mailing list and we tell you about 20 things and it's just got big opt out rates stop telling me about stuff I don't care about so I think and I think things are people are maturing to where they expect personalization so and people are maturing to where they are creepy about it the other day I was in a Slack I'm sure half of you are using Slack and somebody had decided to spam every Slack channel with the same post I was like seriously so my question was around tools and also make versus buy so which tools did you folks evaluate when you were thinking about personalization and what was the process and what did you eventually end up selecting so we got quite excited about personalization and A-B testing when we started the digital platform as I mentioned that was three years ago so we were into it it was one of the things on the list of people might be interested in this and we tried a couple of tests with one consumer brand in the US where we took a multivariate A-B multivariate testing tool and then tried to implement it with this brand and it was separate from the content management system so it was the same I was going down the path of give these guys access to control their content at the same time as us try and personalize it in a separate tool and creating and publishing content is not easy it just isn't creating and publishing regulated content is hard as well and when you have your content then suddenly distributed across many different systems it becomes impossible it's just it's so, so hard to do so we did this experiment and it kind of worked as in like the results from the A-B test were good but it was so hard to achieve it that we kind of put that back on the shelf and it wasn't until much later on when we were working with Acquire and then there was lots of conversation about like Acquire called out like a shadow CMS so you got content here and content there and when we learned and had the content the content creation workflow can be the very same workflow that we're pushing teams to with move to start from the first place it was like okay it's time now we can actually do this and this is going to be possible so we've been kind of farm environment with the websites you guys are so highly regulated with what you can say and what you can't say how are you getting past that to do personalization because from my experience you got to go through a two-week regulatory review before anything's for change on your websites so and then how do you get that submitted past that area if our regulatory process were two weeks I think we'd be very very happy with that they could be many many months at that time and it's a key question to our success here and what I would say is when we're implementing this technology it's not just about picking a technology and then putting it in place you've got to look at the entire picture of what needs to happen in order for a user to use it and that's how creating and publishing content is hard creating and publishing content in a regulated industry as you said is very very hard and it was I see this the same challenge as we had when we went down the path of responsive web design because everybody was expecting a website looked like this and that was approved and that was fine and then suddenly you look at it across different devices and it looks very different and that was difficult for us to get our legal team to understand and figure out that you can't change this the website is going to look different on different devices and get tooling in place to be able to support the approval knowing that some very important things like this text is no longer right next to this text it's below it make sure we're all comfortable with exactly what's happening there and that required some tooling to make that work we're doing the same thing with personalization so we're investing in some tooling to get ahead of this because it's going to be critical we can't say, hey, this is your tool go create five different versions of this content and we're going to mash it up and expect that to be a responsible thing to do there's a lot of work behind me this is actually back to the kind of the I heard the kind of build versus build versus buy discussion earlier on this is something we're building for ourselves because it's a very hard problem to solve but we're making some I did, like I said, I spent some time with your team last night and it would be amazing to me that Chapstick or other Pfizer products have so much in common with Lady Gaga and Eminem because you're scrutiny even forget the regulation you have strong brand managers not protecting the brand and they're used to approving this is it not these variations are and when I demoed these capabilities with Lyft and personalization, my product managers again, we're emissaries we are one part of the ecosystem so we manage Lady Gaga and Eminem but there's a manager and they're like, okay I've got to go to the manager and get them to approve this and so we're all looking at this workflow as being a challenge but I do think that's where we are today you go a year out and it's like I'm going to hear why don't we have this now or what can we do you know, we're just everybody's going to expect it and the forward leaning managers know this they know that there's testing and targeting through all other parts of the ecosystem on Facebook, Twitter, commerce so they get it before I even finish the sentence we're going to do this but there are other people who say I don't want two different pictures and we have the same issue with Responsive yep and you can't ignore these things so with the true story on Responsive is when we first moved to Drupal we installed the Drupal on one site at the very beginning we installed the Drupal module mobile tools and people might remember this is a module when you have the m.websites it would redirect you if you were on a mobile device and we used this module to redirect people on a mobile device to a page that said please don't access this or you can't access this page on a mobile device because the lawyers weren't comfortable with how the content was placed and it was for some sensible reasons it's a kind of response but just not the response responsive right if you had the spectrum what we're talking about here given a user a great customer experience I didn't think that people probably don't think you could get that far down the other end of the scale of what we're doing to our users in that case so that's where we had to it was great for us coming to personalization going through that experience so we don't get to say no to this we don't get to switch this off for the internet we've got to make this work so we had to solve the problem it was a hard problem to solve and we solved it and it didn't it meant that when we were talking about personalization this was top of mind of how we're going to deal with this and we're kind of doing this journey at the same time but you can't put personalization in I think this is I don't know across other industries but in a regulated industry it's very very difficult so that's why we've had to kind of in parallel with saying okay it's much easier now with the personalization tool than being part of the CMS to implement that we've been okay let's go solve this other hard problem which we maybe have to solve for ourselves is we haven't seen like the the market respond with some solutions to those kinds the trick there is and the trick with the platform has been it's I like to say it's been 100% carrot 0% stick right so we didn't get we didn't say okay FISA have 60 content management systems there's this like call to say it makes sense for us to consolidate this you must do this that wouldn't have worked right we'd convinced ourselves it was the right thing to do but it wouldn't have worked so our success has been based on the fact that the brand teams want to come and work with us so you have to be very focused on not checking a box and saying hey you've got personalization now but give it to them in a way that actually works which means you need to consider the entire the entire process and there's sometimes there's gaps in that process that you've got to go and innovate to plug we're almost at the end of the allotted time here so I just wanted to pass to see if anyone else has any additional final thoughts or questions otherwise I guess we can we can wrap up yeah go ahead you need a third party cookie to make that work and in Europe most countries you have to have an opt-in to have a third party cookie which means users don't opt in and they don't say yes which means unless you accept things out properly you've got nothing right you've got no statistics on the user and I think it's good to kind of have that in Europe and kind of if you're building a global platform to keep you mindful of what isn't creepy for the other markets as well like there's a good reason why people are saying like don't be doing this because the users have no idea that you're doing that mostly worry about the US what's funny to me anecdotally is I've had these tough conversations with Germany which one of the most strict about what we can do and then I go to their properties and I see Facebook widgets and Twitter widgets and follow and like and so I just I'm sometimes frustrated that that person you know they're delivering data to Facebook and Twitter and not to the company of the profit they're trying to market with but that's I understand the reality there it's just