 So hello everyone. Thank you for coming and welcome. My name is Scott Truett. I don't speak German and my grade school level French is awful. So I also tend to get excited when I do these things. So please let me know if I go too fast. I work in product at Pivotal. My colleague Mike Goddard is the true talent on stage. He's going to show you everything that he built to make this thing real. Just out of curiosity, do we have any retailers in the room? That's what I figured. The answer was no. And then I'm curious, do we have anyone that's played with the Google Cloud data services? I know Eric probably has. Anyone else? Yes. Like vision API and natural language processing API and TensorFlow. Those are all things that we used here. What about Spring and Spring Cloud Dataflow? Nice. Any Python developers? Good. And then has anyone used Amazon before? Good. Awesome. Okay. So I'm going to talk about just kind of briefly introduce our work. Mike's going to dive into the technology behind it. And along the way we'll talk about retail in general, personalization, data processing, image analysis, sentiment analysis, machine learning, and more. First up, Mike and I work in the data innovation lab at Pivotal. And we're in the business of exploring new ideas around the intersection of agile, cloud, and data. And we love data native apps. So much of what we do today came from the work that we're about to show you. And this project began late last year as a collaboration between our friends at Google and Pivotal. We wanted to demonstrate what was possible with our combined offering. And we chose to focus on retail as our first use case. And of course you can't talk about retail without talking about what Amazon has done to change it. And because I work at Pivotal, I know a thing or two about what they built, that machinery that makes it all possible. And really what we're talking about is software and data. And relentless testing, iteration and refinement. A willingness to experiment and get it all wrong and try again. And most of all, they do this over and over again every 10 seconds, as they famously say. And I also spent a lot of time in the early days thinking about my experiences as a customer of Amazon, how it changed my shopping patterns and expectations. And I always go back to one of my favorite books that I learned, that I read when I first started out called Don't Make Me Think. If you've never read it, you can probably figure out what it's about. But the main point is that thinking is friction. And thinking is a roadblock. And thinking is even a missed opportunity. And what Amazon, I think, figured out very early in the game is how to remove those roadblocks, how to get you to the point where you don't have to think. The expectations are clear. Just the progression is known. It's really simple and straightforward. So I think that's also the magic of Amazon. And really I think this quote from Jeff embodies that sentiment. This customer service and a better customer experience is the key to competing in retail. And it also seems like a really good challenge for our experiment. So let's make that our thesis. And let's tie it together with the fact that a good customer experience is increasingly driven by software and data. And that software is a reflection of your brand, your stores, your people. And uniting all three is a competitive advantage. So hard stuff, but really fun. And we started this before Amazon bought Whole Foods and completely upended many of our expectations going in. But I think what's interesting is that legacy retailers, for lack of a better word, are closing down their physical locations. And yet Amazon is investing heavily and opening new ones and really investing into Whole Foods. So one of the challenges that we embraced was finding ways to bring that physical and virtual link together. And as we went down this path, we realized that we not only needed to think like a retailer, we needed to become one. So we made one up to make it more real. And this is boho chic. So in our experiments, we really focused on the art of the possible and trying to find, you know, an experiment we could run and learn from. But the one we're about to show you was just one of thousands of things we could have done. It just so happened to be the one we chose. So one final piece. We also made up a customer. Her name is Mia PCF. She's loyal to boho chic. She shops with us often, but she has plenty of retailers vying for her business. So with all this in mind, we posed a very simple experiment to run. Let's say that Mia tweets about a party this weekend and her desire to find something new to wear. Can we find the perfect dress for her? Can we make her an offer she can't refuse? And can we do all that with software and data alone? So a very simple tweet and a very simple offer. And here's the original mock-up made after the fact, of course. But this is the idea. Again, we want to use her data to build an understanding of who she is, what her preferences are, and what her specific need is at that moment. We want to match that as a matching item in our catalogue and then find it in stock at her local store. And we want to send her a personalized offer to buy it now in that store for 30% off. Simple, right? So we're going to try a demo that I didn't practice, but how hard can it be to send a tweet? So I'm logged in as Mia, and I should let you know that Mia and boho chic follow each other. We had a very lightweight integration there. There's no I'm open to offers. It's simply a reciprocal relationship. And by virtue of me tweeting with an image and sentiment, we can respond. So I'm going to show you. Yes. Okay. Okay, brief pause. I need to queue up a couple of things so it's more convincing. Okay. So these are our apps that are running. It's a microservices-based solution. So one thing we'll do is we'll tail the logs on this, if I can type Twitter. So there's a Twitter app involved. And then let's see if I can see where my mouse is. It's over here. And let's tail the logs on one of the other apps there. The fact that the app crashed there will come up later. Okay. So there's two apps. We're just going to look at the logs there while Scott goes through the demo. Okay. Thank you. All right. So I'm bad at this, but I can say something like love this dress and want to, whoops, I can actually see what I'm typing, and want to wear it to my best friend's party. Does that make sense? Good enough. And then I can put an image on it that there it is. I'm going to attach a picture of a dress and I'm going to tweet it. And then roughly 10, 15 seconds later, I'm going to get in. Oh, yes. So, okay. So the Twitter app is polling for, well, it's supposed to be polling. Looks like right now it's in a dilapidated state, which is unfortunate. That's something I talk about later. Let me see if I can find that. You're going to look at the logs live. Okay. It's back. And let's see right here. Okay. There's our data science app, which did some work. And now if we go back to the UI, do you have a notification? Sure. So sure enough, there's a notification. I just heard it on Mike's phone as well. So 13 seconds ago, we get this tweet from Bo Hushik saying, hey, we've got some hot new dresses here, a link to our store, and custom code to get 30% off if you buy in store today. So a 10-second demo for a 30-second talk track. Fortunately, Mike is going to explain to you how that's possible. So you want to take it away? I'll let you. Okay. Great. Thanks, Scott. All right. Go back to our slides. Okay. Okay. So it just so happens that everything we needed is available in Pivotal Cloud Foundry running on GCP. And in order to do this, I mean, we first came up with actually there was somebody at Google, a gentleman by the name of Duncan Rutland, who kind of provided the impetus for this whole thing. And his original architecture diagram was actually quite a bit more complex than this. We sort of whittled it down to what we could run with in a short time span. But the idea is that we didn't have to ask for permission to spin up any resources. You know, all the resources were readily accessible, self-provisioning. And there was really nothing to deploy because we have, we'll talk about that in a minute. But architecturally, we envisioned a kind of a left to right data flowing from on the left hand side, social media data sources. In this case, it's just the Twitter one. And then traversing through this pipeline until we got to the right hand side. So as we go through this progression, we start with just broad data. And then our idea is that we get information out of that. We transform that to knowledge and then gain some insight. And then we're finally able to take action. So this is our little experiment in more detail. So we start out. We've got a view of the ops manager of PCF with some standard tiles. We have RabbitMQ for messaging and MySQL and Redis. And those are all used in our solution in one way or another. But really critically important here is we have the GCP, the Google Cloud Platform Service Broker, which provides access to a lot of the magic that we use. And I wanted to credit Colleen Bryant for all her work on that. It was, it's amazing achievement. And I don't think she's in the room right now, but I know she's here somewhere. So to get going, my thinking was that a data flow from left to right, we have the Spring stuff. So Spring Cloud data flow made a lot of sense for this. It provides these nice sort of prepackaged abstractions, sources, processors and syncs. And we use all of those in our solution. They're also based on Spring Boot. Those are all Spring Boot apps. So they're easily customizable, which is something we ended up doing. So this is, we call this the flow UI that shows the deployed stream. So the stream is the data flow that you define. It goes from an HTTP source, which is what the Twitter adapter app hits and puts data into. It goes to the first processor, we call it Proc08 here. And then also to Proc14. And each one of those has a role to do some kind of transformation. Finally, it syncs it into this offer sync, which is a custom Spring Boot app. And it's responsible for delivering our outbound offer. So first of all, the data stream coming in is a Twitter stream. We chose Python for writing this because for one, we have the freedom with the platform. It's a polyglot platform. We can choose whichever language we want because of build packs. And we were familiar with this Python Twitter client. So we went ahead and built it in that. So it has the responsibility for polling and updating the list of our followers, people who follow Boho Sheik. Then we pull their timelines periodically and see if there's any activity. And then ultimately the outbound offers, if they're to be delivered through Twitter, this app handles that. So our user would just opt in on the app there by following us, and that's all they have to do. So by the way, as we go left to right in this data stream, we're just basically taking JSON strings and just sending it across. And at each step, we sort of embellish that string and add features to it. So that's our data structure. Oh, yeah. So in the demo, you saw that the Twitter app had failed. And this is something that I was scratching my head over for a while. It could have been this Tweet Pie client that we were using, or it could have been some issue with the Twitter API. And looking into that, looking out on GitHub issues, I wasn't able to get to the root cause. And at some point it dawned on me that there is yet kind of a magic way that I could solve that without really solving it. And so we went ahead and went with that route. So we just defined a, we said, what are the business rules for this app? What should it do? So every 60 seconds or so, the app is supposed to update the list of followers. And then every 15 seconds or so, the app needs to pull those users' timelines. And if that's not happening, then we have a problem. So we exposed this slash health endpoint, and we updated our manifest file to use this custom health check endpoint. And we said it was an HTTP endpoint. And then we just repushed our app. So what you saw in those logs while we were trying to run the demo is that this thing actually had kicked in and it was hitting that condition where it was unable to pull the user's timeline. It, when the health was checked, it sent a 503 code, and that triggered the platform to knock that instance down and start another one. So all that's happening in this example that I saw earlier within about a five second period of time, which is pretty good. And given that this whole solution is somewhat asynchronous, and the user really has no expectation that when they tweet something, they're going to get an offer within some time. We're sort of, okay, we're well within the parameters of what we set out to achieve there. So the next step is we need to basically say, should we even bother to try to sell them something? So this is where the magic of the off the shelf Google Cloud Platform machine learning comes in. So we use the natural language API, and we want to use that to determine how does the user feel? What's the sentiment? We use the Vision API to basically determine what is this thing that they're expressing this sentiment about? And then finally, we have to decide, is this something that our stores actually sell? So if you look out on Google's website for the natural language API, they have this nice trial thing, you can test it out. So here we've put in the text, this dress would be great for summer. We click analyze, and what we're using in our solution here is we're just taking the score. And if that score is greater than zero, we're saying, okay, that's basically a thumbs up for us. So the first step is are they saying something positive? And if so, then we're good. We can proceed to the next step. So it is similar to that previous UI. This is the one that Google has for the Vision API. So you upload an image. And in this case, I'm looking at the labeling tab. So what this API does for us is it provides labels that apply to that image. In this case, if you look at the image, it's of a dress. The API tells us it's a dress with 92% probability or confidence. And so we say, all right, we sell dresses. We know that because we know about our inventory. So we're still good to proceed with our offer. So in order to, we kind of fabricated this notion of Bo Ho Sheik, this retailer, we don't really have inventory. So we basically went and crawled about 4,000 images on the web. And we took that collection and we encoded the product ID, the price, and the description of what that was, just into the file names. So it was pretty easy database to use there. Then we went ahead and stored those in Google Cloud Storage. Then there were a couple of data prep steps that we used to kind of bootstrap our data science apps. One is we generated a set of the top 250 terms that were found in the image descriptions. And then we ran the vision API against a 10% subset of that overall dataset, just to get a set of labels that we can use as a check in part of that process. So in our demo web UI, and I should show you if I can get my mouse over there what that would look like. It's right here. So this was the previous tweet that Mia tweeted. She was in Zurich at a jazz concert the other night. But if we reload it, so this shows how that evolves across from data to information, knowledge, insight, and finally action based on Scott's tweet that he did during the demo. And you can see the various components there. So that's similar to this. So yes, we sell dresses. We're good to go with an offer. And we use a combination of the sentiment and the image analysis for that. In this case, somebody tweeted, that was me, and I followed Bohoshik. I tweeted an image of a speaker that I liked, and I had a positive sentiment. The vision API tells us that it's audio equipment, a loud speaker, et cetera. The sentiment's positive, but we're not going to make an offer because we don't sell speakers yet. So the next phase of our pipeline as we go along is what does our offer look like? How do we build that? So the offer tweets composed of several items. And you can see an example there. So we have an image based inventory match, which is further qualified by looking at the term overlap between the terms that the vision API gave us and then the terms in our descriptions. So we look for the most frequently occurring term in all that. And in this case, it was dresses. So we say we have some hot new dresses here. So we fill that in on the fly. We generate a custom hashtag from just a set of a few that are popular for fashion. And we provide a Google Maps API to the nearest store, one of our retail locations where we know that we have these items in stock. So when our user comes into the store, they're able to buy this right there because they need instant gratification. So what I would have liked to have done is being kind of lazy by default. Is there a Google image search that I could use and just constrain it to my own images? That would be great. I couldn't figure out how to do that. So what we ended up doing is we got a little more in the weeds. We used Keras, which is a high level API over TensorFlow that makes it a lot more accessible to novices like me. We used a model that was pre-trained for images. It's called VGG 16. So you can download this model. It's a convolutional neural network and you just basically chop off the top layer. And you can use that to extract features for your own images. And you'll get about 25,000 element vectors. And you can basically calculate image similarity by going through that set and comparing the features you got out of the tweet image and look for the dot product of the vectors and find the ones that have the largest dot product. So basically you're looking to minimize the angle between those D1, D2, those are all your inventory items and Q. So you find the set of the top several closest matches. And then of course, because I did it the slow way and it's sort of serial, it goes through all 4,000 items, if you needed to go faster, one way would be to research algorithms and what would you do? And I've been told that we should use K-means clustering or K-nearest neighbors. But my way initially would be just CF scale-i whatever, 8, because the platform magic again can come to my rescue. And if we had a concurrency issue, you know, where we needed to scale up, that would work great. So finally, our Spring Cloud Dataflow sync routes any outbound offers. And it just uses our Twitter adapter right now, because Twitter is the only social media platform we've got. None of the apps maintain state and in fact, the Twitter app dies occasionally. So we had to persist like the maximum tweet ID for each of our users somewhere. So we built a little Spring Boot REST persistence layer that sits on top of GCP data store to provide that. And in the process we started, we kind of bootstrapped a set of Spring Boot starters and auto configs for all the various Spring, sorry, Google Cloud services. So hopefully that'll be coming out soon. The Vision API requires kind of a 640 by 480 pixel image to be, you know, to work optimally. So we had a previous image resizing service that we'd used like about a year ago. So we just kind of repurposed that or just, it was deployed already, we just used it. So the polyglot aspect of Cloud Foundry was really interesting to me here because we had some, we call them pivots, you know, people who work in pivotal labs, a guy named Christopher Jopes. He was what we call on the beach and he volunteered to help us with a Facebook adapter. It didn't quite get done. But the adapter he found and the API that he was most comfortable working with was in Node.js. So it would have been just fine to use that as long as it fed the our HTTP source. So Eric, do you have a question? Yeah, this is like amazing. And it would be so cool if you could open source this so that other people could play around. So I'm just kind of curious. Let's open source it. All right, pull the lever. All right. There it is. You can get the code there. It's a bitly link because I want to track your clicks if you decide to get it. But it's out on GitHub. And it's an Apache 2 licensed project now. And we want to extend our thanks to this great team who helped us build this because it wouldn't have been possible without the advice and elbow grease and sweat and responsiveness in some cases to get some things done in short order. And thanks to you all for coming to the talk. There's our special hashtag. Any questions? I left three minutes for questions. Yeah, it was designed to show what's possible. The goal isn't to say go build this, right? Really what was most interesting for me was that these things are possible. You can put smart people in a room, you can give them access to these tools and turn them loose. And while it took us about six or eight weeks, I think to do the bulk of the work, I think the next one that we do would probably take us three or four weeks and then maybe one or two weeks. And then, you know, as you master these things, you can do more with them even faster than before, right? Do you want to take that? Yeah. Sorry, I didn't elaborate that much on that. But the GCP service broker, if you look, let me go over here. Okay, that's, of course, it is installed in our Cloud Foundry. And you can see it in Ops Manager. If you look here in the marketplace, this is Apps Manager. Because we installed that GCP service broker tile, we see all these services available right here. So if we go down and look at the machine learning APIs and click on it, we see that we've got this sort of default plan. So when you, we call it, you know, pushing an app on Cloud Foundry, you can bind it to a service. So we'd create an instance of the GCP machine learning API. And we'd bind our app to that. And then that's all we would need to do so that it could use that service. We did the same with storage. So we did vision, natural language, storage. And in different aspects of this project, we bound things to instances of our own microservices that we wrote, which are those ones that I described throughout the talk. So I think that for me, as a person who, up to that, this time I had not actually rolled up my sleeves and gotten really deep into creating a solution on Cloud Foundry, I saw it as a huge accelerator because it encouraged experimentation. Yeah, because it's like, I think of it, when I was a kid, we had these things called Lincoln Logs, Tinker Toys, and then, you know, eventually Legos. And I just dumped these things out and kind of go, I want to make a fire truck. And, you know, I could because there were all these things. And nowadays, you know, you've got a pre-assembled Darth Vader. And that's kind of what we're looking at right here. This is, well, I want to say Darth Vader totally here. But, you know, maybe this is like a Luke Skywalker. You know, you don't have to do much. You just put him in the, his little, what do you call that thing? The land speeder and get more lightsaber and he's off to the races, right? So, sorry? Two. Yeah, two. With a lot of help along the way. We pulled in data scientists, data engineers, and, you know, labs developers that were on the beach. We had some design help in the beginning, but I don't know. I'd probably say a week. You know, yeah. Cool. Yes. It's a real desire to work with a luxury retailer. This is very spooky for me because we, because of something very out of force explicitly, have built something extremely similar. Really? On Cloud Foundry, which I'm talking about tomorrow. It's a Cloud Foundry. Oh, that's great. I'll come up and chat to you about it. Yes, please. Okay. Good to hear. Yes. The adapters that I kind of mentioned at the end that, so Google has a number of clients that you can use for a number of languages. I think maybe.net, Python, Java, and probably lots of others. Oh, yeah. Oh, absolutely. Yeah. The reason I use Pivotal Cloud Foundry is because I just, it kind of coddles me. It's like a luxury version, and it has, you know, full leather and double clutch transmission, six speed, and it even has a turbo. And I like turbos. Yeah, it's like a land speeder. Yeah. Yeah, I think you would push it as an app, basically. Thanks, y'all. I really appreciate your time.