 Okay, welcome back. We're here live in Boston, Massachusetts. This is SiliconANGLE and Wikibon's exclusive coverage of HP Verticus Conference. We're here on the ground. The hashtag is HP Big Data 2013. We're watching the Twitter feed, so tweet at us. We'd be happy to answer and engage with you. Again, this is SiliconANGLE and Wikibon's theCUBE, our flagship program. We'll go out to the events. Extract a signal from the noise. Follow us on Twitter. I'm John Furrier, the founder of SiliconANGLE. I'm joined by co-host. Hi, everybody. This is Dave Vellante at wikibon.org. Chris Weger's in this here. He's with DNC, the Democratic National Committee. He's the director of data architecture. Gave a keynote yesterday at this conference. Chris, thanks very much for coming on theCUBE. Sure, thank you for having me. Yeah, so we were talking off camera and I was saying, several years ago, you wouldn't think all about all this innovation coming out of things like the DNC. We've had some folks on from Obama for America and the way in which data is now being used for advantage to raise money, to identify trends. So first of all, congratulations. How did this all come about for you personally? So I actually got involved on the campaign in the first, the president's first campaign back when he was a senator during the primaries. And actually, at the time, there was no, there was no real analytics. There was no big data program. The teams that we have now didn't exist then. And I really wanted to get involved to help with what I believed in. And it's actually been an interesting experience that over time that's evolved to, now it's actually a really interesting technical problem that we have, we've created all of this data and want to do something interesting with it. And so now we're sort of on the cutting edge and it's just happened in a few years. So tell people who may not be familiar with the story. What did you guys do with data and what was the impact on the campaign and some of the outcomes? We did everything with data. What we wanted to do, so Jim Messina, our campaign manager, has actually said that he wanted to measure everything, put analytics in every aspect of the organization. And that was really what we strove to do was to figure out where are all of these sort of distinct buckets of data that for years have been sitting untapped out all over the place. How do we bring those together? And then more than that, how do we take that and start deriving actual intelligence from it quickly? I mean we had, I said during the keynote yesterday, that we had the mother of all deadlines. And that's really true. I mean it's like we couldn't deliver something the day after election day. And in fact it wasn't even that we couldn't deliver it the day after election day. Campaigns are very scheduled and you have to have things at very specific times. And so just the challenge of trying to, without, you can't have a big technology buildup before producing sort of analytic insight. So one of our challenges and why we came to HP Vertica, for instance, was just because we needed that ability to move very quickly from data spread all over the place to actually be able to do smart things with it quickly. It's the ultimate challenge too on the ramp up. And then you said the ultimate deadline. It's like it's got to work, it's got to work. Take us through the rocket ship of the team and what were some of the insights that you've learned from that? Take us through some of the things that happened that were very cool and motivating. Also talk about some of the challenges and what insights you've gained from that. Well, we hadn't had experiences where we give such a large team access to, without a lot of SQL experience. That was the thing is that when we built this analytics team we were like well we want people who are really smart, who get the underlying problems but they're not going to be technology wizards. We're going to have, You've got DBAs, they're people in the field, right? Or experts. Right, people who understand sort of the of voting and all of that or fundraising or whatever their specific target was. We wanted to bring those people and give them the ability to serve themselves to get into the data and just play around with it. And so our team was sort of about removing those, the analytics technology sort of team that I headed up, was about sort of pulling down those barriers so that those analysts could go. So making it as easy as Google search. Right, well it was a little bit harder. But not SQL queries. Right, well no, so our entire team was using SQL. So we had, our internal analytics team was about, I think maybe 50 or so people. And then out in the state, so we had embedded teams of analysts sort of out in the states that maybe totaled, and throughout other parts of the campaign as well, maybe totaled another 50 people. The ones who didn't know SQL, and there were many, we taught SQL and so we had this crew of 100 plus people who were in there and could work with the data. And it wasn't hard to teach them SQL. And, but then they were empowered to, if they could find two tables, they could join them and we didn't have to worry about performance. And that was really empowered. So they had the post-it note of the command and they just did what they had to do. And so I mean we had some queries that were not necessarily the best. You get these giant nested sub-queries many ways and which are how people think and that's great. But people hungry for data can learn basic SQL. Absolutely. I mean they're starving for data. And that's it. They were, and you know, that's what we saw across the organization is that the more people use this, the more they wanted to use it more, right? It's, you know, once you have access to it. To the magic of it, it's beautiful. Then it just, you know, feeds back in on itself. And so we were just, we were just there always slightly lowering the barrier so that people could figure things out. So was this thing powered by Vertica? Are you just here as a thought leader? So this was powered by Vertica. I mean every diagram that I draw of how the analytics operation works, whether it's about how we sort of worked in terms of process or how our data flowed, Vertica is right there at the center of it. And that was a big part of how we were able to do this was because we didn't have to worry about sort of a complicated, you know, ETL process to get things into there or, you know, how is this going to perform? We were able to get our data in there and its raws form right away and people could start using it. So our analysts were, you know, in there right away. And then we were able to sort of build on top of that and grow out from there to the point that, you know, everything that was happening on our analytics team was basically touching Vertica. So what were your data sources? I mean, presumably you started this journey today. What data sources are we going to use? There's so many, but what were the primary ones and how did you get them in there? Sure, I mean there, you know, I can't talk about, you know, everything that we had, but, you know, we had a lot of pools of data. You know, part of our sort of core is a registered voter list. So states are required by law to keep an electronic list of all the voters. You know, whether they have voted in the past but not who they voted for, obviously. And, you know, so that forms our core. So now we know. So that's public data. That's public data. Okay, and then you had some of your own IP. That's right. I mean, so we, I mean, you know, are all of the contributions that are coming in, all of our volunteer field interactions. So when somebody knocks on a door, makes a phone call, we get data about that. And that's exclusive, obviously, to DNC. You don't share that. That's your competitive advantage. Okay. And you had other public sources. You had social data? Yep, social data. You know, I mean, we, I mean, yeah, anything we could find. Okay, so how did the data get into the platform? Sure. We actually had a lot of different paths for that. We ended up building sort of our own, sort of our own system for being able to plug in to the databases that our organization had already set up. So I mean, we had one of everything. There was, you know, MySQL, Postgres, Microsoft SQL Server, you know, raw text files, whatever we had, there was around there. And so we actually ended up building sort of a process to feed that data into Vertica. And that was, again, sort of, you know, we didn't want to transform it. We didn't want to do anything. We wanted to grab sort of just whatever was new was coming out somewhere and drop it into the system. So we ended up being sort of, you know, unhappy with what we could find that required very little effort too. If I just want to copy a table, I should be able to copy a table. So we ended up creating a pretty impressive system, I think, to move all of that around. And the platform, I'm assuming, is candidate agnostic. I mean, you guys are building something that can last and continue and evolve and innovate. Where do you see taking this in the future? Well, that's, I mean, you know, our big challenge now. I mean, if you look at, you know, I think of 2012, the presidential re-elect as, you know, we had one giant tower, you know, it was a gleaming office building of, you know, but, you know, a single focus, right? We all had the same mission, we had the same leadership. We were organized, we had this sort of central, you know, campaign analytics team, central organization. The, our next big challenge is 2014, the midterms. How do we take this and, you know, sort of tip this organization right on its side? Right now we're dealing with all of these different independent campaigns who are all clamoring to have, you know, the innovation that they know is out there. I mean, that the Obama campaign did. How do they, you know, how can we do that for all of these smaller campaigns, which have, you know, fewer resources, fewer staff? And you have to make trade-offs, too. Right. Based on probabilities of success, right? So are you using analytics to do that, or? You know, I can't talk too much about that, but, I mean, you know, we're definitely being. So, yes. You know, smart about how we operate. So what is the biggest thing you've learned? The biggest surprise that you had, that you didn't expect that happened over the course of your journey? Boy, the biggest surprise, you know, I think it was the, the, oh my gosh, this is gonna work moment, right? It's, I mean, you know, we were having all of these analysts, you know, hit the system. It's actually really difficult, at least for us, you know, having no experience doing this before, building, you know, even like a proof of concept, right? During the evaluation stage to figure out like, you know, how are we gonna do this? And so, you know, having the system there and, you know, having this entire organization of, you know, 100 plus people, I mean, at any given time, we had maybe 30, 40 people just in there analyzing data. I mean, that's pretty incredible from an organization where that didn't exist before. You know, we built it in, you know, six months, a few months, I don't know, you know. You have to, I want to ask you about how you balance your public persona and you guys are out, you're talking and that's great because you're marketing yourselves and attracting, you know, new people and new contributors, collaborators, young people, old people, et cetera, but at the same time, you've got some pretty serious IP that you're trying to protect. Do you have that discussion internally? I mean, I presume it's not a free-for-all. How does that all, you know, work? Yeah, I mean, you know, I think, you know, a lot of us have been, you know, in this for several years now and I think there's a lot of, you know, a sense of joint kind of investment in this. It's certainly hard to, you know, we have to balance, like you say, but part of this is, you know, going out and we want people to be excited about this. We want people to know that we are, you know, thought leaders, like this is, you know, this is the image that, you know, we're trying to project and we have to, you know, we need to bring in staff, like this is driven by staff and we want people to know that we're doing really cool stuff and so, you know, we have to walk a delicate line between, you know, giving away the secret sauce and talking enough, but a lot of the technology problems, like, you know, this is out here. Yeah. You know, you can, you know, certainly if you spend enough money, you can have the technology that we have and you can pay somebody to do the work that, you know, my team did and that's not a secret. Right, it's the people and the process behind that. Exactly and so, you know, what we have is this organization that channels, you know, this data and analytics, sort of, you know, and operationalizes it and that's the real secret. So a lot of people watch theCUBE, there's people out there, maybe want to get involved. What kind of people are you looking for on your team, whether it's data scientists, mathematicians, statisticians, programmers, who are you trying to attract? Let's see, data scientists, mathematicians, statisticians, programmers. No, I think, you know, it's working, you know, working on a political campaign, sort of during a political cycle is hard and you certainly have to want it. Like, you know, we work sort of crazy hours and but, you know, we want to build this team of people who are, you know, who are, you know, we want sort of great engineers, great analysts, they're all sort of part of building this platform that... What's the hardest part? I mean, it's a zero sum game, it's like sports in that regard and you've got very tight time frames, you've got intense competition, you've got, I'm sure, demanding bosses. What's the hardest part? Lack of sleep. You're young, so that's not good. No, you know, I think, you know, the timelines make it tough. It really forces a different way of thinking about things but I think it's a way that's been very productive for us. Rather than thinking about, okay, what is this project that we can set sort of a long roadmap for and sort of work through, it's okay, what can we do in a week that's gonna add value? And then at the end of that week, we're like, okay, you know, put it in the analyst's hands, did you derive, and then a week later, okay, did you get any value from that? Yes, then let's see if we can make it a little bit better. No, let's throw it out. Like, you know, we can't work on that anymore and so... John, I was using the Jim Harvock quad, you're either getting better or you're getting worse. You guys are constantly trying to get a little bit better every week. So what's next? So final question, we got a break here, but I wanna ask the final question. What's next for you guys? What do you guys continuously improve on? So, I mean, we've got a lot of different challenges. I mean, I think there's a lot of growth in sort of in the social media area where we broke the surface of that on the campaign. We did some cool stuff, but there's a lot to learn. I think there's a lot to learn sort of across the entire space. I mean, right now, we're really focused, again, sort of on that, how do we take this, you know, big tower operation and make it into a bunch of little houses? Yeah, federate it out, and people empower people to do more. Exactly, when, you know, it's just a very different structure, a very different set of challenges. And how do we get that into thinking even longer term? How do we get that down to even smaller campaigns than, you know, Senate and House that can maybe afford to have in-house staff doing these things? How do we make this accessible to even the smallest campaigns? I mean, that's a real. Final question, just to kind of add one more point. What's your advice to business folks out there? We had Moneyball up here, Billy Bean talked about how they use data, and I'll see that's a big book in the movie. I'll see Brad Pitt, but I'll see it's an example of competitive advantage. Here, your great use case in politics. A lot of business folks want to disrupt, they want to do things differently to make a difference for their business. As George Kedif pointed out, it's not just bottom line, it's like top line or revenue or impact. What's your advice to the business folks out there? You know, empower analysts. There are a lot of smart people out there, and, you know, they can acquire skills like, you know, Sequel and, you know, and to some extent, you know, R and things like that. But you've got to sort of break down those barriers so that they can start, you know, getting at the data, you know, looking across the entire universe of data rather than sort of being in their little, little sector. In their little cave or pen stand environment. Yeah. Well, we were in a cave, but it was, we liked it. Did you see the Billy Bean talk yesterday? Yeah. Were you there for that? Yeah. You know, when Michael Lewis' book first came out, I said, this Billy Bean's out of his mind sharing this information. Everybody says Copycat League, everybody's going to do the same thing, and I guess it's not like the NFL, John, right? Which is a Copycat League, because they've continued to succeed. What was notable is that it seems like the organizational friction in other teams has continued. Did you see that within the DNC? Was there sort of a faction that said, no, you got to have the political experience, the gut feels, the, and you even see that today when people are predicting elections, you know? We were very fortunate to have a team, you know, like I said, you know, from the very top, I mean, even President, campaign manager Jim Messina, you know, throughout the organization, it's, you know, where's the data? And, you know. It's a mandate, essentially. Yeah, and, you know, it's true. I mean, there is sort of in politics, it's a very traditional, I mean, and sort of what Billy Bean was saying did resonate because it was like, yeah, you know, you are dealing with sort of some very traditional ways of thinking about things and just saying, you know, trying to say, well, you know, you can look at it with data, and, but, you know, the reason we were so successful, I think, is because we had this organization aligned and where everybody was bought in. And I think that that's new, but sort of very exciting. It's interesting, you know, the night before the election, John McLaughlin on McLaughlin Group, you know, Romney by X, he predicted it, and you know, Nate Silver's nailed this thing, you know, state by state. What are you talking about? You know, it's interesting to see that dichotomy now Nate Silver works for ESPN, and he deals, sorry, working for a social media company. Yeah, so I think if you can go, journalism is a good step to tech companies. So we'll see the world's changing. The Washington Post is bought by Bezos, and Boston Globe is bought by Red Sox owner. What the hell's going on in this world? So we'll see. All this great stuff, really kind of changing the landscape. Congratulations. Thank you. We're psyched to watch you guys because, one, we really believe that data's going to liberate and create disruptive innovation. You guys are a great example in politics where using and measuring data allows you to be more agile, more responsive, and more targeted, and more accurate, and more accountable. And I think businesses can learn from that. Congratulations. Chris, thanks for coming on theCUBE. Really appreciate it. This is SiliconANGLE, we've got us theCUBE extracting the signal from the noise. Not a lot of noise here, a lot of end users, a lot of heavy weights here, really doing some pioneering work. We'll be right back with our next guest after this short break.