 and we're back live here at dot com 2012 Splunk's user conference annual user conference we're here in Las Vegas broadcasting live from the Cube we're at the Cosmopolitan Hotel I am Jeff Kelly from wikibon.org I'm with my co-host Jeff Frick from Silicon Angle. Great we're back welcome everybody we are we're excited to keep the day rolling and I know you're probably getting tired looking at Jeff and Jeff so we've got somebody who's a little more fun to look at Michael Wilde the chief of awesome so if you if you've always aspired to have a great job at a great company I think chief of awesome would be be tough to tough to beat. I also go by Jeff. You can go by Jeff so we can get Jeff cube in the Cube in case you were a little slow because it has been a long day the rain has stopped the Sun is still not out and we're still having fun here at the Cube so again we're at the Cosmopolitan Hotel in Las Vegas if you're close by stop by if you're not joining the conversation by using the data journey hashtag on Twitter this is the third annual worldwide user conference comp 2012 it's over a thousand people the excitement is tremendous we've got the vendor fair out there people are slowly waiting for the coffee to roll away and something else to roll into its place as we get closer and closer to that time but we're happy to have Michael with us he's going to show us I think some some interesting things that you can do with Splunk we've heard lots of stories but as I say a picture is worth a thousand words and I don't know what that makes an app worth but this is a big diet show so we should be safe. Well first of all we have to we have to get a better understanding what the chief of awesome does. I understand you're also chief of tuxedo fitting. That and collating and filing as well so yeah I work in the CTO's office at Splunk and the main focus that I've had for the past six seven months is on social and mobile using those as data sources that go inside of Splunk and also helping customers or people with challenges around mobile whether it's an SDK or whether it's understanding how mobile impacts their environment so the social stuff is is really interesting to a lot of folks that see it because it's kind of a other than expected data source who would put in the Splunk when you put it in there the data really comes alive. So when we're talking social we're talking Facebook Twitter are we talking the actual content of updates and things like that and drawn correlations with other machine generated data and yeah sometimes right so that's a good use case I have a session coming up tomorrow or Thursday probably after everyone leaves at 2.45 with my friend David Carasso and and the interesting thing is there there is a very compelling use case for IT and marketing IT and the brand to collide right because when we have a service even the you know when people are talking about the cube they'll talk about how great it is or if for example let's say something has a problem they're going to tweet about that as well the marketing folks will understand that brand impact right away but the IT folks that are managing the systems might not necessarily be aware of it right and I did a search on the word site and taking Twitter data taking Twitter data in search on the word site and then the phrase is down you'd be surprised how many people will be happy to talk about their favorite site that's not quite working yet if you take that data and combine it with maybe the infrastructure data you could probably have a pretty good picture we're both groups that really have a sort of an understanding of how IT and brand are impacting each other so that's interesting but yeah data is coming from many of the social networks out there either publicly available or through partners and really don't have to build anything extra in Splunk it sort of just worked but it became magical when we started looking at it I'll show you guys some of that bit later and just to kind of as a measure of scale what is the kind of the social flow of data the social media data flow versus some of these machine data flows that you guys are involved with is it you know just infinitesimally small is it way way big is it small but growing faster we need to pay attention or is it just because we think it's relevant and it's a new important data stream that we want to play with it well I think it's the latter it's the important data stream they want to play with because of the the case that we talked about right some people need to look at social networking data for the purposes of just analyzing their brand or how their brand is perceived it is rare that most folks need the entire corpus of let's say Twitter or Tumblr or something like that but I've been working with a couple of the third-party re-syndicators of data one of them there's some folks running around here a company called Gnep they're out of Boulder and they provide access to the feeds directly so you don't have to go to Twitter and all these other companies but if you look at it on average if I was to take all that the entire social web and put it in Splunk if I were to do that based on the publicly available statistics you're probably talking about two to three terabytes a day if you were to ever index the whole thing most customers really wouldn't do that because they don't need it they need a very small time slice or a very small set of phrases around things that they're interested in okay some of those third-party providers provide that or even Twitter does as well all right cool so let's let's dig into some of the visuals we've got here and I know our audience loves to see kind of these types of products and actions some really compelling stuff so what do you have here for us so what's kind of fun is we we got started with social kind of a on accident the CTO and co-founder Eric Swan I don't know if you talked to him he kind of wrote me into doing a presentation at South by Southwest this year and it was about big data and I didn't want to do something related to firewalls because although firewalls are awesome I didn't think people at South by would really get it so I called up my friends at Gnep and said can I have some social networking data to show because it would be you know an easy thing for folks to understand and out of that we also created some videos and a few things you heard Godfrey talk about the social stuff at South by in in in his keynote and so we started looking at it there and we provided some dashboards for South by and then we had things like the Olympics come up we've got Splunk user conference so there's things going on and then all sorts of other random things on the social network so kind of starting in reverse let's look at Splunk user conference okay so what can we capture we captured all of the stream of data from coming from Foursquare right the social location-based network there's a venue here for Splunk user conference you can check in and then there's also people tweeting obviously and so we have a simple chart here just looking at a distribution of Foursquare check-ins versus tweets obviously it's people would check in maybe once but they would tweet thousands of times you know and this is just over the user conference I built this dashboard in about five minutes it's pretty much just looking at the data point and clicking and that's one of things I like about Splunk even six almost seven years later I've been here for that long it is really simple to use we can look at like the tweet volume here on you know from Monday where we had things like the pre-conference classes to when the actual keynote happens so you start seeing that right so this is naturally to the spike when when the conference starts to kick off the God first doing his presentation right you're probably these things like Loll's during lunch and during other popular time as people will show that my buddy David Carasso who works in the CTO's office he's the one who wrote the book on Splunk literally we have the book I just have the PDF I'm kind of digital I hope so I hope to meet him someday David one of his main focuses in computer science is a natural language processing so he's written some technology that looks at sentiment Splunk typically is good at numbers right statistics eating any kind of data we're doing sentiment analysis on text so we're looking at the actual text of the tweets and determining whether there's a positive or a negative sentiment there's always some measure of error but we have things like someone was talking about how one of the talks was going a bit slow and that you know we'll see a negative sentiment but overall you're not going to please everybody but people are really happy so far about the the conference as well and that's just coming off of tweets we started looking at hashtags here I mean we recommend that people start to tweet with the hashtags as well and what and we're just looking at a very small amount of Twitter data focused on certain terms that might be Splunk conf12 and a few others and then making sense out of them and then of course I can actually look at the live tweet stream that's coming right here so you know I don't know if I've got some issues here on silicon angle it's awesome and you know people are not getting sleep at conf12 because they're having a lot of fun that's great so being able to actually for the marketing people here in Splunk you can actually watch how people are feeling just right here in Splunk which is kind of cool yeah taking a little bit more on how you know useful this data can be what are some of the things you could potentially do and kind of the new correlations you might be able to make well some of the correlations you might be interested so if you're looking at social data you're looking at human behavior or human conversation so you might for example let's say look at the Splunk for Good project which is one of our corporate social responsibility projects out there they're doing some work around the elections right so we could take and look at and the other thing is like Splunk is a tool that not just nerds can use so you could get a feed if you were a marketing person go bring the data inside a Splunk and very quickly develop dashboards without having to hire a bunch of people to do that and we've even used David's sentiment stuff to look at sentiment about the two candidates which is pretty much about the same but then you're seeing bookmarks on Facebook which particular ones of people are actually bookmarking how are they creating bookmarks so that a brand might be concerned about how people are interacting with their web property what devices they're having and potentially what errors are occurring as well there's a lot of fun stuff here so if you start to look at things like four square right four squares the location based social network we mashed up this little dashboard but looking at things like hashtags trending this hour that's Twitter followers followers versus friends and this is all real-time data coming in how are people tweeting that's cute when I first started looking at four square data it gets a little bit interesting because you see people Twitter is conversation or square is doing so where are people checking in right now a lot of people actually check in at home for whatever reason so nobody breaks in we've had a lot of security conversations you know and I started looking at what types of places people are checking in where are they shopping Target and Walmart seem to like always go up and down as far as being ahead but then you start to wonder and the power of Splunk being able to take the next question in your head and drill in is why another reason why I like it because I do things in a very you know just ad hoc way I just give into my own ADD when I originally built this I wondered like okay who's actually hanging a target men versus women so you might for example be a major brand maybe your AT&T or something like that you want to know people are checking in at your stores or potentially checking in at stores that are not yours in that area you know even are you know the the the business is literally right next door social data is really nice and because you can put this all in one platform which is cool Starbucks is obviously the most popular coffee company in the world but you can start to look at that and begin to drill down and things and even look at well you know where are people working out and you know who's actually going to the gym so it gets really fun because the Splunk the Splunk engineering team has built enough things in here where you know a halfway smart person can start to look at the data and just point click and make some cool intelligence well that yeah I think that's a really important point to reiterate because you know we hear a lot about the lack of skilled data scientists and other kind of business analysts with these really deep skills needed for things like Kadoop and other analytic approaches so the easier the tool the lower the barrier to entry in terms of users and of course adoption so what I really like as well though is the ability to drill down you mentioned kind of you wanted to ad hoc analysis right because one of the things about big data is you don't necessarily know where the data is going to take you correct so you kind of need that ability to to kind of me anger if you will make you show us a little bit more about that yeah drill down yeah I mean some of the interesting things here's here's a fun one well we'll actually do some drill down literally so one of the cool things about four square is it provides light latitude and longitude okay so if we were to take that latitude and do something like run a search in Splunk but get to the maps here we'll run a search in Splunk and look at things like who's actually having a cocktail right now in the world wow so let's hold on it's gonna pull this up should be sweeping across from the east as five o'clock is somewhere east of Denver right now you know what I like to have fun 24 hours a day so here we go I almost had it it wouldn't let me use my Mac just gotta get it right down here here we go I'll I'll go to another screen and pop it up here because my man using pretty long but we have an in you guys have probably been talking to folks about Splunk apps right they're out there and one of our partners at a company in Germany called SPP they wrote an app that integrates with Google Maps which is really cool so that means if I have things like latitude and longitude or other data points I can plot those things on a map and make potentially make that the very first part and just on the map it's not necessarily something you could ever just lean in a pile of analytics because you know your own ability to look at a particular part of the country in the world would be the thing that you know you would cause you to drill down so let's look at I'm just gonna search here on maps and grab my good old search so let's look at who's having a cocktail so if I click run here it's gonna run a search we're in real time there we go Eric Swan that's the guy there's two co-founders right there it's a floor wax and a dessert copy pretty much so this is a load up in a second here but so Splunk is running a search against four square now it's doing it in this case I'm looking in the past 15 minutes I can pop it up on real time and we're also gonna go ahead and it's gonna process it pull out the latitude and longitude and then plot that stuff on a map so what we see in front of us is clusters of activity and you know the thing I really like about this particular app is we could drill down into this and really almost go into the enemy of the state type view all the way down into one particular person because you might wonder like why what are they doing why are people this why is this cluster of 16 particular users here checking in on four square and as it goes down you see the clusters go farther and farther and farther and I'm sure if we I'm sure if we get down this thing is rendering right now the map itself so we'll let the thing go but the cool thing is if I go down to one and I just click on that what Splunk is basically gonna then do is drill into the raw data in this case you can customize it to your heart's content where the drill ends happen and they can fly off in other nice places but we can actually go down look on a map go down to a cluster and find one particular bar and figure out well not who we can say maybe it's a male or a female checking it at this particular venue I don't come up here in just a second as soon as this guy loads but it gets really interesting when you start playing with data so in some use cases that's going to be the kind of thing you want to do you want to drill down to an individual level in other use cases you're going to want to look at take a step back right and get a wider view right and that step back might be just even looking in a geographic area and because Splunk one of the key innovations we did at Splunk is we really created a time series engine and that turns out to be the vector that's almost most important on when you're looking at any kind of data because you know let's say it's how fast is your website running the next question is in the past 15 minutes right or this year or last year and because it's specifically designed for time series data it gets interesting because then I can start going well I can watch potentially I can watch people move right if I'm plotting on a map and I'm going over time I can watch the move I had a partner that did some really cool analysis of the New York Marathon so they got actually see where people were moving based on a mobile app that they created that was tracking location of course it was it was the user had to give up their location obviously but it was it was really cool so anyways but this guy popped up again so that one drill in that we did turned out to be someone's having a cocktail at a venue you know called whatever the name of this place is 56 fighter club and that's like really pretty much like now 11 you know or 15 minutes ago this is in GMT but the ability to just deal with the data and I'm not a data scientist right I'm like a half nerd half salesy person business student nerd and so I like things that I can that can easily give in to my own curiosity and literally all this stuff is just developed really really fast well yeah it's interesting because you know we're seeing data professionals really start to span all sorts of types of titles and use cases and you know it's not just okay I'm an analyst and this is what I do it's I can be in marketing I can be in finance I can be in sales and data is becoming really important to how I do my job yeah I mean you often people who have the data have the power right or at least the ability to make the decisions and you know because a beginning user yes you're gonna need a data scientist because there's a lot more than the average you know business or marketing or even IT person can necessarily figure out which you know statistics they want to run but if you can get started with averages counts and things like that over time then you get you know pretty powerful things very fast so so you mentioned a little bit about working some of the sentiment analysis stuff that's a really interesting use case so we've only got a couple minutes left so tell us a little bit about looking forward you guys are doing some cool stuff but what are we what's on the horizon for you and in the office of the CTO yeah the chief of awesome yeah the things that we're focused on so we have the CTO's office is it's got a couple of like sort of main initiatives integration with big data Eddie Satterley one of our guys that's in the office is focused on integrating splint with a lot of these big data platforms David Carroso is focused on the search processing language and obviously the book and writing some of the stuff around natural language processing I'm focusing on some of social and mobile and we're starting to do things like providing access to these social feeds through third parties giving people things I can download and then you know over the next year looking at talking to customers because people are really interested talking to them about what things they want out of social and then perhaps delivering a set of apps for them they can download for free on this one place if they have access to it interesting yeah we've talked a little bit about big data apps today and you know that's kind of an area that hasn't developed quite as quickly as I think some of us thought it would and really that's where you bring the power to an end user who not data scientists can do some really interesting things though with these applications that's where really the power of big data comes through to the kind of the average and it's not just because it's big right well right ultimately it doesn't matter how big it is and if you're not going to draw any value out of it what's the point of you know prepping it and putting it through all different processes and storing it agree ultimately in an application to make use of it totally agree all right very interesting all right well thanks so much for the demo yeah I'm very much appreciate some really cool stuff I'm Jeff Kelly of wikibon.org you're watching the cube live from dot com Splunk's annual user conference we're here at the cosmopolitan hotel in Las Vegas obviously we want to thank Splunk for having us here it's been a great show so far today we've had a full day of coverage lots of great interviews with executives and partners and customers so we will be right back in a moment myself Jeff Brick my co-host from Silicon angle and we will see you shortly after this