 Thanks so much for coming out for this. This is both Vince and I's first time speaking at Rails Compton. Actually, my first time as a developer speaking at anything, so this is a really cool opportunity for me. So I think we're both really humbled that y'all could come out and talk with us about this. So especially since you had to come all the way down to the caves to find us. Hope you brought your jacket, because it's cold in here. So yeah, my name's Shiv. I'm a software engineer at New Relic and I work on what is, we call it our business enablement team. It's kind of our internal engineering team that serves all of the various business units within the company. And you'll see throughout the presentation, I'll give a couple of examples of what that means, so we can get into that a little bit more at that point. And my name's Vince. I work on the RPM UI that you're all familiar with, hopefully, tending to work on more infrastructure back in kind of stuff these days. I've been with New Relic for about a year. I remember last year at RailsConf in Portland, I was interviewing with them, so this is a nice little anniversary for me. Also, just to get a quick raise of hands so we can just understand who's in the audience. So first of all, who here has heard of New Relic before? Okay, good, that was kind of planned. Keep your hand in the air, though. I just wanted to get everybody's hand in the air. Who's currently using New Relic either as a side project or in a production application? Okay, and then keep your hand raised if you're using it specifically in a production application currently. All right, and then keep your hand raised if you have checked New Relic in the last seven days. Okay, that's great. And your hands can go down now, sorry. And how many of you are, let's say, software engineers? So most of you, how about? Software writers. Software business people, business, you consider yourself more of a business. What about a blend between those two? Probably a lot of you, okay, cool. So the purpose of this presentation, what we kind of wanted to do is go through software analytics as an idea, as a concept, and how that can relate to making business decisions. So regardless of what your choice is for a software analytics tool, it sounds like a lot of you have used New Relic before and are using New Relic. But regardless of the choice of tool, what we want to get across is that you can use software analytics to really inform your business decisions. So throughout this presentation, what we're gonna do is start with a bit of a product overview that Vince is gonna walk through. And you're in a sponsored event, so we're gonna talk about New Relic. And then what we really want to do is show you about three or four specific examples of how Vince and I have used New Relic in our actual jobs to help inform business decisions, kind of what we're gonna work on, how we prioritize our work, and that kind of stuff. So some real life examples of how we use New Relic. And then hopefully have plenty of time for Q and A if you have questions. I'll let Vince take it away. So New Relic's been around for about six years now. Started out just as Rails Performance Monitoring, but it's expanded very greatly since then. We've added a number of extra agents, so we work in lots of environments, Java, .NET, PHP, Python. We have a platform component that lets you build your own agents of sorts, and so you can monitor things like MySQL or Memcache, whatever you can write a plugin form, you can hook into our infrastructure and have that data be reported. And lately we've started getting into mobile apps, so we have a iOS agent, and we also have been getting further into JavaScript with a Node agent and a new browser JavaScript agent. So that's sort of the history of how we've been spreading throughout the infrastructure that all of our production apps run in. And so we're expanding this idea of performance monitoring into what we call software analytics. And this is a nice slide that's been put together by people who aren't developers, but it shows the expanse of what it is we're trying to monitor. So it started out as just how is our app performing, how is the server it's running on performing, how it's using its resources, but there's a lot more information embedded in the software that we're writing. Since the software is about a domain, there's all sorts of information about that domain like that we can start using to make better decisions. And that's where we're headed as a company with the new work that we're doing, software analytics. So the extent of what we're trying to do right now is kind of shown here and we're moving it further. But the really interesting stuff is happening off to the right here with this insights. That's one of our first steps into something that raises the level of abstraction above the infrastructure, above the technical, it gets into analyzing the thing that you're doing, not just the infrastructure that it's running on. So APM is what we're calling the site that you're very familiar with. It went from Rails performance monitoring to application performance monitoring. And so this is the infrastructure of what you're familiar with now. We put the agent in whatever component we're talking about whether it's browser, your mobile app, your server, your application itself. Send it up to the collection tier, we start correlating doing aggregation and then we present it to you in our UI. And traditionally APM is aggregated information so we weren't collecting every bit. We were taking averages over time, taking sample traces and aggregating that and showing you the aggregate. But what we're moving to with insights is event level resolution. So we're no longer gonna be aggregating, we're saving off all the information and we can do complex queries on it later so you don't have to, you're not in the clouds, you're actually looking at exactly what's been happening on every request. And that's required building a pretty deep and new infrastructure for storing that and querying that. And that's the backing up. A few of the features in our APM product but also the majority of our insights product. So I'm gonna talk about a couple new features that are in APM. And since we're all Ruby developers, one of the real interesting ones is some of the new stuff that's been revealed with some work in the agent that we've released very recently regarding Ruby 2.1 garbage collection. So this is our staging environment and you'll see this deploy here is where we turned on Ruby 2.1 and that's just stock Ruby 2.1.0 out of band GC or anything. And now that we're monitoring a lot of new information about the garbage collection system, you'll see that our memory usage dropped significantly, the time spent in GC dropped significantly and the heap size got that dropped significantly as well. So this is a little bit of developer flavor for you but if you wanna gather some business insights, upgrade to Ruby 2.1 for sure. It's really a much better garbage collection system. And another new feature that's backed by our insights data store is histograms and percentiles. So like I was saying about averaged aggregated data, there's been other talks about this too, sometimes important details are lost in the muddle of the averages but we actually have histograms available for all the metrics we collect now and transactions. So you'll be able to see things like where are the bumps? There's often like a bimodal distribution and a lot of requests. And you can see percentiles. So what is it like for the every once in a while when there's some weird database contention that query takes forever? Like what are those response times actually look like? So I'm gonna turn it back over for a few more new features. All right, Jim. Yeah, so browser was released, let's say one or two months ago. So it's pretty new, you might have seen it pop up on the left hand side. Some of the features were in APM before but we found out that there was enough of a need for a front end specific tool, a front end developer specific tool that we could break it out into its own tool. So we called it browser, so you'll see it on the left hand nav within New Relic. It's just called browser and then your applications will be in there. And I'm gonna share with you a little story about what happened with my team specifically. So I mentioned before that I work with the marketing team within this kind of business enablement group. And one of our main purposes is to keep up the NewRelic.com marketing storefront site. So anytime you just go to NewRelic.com, flashy, Ajax-y, jQuery stuff going on, it's a pretty site and all that stuff. That's kind of the team that I work on with a bunch of front end developers and stuff like that. So when browser monitoring came out, this kind of crazy thing happened. So we didn't have access to this data before and what happens when the guys in Portland come up with these new tools is that we get to use them first. And usually that's pretty awesome experience because all of a sudden we have access to all this new data that we just didn't have before. So I'm gonna share with you a screenshot of what we saw the first time we clicked into browser monitoring. It's pretty small and it's actually gonna look a lot very similar to what you're used to seeing in the NewRelic APM tool. You've got the charts at the top. You've got throughput. You can dig into page views just like you can with transactions. And then there's these two new boxes at the bottom right. JavaScript errors and AJAX timing. These are brand new features that we pulled out. And I think you'll be able to see pretty quickly if you can see this why it was pretty scary. Scary it was when we first opened this up and found out that 20 to 30% of our page loads had JavaScript errors, which is a pretty scary thought to think about, right? That's a lot of JavaScript errors. We knew about a few of them. You know, a lot of them were not bringing down a side by any means. They might have been older JavaScript libraries that had some stuff in there that we weren't using and it was throwing up some red errors in the console. So not a big deal. But then we wanted to dig deeper and find out if there is anything in there that was more serious, right? So when you click into JavaScript errors you get, again, another very similar view. And it's a breakdown of all of your JavaScript errors over that time period and kind of the amount, the number of times that they happen. So it's ranked by the number of times in this case. And a lot of them we weren't surprised by. And then we started clicking around. So I wanted to click into one and show you what we found when we clicked into one. And we're not using an actual demo here because we can't really get the internet to work. So sorry about that. But you'll see down here that it's pretty interesting. So this JavaScript error is very specific. It's happening basically only on Internet Explorer and probably some versions of Firefox. So yes, slap on our wrist for not checking Internet Explorer, which we should have done. But what I like to think of a lot of New Relic tools are is we as developers can go out and develop really cool things and we're not gonna catch all of the errors that happen. But thankfully, you can use tools like this to catch the stuff that you don't and then you can go fix it. So the people that are operating on IE 7 and 8 can still maybe see your site a little bit, right? So I also want to show off AJAX timing. I don't have a specific story around this, but it's a pretty cool new feature. So you can finally get level detailed information into all of your AJAX calls that are happening within your site. So that includes anything that loads on page load. So things like tracking tags. You'll see a lot of mixed panel and Google stuff happening on this slide. And then you can dig into them and get a level of detail, including kind of the amount of time spent in JavaScript callbacks. So it's kind of a level of detail into your AJAX calls that you may not have been able to see before and I'm sure you'll be able to extract a lot of cool stuff out of that. Anything to add? No, but I really like the JavaScript errors because I helped build it. Cool. Are there any questions before we move on to insights? Because we kind of breezed through APM and browser right there. Yeah. Could you go back to the 2.1 GC graph? Yeah. Vince loves talking about this. This was like, what? I've been working on upgrading to the Ruby 2.1 for like two months now. Our test suite, when I first got it, I finally got it booting and then it had half of the tests were failing. Knocked out of like a couple hundred with a few fixes and then the rest was just like. Continuously knocking away, but anyway, yeah. 2.1. Oh, that was it. Oh, it's a little bit. Yeah, sure. This is available now. We just released it with an interesting blog post that talks about a lot of the agent features. With the recent version of the gem, so make sure you upgrade your gem. Kind of sucks. Do you have a question back here? No, you're just scratching. Okay. Called you up for that one. Animations, animations, and there you go. So, Insights is a brand new venture for us. It's a new product that we built from the ground up, including data collection tier and a brand new UI. And we're calling it, what's this? A real-time analytics platform. So what that means is, like I was talking about earlier, we collect every event that happens. We collect page views, transactions, within the currently existing agent that's already in your app. So we've been doing this for a while. And if you log into Insights right now, you'll see a bunch of data because we've been collecting this for some time now. As well as custom events, so you can send interesting things to the API and use this interface to query and analyze it. Database is really big, really complicated, lots of workers querying, all kinds of cool stuff. But what it gives us is a really fascinating way of interacting with our data. So each one of these, so each of these little segments here is backed by a query that is written in a query language that we developed. Looks a lot like SQL. And it has a lot of nice little functions, basic stuff like counts and averages and whatnot, but also gives you really easy access into making histograms, percentile charts, time series charts, pie charts, all kinds of interesting stuff. And this is all reflecting live results, so you don't have to re-query. You make one query, build up this chart, and it stays up to date with what's happening in real time in your app. And there's lots of really interesting stuff that you can do with this, because any bit of information that you send in to the agent can be used in one of these queries. So say you wanna start doing some performance analysis, and this is something that we've done in a particular view in your app. You could do a query that segments it down to just that controller in action. And then you can do what we call a facet based on another category of information. So we've done facets on how many hosts does this account use. So there's some hosts that have, or some accounts with a couple, three, four, and the performance of that is understood by us. We can simulate that on our local machine, but what if there's an account that uses Amazon and spins up all kinds of dynamic hosts and they have maybe 1,000 or 10,000 in their account. We can do the same kind of performance analysis that we've done in RPM, but faceted by host count. So we were able to determine that there's a threshold, and when you got over 1,000, the response time of this particular view really spiked. And so that gave us insight into what we needed to pay attention to when we were doing our performance optimizations. So we needed to look for not just N plus ones, but what are the effects of having a radically larger, this variable, host count, or another variable. So that's a way in which we've really gotten a lot of benefit out of having a way to do more complex and more customized queries based on the same data that we've been collecting for a long time. And you can make any number of these dashboards and any number of these charts and put them up on a TV or whatever, keep them on your desktop. I recently launched a new feature on my own personal site and I was watching all the visitors come as this registration system opened for both bike polo tournaments that my software hosts. And for me, 100 users on that side of the time was a lot, but I was able to see people are logging in and doing these transactions and I was recording them all just using the agent, like add custom param, new relic agent, dot add custom param, give it a key and a value, and then all of a sudden it's available in the system for you to query. You can stay there. Oh yeah, you wanna go back to that? Yeah, and I think Vince brought up host count, but really for your own usage, you could vest it by customer size or how much they spend with you and then really look at based on what tier they fall in, how your app performs for those various tiers, which can be a useful metric to look at and understand where you need to focus on, focus a majority of your time. So I wanted to share one more example and I have a little bit of a story, so bear with me for a minute here. So we were launching insights about a month ago or so and the marketing team was like, can you build as a dashboard that shows us kind of what Vince was talking about, all the people that are logging into insights, real time, how many of those people, who's viewed a dashboard, who's inserted a custom attribute into a transaction event. So I was like, yeah, yeah, totally. I can build this dashboard and here's the dashboard that I built. This is currently showing you how many people are logged in to insights right now, 815 users. And they were like, cool, this is awesome. We have a dashboard now, but Shiv, we can't do anything with this dashboard. It's just like a dashboard. What are we supposed to do with it other than look at it? It's nice to see, but we want to take some action on this data. So what they wanted to do was take off this data from this dashboard and port it over to Marketo, which we're using to facilitate a lot of our marketing automation. They wanted to be able to take user login so they can understand who's logged in, who's viewed a dashboard and then send off different emails based on that kind of information. Typically what you could do with the Rails application is set up a background job process, that just when something happens, you fire off the event to Marketo. Marketo collects it, you're all good, or you do like a daily sync with Marketo with your application. The insights team didn't want to do that, and they specifically didn't want to do that so that they could test this API implementation that they built. So they built a really, really nice, easy way to be able to pull data out of insights and push it anywhere else you want to. So if I switch back to the slides, here's kind of, I mean it's just a simple curl. It's not anything that, something that most of you have probably seen before, but what's really cool is that you can drop any query into there. So Vince showed off that NERCLE query bar at the top where you can create a dashboard. Literally anything that you put into that NERCLE query that creates a dashboard, you could put that into an API call, which will then output JSON to you to use wherever you want to use it. So where that can be cool is if you're putting in a bunch of custom attributes about your customer size, you can use insights, for example, as an aggregate store for all that data. We'll take all your data, aggregate it up, and then you can just pull it out with performance data or however you want to view it, slice and dice it and then put it wherever you want to. So this is kind of what the flow looked like for us. Obviously we're taking it out of insights. In our case, we were pushing it into our APM app only to use it as a background job processor, which then we can push it over to Marketo from there. Right, so we were basically just using APM as the server to run the process. But what I think is really interesting is that if you switch Marketo with anything, be that like a BI dashboard or really anything that you want to push data into, and then consider using a tool like Zapier, iron.io, then you've set up a process where you can take data out of insights, process it in Zapier, and then put it wherever you want to. And what gets really cool about this is that you don't have to be a developer necessarily. So if your CEO or your business partner comes up to you and says, hey, I want this data here, you might be able to just tell them, like, use Zapier, take the data out of insights, which I've already put in there for you, and then you can push it wherever you want to. I don't have to set up a new background job. I don't have to submit any more new code for this. All the data's there, and all you have to do is use it. So we think we're gonna go in this direction, hopefully using some tool. I don't know if it'll be one of these two, but I think it's a really cool idea to be able to use this data and integrate it with other tools. So the point here is that you guys can brainstorm and figure out the ways that you want to use it. So that's really all we have in terms of slides and examples. We were hoping to open up the rest of the talk just for questions to talk about in your relic. So, yep.