 Welcome back everyone here live in Silicon Valley in Palo Alto. This is theCUBE, our flagship program. We go out to the events, extract the signal from the noise. I'm John Furrier with Jeff Frick here at peer2.o conference getting all the data around all the networks, all the action and our next guest is the Ali Khan director, network engineering at LinkedIn. LinkedIn, one of the big three I call them in social networks. Obviously, you know, you got Facebook, Twitter and LinkedIn all really demonstrating huge success in the marketplace. Also, big fan of LinkedIn and theCUBE. We've been pretty much every Hadoop world since existence that we know their prolific use of data real time and so they got to have the network to support it. Zade, welcome to theCUBE. Thank you. Appreciate it. Great to have tech athletes on like yourself who are, you know, you run the plumbing for all the action for LinkedIn, right? So, is it stressful? It is, yes. It's fun. You got a big smile on your face. A smile of nervousness. I can only imagine the pressure. Obviously, huge success story, Silicon Valley, but you know, the growth has been well documented. Obviously, when public earnings are doing well, but what kind of rocket ship was that like? Describe for the folks out there the challenges that you had to go through, one, to standing up all those networks. Just what was it like and just give some color, you know, the new secrets, but like, give us a taste of what was like. Well, you know, a lot of it was wrong growth when I joined about three and a half years ago. The company was already formed and growing a lot of products and I think that the biggest challenge was, you know, how do you scale this, right? How do you meet the amount of, you know, hundreds of millions of users that we were scaling to? So, the challenge was keeping up, you know, the growth and making sure that your networks were stable and the site was always up. So that was a lot of the work. There still is a lot of work. We're growing users every day, so there's a lot of challenges there, but it's fun building networks at scale. Jeff, what's your take on LinkedIn? You've seen their growth. Yeah, I'm a huge fan and I think it's a phenomenal application, but I'm curious in terms of on that growth. I know there's a lot of like constant innovation inside of LinkedIn, trying to develop new apps and push out new functionality. Are you guys big enough now where things have kind of smoothed out in terms of the growth rate or is it still kind of lumpy based on, you know, pushing out new features or pushing out new things from inside the company? Oh, we're always pushing on new features. We are growing a lot of users, so that keeps the momentum going. It's never slowed down, so we're constantly sending new users and growing the network to meet it. And also the international expansion, I think, has been dramatically increased over the last several years. So, speak a little bit about kind of unique challenges as you increase your global user base. Sure, we have, you know, users all over the world. And every geographical region has a unique challenge from sort of making sure that we deliver content fast. And so we, you know, how do we build our networks to sustain that? So, we care a lot about that. We go into different markets and build networks to reach eyeballs as fast as possible and deliver content quickly. So, that is a fun and exciting challenge. So, talk about the Peering Conference here. It's about openness, a lot of education in the foundation here, first event. The network community is pretty tight. Pretty much the guys at the scale levels that you guys are talking about are a small community and it's growing. But a lot of new blood is coming in. You have a lot of young guns, the DevOps mindset is pretty much part of that culture of the new guard, if you will. What is the big conversation happening right now in the community? And why is it important for companies to get involved? Well, I think, John, a lot of it is that the ecosystem is changing, right? Before you just go in and buy transit and you just don't care, like the internet just happens for you, right? But as you build more content out there, you want to deliver these things really fast, right? So, the entire sort of ecosystem is changing. And so, you have a lot of people coming in and inquiring about, you know, hey, how do I, how do I do peering? How do I actually connect to networks so that I don't have to go through the congested internet? So, that is, we've seen a lot of that. And it's fun to see a lot of people come to this conference, you know, curious about it, to see how other people do it to learn. So, it's been really exciting. So, is it a roll your own market right now in terms of networks? I mean, that kind of saw that with the data centers and hosting people are kind of saying, hey, I can just have them host my own data center. That kind of roll your own concept. Is that what we're talking here? People say, hey, I want to bypass the congestion. I want to build my own routes. Is that what's happening? Well, people care about their applications a lot, right? They want to be in control. So, how do you be in control? You get involved as much as possible. You don't leave anything to a particular vendor. So, people want to peer because they want to control the packets going from their network to another network or reaching their eyeballs. So, this is all about making sure that in the end of the day, you know, you're building your application to a certain level where you're in control of it end to end. And peering is a huge part of that. And what's the challenge before and after? You have to look at, you know, before peering, open peering, to after the ideal state. Is it how much pain? Is there pain relief? Is it an aspirin? Is it steroids? I mean, what do you get for this? I mean, what's the value? So, the value is, you know, comes down to a lot of it. You know, from our side, it's, you know, a faster performing site, right? So, at the end of it, you are improving your site performance. And that's generally what the content companies do. They want to get into peering so that they can deliver their content fast. So, in the end of the day, it's not just a little aspirin relief. It's actually, you know, long term relief because you actually have control of these bits and how you move them around. Just competitive at that point. Yes. Yeah. competitive as well as, you know, you know, whatever you could call competitive or you could just say, you know, user experience. Exactly. It's all about the users at the end of the day. So, how about real time? Because this is something that's come up, mostly the vias on here talking there obviously real time. They don't bump it up. You can't go anywhere these days without talking about mobile. And you guys have a great mobile app and you just live a lot with big data. Real time is real time. It's like latency is an issue, right? So how do you guys, latency is obviously something that network guys are used to. How does that factor in when you have an application driven marketplace that's going on now where people are really looking at the application centric view down at the network. Latency is a big issue. But how does that impact the real time for the end user experience? What are you seeing the issues around real time? What kind of make it better, faster, easier? Well, building better networks, right? Building better networks, connecting to as many networks make the real time piece real time. So as real time as you can get. So what what's happening is basically networks are connecting much more tighter with each other, not relying on, you know, upstreams or various other third parties, they're just going down to the core of the net core of the internet and just saying, let's connect at the core. And when you connect more at the core, that is where you get near real time exchange. See, talk a little bit about the conversations when you're setting up a parent relationship. So if it's a content provider to a content provider, and they've got different types of content, it seems pretty equitable, probably if you're of similar size, like in traffic. But what if there's a disparity there? How does that negotiation go? How are you assigning value? How are you trading value? Or is it just, you know, we're all in this together and we just want to deliver better performance? How are those conversations? So you mean between two content providers? Well, between two content providers or a content provider and not a content provider. So as you said, the the the ecosystem is changing. And we're before it was kind of the tier one providers appearing or the tier two providers. Now you guys are, right, are doing it directly with other content providers and other folks. So how's that? Sure, how's that kind of value established that that enables you then to put together a good relationship and a good arrangement? Okay, so let's talk about the content, the content. So kind of content providers, they're very open to peer with each other, because it's mutually beneficial. There's really no argument. Because, you know, there's API information, or they send emails to each other, whatever. It's just the two content providers just naturally form a marriage, right? It's easy, no problem. What gets a little bit complicated is, you know, when when ICPs, and then content providers, you know, one of actually peer with it, some of them are strictly like, you know, no, I would just, you know, you have to buy from us. And that's that's their stance. There are many that actually a little bit more open, they're there. They see the value in it, right? So if you're a heavy content provider, and they have access to a lot of networks, one thing to note is that transit price has gone down over time. So it's really, really cheap. So, so why do you want to charge a content provider so much? Money for it, whereas you know that at the end of the day, you're going to make that money out of the access people. So you benefit from peering with content providers, so that in the end of the day, you provide better performance to access networks. So that is actually a value that many providers see themselves. And those are the ones that see the value in the relationship with content providers. And so openly peer with them. Okay. And that is actually beneficial for their business at the end of the day, because they actually do make, you know, they can deliver the condo a lot faster. In the end of the day, their users are happy. They will not leave their networks, right? And stay on with them. So it's a mutual win and win. Yeah. So basically, the value of the content has increased so much. And then the revenue opportunity on the transit has decreased so much that they've kind of, they've kind of passed now, which is much more logical to go ahead and do that peer relationship. So at the end of the day, you're getting the money down at the other end of the pipe. Exactly. Interesting. So what's next on networks? Obviously, virtualization has a huge impact on it. We'll be covering VMworld coming up for our 50 or now. And, you know, Paul Moritz laid out his vision, you know, five years ago, now he's at Pivotal, you know, essentially the main software mainframe is the cloud. And the network has always been kind of the last stomping ground to really get tweaked and seeing, you know, it's actually moving down the stack versus we used to be saying, everyone's going to move up the stack in the old, old, old conversation, you know, now that's happened. Now all the force is coming down the network. SDN has been a big hyped up area, software defined data center. How is all that those drivers changing networking? Is it forcing behavior change, some technology change? Can you comment on that? Yeah, sure. So I actually just gave a talk on on peering automation and and and the way we look at it at LinkedIn and a lot of content companies like us are doing is the role of the network engineer is changing. Before it used to be a network engineer getting on a piece of routing equipment and typing away at the CLI and making, you know, routing changes, you know, real core engineering kind of stuff that feel good at the end of the day. But that is not the best use of an engineer. An engineer should be spending more time writing code to manipulate these devices. So these devices now have gone to stage where they've become far more sophisticated. So the trend is actually going towards writing software to manipulate the changes. So the network engineer spending more time or in the future will be spending more time having to shift away from the mentality of getting on the router and writing and making these changes as opposed and moving to like actually writing code to make it. It's a better use of that time. It's more challenging. It's more innovative. And it's the future. The old days we used to say I used to do a lot of networks stuff that network plumbers. Now that's changed. You mentioned software. So how is this definition change amounts? So it's kind of in terms of a dear me kind of joke, but you know, it's certainly it's kind of plumbing, I guess, but like with software you mentioned the roles are changing. So what do you see the key change on the software side? You mentioned productivity. Is there a new discipline when you're hiring folks? Is there a new skill set? Is it the same? How do you look at hiring? What do you look for? Is it the old days? Hey, good disguise smartness networks? Now, is there a new criteria for hiring? Yeah, so I think someone that thinks outside the box differently. Someone who has some programming skills is interested in programming skills as definitely appealing to sort of the next network engineer. So we look for people who are, you know, have a Python skills and stuff like that so they can actually write sophisticated code to actually make changes on routers and things where so that there's less human involved, everything is getting more automated. And that's the way to scale. It's interesting. There's a lot of talk here about the young guns that you guys, you know, the old the old dogs are training the young guns on kind of what was before and it wasn't always so easy and it wasn't always just so push button. Is it refreshing or is it frustrating when you got guys that just expect everything is going to act like like Google or LinkedIn? I mean, you guys are you guys are probably part of the problem in terms of being a really awesome and efficient application that people have built their expectations around web behavior between you and Amazon and Facebook. You know, people expect web applications to operate like yours does. Is it refreshing that they just expect that? Or is it frustrating because they just a lot of them don't really appreciate either really what's happening behind the scenes and the complexity and or you guys have solved a lot of the big issues now. So in the reality, they don't really have to worry about it and they can move on to some higher value items. I always think that you have to constantly reinvent yourself, right? In any any occupation, you constantly have to invent yourself. And I think the people that I've been doing this for 20 years, I'm still going to be on the routers. I'm not going to, you know, do this new SDN stuff or whatever. You know, I think it's just one of those. It's unfortunate, you know, because there's so many things that you can do when you just get out of that mode. And I think it's just the future is just going to be that. I mean, I'm sorry to say, but people will just have to step aside and let the new sort of way of thinking take over. And how do you do more automation? How do you do software manipulation to drive changes in your network? And that is the future. So we're here at the peer 2.0 foundation event, an overall event, and talking about networks, LinkedIn, huge stuff, pressure on the network. What's the number of active users now? I mean, the public number, I'm not asking the confidential information, you know, what the, can you share any public numbers? We have more than 300 million uses. And so get a lot of action on the thing. One of the things that Twitter has always been kind of, you know, been holding on that rocket ship with their, their fingers holding on, not trying to get it fall off with the infrastructure has been well storied with Twitter, the fail whale and trying to keep up with support. You guys have done a good job at LinkedIn. What's your biggest lesson learned in the past three years than this growth? LinkedIn has never really had any big public snafus like Twitter. But you know, I'm sure that there's been some channels, what have been your biggest learnings that you can share the folks out there? Well, keeping the site up, constantly worrying about it, keeping it up. That's that's, I think, every any network engineer, any network team, you know, cares about, and scaling and meeting the demand. That I think is, is one of those things that we got constantly. I learned, oh, my team actually learns, you know, new things, you know, user behavior, you know, patterns and things like that. So that kind of makes you think differently, you know, how do I constantly build a network to scale? And, and that's the challenging. It's fun. Yeah, I love the Jeff Living in Silicon Valley. You get a lot of pioneers, certainly, you know, we were based in the Cloud Air office when they started years ago, Amaral Adela came from Yahoo. And a lot of the web scale companies really were pioneers in large scale. Now on the social network side, you guys have done a great job of scaling up and in a modern era, I call it a little bit later than some of the some of the older web scale companies. But now every corporation is trying to do that. You're seeing almost like the vertical stacks, you're seeing DevOps. What is your advice to companies trying to do a LinkedIn? I'm not saying they're going to try. You guys are a black swan in a way. You're you're a unicorn or whatever you want to call it. You're not whether your enterprise is going to do what you guys have done, certainly from a scale standpoint, but they want to get there. Hyper scale has been a huge thing and it works out as well. So what's your advice to enterprises saying, Hey, you know, I want to be more like LinkedIn, but I'm going to be agile, want to be fast, want to be real time. What's your advice? Well, I would, you know, more probably focus on like, you know, what kind of, you know, I mean, I assume that you want to be a great content, you want to provide good content. So any company that wants to provide good content, like LinkedIn, Twitter, or any, you know, the focus is you end user care more about them, like make sure that, you know, when you design products, when you design infrastructure that you're actually making the end user experiences as as great as as possible. And that should be only driver because at the end of the day, that's what matters. That's what actually makes people come back to your side and be engaged. So so that is like the fundamental thing that I would I would say that every DevOps or infrastructure person should really care about is how do you make the site scalable and stable and and and and provide the best end user experience. Final word for you to get the final word into this interview. What's what is the peer to event like what it's the vibe here? What's the mission? What's your takeaway? So peer to is I think a great event for people who are a step back. So basically, peering has been, you know, around for a long time, but but it's been sort of a very select group of people, small and people know each other. The community is very tight. But we found that some of us found that there isn't a forum where new blood can come in or new people or people just want to find out, you know, what is what is peering? You know, how do we get involved or how do how does it benefit me? So this event is the first educational forum that actually creates that atmosphere for people to come in and and and and just want to learn that, hey, you know, how does Pandora do it? How does LinkedIn do it? You know, so we are able to share like our information on like, how do you get involved in the peering community and and and help build your system. Some big technical challenges to solve some big hard problems to solve, right? And to this business opportunity. If you get some challenge, right? Right. Absolutely. It's great. So Jeff and I are covering like a blanket. We appreciate your times the at Ali Khan appreciate that from LinkedIn director of network engineering. Big job at LinkedIn. Obviously great site that great social that we use all the time. And this is the cube we're live here in Silicon Valley in Palo Alto for peer 2.0 Foundation event. We'll be right back after this short break.