 Hey everybody, this is Tom Merritt and this is the first of what we hope are going to be not only a series of interviews But eventually a special to help people understand various topics around technology and the one we're tackling first is peering as you know We've talked about this a lot on the show tried to explain What's actually happening not only regarding net neutrality not only regarding? Netflix and their accusations of being throttled versus accusations of just having congested ports Level three and cogent with their peering disputes where that all fits in So that's the topic we're tackling and very happy that John Milburn Agreed to join us today now John you have a long resume that I couldn't possibly summarize Adequately why don't you let folks know a little bit of your experience that is relevant to this because you have a lot Well, I've been Internet user and manager since long before it was called the internet Long before Al Gore invented it started on Unix and internet systems in the late 70s at Berkeley My by training. I'm a nuclear engineer worked on particle accelerator stuff like that been in Korea for 23 years now built out a lot of the broadband internet infrastructure here in Korea and Delt with both the domestic rollout data centers all the peering issues and the interconnections within the region and globally So it's safe to say you you have a fair understanding about how the internet works Built a lot of it especially here in Korea, which is you know unlike the bandwidth ghetto Which is the US our internet is actually pretty functional And Jenny Josephson our producer is going to be sitting alongside to throw out some clarifying questions If we get too lost to down the rabbit hole as well. Thank you Jenny. Yes. I'm glad to be here as the designated normal DN is your position all right, so John let's let's start off because What we've tried to do on the regular Daily Tech news show is sort of give people the really broad overview That the internet isn't just you and your ISP There are things called autonomous systems. There are people that are transit providers There are CDNs content delivery networks, and then there's sort of this new weird Category that is sort of malformed Sometimes a Google acts as a CDN sometimes it doesn't and Netflix particularly is really odd If you could if you were to explain to somebody who knew who was familiar with those terms a little bit How the internet is laid out now versus what maybe they heard about in the 90s when it was backbone providers, right? How would you explain that? Wow, that's pretty broad Yeah, we're gonna start broad and we're gonna There's nothing new about Netflix In some sense, you know Netflix is a content provider. They don't have users They their their traffic pretty much goes one way it comes from from their service and they push traffic out There have always been such players, you know way way back probably the earliest one was the first big search engine the Alta Vista and You know they had their traffic all going in one way The one thing I think People don't understand in this talk about open internet is there's no such thing. There never was Everyone has always paid for their access in some way The large content providers be it Google eBay Amazon They pay to connect to someone else the the idea of there being these large Backbone providers or transit providers Came up because it was a natural way for networks had to be built a mesh network in the US between all the major cities and so on even In the days when we had lots of small ISPs in the dial-up days You had a few large networks that had those fiber links between cities, you know coast-to-coast and all of that the That in contrast to the to the guys that were very content specific You know as I mentioned the early ones and and eBay or Wikipedia AOL Facebook today Facebook basically has two primary data centers one on the East Coast one on the West Coast and they serve the whole world from those the all content providers have always had to buy their access from somebody all users and users buy their access from somebody And so all of these issues are basically about the relative economics of those costs of interconnection So the big question that started me down this road and this was years ago when I when AT&T CEO said Google and YouTube want a free ride. I was like I immediately went to what you just said Which is well, wait a minute. Everybody pays somehow It's not like they're they're they're freely accessing the internet and AT&T is mad that they're stealing bandwidth So what has caused in your opinion? The telcos to start to push this line that well We we are getting too much traffic are and the you said they push out traffic But another way of describing it I think Would be to say that their customers the ISPs customers are requesting too much traffic To be to be accessed from these companies. Why is that? Why is that happening? I? Again from a from a historical perspective. This is Ironic and hilarious to me if you go back to the late 90s early 2000 we had a situation where The telcos were the ones making all of the money all the content providers were desperately in the dot-com bubble Everyone's was desperately searching for a business model for a way to make revenue Content providers frequently approached telcos and said look your users are requesting so much information from us Why don't you give us a share of that revenue for all of the content? We're providing to your customers and the telcos uniformly told the content providers to piss off Well, yeah, as they would they're like you know and this is and this to clarify for people Was at a time when there were a lot more telcos than there are now because we've seen so much Consolidation over the past 10-15 years, but they're saying hey, you know what they're paying us for access You want to provide something out there for them access go ahead, right? Fast-forward 10 years later, and you've had a shift in in the the revenue models the lot of content providers are You know have figured out how to do this Netflix is always the poster boy for this because their users pay to access it and They send a lot of traffic It's just the nature of its content with video streams is that that that it's it's fairly high bandwidth the Telcos are You know a bit a bit disingenuous a bit schizophrenic here where On the one hand they're providing a service there for the customers pay for that access and pay for traffic But at the same time every telco considers its own network as its own property and That anybody who sends lots of traffic into that network is potentially degrading the quality of the total network This whole term of over-the-top OTT is is one of the dirty words for all telcos it and and and it and what it comes down to very simply is The telco is jealous of content providers who are able to deliver their products to the end customer and the end customer has a direct Economic relationship with that content provider and the telco doesn't participate in any way And and that seems to be based on their historical model as a telephone company If you think about it is in the way of 900 numbers and that sort of thing and the telephone company owning their wires That all makes sense right and and you know telephone companies are in a tough situation here. They are They are accustomed and they still have lots of people with a business culture inside the company of We build this thing for a very large capital expenditure and Then whatever service runs on top of this It's an it's a marginal incremental cost for us to provide but we charge much more for it because That's where we get the revenue that lets us build more infrastructure later to to to pay for that infrastructure We already built and to Build more infrastructure in the future. So we must be highly profitable To serve the public good and telcos which started out as government-owned companies in many in most places You know absolutely have this culture deeply ingrained into them and and you know they they resent any type of competition and Being unable to participate is Particularly egregious to them and cable companies would be similar as well For different reasons, but they have carried The content to people and and collected money because of that content. It's even worse for cable companies. You've got Yeah, I say the most hated company in America Comcast But their their their primary business is delivering that video content the TV signal to you now that they're heavily moving into data Players like Netflix who also deliver video content Are offensive to them they're directly attacking their core business model of delivering video and so that the The tensions the stage is set for lots of tensions between these larger players and it's become a bigger issue as there More bigger players. It's become a bigger issue as People become court cutters Now I understand how a company might say okay We've maximized the revenue that we can get from the customer itself, right? They're just not going to pay anymore right now And we we we don't want to sink that money into new investment until we can budget Some more money coming in and that's why they're going after the content providers and saying well Netflix Maybe if you paid us a little money That you know that would help that that that all makes sense from their point of view as you've explained it But it is unique at least I think unique to the United States that this is playing out this way I know European telcos would like to be able to do this I've heard that expressed in a couple of places, but in the United States. It seems like they are able to do it Why is that? It's very simple issue in the US the most People and businesses in the US you have for for your fixed line broadband internet access you have at most two choices for the vast majority of the population and Yeah, this is basic economic Theory is that when you have only two players when you have a duopoly market you have implicit collusion at work that that you have You have in most cases a telephone company and a cable company those are your choices and They will have some relative market share and it's in both of their interests Not to cut their costs not to cut their prices, but to maintain Margins and market prices so For for those companies in that situation then their drive is to find a way they have a captive market of users No one else can enter that market To compete against them that as a user There's no place you can go but to one of these two and if they behave much the same way it's not a real choice and So that gives that Access provider leverage against those whose business model depends on sending content to those users now in in much of the rest of the world The situation is different There there are very few markets now that have monopoly or duopoly for fixed line access Here in Korea for instance Every home user has a minimum of four options You have three fixed line telcos and the local cable company In in I love pointing this out because you know Americans are Tend to be so negative about everything in China the boogeyman China But in China now you have a situation where there are a minimum of three and often four fixed line broadband providers and and Yeah, they're all state-owned in China, but they're truly really competing with each other and you have a regulator that is You know pushing for and managing that competition So the result of this is in Korea in China in Japan Our basic home service now is a hundred megabits symmetric We're rolling out gigabit service to the home and it's typically $30 flat rate With no caps Now here's the thing a lot of people say well, of course Korea can do that. It's a smaller region It's harder to do that in a vast expanse like the United States You also have folks who say well in my in my area. I have three or four providers so Addressing those two things first of all is is it different in Korea because of the geographic situation in your opinion? Second of all if there are three or four operators in the US Why aren't we seeing these higher speeds that we're seeing in other places with multiple operators, right as to Korea? No, it's it's it's certainly true that we have the topology and demographics to That that allowed us to drive those prices down very quickly. We have extremely high population density People tend to live in clusters of high-rise condos Seoul city itself You know, we've got a little over 10 million people the metro area is about 23 million people Which is half of the country's population so we we fundamentally have a hub and spoke network With with the traffic concentrated in Seoul all of this drives to the most efficient low-cost network Further we've got the our environment here is that we are a unique language country that the both the producers and consumers of Korean language content are Fundamentally here nearby and and so that means that our internet traffic tends to be These days it's about 99% domestic Less than 1% of the traffic here ever hits the international pipes and You know, and that's a progression because it went when When we started the commercial internet services in 96 we were at about 90% international 10% domestic Your your bits are crossing the Pacific Ocean quite nicely right now It's where you know, we're on a you know 200 some millisecond gap and you know sure we've got Said it's only about 1% of the traffic but 1% for us still You know my ISP that I'm using right now their connection to the US is about 20 gigabit PS and That's about 1% for them So there the more backbone in Korea is tremendous So so that brings me to that second question. Why? Okay, let's let's take Manhattan, which has a very high density of population and in many cases you have more than two Internet providers to choose from why don't you see that kind of capacity in some place like Manhattan or Los Angeles or Chicago? Yeah, a lot of that is is driven by regulation you know Manhattan New York City Manhattan particularly are just notoriously difficult to do any type of cabling work anywhere due to regulatory issues permit issues you know dealing with the city planners and then dealing with the Unions for any type of installation or construction work and that wasn't the case in Seoul rolling out. Do you think? There was very much By the city and by the national government there was you know very much a Can-do attitude of yeah, we got to go and push this and do it now part of part of that is Yeah, just that Total infrastructure of getting it done quick and easy and this is much the same in China And you've got Japan that's in between those two More regulation things are slower and more burdensome there, but still it gets done Another big part of this is is the type of networks that get built is is We have a I think because of the competition because of how it was driven in many ways We have an extremely simple network here in many ways you know the entire network is Just a great big Yeah, land and that's because of the centralized nature of the of the population No, it's absolutely not because of that. It's a it's a choice by the telcos based on the competition to Reduce the capex per user by using the Simplest lowest cost access technology to get the best let's say price performance Ratio out of what you install so I mentioned our basic service here now is a hundred megabit symmetric that means You know the internet pipe coming into my house is a fast Ethernet cable That is about the cheapest thing you can possibly do in terms of high-speed networking right now There we don't we we don't have DSL modems. We stopped deploying a DSL here in 2003 we stopped Because it was obsolete. This is ten years ago, and this is what you're still deploying. Yeah, yeah It's nice and cheap and all of that We went to BDSL then After VDSL we went to pure fast Ethernet probably around 2005-6 so if that's more efficient why the hell would any self-respecting AT&T or Verizon not do that Why are they dragging their feet along with DSL and and and resisting rolling out fiber? So partly the the requirement for DSL in many cities because your population density is lower Because the distance from the home to the central office is longer It means Yeah, ADSL is the thing you can use on existing copper wires And and you'll often hear this oh your speed is lower because you're farther away from the co And and that's all true. It's it's the distance and it's also the quality of those wires how much Parasitic capacitance they have crossover how well twisted It is there's lots of factors that go into that The so they are definitely trying to avoid You know changing their topology changing their you know pulling new cables in and so on Yeah There there is cost associated with everything of course, but you know they They're making the economically rational decision to Yeah Keep the speeds lower and continue using the old infrastructure as long as possible And part of that is just it costs more to build infrastructure I think everybody understands that but is part of it also dealing with the difficulty of regulation and permits, etc That's part of it. It's it and that all goes into the total cost of build Um But part of it is without a competitive pressure There's no reason for them to do that and and you can see those effects very clearly you have You know AT&T Verizon these guys, you know suddenly can roll out much faster broadband services in cities where You know the google has gone to roll out there Austin texas got fiber from AT&T. Yeah, indeed and and so yeah Nothing should be a stronger indication than that that that that competition is the true pressure for for for change and upgrade the Another big part of this what when I said it's a simpler network We've rolled out this pure kind of ethernet distribution like that. So the connection to my home I mentioned it's a fast ethernet cable coming in well that goes to Uh a hub in my building Where a bunch of people have fast ethernet connections into that and then there's a one gig fiber Connection from that hub that goes to the telcos office And from that it goes into another ethernet switch which aggregates into 10 gig connections and then into the router and the backbone Now how did competing telcos agree to roll it out so nicely? Well, the the each telco has its own network. They're not they're not using the same infrastructure There's overlapping infrastructure. It is not it is not the efficient way to do things in in terms of having a single infrastructure or or what what You hear about a lot, uh, you know cities that want to do their own fiber plants and so on We really don't have that here the the Often what happens is Someone who does You know one company may own a fiber bundle to some location and and they will lease it to the other telco There's there's cross swaps on the fiber there, but that's None of that is driven by government regulation or a requirement. That's just pure, you know, you know economics of companies, uh, you know Co-operation So that's interesting in korea. You don't have an unbundled local loop situation like you have in europe. No It's it and and each telco found it worthwhile still To roll out their own infrastructure It sounds like the government was more cooperative in helping them to roll out, but The other side of it. I mean what what caused that competition to happen was was it That you have that small geography It's partly that it was it was driven by You know the fact that we did have sort of three players to start with In fixed line and then in mobile as well We actually had five mobile players In 97 and it's it's consolidated to three uh the The Fix line was driven. There were there were three fixed line telcos and Yeah, again when you have three real competitors and one and you know a well-funded new entrance and and that that third player entered the fixed line market in 97 here And it was its largest shareholder was the electric power company And which had a very large fiber plant because they had been installing like all power companies everywhere For for the uh since the Early to mid 80s Every power cable that got installed also had fiber in it That that fibers embedded in almost all power transmission cables for All circuits and can also be used for data transmission So i'm trying to put this together because it sounds like Based on what you're saying that one of the issues in the united states is the historical nature of regional monopolies That exists for cable television, which is now rolling out data It has been for a long time and that exists for the regional telco operators Even when they broke up at and t they broke it up into essentially regional monopolies And those regional monopolies have been consolidating But verizon and at&t don't really run infrastructure over the top of each other in very many regions anyway Do you think that that's one of the problems that we've had is that we we never had multiple telcos covering the same area? Exactly that is a significant part of the issue the the The the local telcos have never had true competition Uh And the the local cable companies have never had true competition And when when that competition, you know reared its head as a possibility we did have it in long distance Right we had the mcis and sprints that they came up in long distance But never very strongly in the the local area when uh when it came up with the Especially in data with is p's where you had is p's just running over the top of the phone lines the the pstn That became a huge threat to the telcos And so you you saw that as things moved to dsl broadband type connections the The us telcos saw what had happened elsewhere in the world and took very effective measures To basically block and kill off all potential competition in broadband by You know By on the surface saying it was all unbundled and available to all comers But the the fact is getting Yeah, they're whether it's their local Uh puc public utility commission regulators or national they put lots of restrictions on What equipment could be used what kind of testing and certification that equipment had to have How the equipment had to be inside the telcos facility that access had to be scheduled with the telco They they they they created lots and lots of non regulatory But regulators supported blocks Uh that just killed off the economic viability of any type of broadband competitor They did it very effectively and uh, and so you went from a situation in you know 98 99 in the states you had a choice of 20 to a 50 isp's wherever you were And uh to now having that choice between one or two uh The and and to boot the telcos got reclassified on data as an information system So they didn't even have to bother putting in the non regulatory blocks anymore. Indeed. So uh, uh that that Uh exclusion from any common carrier type Uh, uh regulations has been a godsend for them Although I have noticed that they use the common carrier Legislation when it's beneficial for instance verizon Uh, there's a couple of stories recently about how they used their classification as a common carrier to be able to roll out Infrastructure because that that made it easier and it made it cheaper in some ways and they got easier permits But then they were able to use those lines to have their information system rolling out over it Because they were like, well, we're going to use this for telephones So we want those old regulations to apply in the rollout But then when we use it for data, we want the new regulation the information system to apply They play of course That that that you know, I would do the same anyone would do the same So so now so that sets up pretty good Why we have the the isp sort of in control Of things and why they're able to say, okay, we've got a captive market You know, there may be a little bit of wiggle room between us and the cable company But we kind of know where we sit as far as what market share we're going to have So where do we get an increased margin? Where do we get new revenue? We have to go to the content providers now Explain if you can and if you can't that's fine What is happening between transit providers like cogent and level three? Because isps and these transit providers are having disputes the transit providers accuse the isps of deliberately not upgrading their equipment Thereby causing congestion the isps say the cost of the equipment upgrades should be shared And this all just sounds like gobbledygook to the average consumer Yeah, and it's it's it's all True You know, uh You know these guys are pretty careful to not Directly lie about these things when they make these statements. They're there's they're always true from some perspective uh so In some way cogent cogent is probably one of the Uh outlier good examples of of what's gone on here so Cogent is a network which you know is well known for cheap access cheap cheap pipes and they If i'm a content provider and i can't afford to directly connect with everybody go to cogent They'll give me a good deal and make sure my traffic is accessible. Is that what you're saying? Well, they'll give you a good deal. I mean the the the joke amongst the I'd say the the Core network engineers, you know For for from many years ago. We said, you know, there is that is that bandwidth or cogent bandwidth? Where it was it had always been, you know, sort of a derisively known as a lower quality bandwidth A lot of a lot of cogent's ability to keep their costs down comes because they had Legacy peering connections to some of the large networks Way back when cogent bought a company called ps i net which was one of the very early backbone isps And from ps i net they inherited The the the peering agreements that ps i net had with folks Sprint AT&T mci Which is now Verizon's backbone, right the old mci net is is Verizon's backbone that goes back to the unit ways days. Yeah in indeed so so ps i net Cogent inherited those contracts And the the the peering agreements with them and that gave them Essentially free connectivity to many of those networks Now all the details of these peering agreements, you know, they're they're always done under non-disclosure agreements And so, you know that the finding the details of them out is is always quite difficult But yeah Typically there is nothing binding in those agreements about Either schedule or requirement to maintain Uncongested links You're talking about the interconnection between me as the the the client of a cogent And and cogent where they with and so the ones in dispute now are between a an isp say like AT&T or Verizon They have places where they plug into cogent's network, right? So it's it's the agreement between AT&T and cogent Probably has No anti congestion flaws It's it's the typical thing in these agreements is that both sides will upgrade at upgrade at their convenience when The links are congested and And So on and yeah, there's a cost associated with with changing those links Typically they meet at multiple points and so some some link it might be congested in San Francisco, but not congested in Dallas and then Someone like cogent might try and engineer their traffic that if it's going into AT&T's network Even though the AT&T customers in San Francisco, they might reroute it to Dallas to put into AT&T's and that's carried on their back bone There are lots of ways to people why some they're like, well, wait a minute Why if if the problem is between cogent and and netflix traffic coming over cogent Why does it work sometimes and it does in others part of it is the number of people watching at any given time But part of it would be this kind of like network rerouting and and shaping Right that and that's you know, we that's called traffic engineering in general so so Yeah, the the the best Operating engineers in these networks try and look for these problems and try and adjust The routing to the extent that they're able to in their own network. They try and adjust it so that Their traffic goes, you know, uh through uncongested paths Um, I want to jump in with a question and I may take us on a slightly different tax So forgive me, but one of the things that I'm hearing When you're telling us about the way things were say in the late 90s Is that you had a lot of competition in a lot of markets and now I'm hearing you tell me you have A duopoly in most markets. So I guess my question is between 1999 and now What happened to the climate? What was there a change? Like what what would you say is a huge difference between then and now? Oh, the biggest thing was the growth of broadband you had you had all of that competition when most of the internet access was dial up Because everybody could do a dial up isp and the economics made sense because if you were the isp Uh You were receiving phone calls from your users Well, you didn't pay anything to receive the phone call And in fact if you if you you could make a deal with a what we called then clex the competitive local exchange carrier A competitor to the local phone company Some of these clex had a business model of partnering with the isp so that when You the user made a phone call to to me the isp It got delivered from your phone company to my Competitive clex phone company and then the main phone company actually had to pay a per minute settlement Feeded at clex for terminating for receiving the call of his user now the The the telcos hated this of course Because they had users who would be on hours and hours because it was all flat rate phone service then the uh, the clex Received the money for just a simple interconnect to the incumbent telco The isps Basically got charged nothing for any of that connection And and so, you know the telcos reacted to that and as broadband rolled out they they made it They created structures and uh, uh, that made it economically impossible For the clex or the isp to uh survive in that environment But is there an alternate history? I mean, I guess that's what i'm asking is is there a way that that could have been avoided or It seems to me what we're fundamentally saying in this is that that this is A bunch of big businesses that that That we're like no, this is ours and we're just going to hold on to it And we're going to make it difficult for others to start and And now we're here in this mess where if correct me if i'm wrong But we are significantly behind the rest of the world in terms of the speed of our internet in most parts of this country I mean first tell me am i right about that Oh, you're certainly right about that. I think that you know the the us, you know just in terms of Average access speeds. I think the us ranks something higher than 20th in the world now It's in the 20s or 30s And in terms of the cost per Units of data It's amongst highest Uh, so so it's behind in many ways the right the the alternate history Um, you know, there are many examples. So one of the best ones is the uk Um, and they split up british telecom Uh And but rather than split it regionally what they did is they made BT the service company and bc bt the infrastructure company So that bt wholesale Uh Was the new company? Uh That was was split which owned all of the physical infrastructure, but their mandate was to sell to all comers So all of the other telcos isps can run on top of bt wholesale infrastructure It goes back to what we were talking about earlier of the united states creating these regional monopolies, right? the the regional monopolies were You know that goes back to the I think what 82 or 84 split up of atnt uh But right the opportunity You know was still there to to split up the infrastructure the telcos have always considered All of that infrastructure is their own property. And this is where the regulators missed a beat I think is that To to not take into account that all that telco infrastructure Was originally built under monopoly conditions Uh, and and where their profits were guaranteed by that Yeah, monopoly environment and so The public absolutely had an interest in how that infrastructure would be Uh handled going forward. In fact, what happened is the uh, the regulators, uh, just Let it all loose gave it all over to the to the uh shareholders of the telcos and Did nothing imposed no requirements Of any meaningful type To create competition at the you know The FCC and the public utility commissions across the us have almost uniformly failed in serving You know the uh, the the social interest Of maximizing the benefits of society that they're supposed to do and in fact, they're almost all you know Uh Working for the benefit of industry And and yeah, I say this is very easy to compare and contrast with a lot of other countries where you do have true competition And it's largely because The regulators have behaved differently So so now we have the story, uh, where the government, uh, favored a monopoly Allowed an infrastructure to be built up in some ways they it was subsidized in part If you go back to the depression era, there's certainly lots of works progress administration stuff contributed to that infrastructure Some of it was funded directly by the company They break it up, but they let them keep the infrastructure However, they don't make it easier for anybody else to roll out infrastructure. Certainly not in the local level They have regional monopolies so that we don't have a situation like korea where you can have three or four different Telcos in a region competing and driving each other But you also don't have a situation like the uk where you have an unbundled local loop where the infrastructure provider has a different mandate than the service provider So we end up with the limited choice And then we end up with what we were talking about with the cojans and the level threes and the transit providers Having had It sounded like you were saying sort of a spirit Of of of good nature a gentleman's agreement that it's in our best interest to keep this interconnection working So we're not going to write it into the contract who's responsible, but over time will of course We'll be wanting to upgrade our our connection infrastructure. Is that How it worked until recently? well, it's uh In the early days it would absolutely was the gentleman's agreement and it was uh, it was a lower level agreement largely between You know the core network engineers and you know If I wanted to upgrade my link Yeah, it's sitting in korea. I wanted to upgrade my link to telstra in australia You know, I didn't go through the business executives blah blah blah, you know I picked up the phone and called jeff houston and said jeff look at the link. Oh, yeah, it's a little full Okay, let's upgrade it. Okay. I'll tell my business guy. You tell yours and uh Yeah, a lot of the the early internet was done that way and I mean that's how we made big pipes throughout asia was uh Yeah, the same way it was done between the ispes and the states It was a few core network engineers who just call up each other and do it The when they when they came to writing these contracts The thing was nobody wanted to be bound Because we you know, we knew there's so much uncertainty. We don't know how this is all going to develop But nobody wanted to be bound in such a Peering contract to things that would create You know capital expense burden where you don't want to be forced to upgrade You don't want to be forced to buy equipment so all of this is you know, it's uh You'll often write, you know commercially reasonable effort is is one of the code words you use in there Again, I don't know what's in these uh At FCC chairman wheeler seems to have picked up on that phrase recently for sure And and uh, you know that I thought it was funny when he when he used that because uh, you know I've seen that in so many of those contracts And it's it's a it's a wishy-washy word. We put in these contracts so that we can do whatever the hell we want That's what it seems like it seems like that kind of phrase as that Realistically pragmatically. That's what it is So it seems it seems reasonable then it doesn't seem reasonable But it seems it seems to make sense to me That an isp would suddenly catch on to the fact. Hey, we're not required to upgrade this equipment Uh, and in fact, we could have a little leverage here because we want to find this new market to make money We've made as much money off the customers as we can. We're not feeling any competitive pressure there Let's figure out how and and that would be a very defensible and yet, uh effective way to kind of nudge Content providers into striking a new deal by just saying you know what no, we're not going to approve any any equipment upgrades right now right and more than that you've got You know, let's go back to netflix particularly. So you've got netflix pushing all of this traffic out there netflix wants to pay as little as possible to send that traffic they They go out and make deals and you know and they they they go out and ask lots of Backbone providers or cds and so on they they ask them all for You know what how much you know, will you give me or how much? Yeah, will you bid for this level of traffic? the they want to pay as little as possible so cogent comes in with a low ball bid and So netflix says great Cogent out yeah, I'll do this and you know and cogent is planning on Say sending their traffic to you know comcast end users And they can do it cost effectively because they have this Low cost or no cost pipe into comcast Well at some point the traffic levels there become so large the Comcast says Whoa, you know, you know The links getting full. It's congested. You want to upgrade? Well You know we comcast know that You cogent are being paid by netflix to deliver this traffic so share that with us So you cogent pay us for this upgrade in this link We want a piece of that Yeah, cogent You know desperately does not want to create a precedent of paying for connections that have traditionally been free for them And so they bulk and you get a game of chicken And and this this has often often from the beginning been the case in peering issues between isp Is it's a game of relative power? Of You know and it it is a game of chicken who's going to blink first whose customers are going to complain more Who needs that connection more? uh Yeah, the the extreme example of that is uh Basically every isp in asia when we started up to connect to the internet We had to buy a circuit across the pacific and then pay somebody to connect into their internet Port you know someplace on the west coast and So we were paying the full cost of getting there and the full cost of a port To talk to the internet through that transit provider Nobody in the us has ever paid anything to connect to the rest of the world Because the rest of the world has to pay to connect to the us All right, so Before i i'm circling us towards the what do we do about this question? But before we get there I do want to talk a little bit about netflix in particular because what I do think they're doing that's interesting is they've come up with a clever way of Avoiding the cogents of the world when they can get the isps to play ball They can go to a cable vision and say, you know what? We'd like to interconnect some servers directly with you and that will help your customers They'll be happy Because they'll be getting these perks and the perks have changed over time for a while. It was a higher resolution Now it's just a higher level on the chart of who's greatest at being a netflix provider on an isp But what's happened recently as people probably know is comcast and verizon Have both struck deals to say yeah, we'll interconnect with you directly But there needs to be a there needs to be a payment here We're not going to just put your server in our system We're going to interconnect somewhere else and and we don't know what the settlement is But the implication is that netflix is paying the isp for the connection A lot of people look at that and say that's a net neutrality violation Explain what that actually is and explain why comcast and verizon can get netflix to pay whereas cable vision is essentially paying netflix Yeah, so so All of these are just Levels on a scale. I mean there's there's there's a continuum here As we said earlier everybody always pays for their access at some level the uh Netflix, you know, they don't want to do simple caching like a lot of the cds do They want, you know all the connection to come from their own servers So right they make deals where with with some isp's where they come and put a set of servers in in that isp's core network We don't know What if anything they're paying there? uh We you know We know that for some small isp's who've asked for that service Uh netflix is actually asking them right to pay netflix To put those servers in This comes down to again. It's uh, it's relative power Of of who needs it in the case of a comcast uh netflix Cannot afford from a business point of view. It's a not deliver traffic to comcast customers They need to do that somehow because of the sheer numbers because cable vision doesn't have much any more competition than comcast does in most markets But there are much smaller footprint And and right it's it's sheer numbers number of customers and and and it's also that that you know the nature of the company Comcast is known as one of the more aggressive players in this area Yeah, yeah, uh, uh, you know, they're they're tough part of it is Definitely, you know netflix is not you know, uh Without blame here netflix, you know, uh You know went to And and it's you know, it's cogent Recently, but they were at level three before that for their uh, comcast traffic they they They actually had to move from level three to cogent because uh, this was probably Oh three years ago uh Level three's interconnection to comcast got swamped with netflix traffic and uh And there was a big conflict You know back then Comcast sees this huge traffic coming in through appear and says no when the traffic's that big That guy should come and negotiate directly with me You know you netflix should make a deal with me comcast and you know and will come up with a commercial solution You netflix are already paying somebody Whether it's level three or cogent to deliver that traffic So don't pay that intermediary who's not paying me Pay it to me directly Seems perfectly reasonable Right and so you know You know, then it's like the old oscar wild story, you know, we we've established what you are now. We're just negotiating price uh The the and and that's what this really all comes down to in that in in this particular netflix thing It's just an issue of who gets what piece of the pie And and comcast has a large enough customer base That right they can threaten to You know cut off or or actually their threat is to just do nothing Right and and netflix service just suffers and people decide i'm not going to pay that eight dollars a month anymore Indeed it continuously degrades Over time, you know as you know because that traffic needs to go up people want more hd There are more subscribers blah blah blah The so by doing nothing by taking no action at all They degrade netflix and they also degrade every other cogent customers Uh that wants to deliver traffic into comcast that they are also degraded Which which by the way explains to people who are like yeah, and when I use a vpn My netflix service is great because suddenly it may not be going through cogent for so Yeah, if you if you're if you're using the vpn it might be going out through a totally different link Yeah, right and not traversing that that's a congested line So this is where I have been and I'm curious what you think I think that the netflix disputes And and even the level three and cogent disputes are perfectly natural. They're they're not unusual They're certainly not examples of net neutrality gone wrong Uh, they are the way business is done in peering and it seems like that's what you've been saying is these aren't new kinds of arguments The thing that's new in the united states is that the isps Have more leverage and power than isps normally have had in the past And do have in lots of other parts of the world Yes, exactly the the because it's concentrated down to You know in most areas Uh, a monopoly or duopoly of broadband access There the power of the big ones has gone up And so you've got AT&T, Verizon, Comcast who are the really big ones Um, and you know Time Warner As well And these guys You know any content provider has to deliver to the users in those networks. And so they do have That you know, uh relative power in such negotiations the That as an end user You know and this is part of the frustration you get in the u.s. As an end user not having much choice there it's very frustrating because You don't have an economic vote in this you can't change your business to someone else to to punish a badly behaved telco So john I know I know you don't have the power to do this and you've been very kind and in helping us understand All of this but uh, one of our listeners sub guns Wants to know what is it going to take to get average access speeds to be competitive With the rest of the world if if you were put in charge of the FCC Do you do you have an idea of what you would do? Oh, definitely the the Yeah If it was me with the FCC the first thing I would do is uh try to remove all of these local county city state laws that try to stop um municipalities from Building and using their own infrastructure. I would like to encourage anyone with infrastructure to be able to use it quickly and effectively to create competition And the the it's it's not just FCC, but it's the it's the state puc's also that need to aggressively encourage competition The and and whether it's through You know municipally owned fiber whether it's through inviting in Google for making you know, it's clear google when they come into a city They ask for certain regulatory changes. They ask for a streamline permit Uh, uh and and construction Oh, it is screams bloody murder about that in austin at the same time hustling to get their fiber rolled out right so so so but you know The the An aggressive stance by such regulators to drive Competitors in this market This is the only hope To to to change this environment at only when the end users have that choice Uh Will the other market forces work themselves out and you don't you don't you know, i'm i'm just i despair that that Anything about these you know open internet rules that that that you know either side wants to put in or not put in They always have such horrible unintended consequences uh Yeah, that that uh They all end up stifling innovation and growth in some way Yeah, just opening up Municipal fiber allows anybody to come in and make really simple sort of ethernet based access Now you know You need yeah, you need to get into a backbone someplace But that's where you do have the level three and cogent and sprint players that that that sort of have those alternate backbones uh Most content providers are accessible there. It's really Uh This will work If those barriers come down, but but the the trend has been in the other direction The regulatory trend has been to lock up and restrict municipal networks It seems to me. Oh go ahead jenny. Yeah, so I guess the question that comes off of that is We're essentially talking about um What most politicians are always talking about which is you have to give control back to the local level I mean essentially what you're saying is this cannot Then the easiest way to fix this is at the very is in the last mile It isn't offering faster speeds Uh through the municipality And if we try to regulate it from the top down, I mean, I can't even believe I'm saying this But like if you try to regulate it from the top down Maybe you're not going to get the best outcome. So if you're if you're really talking about action Are you talking about the city council or the county or You know Concepts that maybe they hadn't thought they were in that business before I think absolutely. Yes. That that that the action Can be taken at the city level and we've seen that in a few cities uh But as soon as a city tries to take that action the The telcos and lobbyists will go and try and get state laws imposed. So so the The attitude has to be there from the FCC from the state public utilities commissions But then the action Definitely has to come in at the city and county level to Allow for cost effective and rapid build out by new competitors When that happens The investment funds will be there right now the investors won't touch things like that Uh Because all the cards are stacked against you So it it sounds like if I were the if if the FCC were to Just forget about open internet rules And focus on Encouraging that kind of competition and there's probably going to be different solutions for different marketplaces for different localities But saying localities that want to go like chat nougat tennessee and roll out their own fiber We we will put in rules that make it easier for you to do that Uh encouraging places that might be able to handle three or four telcos We're going to change regulations to make it easier so that the incumbents can't block you is Is that too complicated? Is that too idealistic? No that that that all makes sense Yeah, if you wanted to take this to the most, you know How could you really affect this very rapidly? Yeah, as again, I go back to the uh, UK model You know if if you had a really interventionist uh regulator and and uh Who wanted to to really change and create competition You know that way is pretty clear you're going and break up the incumbent telco You just break out the the their infrastructure as a separate business But that's not going to happen. I'm I'm I you know, I'm trying to get you know that one John I think I figured it out you let time Warner and Comcast merge You let them buy cable vision in fact you encourage quest AT&T and Verizon to merge But then at the very last second when they've almost created a monopoly again, you break them up into infrastructure and service That would be perfect That would create exactly the environment you need you just lure them down the primrose path Everything they want John this has been fantastic. Thank you for taking the time to walk through all this with us Thanks for uh, uh being so gracious and letting my nerves settle. That's uh, as I said It's the first time I've done anything like this. So, uh, uh, I had a few butterflies here Ah, well, you did you did a great job. Jenny any any other questions we want to Well, I think and maybe John you can just you can answer this quickly because it really is a question I think for tom which is all right. I feel like now as your dn your designated normal I have a better understanding of what could be called the politics and the economics of the hardware Okay, so so we have a hardware question and we have a hardware problem That's what I'm understanding which is that somewhere along the way We fell a little bit behind for the reasons that we've already discussed So what is the bridge between this hardware issue that we have been talking about and this economic and business issue that we've been talking about And this concept that I keep hearing day in and day out called net neutrality Now tom this may be a question for you to answer which is Where are we going next with this series because I now understand the hardware But I couldn't possibly tell you how it connects to what everybody's squawking about So you say when you're saying hardware you're talking about the infrastructure the infrastructure Yeah John what do you think is is there a bridge between net neutrality and and this infrastructure issue? the the It seems to me that the fear that's driving a lot of the discussion about net neutrality is that the ISPs Who own the infrastructure will get so much power That they can start now blocking and restricting the things that you can Access and this is a well-founded fear when When you have AT&T for years now now talking about OTT over the top services This is exactly where their mind is That oh over the top is something we haven't permitted and we're not participating in so we want to be able to block it Or slow it down or degrade its quality and so a lot of the The the the fears driving The net neutrality discussion are the fear of those large companies being able to throttle or block certain content and That is well-founded The We've seen You know Comcast do it years ago by putting Basically packet filters rate limiters in their network to try and Uh suppress p2p traffic and sky traffic and so on they have the torrent sky print. Yeah the The solution to that Is not clear. You know the obvious clear simple solution Again, it's when you have competition any isp who behaves badly that way loses customers. They move When you don't have competition the only way to solve it is is Through some type of regulation and but this is where you have to be careful the Yeah regulations that require You know a telco to for all kinds of access and Reporting and and so on One danger here a big danger is that you create a barrier for Competitors But now if you're if you're a new isp and you want to come into this world and you have all of these regulations about open access and so on And reporting and compliance it can create A both a laborer and a technical and a and a Reporting burden That actually drive your cost up so that as a competitor you're not viable again This that that i'm i'm saying the regulator needs to be sensitive to this to monitor it But be very very careful about putting in Reporting and infrastructure rules because it could become its own deterrent to competition and don't think that these telcos don't know that And that they're they're playing this Yeah, any kind of rule any kind of regulation Yeah, those They have you know lots of lobbyists looking at this from lots of angles and their overriding Goal is to make sure that new competitors don't come They got to where they are in part by making sure the regulations were constructed And just such a way that it made it difficult for competition To start against them Yep, and the new round of regulations will follow the same pattern So when you see them fighting the open internet order, they're actually not fighting the open internet order They're fighting to make sure it's implemented in a way that they feel will best benefit them That is because they're not anti regulation. They're anti regulation that is bad for their business model. That is correct Um, they love regulation. Yeah To me the open internet order is like treating a tuberculosis patient with cough medicine Uh, it doesn't address the underlying problem at all And in fact, it could make it worse because if it suppresses the cough Then you may think he's cured and the tuberculosis gets even even worse It is it is a band aid Uh, and I and honestly, I didn't realize it as clearly until we had this conversation today Why I was so interested in hearing Part of this versus the net neutrality part uh, and and it's because to me jenny the bridge is That the infrastructure problem is the thing that could actually cure the tuberculosis It's the thing that can save the patient or as the open internet order It might be able to do something but it actually might have worse effects And we really you know, we're not in control of what it will or will not do and it certainly doesn't solve the underlying issue Okay, that makes a huge amount of sense to me. You guys are awesome Good. Oh, that's fantastic. That was a great great way to pull it together. That's a lovely analogy also That's exactly correct that that that if you don't deal with the fundamental issues Uh, all this other stuff is smoking mirrors Well, we're gonna leave it there. Uh, john, is there anything you want to let folks know about, uh, regarding yourself before we We stopped the broadcast here. Nope. I do everything I can to stay stealthy. All right There's a particle accelerator right up here that I built in korea. Oh, that's fantastic Well, let's uh Thank you for for going a little bit off stealth for us today. I really appreciate it All right, take daily tech news show dot com Uh, is the place to get the daily show that discusses these kinds of issues and hopefully we're going to do more of these kinds of interviews And eventually maybe Take a special out of it that that really attacks this issue and and tries to sum it up. Thanks everybody for watching