 Hi everyone, thanks for coming especially on like kind of the most busy part of the semester to the final lecture of the mstdp conversations and practitioners lecture series For those of you joining for the first time Computational design practices and MS program in its second year that introduces students to a range of possibilities for design work at the Intersection of architecture urbanism and computation I'm sure everyone's heard this at this point, but my name is Adam Vosberg I'm the assistant director of computational design practices along with Laura Kergan who is the director of the program So this lecture series Intends and has intended to showcase practitioners who are working with computation on various media and subject matters at a wide variety of scales We've kind of accidentally been going up in scale with each lecture and are now closing with Lai Olson Who among other things talk about or work with internet measurement data and how it can both reveal and influence the key networks of Decision-making through which industry Policy standards organizations and academic research are inextricably the sometimes imperceptibly connected Laie's an artist writer and researcher whose work considers systems and the narratives of their histories She is the lead data scientist and measurement lab where she studies the evolution of the internet in a part-time lecture at the New school where she teaches critical computation core lab systems and statistics in society She was a 2019 artist and resident and movement research a year eight member of newings our encode track and partnership with Rhizome And was a spring 2020 technology resident at Pioneer works Her work has been shown a tech zine fair movement researches fall festival New York art book fair the internet archive to Centralize web summit and our networks. She loves the Phoenix Suns and the New York Knicks and that order and we'll talk to you about basketball anytime And as usual we'll have a brief Q&A afterwards So make sure to take the opportunity to ask some questions or share thoughts about life work. Thanks again for coming y'all Join me in welcoming Laie Olson When you write things like you love the Phoenix Suns the New York Knicks. You don't imagine someone having to say that out loud So thank you for Thank you for doing that. Thank you for being here. I understand It's a busier week at school. So I'm really grateful that you're here. I said my name is Laie Olson It's like the word lie and letter e if that's helpful to you I'm here today to talk about internet measurement Which I have learned a lot about through my work at measurement lab where I said I'm the lead data scientist was formerly director and I'm wanting to kind of just give some insight into the technical and logistical bolts of what internet measurement is And I'm gonna call this like the what section of the talk But and then also some practical applications relevant to policy Internet governance and industry But then also talk through some of the more Conceptual squishy artistic questions that fall out of this practice for me things that I sort of think about While I'm doing it that are these larger harder to answer more nuanced questions that I think are equally as important But are less often asked But again to start all you know sort of talk through the just what even is internet measurement Because the talk is titled how how tall is the internet sort of In reference to a joke that a lot of people make and admittedly that I also make when I say that I Measure the internet for a living because it just sounds like it sounds like a punchline It sounds like this like impossible task like you're measuring, you know, like the wind or God or you know The Kardashians or some things are sort of like incredible that you can't really wrap your mind around Especially something, you know because the internet is so big and you can't see it And there's obviously not maybe not obviously but there is a physical component to it But there's also a lot that you know extends beyond the physicality of it and is just sort of Surrounding us like this amorphous bubble that we're always within and always intertwined But again can't really conceptualize the bigness of it or the scale of it And so when I say I you know measure the internet it does it's it almost seems laughable because how how can you do that? What are even the units of measurement that we could conceive to apply to something as Ephemeral and consistently moving and big And what I found is when you start Trying to think about measuring the internet or really, you know Measuring anything one of one of the first questions you have to ask is you know What is it that you're trying to measure which I've I found actually and maybe you'll find this in your own work can really Quickly turn into an existential question because you're really asking like what is a good version of the thing that I'm measuring like What is the thing that we want to be able to track the growth of or the health of and so, you know Often and I'll talk about this more Internet measurement is sort of reduced to this idea of speed and availability. So like how fast is it or how? Accessible is it and those are interesting, you know, sort of contours and and metrics to Think about but there's also this open question for me around what is what else is there to be measuring about the internet? What else is there to care about in the infrastructure that we're building? And I think there's there's basically just more to be thinking about besides is the internet fast and available And so part of what I want to talk through in the later portions of this talk is like can we imagine imagine measuring? the internet as a way to imagine what else we want from it what other futures we want to realize within the infrastructure and and to Be able to to utilize within its structures Another thing about you know again measuring anything is that you're you know Essentially just choosing a metric to check in on from time to time or a way to sort of Standardize the understanding of so I think that a good analogy is like we talk about GDP in relation to the economy or you know Air quality metrics, but those are also good examples because there are many ways to measure the economy in many ways Ways to measure the quality of the air but we sort of settle on these specific metrics because they kind of gain a fluency of their own and they sort of gain a momentum of their own and so like speed and availability with internet measurement and That's what we see in other sectors too where we just sort of grab on to one metric and so there's this question of like What else could we be thinking about? But before I go into you know all these larger conceptual questions Like I said, I want to go through kind of the the basics and just have a shared terminology to talk about because again What is what is it even meaning to measure the internet? So this is a loose framework that I Use to try and understand like what is it? That's actually happening the one there's two things I want to say before I start is like there are so many ways to measure the unit there's there's there's endless ways that you can think of and This entire discipline I think could you know take an entire course So I'm giving you a crash one But there are there is like a framework that I think you can think about when thinking about internet measurement to kind of Ground it and to be able to think about different measurements in relation to one another So this framework is essentially that there's almost always a point a and a point B That someone is measuring in the network and we'll go a little bit more into what I mean by a network But essentially, you know, you're measuring from one point to another and maybe back and then you're almost always Incorporating some methodology about how you send the data. So how and what shape or how much or what speed or in what Size and and then also you're almost always collecting a metric along the way So you're essentially sending data from one point in the network to another and saying like I'm going to send it This way and see how it reacts under these conditions and then you're collecting metrics to understand What what what's the what the behavior of the network is in those conditions? So where point a and point B are in the network and how the data is sent and what metrics are collected along the way are What I think of as the defining characteristics of a measurement and are also what distinguishes them from other Measurements so again, there's so many ways to measure the internet But I think a way to sort of wrap your head around what is actually happening is to think okay Where's the point a where's the point B and what is the methodology that's being used? I kind of use this as an entry point because you know internet measurement like art I think can be really overwhelming, but when I'm looking at an art piece. I just think to myself, okay What am I seeing what colors are there? How are they organized? And what is the composition and kind of try and go from there? And so I use this like point a point B What's the methodology as a way to do that with internet measurement? Again, this is a bit like how we measure the economy by measuring GDP or like the price of eggs or the Unemployment rate like these are completely different measurements But they're almost always trying to give a sense of how the economy is doing and these internet measurements are often trying to do The same thing like how is the internet doing from this perspective? Digging a little bit deeper just as I mentioned so Point a and point B are always placed within a network and when we talk about the internet We're actually talking about a network of network So these are we're not you or I or myself right now are connecting to the internet our data is not only being sent through the Network that we're connecting to but also to other to networks that connect the networks to other networks so it's again the internet is a network of networks and When we're placing when we're doing internet measurement. We're often placing The point a and the point B Somewhere within that so you might be placing it closer to the user device You might be closing it placing it closer to where networks connect with one another But these are again the defining characteristics of a measurement with the hubs that Situation or in this diagram the hubs that connect these networks are typically called interconnection points And they're run by what we call transit providers or internet exchange points And the act of exchanging traffic at these points is often referred to as peering Not the most important thing for you to walk away knowing today But it's an interesting part of it in terms of there's so much more than Just us subscribing to Comcast or spectrum etc. Those these networks are part of a much larger picture and infrastructure that internet measurement Is often helping try and break down and trying to sort of segment and understand what's going on on those Lower level or smaller segments The other helpful concept to try and understand internet measurement worth thoroughly is Thinking about the protocols so to communicate with one another You know for example the sender and receiver when they're sending data to one another They use protocols And so product some examples of protocols are you know TCP, which you see in the transport layer here IP Which you see in the network layer protocols Grant TCP is thinking about how to get packets there in the right order IP is thinking about how to get them to the right place And they all sort of play a role There's a concept of modularity here where they're sort of depending on one another to do their job So they can do theirs but I think a really simple way to think about a protocol is kind of like a script or like an etiquette almost I as of this week started trying to learn French and The language app that I'm using is giving me a lot of sort of rules about how you talk to different people talk to professors talk to fellow students And I realized that's that's a protocol like that's a sort of protocol And I think you can think of these internet protocols as ways for computers to understand how to interact with one another so internet measurement going back to this diagram is Utilizing or is trying to understand how these protocols are behaving in different Environments when sending data from point a to point B. So sending data through these networks and then Examining these different protocols and collecting metrics along the way to understand how they're doing So when we Uh So when we ask these types of questions that or when we start thinking about how to measure the internet We can we can basically start asking like how how is the internet doing? How is it doing today? How has it been doing over time? And again, you're you're picking up on these lower level characteristics of how it's working to see You know, okay. Is it doing the thing that I expected to do? How well is it delivering data? How common is it for data to actually get there? If it's not getting there, what are the reasons be? Are they political? Are they technical? Are they something to do with the environment? How consistent is it that the data gets from point a to point B under these conditions? And most of the time I would say most of the time people are thinking about How to improve the internet such that if something's not getting from point a to point B the way that we want it to How can we improve the internet? Such that that happens more often And so as I you know as I mentioned at the beginning this can this can quickly become existential because it gets to this question of What does it mean for the internet to be doing well? Is it is it just that packets get from point a to point B? I think that's often how we think of it within the discipline and are just you know sort of teasing apart these different questions around or the sort of Micro improvements that we can be making for it to do well according to The idea that you know speed and availability are the most important aspects But you know I think there's there's just these larger questions to be asked of what does it mean for the internet to be doing well? And I and I I'll Invite you to think about that as we go But for some the answer to this question is actually quite simple So for example for internet service providers like Comcast or Spectrum time Warner, etc They mostly just for them doing the internet doing well is just that it runs smoothly Their customers are happy and people continue paying the the subscription fees So for them the internet doing well is really just a maximization of profit This is also true in a sense for cloud providers like Google Cloud AWS Microsoft, etc They're happy if their content that you know their clients are wanting to have hosted is available same thing for What we call content delivery networks or CDNs like Cloudflare or Netflix or you know Companies that run CDNs where they're just really interested in getting you as much content as possible because also ads come with that And so there's you know There's a very simple answer to this question from that perspective of as long as the profit is maximized The internet the internet is doing well And so all of these companies that I mentioned are doing internet measurement routinely because they're interested in keeping this sort of answer Answer they want to know you know What are the improvements they can make to continue having the internet in a way that internet work and run in a way that? Reaches their goals Academic researchers are often always are often also running these measurements but often on a I would say a smaller scale and they're often thinking about again these sort of like tight corners of you know you can imagine like Just like tightening the screws and things that can make these smaller improvements to make the existing system function better But what's interesting is that you know often these even though all these different groups are measuring the internet and sort of keeping tabs on How it's doing according to their own interests They often don't share it like there's not much of an incentive to make it public and there's also a lack of often a wider perspective outside of the parts of the network that they control and And same for the academic use case often You know researchers will sort of run an experiment either in the real internet or a test bed that emulates an internet But often on a smaller scale such that The the data is only talking about you know a particular part of the network one one network in the network of networks And this is the void that measurement lab was created to fill this idea of needing to be able to share Public data as well or to share internet measurement data publicly as well as be able to measure the Internet at a larger scale and again in the quote real internet where sort of real packets and real content were being hosted And so measurement lab was founded in 2008 in part by event surf as a way to try and address this idea of like how do we measure Something as big as the internet in at a scale and over a long period of time And in a way that could be shared publicly because as we'll talk about there's a lot of public applications There's a lot of applications in the public interest of this kind of data and of this kind of research So just to continue our shared vocabulary I'll talk a little bit more about how n lab specifically measures the internet remembering that there are so many ways But as I mentioned, there's always a point a and a point B almost always in internet measurement and for us The point a and point B are the user meaning, you know user devices like a phone or a tablet or even a router and then a server within an interconnection center. So going back to the Vocabulary we introduced before ISP be in the situation is ISP that's connecting other networks. And so historically, that's where M lab has placed our servers as a way to measure the what we would consider a full path or You know the entire way or the entire journey that a packet has to make to Traverse the internet As of 2022 we started also adding cloud networks or cloud Servers in cloud networks as a way to also measure where more and more content is being hosted But just to give a you know a concrete example of what specifically M lab does. This is how We've gone about measuring the internet in terms of where point a and point B are And as I mentioned, there's also, you know, the other defining characteristic of how one measures the internet is The methodology and so on our platform We're able to host up to 12 but right now we host for different measurement methodologies and the way that those are sort of technically or Logistically implemented is by hosting the server side of the measurement meaning where Acting as the thing that gives data and then clients are able to run tests clients typically meaning like a user device When they run a test they run against one of our servers So they're able to Traverse the network through the path that a packet would take through from their device into the interconnection point And when this happens when it when a user runs a test from a client They are then sent to our server. They're shown the results and then their Result is archived in our public database. And so over 14 years of doing this we've amassed 6 billion rows of data and collect about 4 million per day So you're probably wondering where does that data come from or who is running these tests? This is a map of Where our servers are in the world So one of our tests is called network diagnostic tool or NDT and it is integrated into Google search So if you Google how fast is my internet or speed test or something similar? We're the thing that is Integrated into that search page so we get a lot of our data from this integration again 6 billion over the course of 14 years and When someone runs this test they're shown an upload speed and a download speed and again We then archive that result and publish it publicly in our database and Essentially what it's doing is it's just opening up a connection a TCP connection for a single stream to be specific a TCP connection for 10 seconds and is just trying to shove as much data through as possible to make a calculation about how again fast or Speed the the connection is So just breaking this a little bit further We initially place servers in the interconnection and Facilitated tests coming from the client as a way to measure user experience Thinking through the idea that when you're accessing content you you're not only staying within your own network so there are speed tests that also measure within the network, but we Historically have placed servers outside of it as a way to sort of account for the fact that when you go and fetch content from the Internet you actually have to go into the internet and so this is a just a differentiating thing or detail between Some speed tests and our speed tests But through all this work, I've learned Oh, sorry. This is that this is an example of the data that's being archived in big queries So this is actually just a subset of it But essentially this is just a JSON blob that's giving you information about the or connection Upload or download depending on which way the test is going When you're testing against one of our servers So Through all this work, I've learned that there's there's a lot of opinions about What speed means and what is what you know how we should be measuring it? And if that's even the right term and there's a lot of debates about basically all the details I just shared if you got someone else up here, they would say something different And so with the next person the next person there's a lot of different opinions about what? Should be measured and I I think you know even though a lot of these conversations get quite technical and quite intricate and You know really in the weeds I think I find that they're also kind of beautiful because they're essentially thinking about like what matters like what should we be measuring? What is the thing that matters to us about the internet? What is the what are the characteristics that we care about seeing and checking in on and day-to-day saying okay? Is this thing still working well? What is what should we measure and what behaviors of the internet matter? And so again, even though you know these are often really detailed Conversations I think the measurements help us to sort of one all be looking at the same thing and have a conversation around Details that we can all see or you know Understand and then it's kind of like also this like sifting through the noise and trying to understand What are the characteristics that matter and what are the things that you know with something like the internet that's continuously evolving and changing over time? What are the things that we want to consistently? Measure over and over again and in the case of mLab publicly and at scale and something That can create a sort of baseline for how the internet is changing over time The internet is after all and an infrastructure and there's this quote from this essay from Adam Rothstein on Rhizome it's called how to see infrastructure and In it they say Infrastructure's power combined with its lack of visibility is the stuff of our society's physical unconscious So I think there's something about measuring the internet like over and over and over and over again over a long period of time That you know allows the internet to sort of drift out of that societal unconscious and become seen and and When something becomes seen then you can ask you know better and more specific questions about it And ask, you know, what is it even doing? And is that what we want it to be doing and and how can it be improved? I often think of internet measurement through the the lens of her through the this image of this piece by Franz Erhard Walter Where you know, I kind of feel like the internet is this thing That's always around us and holding us in this in this way in a way that we can't see and I think of internet measurement Is almost being able to see this fabric like see this, you know Without internet measurement or without being able to see the infrastructure understand the infrastructure We're sort of held and like tethered by this material that we're not really aware of and I think of internet measurements And internet measurement sometimes as a way of being able to visualize this fabric That's tethering us all together and understanding the physicality of it and understanding the way That it exists in the world So with that shared vocabulary, I want to talk about just like who's actually using this data What do they use it for and in this section? I'll talk about mLab data But I'll also just talk about internet measurement or other internet measurement projects and the efforts at large so Broadband mapping and the digital divide in the US is something that I you know This could be a whole semester as well, but I'll try and keep it shorter So here is the map of the digital divide according to five different data sources three of which are crowdsourced like mLabs And it's including mLabs And two of them federal one is actually the census data and one is provided by the FCC The red areas are that are those that are reported to Buy these data sets to not meet the threshold of the definition of broadband in the US Which is 25 download three upload again that sort of Centering around the metric of speed but in any case the all of these data sets have a way of measuring speed Have a way of measuring that that threshold and the red or the area is the more data sets Say that that area is not meeting that threshold of 25 3 So one thing to start is like this there's many versions of this map Many of them do point to the conclusion that there is a digital divide meaning that there are some people without access in the US Or adequate access, but there's you know mapping is a very contested Project and and and definitely not only specific to broadband mapping. There's almost always a Narrative that a map is trying to communicate and you know There's always, you know an effort to make it as objective as possible I mean, I shouldn't say always there's often an effort to make it as objective as possible But it is a difficult thing to do is what I was what is all that I've really learned So this is one map of the digital divide And as you can see it's it's there's there's an issue here according to multiple data sets Part of the context here is that broadband or internet in the US is not regulated as a utility like water and electricity It's very much treated as a commercial product But that said the the federal government and the state governments do have an interest and an understanding that especially post-covid that The internet is a vital part of living and participating in the economy and the democracy So they they have a stated interest in improving the state of the digital divide But their solution to this is I find to be quite a funny one So to measure internet availability in the US every internet service provider is required To to tell the FCC what level of service they provide and in what area which essentially means the ISPs are grading their own homework They're just you know telling the FCC What's what they don't have to provide any measurements. They don't have to provide any evidence of this They just fill out this form and say I provide service here And so again, this is sort of like, you know, all water companies are saying like it's safe Don't worry about it And so this is the FCC map of 2011 it's changed some but for the past 10 years remained Pretty similar and there's two of one to talk about the map the Oranger it is The more service it reportedly has but basically everything that's orange Is what has the available or has the the threshold of service that's Required by the FCC and so obviously there's a difference here and again, you know maps are Maps are narratives and themselves and so again this data or this map was Formed and developed by just collecting information from the internet service providers themselves and saying you tell us you tell us what you serve One of the intricacies of this form too is that it only requires ISPs to report at the census block level meaning so if One household if they could and it's not due it's not currently But if it's they could serve a Household in a census block which can you know is a smaller unit but can be Larger and include a good amount of households if they could serve one household within that area they can report it as served So that's you know, you're getting these like larger swaths that are all this Is an issue and it you know just sort of creates an experience that's quite different from the lived reality the reported reality of a lot of People who live in these communities who are saying I don't actually have the axis I need and what's interesting to me too is that you know, they're not going and saying I don't have 25 3 They're saying I can't do homework. I can't work I can't do telehealth and there's these really sort of functional realities to what they're describing that are Captured by this threshold of 25 3 but it's often expressed and felt much differently And so I think that's also something interesting in this work of you know We we center around these metrics so that we can have a common language and a common vocabulary But there's also often very practical and very visceral ways to express what these metrics are trying to So, you know, you might also say like well, yeah, this isn't what's happening But like so what so what if the government is wrong? What why does that matter and so in one instance or one way to answer that is just generally This is considered to be has been considered to be the single source of truth for the White House and for Congress to understand the depth of the problem So as a way to understand where they need to invest more where they need to Provide more solutions or you know provide more regulations And so if the map isn't showing that there's an issue there's there's not so much of a call to action But also in 2022 the infrastructure investment infrastructure and jobs act. I think I'm That wrong the eyes are confusing made an historic investment in Broadband specifically, so this is the first time that an investment this large has been made in the US into broadband infrastructure so 65 billion dollars was Allocated within this Project to be able to invest in to broadband and 42.5 billion of that it was meant for allocation to the states and so this map Not this one in particular, but the the map that the FCC has created was what decided where or what how much money each state got so it became a very political and Logistical reality that whatever was depicted in these maps was what decided who got how much money? and How they were able to invest in there in their state and so in preparation for this Congress created the National no broadband data act and it mandated the FCC to create better maps because over the years I showed this 2011 one you can Google it there were there meant there was a lot of just known described known Unhappiness discomfort with the map in the way that it was portraying the state of broadband many leaders were saying That's just not the case Many state leaders were saying that's not the case and urging the federal government to do better So that was known so they tried to basically improve this Through the broadband data act by then making it such that ISPs had to report at the address level so no more of the census block thing You have to say this household has this service or we provide this service But the lines are still blurry between whether or not they say I could serve or I currently serve and it's also still It's still the ISP is grading their own homework. It's still them saying we provide the service Don't worry about it. And so it still does create This idea or this this narrative that you know, there's Not that much investment or that there's a there's a perhaps different level of investment needed than what some would argue is necessary And so again, you know, I just find all this work quite interesting because it really drives home for me How mapping is a narrative data is a narrative and you can often You know use it as a reference And as an objective reference to an extent but also there's always the question of who made this map Where did the data come from? How long are they collecting data for? You know whose interests are represented in this data and in this mapping and you know What is the overall sort of? Narrative that I meant to walk away with from looking at this map So another interesting aspect of this work And this is how mLab pulls into it is that because this map just doesn't seem real to a lot of people They'll often take it into their own hands in a sort of citizen science way by running speed test campaigns within the local communities to say like Okay, let's if that's the data set that we're being given let's try and make our own So let's try and get a better sense of how the internet is behaving in the areas that we're interested in and so This is a study or a report by Michigan moonshot in Michigan in these two different counties that ran crowdsource data collection campaign integrating and labs and UT and essentially on the left the Purple areas are served according to the crowdsource survey and the FCC and then on the Right the blue so you can see there's there's discrepancies in sort of the takeaways around what kind of investment is needed And how much they're essentially showing you know pretty different versions of broadband availability in these areas And so this is an interesting case to me too in terms of like You know one just you like Local versus federal government in the US and sort of like the tensions that can come out within that Within that structure of you know the federal government saying like this is reality and the and on the lower levels of government They're being a pushback and of a sort of exercise in trying to Depict what their constituents and what their communities are communicating as the reality and of course within all this mapping and data becomes this tool for people to Have this conversation So that is a those were all US centric Examples, but I think I should note that the digital divide is very much felt elsewhere but it's what's interesting to me about that is there's very different barriers of Fixing these issues in depending on What area you're looking in or what region you're looking in so In the US, you know by nature of it being originally idea the internet being originally a defense department project and the internet being Kind of you know commercialized here first. There's a lot There's a lot the barriers are different to providing access here than say somewhere in the global South or a country that got connected to the global internet at a different time and there's a lot to be asked around you know, what of the policies or what of the different social structures bend to Prevent digital access in the US versus in other areas. It's just it's my point there being it's not necessarily always An apples to apples conversation when you're thinking about digital inclusion globally, there's almost always different barriers and different systems to consider within within that conversation, so that's one practical application of Internet measurement data the next is a bit different but very much relates to this question of How is the internet used and and for what so this is these are graphics from a Project called the open observatory of network interference Mooney for short if you Google them a pizza company comes up, but it's it's not that one It's this one and they measure they really tenure project really ongoing project around Collecting evidence of internet censorship, so they will you can download an uni probe They called a probe on your phone, which is just a mobile app and run measurements to see if a website is blocked in your region and This is particularly used and useful when During elections in some countries or during protests or just Times where the government doesn't want you accessing certain information depending on the region So, you know on the on the left we have blocked domains in Iran from 2014 to 2017 So these were based off of measurements of people running that people ran from Iran seeing just what kinds of websites were blocked And they and they do this by it's that I actually Probably too much detail, but just they basically Show what it happens when you try and reach that website and then try and characterize it as a block or As not a block and that might seem simple, but it's actually actually often Sometimes unclear What is happening on the network level when you're trying to reach a website and with all of these? There's this kind of open question of like was it by accident or was it on? Intentional and that's actually kind of a difficult question to answer when you're just looking at the data When you pair it with the political context, there's there's more conclusions that you might want to go to but often The reason people are looking at a network measurement level is to provide more objective evidence of the thing happening So people are reporting it happening But if you can actually see it in the internet measurement data that gives that much more validity to the argument And so they're trying to capture these websites being blocked on a technical level such that there is that much more evidence That to hold the sensor accountable There's also, you know on the on the right side. This is an example of Measuring during a specific event. So it's kind of tiny, but they're looking at how often these Various apps Facebook messenger telegram and what's that were available during the course of the 2021 general election And I won't go too much into that but just you know, basically this is the project of Documenting how often this happens for how long and to who and where This is a project out of Georgia Tech and previously you see San Diego Called iota or internet outage detection. This is a graph of The signals from the same Uganda election in 2021 and the iota project is looking at when the internet is Shut down And so this not meaning, you know, just one website is blocked This is meaning there's an entire You can't if you log on to any website. It's not going to connect and so Again gathering evidence such that Sensors or people who are being censored have that much more evidence to say this is happening This isn't just something that we're saying this is you can see it on a network level there is evidence of this occurrence and Both of these projects really also focus on open data and so This is also where going back to the this the idea that there are a lot of people collecting this kind of data But not many publishing it by making it open the the ideas that we're able to all See where the data is coming from see how it's calculated And analyzed and aggregated and have a common shared understanding rather than just being given a number as with the FCC case about what is happening in the network and have a end-to-end Transparency in terms of how this data was produced because you can imagine it becoming quite political quite fast If a number if someone just said look the internet is being shut down I have a graph but don't ask questions about where it came from so these these projects really make an effort to report and to write and to Publish the data around how they're coming to these conclusions as a way to sort of build a common understanding and trust around the behaviors that they're seeing So I'm going to pause here for questions And before I go into my lofty conceptual ones are anything I can answer in terms of you know internet measurement or what we just went over Any terms I can clarify? Yeah, I love this question. Um, I I'm curious what others in the room would say I think I would almost never claim a data set to be better than another because I think it's just a difference in Perspective let's say so I often actually do say that the ISP or the 477 data or the FCC data is helpful because it's kind of like an expectations versus reality thing and I think there's Something to be said for having that but in terms of better I think it does Projects like M lab make an attempt to represent the user experience So I think anything that is really centering that is going to be to use the word I said, I don't like to use an improvement or you know a little bit better in perspective But this is where I also would point you that there's just there's a lot of different ideas about how to measure The very floaty ambiguous concept of speed There's a lot of different ideas about where the point a and point should point B should be And there's a lot of different Ideas about what it means to measure if the internet is working well And so I think while yes M lab would say that we try to do that better and other projects would say the same. There's definitely just an ongoing Debate that again, I find quite interesting around what is the right metric? And I think where I've landed is that there's no one right metric that it's always going to be in my opinion a sort of Consortium is the wrong word but amalgamation of These different metrics because they tell you different things they're often measuring different things And so where I've ended up as I'm saying or I'm when I'm looking at these data sets that are often Reporting to measure the same thing speed or throughput, but doing it in a really different way If they're both saying if they're saying different things that's information because it can help you understand Okay, so this measurement is measuring This part of the network and it's bad. Let's so there might be an issue there that might be the bottleneck While this one is saying the network is good So maybe the portion of the network that they measure is doing is doing well But if they're both measuring if they're both saying that the internet is struggling then there's that much more evidence that there is an issue And so chances are long-winded way to say that I I would actually say there's no like one better metric than the FCC But but the use of more data and more open data specifically I think can help get to a more realistic understanding Yeah Well, the other thing I would say too is the this project and others like it often try and pair the internet measurement data with Qualitative data just surveys and so often are trying to pair it with people that are just saying like yeah My internet sucks and like that should be enough. You know, I think there's an interesting Thing that a lot of data driven or quality quantitative assessments do where they're trying to sort of legitimize something that people are Just saying and we're just not necessarily Taking into consideration And so yeah, I think there's those are also attempts to answer the question better by just asking the people and just saying like what What would you say about your connection? Yeah, how did you manage to get the search results saying how fast is the internet to point to your website? That's a great question. I mean honestly it predates my time at measurement lab So I'm not entirely sure but I will say that the project is supported by Google And so I think they're more likely people within the organization that saw the value in being able to collect this amount of data at scale But yeah, I my kind of understanding of it is when we got in there and we stayed Yeah, yeah, good question in the Uganda The Uganda example, where did that data come from? I know less about iota I know uni is all active measurements meaning these are people running tests from their devices or their Or their phones or their computers and so they're running measurements to contribute to this data set as a way to sort of Advocate for their experience Correct. Yeah, that's not part of you. Okay. So it's people there. It's a local They're connecting to local people and yeah testing local servers. Yeah. Yeah, okay Yeah, when he does a lot of actually the last part. I'm not sure I'm not sure where the servers are located But the unit is a lot of really great work with working on the ground digital rights activists. So they'll do like sort of Preparations and trainings for when a shutdown or when a censorship event occurs to be able to collect this data So there's people on the ground saying like this is happening. Let's all collect Let's all run these tests so we can collect data about what's happening Right because the local servers aren't there anyway. Yeah Yeah Do you guys collect data on the? Like emissions associated with the exchange of data packets Emissions No, I think that would be really interesting to you and it's actually interest I spoke at the internet engineering task force recently and a lot of those Audience members were also interested in this. I think it's becoming an ongoing awareness that also everything that we're doing is you know Contributing to the environment in some way it would be interesting to test that I think one of the ideas around methodologies would just be to gain to collect metrics around how much energy consumption the servers that we Place in these data centers are collecting and are using And just getting more information around what our own contributions to that to the environment are as well because if it's going to like multiple Centers in the network like they're all Like emitting energy. Yeah, I know far less about that, but I think that would be a really interesting use of our infrastructure for sure. Yeah Relevant, thank you Because I was more thinking also now I think in this parallel of what is public and And now more private companies To develop Is there now more private companies just like styling for example with Tesla Pushing to really develop more privatized internet and more privatized access to the internet And I think then in parallel also tendencies with web three and like other protocols Where it's also more about protecting your own data and what your thoughts are and how this could be Yeah embedded into I think your research and also in how your research and the data that you Analyze really becomes manifested and actually I guess working with your clients, but working with the federal government to really Yeah strategically update the existing infrastructure because I think it's interesting how it really reflects Yeah, I guess the issues that exist in the build environment with built infrastructure, but also the actual digital infrastructure is equally fragile Yeah, so if yeah, oh, yeah, I really there's a lot in this question. That's really good I think what I would say to the first part in terms of like the privatization. I think it's actually interesting or open for It's open for question around how much one private company can actually change the internet I think something that's also that we don't time to go to in this talk But what's less understood about the internet is that there's a lot of Mutual cooperation between these companies so they might you know sort of battle it out in the in the news or whatever But they're often participating very much as co-conspirators at organizations or at Conferences and meetups like the internet engineering task force and I can and I triple E in these different consortiums meant to sort of mediate how standards and how Networks operate and so I just to say like in the in the starlink example Like they're very active at these types of conferences because they they need I think that's a really interesting thing about the internet is like it needs the rest of the internet to work Like a kid there's only it can't run the entire thing itself And so there's some level of cooperation that happens and these standards organ organizations That often just to say is also how internet measurement finds its way too is because they're paying attention to these protocols or these standards that are being Implemented and defining measurements based off of them, but in the terms of yeah, like how can these are? How can also measurement understand the sort of impact if I'm understanding your question correctly like I think one way It would just be to measure it like we can see you know more people testing from Starlink in our data And so there's a degree to which we can see like how over time, you know more tests have been running from Starlink and these examples of just Ways to sort of deduce how prevalent they're becoming or how common they're becoming but then I also think there's there's Ways in which measurement can like so for example if Starlink's wanting to change something fundamental about the internet I don't I don't know what but just something you know in particular We could measure what it is they're doing in a public way that could sort of Provide more public visibility around what it is, you know, they're doing there's a there's a way that Measurement itself is like a flashlight is often a concept that I find really compelling of like if a thing could be happening And we you know if all the differences that we just start collecting data about it It becomes that much more known And so I think that's one way I could see internet measurement being a part of Trying to understand what it is that's happening with these new companies Thank you, okay, I'm gonna go into the last little bit So the last bit of what I want to just talk through is Different questions that I it's like a wish list of questions that I wish more people asked and that I myself am very Excited and compelled by asking So in the in the different situations or the different Research cases that I laid out the practical applications of internet measurement data namely the digital divide and internet censorship There's always a sort of like trying to understand what to do to fix the problem and that makes sense and there's you know, very real Impact to asking that question But I think it's almost always going to benefit from also asking like how did how did we get here? Like what are the contexts that created the digital divide? What are the contexts that created an ecosystem in which? Censorship is prevalent like what are the different sociopolitical and geographical and Economic concerns that created a context in which the digital divide is able to Exist the way it is or you know Elections are able to be under happen under the context of censorship. There's always just a bit more to the question I think Then just you know, what is happening now and what can we do to fix it? And I think it will always benefit from this longitudinal historical framework around Again, how did we get here? How we're looking at this data as evidence of what's happening right now but what led to this data and what are the things that can't be represented in internet measurement data that Help explain Why the data is saying what it is so I think of internet measurement as you know evidence and sort of an artifact of the present time, but what are the again context and systems that led to that conclusion that's being drawn and so I've been thinking about this term of infrastructural histories or I'm loosely defining as the evolution of infrastructures told through the context of their political social and geographical Histories so placing these questions that we have about how the internet is running in this larger timeline and this longer Framework around what it is or how it is that that came to be I think we can also flip this and say the political social and geographic histories told through the evolution of their infrastructure basically How is infrastructure like the internet like public transportation, etc. An artifact of sorry a way to tell These different longer broader histories so it can be a lot to say I'm going to tell the social history of You know xyz country, but looking at their infrastructure can be a lens and an entry point into understanding it as a whole and so for me I think this is just a Wish that you know when we're talking about these really difficult Problems like you know closing the digital divide and providing access to the internet where it's not Being able to take being able to answer that question by taking a step back and thinking about how it is we got there again What are the political contexts that led to that and I have this like sneaking suspicion that often what we'll find is that The issues that we're seeing with the internet are perhaps just these Extensions of these larger political contexts of these larger political dynamics that are happening So I don't think the areas that are experiencing the digital divide I don't think that's the only inequality that they're experiencing I don't think the regions that are experiencing internet censorship. That's the only form of repression that Is happening in that region and I think if we look at the broader context There's going to be a lot more information for explaining why the technical stuff we're seeing is happening. Who are the actors that were In power, you know when the internet was introduced to the region Who's benefiting from it being there who's not maybe benefiting from freedom of expression? What are these larger? Cross currents of power that are happening around the infrastructure and how can that help us answer again? Why it is we're seeing you know graphs like this one The other thing that I really grapple with in this work is sort of just assuming that the internet is a net good I I will say this is the squishiest question that I have. I don't really have a conclusion About the answer to this question, but I think there's a way I what I want to at least call out is that there's often an assumption so often in the digital divide conversations and An internet freedom ones by nature of the work It's this kind of the way that it's talked about the way that the internet is talked about is often in this like Well, we got to get people connected. We just just like full stop. That's the goal That's the that's the thing that we want in the you know, even the speed test often points to this Idea of the internet that as long as it's fast it's working and I just I just think there's probably more to ask about that and You know in the program that I teach in at the new school and a lot of New York art circles there's a lot of people thinking about this in terms of You know, is it good that we're all on our phones all the time? Is it good that we're? Connected to all these platforms and are consistently being extracted for data to perpetuate all these different structures Like is that good people are asking these questions, but I think this is My wishes for these two conversations this internet as a net good and internet as a complicated pushy thing to be in conversation with each other more and to understand One another in relation those questions in relation to one another instead of sort of separate conversations because I will say also there's a there's a part of me to that recognizes that anyone who's Questioning whether the internet is a net good probably has pretty good access to the internet like they probably have a really you know consistent relationship to being connected and so what is that like what is That political there's a politics to that in terms of who's thinking about Being too connected versus not being connected at all and I think there's a way to have this conversation. That's like, you know I Part of my answer to this is that because some people have it everybody should have it But I think there's a more nuanced way to think about what is it that we're connecting people to what is the Reason that we want to connect people. I think you know in the digital divide conversations It's often centered as an economic good people can participate in the economy in the internet freedom Conversations often, you know framed as so they can participate in a democracy But there's also all these other sticky things that come with that and so how do we talk about that in a way? That's not Binary how and you know, it's not that the internet is totally bad and we need to get off of it because we've had too much of it Or we need to get people on it because you know, it has everything that the world can offer Like what is what is a way to have that conversation? The last thing I'll say is to is that the conversations that often happen about the internet being maybe? All too much or not something that we consider in that good They're often proposing alternate solutions But they're often very limited to the what would I would call the application layer or like things that are centered around like design and Sort of what apps we use But I think the conversation can go down into the infrastructural layer as well and people like Network operators and people like internet service providers perhaps could start considering these questions as well And they can become a more I think, you know, well-rounded consideration that we're all thinking about And this gets in again to this question of like what what is it for? What do we want to use it for? I think there's a inherent understanding or an implicit understanding that Because we have because we already have it we have to make it work better the way it is But I mean maybe that's not the case and I think that's what internet measurement makes me think of it It was like we're sort of using these metrics to perpetuate the current reality But is is that the reality of the internet that we want? And you know the squishier artist question is like I think a lot of what internet measurement can help me rap At least at least me wrap my head around is like what shape is the internet? Like what is the thing that we're building? What is the the form if it were to be the fabric tethered around our heads like what? What shape would it be what? movements would it allow? what sort of Physicalization would it become like what is a sculpture of the internet event and I should say you know a lot of the internet is Physical so I'm talking about the internet as a concept, but you know what shape do we want? The internet to be and then I think another way to ask it is what shape do we want the internet to facilitate? So if we think about the internet as an architecture as an infrastructure, you know the way that public transit Facilitates how we move around a city how you know many people in this room are more qualified to talk about this than me But how architecture? Enables you to move around a space in a specific way Like what is the way that we want the internet to allow us to move around and to enable us and to empower us I think also I think about you know I think IOC talks about this but this idea that you know we can only advocate for like pure existence Like what is it to advocate for an internet that actually provides joy and strength and solidarity and things that are not just Existing as it is and existing on the day-to-day like what is an internet that? Facilitates the interactions that we aspire to versus the ones that will just sort of live with So thank you for considering these questions. That's my email if you want to reach out, but I'm also here. Thank you Great, thanks so much. That was really amazing As always please anyone raise your hand with questions. I'll go around and distribute some mics I'm sure there's kind of a lot of questions after that if anyone has a Media opinion on if your net is good or not Thank you. That was a great talk I get curiosity on What? Conversations or musings you've heard around You know thinking about innovations in different Structures or ways of I don't want to say distributing the internet, but I guess like Maybe like providing it in a sense that's on like an infrastructural level from you know service providers or even like star you know Starlink or Other places like that that are Deeper than just the you know, then what we can affect in the market places that are provided by you know Giants on the West Coast I think my initial question to almost all of those initiatives is how will they scale and how will they? If there's if the hope is that they'll replace the current infrastructure that we have how will it do so while? it still runs like you know, so There's always these ideas that sound that work relatively well in a smaller population So like one or two or even hundreds of thousands of servers, but when you scale them up to The level or the you know breath that the internet the current internet is working I think they often that's where they run into issues So I think there's a lot of really compelling ideas, but the question of how can it scale I think is always gonna be the hard one which is not to say we should stop trying but it's you know I think that's the question that most of those initiatives initiatives eventually have to answer like how do you? Build something that's going to work at a scale as large as the one that we have now, but do you? Do you see or maybe think about how? Initiatives can scale in a different way that are like localized maybe like if you think about how you know In the US we have a central, you know rail like transit system, but if you look at other countries that have different You know operators like in Japan where they have, you know different rail companies or whatnot like is Is there Maybe this is this is too speculative of a question in a sense, but like If there are different models that like maybe you make sense for different Locals and regions and sections of the world, you know as like the politics is not equal of how You know we can use the internet in a sense. Yeah, I think that yes great question. No straightforward answer, but I think What it makes me think of is again this question of what is the internet for so if an internet is just you know to help you communicate In a way that is different than just watching that 7,000 Netflix videos like all day Maybe that's a different internet and maybe that requires different kinds of resources and maybe it's Structured differently, so I think like that gets into like maybe we can rethink the structures and the shapes and the Systems to your example like you know the structure of a transportation system would change based on How many people you're trying to transport, you know who how quickly and all these things and so I think there's an Opportunity to I think that's where it has to start is like what is it's for and then we can think about what the different shapes But yeah, I this is a Because the you know transport is very often still a government Government program and hasn't completely been privatized yet. So the internet is Mostly privatized so I'm curious about the Twitter crisis What you what you think about that right because for so many people the internet is synonymous with either Facebook or Twitter Right, so in my world my small little internet world Now everyone's disgusted. They're all leaving They leaving Twitter and you know you have to get an invitation to mast it on an invitation to blue sky And there we all are all our little liberal bubbles Talking to each other about how we hate Elon Musk, but on the other hand, you know Twitter is very Inexpensive and very powerful tool for political Organizing in all parts of the world and that are both safe and dangerous and you know all of those things So we kind of watching are we really watching Twitter crumble or actually are we watching it? Strengthen and running away from you know something that's gonna change You know into something with no federal regulation. That's even more dangerous Yeah, I mean like there's a conference to this slide where I was getting at this like What is it to question a thing that you have access to but someone else doesn't like I think that so often You know again in like New York art circles people will be like boo Twitter boo Facebook and like I you know I get it. It's doing all well and fine and yeah the rest of the world That's they're often used as tools for organizing because everything else is blocked and there there are they have enough resources to not go down during You know an over flooding of usage And so I think there's a complex conversation around like private infrastructure being used as a public space like what is what is that but then Yeah, I'm like what is the role of I remember like I've now invoked AOC twice, but When she was questioning Mark Zuckerberg In Congress she was you know saying like so you wouldn't take down hate speech or even etc And I and while I think that's an interest like a of course That's a question to ask like I don't necessarily want Mark Zuckerberg being the one to decide what is free speech and what is Not like I think there's a tricky responsibility To give to private companies in that way. These are all just stream of consciousness thoughts I think another thing that's interesting about Twitter in particular that I've meant to look more into but just like I've noticed that they're less active at these internet Consortium groups like they're not very active in the standards bodies. They're mostly depending on like AWS infrastructure as I understand it So they're they're really depending on a lot of infrastructure. That's not theirs. Whereas meta And Google have really put a lot of money in time SpaceX have yeah SpaceX have put a lot of money in time into creating Infrastructure such that they can exist more independently and more sort of on their own So I don't know how that folds into you know, whatever He's thinking of doing but I don't know in terms of the like what's gonna happen. I I I don't know and I Yeah, I think I think you know people like you are so rare in the in the tech world and what's so great about your work is that you You know all your conclusions You're making are through very very close work with the actual thing and you know like you can say Oh, you know Twitter has a different infrastructure than there's so few people that have that knowledge So have you written anything? Have you published any articles like you know, not him? Golly Is that the word academically? That's the word that might be why I can't think of the word At least people thinking about public interest technology in the tech world And they can't they don't even know the definition of the world public as it exists You know in our in our field of architecture. Yeah, yeah Yeah, I think I'm well. Thank you first of all, but then I I think one aspect of this that I'm interested in is like I Think I'm rare because this knowledge often presents itself is very intimidating and very opaque and something that is not meant to be understood and it's not meant to be studied sort of I Don't say lightly, but you know outside of the academy and so I think I am interested too in like the ways in which this knowledge can be Understood and disseminated by just paying attention Like I think that's also a compelling aspect of internet measurement and just data in general is like if you just keep looking At the thing there's often something that's revealed through that act There I wrote an essay not an academic publication but on the arena blog about measurement as a form of Tending like as a way of just continuously checking in on whatever structure it is that you care about And so I think there's something in it for me to not to resist the academic Structure, but just to also this way of just like it just being something that you're interested in caring about and that being Pedagogy in its own way or a discipline in its own way Yeah And this is a question for you. Well, this is this was an amazing talk and really relevant and and in a way also bringing an agenda of what questions what are the right questions at this point to ask about Internet and and this is a question for you and and also I think it's connected to your question Laura and and your work And it's how how do you see the the connection of? Let's say internet measurement and Scholarship now and you also work on data for for humanities and in my opinion a big part of the Capacity to produce history now in history of Infrastructures is very much as you explain related to the capacity to measure internet And I wonder what is your experience with basically the intersection of your work with a scholarship and how a scholarship can be Empowered through an understanding of how data operates collectively. I really like this question in terms of yeah Also how internet measurement can be part of knowledge production as you said So I should say like this isn't an entire burgeoning. I wouldn't maybe niche but still Active academic discipline and community. There's an internet measurement conference There's internet measurement specific tracks at the IETF and that Conference is like SIGCOM etc. SIGCOM is a computer networking Conference, but so there are people thinking about this really deeply in a rigorous research way I think what's interesting is that most of them though are asking these I don't want to call them short-sighted But very focused questions. They're very focused on a specific particular behavior of the network At a certain protocol level. That's you know, this like tiny finite Detail that is absolutely important to ask and study and think about but it's often I think that's sort of where this Small rant comes from is this idea of like but what what is the larger context of that question? And so I see a lot of internet measurement research happening And I think I'm I'm interested in how that can be expanded for knowledge production and be intertwined with other forms of knowledge like history like sociology etc. How can it be? Considered not just a technical and not just a computer science interest But something that is an artifact of these other disciplines as well I have a question kind of then you know How does it sort of relate to scholarship that I'm also kind of wondering how it relates to advocacy? You know because I'm totally super interested with the questions like we define whether internet is performing well But how fast it is right and it's like a very kind of consumer Centric way of looking at things but at the same time, you know, we are living in a world of digital inequality You know, I'm thinking of the Nobody saw the markup piece from last year called dollars to megabits. Yeah Yeah, they produce a series of maps provide a very kind of good argument of kind of digital redlining in a way that a lot of the You know areas for red lines 50 40 or whenever there are red lines have carried over and having sort of bad broadband So like within those communities of the M lab and also kind of internet measurement conferences It like is there a kind of Direct tie-in to advocacy and if there is I don't know Is there like some things in particular that people are interested in it at the same time that it feels kind of weird To say that basically like everyone should just have a better internet service provider It's like a very consumer centric way of looking at it I don't know. Yeah, I think yes So just to answer and somewhat directly like this is an example of what I would say is Is a form of advocacy also these projects as well in terms of collecting data that otherwise would go unnoticed or Unseen about events that are you know being reported, but not necessarily taken seriously So that you know, there's a level of advocacy there. I think we're it's interesting too is how it intersects with open source culture and histories and communities, so there's an Alongstanding argument that all of these systems should be again and to end open such that the data can be It can be clear how the data was produced and I think that's the advocacy level of like finding not only Or not only enabling communities to collect this data But to understand how it works and to understand and to maintain it themselves Which then easily lends itself to or like is neighbor to Efforts to have community infrastructures where people people are maintaining the their own internet You see this a lot with tribal communities In terms of and their concern tends to be like data sovereignty and being able to manage their own infrastructure And so I think the the advocacy comes in and like not only do we want to build our own things But we want to be the ones to see how they're doing. We want to be the ones to say Yes, it's yes. It's good or no. It's it's not it's not enough So there's a way in which it ties in with the open source nature by making it theoretically at least easier for communities to do that kind of data collection on their own and then also be able To do that maintenance of their own infrastructure on their own Anyone else? Okay, well, I think we covered a lot of ground here. Thanks so much. Thank you