 Hello everyone, my name is Eric Goldman, I'm a professor of law at Santa Clara University School of Law. And today I'm here with near of Tolia who's going to talk about his journeys through Internet entrepreneurship and some of the ways in which his experiences can help us and understand and think more smartly about how we can build online communities and in particular what we can do to encourage informed productive and equitable communications online. So what I'm going to do is I'm going to run through a series of questions with your and we're going to start with the first one. Let's talk a little bit about your experience at Yahoo. Tell me what you did there. I joined Yahoo in 1996 as one of the first hundred employees and I would describe myself as a lottery ticket winner. Having joined Yahoo at that time with no qualifications really no experience certainly not as a technologist or as someone who was facile in the ways of the internet from a business standpoint, although even back then I was a very heavy user of the internet and a very, very strong believer in the community aspect of the internet but I joined the company early by virtue of knowing some of the folks who started the company because I was at Stanford and and the founders of Yahoo or of course Stanford grad students. And so I found myself there as a surfing Yahoo that was my title, and the company at that point was predominantly made up of engineers who are writing a code that operated the Yahoo set of websites and surfing Yahoo's and we were the folks who would build the Yahoo directory. Remember that Yahoo was the first directory of websites, maybe not the first but certainly the largest and most prominent directory websites because at that point the internet was just getting started the consumer internet as we think of it was just getting started and one of the biggest challenges was finding websites that one was interested in there was no Google there was no real basile search engine. And so, believe it or not, the founders of Yahoo would take submissions of websites with self submitted descriptions. In many cases they would look at those descriptions and then visit the websites themselves and then they developed an ontology a kind of taxonomy of websites a series of categories, and they would place the websites in these categories and when you visited Yahoo.com, you would find essentially a series of web pages divided into an ontological tree, and using the usability of Yahoo, you would drill down into topics of interest so for example if you were looking for a site on politics, you would go to the front page of Yahoo you could type in politics and it would try with its search engine to find the pages that matched that search query, or you could browse through different categories you could start with the front page of Yahoo and then you could drill down to politics and then of course there would be international politics, national politics, local politics, city based politics it would go all the way down, and you could keep drilling until you reached an end node of the directory and within that node, there were a series of websites and you could visit those websites and typically, there was a very high quality bar to be included in Yahoo and so you would believe as an end user that if a site were listed on Yahoo, it was a high quality site. And so our job as surfing Yahoo's was to have areas of the service areas of the taxonomy, in which we had expertise and then they would shuttle the submissions towards us and we would submit the websites into the directory, and they would be added, and that's how your website would be found, because everyone at that point was using Yahoo. So in terms of building communities or encouraging pro-social activities online, what kind of lessons did you learn from that process of taxonomizing sites into a directory? Well, I think that community online goes hand in hand with user generated content. And so community tends to be a set of people that have a shared interest. And more importantly to me, they're able to engage in conversation about that interest. So it's not just that they visit a series of websites and they have that interest in common, it's that they themselves can connect and reverse about that thing. In many cases, the submissions to Yahoo at that point in time, we're talking the mid to late 90s, were not professional websites. They were amateur websites that were created by enthusiasts in different areas of interest, different topics. And these enthusiasts would then actually almost create little online communities around that area of interest. And so while we at Yahoo may not have thought of ourselves as community builders or community cultivators, we were because we were bringing to light the websites where enthusiasts could gather around a particular topic or area of interest and create community. And so in a way, my job was to find the best websites that could create community and then make sure that the people who wanted to be part of those communities could find those websites. That was the beginning of Yahoo. Yahoo then grew to become a series of different what we called properties. There was Yahoo Sports and Yahoo Finance and Yahoo News. And these were less community websites, as much as they were destinations on Yahoo itself. But the way that Yahoo started was simply as a pointer to websites that in most cases were created by enthusiasts created by people who were creating community. Ultimately, Yahoo not only had its own destinations, it ended up building or acquiring a number of different services that I would also consider to be community centric. For example, GeoCities was a series of websites or it was really a hosting service for websites that in most cases, again, were devoted to creating online communities around areas of interest. That was a company that was acquired by Yahoo. Yahoo had one of the first instant messenger applications. Believe it or not, it was called Yahoo Pager, which, you know, is a strange word when we think of messaging services today, but messaging services are natural vehicles to create community. So whether or not Yahoo would think of itself as a community kind of website or a community service, I certainly felt like most of the things that I was exposed to in that job had elements of online community. Yahoo would think of itself as a media company, but I would say in most cases that media was user generated, and it then engendered community. Yeah, and I think about just going back even to the initial function of creating the directory. You mentioned that you had a bar for what determined site to be high quality enough to be included in the index. And of course then there were lots of people who were trying to gain that system in order to advance their agendas, whether it was spammers or just, you know, people who are looking to mess with you. Can you talk a little bit about the challenges and opportunities that were posed by establishing that standard for high quality services that would be indexed in Yahoo? It's a great question and I think it is a natural question, particularly when thinking about keeping communities high quality, but let's talk about it more generally in the case of Yahoo. What was once Yahoo's greatest strength in many ways also led to its downfall. So Yahoo's greatest strength was that human editors were looking at website submissions and then making a value judgment based on their own expertise. The employees of Yahoo had an expertise in areas and they would use their judgment to say this is a website that should be included or this is a website that should not be included. This is a website that should be highlighted or this is a website that is just kind of average so it can be included but it shouldn't be highlighted. That process was extremely effective, particularly when the web was extremely wild and woolly, which it was at the beginning. However, the popularity of the web and the explosion in the amount of content that was created very quickly created an unscalable problem. I remember being a surfing Yahoo was in many ways a dream job. However, it felt very much like being on a treadmill, because no matter how many sites you looked at and categorized and made this value decision on. You had a queue that was growing larger and larger and larger. If you did 1000 today, there were 5000 waiting for you. If you worked your way up to 5000 a day, there were 15,000 waiting for you. And if you start to think about these numbers, how much of a value judgment can you really make when you're trying to submit hundreds if not thousands of websites for submission into this directory. It's extremely difficult. So then of course Google comes along. I joined Yahoo 1996 by 1999 Google had come along and Google had invented this magical algorithm called PageRank, which looked at the reputation of websites and determine that reputation based on the number of incoming links to that website, and use that as an automatic way, an automated way to determine whether something was of high quality or low quality. And that ended up being infinitely more scalable than Yahoo, which ultimately led to Google dominating the search engine wars and Yahoo starting to lose prominence. And so in the early days, the ethos of Yahoo was very, very, very clear. Yahoo was a directory and our users trusted us to ensure that that directory was high quality. We felt a very strong responsibility. I remember extremely passionate debates about websites being submitted that taught end users how to make bombs. Should those be submitted? Was there an issue of free speech? I remember there were websites that talked about suicide and how to commit suicide. Should those be submitted? In almost all cases, I would say the folks at Yahoo were some of the most principled, high integrity, highest quality individuals that I've met in the industry, and I was extremely lucky to be there. However, it was an insurmountable challenge, a problem, a mountain of content to categorize by hand. And ultimately I feel like Google came along and Larry and Sergey, they used the nature of the web itself, that interlinked nature of the web itself to come up with a better solution. We're still wrestling with all these problems today, not only the content that might be legal but still otherwise problematic and harmful to society, and also this balance between human moderation and automated moderation. The initial Yahoo vision for this directory was in our Teamcrafted directory where each was lovingly added. All the way to, you describe the Google scenario, it's just an industrial grade machine. That's char data without a lot of human intention. And I think we're still wrestling with all of us today. It was just like a microphone of all the things that we're still struggling with. Well, I think one of the big challenges is anytime a service gets successful enough to become a place where millions, and in the case of Google now, hundreds of millions, if not billions of people will rely on that source for information. Then all of a sudden the incentives for gaming that system, the incentives for being a bad actor, those actually are dramatically high. And so it's almost in a way an arms race. If you are a director if you're an arbiter of quality if you are the quality standard, you're constantly having to reinvent how you determine that quality you're constantly having to stay ahead of the people who have so much to gain. And in many cases those people, they're not evil. They're not evil minded. They're just trying to do what's in their own best interest. They're self interested. Now there are some truly bad actors, but most of the people on the internet today. And so it brings up a very, very interesting question which is, how does a service ensure that self interest and and equitability and quality and all of those things that get added to this recipe this mix. How do those things ultimately manifest themselves for the end user, and you're right, it's the same challenge that we're wrestling with today 30 years later. The whole idea behind capitalism is that we let each person act on their own self interest and collectively, we make more efficient markets but as you point out, it becomes somewhat of an arms race where the people who are self interested can overgraze a resource in a way that actually ends up being to everyone's detriment. And so, I don't know, we're going to move to our next topic in just a moment but before I do do you have any further thoughts about how you know is there are some sense of best practice or some sort of lessons learned about how to fight these self interested over grazers. What's a natural transition to when I left yahoo in 1999, and joined a group of extremely bright, extremely ambitious co founders to start a company called opinions and just to back up for a second. When I was at yahoo yahoo was one of the most incredible places to be working in the world. I would wear my yahoo t shirt and walk down University Avenue and Palo Alto and I would get mobbed by people who would say, I love yahoo. I love using yahoo I can't believe you work at yahoo that must be so cool. But what I realized being there and being one of the first 100 employees and seeing it grow to 10,000 employees and being there when the market cap was less than a billion and seeing the public stock go to over 100 billion all these incredible things. I realized how lucky I truly was, and how in many ways, as I mentioned I was the winner of a lottery ticket. And if I were walking across the street one day, and, and didn't look up and a truck ran over me. I don't think anyone at yahoo would even notice that I was gone. And so, as I was thinking about what I was seeing. And as I was so infatuated and, and blown away by what Jerry and David the founders of yahoo had created. I really wanted to try to do it myself. I just wanted to try to create that winning lottery ticket for many other people. And I remember my going away party telling that that to Jerry Yang one of the co founders of yahoo and he sort of looked at me like I was crazy. And he said, Do you know how hard it is to start a company. And I said, Well, I mean you seem to have done a pretty good job and he said, Don't be fooled. This is one of the hardest things in the world, which of course I found out so we decided the co founders of opinions that there was this incredible opportunity to take this explosion of user generated content, and to apply it to a kind of self regulating self organizing marketplace of experts and our idea was that everybody has expertise in some area. Can we find a way to be the definitive resource or the definitive destination for that expertise and create community and utility around that expertise and to your question about what are the best practices. We thought a lot about how to architect an online system for user generated content and online community at the beginning of creating opinions, and we did think about three very important things as part of our framework. We thought about structure. We thought about incentives, and we thought about, we thought about this idea of reputation. And so this was the beginning of taking, I would say a more systematic view towards user generated content. And we looked at Amazon, who had pioneered this incredible seemed very simple at the time but this incredible utility called the customer review. And we felt like that was such an incredible revolution even though it seems so simple. Because remember back then, most people were getting information from Walt Mossberg, when they read the Wall Street Journal or from consumer reports. And there was a scale problem there to those experts couldn't possibly write about everything and those experts may even be biased. And so we felt there was an opportunity to really even the playing field. However, we wanted to do so in a way that was scalable, because we felt there were natural network effects that if we were successful would make this marketplace larger than anything in our wildest dreams. And so structure incentives and reputation were three things that we thought about very, very deeply. And as I think back about our time at opinions I think about how many things were, I think opinions with a little ahead of the curve, honestly, that it spotted some of the things that you described in this structure incentives and reputations that really were an advanced forward for the industry in some ways maybe even the industry hasn't fully caught up or it's moved on in a less healthy direction. Can you talk about what made the opinions unique about structure incentives and reputation what what were the elements that you thought would solve those problems, compared to what you were seeing at the time. Well certainly I agree that opinions was early and as many early pioneers, ultimately have arrows in their back. We to receive many arrows in our back but it was really exciting because we felt like we were, we were pioneering as pioneers we were inventing we were creating, we were not necessarily refining. We were taking concepts that were brand new and trying to put them into action so let me quickly describe structure incentives and reputation and then I'll talk about how we applied them and I think you're right. So these are things that all UGC and community website struggle with even today. The idea behind structure is that if you simply give someone an open web page, an editor that just lets them go in and write whatever they want. The quality that you end up getting is is very variable. It doesn't necessarily have to be bad, but it's very difficult for it to be consistent, because many different people have many different ideas on how they want to submit this user generated content to any marketplace. And so with structure, you can use forms, you can require certain things like a minimum word count, you can ensure that the formatting of the submissions, and the intent of those submissions ends up being high quality and uniform, at least that was the theory. And so, unlike Amazon at the time where you could write anything you wanted as a customer review on a product or service at opinions you would show up there would be a subject there would be a title there would be a description there would be the body there would be a minimum word count. We would have a spell checker built in all of these things that at the end of the day were about uniformity and quality. So when you came to opinions that uniformity would help you feel like, okay, I know what I'm going to get. And the quality of course was most important. When you came to opinions you would know that at the end of the day, the submissions are high quality. There's good spelling. They're well formatted they look the right way. We wanted to edge user generated content closer to editorial content. And obviously, you know that even today, whether it's digital or back then when it was mostly on dead trees printed media. People spent a lot of time typesetting this content so that it looks the right way. And that was one of the ideas we had. So that was structure. The second thing is incentives. We thought a lot about why do people do this. And, you know, ultimately I think this is the one that we got the most wrong. We decided to pay people for content. It was such a novel thing to do, not really novel if you consider that most of these professional publications are actually hiring writers, and the writers make a salary but for an online website like that to pay people for writing reviews, any person, you're not an employee you haven't been vetted. We would just pay you some minimum amount and some maximum amount for your review. And there were lots of interesting ways that we determine how much you would be paid. And that was ultimately our incentive system and the incentive system then had a kind of witty little approach, which is, we would pay you according to how popular your review was and we would measure that popularity by how many times it was read. In America, as you remember this led to all kinds of things like reading circles, where reviewers would read each other's reviews, there would be bots that would be created so that it would look like people reading the reviews and they really wouldn't be I mean, it ultimately was the wrong thing to do is that I think globally speaking, I would argue that most online communities that rely on a real currency, like money, as its main incentive, those create almost two capitalistic kinds of marketplaces, and I don't believe that those ultimately work. So all the people out there who try to pay people to do things online, people that aren't their employees, people that don't have, in most cases, any intention of taking that job seriously, those those systems. I don't think work very well. And then finally let's talk about reputation because remember back then, you could create an account at really any online website, and not use your real name in fact no one did. And then finally girl 13 that was the sort of archetypal username. And so we had this idea of reputation where you didn't have to use your real name and it's not just that you didn't have to use your real name. You didn't use your real name. However, you would build a reputation based on the number of people that would follow you or be your fans or read your reviews we used various different words for this. And that too ultimately led to lots of problems because we felt like creating these fan bases would be valuable, and it was, except for when people started to say if you scratch my back I'll scratch yours. And that created all kinds of strange, strange kinds of dynamics within the marketplace. I do believe however that structure incentives and reputation continue to be three of the major. Important items that you need to really think deeply about when you're creating an online community. Interestingly, there are examples of flourishing online communities that are high quality that have a very different approach. And by that I mean they don't think as much about these things I'll give you one example it comes to mind Wikipedia, what kind of structure is there on Wikipedia. It's just an open text editor. So on the structure side, they do actually adhere to that rule of uniformity that I talked about all Wikipedia pages look largely the same. However, they rely on editors to make sure that over time, those pages adhere to certain quality standards, because it's just an open text editor when you open it up. There are lots of different ways to skin the cat. And we got some of them right at opinions and, and unfortunately we probably got more of them wrong. Can we drill down a little bit more on this because I think the each of the three points you raise just raised so many additional interesting questions. I think about something like social media today, Twitter or Facebook, where really they are just as open text editor, and they're actually encouraging relatively off the cuff, and, you know, not particularly of well formatted submissions. I think a step backwards from the type of scenario you described that opinions was was was trying were pursuing where it was very tightly structured in how users navigated and it seems like social media almost has gone the opposite direction. I, I'm not sure about how we bring together quality with structure, but I don't think that I would quite agree that the services that you mentioned in the other very very dominant social media service today. I'm not, I would agree that they don't think about structure and I'll give you some very specific examples. Twitter's lasting contribution to structure was having character limit is having a character limit. The tweet was designed to be a short set of words, if not a fragment. And of course that's been waved now but you still see people do these tweet storms, and you have this idea that it's better to express a thought in as few characters as possible versus being long winded, both because it forces the creator of the content to think more deeply about what he or she is saying, but also because from a consumption standpoint. It's a lot easier to read tweets than it is to read essays. So I think Twitter actually has pioneered this very interesting structure now let's talk about Facebook. Facebook would use these leading questions. What are you doing today, what are you feeling today, tell your friends what you're up to. And these are ways of getting someone into a mindset when they post about about what kind of content you want them to create that structure. The fact that Facebook was really one of the first social networks that prized photos above text. That is structure. Now in the case of Facebook yeah you could do anything it could be really low quality. Unfortunately, frequently is, however, Facebook if we move on even from structure to reputation. Facebook was your real friends. It was real people with your real name and so they got at the quality thing a little bit differently. Let's actually move to a completely different social network that doesn't even really focus on the written word Snapchat. So how does Snapchat deal with structure. Well Snapchat is also pioneering and very very novel because you open the app, and the camera opens up. Well that's a form of structure, because Snapchat is telling you what we want you to do when you open the app is to post a picture. And so all of these things end up creating I think a kind of output, and that's really at a high level. The chief aim of structure. It's this idea that the creator of the services, I envision this type of content to come out of the service, and I'm going to nudge people in some cases it's nudging in some, in some cases actually absolutely rigid like it was on Facebook, you have no choice you have to do the structure that's laid out. But in those cases, I would say that social media has done a decent job of, of keeping the structure. I'm not sure that structure is the thing that if you only get structure right, you're bound to have quality as a result. What really makes these services so difficult to monitor to regulate to ensure that they keep quality bar high is that these three types of structure reputation incentives, they're sort of interdependent, and, and they are, they're all coalescing into some output. They're not, they're not sort of, they're not unique or independent of each other they're not mutually exclusive. They come together to create something that creates an output. For example, I would argue in the case of Twitter, which unfortunately Twitter has some of the highest quality content that you'd ever find and then it's also got some of the lowest quality content. Twitter on the reputation front, you can use any name you want. Now they have verification, you know, a little blue check mark that says that you're a verified user but for many years they didn't have that you could use any username you wanted, and ultimately, followers which I, I wonder about that word even followers was one of the ways that you would build reputation as a result your incentive was not to tweet the highest quality content but to tweet the content that you think would get you the most followers. So, these things are all interdependent. I mean I would say that Facebook has done maybe the least work on structure, but on the reputation front, Facebook is your real friends. Facebook required a real email address from a university to join initially, and that kept the quality bar extremely high. However, because you knew that these were your real friends and because you were building the friend graph yourself and because you could only be in someone's graph, if they actually had chosen to accept you because remember you also have this either political acceptance of a request or asymmetric right so followers tends to be asymmetric I can follow you without you even knowing who I am, whereas friending is a symmetric kind of reputation builder which is I ask you if I can be your friend because you have to say yes or no. And so that's where Facebook relies mostly on quality, you know, because on Twitter you can have people follow you that aren't very nice, and they can actually respond to your tweets in a way that's quite rude. And that's one of the biggest issues that we find on Twitter. And it's because of the structure reputation and incentives that all come together on that platform. And then sentence their reputation a moment but one more thing about structure I just, when I think about the opinions versus Twitter you said some of the ways that Twitter structures, the conversation with things like a maximum word count or a character count, as opposed to opinions that had a minimum word count, or Twitter, you know just having the box. And so it says, you know, there's, you know, basically, whatever you want to say, go, go for it. As opposed to opinions that is you said had much more structured prompts. It almost seems like it's solving different problem but it seems like it's lowering the barriers to people express themselves. And I wonder if that's ultimately healthy approach. Both, but I'm curious about your thoughts on how we take a step back in the kinds of reflective qualities that the opinion structure might have produced when we see some like Twitter. It's a great question I think ultimately you have two macro trends that are bigger even than the question of online community and user generated content and those two trends are. When you have mobile devices taking over the world. And on a mobile device you are much more likely to both create content that shorter and consume content that is shorter. And so Twitter is a mobile native application. And you see that from day one I mean it was modeled after SMS, which is a mobile specific technology. It was a service that was created when there were no real mobile phones that were being used. It was all desktop computers and some laptops. And so that's one big macro trend and I do believe that this idea that we are less conditioned and less able as a result to consume other pieces of content our attention spans are actually shorter. We have we have less patience in general to care about quality we want to just flick through a timeline and get to what we're looking for as fast as possible. Those are all trends. It's hard to understand whether those trends are in effect because of the social media services, or the social media services are being built in such a way to take advantage of those trends that are already there. But that's a giant trend that mobile trend that is much, much bigger even than the rise of these social services. And then the other thing is, we've, we've now created a world where everyone feels like they have a voice. Everyone. That wasn't the case when opinions started the idea of opinions this idea that there were amateurs who had expertise. That was something that people that they were a little puzzled that we believe that to be true. No one's puzzled by that today. In fact, we're at the other extreme they're people you've never heard of that are the influencers. Right. They have no credentials. They have no qualifications. They just have large fan bases. And so, I think it's an interesting question to ask which is, are the social media services the way they are because of these two macro trends, or are these two macro trends there, because the social media services have pushed us in these directions. I think probably the answer is a little bit of both, a little bit of both. But, Eric, as you and I think about our children. The reality is they are going to be interacting with user generated content and online community on phones, not on desktops or laptops, and they are folks who have shorter attention spans, and are more impatient and are used to scrolling like crazy until they get to what they're looking for. And so I think probably rather if I were creating opinions 2.0. Right, and you remember there were many opinions 2.0 kinds of things that we created over time but if I were creating the next version of these opinions, now 20 years later, I would say to myself, how can I, without ignoring the fact that we're in a mobile world where people have short attention spans, how can I still raise the quality bar. So maybe there's a different approach, maybe something goes into some kind of beta area before it gets pushed out and is read by everyone. I don't think that swimming against this tide of short form content in snippets on a mobile phone. I think it would be very difficult to go against that extremely difficult. The closest I've seen that people have done that is the emergence now with the user generated content and online community of newsletters. The most important example I can think of of longer form high quality content that still creates online community and is in most cases created by what we might think of as amateurs or everyday people versus professionals is things like sub stack, which are quite interesting to look at today. Well and sub stack and entire genre of other payment platforms like Patreon or only fans, they all have that incentive issue and so can we come back to this incentive piece or moment. The idea, if I, if I understand your perspective, you would say now, paying for content in the opinions environment was ultimately a mistake. Did I catch that properly. Yeah, I felt like it was a mistake for for a couple of reasons. I don't think generally speaking that Patreon and only fans are making mistakes. In fact, I think they're on to something but they're doing something very different than what opinions did. And these are the two very specific ways that they're different. The first is, Patreon doesn't pay its creators. The end users who consume the creators content pay Patreon, and then Patreon pays the creators the majority of what those end users have paid them. In the case of opinions, it was a very strange kind of marketplace because it's not that readers of reviews would come to opinions and then pay opinions and then opinions would go and pay the majority of that to the content creators. In a way, we had abstracted that and said, we're going to generate advertising revenue based on the content that's created and then we're going to share that advertising revenue with our creators. But it was not a direct relationship between the consumer of the content and the creator. Because only fans in Patreon, there is that transactional relationship that customer relationship between the entity that's creating the content or providing the experience, and the person who's paying for it. So that's number one. Number two, in most cases, Patreon and only fans are catering to folks who have an existing reputation and in most cases have decided to create an occupation. Maybe not a full time job, but a mostly full time job out of doing this kind of thing and patron or only fans is just another another channel for them in their full time job. If you remember some of the greatest review writers in opinions history. They were not professionals. They were not looking to quit their jobs and write professionally. Now maybe opinions made the mistake of paying them so much that they thought they could do that. But that was not their intention in joining the service. Their intention was to share their expertise, but then we messed everything up by deciding to pay people for sharing their expertise. Where in most cases, and you see this in Wikipedia and you see this all over the web today, people are perfectly happy to share their expertise because it gives them joy. That's the currency that they're receiving. That's the incentive. The incentive is that they are sharing their expertise with the world they're helping people, and then if it gets to be big enough, they find ways to monetize that. But I feel like opinions taking the central role in that monetization. I feel like that was a pretty big mistake. I mean it almost caused the company to go bankrupt and die. And I'd like the analogy or compare contrast with Wikipedia because if you would look at Wikipedia and say that can't work, that people would invest all that labor, and Wikipedia's rules don't allow them to monetize their work whatsoever. And in many cases the Wikipedians don't even develop a high profile enough reputation to monetize their reputation outside of Wikipedia. The money just isn't there for Wikipedians and he would say normally from an economic standpoint that people will prefer the sites that that's paying over the one that's not and yet you're suggesting actually it was almost a reverse. Well let's talk about Wikipedia because I think it's it's particularly important in the context of this question you've asked which is about quality. When Wikipedia started people said, my gosh, this is a joke, this will never be encyclopedia Britannica. This will never be a real encyclopedia. And today, it is the reference standard. It's more timely, it's more complete. It's more up to date. So let's run through our structure incentives and reputation and think about what Wikipedia did and I think what we'll find is sometimes thinking about those three things, but the other extreme is what yields success so structure Wikipedia has a set of rules that that you need to follow when you submit things but the text editors completely open. It doesn't rigidly force you into those rules. In fact they have editors other volunteers who will then enforce the formatting correctly. So that's where they went on structure. It's the most simple website in the world. I mean, I don't think they have specific mobile UI. They don't have a specific desktop UI. It's just a text editor that looks like a very simple page in one font with some formatting with very few pictures. I mean it's a throwback that is one of the most important services in the world today so that structure. It's the most important incentive. As you mentioned wikipedia go so far as if the people on wikipedia the wikipedia is the community that is about creating the asset, if they believe that you have an incentive to gain by adding content to wikipedia. They will either note that, or they will take your content down. So they have gone to the other extreme instead of saying, we will provide these incentives, whether they're explicit or implicit whether they're monetary or whether they're ego based. They have said, you know what, you won't actually have incentives here, your incentive is to build this thing that's larger than you. That brings me to the last which is reputation. You don't use your real name on wikipedia. You don't really get attribution on wikipedia, you have to go behind the curtain to understand who wrote these pages, and I would say less than 1% of the people who enjoy wikipedia every single day actually go behind that curtain. They don't click through they just read the content. And so wikipedia said, Oh, your reputation is not important to us. What's important is the reputation of wikipedia. If you're up for contributing and making sure that the reputation of wikipedia is here, then you can contribute. If not, we're not interested. And so maybe an interesting topic to consider is wikipedia is the model for a high quality output of online community and user generated content. What about reputation. You haven't mentioned the web of trust but could you tell us a little bit about the web of trust and how that was the structure for for opinions reputation model. The words that you choose when you're architecting one of these systems are so important. Trust is a word that we used the opinions and it's such a loaded word. I mean it's it's a little bit like the follower word right I mean follower. Before we get to trust follower is a word that, to me, harkens to. I'm an acolyte. I'm, I'm a fan. I'm someone who's following you right. And it matters to people that they have followers. So that was actually quite a genius word to use. In the case of wikipedia, we would ask the question do you trust this reviewer. What a loaded question right because ultimately, do you choose to trust someone based on the quality of their content online, which they are writing behind some pseudonym. Do you think that judgment, you know, we might have asked a simpler question such as, do you think this person writes high quality content, but it doesn't quite have the ring of do you trust this reviewer. Right. And so we would ask that question. And if people would click yes, you would create something as an end user called a web of trust, right you would have this web of people who trusted you. And ultimately, those writers who had the largest webs of trust, were the ones who ultimately received the most promotion, and promotion was simply their content went to the top of the list, and the people who didn't have a very strong web of trust in the community, their content would sink to the bottom. And the problem was, once again, it was a it was a brilliant kind of, of idea, it was a very catchy set of terms. Interestingly, we tried to patent it. Eric, you may remember, and we got some feedback that the term web of trust had been coined by one of the inventors of the internet if I remember correctly. And so we had a prior use issue with it. But ultimately, it led to, you know what what we think of as, as again, this kind of circles of trust not webs of trust where people would say hey look, I will say that I trust you if you say you trust me. And then, you know, we'll sort of do this quid pro quo kind of thing. And then all of a sudden, you know, we can't tell where the trust really is and so what would we do differently well. Maybe you would get 20 points of trust, and you could only give out 20. And after you gave out 20 after you trusted 20 people, you can't trust anyone else. And maybe you can't trust anyone until you've read we know that you've read 30 pieces of their content, or you have to write why you trust someone right but again, once you're off to the races and and I think this is an important point. The reason that it's so critical to think about these frameworks before you get started. Once the system starts. The system takes over. You're no longer in control. Imagine what it was like when we told our opinions community, who we were paying to write reviews, we're not going to pay you anymore, or we introduce something first called income share, which was, you know, obfuscated and difficult to understand and essentially just an excuse for us to pay a lot less to our reviewers than we paid them before, because we were going to go out of business. We weren't generating any revenue, and we were still paying for this content. And so, you know, you, you, you start these systems, and you don't realize at the time that if you're lucky and lightning strikes and you have some success. But then you realize that something you put into the system from the beginning is not leading to the right output. It is so difficult to change. I think one of the reasons why it's almost impossible for Facebook to copy the services I mean the next company I started next door there's now something called Facebook neighborhoods and of course Facebook is quite well known for building competing services and in most cases the competing services don't get off the ground. Because you really have to, you have to architect one of these services or systems from the ground up for it to really have the right kind of output. And when you have an existing service. It occupies a space in the consumer's mind and it's very difficult for that consumer to think of it any differently. And so as an example of Facebook says we want you to post pictures like you do on Snapchat, people will say but I'm used to status updates. Why would I go to Facebook to post pictures I'm used to status updates. Right. And so this is actually one of the challenges but in the case of opinions, using words like E royalties, which was the word that we used when we were paying people for writing reviews, expressions like Web of Trust. Those were very novel at the beginning and we love them because they were catchy and people responded to them. But when we realized they weren't quite right. It was very difficult if not impossible to put the genie back in the bottle. I want to come back to the trust piece in just a moment but the solution about switching gears on your community I think it's a pretty interesting issue. It's almost about what value proposition aggregates an audience and then if you change that basic value proposition it seems like that audience is going to say but that's not why it was here in the first place you're asking me to do something that didn't track me at first place. I think you're exactly right it's it's a question of expectation management but I think it goes deeper. And so your community that's successful feels like it's owned by the community, not by the commercial entity that started it. And so your website. You want the community members of that website to feel like they own it. That's why they care. That's why they spend all the time there. They do the things that you want them to do to create this incredible marketplace of user generated content. And so when you change something, almost as if you're God in that system. People don't like it. They weren't consulted. We went through the city opinions we created councils and we had conference calls and we had interviews and it was never enough, because ultimately, we had succeeded beyond our wildest dreams in one way which is our users felt like opinions was theirs. But then our ability to change things my gosh, it was almost impossible to change those things so at the at the low end it's expectations management. And if you're really successful. It's about ownership. It is about ownership and you are changing the core of something that someone else feels like they know. Yeah, it's a pretty powerful statement honestly this notion that to succeed, you want your community feel like they own the community. I think we have an experiment about a decade ago, I don't know if you remember this where Facebook actually tried to put its content policies up to a user vote. Do you remember when they tried to do that, and they got almost no participation from their users and so they realized there's no point in us building a system with the users aren't going to engage. And so I wonder, is that a contrary example that maybe, maybe Facebook is architect so differently that we never gonna feel like we own it and therefore, if they ask us to try and own it, it's actually fighting against the value proposition. No, I think in the case of Facebook. Something is obscure and vague as content policies is not what the users on Facebook feel like they own. On Facebook, what you feel like you own is a mouthpiece of communication to your friends and family members. That's what you feel like you own. And so as an example, let's take the competitor to next door that Facebook is is building or has built now Facebook neighborhoods. If all of a sudden you start to see in your feed posts from people you don't know, but they happen to live in your neighborhood. So you might ask yourself two very interesting questions which are completely different than anything you've experienced on Facebook or prompted by the fact that it feels like a very different experience. The first is, how does Facebook know where I live. How does Facebook know that I live in this neighborhood, and they're showing me, they're showing me content. I mean, does this mean that in some way, Facebook is is taking advantage of information it has about me. And the other thing you might ask yourself is, Why am I seeing something on Facebook from someone that isn't my friend. And so what I mean by the ownership of the experiences, it's less literal than I own this car, or I own this computer I own this book. It's about an experience that's owned. It's about Facebook is not the service itself but I own the experience of seeing things from people that I have a symmetric friend relationship with, and those people can live around the world, or they can live where I do. But where I live shouldn't be important to Facebook. And so if Facebook then changes its core value proposition. Right. It's the ownership of the experience that I feel like has been changed. It's, it's a, it's an important concept. It's not a literal one when it comes to ownership, but what it essentially means is, you've built this thing and now I have curated and created because of my own contributions. And I have this experience that I feel like I own. I have my friend graph, you don't even think of it that way because friend graph is a very Silicon Valley term, but then when all of a sudden it changes. It's very, very difficult to reconcile. It makes me think about my experience and I was a contributor at Forbes. They gave me a platform to publish on Forbes, but I was bringing a lot of the audience to the table. And then they put up an ad system that basically froze people out if they blocked ads. And my readers were just flaming me in my inbox saying, you know, you're asking me to unblock these ads. I'm not Forbes is, but it was Forbes interposition with my relationship with my readers that actually drove a wedge in my relationship with them. And that's what ultimately, I think kind of put me over the edge and said, this is not working for me. They've changed the relationship between me and my readers in a way that ultimately doesn't work to my advantage. So maybe that's a better word relationship. It's ownership is also a loaded term because it makes you think of something that is maybe, you know, about possession, but this is really more about the relationship that a platform has with its end users. And when the platform changes in substantial ways that relationship. And look, the really, I think insightful thing about Facebook is people don't trust the company based on surveys and what we read today. However, Facebook is exceptionally good at evolving its product by essentially telling its users say look, we know better than you do on this. And we're going to show you that over time and you will trust us and stay with us. And they've done it over and over and over again. Remember when the newsfeed was invented by Facebook, which is one of the greatest innovations ever in the history of social media. So many people didn't like it. And there were all kinds of revolts. And I give Mark Zuckerberg so much credit because like a true visionary like the true visionary that he is. Even though that relationship that Facebook had with its users was changing a lot, but it was changing for the better. I want to get to next door, but I did want to mop up this issue about trust because so much of the debate today about the internet is, how do we know whom to trust online. What are the indicia that someone is trustworthy. And what happens when we misdirect our trust or when we accept content that isn't trustworthy and treat as if it is. And I just remember, we're back in the opinions days that this notion about trusting another author was always uncomfortable to me, because I don't trust people categorically. I don't trust somebody has been an expert in consumer electronics, and I'm not likely to trust them as equally about vegan foods. And so this notion about trust seems so contextual in nature. And I'm wondering if you have any further thoughts about your lessons from these trying to force these trust relationships on on top of the, the interactions on opinions. And learn that about today's concerns we have about who do we trust online. How do we know we can trust them. Well, two things one is you you essentially answered the question when it comes to opinions but I would say, in a very general sense, trust is such a loaded word that means so many different things to so many different people that I would use that word at peril, really explicitly. Now you can say as the creator one of these communities, I want to build a trusted community, or I want to make sure that trust is one of the pillars of this community. But when you start to ask your end users, whether they trust something or not or in the case of the opinions whether they trust someone or not. I think that's fraught with danger. The second thing I would say is this is a perfect segue to next door, because you created next door in 2010. And so it's over 10 years old. I would say that the thing that we're probably most proud of with next door is that we feel very strongly about earning our members trust, and we have done a number of different things along the structure incentives and reputation fronts to not just make what we say, but it's something that we have done. And in most cases in the early days of the opinions, when we did things to try to build trust we never use the word trust because that was a mistake that we used at opinions, but we did try to build more trust in our system and with our members, and many of the mechanisms that we use that we felt would do that were poo pooed quite a bit. In many ways they hobbled the system and its ability to grow quickly. But we felt like those were the right investments to make and and ultimately now with 99% of American neighborhoods and one in three households and and 11 countries around the world using the service. I think it's, it's been working pretty well, but we never said, do you trust your neighbors, do you trust next door, would you trust this particular person, but we tried to do things that would increase the level of overall trust and I can give you some of the specifics. So let's drill into that and I have some other questions about next door but what you said you did some things that people advised you not to do that look like they were going to slow the growth but built the company for the longer term what are some of those things. At the very top of the list are two things, both of which were very, very antithetical to the way that services were being built back in 2010. Because all content on next door is private. So this idea that, you know, none of the conversations on next door are indexed in Google, or can be accessed by anyone outside of the network. That's very, very unique because most of these services grow, because the content can be discovered outside of the network. So we built a closed network, and we felt like we built a closed network because we said to ourselves look, we need people to feel that these conversations online are as similar as possible to the conversations that they would have offline in their neighborhoods. When one has a conversation with a neighbor. That conversation is not broadcasted to anyone who wants to read it, or listen to it, or hear about it. That conversations between neighbors. So let's create a private social network for the neighborhood and those were the words that we use. We didn't use the word trust but we said, next door is a private social network for the neighborhood. And so there was no discoverability of content you never had to be afraid. Oh something I posted on next door is being read by some random person across the web. Now who is it read by, and that was the second thing we did in the early days of next door. We felt like it was critical that once again to proxy what happens in the real world when one speaks with a neighbor. That person's one of your neighbors, because you walk down the street you know where the person lives, or you bumped into the person at the corner park, or you know the person because that person's children go to the same school that your kids go to. And so we said in the early days of next door you cannot join the system unless you verify your address, because in our world address was an objective identifier of whether or not you lived in the neighborhood. And so those two things restricting access until you prove to us that you lived in the neighborhood, and even then, ensuring that all the content that was created was kept within this closed network. Boy we had a lot of interesting board meetings where we debated those things right I mean the third thing we did was from the very beginning people said you know I don't live in that neighborhood but my parents live there or my kids go to school there I visited there and I really want to read what people are saying there, not because I'm being pernicious or because I want to do something harmful but I just want to know the best pizza place there because I like to go there on the weekends. And we said, sorry. If you live in the neighborhood, you have access, but we're not going to let people do drive buys in neighborhoods that they don't live in. And that that not only was very antithetical to what people wanted, what our end users wanted it really cut down on the network effects in the system, because there were only network effects within a neighborhood. So, if you think about it, growth is the lifeblood of any consumer internet company, particularly social media service. And from the very beginning, think about the friction we created. No discoverability on search engines are across the web. No ability to join and even see the service until you verified your address. And finally, no ability for membership to spread from one neighborhood to another, in a viral way. And so doing these things were incredibly. I use the word antithetical, I mean it was just counter to all the best practices, and we weren't doing these things to be stubborn, or because we felt like it's just better to think different, or something like that. And so doing these things very specifically because we felt like we needed to build a system that was based on trust and the way we could do that was as well as possible, understand what goes on in the real world, and then create an online environment that proxies those customs. And so that was really the beginning of next door, which was very different than anything that had been created until then. And that's the same way you frame it because in, I think, all respects next door mirrors, the physical space neighborhoods that we have, but the mirror also shows us some of the ugly sides of our neighbors. And can you tell me a little bit about how next door has been dealing with the fact that we, we see sometimes the ugly sides, because of this mirror effect. Well, probably the ugliest part of next door is something that that is not, I mean, there's sort of an obvious ugly thing and then there's a non obvious ugly thing so let's talk about both of them. The obvious ugly thing is people are passionate, and they have different points of view. And so they can get into fights, they can get into fights in the real world they can get into fights in the online world. I'm talking not about physical fights but I'm talking about fights when they have differences of opinion. In neighborhood, when your next door neighbor puts a Trump sign in his front lawn, and you don't like Trump, that might be an issue for you. And the same thing can happen online. And so it's a mirror of the things that are going on offline when people have political conversations on next door and so on a wide variety of those kinds of topics for the longest time we've said this is not content that's suitable for next door. Next door in many ways could be an unbelievable place to talk about politics, because you know where people live, and people tend to vote based on on where they live. But we have said for the longest time that national politics really has no place on next door local politics it's different story, but you do have disagreements and you've got neighbors that get into arguments and and that's the obvious challenge that we knew that we were up against from the beginning. There's a non obvious challenge, and it's because I would say it's not intentional and so next door has received many allegations of racial profiling. And I will describe what racial profiling is it's essentially when a neighbor posts on next door that they see a suspicious person walking around the neighborhood, and they assume that because of the person's appearance. The person is suspicious, and in many cases the person's appearance is based on skin color based on the way they dress based on the way they wear their hair. And in almost all cases, it's a person that quote unquote looks different than the person who's posting. And unfortunately in, in almost all cases, the person that is being posted about is from an underrepresented minority in African American Hispanic, someone who tends to not be Caucasian. So, when the first allegation occurred or when the first set of major allegations occurred, I was stunned, because I am a person of color, and I would never be part of creating something that would racially profile. But as we dug a little deeper we began to understand that the issue was one of unconscious bias. And that's where neighbors were posting not to be racist, but because they truly were trying to help their fellow neighbors, but they didn't understand that simply noting that people look suspicious, based on the way they looked versus based on their actions. That was a form of racism, whether it was an unintentional bias or intentional. Many companies would say all right, and we did this in the beginning as well. Let's do a statistical analysis to understand what percentage of content could fall into this category of unintentional bias or racial profile. And when we did that analysis not surprisingly it was less than 1% of 1% of 1% of our content, because mostly what's going on in next door is people are recommending babysitters, and they're asking about lost dogs. And they're talking about school fundraisers. But suspicious activity is something that people post about. And we felt ultimately that even one post that caused someone to feel like they had been racially profiled, even one was too much. So we rolled up our sleeves and said can we do something about this, and we created all kinds of systems and the company continues to create all kinds of systems, again largely around structure incentives and reputation that discourage people from just posting as quickly as possible, and encourage people to stop and think before they act, because in most cases, the best way to combat unconscious bias is simply to make people slow down and stop and think about what they're doing. And this again is so counter to so many of the things that are happening in the world today, and even the things that you and I have talked about. Because we're in a world that wants instant gratification that wants to post as quickly as possible that wants to remove as much friction as quickly as possible in the process. And what we are essentially saying is, before you post something in crime and safety on next door, think about what you're doing. And if you're going to say that there's a suspicious person, help us understand why that person is suspicious, based on behaviors, not simply based on appearance. So it's a, it's a kind of of two sided coin, I would say of all the things that I'm, I'm most disturbed by that occur on next door racial profiling is at the top of the list. But the other side of the coin is of all the work that we've done to build this online system. The work that we've done around trying to reduce racial profiling is some of the work that I'm proud of stuff. So let's talk about next door. To me, it's the at the vanguard of, I think the opportunities that we have online to change the dynamic of people treating each other worse, because they're online because they can hide behind anonymity or they lack the physical presence that might inhibit anti social behavior. So next door seems like it has an opportunity to actually make us be nicer to each other and I think about the kindness filter in addition to the tools you mentioned about anti racial profiling as basically ways of trying to be nicer to each other online that we might otherwise be in person. There's no one who's sitting on my shoulder saying you know Coleman, you're crossing the line and you're saying something that someone's going to find to be a product of unconscious bias or a product, you know, going to be interpreted as mean. But next door actually mediates the conversation way that that acts as that little good angel on my shoulder saying hey, you can do better. And it's really it's all based on that very very powerful word which is neighbor. What does it mean to be a neighbor. What does that really mean. And to us it's it's really sort of tied up in the golden rule it means that you treat each other the way that you treat other people the way that you want to be treated. It means that you know that your words have impact. And you know that because you will say things and then you're going to drive and walk around your neighborhood and people will see you and understand that you are behind those words, because on next door Number one, you have to use your real name. Number two, they know that you live in the neighborhood because you verified your address and number three at this point with 99% of American neighborhoods using next door chances are that your neighborhood is on next door and mostly everyone is reading what you're saying. You can think about that as the kind of angel in the devil on the shoulder telling you, you know not to do bad things and, and to be careful, you can also think of it as an opportunity to engage with people that you might not see because people's commutes are longer today. And because our neighborhoods are not nearly as active during the day because in America we have such a pro work culture. We don't really sit out on the front porch as much as they did anymore. Right. And this is one of the reasons why we don't know our neighbors. I mean, there was a very, very simple statistic from the Pew Institute that we held on to so deeply in the early days of next door and that was that 30% of Americans could not name a single neighbor by name, not even one. That isn't the way that I remember growing up. That isn't the way that I wanted my children to grow up. And today when we say on next door one third of all us households are using the service. Well look, you open up your phone. You tap on next door and you will immediately see dozens of names of your neighbors. And that's to us tangible progress, but it's all back to that idea of being a good neighbor. I don't think this is something at all that next door pioneered it's just a trend that we've tried to be part of which is, we believe the world should be a kinder place. We believe the world should be a more accepting place, and we refute the notion that the only result of online community and user generated content is the opposite of those things we refute the notion that the internet has to be a wild place where people just insult each other. We believe the technology is a force for good. And it can be. You just have to be thoughtful about the application, and you have to be committed to sticking with it. Because as we talked about the stakes are really high, and there are people out there that are acting their own self interest and instead what we need to try to communicate to them and what we need to try to build in these systems is to make it easy to act in everyone's self interest in our collective self interest. And the reason community is so important to me is because that to me is the definition of community, being part of something that's larger than ourselves. You have been extremely generous with your time. And I actually know it's late, those who watch in this video near of is nine hours ahead in Europe and so we've actually building community in Florence Italy. But, but we'll last open end the question here for you. We've talked about so many different lessons learned things that you would have done differently at each of the three stages, but, but it's any other broader take away lessons that you want to offer up now about how we can go around building online communities encouraging informed productive and equitable conversations online. Is there anything else that we haven't talked about that you want to highlight. Jeff Bezos has this very famous but, but I believe very powerful notion which is it's day one. And I believe that about the internet and I believe that about online community and user generated content, but I will say that different areas within this day one I guess we could say different times of day one have different, different areas of focus and where I believe we should focus now is we should be positive and we should try to create systems that encourage people to be good. I fear that there is too much focus on the bad actors. We're also trying to remove or eliminate or punish the bad actors that we're not spending enough time, trying to figure out ways to encourage and amplify and bring along the good ones. And the reason it's so important to do so is because you have this bimodal distribution of lots of of bad actors and good people on on the other end but in the middle is where you have the largest potential community. We want to nudge people to be good. We want to make it easy for them to be productive. We want to show them that the internet is a force for good as is technology, not just an abyss, where people can scream whatever they want, and people end up turning off. There was an interesting essay that was written. And I'm, I'm not sure if this is the exact title but it was something around the cost of silence. And it was this idea that the internet's become so difficult in terms of expressing one's opinion that it's just better to be quiet. But what is the cost of that. If I decide tomorrow that I'm not going to participate on social media, because it's just not worth it. There are too many people that are abusive. There are too many things that are unpleasant, then I'm not sharing. I'm not building community I'm not connecting. I'm not providing examples that can uplift people. I think in this new era now this part of day one. Let's focus on the positive and let's build systems that reinforce that. Thank you so much for your time. You've given us lots to think about. I'm going to stop the recording. And here we go.