 So my name's Michael Coran. I'm a journalist. I found a public which is a multimedia publishing platform digital publications so I think why I was interested in this topic and I think why we're all here is understanding, you know, where we're going with commenting on the web and conversation on the web in a way that maybe hasn't been lived up to all of its promise in the last few years and I think, you know, we we look at comments as like a side something that's additional or on the side but but not maybe that important but in fact, I mean when you look back at history annotation and marginalia and commenting was incredibly powerful So the Talmud is that Jewish civil law and ceremonial law is commentary Furman's last theorem was scribbled in the margin of the book When you look at a used book dealers will tell you that students look for books that have the comment and it's more valuable Galileo ancient books that are more valuable. So there's a distillation distillation of wisdom I think that that comes with yes the canonical text but also with that commentary that comes with it and so that wasn't lost on the founder of the World Wide Web Tim Bernard Lee Tim Berners Lee actually envisioned having an annotation layer across the entire web but because the technology hadn't been caught up with the reality or his desires that never actually Materialized and at the time a lot of people didn't see the value the NSF said that's that's actually that he applied for a grant the NSF They said it's not it's not worth it. There's no technical merit to it. And so here we are today and There are hundreds of millions maybe billions of comments and an annotation being made each year and Obviously, there's a lot of interest in what that means for the future of an audience that writes more than you Which they will always do so and that's a journalist. That's both exciting and But they didn't write better than that so I just wanted to do both you and Because we live in an era of transparency government surveillance surveillance I just decided to take everything on the web and make up a bio for you. So please So Dan Dan Lee Lee's from Hypothesis, which is a open source project building a conversation layer annotation layer on the web and He has a 1282 tweets he's Incredibly fortunate to have a front seat during the most extraordinary extraordinary moment in human history. He's a coder an entrepreneur he started the first on all the largest online trouble company on the web back in 95 and Eventually at 50% of the market at the time and It's a faculty at Singularity University very interested in climate change and was working on ways to sequester carbon through iron fertilization Actually, so I like to hear more about it at some point and like both people here humanity's major philosophy philosopher from the University of Illinois in Ivano Champaign and he now directs Sauce Labs and get around on the board as well as processes It's two tweets. I picked your tweets. I think they're funny So the team is on fire so much development and design were getting done incredible to behold and then Jeff Bezos retail gutted now onto news so Head of product. This is Sam Parker. He's the head of product at discus and Sorry discuss Fan of food Cycling baseball big data analytics people as those things all go together And so he started off. I think at gamers calm the one PC editor and ended up CBS See that okay, so quite a bit of technology media mixed up and of course he also comes from humanity's background history and French Berkeley which I find fascinating and So two great quotes two or tweets from you one retweeting from Bill Murray grammar the difference between knowing your shit and knowing you are shit My favorite part of winning the major sports event random acts of congratulations so many high-fives my hand hurts So thank you both for coming. What we're gonna do is have a five ten minutes for each person talk And then I want this to be a conversation appropriately, so What Sure, okay So discuss does comments, but more than comments so our goal is really community and we think of ourselves as the community of communities and From comments to community is a pretty long way All posts anything that a user can contribute across the web can be a comment and on some very real level a YouTube Video that someone posts can be a comment. There's comments on comments Tweet can be a comment all these these different elements these different containers Aren't really all that different discusses just doing Comments that are on articles usually on videos usually, but we have people who are implementing us in a wide range of ways We're really a platform and and our product makes it really easy to just drop us us right on a web page, but We've seen a range of Implementations we see on one side communities one community that a lot of people know is abc.com Fred Wilson's blog one of our early investors and it's a really really Active community where at this point there's so many other VCs that participate in there that that people go just for the comments and he's basically just creating a kind of an initial header a theme and then there's all this discussion where Someone will pile on the other people pile on and one of the things we love about that dynamic is that critical mass where? Someone is posting the original blog post But then there are insightful comments that bubble up and we spent a lot of time trying to bubble the right ones up for the initial visitor to that page and then this whole thread of discussion and And one of the challenges is is even finding that I You know before I joined discuss I think there was one moment when I when I did stumble on abc, but finding a great community is a is a pretty hard thing online So creating on ramps To that is something we've been actively working not all posts not all comments are created equal Not all communities are created equal. We really love seeing active Active discussions where people are replying to each other not just replying to the parent To the the original blogger and and that's just so we go in there And just personally we find those more interesting to read we talk to people and they find that their participation and the feedback that we give because people love to get sort of the bit of ego stroking of how many people Appreciated my comment. How many people replied to my comment. We do that in real time as much as we can It's that critical mass and even at that trending velocity of Discussions that we use to help people find really interesting things to talk about and so when we look about look at Sort of the range of things that that our comments can refer to that generally it's referring to a URL, right? But often people can actually start a discussion in say a mobile app We've seen people do it off of force fair location data Where the reference to the conversation is just a location we've seen Where people referring to particular video game character and they have a whole web page where they're all these different Discuss-powered conversations through the whole page That are about these different Video game or movie characters. So we've seen very annotation like Implementations and when we think about okay, how should we develop our software? How how should we recommend to people based on seeing a lot of people use our software how to build a community the the kinds of Communities that we zero in on For and and say we want more like that are ones that have critical critical maps where they have velocity and one of the Things that that really tends to determine that and when I go and I run my data And and the product managers go and look at it some some really key stats are you know What's the what's the ratio of? Replies to different people. What's the number of comments and sort of the the velocity the time in between comments and we like the Discussion threads that are somewhat bigger Dozens maybe low hundreds of comments once you get much beyond that it starts to feel like this big public place And everyone's shouting to get attention But if it's dozens or low hundreds it feels like there are other people are really going to see you Intuitively you know that other people are going to see you because they're reacting right next to you and so sort of creates this this natural soapbox and when you Sort of chop and chop and chop the the size of the reference down to like the event level or There's video game characters that are being annotated that I mentioned We see kind of a problem with with generating critical maths We have not yet figured out how to do both of the things we care about which is get enough critical mass and Allow people to talk about one particular part of the page and we haven't productized that Though that's certainly, you know actually pretty interesting to us So hi, my name is Dan whaley there's two of us here me and randa leads my our lead developer and We have a team of about six or seven distributed around the planet and So we'll tell you a little bit about what we're doing. So this is a proposition a proposition a Goal, maybe this is kind of how we think about what we're trying to do and So you might ask what why? Why annotation right we know why knowledge is important and the best way I've Found to kind of discuss this or to kind of illustrate this is to think about going back in time a thousand years and looking at key things like the Magna Carta or the Declaration of Independence or Documents that are important to us that form our our documented history and realizing that we really don't know too much about the What led into forming that particular document the discussions that led into it? the the back channel between co-authors the maybe the peer review of Right after the initial drafts were created or even the discussion that formed immediately after the thing became publicly available or anything in the ensuing couple hundred years and when we have points of inspection From historical documents that may be reference it, but we don't have fine grained commentary and so a question is This something that we can create today in 2013 we have more information being created per second per second than probably in all history laws journal articles news Religious texts things like Wikipedia tweets blogs But our commentary our ability to have a conversation about those things is still fairly limited. It's Balkanized it's not open and the Ways that that that common Kerry can involve oftentimes get shut down things like posterists and Places that we store our thinking our subject to the whims of the places that we store them so You know 2013 is one question. What about in ten years? What's the landscape going to look like then? And how do we think about starting to preserve what we're doing now not just even for ten years? But maybe a thousand Imagine being able to look back in a thousand years and see the conversation About all the world's knowledge with powerful search tools that would let you dive into a specific year to a specific document and see the conversation that have had evolved around Around that document at that point and it sounds kind of fantastic or maybe crazy But I actually think we actually think this is something that's within our ability to to create So hypothesis is a nonprofit It's a it's a nonprofit project to enable the community moderated annotation of the world's knowledge We're funded right now through grants from foundation Sloan shuttle worth night and Mellon Foundation And this is a pretty old idea As Michael mentioned a few of the data points our first point of reference in the digital paradigm for annotation goes back to Vannevar Bush who wrote the Famous 1945 article in the Atlantic called as we may think where he imagined this thing which has become the web and The thing which has become the web browser that he called the memex machine fast forward to 1993 this is Mark Andreessen's call for people to beta test their group annotation Plug-in for mosaic the first version of mosaic which they turned on for a little while and then turned off because They realized the scope of the service that they would have to run to operate this thing and In a blog post last year he kind of regrets Having turned this off and wonders on what the web would have been like if we would have had all of that 20 years of layer discussion as a result and another one Larry Page out of Steve Levy's book a very interesting reference I just discovered a couple weeks ago that pages original dissertation topic was on how to build a collaborative annotation layer on the web And as a result of that thinking about how you would decide which comment would get Preference and and how to sort signal from noise ultimately settled on page rank As as his thesis. So this is something that's been in people's minds for a long time There's a ton of projects roadkill some of them that Have tried to do something more like this, which I have a History that we try to keep in a Google spreadsheet of them And we have 60 projects. We've been tracking over the last 20 since mosaic that have been trying to do this. So this is a a rich landscape But what's new is that there's a group of people over the last couple years that have come together to form an interoperable standard for how to create annotations how to Share them and how systems can be interoperable with each other One system creates an annotation another can read it and this is now a w3c a working group called open annotation and there's They've made a lot of progress they have Data standards and they're settling on kind of a topology for what an annotation looks like and some thoughts about Things that an annotation can do and things that it can't So For me an annotation is kind of this very simple thing, which is like a tail Or it like an arrow rather with a payload it has a tip and a tail and a thing that it kind of carries with it So this is similar to marginalia as Michael mentioned Here's this is Sir Isaac Newton's Scribble in the margin of a treatise on optics And this has two elements of of a modern digital annotation It has the thing that it's talking about that and the very specific location within that document and then it has the payload the thing that's being said so this is what Isaac Newton actually created and What we get with open annotation with digital annotations now is the addressability of that annotation so the annotation itself is a web object that can be shared and With other people and pointed to so that it forms the continuous chain very much like a hypertext length So now on the web we have hypertext lengths that point generally to the top of things We have anchor tags, but anchor tags inside the documents are generally established by the people who create them so what's possible with annotation is The ability for people that were formerly the audience as Michael said to Now create links between very specific things and other very specific things on the web that other people Like them can discover Not just once on a page, but because we're pointing inside pages Maybe fine-grained analysis as a person points out the points of an argument to elements of the evidence or the basis for and not just in documents, but in things like media video to specific time codes or areas and Video or images and also one of the biggest use cases for annotation now that the open annotation group is focused on is data So how can people that are studying a specific gene inside of a genomic sequence for the fruit fly? see and discover Papers and threads of research that other researchers are doing because they're at the key thing Which is the real estate? It's the gene that they're interested in and they're there at the gene and from and the gene is the point of discovery for all things that relate to it So we this is a Lot of people are starting to work on this there was a conference earlier this year in San Francisco We had a hundred people from around the world that are working on projects and also implementations of annotation come together in San Francisco to Talk about how to build that shared vision There's also an interesting project called annotator that a lot of these efforts are built on top of including our own Which is starting to become The platform on which in the API layer for which a diversity of different annotation projects can be layered on top So we have our own Effort which I can demo for you So you can download that and try it now if you like at hypothesis slash alpha You can also take the code and use it run it yourself run your own annotation server and and layer and then help us build it in fact This today we were talking to a group of folks in at the University of Mary Washington And one of the folks there has just built our first wordpress plugin So that you can add hypothesis to your blog for instance on wordpress, which is great So I'll just give you take a minute and just give you a quick demo of what this would look like What this does look like so we have a this is the hypothesis alpha page We have a chrome plug-in right now. So there's There's a there's an extension rather. There's an extension. There's a bookmark lit This is also code that you can embed on the page itself if you want to layer Annotations on a page without depending on your audience bringing the extension to it Using for instance like the wordpress plug-in so if I click on the plug-in I can add it to my browser and Then I'll show you a page so we did a workshop at Berkman this last week on annotation in the law and we I invited them to kind of collaboratively annotate this Canonical version of Roe v. Wade So the hypothesis extension and pull kid is visible on the right If you open it up You can see that There's a variety of different annotations That people have made you can see that as I scroll down them that you can see the actual text That the person selected these are you know kind of nonsensical so I wouldn't read too much into them, but There if you click on one you can see the source that the person annotated What the person said and then the threaded reply structure Below that initial annotation and you know I can create a new one right here, so I can add a tag to that if I want So there's kind of a tagging capability There's a search function here. So if I want to search all of the annotations On this particular document I can search for a word like test a lot of people have said the word test including myself So I can get results that that relate to search This is a faceted search so I can add new terms to it like particular user and I get that result Every annotation is like I said a web object. So not only Is This annotation here a web object, which means I can grab a URL to it but all of the Elements in the threaded replies are each their own object so I can go to a new tab And I can see what that particular Annotation of that particular reply was Eventually we'll actually build the capability to take you back to the original source page as opposed to this interstitial page So that's kind of a quick Demo of what annotation might look like layered on top of the web as we know And this is the beginnings of what we're working on We also if you go to Get hub we have a roadmap, which we're very interested in feedback on so We have released that we're focused on right now, which includes some of the things. I'm demoing to you right now tagging and search and things like that but also importantly Features that we're focused on after this next Month or so like groups and things like that. So I'll end it right there. Yep. That's great. So I guess the first thing I wanted to ask and a bit of the elephant in the room when he comes to commenting and Annotation is how do you maintain quality? So I think when as soon as you talk about user-generated content new things of Galileo noble thoughts and free content not that many so I mean you had a lot of experience with this you have millions of comments a day And I think you probably thought about this quite a bit. What's your take on? How do you keep that maintain quality? so To general approach is one is highlight the good stuff and the other is the bad Policing the bad stuff or hiding the bad stuff So on the the bad stuff side of the first sort of Relatively solved part of it is just spam. It's just like email spam the the Problem with stuff that isn't spam, but it's bad stuff is that every different community has their own standards That's what sort of defines a community is that is the social norms and the sort of shared interests Of that and our software one of the most basic functions is is the ability for a community leader to say hey This person can't participate in this community because they're just behaving for There there's a range of things that we need to do to Make sure that someone who does not choose to identify themselves with a real name Does not does not choose to sort of you know, really put put you know Facebook identity out there like that, but that they Are hey, I'm I'm using my alias because this is a particular Maybe for example video games. I use my video game handle when I'm talking in video game communities separate from my business Communities where I use my real name so discuss needs to sort of Intermediate and say hey even though this user isn't choosing to use their real name The community has chosen to reject this person because they're about after okay we'll discuss now needs to make sure that this user can't just stop pop it and come in and Choose to identify themselves differently and behave just that that just creates a big mess for the community So that's a that's a class of things that we need to sort of provide a platform We find because at the end of the day our job the sort of the user promise that we have for community leaders bloggers new sites like CNN and the Atlantic that use us is to Take as much of that effort off of their You know really small management teams their authors who are brave enough to to venture forth and participate We want to make that as easy as possible. So the software does as much of that as possible let's go one to ten in terms of feel like you're Effectively releasing your kid or letting a community to replace itself or do you feel like you thought I would say on the On the spam side where we tend to be an eight or a nine on the identity management side, we're probably We're not in that great range and we're also not terrible in terms of the blacklist and us being able to sort of identify users across Accounts and respect the community leaders ability so there I would say What our intent is to do a lot better to know how to do that And then there's there's sort of a whole class of other things where we have We have a user reputation feature. We look at how people behave across the whole range of sites We use community signals like people who are down voting and up voting. That's sort of the most most effective ways I sort of got to the highlight the good stuff. I did that stuff our sorting mechanism. It's actually pretty effective Across most communities. There are some communities where There's just a bunch of jokers and they end up like a voting really really bad stuff but Generally speaking that that works pretty well Use a reputation works pretty well to look across the whole network said this user generally gets down voted this user Generally is behaving badly. Let's warn the community moderators and then provide some interface elements and sort of the power tools so they can more quickly Weed those users out and say, okay. I don't want them on my site We could do a lot more on the curation side to bump up the good stuff So We are not aware of a way For us not to impose some sort of Discussed way of communities to run and to have that all be algorithmic so in order for us to Believe that communities need to be different and lots of different communities exist and we don't impose content standards on those communities We really need the human signals the community leader curation a little bit of effort everyone's allowed to do to deal with the bad folks and then The community participants really to say what they appreciate Then you you're taking probably a little more open approach or like you're your wild west, right? So your whole structure is open. You really can't control in any way Or I guess in some ways, but tell us where you feel like the line between control and openness I guess I don't see control and openness as being opposite poles. I think you need Any community that doesn't moderate itself or have moderation If you like if you give people a cram and you tell them they can scribble on the web They're going to scribble so I think we need to you know communities need guidance and they alt You know our ideal is that the community moderates itself. So this is a pretty well established Line of research and practice in online systems reputation theory And I would agree that there's a basic noise floor that you need to establish through pretty simple algorithms and techniques to detect spam and maybe obvious trolls Those are sometimes hard to define but you can probably get into that territory and then we need to find ways to create headroom between the best stuff and the worst stuff and I think there's a variety of very interesting techniques stuff that we're exploring right now to use Not only the community to do that, but also to push Moderation opportunities to people that have knowledge in those particular areas. So kind of topic clustered moderation As opposed to just getting everybody to ever moderate everybody. So that's you know, obviously We're still pre-launch and we're not dealing with the reality of tons of traffic but it's something we're very focused on it's why we're still in alpha right now and it's Looking forward to Working with that and getting feedback from people Yeah, sure Fascinating really really interesting area Right now we require a pseudonym We were fans of pseudonymity and the ability for people to have identity that they can either Blend the real name to or not But also but obviously anonymity play has played a really interesting role in some very very powerful communities Wikipedia 4chan and Actually Huff Poe which for the news this week I don't know if other people have seen ever are actually going to turn their anonymous comments off for the first time that they've Since they've started so Because of you know, obviously difficulty in managing that I think our best Guestimate is that persistent identity is probably important to have a high quality community Maybe once you establish a persistent identity, then you might be able to make anonymous comments occasionally Leveraging your reputation, but because in a certain circumstance There's reasons why you'd prefer not to reveal that But stability of identity seems to be important with I don't think real names are an essential piece Do you require to just tie it to a real ID to we need an email address at the moment But we are we are looking at supporting external systems like Mozilla's persona and also You know things like a lot of stuff for people who can authenticate users and for us So one thing that I think you need to build there seem to be an interesting difference is this idea of Comments and comments and community versus annotation and annotation is can be very much of a solo activity Tell maybe talk a little bit more about what you feel like how important community is to your product and what you do and the idea of interacting on the web Versus the idea of adding one more piece of information to this Rather this topic because there is quite a bit of difference and I want to get your thoughts on I think our reason for being is is community and people commenting in a way That's public where other people can see it and other people can participate So we have a strong bias towards discussions Versus reactions We used to have a feature that was called reactions that where we were pulling in a feed of Twitter Comments against a given article a given URL and we actually turned it off It was just really really noisy It was the kind of thing that you see on Twitter when people are reacting to URLs Twitter discussions exists There are real people who talk about things about with each other But they're not tending to talk about a URL most of the stuff about a URL is just a reason So that was just one specific example where we built something we had it for a long time People thought it was kind of important because a lot of a lot of site owners like the volume They like the perception of of engagement. Oh, there's I'm a small site But but at least I've gotten a bunch of retweets of my page That's just one specific example We turned it off because it just does not meet our quality standard doesn't feel like a community doesn't feel like discuss And discusses really about discussions so We have a we have a bias towards looking for features that help the community leaders achieve their goals Which tend to be around generating engagement and feeling like there is it is a community There are participants around there their content around the the contents that they're trying to create So a lot of what we the value that we provide and how we prioritize and improvements is trying to generate income out So I think if I understood the question that there's a continuum between personal research highlighting annotation for my own purposes And then the kinds of interaction that might have with people around me Yeah, and also Showed it here, but there's a you know highlighting tool That we can that you can turn on and just make highlights on a page for yourself So this is actually interesting feedback from a journalism workshop. We did A month and a half ago where the journalists were kind of like a lot of what we need is just tools to examine long-form documents that are the basis of Research inputs that we that might form an article. We want to be able to create digital annotations Or digital highlights of those that we can store and record in our timeline And then go back to and turn into an annotation that we can then reference in an article later So I I guess I see this spectrum between personal use and social use as being something that we Hopefully can do with the same toolkit And just at different times in our workflow and also in kind of you know private modes where we're working with a team To collaboratively annotate a draft of a document before That document becomes public. So you wouldn't say community is one of your explicit goals necessarily Community ultimately is The goal of humans because we're social animals So I think it's it is the terminus that the endpoint that everything's feed into I mean there's a lot of skill sets required for this But I think there's also a need To give attention to attitudes How attitudes and change are regenerating and if you have any research or any information young younger generation that maybe as far back as that Pre-school or elementary school Thinking of the work of Seymour Pepper and how that influences a lot of the behavioral Outcomes of the use of technology My quick reaction is that whenever I meet a young person today even a very young person. I am blown away at how Totally all over technology. They are at a very very facile levels. I don't I'm personally not knowledgeable about maybe the kinds of paradigms you're talking about in terms of Enabling very very young people to do these kinds of things or how they might interact with this, but they're fast learners I don't have any particular data or learning on the Early-early age group and we we certainly see communities where There are plenty of teens and college-age participants, but I haven't really seen Specific use of discuss in pre-teens so Seems like a lot of what you guys are doing is blurring the lines about What content with annotation or comments or just content period is and I look at Wikipedia for example and that essentially is a series of comments or annotations by an audience that has created a coherent article narrative or if you want to call it and so that line between a Articulate document and then what has been commented on is completely destroyed or it's a fluid spectrum would say and so What what do you feel like the ability to do it you're enabling? In terms of comment annotation changes the nature of knowledge Does that mean that we're going to get to a point where Wikipedia at the time is really a counterintuitive that that ever be so At more accurate Do you see us going to a place where the Wikipedia is actually probably the more Dominant form of knowledge a group created knowledge through annotation or do you and because we live in the opposite world now that's the exception I would say that We're getting to the place where these kinds of tools increase the metabolism of knowledge I mean knowledge is something that is this Nebulous thing that we all kind of mutually agree on and realize as being the the event horizon of what we collectively know right and if if our ability to interact and our ability to have a conversation is Increases then it's I think the impact of that is quite profound if you take the vehicle of Scientific knowledge has generally been you know the the scientific journal article and you And you realize that there's no commentary on that that the only Commentary on the journal article has been its citation in another paper that was peer reviewed that may come out a year or a couple years of three years afterwards and But what if you were able to go to the journal article and see the layered commentary as a on top of that? You know days or weeks after that after it came out. What if you were able to see the pre publication? peer review as a component of that and and even extended forward past the publication date because somebody Read the article after was published commented on something that one of the peer reviewers said from before when it you know The publication happens so I see what's coming with some of these new tools and paradigms as being completely disruptive from in terms of the metabolism of of Knowledge and I think it's amazing the immediacy in the context of reactions With what technology today is is pretty radically different than when I was a tech writer at the start of Decade and I get emails about how terrible my reviews were for correcting me on this or that and then Had a forum that was a you know sort of just slightly Apart from the article pages to then having comments that was sort of an evolution over over five or ten years there there was something really I think energizing about seeing all of the reactions right in place and and sort of seeing Getting email notifications about that but also it was so much more likely that that someone might say something and then someone else would challenge it when it was in the context of either a forum or a comment and And even if I didn't go back and change my article it was available for everyone else to read sort of right there in the moment I think I always still though kind of And I think there's there's some part of this baked into how discuss things There's this moment in certain boron communities where really long-form Discussion that that lasts some threads last over months and years. That's something really special and how those kinds of Make sometimes forum ticks and the way people behave in form. That's pretty special and and you know, it's one of the reasons why we love to get bigger deeper meteor discussions But I can't Yes, so for example, there's one of the sites that that uses us it's very really large in was my commitment for a long time my gin they It's great to have them on discuss and they have a really interesting video game very young audience and sometimes they've there's certain Jokers griefers that will just put the funniest one-liner they can and get as many upvotes as possible Has nothing to do with the article, but it's actually a very simple way of defeating this of basic popularity Discuss common So the the specific voting algorithm that we use is it's really trying to Look at how much signal we get and so the Comments that are older if they get enough downloads they will drop. It's not just time. It's not just Kind of the pure percentage of upvotes to them So if somebody Cares enough to upvote it and enough people care enough to them vote it it tends to sort of find its natural buoyancy One of the things that we see that that can be a challenge with that kind of system is people who come back Who care enough to come back to that article? Kind of session after session over course of a day There there's some discussions that I find that really useful for people tend to switch immediately back into the newest views They can really sort of immediately scan What's new and was we think about where there are really important ways for us to Make comments feel make a comment the quality feel better for as many people as possible It's in those sorting modes and how did we do create curation and allow different kinds of users to Really have a say on showing those those are really We've seen stories break because of comments and it's The internet as we know it today. I mean there's there's awesome stuff that's happening in comments it's happening on Twitter and When it's when it's the it's big enough community and there's a big enough stage and someone has something to say and other people see it It can have a massive impact Yeah, I mean I would agree that I mean the disc you know the discussions that we have about things that are important to us are are as Important as the thing itself. So I mean if That's part of our discussions are part of real life. It's not just the internet. It's it's who we are So I would say yes You know So we see and we have some some stats that I think we published that that are When an author and we can identify the author because the site is told us has done a little bit of CMS integration When the author participates the engagement goes up we give the author a special badge that designates their their their role in the community and that tends to call call out more reactions and Just in in general the tone is set I think the more that really good article really good comments can bubble up It also sets a tone for all the comments that follow. Yeah, I mean absolutely. I think author Participation is key You know authors are you know different though some hate to engage with the audience Some don't I don't think we're going to change that I think when you can provide better tools that increase the quality of engagement that the audience can provide You know, there's interesting features that you can do with with annotation like somebody can highlight a Spelling error or something like that and if we can identify the author through integration Then you can you know the author can say, you know, I got a text message that there's a you know a spelling error And I think that's us the audience begins to provide a service To the author which can increase the engagement and they're not just an irritant anymore I Think anecdotally We've probably all seen examples where discussion threads You can get a side conversation that side conversations no longer really interesting to the general group of people that are just reading through the main conversation thread and we've definitely looked at at UI Implementations that below a certain depth of conversation. It just doesn't need to show up We haven't productized that but things just you look at enough of these and That tends to happen the the other thing I would say is that you can really profile a community based on the percentage of comments that are replies to other comments versus Comments that are just replies the main article. I mean when I when I just basically sort of all the all the communities on our network and you see all the Communities that are very just response to the original article It doesn't feel like a community. It's just a bunch of It's not out when they necessarily post but it tends to be I agree or disagree And it doesn't feel like a discussion. So I think they're there's some broad strokes examples There have been communities that have disliked that feature There are communities that also dislike the fact That people can see who upvotes. I mean, there's that's sort of the funny thing about Every decision we make and certain decisions we make are so core to how discuss works and so core to how We need to make sure that the people who go from site to site have a certain set of expectations for how they work That we don't make them optional others other decisions like whether or not someone can post as a guest So they don't even have a real email account We allow that as part of our feature set but a community can turn that on and off Community calls to turn on or off whether or not they want to pre-moderate everything But upvoting and seeing who upvotes and down voting but only allowing registered users who identify themselves to down vote Right now seems to be a good Balance so that naturally there are more upvotes So people who who feel sort of attacked by having any downloads really they're more likely to get a Positive up-vote down-vote ratio and and we don't need any I mean We don't need them to be on the same system for the algorithm to work there It's just we're sort of tuning for human psychology and for the reward mechanism and generally speaking people want To have a little bit more positivity There's a range of different approaches that different communities take Reddit hacker news and some disallow Stack overflow. I think disallows Downvotes up until you have a certain reputation threshold. So for us, it's really a question of experimentation Let's see what works. Let's get some data and then tune the system based on that Downvotes and Communities kind of Question is really interesting is when you have this system that crosses many communities and then you have individuals from one community Visiting another and then imposing their idea. So if you have someone from a Conservative website visiting a progressive website, they're going to make the way that they use that community with them In size, you know, and so I know that from my screen where we discussed I was very curious. How would something similar apply to the annotation side of things where you have a much more seeming like you know, seeming more with a PDF where, you know, you're expecting a very culture Curated discussion or something where it's like, how would something like that work? I think it'll only be necessarily cultured at the beginning when early adapters Early adopters, you know tend to you know, maybe fit that paradigm What's that? Yes, right You know The more adoption you get I think the more that you you know start to reach out to to everyday normal people I Think the very interesting thing about what you said is is the cross-pollination that you get between different communities and the potential for these systems to break down the filter bubble and to Create a layer that is shared Conversation layer that bridges Different sites and different domains and I mean that's part of the reason why we're in this project is to engage in that Inquiry But just the last question you guys are both looking out ahead and you've solved a lot of problems What do you feel like is the largest Challenge that you're facing right now and to achieve what you want to do from a commenting or annotating or community perspective that you just haven't solved and you're just not sure how Haven't solved and not sure how Well, we have a road map. We have lots of opinions. So, you know, I think what we're focused on right now is is getting launched and getting real users and and getting feedback and and Working from that and feeding that back into our plans and how we grow from here And there's lots of stuff that we don't know That we're hoping to find out Say that since we're on so many different sites and we see so many different parts of the web One of the things that that surprised me when I joined discuss two years ago coming in from the media side was just how Much the messiness of the browser technology space Influences our product space because we're we're really a third-party application across the web. That's being embedded in all these different situations and as we are in the middle of a major upgrade for our mobile web Commenting interface and comment reading interface, but we're taking a pretty radically different approach it's mobile first and and and just really trying to understand what we can do on mobile browsers to make it feel really modern and mainly really Interactive just that the fact that the mobile browser space JavaScript performs a lot less well on mobile Processors, I mean those kinds of issues. Honestly, I wish if we weren't talking about that all the time and as a constraint But the basic technology of the web and web browsers We love the fact that that we're in the web in the mess in the sort of messiness and wildness of the web It creates the opportunity to connect the dots between all sorts of different pages without being a walled garden but we also have to deal with the fact that there are all these different browsers and Trying to make sure that we're always like continuously sort of realigning to user Expectations and how user expectations about how mobile works and mobile apps work That's actually I think a really interesting challenge for us from a product development standpoint Because mobile usage we've seen massively ramping The expectations are very apt like but how can we be very apt like but but in in mobile web So that's not so much about community building except that Communities are going to form wherever people are and in wherever they're interacting with content. And so That's that's a really interesting challenge for us Great. Well, thank you to both