 It's with tremendous pleasure that I'd like to introduce our closing keynote for for annotate 2021 Nine years ago in 2012 we wrote a blog post called soul searching and then it was one very specific observation And I quote we suspect that if an open annotation Standard for annotation is successful long-term that our front end will eventually be built into the browser where it belongs In addition to being duplicated enhanced or possibly reinvented by others leveraging a common API That sounds like success to us It means our job on the front end right now is to create a phenomenal reference Implementation for the use of interface that for instance Mozilla might built in to Firefox I'll say it again now if annotation is ever to become mainstream and fulfill Our larger dreams and ambitions that it must become native to browsers not just one browser, but all browsers as Native to browsers as the link or the tab And that doesn't diminish the role of the annotation community or Particularly us at hypothesis in my opinion Rather it complements it and extends it in important ways And so for all of us who believe this deeply we have a new champion and that champion is David Bokin David is a software developer on the web platform team at Google who works on chromium the engine under Google Chrome That is used by lots of browsers now from Microsoft Edge to Brave Opera Vivaldi and of course and of course Chrome itself Over the last several years David has helped think about and ultimately deliver a precursor to web annotation called scroll to text Which is a capability now live in Chrome that allows the browser to scroll to any arbitrary text on any web page through a simple URL enhancement And it's a feature inside of Chrome that if you go to Google search results in Google search and Click on it on a result you may see or remember seeing that when Google takes you to that page It scrolls you down to the to the part of the page that referenced Ultimately the search query that you made and that uses scroll to text And in bringing that feature at Chrome David really worked in the open Posting his code and prototype code on on github and recruiting a lot of us in the annotation community to the discussion About how the feature should work and how it should relate to the W3C web annotation standard So about when about a month ago He sent me an email and ensured a document with me that that outlined a vision for bringing open internet in Interoperable federated annotation to chromium in a much bigger way. I was pretty excited and in and in short David's without stealing his thunder David's concept is as close as I can imagine to come to checking all the boxes that I think Would need to be checked in in such an implementation So I won't get in his way any longer Let me allow him to explain this to you for himself David the floor is yours. Thanks Dan for the introduction and For inviting me here to speak. I'm excited to share Some of the explorations that we've been looking at and the vision that I think we're trying to build So I've got my email here on the screen Feel free to reach out to me with any comments or questions after this talk And yeah, if you want to I've got a couple of links in the slides So if you want to follow along I've got the link to these slides here as well But yeah, I just want to start off and build a little bit of context So Dan already mentioned some of this but about a year ago We on the web platform team in Chrome Shipped an extension to URL fragments that we call text fragments And a text fragment allows a URL to point to a specific snippet of text on a destination page So it makes links a lot more granular and precise And incidentally this is something that I think annotations also really want to do So like Dan mentioned, we got a lot of engagement from the annotation community in that work And that was kind of how I got introduced into like what annotations are and what the community is trying to do So fast forward a little bit More recently our browser team has actually been working on building on top of some of the work we did for text fragments And just building out like more useful sharing scenarios So actually just last week we launched this new feature Which is allows users to generate these kinds of links So if you go in the latest version of Chrome, you should be able to see a copy link to highlight menu item and on Android you have a similar option now in like the sharing hub And so now that we have all these sharing features built on this There's some really natural next steps that people are asking about For example, like it would be really nice if you could share a highlight with like a comment or a little bit of discussion Or maybe some reactions and like emojis and stickers are really popular in other apps What about making these highlights like persistent? So when you come back to a page and if you've highlighted things before maybe you want to see them again So when I heard a bunch of these like something clicked in my head and I was just like oh, these are annotations So when you're sharing these you can't assume that the recipient is going to be using Chrome or any specific Client, so this is something that would really have to work in like all browsers in the long term And one of the principles that we have on on the web platform team in Chrome is like neutrality So we don't want to build things that are specific to any one party So if like Chrome or Google can do something then others should also be able to have the same capabilities So this brings me to what I want to talk about today Which is how do we enable? Annotations that are built into the browser So this talk is based on an explainer that we recently published. So this is the document that Dan was mentioning You can check it out at the link on this screen And by the way, feel free to leave comments as annotations there So I've been using hypothesis and I had a good discussion with Dan about it But feel free to jump in and share your own thoughts so in this document we're trying to Clarify a vision for how annotations might work in the browser and what we would need to add to the web platform to make it possible I also discussed some of like the reasoning of why I think this is specifically a challenge for browsers Since we already have annotation tools that work quite well So I just want to clarify a couple of things before we really jump in this is all really early stages I'm quite optimistic, but I also don't want to over promise or give a false impression This kind of a change does carry quite a bit of risk So it's very possible that like we'll do a bunch of explorations try some things here and it doesn't pan out That's just the nature of the web and technology in general Doing this right does require a ton of work across many fronts So this is something that we want to socialize and get feedback early on so that's why I'm here We want to work out in the open and be very inclusive in gathering contributions Since this is a change to the web platform and it affects more than just Chrome and its users And everything I'm discussing may sound somewhat vague at this point And that's because we haven't really made any decisions yet We still have quite a bit of convincing to do even internally within the Chrome team So please keep in mind that nothing I'm presenting today is a completely fully baked proposal or something that we've committed to These are more just ideas and kind of the directions that we're looking into But the flip side of this is is that if you have opinions you can still This is the best time to have influence. I think And I'd like to focus here mostly on the platform aspects And not so much on like the UI and product decisions So I'll clarify a little bit more about what I mean here in a moment And the last thing is I'm somewhat new to annotation Certainly within this crowd I think and so my thinking isn't going to be super sophisticated here Part of my goal is to see the discussion here and get more awareness for this work But really I'd also like to hear and learn more from you all about some of these issues And how we might solve them and how best to think about them So I'm hoping that we'll have some time at the end and have the discussion But if we run out of time or if we don't get to your question or or just if you need some more time to digest this Feel free to shoot me an email or a tweet anytime or or check out our github repo and then leave comments there So I just want to start by explaining what we're not trying to do When people hear the words browser and annotation Often immediately their mind just jumps to like here's a commenting section for the web And we really don't want to have this one big commenting section There's a lot of examples. I think of like large platforms with commenting And in the best cases it requires just an immense amount of work to even make it usable Personally in my own opinion, I think like when a community is overly broad The commentary becomes less useful and less thoughtful So this is a case. I think we're having some decentralization would be very helpful And it lets communities and groups self organize and self moderate at an appropriate scale for them And I think this is really helpful to mitigate some of the biggest challenges and criticisms that we might face From bringing annotations to the web So I've mentioned that we want to focus on the platform aspects. So I just want to clarify what I meant by that Very coarsely The platform is the bits that we would actually want to standardize somewhere in a specification So that clients and other players in an ecosystem can interoperate So I'm a biased browser engineer. So when I say web platform, I'm thinking about browsers and web pages But the web is a lot bigger than just browsers There already exists the web annotation spec, which is already standardized like what an annotation on the web looks like But it doesn't mention web browsers at all I certainly missed it And as somebody who's like spent the better part of the last decade working on on the web in a browser I actually only just learned about web annotation in the last year or two So this is a gap that we would actually need to bridge So the platform is things like formats and protocols for annotation data Which web annotation already provides But we also need to think about Things like opt-in signals for pages and ways for pages to announce That an annotation source exists For example, like how does it how does style highlights? The reason that I want to focus on this Is that I think in order for the ecosystem to flourish More or less everybody kind of needs to agree about these issues I don't want to talk too much about like the individual clients and how how they're implemented and how they present annotations Because I think that those are things that can easily change And those are the things that would likely be iterated on based on feedback and experience And they're also the things that chrome or another browser or another agent They can disagree on that and make different choices depending on their own product goals and user populations And I think that this disagreement is good. It allows differentiation and innovation So for example, chrome serves like billions of users And so it's limited in how complicated we can make our ui Because it has to be usable by basically everybody On the other hand, if you're an academic who spends every day annotating you might require something that's more purpose built So I think it's healthy to let clients be opinionated as long as we make sure that they're all speaking the same language So I want to talk a little bit About the players in this ecosystem and how we've been thinking about them The w3c has this really great priority of constituencies Which puts basically the needs of users ahead of the needs of page authors ahead of the needs of implementers or browsers However, now we also have annotation services Or annotators just to be clear. When I say annotator I'm talking about like an annotation service Which is not to be confused because you can also think of a user making annotations as an annotator So yeah But if you look at this picture, we already have some questions So annotators seem to be somewhere between like a user and a browser But are they more or less important than page authors? And even within users, there's multiple class of users We have just everyday users browsing the web, but somebody who creates an annotation on a web page is also technically a user In a way, there's some sort of similar to a page author in this case, but not really So clearly we've got a little bit of ambiguity But I also don't want to rush to be explicit about creating and ordering here Because I think in a healthy ecosystem, the interests of all these parties are not going to be zero sum And so we should look to find a healthy balance of interests rather than trying to find like a stack ranking and elevating one over the other So annotations aren't new like hypothesis has existed for a long time as I have other tools So I think it's worth discussing why I think building this into a browser introduces some unique challenges The main one is that we have a lot of users like a lot a lot And they come with a wide range of like tech literacy and diversity and everything So really users need to be able to intuitively understand All of the different features in the browser And so this makes it quite challenging The browser is already super complex and has like millions of features and many of which you probably don't know about So even making small ui changes is really difficult and annotation. I think is very ui heavy Unlike when a user seeks out and installs an annotation client Our users might not actually know about annotations or want them at all So this is a very big difference like if you go and if you install hypothesis the extension You're signaling that you actually want to see annotations. This is not going to be true for for most browser users, I think And since that they're not intentional users We have to make sure that they can understand what's going on So it could be very bad for like security or privacy if if users get confused, right? So if they're seeing annotations If they might confuse them for as coming from the page or coming from the browser So we really don't want this to become like a new vector for phishing or security issues And this does affect authors, I think at a different scale So even with existing annotation tools where the user set was fairly limited Some page authors were already fairly upset and they don't want their content annotated So there are legitimate reasons here either for like branding reasons or just to avoid abuse This will definitely become a bigger issues if annotations are now visible to all the users And I think this really boils down to the browser users and pages Are not primarily annotation focused It's a challenge for us to figure out exactly how to mediate the different needs In the absence of having a good interest signal That said, I think this is a really worthy goal to pursue There's a lot of really exciting potential for like new ideas here I have a couple that came to mind. So I've listed them here You know, there's shared commentary on the web, which I think is the most obvious and biggest use case and everybody here is probably familiar with it But there's also like this notion of like a more collaborative browsing So I would like to see the web become less lonely Being able to share commentary and reaction with friends Maybe you can even publish a feed of annotations that your friends can follow And be able to tag friends on a page. This would be really nice You know, perhaps you can have like wikipedia, for example providing contextual annotations as you browse a web page Or some kind of fact checking services Another use case that has been pretty common In the past is like allowing users to suggest corrections and provide a bit of a feedback loop with authors Which would be helpful in cases where pages are abandoned We also have like aggregator type sites. So like if you think of like reddit or hacker news It's kind of a regular trope that people don't really read the article and they jump straight to the comments What if we could move the commentary to happen in line with the article? Would that make the discussion better? And I think providing a layer on top of the content can also enable some more automated use cases So we can help users be more productive and navigate better So for example, like at work it might be helpful For your email client or your document client to Provide links to like relevant emails and documents depending on the page that you're looking at Or having an annotation show you how long a page usually takes to read or a tag cloud of the content that it has What i'm really excited about here is that this really does open up something of a new dimension And by building it out as a platform I can see it being used like very creatively So that's the good, but like what makes this really hard If you read our explainer, there's a long list of challenges But i'm going to focus here on what I see as the the biggest ones Some of them are more technical in nature I'll discuss those a little bit later And i'm sure that there'll be a robust debate when the time comes to actually implement those but I think at least those are somewhat more quantifiable. So i'm less worried about the technical aspects It's the more subjective issues That I think are the most challenging and the ones that I want to highlight here So they have some smaller sub issues, but I think they can be broadly bucketed into two themes The first is moderation So how do we deal with content that's like abusive or dangerous? What about like security related challenges or spam? No one's going to use annotations if 90 of what they see are ads for pills And if there's one thing I've learned about the web is that if they can somebody always wants to sell me pills So one option is that we can pass this challenge down to annotation services So if users can choose the services that they subscribe to Those that do a good job of moderation will be preferred and so those like the incentive structure here would be good This seems like a really attractive option, but I don't think it can be the end of the story So users aren't always very intentional about what they install So I'm sure that this image is maybe not totally in It's a little bit of an exaggeration, but I'm sure it brings back some memories for some people Um So I mean if we do have this concept of a subscription We would need to consider how we can make it clear and safe And basically provide enough friction that it's it's difficult to do unintentionally and and easy to undo Which really brings up the next question is like can we Make sure that annotations are always clearly attributable If users can know where an annotation is coming from It would become a lot easier for them to either block or remove bad actors But this does rely on users having a good mental model of what's happening. And so I don't think it's it's that simple At the very least this will need quite a bit of thought There's also the question of like how much can the browser do to detect spam or like malnotes If if there are things that we can do There might be room for like the user agent to surface that to a user somehow So for example, if we can figure out that some text is is spam Maybe those annotations get hidden or grayed out or at least like Shifted to the bottom It does also seem like it would be really useful to give services Some signals and tools so that they can help do this job better So for example, a lot of existing tools Have a method for users to be able to flag specific annotations Should we standardize this kind of feedback? Maybe there's also other data that a browser could provide To a service that might be useful for them to do moderation I don't necessarily know what all of this is yet But depending on how the ui looks like the browser might be able to tell how useful an annotation is to a user Surfacing that to the service might be something that would be useful So finally, I think we can structure the incentives here in a way that might discourage misuse It wouldn't completely eliminate it But like for example, if annotations are hidden behind a click They become much less as a surface much less attractive as a surface for like spam So the second major challenge is that at least that I see is how much control do we give page authors? This is a complex and contentious issue And I think there's good arguments on both sides On the one hand, like the universality of annotations is kind of the point If annotations work only on some pages, it'll be really confusing to users and they'll be much less useful And we also don't want like unscrupulous actors to be able to shut down legitimate debate and discussion But on the other hand, some authors don't want to deal with abuse And particularly those from vulnerable communities People also have built businesses on the web and we don't want to enable mobs to go Brigading across their virtual storefronts and destroying their livelihoods Separate from that in some cases it may actually be better for the user experience For annotations not to be shown at least by default So for example, there are some very dynamic pages like basically video games for example, probably wouldn't Wouldn't benefit from this So I know this has already been a challenge for existing services And this is something that they've dealt with in the past and I'm sure are still dealing with But bringing this to a broader audience is definitely going to amplify the stakes So personally, I think that giving authors at least some way to signal their preference would be a good step At the same time, it seems like users should be able to override the page preference After all the browser is the user's agent But something else to consider is are there any signals that The user agent can actually gather an aggregate here to help make better decisions So for example, like if many users are overriding a specific page's preference Should the user agent start doing that for all users? I'm not sure yet. This isn't the question that's like necessarily answered But it does seem like that's something that different user agents could at least take different positions on And personally, I think that a lot of this will come down to what the default presentation looks like So there's a trade-off here to make on how how visible annotations are by default We don't want them to be annoying But we do want them to be discoverable So On one hand, like the more visible they are the more objections will get from authors Maybe on pages That wants to like opt out Annotations and their highlights Wouldn't initially be visible, but maybe an icon in the ui somewhere might light up Which would then show the annotations So this is just to give you an idea But I did a quick little mock up here As a disclaimer, this is just a mock It's not indicative of anything that we're doing yet But this does seem to me like it would strike a fairly good view Of the balance, but I am curious to hear what other people think about this So getting a little bit more technical here Everything I'm about to discuss is just ideas. We haven't put down any code or even proposal text yet It's likely to change due to feedback or just meeting reality But if anybody does have strong opinions feel free to leave suggestions or criticisms on our github repo So this diagram is kind of like the main one that I used to help explain how this might work in a browser The green notched boxes at the top represent what I call an annotation source A source is basically just an endpoint from which the browser can go and fetch annotation data So these would typically be an annotation service But for example, the browser itself may actually be a store of annotations as well So you can see here. There's a user box. These would be like private annotations that the user doesn't necessarily want to share So maybe annotations could also be embedded directly in a url I'll discuss a little bit of these A couple of options here in a future slide The browser would fetch from all of the sources that it knows about For any given page and then it would aggregate them and then display them in an annotation surface So in my view a surface would either be an installed extension or the browser's native ui If an extension doesn't consume a given annotation, then it can be passed over to browser ui for display So I can think of a few ways Annotation sources might be added to a page and maybe there's others, but this is basically How I've been thinking about things You could configure it in a browser or an extension settings page So this is basically the model that we have today with existing tools And each page load would go and query the source for any annotations it has This of course requires that users really trust the source Since naively implemented like their browsing history would basically be leaked to that source It'd be nice if we could make this more privacy preserving For example, maybe you can use a trusted proxy to fetch the annotations Or you would fetch all of the annotations or large buckets so that the source can't find out what you're browsing Another option is that URLs themselves could specify a source So this wouldn't require users to subscribe to To an annotator or do any kind of configuration So this would be really useful for like aggregating websites like reddit So this this could be a way for example, I mentioned earlier of like bringing commentary to the destination page You could imagine like linking to a news story and then saying like Go go to the news site and then also fetch annotations from this other separate source I think separate to that this would also be really useful for discovery Since users probably aren't going to go out of their way to go and install an annotation service But this is a way that they would Kind of a stepping stone into getting into annotations Related to this a url itself could be a source So we could actually embed the data into the url this this kind of mirrors the text fragment approach we had This would be really nice for like very ephemeral type annotations So say that you just want to share a quick comment with a friend You wouldn't even need a server in this case. You would just encode all of the data that you need into the url And finally pages themselves might also want to be able to specify annotations so for example, you could imagine that Today pages already host like some commentary sections But it would be really nice if they can just dump a link rel element on their page and say like annotations come from for example hypothesis And then all of the commentary on the page would actually appear as annotations So speaking about ui I think the browser ui very likely would just provide like a lowest common denominator. I think You can think of this as like basic text with some limited formatting There are questions of like how far should we go over like how How much pages or the annotations can offer like styling? Maybe some pages can provide some hints as to how annotations should render Just so that they fit well with the page and its theme As a very basic example Text fragments today already allow you to style the highlight using some css We'd probably want to allow that for annotations, but Yeah, there's a lot more freedom. I think in annotations I don't think that we want to allow just like arbitrary rendering certainly not like javascript But there's a lot of questions over like exactly how much control do we give over presentation? I do think there's still room here for extensions though So if the browser is like the lowest common denominator Extensions would be able to supplement this and provide extra features that aren't available in browsers So this would be really useful for just experimentation and trying out new things But also for building more specialist tools So ideally specialized features would provide some kind of like a basic fallback That would at least be readable by users. You don't have the extension So to give you an example, like maybe the browser can't display images Uh, maybe initially Uh, as a fallback, we might just show like a link so that a user at least a user who doesn't have like an extension that does allow images would then be able to go and see that So to integrate well with the browser. I think extensions would need some new apis Um, basically so that they could receive the annotation data from the browser It would also be helpful. I think to expose some common annotation functionality So for example like text matching apis So that they're consistent with the browser And we definitely wouldn't want to duplicate annotations both in the browser UI and extension UI So I have a basically an idea of how that might look Again, this is all very speculative and not necessarily fully baked but Here you can see kind of an event based model Where for each annotation, you can call prevent default on it like you would in many other cases In the web platform And that would let an extension mark the annotation as being handled. So it doesn't end up in the browser UI in that case So finally for consistency, I think it would be great if we could allow extensions to directly render into the browser's UI This would let all of like the UI and and controls be consistent But let extensions add new functionality This I think would be best case scenario something like very very long term But if if this does turn out to be like a wildly successful idea, I think this would be a great direction to go in Rather than having an extension build like its own side panel that looks very different Let them kind of customize the inside of the annotation that the browser provides So as we talked about Author controls might be necessary I think one important criteria here is that some authors Don't necessarily control their server or domain And so we need a way for this to be applied By specific page content So hosted blogs here are a common example For these kinds of authors They might not be able to set like HTTP headers Or per domain mechanism won't necessarily work for them I'm not yet sure how complex or granular this needs to be It's possible authors might want to be able to block By a URL of the annotation service This would be consistent with some other like security or Like content security policy and other features on the web Let you block by origin On the other hand, maybe this is a little bit overkill And it would be simpler just to have like an allow block for all annotations Because again, at least in my in my mind, I don't think that we would want to block annotations But more just like control How visible they are and how much action a user needs to take to see them So yeah, that's pretty much all I've got. I've got some parting thoughts I think that this would be a pretty major change in how users interact with the web from today It's really exciting, but it also comes with quite a bit of risk And so I think we want to be really cautious and thoughtful about how we approach it We want to do this in a way that includes the whole web ecosystem So we're trying to work in a way that's transparent and includes anybody who'd like to participate And we can't do it alone For me, success means that this is available in all browsers, not just Chrome And we have a healthy ecosystem of clients and services So I'll just reiterate again This is all very early stages and it's premature to say that like Chrome is committed to anything There is a chance this could all fizzle out But that said, I think so far so good We've had some very supportive comments both internally and externally And so I'm looking to Kind of broaden the the community that's that's involved here And get more feedback. I think is is the most important thing Frankly, I think like this community of Annotators and people interested in annotations. I think I kind of know I think what the opinion will be just because like people are excited here to see annotations become more mainstream That's not going to be the case for all communities So I am trying to broaden those circles But to maximize the chance of success here I'd like to work in like small incremental Stages rather than trying to like put together one big vision and then deliver it all in like a big bang So the main benefits here are that like if we can This would let us deliver some value much sooner And if the priorities do eventually change like that value would still remain I also think it's easier for us to get feedback in smaller chunks and we can learn from our mistakes and iterate on those And this is maybe a bit of a guess But I also expect that it'll be easier to digest for the entire ecosystem if we take like small steps in incrementally So just one thing I would note is like once we do eventually Publish some kind of proposals or or build features. Don't be surprised if they don't Completely like satisfy this vision that I've outlined We are going to try to do this in like small pieces So yeah, that's basically it. I would be happy to answer any questions or just like hear comments and see some discussion Awesome. David. Thank you. That was awesome. Really terrific Um, you uh, you definitely I think get a lot of questions and so forth that we're in in your paper um, I will note by the way that the url that you Pud pushed published in there, which was I think an alias for the github page Isn't working for us. So we we did publish the url to the to that github page in the chat here and So that they can see what you might not have So I'll just I'll start with with one or two and and let people post more in the q&a and Also, at some point Nate can come on and maybe pull some of these q&a questions into the Into the window here but No, right here at the end you talked and I know you've kind of put together a proposal Which is at your page and you know really enjoyed annotating That with you But is there another More large larger or more formal proposal that might come from google You know or or kind of white paper or descriptive object Before you got you know kind of involved to work Or got down to work Sorry, can you clear is there a is there a more formal proposal that might come from google about this work More formal than for instance your You know speculative paper Yes, so definitely like once we actually Have something more concrete We would be putting forward like we would have a spec for this or like a A draft spec So the way we worked on this in like text fragments is we put out an explainer We had some ideas and then we In chrome the way we do this is we will build the feature like behind a flag And we won't ship it until like we have Approval from like our api owners and the entire kind of like chrome shipping process But we would have like more formal certainly like a Like I mentioned like a draft spec which would actually specify exactly what you need to do in other user agents to Have an interoperable implementation I would I think like anything that we do I would also put out Another explainer I guess with more specifics Just because like specs are difficult to read even for the experts And so having something that's like human readable about like here's what we are proposing And then have some debate and iteration on that great Um Let's see. Let me just start going here to the community Not in any particular order, but just because i'm looking at it a mark ram from the internet archive director the wayback machine says a couple things but Can you speak to annotation interoperability? and and maybe What that means fully expressed in the kind of implementation you're talking about Yeah, so I think the the data and protocols Are something that we would want to be interoperable And luckily we already have the the web annotation standard So I think that is something that we would want to leverage And and basically build on that So in a world like that An annotation created by chrome would then be viewable in like hypothesis It would be viewable in safari or whoever else decides to implement that Of course web annotation is an incredibly detailed and large spec. Um, so for example, chrome might not interpret every possible Property that you can set on an annotation, but at the very least they would understand what those look like and Yeah, basically let you interoperate Um Brad, uh, brad meyers from Carnegie Mellon asks Um Could annotations also apply to web apps like gmail or google docs? Or maps, which is an interesting one Um, so we could uh, so that the interoperable nature Of how we imagine annotation might apply to some of the things which implement Versions of that I guess Yeah, so the the distinction between like a web app and a document is kind of blurry. Um, so naively. Yes Unfortunately like dynamically generated content is something that's a challenge I think for like existing annotations already. Um, and it's something that we would want to solve In the case of like maps, for example I think that's one that would be like naively very challenging Because you basically have like some kind of a canvas rendering and it's it's not really clear Like what you can attach an annotation to Um, but we could certainly look at like providing uh, api so that uh, like to take maps as an example Um, maybe they want to actually be able to specify things in their canvas that like this is an annotate Like an anchor for an annotation and you can add things to it I think this would be a very like long-term Play because generally I think like until we have a lot of users and this is something people are asking for It's going to be a tough sell for for teams that already have like limited resources to go and add this But yeah, it's possible. It's just like this wouldn't be our first priority Um, great. Um, matias, uh From uh, norton asks, uh, how might we bridge Annotation features already built within a web app Like for instance, uh, the annotated features in norton's reader Um with a broader platform So that annotations in an ebook can cross the boundary into the browser's annotation stream Yeah, so this is interesting. Um, if if I understand correctly, um, maybe to put it in an example I'm more familiar with um, is like in google docs. For example, you can uh, go and make comments on a document And so that's like a form of annotation And so how do we uh, make those more consistent? I guess Um, that is something where I think like long term, um, like docs shouldn't have to build that kind of feature themselves Or at the very least should Should be able to go and like augment the browser's ui with uh with like apis Um, but again, this is I think relates to the last answer where it's like this is like a long term thing And if it's very popular to some that's like a direction that we would want to go in um, but it is uh It's it's possible, but uh, yeah requires lots of thought challenging indeed Uh, chris chris aldrich, uh asks how can we better annotate embedded images? Audio video it would be nice to annotate an audio file directly at its file page rather than pages in which it's embedded That's an interesting one. Yeah, to be honest. I haven't really thought about this very much. Um Yeah embedding like rich media in particular things like videos. Um, sorry annotating things like videos Seems a little bit more difficult um Yeah, this is this would be something I think we would consider later on um Certainly the other direction where I think uh, you might want to add things like videos and audio as an annotation Um seems like an easier problem to solve I think Um, but yeah in in terms of like the generality of annotations It would be nice to be able to annotate videos. Um, but it would be a bit of a challenge um Let's see, uh Mark Graham asks, um How might you work? with mozawa other other browsers Who are not even built on chromium for instance? um And he also asked how might others collaborate with you if I may be two two halves of the same coin Yeah, so um, I mean we have standard processes for how browsers work with each other. Um It's a bit of a challenge because different browsers have different priorities at different times And so getting all of the relevant people in the same room can sometimes be challenging Um, certainly once I think we have something a little bit more concrete I'll definitely reach out to like engineers from uh, like webkit and gecko And just make sure that at the very least they're aware of what we're doing and I mean provide any kind of feedback If they want to Um, if they want to be more involved and like kind of help push this along I would be all too happy to have this um Sorry, can you just repeat the second part of the question? Well, how might others collaborate with you like, uh, if You know somebody at the internet archive Had had ideas or capability You know notions in mind Yeah, so the the github repo which, uh, I think you mentioned you dropped the link here somewhere That would be a great place to just go and file issues If and when this does become a little bit more formal, uh, we would want to move that github repo into the web incubating community group web platform incubation community group the ycg So that's kind of like the the area where we go and incubate new projects on the web platform to let others chime in and and see progress and give feedback And help out if they want to Um Talk a little bit about timing. Um big big thing. Um, you know, what kind you know, how does this evolve Given your experience in launching, you know, kind of other big features within crone What what might people expect to see? In What kind of time frame? Yeah, so this this is probably the toughest question. Um, so just to give a little context like text fragments Um From when I started working on it. Um, it kind of predates me by a little bit But I think from from that until when we actually shipped was probably about two years And text fragments in comparison seems like an easy problem. Um, so, uh Yeah, if if this does turn out to be successful, I think this would be like a long-term work stream rather than just like one feature launch and then done Um in terms of like when you might see things, um Uh that we're building I I don't actually know the answer to that question. Um, certainly not within the next like Three to four months. Um, I think at the very least this would be kind of like a half year to a year thing where you might start to see like changes in chrome Um, I'm hopeful that we might have some kind of proposals out. Um and Before that so that at least like the The gears of it, uh, and the process behind it might uh might be more visible But like in terms of actually getting features in the chrome that can take quite a bit of time um We've got a question about accessibility. Have you thought about um accessibility and maybe in the In the narrower sense around things like screen readers and so forth or maybe in the broader sense just in terms of You know general accessibility for for everybody Yeah, I think this comes down to uh more on like the browser ui side So for example, like when when our teams would be building out the ui for this This would be like front and center of like, how do you make it accessible? How do you make it? Uh usable by like diverse set of users so like different languages Yeah, I I think building on like web annotation. I think is another thing that it's helpful here where W3c standards I think have to go through this already So in terms of like, uh, the design of of the feature itself I would expect a lot of that has already been thought through Again, this is all very early. So I I don't think that we've uh, personally, I haven't gone through and thought about this that much yet But like by the time we get to shipping This is why it actually takes so long to get features and uh into browsers Is like we have to go through like all these different steps and accessibility would certainly be one of those Yeah, but if if there are things um that we should be thinking about here This would be a great piece of feedback to get on on like our github repo um the the relationship that you've kind of laid out or the or the Door that you've opened in terms of Suggesting that this is a general capability that services could plug into hypothesis or maybe like a reddit or whatever um Is is is I mean love to hear that obviously Are there other examples that you can point to of web features that have been built into the browser That have this kind of service other than the browser itself, which obviously the tabs so pulling content from services But other than that, you know, kind of obvious example Of a kind of a service provider browser feature relationship um that are maybe um suggestive So the main analog, um, I could think of here was like the like the user's default search engine Um, so all browsers, I think let you specify which search engine you want to use like when you type into the address bar Um, I think that it's somewhat similar. It's a little bit different in that You wouldn't want just like one annotation service. I think you would want you might want to add like a list of them um And the other difference here is that I think like like you mentioned different pages might be able to Uh add services like through links. Um, or through other methods So, yeah, there are some analogs here, but I think this is fairly, uh, it's a fairly new capability Um, we'll see if we have a question about vandalism, of course, um content moderation. Um, you know, you spoke a little bit to that, uh, but Can you, um Um, you know, I I guess can you do you do you have any more further thoughts about um about that challenge and and you know ways that you think, um Um, you know make sense to you in terms of how to have a person problem Yeah, I mean most of this will just be like my own opinion. Um, I think keeping things in smaller circles would definitely be helpful Um, maybe if not even smaller just like user, uh, where the user Can choose what they see So for example, uh, we don't want the situation of like when you go to a web page and you see what the rest of the world has said about that Web page, um, maybe you just add like a circle of your close friends and whatever you go to a web page You see things that they have said about it. Um, so I think that would help quite a bit Um, I do also think like the default treatment that we give annotations in ui can be helpful here. Um So for example when you visit a page, uh Maybe we don't show like all of the comments immediately on the page Maybe we have like some kind of subtle hint that like hey, there's something here But you actually have to click through to go see it and so the vandalism might not be visible unless you actually go and seek it out Um, yeah, this is like I said, this is one of the biggest challenges. Um, and so any kind of ideas we have here on how to better address it I'm I'm open open ears Um So this is this is a little bit of a provocative question. So I'll apologize ahead of time google um annotation in in You know, there's a lot of things it can be very public. Um, it can be very private. It can even just be personal notes Um, but when it is public it is social Um, and google has experimented with some more social features before and social networks and so forth Do you how do you see this with respect to the To the notion of social networks and you know, obviously google I would imagine would want to be one of those service providers In a feature like this Yeah, I mean, I think it's right that this is very social. Um, I'm not sure yet What like the non-chrome part of google would want to do? Um, certainly we haven't heard that yet Um, but uh, yeah, I think there's room for for different kinds of use cases here So there are also non-social aspects to it. Um, so like I mentioned some of like the more automated use cases um Yeah, I I don't think this will replace like social networks, for example Like there's still going to be room for twitter and facebook and so on Um, but this this does feel like a bit of a social network at a smaller scale So like the exciting thing to me would be like if I can take my my small circle of like, uh, like a WhatsApp group, for example, um and be able to like go and read and annotate the web together Um, that seems like the most exciting aspect of it to me Right Um, uh, let's see. Um I do also talk about incremental pieces You know some of which may even be um under the hood and not even exposed as as user facing features, but could be underlying you know kind of wiring that That the browser could You know could could support Can you talk to what some of the first derivatives of that kind of? Nature might be that would Obviously support a kind of an broader annotation capability Yeah, so, uh, I think like the for example having, uh, author Author controls. Um, so being able to specify like a preference for a page to me that seems like A fairly easy one to kind of break out and something that would be useful Like even if we don't have annotations by the browser Like that's something hypothesis and genius and others can uh can leverage And and use so that's like one example of something that I think What could come early? um, I think We would want to look for things where like we can build A bit of ui and try this out without having to do everything um, so Perhaps like I mentioned the idea of like you can embed annotation content into the url So that might be an easy thing to do just in terms of like we don't have to set up servers We don't have to set up like a lot of like the networking aspects of it But it would help us build Like the ui and and start to accustom users and and see how users react to actually seeing these So that could be a first step um, what about things like that are not traditionally Thought of as like annotations like a tweet or something like this is it are you imagining that the Harness that you are creating might support Um Surfacing a tweet that that tweeted about a page on the web um as Something that could be in line So I think this is what's really Citing and powerful about building this as like a platform feature is that This is something that like we as as chrome or whoever wouldn't have to Consider this If if like twitter for example wanted to surface tweets in the form of annotations Twitter could set up an annotation service and start Serving those out So I think that's like really the uh the important part about like making things Interoperable and and using standards is like different services could be built And there's i'm sure there's like millions of ideas that I haven't thought of About like how this could be used and so we want to build something that can be extended and and New ideas built on top of it rather than just like we have an idea for how to do sharing on the web And then just like build the static version of that Um What about personal note taking so one of the um things that's most uh useful about annotation is that you're Uh, you know you can sit there and take your own personal notes over any page on the web and um in in fact annotation in a certain Constructs and kind of in the from the web annotation point of view is identical to like a bookmark or something so tagging and bookmarking be you know um and an annotation might become kind of a Of a continuum rather than a set of discrete, you know unique things. Have you um thought a little bit about the personal aspect of this Yeah, I think that is an important use case And maybe one that's actually easier than a lot of what we're talking about since uh, it doesn't touch on those issues of like page authors and content moderation So it definitely seems more tractable We have like thought about like for example text fragments Is like an easy way to go and do highlights on the web So like if you can make those persistent that's like a little bit of like persistent highlighting And then it's basically just a short jump to add comments to those and and then you have like annotations So yeah, we have been thinking about that I'm not necessarily sure how and when that would materialize but That is something that that could happen like in that diagram. I had you saw like We had like a special bucket For the annotation sources inside of the browser So this is something that like we could do and and keep it all Consistent with with sort of the rest of the vision Thank you. Um, that was amazing. Um Um, really appreciate you coming sharing your thoughts with us This is super exciting and I just also want to give a big shout out to the way that you that you've gone about not only this but also the scroll to text functionality and the kind of working in the open and bringing us all into the conversation feels great and And then just you know, can't say thank you enough for for that No, thank you. I'm I'm excited to see how this goes and uh, yeah, if I would love to see this continue So, yeah, just uh, just leave feedback and comments and ideas on the github repo terrific