 And today I would like to talk to you about research road mapping with discourse graphs. As Eugene said, my name is Carla Kirstenow. I'm the Network Research Team Lead at Protocol Labs. And at PL, we build tools to move science forward faster. And we think that these tools are important public goods. And today I'd like to talk about one of those tools, the discourse graph, which we see as a protocol for sharing and synthesizing knowledge. And how we use it in research road mapping, which we see as a process for aligning incentives and managing information flows between researchers and research funders. Oh, this works. This is awesome. Great. So discourse graphs, a protocol for scientific coordination. What we mean by that is there's been a lot of really interesting work being done in the road mapping space, a lot of it by, for instance, the Forsyte Institute has done some really excellent work with their tech trees and maps map initiative. We had a really great talk about that a little bit earlier. And so I'd like to talk about a tool that we've been supporting at PL, which we think is a nice complement, a powerful support for road mapping efforts. We think that discourse graphs can be used to describe the present state of the knowledge frontier, find opportunities to move the frontier forward, to evaluate different research initiatives, and find leverage points in a research network. So we like to think of science as being a complex system with lots of moving parts. And we think the discourse graphs can give us a little bit of insight into those moving parts and allows us to see little points where we might be able to influence the system for maximum impact. And the work I'm talking about today is largely the creation of some of our grantees, some of the people that we've had the privilege of working with. So people like Joel Chan, who is a professor who studies creative knowledge work at the University of Maryland and the creator of this particular discourse graph protocol. And David Vargas, who is an independent developer who builds tools for thought. And Madoka Matsu, who's a molecular biologist who expanded the discourse graph grammar, the discourse graph system to meet the needs of modern experimental science. And then also, of course, the discourse graph community of practitioners who are researchers, scientists, thinkers who get together and do cool things like meet up in Hawaii and talk about image annotation and genetically engineering novel fungi. So these are the kinds of cool people that you get to work with if you're interested in discourse graphs. And as was mentioned, Joel and David were kind enough or foresighted enough to offer us an impact certificate for supporting this work. So I want to mention that, that I'm talking my book here. I'm very incentive aligned here with this talk. And this was co-issued to me and Evan. So at some point, we will of course fight to the death over ownership of this impact certificate. I've been doing my push-ups, Evan. So let me describe the discourse graph grammar before I get into how we use it in road mapping. What's really cool about discourse graphs is it's a very simple, very naturalistic system. And it replicates the parts of natural living conversations. Really, there's some vibrancy and dynamism in the protocol. So you might, you will recognize all of the parts of the graph. It has pieces like questions, asking questions, making claims about the world, adducing evidence to support those claims, and then referring to sources to provenance that evidence that you've reduced in support of the claim. So of course, the Darling of the Internet forum, the source, I made it up, is not good enough for a discourse graph. And those, let me move back a second. So here we have the nodes of our discourse graph. Now let me talk about the edges of the graph. The relations between the nodes are the relationships between these different pieces of the discourse graph conversation. So evidence informs questions. And it can be used to say support or oppose a claim. So there's a charge there. Claims can support or oppose other claims. And new questions can be informed by existing claims. And this is interesting because it means that there's sort of a little bit of pre-adaptation for the generation of new questions. You may have a claim out there that's sort of in search of a question. And it hints at the modularity and composability of the system that something that seems to be settled in one discourse graph system may invoke or stimulate new questions in a new or adjacent field in a new discourse graph system. A lot of interoperability. It's like playing Legos, and it's really fun to build. A little bit more about the data model. If you think about a discourse graph system as a system of nodes, and indeed it could be expressed in a system of postive nodes. Very simple. Question nodes express an open research question. And claim nodes synthesize observations. What that means is that they bring together a lot of different things you've noticed about the world, and you generalize them into a claim about the world. Evidence nodes, on the other hand, a little bit lower in the hierarchy, express a specific observation. So they carry with them a lot of context, a lot of specificity, and source nodes contextualize those observation nodes. So the build process, when you're creating a discourse graph, you start with a set of questions, and you collect observations to begin to address those questions. Now, I should mention that those two steps are often reversed, especially in empirical science. A lot of times we find that we make an observation about the world, we say, huh, and then that generates some questions. I think a lot of science is precipitated by, huh, more often than Eureka. So then we synthesize, we've made a bunch of observations, we synthesize them into claims about the world, and then we compose those claims into arguments and theories, and then we go off and we find new questions or hypotheses to test. And I think you'll find this is a very familiar recapitulation of the process of scientific inquiry. This is what you do, and we're just kind of breaking it up into parts and making them explicit and, you know, sort of describing them with pretty little boxes on a slide. So discourse graphs are a client-agnostic protocol. They can be implemented in anything from, like, a cork board, murder board, to a marrow board. I think some of the more popular implementations now are net book notebooks like Athens, Log Seek, Obsidian Rome, for instance, is probably the most mature ecosystem for working with discourse graphs, largely because that's where David Vargas, the open-source developer that I mentioned earlier, has developed a powerful suite of extensions for implementing the discourse graph protocol that's spreading to a lot of different clients, and David has a project where he's going to make that spread even faster. So I mentioned there's some flexibility in the protocol. I spoke earlier about the biologist, Madoka Matsu, who's sort of adapting the discourse graph grammar for experimental science. All that really means is that he's changing the terminology that we use to describe the different nodes in the graph to better reflect what goes on when you're working at the bench. So the original discourse graph protocol was developed from the needs of literature synthesis, sort of examining and dealing with the needs of the current knowledge frontier, assessing the state of the current state of the art, the current body of knowledge in a particular field. Results graphs, which is sort of Matt's twist on the subject, it's a little bit more geared toward experimental science. So the question, claim, and evidence backed up by the source of the discourse graph in the result graph is a hypothesis conclusion result, and the source of truth there is the experiment or the model or the simulation. A slight difference, but it has particular implications. What it does is it emphasizes that the source of evidence in the results graph system is the experiment model or simulation, and it makes it very clear that a hypothesis is a request for experiments, and it's a system that makes it easier to formulate well-structured requests for experiments. And we'll see how that works when you incorporate this into the road mapping system. Okay, so I've said roadmap, I've invoked the term road mapping, but I haven't really defined my terms and it's sort of like I'm burying the lead here. What I mean by our research roadmap, it's a goal-centered model, essentially centered on a breakthrough innovation. It describes significant milestones in a technological effort and the relationships between those milestones. It is meant to concentrate expert attention on a research problem. It identifies leverage points, as I mentioned, in the research system. It surfaces them, sometimes serendipitously, for expert attention. And it allows us to drive distributed coordination around a problem. It sort of makes everybody, it gives a shared context for attacking a problem, for examining a problem, evaluating it. I would say that a discourse graph is a user interface for distributed coordination. That's well with the road mapping system. It helps us to understand the current state of the knowledge frontier. That's the synthesis process. To identify dependencies and missing links between milestones and to create requests for experiments to incentivize distributed pathfinding. So the process where different distributed decentralized researchers can follow their own local incentives while keeping in mind global incentives, the global state, and the global optimum of the system. And so I'm going to describe how this works in the context of, as an example, an RFP. In fact, an RFP that was developed and funded by the CryptoNet Lab at Protocol Labs last cycle. Just using this as an example, this roadmap was derived from the RFP. The RFP was not developed explicitly as a roadmap, but I think it's a good illustration of how, because an RFP is a goal-centered research initiative, it can be represented as a roadmap. And so I'm using this example here. Any errors in the diagram are solely my own fault, and I apologize in advance to CryptoNet Lab. So the first step of building a research roadmap is to define the goal state and its properties. Ooh, I have a laser pointer. I think any time you get a laser pointer, you should also have a conference cat. So you define what is your definition of success? How does the world look different or better when you're done? In this case, CryptoNet Lab wanted better vector commitments. And then they defined the properties that constitute what is a better vector commitment, and those you can see in the purple squares outside of the green triangle, goal state. Then you draft open problems for creating the necessary properties, and those can be kind of thought of as research projects that are necessary that you've defined to address the properties that your future state needs to have. And then you might, if you're particularly kind, suggest potential directions for solving the open problems. So these are ways that you might address or ways you might approach the open problems that your grantees may make some mention of in their grant applications. Here you can see each roadmap node in the RFP generation process can be seen as being supported by a discourse graph. This is where that composability comes in. You can just sort of drop in a discourse graph at any one of these nodes to support your reasoning as you're building this graph. For instance, what is the desirable goal state? This is meant to represent an argument or a conversation that you may have internally as you're building out your roadmap. What are the essential properties of the goal state? Another argument, what research projects are necessary to enable those properties? And what are the highest impact directions to take within each research project? These are opinionated roadmaps. They express your beliefs about the path to progress here and so they should be supported with some evidence. And then finally, these are all synthesized into a roadmap and you make the crucial distinction of which properties or projects should be de-prioritized. I'll get into in a minute or two, actually just one minute, and I'll start with discourse graphs. Again, an open problem statement is a request for experiments. So while you were shifting now from the internal discourse graph-like project of creating the roadmap that you may do with a small circle of collaborators and externalizing the problem, you're inviting people to the conversation by asking them to contribute results graphs to solve the problems identified in these nodes. The nodes are still here. And now I'm going to describe something. This is about what's called the evidence store. It's not very easy to see in this slide, so I'm going to go to the next slide. It's already been implemented in software in a couple of different client softwares, but this is just for illustrative purposes in a little narrow board. What it does is it's a querying tool, the David-built, which allows you to attach attributes to any one of the nodes in your discourse graph. So for instance, when you're building a roadmap, those appropriate attributes might be the robustness of your confidence in a particular direction, the amount of support, the amount of evidence and support of going in that direction, the amount of evidence opposing going in that direction, and even whether they are at the lower level, whether there are sources that support or oppose a particular, support or contradict a particular piece of evidence. Now if you're doing this collaboratively, it may be appropriate to describe those bits of evidence, those attributes as distributions, which essentially could represent child-source credences about the true value of whatever you decide to attach to your discourse graph. There's a lot of flexibility here in the different implementations. And essentially what you're doing is you're building an investment thesis. We're seeing here not only can roadmaps be used to drive what we call an RFP project, but also there's meta-roadmapping, where you may roadmap the process to drive adoption of a research public good that's sort of at a different scale. So these discourse graphs are kind of scale independent. And essentially each one of these graphs represents an investment thesis for the direction that you want to, the direction that you want to support, incentivize, accelerate. And so we can see that there's a little bit of similarity and complementarity in the way that we build discourse graphs and the way we build research roadmaps. We start with a set of questions, start working with the current knowledge frontier, synthesize claims about its position, locate gaps in the frontier, estimate how important it is to fill those gaps in a comparative way, request experiments to fill the gaps, collect conclusions, re-synthesize all of our data, and push the frontier forward, and then start with new and better questions. And I think I'm contractually obligated to show this particular slide, crossing innovation chasm. Our missing effective coordination systems to make this part happen, Juan has mentioned this several times. I think I'm going to cheekily drop in the results graph protocol. It is a small but significant contribution to the types of coordination systems that we'd like to see built to move science forward and to coordinate activity around a research goal. If you're interested in these types of problems in building research portfolios and talking about research road mapping, research team, that's my team, research program managers, that's my role, research scientists, research startup operators. We also have a discord for network goods, public goods, and nerds. If you haven't joined it already, please do join us there and nerd out with us. And Joelle and Matt and the discourse graph community of practitioners, really awesome people. They are in our discord. They have their own discord. They're all over the place. If you want to read the docs there's a really nice wiki that Joelle has put together. And I should mention Madoka Matsu. If you want to work with results graphs more closely, Madoka Matsu is hiring a Siberian, which is an even cooler form of librarian to help work with the results graph syntax, results graph grammar, protocol, and community. And thank you very much.