 So, I'm the infrastructure lead and the thing that I want you to understand by the end of this talk is what you can do with our infrastructure. So, when we look at how people set up and these simulations. We're pretty familiar with starting by finding a PDB you like or docking a drug you like, and we can't help you there. But where we can help you is at parameterization. So, you can put a drug through gaff or CGEN FF depending on your favorite MD engine. You can ignore the warnings about how you should benchmark the generated parameters and not just blindly use them. So, you can do your all your black magic to the PDB file like I don't know adding caps adding waters keeping waters setting pro nation states. However you however you feel like it should be done. Then you can put the PDB file through your MD engines for preparation scheme to get the parameter files, you set your box and size and shape, you add your solvent, your salt, run an you realize that your drug had the wrong chirality, you re parameterize, resolvate re equilibrate everything you run your simulation, you feel like things have gone surprisingly well, and that you should try this out in a different force field. Then you do everything again with your new force field, hopefully the same way, but your notes weren't that great. So make sure to change your non bonded cut off because that's what that's the way you're supposed to use a different force field is you need to be changing your non bonded cut off in your simulation settings, otherwise you're doing it wrong. But then you realize your MD engine doesn't support the enhanced sampling method you wanted to use. So you try to get plume at working and like some numbers come out and you're like 60% sure that they're good numbers. And then you just give up and redo everything with a different MD engine. The tools and experience help, but preparation tools are permissive and failure is often detected late tools tend to move from chemistry worlds to model world as early as possible. Protein and ligand force fields use completely different tool sets and you're kind of left to figure out how to put them together at the end if you want to mix and match. The tools often need user oversight and are difficult to run in batch and automated pipeline. The open force fields are often engine specific and you can't really interoperate easily. And the parameters that affect the energy are not all encapsulated in the force field. So at open force field we thought what if your tools remember the intended chemistry throughout the preparation even after the parameters were assigned the chemistry could be known and referenced in the appropriate times. What if the same tools and force fields worked for both ligands and proteins, just throw molecules. What if the same tools and force fields could be used for any MD engine, and you could prepare the whole system and then apply the force field and export to any engine. Where everything was just done in a Python interface not jumping between tools with different styles of arguments or different file formats. If the tools warned you when you were doing something that didn't seem to make chemical sense. And finally, what if all of the energetic parameters, everything that you need to figure out the energy of the system except for the coordinates was included in the force field file. So, advances in force fields are cool. And we built a lot of infrastructure for applying them to. So we can take in just about any chemical format from smiles to PDB and give you a parameter I system. You can prepare the system once and then parameterize it any number of times with any combination of force fields and MD engines. And I think our support for new force field technology for new functional forms is going to make this even better. So we want to take you through what's available now and get you excited about the changes that are coming up. We have an FF workflow system preparation happens between the chemical structure files section and the topology section. And that's where you're putting things together that's where you're salivating. And so you can bring us starting there you can bring us your prepared system in any, in any format. So if you have the open eye wrapper in particular, or if you have an open eye license, the list of formats that you can come with is ridiculous. And with our decade you can take quite a few can get basically all the common formats, except all two that's very tricky with our decade back end as well. But more or less anything that defines a chemical graph is in scope as a starting point for open FF. And especially one thing that we've been struggling with is the loading of PDBs because PDBs sometimes they have components or you know residues that meet a set template and so we can look them up from a library like oh is this supposed to be a double bond is that supposed to have a formal charge. But oftentimes it doesn't oftentimes you've just got a small molecule in a PDB file. And we are now building out we've we've had some rudimentary functionality sense very early on but we're really building out right now the ability to load unrecognized components from PDB files. So you can create a topology, which has a complete physio chemical description of the system. That's the protein your drug your water and salts and co factors membranes buffer chemicals, whatever, along with the starting positions and the box size and you can save that out to a disk if you like just as a topology with no parameters yet. And then do other. And then you can combine that with a force field to get an interchange. And so again the spirit of force field specifies everything you're cut off so you don't need to worry about saying that correctly so you're not misusing the force field that's already specified in a spirit of force field. How to apply the parameters the P&E settings whether constraints are used. It's all defined in the force field. And our force field format. I think we can safely say is more extensible than other formats. It's easier to debug problems with Smirnoff force fields like we saw this morning without getting tangled up in atom type relationships and things like that. And then we can use interchange to get output files suitable for simulation and a number of md engines. And so, right now, we can export to open a man grow Mac open a man grow Max we're pretty confident about that I'll show you an example soon. We have rudimentary support for amber and lamps, and we're experimenting with importing components from these systems as well. So, basically taking things that were parameterized elsewhere, and if the force fields are compatible and that's a big if we're building out experimental functionality so that these things can be combined. And so if you parameterize your system and something happens and you don't like it, and you wanted to use a different force field. That's fine. The topology object itself before it's ever met a force field is serializable. You can save that to a disk and load it up later switch out your force field. If you want to use a different engine later, you could save your interchange, totally serializable. Come back to it, export to a different format, must with whatever you want in the interchange. If you want more accurate parameters for some drug series that you've never seen before it's got some exotic chemistry that's not from any of our training sets previous talk we've got bespoke fit it slots in perfectly. And if you want to do anything weird, we've got documentation there all the codes open source so if you can't find your answer in the docs if you're comfortable with Python, you can probably find it in the code. And we have first class support for Jupiter and then Jupiter notebooks so this is a really easy way to record and communicate the workflow and what you did such that other people can use it modify it. So I wanted to announce that topologies can now be created directly from PDB it's this came out somewhat silently in a release about three weeks ago. But where it used to be very painful process of loading all the components that you want in your system from PDB, you know, one at a time. Here's a protein and let's go get some waters. No, now we can load a box with proteins waters salts small molecules supplied that they have connect records. Yeah, it's, it's very powerful. Yeah. Yeah, so the point was in this code block, we have unique molecules equals ligand, and there's more than one unique molecule in here there's water and ions and a protein. And we have built in recognition for the water ions and protein. It's just when you come in with an exotic new species. You have to know the chemical identity of that and help us load the PDB by telling us about it. So, when people try to sell you Legos, have you ever gone to the Lego store. You know maybe you've got a kid, and they see a big block of red Legos the two by four is and they say wow I want to buy some like, look at that and then there's another big blocks of, you know the flatter pieces and other blocks of the two by twos, and they say, Oh my gosh, I'm in love with Legos. No, that's not how Lego sells Legos Lego they show you the dinosaur you can make the roller coaster and once you're interested in the things that other people have made, then you start asking about the blocks. And that's what we're going to try to do this. So, the remainder of this presentation is basically just a bunch of cool stuff that you can do with the toolkit, and to prove that you can do it we already did it. And if you go to this URL, Matt's put it in the discord. But you can open up a co lab for every one of the things that you're going to see in the remainder of this and do it. You can try it out with your own stuff you can install it locally. It won't help too much to pull it up on your phone they so tempted to put up a QR code, but your phone can't run co lab, but your computer can. So, these are our vignettes and the repository is 2023 workshop vignettes. Again, there's a link in our in the discord. And each one of these, we've given a color code so previously we've been very paranoid about sort of product management we don't want to release anything that the most picky of users would complain about. And so we keep things very under wraps for a very long time. And we found that's not helpful like you heard about this talk yesterday we showed some kind of wacky stuff you could do with open force field and she loved it and now now Jensen has been investing heavily that they've had a person doing this stuff for a while. And so we want to give you a measure of confidence in each one of the things that we're going to show. So we've got a stoplight system where green means it's production ready. You should expect this to work if your inputs are valid and if it doesn't work it's probably a bug that we need to fix. Yellow means it's something in the public API, but you should expect to be, you know, if you feed in a bunch of inputs, something weird's going to happen and we'd love to get your bug report we're going to try to fix that. And red means that this is something that's a prototype. It's something that if you're, it's part of a private API somewhere or you're going to have to acknowledge as some point in writing this code that you're using a prototype or something experimental the different packages do it in different ways but we show you how they how they work in notebooks. So for example, here's a vignette. We said, you know, hey, we want to show that open force field can do something useful that people haven't really tried before. And so we've made a myself self assembly simulation. It's actually not very big this isn't all of the code. This is some code showing that we've defined DLPC. We've used pac mole to make a box of them. And then we simulated that box. And sure enough, we get same dynamics we don't have a graph here I'm not sure if this is precise to, to a great extent but if you have a system like this that you want to try out. Go for it. This one's a yellow light only because the pac mole wrapper isn't something that's in the public API yet. It's something that's kind of a. You know, apparently, but if you want to go ahead and use it feel free, but if something weird happens at, you know, don't be surprised. Another thing I wanted to show off that we can do is, oh, yeah, and we didn't do it on that one. So what I want to show off we can do is modified functional forms. So we have a repository called smirnoff plugins, and it has a whole bunch of these handlers for modified o f f x m ls non standard o f f x m ls. And this is what our research team uses a lot. So here's an example of an octane and two metal purity and I think. So this is part of a hydration or a mixture salvation for you or something Josh that may help with recovery. But as you can clearly see here. This is using a double exponential non bonded for this isn't Leonard Jones. How incredible is that, and you can actually run this, you can, you can follow over to chocolate and repository here, and take a look at how you could get running in a non functional form. And so let me say a little bit about our docs. We've, we have comprehensive docs, we've now our documentation we've consolidated it under the URL docs dot open force field dot org so no longer do you need to go to every packages GitHub and scramble around for the documentation link docs dot open force field dot org is the landing page for all of our packages. And there's a search bar up here that will work to find you code from all of our packages API references theory documentation. If you don't know what package you're looking for, it'll search all of them. And we've heard a lot back from users and in fact mark Forrester has been really helpful in getting us to understand the new user mentality like, you know the wall that you hit and like the subsequent wall that you hit as you're getting started. And so we've added a page of what we're going to call wayfinding docs. And this is sort of like that diagram that I showed earlier but it's vertical. And this shows the information flow from chemical systems down into molecules and apologies and force fields out to an interchange. And so if you're wondering, Oh, what do I need to install what component do I need to be using for this thing I want to do. So you can come to this page and you can click through all of these, these go to the documentation pages for the different, the different projects and to the specific API reference for the component that you're looking at. We've also got consolidated lists of projects we've got theory documentation as well. We've got a number of example notebooks. And so you can you can go and check these out in the docs and run them. These are separate from the vignettes these are better than the vignettes and really high quality API references. So, yeah, I think Josh had a specific thing you wanted to walk through here, but I'm not going to do that. And beyond that we have video tutorials like I said we've done these advanced workshops and we record them we edit them we clean them up so you don't get the ums and the weights and the the room windows being cast. They're really kind of snappy I don't think any of them are longer than an hour, and they go through just very punchily the some of the advanced workflows that we show off. But docs are always going to be a work in progress, just like the software just like the force fields. And we want to gain a reputation for stellar documentation this was written before the feedback yesterday so Josh maybe should have updated the slide. But yeah, we want to hear the bad things about the docs. It's really hard for us to tell if they're doing their job because, you know, if we have bad docs, people ask us how to do things all the time but if we have good docs we don't hear anything. So please tell us the things that you're having trouble with please tell us if things are going well. We want to make sure that this is something that people like companies who don't want to get all mixed up in scheduling and IP security can just come to our docs and passively find the solutions to what they need. We've got a new centralized examples page coming soon so right now probably you've had a really bad time of finding our examples you have to go to GitHub and then click examples, and then it's just a bunch of folders any better guests that you want the right one. Instead we're making a central examples page as well. That should make these a lot easier to find. Yeah, and for all of these will have links to co lab will have links to installation we really want people to get stuff installed liquidity split and start using our tools. Yeah, so bespoke to it I don't need to go over this again bespoke fit can save itself some work. If you run it in a certain mode and feed in a bunch of con generic ligands it will recognize if the ligands are con generic and not not overdo things. And once you've run bespoke it. This is literally what it looks like by the way I know people sale I did this amazing thing in a talk and you don't think that you'll ever find the code or be able to run it. So here's the code. This is literally a vignette and it's just called the bespoke vignette. If you're if you go to that link from the discord. And in a Jupiter notebook you just run bespoke fit and then you load the force field and then this is 20 more lines of code to your protein ligand simulation. So bespoke fit the product itself. So we've done a lot to improve the stability and the verbosity of, you know, bad things that happen as William mentioned yesterday, when running bespoke fit but there's still work to do on that. We can detect connectivity changes because when we do QM optimizations, they don't care what come informatics that the bond is protons can wander around you get these awful fits. And we don't prevent them, but we do prevent you from getting a horrible force field, we have it safely crash out when there's an activity change. We've done some routine maintenance to follow the biopolymer support the toolkit. And now there's better intermediate file preservation so when something does go wrong you can see, though it's still not perfect it's a complex beast and we're working to make it better. As William said yesterday, sometimes force balance runs for seven hours, and then you don't know what happened but you decide to kill it or maybe it kills itself. We've made a fork of force balance that we're intending to use as a dedicated bespoke fit back end. And so force balance is great as a research. It's got all these bits and bobs all these interfaces where you can plug things in and all of those slow it down when it acts as a back end for bespoke fit. And so we're looking at just stripping off a lot of the extras from the force balance fork and making it just a very liquidy split torsion fitting tool to be a back end. Oh cool and now over to me to talk about the rest of our infrastructure. So, like I mentioned before, we've made PDB loading way easier. We've had it for a year, but it was very painful you could only load one component for PDB. And now you can load an entire solvated box. So this is one of our vignettes this is a green vignette PDB files, and it's, it's got one more block below this to show the simulation and that is the entire Jupiter notebook. All of the open FF code to set up the simulation here, and the rest of it is boilerplate open MAM to set up the integrator. So, if you want to run a simulation, easy peasy. Open force field will can't function without RD kit and open eye. That's how we do our chemical substructure matching for parameter assignment. And as a consequence of this so we have pretty high quality to and from RD kit methods and to and from open eye methods. And just to show you something that you could do with that. So, in that protein ligand simulation I showed you a second ago this was our starting ligand. And if instead before starting the simulation, I take the ligand from the topology and I send it to RD kit and I run this, this reaction on it that's this taken aliphatic carbon hydrogen bomb, and turn it into a carbon fluorine bond, and then take every product that could come out from applying that reaction once put it back in the protein and simulate it. So here you can see these different aliphatic hydrogens have been substituted with chlorine here you get to kind of at the same position because of sclerochemistry. And you get one over there on the methyl but we do filtering to make sure because it's replacements around the methyl you don't need three of those you only need one, and then send it out to simulate. This whole thing with 70 lines of code from loading to the chemical modification to simulator. So, not quite alchemical free energy, but still very fun. And so the toolkits getting really exciting. Right now we're aiming to do a little bit more for polymer loading, we can load unique molecules, but we can't load unique new like residues from a protein. And we want to be able to do that and we're working on that with the shirts lab right now. And now I'm going to talk about interchange a little bit. I showed it on the diagram but in greater detail, interchange is the thing that comes out when you combine a force field in a molecule. It contains all of the physics, physics parameters for your simulation as well as positions, and I'm not sure about the losses maybe an old figure, but box vectors. And we can export that to open M and grow max for sure amber for kind of charm and lance eventually, I think our lance exporter works pretty well now. And so that's our big picture. This is, this is our spider that interfaces with a lot of the other members of the molecular dynamics ecosystem. So what's inside of an interchange. We've got a vignette to answer that question. So we can make an interchange from a simple small molecule like this. Here's the code to make it. And then we can say okay interchange what's in you tell me about your bombs. Tell me about all the force field parameters that get exercised in parameterizing this molecule. And it says well there's three unique force field parameters, and I'm going to vectorize these out so it's sort of a linear but here's the case and here's the length so these case are up in the hundreds of lengths or angstroms. And then we can say well, where are these unique force field parameters applied, and it can give us a number of more useful representations but for presentations. This is a great representation it will give us a sparse matrix, telling us where these different parameters get applied to the actual top logical bonds and system. And if we multiply these matrices, we get the system parameters, which are again two columns pay and links. But these are, and I've numbered the rows here so a carbon fluorine bond has these parameters number four is also carbon fluorine it should have the same parameters. And so if you were interested in doing some like ML optimization force field interface stuff. Interchange is there for you it's got these API it's got a few others, you can just pull physics parameters out, you can pull the assignment matrices out or the parameter assignment matrices out and get to work using ML tools. And here's a similar vignette. This is showing. We parameterized a molecule, and then we're just showing the bond length because we just wanted to pick a parameter. That's assigned to each bond in here. This is the smiles parameters vignette. Interchange can export to different MD engines. Here, for example, this is a box of DMS O ethanol and some sort of like sugar thing. And we've created it using pack mall, we've assigned parameters, we've created an interchange, and on the left, we exported it to open mm and this is a trajectory of the open mm simulation. On the right, we've exported Gromax and Gromax, I did an energy minimization and NPT thing so it looks a little bit more packed. But yeah, simulations run. They don't explode. Things are looking pretty good. We had a slight bug with water but I think I saw it got fixed this morning. Yeah, so go ahead play around with Gromax. Yellow light, we haven't exhaustively tested out the Gromax exporter but it doesn't explode so far the energy single point energies look good so far. And if you smell anything funny when you try it, open up a bug report for us. There's early support, and this is prototype so this is red. There's early support for creating interchanges from already existing open mm systems. So you can get open mm systems from a variety of places like the open mm force fields package can make you a charm lipid or something, and you could load that into an interchange for combination with other components. Force fields are hugely detailed. Not all things are compatible. So this is going to be a little bit rough going and there's going to be a lot of user experience improvement that we need to do in this field. But here, for example, we've taken a sample six host guest system, and we've loaded it into interchange using the from open mm method so here's all interchange from open mm. And indeed we've evaluated it the numbers are maybe a little bit funny, but we got it in there and it's to a place where we can re export it to open mm we have the data structures it's holding on to it. You can try to export it to grow max that would probably be even uglier. So this one, this one's not wiggling as an indication of how much we pressed it. This is a red vignette. And another red vignette is the plus operator and this is one of everybody love this about permit is you've got the plus operator you can load arbitrary components stick them together and watch them wiggle in the same box. Over here, we create a ligand interchange just using our standard ligand parameterization workflow. We make a protein interchange that's when it's using the FF 14 SP force field and then we combine them together. You don't need to do this to use this workflow by the way, the normal way to do it is to make a single force field and apply that to a topology with both the ligand and the protein. But if you wanted to do it this way you can. And we can use that. And here's the example of exporting it to different formats. There's a little bit of gobbledygook because of bond constraints everybody handles them differently they get a little bit mangled and the exporting so again this is red. But for the most part open mm and grow max are matching closely on a lot of numbers amber looks like we got something fun going on. But these are things that you can do and we're going to be working on improving the reliability of these we're not getting parsing errors anymore we're getting numerical errors. So that's good. Beyond interchange. I believe it was bill spoke mentioned yesterday he tried to tangle with QC archive, it was very difficult. And as scientists tangle with QC archive, we have thick rubber gloves that are called QC submit, which reached out to QC archive, but it's your guide as an mm practitioner in QC land. The open force fields data sets on QC archive have additional metadata saying which chemical graph they're trying to encode along with some other things. And if you use QC submit, we use QC submit to submit those but you're not going to be submitting data sets. If you want to pull down our large data sets you can put on those same gloves pull them down and get graph molecules with energies patch. Here for example this is one of the torsion drives from our protein fitting project which you'll hear about after the break. And this is Bailey and Alan in Bailey and I believe, and this is a 2d torsion drive so it's it's got like a bajillion points, but in the end, you get this nice energy plot, which I put in hard trees, because I didn't have time to convert infrastructure wise we're also building out Alabama scale. This is what David Dawson spoke about yesterday. This is our large scale orchestrator for big numbers of free energy calculations we're going to be using that routinely in four scale benchmarking. I'm not telling you anything wiggling here, not because nothing's wiggled thousands of CPU hours of things have wiggled, but it's a very large data structure in a database and it's not practical to go get wiggling out just for a top. But this is running. It's very exciting. Right now it's sort of a tool for internal use our deployment docs are pretty good we hope we have a lot of researchers at different institutions getting access to this, and I think that you could passively try spinning your own institution but if it has trouble, I wouldn't be too surprised. It's it's kind of a complex piece just by nature of its scale. And so one of our big goals in the next year is to make the user experience for going through the toolkit and getting to a simulation, a lot easier. And so for example, this is the non canonical amino acid vignette or workshop that got Rebecca on the track that brought her ultimately here nine months later. This took 40 seconds to run just execution time and had about 160 lines of code you had to chop out the new amino acid and put caps on it and calculate charges and it was a whole mess to load it to to load it from PDB it's just awful. And with topology dot from PDB, and I didn't use the same modified amino acid in this case just for a technical reason. But here I've made an amino acid it's like Alanine Alanine free me and Alanine but the way Alanine Alanine Alanine free me Alan Alan. But the three means really weird, I modified the free mean it's not just fossil free mean to make sure that this is not anything that might be recognized from a known set structure library it's like phosphorus, oxygen carbon nitrogen, or something. It's completely arbitrary modified amino acid. And instead of needing to do like 50 lines of loading this weird unrecognized thing from PDB. We just, we've added a new argument and it's read, because this is private, but you can feed in now an OFF more where some atoms are identified as leaving. And it will now recognize this as a substructure alongside all the other amino acids that can be loaded. This is not the final API, you can play around with this, have a lot of fun. But your code will eventually break when you remove this option this is just here to show the possibility of how easy things can be, if we can load modified amino acids. So this outputs to over here you can see this is from a PDB it doesn't know anything about bond orders the carbonyl oxygens or single bonds. So the output of this is a full chemical graph suitable for Smirnoff parameter assignment. And so we can go ahead and run the simulation with this using the naggle prototype instead of the AM1 BCC like semi empirical charge assignment. This has a runtime of 25 seconds and it's down to 90 lines of code. So you can go ahead and try running this in the custom sub structures and naggle. So our big goal for the coming year is to improve our infrastructure is usability we have amazing things coming out for the force fields and we want to make sure that we can get those force fields into as many hands as possible. So we want to expand interchange to have sort of vanilla importers for open and amber and grow max and by vanilla. I mean, you know, don't bring. Nope, we can't import things that are already parameterized and have polarizability. We can't do exotic things but we can do Leonard Jones bonds either constrained or unconstrained sort of the manila stuff that's like the sage tile parameters, we want to be able to import those. And we want to make this really easy for protein ligand workflows like I showed you we can cut down the lines of code for modified amino acid by about 16. And we want to keep cutting that down we want all of these to be very snappy and not require user customization. We want to have expanded polymer loading like I said, and because we run out of PDB codes. We want to add support for PDB X and M M SIF, and we want to make a major push to improve the quality of our documentation. And one of the big things is that we want to centralize our documentation that was organized into all of these little packages that don't talk about each other so if you have a problem and you go to one package docs, you don't know that there's another package that will really squarely do what you need. So we want to go ahead and make it really easy for new users to figure out what tools they should be using and how to how to get started with them. And finally, I want to say, this is going to be the year of benchmarking. So you've seen so many exciting things with polarizability with alternate smirks typing this morning with with all the different things. So as an organization we're looking at this and we're saying, wow, which one of these should we put into the mainline force field first, we can't do all of them at the same time that would be a huge mess. And so we need some way to fairly compare the accuracy improvements provided by these different, these different suggested changes to the force field. And the way that we are standard benchmarking suite had run before was that it was carried over from the season one industry benchmarking project, which was a very tightly knit software stack that was made to work with exactly one version at a time. And the only goals for this were accuracy and deployment. We had to deploy this on pharma computers that we couldn't see that we couldn't debug and it needed to work on the first crime. So as a consequence, this workflow is totally inflexible. It does RMSD, DDE and towards and fingerprint deviation if you want anything else, and you will probably break the entire workflow silently with scientific error, and the numbers will come out gibberish but it won't tell you why. There's a huge demand for benchmarking keeps getting reused. And so we want to modernize it. And we're making a big push this year for what we're calling generalized benchmarking. And that is where instead of being locks on all sides, there's plugins on all sides. So instead of loading, you know, a preset exact, you know, only QC archive data. We want to have different plugins for different types of data inputs so we can have local QC data, QC archive data or physical property data sets or other kinds of data defined by users. We want to have a number of analyses that can run and again just another set of plugins. The first generation of generalized benchmarking will do the season one benchmarks, along with the sage release benchmarks so these are condensed phase property evaluations. And we want to add support for alternative functional forms, because this is a big point we have really smart people at open force field suggesting changes to our force field that could make it much better. But we need to make sure that we have the benchmarking infrastructure to fairly compare them and do the exact same tests. So phase one is we're going to get this put together the plugin infrastructure and functionality equivalent to the season one benchmarking and the sage benchmarks. Phase two is we're going to roll this out to the entire internal team. And I think this has been a long time in coming, and I think everyone's going to be really excited when we can give talks and have you know the exact same plots saying how our methods are improving the quality of the force field. Depending on the state the state of outcome scale. We might be integrating that so when you have potential force field you want to test using the benchmarking harness it will also dispatch a number of calculations to pulling it home. And we may also do the protein observable benchmarks that you hear about from shape in a few minutes. After that, we're going to be discussing a scope for phase three, and phase three could include more sorts of benchmarks or data sources, or depending on the state of the stack at that time and what our boards think this could become a season two industry benchmarking which does everything that season one did with the program analysis, as well as condensed phase analysis if there's interesting things about that, as well as free energy calculations using the open free energy staff. But that's something that'll be for discussion. Listen for this and advisory board calls if you're on our board. And I think that's the end of my talk I'm a hair over time. So what I'll do is I'm going to release everyone or the coffee break, but feel free to approach me if you have any other questions. Thank you.