 Thanks so much. Thanks for having me. So before starting the presentation, going into the meat of it, I thought I would do comments about what we have done over the last few years. It's been about five years since I was with you guys at the San Diego meeting, and we wanted to get a bit more active in the community. So I hope this gives you some context of what we might be able to contribute. So first, my lab is consisting of a diverse set of people having a lot of different interests. So we're working in a variety of areas, but I think the tools that we are applying and the lessons that we gleaned from it, they transfer very well into what the old proposal was trying to achieve. So I've run through a couple of these projects and make some comments also relevant to the discussion we had last night about the machine learning and AI infrastructure while I was at the intensity. So in my lab, we are working on lab automation, back to discovery project and engineering, our methodology. And we are also in this consortium which I'm writing which is called the Consortium Molecular Designs. We're trying to a very close test and build cycles together with chemical engineers and biology colleagues. So our journey into AI really started in 2018 when I attended the CAS 13 conference right after getting the academic position, and then shortly after we had the San Diego meeting for the open postage group. And there we were just done retraining Alpha 4.1. It was sort of our hello world program where we tried to democratize something that didn't seem to become an open source solution at that point. And so we put a couple of reports on it and one I wanted to point out because if you have you have asked me about it is this one where we have shown that if you take two amino acid sequences that are very similar to each other, they only have a couple of mutations different. Some neural networks are actually able to get structural differences right. And so here we showed that across one network was able to get the contacts correctly predicted for the different folds, which was something that was at a fold, for example, couldn't do at that point. And then there were a few other articles. One thing that we were really interested in understanding is if these type of deep learning models infer any insights about physics. And while looking at it, we realized no, it's not the case, because deep learning is always some sort of shortcut learning. It will always exploit some underlying bias and transition that from your training data into the test data and hopefully it generalizes, but it won't necessarily use anything physical to get to it. Yeah, so the code is our first attempt to be open source and do some things when we find afterwards published their reports, they kind of mentioned it. So we hope we have a little historical footnote that maybe we have a little bit reshaping the publication strategy and making some of that outcomes publicly accessible. After that, we thought, well, what do you do when you do machine learning? The standard problems are work the docking and doing hand side identification. It seems to be always the first application everyone gets their hands dirty on. So what we have tried to do is to go one step beyond basically docking tools by saying, hey, there's so many docking tools, they all have different random number generators at the end. So maybe there is a signal there if we build a consensus model that uses machine learning to find the underlying pattern. So we trained the machine learning model for mid-dock that uses five classical force fields and uses all of the terms that come out of them. So a set of about 50 teachers runs that for a shallow neural network and then was able to differentiate quite well, at least on our test data set between actors and decoys. And we're currently applying that actively in the cash challenge that some of you are aware of and we have used it now on enemy and sink. It's pretty expensive when you're in that many docking to it. So not everyone is excited about it, but we know that there will be some results that we're using it going forward. I'm personally very excited about the work the open three energy group is doing and I hope that we can contribute with this pipeline to it in the help of the benchmark. Then with regards to image annotations, everyone wants to build a really nice image classifier for cats and dogs. It works wonderfully because we can agree what they look like. When you ask pathologists about cancer types, there's a lot less consensus in the room. And so why for the last 15 years has no one been able to put a clinical approved prostate cancer machine learning model into operation? It's a good question. We wrote a review paper to come out where we show that the problem is really that there are no unbiased data sets and the bias that exists between a pathologist looking at an image in this orientation. That orientation is as bad as the difference between a pathologist and the best AI that you can have today. So very interesting and I'll answer the question to what type of data sets do you need to use if there is bias in the data and do you want to capture something that has to meet the relevance and I think there are some insights learned here that can transfer in our machine learning situations as well. We're also doing some more classical bioengineering. So here's a study where we looked at a biosensor that has a very broad ligand spectrum and we tried to reduce the ligand spectrum by rational design. So we went some knocking on it on some active site neighbors that seemed like potentially relevant, created the library and then tested it and we found one point mutation that was able to become inactive for a specific substrate. So making that sense a little more specific. The reason I'm presenting it here is because after the rational design we run in these simulations already because the crystal structure sort of all looked exactly the same. So the question is why is the difference in behavior and in these simulations they showed that there's a shift in the Boltzmann ensemble of confirmations and that is really what determines the function here and so when we talk about Alpha 402 or some of the structure-based models we are just not at the end of the journey yet because it's not enough to get static structures. We really need to go into structural insolments and probabilities of structures given whatever conditions might be present in the cell in order to fully understand what Boltzmann does. And so that led us to the question of what can we do with machine learning in the field of enzyme design. We just wrote a video paper on that and found that it's a very hard problem. So pretty much everyone tries to guess that and machine learning at least in 2021 was very limited in its application. So we started working in that field and we applied some large language models. So by now there are a few other tools out like protein MPNN for example, Sega Eugenicov is doing a lot, Baker is doing a lot again. But when we investigated these models we realized that there's a bit of a systematic error in there because when you look at the learning objective that these models have is they always try to take a structural confirmation and try to optimize the free energy around that confirmation completely ignoring the rest of the energy landscape. And so there's a high chance that you find the structure that may be very optimized but it does also have a different energy minimum and then you can get misformers and misholding events. And so when you look into the reports you can find that people are always surprised at why the natural language model is giving you such a great idea of where to go. It works only in a certain sense of the cases. And our hypothesis is that there are actually alternative confirmations that are being also optimized and people are not aware of it because the learning objective is just not properly defined. And so this preprint, which hopefully is going to come out soon, in the journal we show that there is a way of rectifying that by using base rule and by doing some tricks to the way we use these language models and we applied that to an enzyme in nanolubes and we're able to stabilize over a wide range of temperatures which gives us a good idea that this model might be able to help with the hard task of protein engineering in the future. What it really showed us though is that if you bring people in the room and also the digital pathology showed us that they have a very strong machine learning background they might be able to do a really nice job at giving you an architecture that predicts something that's very accurate but it is just not the right objective function because if you don't have the domain expertise and all the implications of it and what else could be happening it's very easy to be misled and go for something that has absolutely no relevance like a capital value of one when you train an AI model for cancer classification where the data set was created by one pathologist on data set from one hospital and completely ignoring the heterogeneity that you normally have when you travel across the country and ask multiple pathologists. A great machine learning model people publish it but at the end of the day no one will use it because it just won't generalize the remains of the AI bar and the same thing that happened in biology systems as well. So we need to bring together people that have domain expertise as well as the machine learning and background to get these products to work. Then CAS 14 happened and I was really invested into the trust by product and we thought we could make this something that would revolutionize the field had some nice attempts to improve on it and then of course after fall two put us all into shock. During the conference we created this little self-help group to meet once a week and kind of overcome the stress of having the likelihood of our funding completely destroyed. I'm actually called an open fold and we had a weekly meeting trying to understand what is it that makes AI bar 4.2 so great. We had a lot of external speakers come in, Fabian folks talked us about the SE3 transformer and we were wrenching into these mystical fields of math and physics and how to integrate them until eventually we realized that Alpha 4 talked a little bit more about what could have been done than what they actually did. The interesting thing is of course that these concept like SE3 variants are very powerful and can be used in other contexts. However in these discussions we realized that retraining Alpha 4.2 would be too expensive for the resources that we had. I checked this morning who is left from the group and the only one left in our lecture was Mohammed. I guess he liked Alpha 4 so he stuck with it afterwards. Good. We are stuck with equity variants instead. One thing maybe that I learned and one of you pointed that out to me. They said if I come up with good names I should do a better job of announcing them to people and so here's the model of our lab. We call it the data rhinos because we created this integrated data science path at BYU where we take students that are very promising STEM students like Bryce who is presenting with me. He worked with me on the prosper model since 2018 and then give them the ability to work in consortia with industry partners together so that they come out at the end with a degree where they have a data science applied background that helps them to become a real contributors for the rest of their career. I like the tagline we're quitting unicorns, only real ones, tougher ones, right? I don't like computing university to end on the big corner just to be stable and data rhinos. Anyway, so one thing we also did is we were very heavily invested in a company called Zonto that does lab automation. She signs up inside that organization with us acquired by Broker two weeks ago, which is nice because it gives us some funding routes that they didn't have before. So due to that I'm able to find the consortium to make it a design a little bit better at BYU and for example contribute Bryce for the next three years to the open forestry initiative and have them work between some projects that will hopefully enable science a lot of scale. Yes, for sure. I forgot that I picked up somewhere. Okay, so but what we really took away from these CAS 14 results was that the idea of inductive bias is super important. And when we train machine learning networks the way we can go a level beyond what everyone can do is by putting some hard math and physics into it. And so what we did then with Fabian Fuchs together is took one of his SE3 transformers and pimp it up a bit. He had trained it on the terrible QM9 data set if I can quote John from yesterday and we retrained it on the Any1 data set. And just to show that this will be able to give us a nicer performance and the initial results look pretty good. So we presented it at a new website event and today we're going to show you what happened in the next iteration of that network. Before pointing that out I just want to also highlight a different area of application because the question came up yesterday when will ML replace normal MD simulations for force fields in our domain? I don't know in our domain but we wrote a review paper on what happens in the field of molecular simulations for molten salts and in that area we see an exponential growth of purely ML based simulations because they need to have very polarizable force fields and you can only get that with AIMD and all this up-in-issue stuff is always limited to 100 atoms, few picosecond simulations. It's not enough to get them the thermophysical properties that they want and so they're applying right now all types of crazy potentials that have a bunch of underlying systematic errors. So one thing that they love to do is run an MD simulation up-in-issue, the DFT one, and then they fit a Bela Parinello system on it which was optimized for single atomic types and when you read the literature you will always see them say this works great for two species molten salt but when they go to four or five atomic species it completely breaks because the input vectors are hard-coded to correspond to an average chemical environment which obviously shifts inside of the simulation box but they haven't quite caught up on that yet and so I think that collaborating with the open force field group can help us transferring some of the insights that you have developed and best practices into that field and hopefully have this vertical domain also to make progress but with this I'm going to transfer over to Brice so you can tell you a bit more about the new model of TRIP. Yeah so the TRIP was the next iteration of our SE3 transformer-based neural network so in context there's there's a ton of different neural network potentials for example LaShunet, Bela Parinello, Annie but the majority of these neural networks they only work with scalar type features which for molten salt simulations you need polarizable force builds which can't which aren't be able to which these scalar networks can't predict the dipole moments correctly and another big limitation of these is that a lot of them require handcrafted input features using symmetry functions so what we wanted to do is instead work with an aqua variant type neural network which has a lot stronger math which has a very strong mathematical foundation based off of spherical tensors which transform co-variantly under rotation so an example of why these are useful is aqua variants is a principle that allows physical properties to be encoded so for example on this figure in the bottom right diagrams what exactly aqua variants can mean so if you go down then you calculate forces on a molecular system and then go right you can rotate those forces but you end up getting the same forces as if you had first rotated the system and then evaluated the forces so you can generalize this beyond just vectors to higher order geometrical objects that these tensors that we work with so we're just to say we're not the first ones that have done this other people have also been working on this but our method is more general than for example neck whip which just worked with tensor field networks our method uses sd3 transformers which also incorporate attention mechanisms which allows it to then have angular degrees of freedom in addition to the radial degrees of freedom so some of the features of our network is we trained it on the anime 1x data set which supports carbon hydrogen not nitrogen oxygen species and we use an atom agnostic architecture which is something that we don't really see many other people do and what I mean by an atom agnostic is that the majority of our parameters are shared between atomic species the only atomic species dependent layer is the atomic the species embedding this allows it to then work for many different species in the future where we want to transport and transport to new systems for example molten salts a problem with like Baylor polymelo is that as you increase the atomic species the input vector increases dramatically and so does your primary account so by having this species agnostic architecture you're able to limit the parameters and then also allow information to be generalized between atomic species another thing that we added is we we have added correct atomization energy and repulsion as someone mentioned yesterday if you run with any potentials you can end up with fusion and we've corrected that by adding a nuclear repulsion term and then we also added an atomization energy so that your molecules don't cause power bearing simulation and to help what the areas that might not have been trained by data we we bias it by using our physical knowledge of how the system should behave and then the last thing that we added the last feature we have is that our network is smooth it says in the mathematical sense that our derivatives and all their derivatives they're all smooth they're all continuous that helps prevent some of the instabilities that you have with other neural network potentials so we benchmarked our network on what's called the Comp6 benchmarking set and this generalizes beyond the molecules that are in the training set so it has larger molecules that the network was trained on so it really gives you an idea of how well your network can generalize beyond the molecules it was trained on that's one of the things that a lot of the network potentials are looking out is that they're trained on a very small collection of molecules just with different confirmations from the test set it's just different confirmations of the same molecule but to have a general potential you want to be able to say how will this potential work on molecules it's never seen and we find that our network performs better than any one X anyone all the other published methods and we're really excited about that because it means that we're going in the right direction and that our architecture has potential we also did energy surface scans of H2O and this kind of diagram's what I was saying earlier about the atomization energies and the nuclear repulsion because if you look at that all three of the different methods for calculating energies they all have the same similar minimum around one angstrom but if you look at any if you go smaller than one angstrom then it actually doesn't have the repulsion term that you'd expect from physically but like DFT does but our method does have that because we have hard coded that into our architecture and then another issue is that if you start pulling it apart then you end up with another minimum which isn't physically realistic but our potential overcomes that issue by having the correct atomization energy we also did a torsion scan of a bedroom and we found that our method got slightly higher RMSE values and that we were able to predict the correct structure of the torsion scan we're hoping to benchmark it on the bigger dataset of torsion scans that was introduced yesterday though and then we also ran MD simulations using our different using different potentials we used the amber 14 with the tip 3 water uh and then also any 1x and trip potentials using um solvated models that and the water molecules their forces and and energies are actually predicted by the networks rather than tip 3 um we find that we're able to simulate molecules for 20 nanoseconds and then get a good understanding of the statistical landscape of the MD the MD system so we calculated different MD statistics of the secondary structures what what are the average pi triangles what's the standard deviation of those and then how often is each of these uh potentials how often do they predict that they should be in one second their destruction versus another and we found that the anyone acts um it's it samples um regions of the the rhomachandron plot that generically aren't accessible but our our approach um limited the limited mainly to um the more common rhomachandron areas um yeah so going forward we're hoping to apply it to enzymes and I'm gonna pass it back to Dennis just a quick conclusion so uh thanks Bryce I think the potential has a lot of potential the question is now which data set do we need to train it on to get something really important out of it one thing we hope is set by teaming up with open force field some groups from the molten salt community will feel more inclined to share their trajectories right now everyone runs these expensive trajectories and locks them up which I think is a waste we could train on those and then really expand the network to different atomic species like Bryce mentioned transfer learning is completely possible you don't have to retrain anything you take the model as is and fine tune it on the next atomic species and it shouldn't lose its ability to work on the initial data set um yeah and then of course it opens up the door to reactive force fields which I'm really interested in for the case of enzyme design we have done so much enzyme simulations and tried to work with the QMNM simulations it's always a pain being able to have one force field that does the reaction of the active site correctly but then also models the water molecules correctly about being dependent on some our integration is a this powerful promise we have to see where it really needs the future so with that I'm just going to point out that I'm excited about collaboration if you have some data sets where you'd like to try to try this out we can train that guy very rapidly takes a couple days on our GPU cluster and we can bring you some benchmarks on that and we hope that together we'll define some best practices that will inform our collaborators and colleagues in domains that have not yet the 40 years of experience that we have with water models and solvents here so with that I'm at the end of the presentation thank you for your attention