 Thank you so much and thanks for opportunity to speak here and crush your party and I think you know, I Learn a lot and you know, I think there are interesting opportunities because It's all kind of the same problem. I'm we have the zoo of the sports fields and they try to Make something out of you know, the open FF consortium kind of started top-down I use that Microscopic properties to fit your parameter. So I'm coming from conditional chemistry background. We do You know bottom-up approach we start from from crystals and going up. So probably we'll meet somewhere in halfway in between Basically today, I'd like to talk about some of our recent projects when we use machine learning to solve some of them Long-standing issues in conditional chemistry You're currently in transition, so I'm at UNC Chapel Hill, but we changing colors and moving up north and but hopefully we'll do the same and I'll skip the you know introduction. I'd like to start, you know, acknowledging You know, really smart students Privilege to work, you know my lab at UNC, especially Roman entity, you know My partner in crime Group of Adrian Royberg and you know Florida funding mostly an asset when our collaboration with Los Alamos and big shutout to the big machine because as you will see we use a lot of Mechanical calculation, so I kind of think about machine learning and I had tools at the CS army night So it has its own utility, but it's not perfect But we found, you know, several use cases. So My lab is also part of the what's called up low consortium It's a high-troupled computational initiative for inorganic material and we develop a lot of predictive models for Inorganic material think about you know ceramics alloy things like that We also apply to two-dimensional materials. Yeah, we use a machine learning to guide chemical experiment and material science We also use some tools for visualization of chemical space One of my interests been in a school of pharmaceutical science with drug discovery and we work also in what's called Generative models and machine to kind of open a brain and in this all chemist and kind of start to dream in in in molecules, but in this talk, I You know, I talk about our work in quantum mechanics and how machine learning could accelerate Mechanical calculation, so essentially if you think about this kind of cartoon representation Do you have a graph when you have a methods of different scaling where so some kind of medical factor? We can argue about that, you know force fields can be Extremely accurate if you parameterize but generally, you know, if you look for, you know transferable force field The problem is you have this uncertainty that can be really good I'm quite accurate, but they can be terribly wrong and in many cases. We have no idea Because if you take, you know, you affect you you put some uranium fix up, right? You can still run it and it will not complain, but the question is what you get and Then we have, you know, the empirical methods for example X to be, you know, the functional, you know conventional DFT theory is scaling and Q and the gold standard in the organic wall would be CCSD with with triples and And basically you can put those error bars and basically this systematically Converge until you reach the heaven of the PCI Which is a dream Basically what we try to do because of that extreme scale and his we try to push those methods to this left top corner when you have advantage of fast compute, but hopefully maintain You know the low error bar of quantum mechanics. So one one up, you know, if you think about You know Quantum mechanics one-on-one to the Schrodinger equation some independent in our case. We use Concham DFT equation but if you look on it and Kind of rearrange all the complexity you can come up this this way so the ground state energy Surely the this is the magic function F With respect to molecular coordinates, that's a regression And so this what we do take neural networks in molecules of neural network you get ground state energy Neural network is differentiable. You can back propagate your gradients. Now we can do generalization Dynamics, we can also do hash and something like that Now very briefly how it works. We were inspired by ideas of Bella Bella and Perinella who are Take molecules and think about as the in terms of atomic environment So when each each each atom is represented by a spherical environment in our case about the nanometer Sphere to encode its neighbor. The problem is that is additional Bella and Perinella Environment it was designed for I Think silica. It's very easy. It's the easiest of the homogeneous and and basically in that paper we reformulate the idea of atomic environment, which is more transferable for complex environments in organic molecules and can handle Fancy fragments. So in a sense what you took are the neural network kind of zoom each or one environment at a time and and essentially the Total energy of the molecule is going to be the sum of those those environments So as John mentioned essentially what we see emergence and convergence of this hybrid force field Tokyo's honey generation essentially your proximity short range Who live in the neural network, but they also have a dispersion. For example, you can use D3 D2 grimoire empirical dispersion and And again, we use a particular choice of But as that's the empirical To describe non-bonding Wonderwalls in fraction. It's it's empirical essential kind of Leonard Jones kind of type Yeah, yes So now as as we get more experienced We can we can predict atomic properties for example partial atomic charges atomic volumes So you can be a little bit more sophisticated and you can Include a dispersion now we can extend for example, but check a chef left side dispersion or even many body dispersion but also we can Include electrostatics and it can be as fancy as you would like can be charges, you know dipoles and you know the multiple but still you you You parameter you you describe your short range chemistry with a neural network and And you know would hopefully I'll have a time basically we have a new architecture which is Which is kind of long range because if you if it's in our atomic environment. They're short range we have to use some kind of physical equations to Describe long range interaction, but they have a new architecture. It's we call a net which is Inherently long range and then you can you can you can implicitly describe all kind of interaction. It's fully inside the So this will give you absolutely black box method Now very quickly how it works basically we run a lot of anti mechanical calculations For small fragments like organic molecules from one typically to 1012 cave atoms then We feature eyes them through atomic environments, but then through a neural network and and basically do a summation of those essentially fictitious Energies of each atomic environment and Then when you do summation you you basically check with you is the reference Quantum-mechanical data, so we don't really care about those fictitious energies Because of the summation then they compute the the error compute the cost gradients and update the new element I can basically use rate Currently we probably try seven elements So if you give if you draw me a molecule with those seven elements, you know Again, you see it's mostly biogenic elements or kind of drug like space We'll give you energy forces and Encussions we're working to extend it to phosphorus and you know more collagen filenium and bromine we use Omega B 97 X functional from marketing head Gordon This is range-separated hybrid functional for those a DFT of shadows in the in the in the audience We started with double zeta and you know upgraded to triple zeta and currently We we switched to a different more modern functional. This is your Here which is arguably, you know, the best functional what we have today and also we have a couple cluster CBS You don't know there's a bit train on the couple This is fast less than a minute That's ours Not on a laptop. That's on a Again, so the Pathological paper been published in the 17 we call it honey with neural network It's it's it's an approach to to train neural network potential and also sampling and the question I've been asked why we call it honey So this is any and what we would like to do given proper training So we want to train my padawan to be DFT Jedi master I said and hopefully Yes, exactly and hopefully We would avoid that Scenarios that, you know, somebody would be killed in this And probably you should appreciate how hard we work on this convoluted acronym To satisfy, you know, my and Adrian's laughter Star Wars, but also, you know, for legal reasons not to be seen by So basically we we try to push, you know, and make this functional into the Black corner. Now, let me show you a couple of example, you know, the machine learning methods and you know models typically criticize the Blackboard method and I agree with them So we we try to try to develop a way how to sense when we can trust them So and of course we use methods from active loading So for example, the simplest way would be to look for the ensemble of Disagreement so you train an ensemble of neural network and see what's the what the difference between predictions And you know, very simply again the technical in this paper. Imagine you have a certain space space and You know, you you predict the sort of one can be an energy or properties and you train us in a special way three different Now and the absurd certain prediction now what you can see Because we can run quantum mechanics so we can query the oracle and we can Basically look for the our true answer as then you can observe. Okay, so there are regions where neural networks will agree So we have good data coverage. There are some bad regions when we have either overfitted or have bad data coverage when When they disagree, however, what you can notice it that when the this assemble is disagreement with large We can monitor those those region and you don't need quantum mechanics to identify the point So therefore new neural network itself will give you kind of uncertain and uncertain the quantification Signal could say hey, don't don't trust me now, you know, go and you know, you know, run more quantum mechanical calculation and improve and and basically what it allows us to do essentially Use self-consistent almost fully automated framework how to train those Neural network parameters, so you you throw all your kitchen simple data sources You score them for this for example ensemble disagreement, you know, and we've established certain Criteria, then you run a compute cluster, you know, pay the database retrain the ensemble and basically this loop is fully automated so students can I don't know it pizza drink coffee and you know the Project it's nice and it also, you know remove this barrier of that, you know, of this Manual labor that went to that Making those The ugly side of the thing it those methods are extremely data hungry so original any one We had to run 20 million safety calculation. So our carbon footprint is But again, we rely on the on the Facts that you know chemistry semi-local. So you use small organic almost pregnant like Now and then you can test it in in different regions you can To the question was how many molecules Yes, so yeah out of this 20 million we have about 70,000 organic molecules and for each of them we have a lot of different confirmation out of equilibrium Structures, yeah, so so you had you know most of most structures come from out of equilibrium This is a GDP database from Generally, Premon. It's basically enumerated all possible small molecules and candle You can test those things in two different regimes You can draw your test molecules from the same distribution take on a small molecule and this is what called interpolation But for us for chemistry, it's not very interesting because we want to try something new so What all all test I'll show you basically they've been done on this unknown essentially in extrapolation regime when we For example, so this distribution show you sizes of the molecules of blue corresponds to our training data. You see this quite small, but we we develop set of You know a test Test data sets that goes to a 50 hundred heavy atoms, you know We just that includes, you know a trip up ties, you know FDA approved drugs and you know larger lunch organic system so we see how it works in kind of more realistic scenario and We developed, you know, several data sets so this original data set has 60,000 molecules 25 million data points Soon to be depreciated Because using active learnings. We can be much more data efficient Also have much more diverse and easy easy to work but we also have the couple cluster data and that will be released soon and We currently work on the extension the sulfur Enhalogens, you know being run Now, let me show you kind of and again, this is this is available today and This part of the relief from the next month Paper been written now show you kind of accurate. So this is this is our test molecules You see you have greasy drug like molecules. You have small proteins and you can afford to run quantum mechanical calculation and So what's the sort of accuracy you can achieve? So you you you will get about one to two KKL from all in total quantum mechanical energy Which is very nicely correspond to our relative energies Better than one KKL from all for most of the system. So I'm sure you a couple of examples of kind of hard system for many Forts filter you have sulfur, you know a lot of colorization you have halogens and What I'm showing you here. So here on the right is the potential as a surface one is DFT So if I take a couple of the hydrols and you know and and and do potential energy scans and then the left From our potential and as you can see in any case is basically Brilliant of each other and potential can represent, you know the small little details This things and potential energy surface and the total MSE you see is in one or two KKL for the full To give you very very nice approximation for the potential as a surface and Unfortunately, we've been a little bit scooped by Johnny de Fabry from a Celera who implemented actually And beat us in our own soil. So you he used any and to parameterize force fields with gas and So basically it's from a paper from last month But essentially already, you know keep fields, you know And in many cases you can you can easily replace our force field and do torsion scans and get high accuracy essentially this paper what they show You you get the same accuracy feet of your classical force field Doesn't matter you use our potential on and get you nice It's neural net so yes, it was under gaff Gaff to I guess other advantage is fully reactive and Currently we don't have much of training data in the reaction, but in principle you can do simple chemical reactions What I'm putting here, you know, I'm running IRC for particular chemical reaction. Those vertical bars gives my IQ Uncertainty quantification as you can see we have low uncertainty In the products and reactants very high and as you can see the transition is physical However, if you add a little bit training data, basically you can fix that and This is your in the black as the actual gear and again, I show you out of sample example So this particular actions were not trained specifically for this But you also describe chemical reaction We have Orsters so you can run molecular dynamics and and basically what you will get so for example this particular cheesecake compound Again, it has sulfur and halogen so you know molecular dynamics you run abinish on our current dynamics and then the Our potential basically RMSC in energy It's essentially within one k kelp from all. So this is the The magnitude of the force component is within three or four k kelp from all per answer again. It's a very low error Quantum forces Absolutely Because our our forces are true gradients of the energy. So it's flat as We can do Hessian so we can do harmonic frequencies RMSC about 25 to 35 wave numbers so you can do decent thermochemistry Calculate we get the energy harmonic approximation, for example So John, you know mentioned the charges reaction Also trained to charges, but in our case, it's a conformer dependence. There's a wonderful patient charges It's a three-dimensional geometry dependent. However, the problem is charges, you know, the whole zoo of charge scheme and You know, for example, there are several here And they all as you can see if you do charges They very little correlation between different charges. So we weren't asked different question To dipole and quadruple are physical So they ask neural network k neural network reconstruct Dipole and perhaps constrain the quadruple but assign charges What what and what this paper describes essentially we rediscover a same five in which been fitted with experimental dipoles, but this was unbiased and A little bit surprising that essentially if you constrain if you constrain and and and try to reproduce a dipole and quadruple Basically, you can essentially Defeat him and and again since it's Yes, so so so we went to the, you know We constrain dipoles and the neural network was recon was assigning charges to reconstruct the site simply from the derivative of the quantum dipole and in fact we kept If you insist that the Dipole is is fit you you get a A unique set of charges which reproduced the energy as you said and you can go on and get dipoles from the quadruple But in principle it was interesting surprise as those charges might be But they were nicely correlated Yes, yeah, this is all that space at this point Right, so I'd be pleased to see what you've done with solution to because this This is this is great, but it but it's always a challenge to do things in water and things like that We have not done anything Then what we can do we can now we have energy our forces we have our charges and dipoles You can run, you know, for example and simulate our spectra and gas phase, you know and compare with QM for example And and basically you can get very nice Run and essentially and the time the main simulation so everything done by you know few machine learning Very easy to use we are you know use Python so for example Again, I'm not venturing to run in a light simulation. So In shorts, basically you read the XYZ file You can instantiate the calculators. You use any model you can get potential energy and then For example, you can get forces you can use your standard methods minimize geometry and check that more or less converged and Then you can do for example vibrations and then so that was a water molecule. So again, you get three free three Normal modes so it looks like water all positives of this. So it's good. And finally you can do full thermochemistry You know zero point energy In the harmonic So there are a couple of examples in our github now There are certain applications where the DFT may not be enough And I've been looking on that and in particular are what we try to attack the problem So the amount of data required to train the spins is very high. We cannot do conventional couple Despite the progress in the big machines. So we have to be smart. So what we come up with the extrapolation scheme Which essentially use approximate couple factor with so-called deal-pinner operation and Essentially, if you if you familiar so this is a standard scheme of the hill gacker how to do a complete basis set extrapolation to the Using the the standard couple factor scheme. So essentially this terms would kill you because 7k alien. So what we come up with this? Inside extrapolation scheme inside the extrapolation. We have kind of like in this moving inception, right and However, what you what is interesting is to take couple of standard benchmarks you run standard clapa cluster And you can really see for example, you go from alanine to aspirin to double the system size So you your CPU, you know Essentially grows financially, right? So in in our work basically the we can much we can mitigate this one while maintain the Using their personal So what the scheme allow us to do we can scale up couple factor calculations and and we actually run half a million of them on a large machine with all of us and And then we also had to use a different tricks with full transfer learning when we use our DFT potential as the as essentially as a cheap proxy and You know, but but also gives you a nice apex, you know I rough approximation of the country and energy surface and then we can use a smart sampling Where DFT is wrong to kind of fix that so what we do you you you basically you copy weights of your neural network that been trained on DFT then you you know you freeze part of the weight and Then you use, you know this transfer learning technique with the sampling to retrain only part of your neural network is couple plus today and Essentially what you get this Frankenstein? It's neither couple cluster not the DFT however What we show in the in this paper as published recently basically in many applications that neural network is fire and accuracy Well, we tried but typically you constrain bottom layer and you train Little so I'll show you just a couple of examples of skip technical details So for example In reaction energy, so there's simple hydrocarbon. So it's an interesting data set of hydrocarbon reaction energies And if you look the simple, you know, there's nothing fancy But if you look for high-quality reference for energy, so the plots give you so there's a six You know seven reaction energies and on wine access is it is an error If you look for our functional so a mega B97 X in red There are in this reaction energies where I from 12 to 40 K Error, it's a disaster if you have reaction at you know error in reaction and they're It's total disaster, right? so obviously neural network potential been trained on on this DFT me mix and Those errors are as bad as it would work. However, given a little bit couple cluster data. So this light You know two kind of color the neural network potential been trained with transfer learn essentially You know get it to the one or two and again, none of the system when the training data No, those are a thermochemistry products You know, it's yeah, so it's it's it's reaction energy Yeah, yeah Yeah, so this is gas phase again. It's it's gas phase reaction energy. Yeah here There isn't there is no barrier yet. So this is just a reaction yet. Yeah. Yeah Yes, exactly. It's a thermodynamics. Exactly. Thank you There's an interesting test data set from Genentech right or Bertha run various Distortion benchmarks with different levels of theory for drug like molecules and and essentially we can get all the The hitter all profiles quite accurate, you know, again DFT is not bad, but it's overestimate. For example, it might be underestimate the you know Confirmers, but basically with the with the couple plus the chain and you you get essentially of that and In this paper there are many of those now, so This is kind of a summary of the benchmark of the of the whole of the whole data set. So this is your force field you should not name and There's a various types of any 1x train on DFT essentially the way in between the MP2 and you know the D3 lip with the couple plus of data So it's essentially it exceed all the all the all the DFT functional Been tested and it's approaching the your fancy, you know, a post-hardware of methods that you see there Some composite scheme as MP2 2.5 or some kind of extrapolation Be basically we can get the torsion really really accurate. Ah, okay Very recently we've been working can we go beyond just simple and We've been thinking about, you know, how to expand different architectures. So for example, you know, this is essentially a place on architecture when you Use atomic environment, which is essentially fixed type Scripter for your and then you feed the neural network. So we've been trying to do, you know, work hard to explore different You know fancy architecture and in particular can we use? Learn embeddings. Can we use message passing to describe long range instructions? So I'm in the and I'll and also can we train to multiple? There is no reason why you To prevent in you to train to multiple things and in particular, you know, I'll fortunately have to keep technical part, but This particular paper we train to eight point it Space energy we get SMD energy. So this year are continuum solvent Approximation and then atomic charges atomic volume things like that. So you have you can have multiple properties an advantage of that So inside neural network, it's more a data efficient to more quantities you train easier each one is yet and also In terms of simulation, you will get all of them in one single class So this will additionally accelerate, you know, you kind of simulation you have to property predictions as a separate exercise and And also, you know, we have this message passing layer We have this message passing layers that allow us actually to put all those long range interactions inside the neural network and maybe we can Keep using those, you know, a standard Show you a couple of examples. So for example So a net with this is equal one. So it's it's it's one pass. So essentially, it's no it's no long range Is yet so essentially it's equal accuracy in any but as As you as you increase and you pass messages more environments, essentially you you use the error and it allows you to get more accurate Energy and for example, what clear example would be if I take those substructure We have sulfur, which is very polarizable. I have the subsity and are and if I modulate, you know electron this drawer or donating Type of the arm I can modulate the sulfur charge So if we use atomic environment and for example, this are More than five months from away So this the standard way how you predict charges will not feed are and essentially all my charges would be equal Because it doesn't it doesn't feel this are but as I pass messages I can essentially recover the correct behavior and they eat and kind of the long long range influence of the are disrespect To the charge is one example We can get again Salvation free energies and and that's we use them and sold database and which which reference the Pre-energy of salvation and basically now you can do relatively accurate salvation free energy just this machinery so you can use the standard Equation and we'll get I must say about one point a little from all which is kind of on the On par if you use two different models or two different functionals or you know approaches Now let me spend a couple of minutes just to Show you a few things where I think it's going so we currently work with the group of Phoenix It's it's a software for protein physical structure fine from Pavel who's at work Lawrence where to lap and basically there is a there's a lot of excitement about excitement about hi, I am and one of the one of the way how you A crystal structure use quantum refinement and this lawn can very complicated pipeline We use quantum mechanical calculations to get better, you know, met and of the atoms So what they use right now probably the fastest quantum mechanical code for terra camp However, there are a few drawbacks. Basically, it's very expensive Basically, you you must have a Tesla GP news and and and for large proteins takes weeks So if you work right now, basically if you plug it in on it, it's free for academia It runs on a laptop and basically you can you can refine me. I Will not show you Fresh crystal structure fun is honey yet, but I'll show you a Quick example what we will be able to do for example, you go to PDB Take a particular protein We have a ligand here So it has all seven elements In the structures and again, I take my potential which is currently in development It has not been trained to this particular protein. So we internally add some water data and, you know, simple Diamers and and basically I can set up a simulation. So it's 35,000 atoms explicit water. It's a box Thunder the only place I'm cheating. There is no iron. So this is your distilled water for things and and I run it and The trance we run it for several nanoseconds. This is really boring movie But I argue this is a great achievement. It didn't explode didn't default and Things didn't go weird. So given enough data and not not seen a you know food protein At least it's state stable for a low number of nanoseconds The the ligand is side the active side. We see the hydrogen bond in contact, you know, there's your loop So things like okay, so they give us a hope that you know, we can we can parameterize You know protein ligand force field given Not infinite amount of content and and only five nanoseconds It's been shown that you know, you need to go out past microseconds Yeah, absolutely, and I totally agree and it may be milliseconds to get a protein No, I totally agree, but just to show the qualitatively But you can do it we hope to do it but also I think it's What is more interesting make the students run this relation make probably the most common mistake So this is a Spartacus, and this is Arginine and The the the carboxylus fraternity and this is probably by far the common mistake When you run a simulation Seed of the movie, but basically you see the the proton transfer So this this this potential is reactive. It's a neural network Everything inside the neural. Yeah, so that protein simulation didn't run inside This is a neural network. So by potential I mean The neural network which we trained on the content. Did you actually have Training examples where you had proton transfers or is this just not specifically this so we have since we use normal modes basically the protons we have enough examples where proton far But it's between They they fly Because the sample in the energy window Yes, but you have to have an example where there's something that accepts the proton at the same time as something that donates You might have you might have something like It might have said with a with a better carbon you or something like that But we have not trained specifically so take the Arginine and you know, I'm not trained But maybe if you would be Okay So it's reactive and now One more question. That's okay. If you come go back at the very beginning you talked about the composition in this whole thing into various input vectors We have five angstrom regions that scan for the structure. Is this still the process that you're using you That's we run the number Okay, so there is a there is a wonderful. Oh Okay, that's all guys But they but this is handled essentially writing on it What this what this? Okay, can I have one more question? Yeah, so I believe that everything was trained on single molecule data, right? the energies so and I also understand that if you expand this to a Many molecule problem. It's not a problem because there's no bond in your model, but did you Did you make an adjustment for the fact that some the reactants might be entropy driven and Did you incorporate? Do you think it's might be interesting to approximate some of the entropy factors in This model if you're doing MD then the entropy consistent with the potential energy surface is correct you just have to you know calculate the right function of What you're doing so you'll get crossing Re-crossing stuff. All right, but in theory by doing MD your entropy is included If you're doing you know Q8 quasi-harmonic then you know more complicated And the reason for that is if you're training if you get the energy surface, correct correctly your all properties follows so then If the energy surface is correct and you do MD you get So you don't necessarily have to train on that specific Now you might get to it later the question I always ask is What's the density of you know water ethanol? Lexing, yeah Okay, so like then like the temperature dependence and what about you know, I'll call this is not the best right water one Oh, right, but You know, it's okay for It's not terrible, right? That's it for you. Let me tell you also liquid under ambient condition It's a big achievement. So I think I think that's what I'd love to see in a lot of these force fields is actually No, what are the condensed phase properties? Looks like I there's there's all these matches to QM But if you want to use it for proteins and the question is No, I would love to see in more of these papers. Let's see what the liquid properties look like. Yes So we started from down and we probably creeping up so we'll do test on on diffusions and densities So is that the version with Lena Jones and Electrostatic correction of any so where you have both of them. Yeah, okay So then you get water Yes, yes, so the development version has water clusters molecules and high channel. Yes Limited but So on top of your purple helix there in front is that a hydronium I'm I see I use a right day to visualize Sphere so this thing is just what's the cut off? So just from water, but it looks like there's a water with three protons attached to it. This one. Yeah That's that's there's something behind I see Okay, and so that's probably not a hydroxide a little bit Okay, as long as they're VMD artifacts and not your own that yes Okay How long did it take the simulation for five minutes? So since it's reactive I cannot do Which is what you use for ab initia, and it's probably five five to ten times at this point more expensive At neural network site and class But at this point it's probably 50 times Lower so you cannot fold the protein quantum So I I have another comment, sorry to be piggy, so I looked at this Picture now for quite a while. So your your arginine is this also a VMD artifact that you don't even start from neutral Arginine residue. Yeah. Yeah, so that was a student mistake So that was a mistake because I would be impressed if it's like a neutral arginine And you have a proton transfer from from your assets to make it a positive arginine There's a negatively charged arginine. No, no, no, no, that was a neutral arginine. It's a game plus minus Because I see a two binding nitrogen to well and sniffered in there Okay, Alexander if I may ask a question this is Thomas Fox from Burring-Ingelheim from remote I Like I'd like to see this work kind of having in the whole protein With with a neural net approach my experience I tried that same thing with Say poly alanine polycystine With increasing size some like five six seven eight Alanine's in a row Come came up this well with three hundred four hundred confirmations and then calculate the DFT energies and the aim net energies of these and I get Kind of differences of five seven ten Cake hells I get an overall R squared of point eight. I wonder Given all the rated at these small Very small systems. You get such such a difference between the I am that's energies and the DFT energies, how would you ever think of being able to simulate whole whole protein? so that we hope to do with the you know Active learning when you do a few cycles When when you pass your data sets and you do few iterations to kind of look what parts of the potential energy surface have higher error But when you do this few iterations Then it's rapidly get, you know much much better quality if you just do single points of probably and Did you use the pop? Yeah, obviously, so you use that the amnesty from the public repository exactly Which doesn't include any, you know intermolecular interaction data What I'm showing you is a developer's version when we have some you know clusters and Intermolecular interaction the one which is in good hop Just isolated Yes, but even those it's just it's just isolated molecules 40 atoms 50 atoms Then it should work according to your your presentation. Let's let's you know, let's go offline and you know, send us things and we can Be happy to look okay, perfect Let me finally show you even crazy things what we add is the Just reaction carbon carbon bone break this this movie is probably Wrong, but the the thing is that So far what we train that Small molecules and you know, I break in a carbon-carbon bond in them in the organic more As anyone the students about this crazy simulation We have carbon vapour so it's 4,000 atom 60 atom 60 and 60 and strong books high-temperature simulation so you start from random gas and When you observe this reactive simulation you see formation of flakes There is a formation of cool it in here And you know wrapping things and you know And it's interesting so the quantitatively this simulation probably won't because they actually only an Equation through the root, but what is interesting it? It learns enough to suck carbon of six-membered rings into flakes Bucket balls and nanotubes So it learns enough chemistry or physics so To kind of you know get that it also get partially right the way how this second flakes Grow by doing a chain of you know floppy chain of carbons and then you know taking next carbon So you get you get some physics already correct and That's actually gives a hope we can do you know high-quality Reactive potential and we will not need to do it reaction by reaction when all 50 volumes of organic reaction books And what is also interesting that probably seven years ago So late professor maracuma published a science paper doing this simulation took them a year Only to run on a big machine That can be done on the left So that gives you you know us as Ross used to say 99% of chemists who don't have access to big machines I think it will give us That's an incredible result I think but at least to me getting With the react of getting nanotubes, etc Your input information to this neural net Had nothing but for example, but molecules so the way your neural net knows about The carbon-carbon reaction. We add a little bit Carbon-carbon won't break up. Oh from molecules. However, we not show any carbon nanostructure We show carbon bond breaking information as That you know, I showed the reaction exercise essentially we show different like carbon carbon carbon distillation formation those alder, you know Like that. Oh, so you trained it for that but it's been trained only to molecular Right, okay There was you know, IRC's for this reaction So this is literally awesome stuff like Incredibly cool. So I just wanted to ask about two aspects one of which is the speed you mentioned a 50 different time speed difference But between like open mm on a similar system. I get like 15,000 times So how do we close the speed gap between that and that we had this discussion You can probably fix it but Harmonic restraints, but then you kind of you see the purpose, right? You want to avoid reaction To increase the time I don't know maybe the custom is it really the time step That's the main limiting factor because I think we could do next like there's a lot of tricks that one could do in To get that close that down. So We should talk about we should try one of your integrators The other question is the you know, the the folks over here have been finding that Um Quantum nuclear effects are a major Effect in anything that involves hydrogen bonding properties, right? So anything water anything hydrogen bonds That's probably important for proteins and small molecules. So How do you deal with the quantum nuclear effects? Are you thinking this would just go full path integral? Are you thinking that you don't need it or are you thinking that you could fit to condense phase to correct? Yeah, even if you run ab initia and v you need to couple it plus the integrator or some kind of method So that's as easy to interface. Yeah. Yeah Uh Maybe, you know, we'll have because it's a lot of startups could do the fancy neural architecture chips If they can, you know You can run, you know, we use standard fight works of respect Because if the is it going to be custom cheap, you can run it faster You can use it Okay, great. Yeah Excellent Okay, great. Yeah I'm just uh, I think it's really cool and thank you for sharing it I I sort of wonder whether as you go through this do you eventually occasionally just come up with something weird that happens. Oh, absolutely Oh, can you tell us about that? I should do well There are you know, we work with several groups and of course, you know, there are some geometries get born You know, for example, because at short range, there are some conjugated system Not fully planner. They're a little bit There are some issues early on and the nitrate group get too Because we get too many, you know, there are there are some things but I think the advantage of that Because as a user community, you know, at least the people who work with They send us those weird structures for a while And then the retrain we can retrain and typically fix the problem, but don't break And it's not at least what There as you go with reactive stuff, it's it's it's it's it's not going to be simple I'll show you good cases, but you know, since there is no physical constraints, you know things get really weird You know, for example, you can get two atoms too close to the other and it's kind of You are minimum. So we still probably have to Carefully inform users what is the main applicability of that Because as soon as you go out And there is no data. So you have absolutely no physical guarantees whatsoever To run it And at this point, since it's a student driven development, so we have zero warnings in our code whatsoever Okay, thanks Let me finish. So it's uh so on github It's coming, you know soon in your favorite Packages, so it started from amber Alberta did open mm. We're working with lamps. So if you're interested and you know, tell us, you know, what other packages we need to consider Again, mnet on github data status at the sphere to work with a number of academic labs some of your friends are competitors And thank you very much So what sort of investigation is going on for are there better basis functions to be using different common eight, you know different ways of Yeah, basically different basis functions what what potentials are there for getting something that's maybe a little more transferable With better basis functions. So when you When you triple zeta basis, sorry, no, I meant like the basis functions for the neural net like how do you Not not the quantum. Oh, you see that It's kind of settled. So we knew, you know, we kind of know it works. So at this point, I don't know If you ever hit the limitation of the descriptor Or, you know, we get the discretion of use the word descriptors. Yes. Yes But how do you know if it's like the best set of descriptors are better I have no idea. We come up with one We use it We kind of advertise it, but there are a few other types different neural networks What is kind of interesting? So it's almost like a wild west, right? Yeah, so people come up with this amazing things and trying You know, maybe in a few years, it will settle down. There will be few more mature It seems like there must be some way to like, you know, what is the minimal, you know Descriptors space that's the smallest that that captures things the best. It seems like coming up with like a good way to quantify Quality of descriptors is important to make things transferable Or more transferable The thing is, you know, for you it seems Fascinated and I've been on another conference and say no one kick out for more. It's terrible. It's awful So some people think one kick it's it's unacceptable accuracy But there are many many applications. We can have a lot of mileage out of this accuracy So you never, you know, there will be always many opinions about What is acceptable or not? I'm just curious was the conference you were at a crystal packing conference No, that was a quantum chemist that there are people who do very accurate You need like one way number accuracy for yeah, those people are yes um Would it be possible to go back to the slide where you showed the fit with the forces and the energies Yeah, I think it was an md It was a while ago I guess I have a good memory. Yes, there you go. Um this was done with uh A force force field representation for the non-bond or this was done with this is isolated molecule Just neural network. There is no larynx or range. It's just one molecule And the whole simulation is done with the neural net. Yeah, and then we take Snapshots run qm Calculated energy and forces and that's the error for forces Because one of the things uh, the reason or and I found was that there's The forces there the a huge component of the forces on atoms in polar systems is uh, the Is charge flux Which is not accounted for In standard representations, but I guess it should be here uh again, you approximate whole physics in a Like box neural network. It's hard to partition But but this is kind of a simpler case because of gas phase Thank you Can you elaborate on how you transfer the the long range information a bit more? Um, you can do two ways one range. You plug it into standard mb package. No, uh, the implicit They implicit. Okay. So we have update vectors And so you originally have embeddings, which is kind of atomic environment When you do one pass Your central atom essentially collects information, you know, think about passing message As soon as you is in one environment, we have vector for every atom inside this environment Through through through space. There is no bonds basically Yeah, yeah, so basically you you you sort of learn interaction inside your one environment And then we have this pass essentially like a residual connection That you can loop over and then you pass this message to your neighbor environment And when you iterate you essentially pass messages through this environment Yeah, so you have learnable vectors and and basically you iterate Kind of like it's almost like scf loop Which you know, it's kind of iterative, but there's no variational principle. You know, it's just a nice analogy So when they iterate so essentially you kind of You know look what neighbor of neighbors in in the you know in the space And basically by this you learn. Oh, I have something over there and there Does it And if you if you're interested in basically in the equations and I'm happy to talk with you. Thank you