 Hello, okay. Hello everyone. So I'm David and I'm a PhD student working with Gabor Chani in Cambridge in the engineering department. And so today, we will actually give a joint talk with Ilias within the audience so we'll join our talks. And we'll talk about a work that we have been doing over the past year or so and it will be about generalized atomic cluster expansion. Okay, so let's see what I'll talk about. Basically, in the past years, there were two big advances in the field of like machine learning for materials, and one of them was atomic cluster expansion. Boris mentioned the other one is probably a necklace. Yeah, share the screen. Yes. Okay. So, so basically, all the motivation for this work that we started about a year ago was to look at this atomic cluster expansion and, you know, which was a basically a huge advance because it really put all of the in-base representations on the same footing. And then there was this message passing an equity and message passing that works like an act we've, which achieved, you know, even better accuracy than the previous item centered representation is based models. So we created this framework called multi and this really builds on on the ideas that have been around for a long time so basically Marie she condor has been thinking about it so that became a child as well, they have a recent preprint. And the idea is that how do we kind of unify or how do we connect these atom density-based representations and message-passing networks. And so what we will present today is a very careful formulation, which we believe really helps highlight the key connections and helps us understand, for example, what makes NACRIP as good as it is. And so in particular, I will, in the first half of the talk, talk about something that I call here generalized equivariant ACE. So ACE is the atomic cluster expansion. And I'll just show a few results as well, what a single ACE layer can do. Okay, so this will be very brief because I think everybody in the audience kind of has seen this, but we want to represent atomic structures on computers. And these structures are usually made up of particles. And the particles have two types of things. So one of them is the position. So they are embedded in three-dimensional space. And the other one is something that I call here attributes, which is, say, a chemical element. It can be a charge or any other fixed property that defines the system. And then we want to do some machine learning on this structure. And for that to do machine learning, we need to represent the atoms on the computer. And obviously, this kind of XYZ representation is not going to be the best we can do. So the question is how do we transform this into something that's more suitable for machine learning? And then we want to predict something, for example, the potential energy, most of them. So what are the symmetries of representation that we need? And so here I just want to, before or else, introduce some notation. So whenever you see sigma, sigma will be basically this configuration, which describes the state of an atom. So an atom will have a position. And then it will have these attributes, which are, for example, what is the chemical element. And then there can be also some learnable features. So you know, in that field, there were some learnable features which were maybe vectors or tensors. So the usual target properties will have some symmetries. So, you know, potential energy will be invariant, binding affinity will be again invariant, something like a dipole moment can be a vector. So but anything that has a symmetry will obey some kind of symmetry relationship. So say you have a model phi and you apply a transformation Q on your inputs, and you make a prediction. And then if your model is equivariant, that means that you're, you could have actually made the prediction on the original system and then apply the transformation. So if your model is invariant, then transforming the coordinates and then computing the energy is the same as computing the energy and then transforming. If it's equivariant, so if it's something like a vector, then if I predict this vector on the transformed, that is the same as predicting the vector on the original and then doing the transformation. So this is kind of an equivarian. And if you want to represent equivarian properties, then we need to choose a basis for it. And the basis that we'll talk about or that we use for this small case framework is the spherical representation of the tensors. So basically, kind of like the atomic orbitals, so L equals zero is this spherical, and then L equals one is going to be these kind of p orbital like, and this is important because the L equals one spherical vectors are the same as the Cartesian vectors. So, so for example, other some neural networks like pain use Cartesian vectors, but they are just up to change of basis equivalent to using sphericals. And the M just indexes these. And then the model predictions should be consistent with the symmetries by design. So we want to create models that have these symmetries built in. Okay, and then the symmetries will be translation, rotation and permutation, as Boris said, and probably all the other speakers before. Okay, so what do you what what is generalized atomic cluster expansion. So first, before going to start the cluster expansion is this framework that of symmetric polynomials that was introduced in sort of 2019 by Ralph droughts. And it builds a complete polynomial basis for atomic environments. And the most basic building block is something called the one particle basis. And that has this kind of form. Okay, so what what is what is this, it has a radial part, which takes the argument the distance between the atoms. And then it has an angular part, which takes these unit vectors on the sphere as arguments. And this will represent all of the angular information about the environment. And so rji is our central atom, and j is one neighbor. So this is in some sense an edge feature. Right, it is a feature that is connected to the rji pair. So what we want to generalize this because this as it is wouldn't really fit in with something that like a neural network. So we came up with a generalized one particle basis. This is going to be a bit tricky. So they just one particle basis will still keep the radial part, okay, that's on the distances. It still keeps the angular part. But then it also has this T, which is a very peculiar object. And T basically is everything that is not a radial and angular. And that will describe the dependence on, for example, the chemical elements. Okay, and this allows us to generalize this one particle basis from the having species indices to, for example, continuous embeddings of neural networks. Okay, and then we have two indices here of T, the K and the C. And this will be something that becomes very clear later. But basically the key is that one of them, we have this degree of freedom of collecting different indices into one or into one or either K or V and you will see in two slides time what the point of that is. But it's just something we can do in the theory. And just to sort of give some examples of what TKC can be. So if you want to recover the previous one particle basis with the species index, then we can encode the neighbor chemical elements into this TKC. Okay, so we want to recover this type of functional form. Then we can say that, okay, TKC should be essentially an index selector. So if theta i is z i, so if theta i is the i th atomic number, then and it will select basically this TKC will select chemical element z i and chemical element z j and then kind of pick the corresponding radio basis. Okay, so here the radio basis depends on what is the carbon hydrogen radio base, the carbon carbon radio basis. And then here it is agnostic, but it has the index C and that index C will now correspond to picking the TKC will pick the correct carbon hydrogen radio basis, for example. Okay, and if we have channels with chemical embeddings, then again that can be expressed. So this TKC could map, for example, the chemical elements into some continuous space as well, like the neural networks do. And then this TKC could also introduce other dependencies. So for example, there is a magnetic moment or a dipole on the on the neighbor atom, then that could also be incorporated into this framework via these TKC functions. And this also doesn't have to be invariant. So it can have an aquarium output. And then that would be expressed in a spherical basis. And then C or K would, for example, collect the LM indices of the symmetries of this output of this TKC. Okay. So moving on, what can we do? So this one by the basis is essentially an edge feature. So now we can collect it. So we collect by a kind of like a message passing away into into this a basis. So this a basis is just summing over the neighbors, these edge features. Okay. So it's basically like a message passing network, but also like a so this is a notation. So a is the atomic base sort of permutation invariant basis set in an atomic cluster expansion. So all we have done is we summed over the neighbors, this one particular basis. And then we will create higher order basis functions. So why do we do that? Basically, it is known to be a systematic approximation of, for example, a function of an atomic environment. It can be systematically expanded in this many body expansion. Okay. So many body expansion is essentially taking a property of atom I and saying that it depends on its own, what atom I is, and it depends on terms that sum over atom I and one neighbor, atom I and two neighbors and so on. So this one body, two body, three body and so on. And we can parametrize each of these terms with these a basis functions. Okay. So if you look at it, we have a shift, a one body energy, and then we can have two body energy. So this a, I was basically a sum over neighbors of two body things. So that that can parametrize any two body functions. We have studied the complete basis. And then if you, if you take products, then what you get is basically three body functions, right? Because you had the I and then one neighbor, and then the other eight depends on I and then another neighbor. So if you product them together, then you get these three body terms. And this actually is very similar to Allegro, right? Because in Allegro, you also had edge features and then you product, tensor product them together. And we do the same here with the violums, it's essentially the same kind of tensor product. And you can see that we only take the tensor product over the V indices. So that's why we had two indices K and V, because we can have the freedom of mixing some channels and not mixing others. And then so he created this product basis ball A, which is now a many body basis function. And it can, we can go to arbitrary high body order at very low cost this way. So without having to actually sum over all triplets and quadruplets and so on. So now all we have left is to actually do something with rotational symmetry, which we have ignored so far. But we haven't quite ignored it obviously. So what we then do is we have to symmetrize these A basis functions so that we get, we pick out the correct equivariance that we want as the output. And basically, you don't need to worry about the maths. What we do is we have to take an appropriate average of the spherical harmonics in the rotations. And the key is that we did use the this violums to parametrize all of the angular dependence. So then we can use the spherical harmonic algebra, you know, from angular momentum, like big Russian books, and then we can look up what is the correct kind of coefficients, like generalized clubs, Gordon coefficients, that, you know, if you sum these these products of violums, you're using the appropriate coefficients, then you can get out, for example, a big L big M symmetry features. Okay. And then here. So basically, what I am showing here is that how we can create from these many body A basis functions with the appropriate coefficients arbitrarily symmetric many body functions. Okay. And that is the that is kind of like a generalized ace feature. So basically, we get an output by by taking these features and applying some weights to them. And if you wanted to reduce this to, you know, original drought linear ace, then we would need to take K equals one. So you have only one, there is no uncoupled channel, you take the product over everything, and then L equals zero and equals zero would be the invariant, the energy, for example. So now I think that was the theory. And so from here on, I will just show a few results of invariant and equivariant ace as a single generalized one ace layer. And then Ilias will continue by showing how you can connect this to to the message passing networks. So what we have done first is okay, we have now with this many body basis functions of ace and they are super efficient to compute. So the question is can we actually just do linear regression on them without having to go to neural networks or kernels. And what we have found is that what you can see here is that in something as easy as MD 17, even with linear regression, you can be kind of on par with most of the previous methods, including some message passing neural networks, even some L equals one equivariant neural networks. But then you can't quite be on par with something like NACRIP. So clearly there is something missing in simple linear ace from the graph neural network from NACRIP. And maybe Ilias will maybe talk more about how to go from ace to quite there. So then just a few other illustrations is this data set that Boris also mentioned. You can see that if you train the potential energy of this molecule at 300 calving configs only, then you only take certain stable combinations of angles in your training set. And then when you test at high temperature, there is a much broader distribution. So this is a very good kind of challenging benchmark data set. And you can see that ace does really well compared to other methods. And then we can also feed equivariant property. So I just want to show here, this is percentage error on negative neutral and positive water clusters. And now the dipole moment is an equivariant rotating quantity. And again, just with a linear ace, we can get a very, very accurate dipole. So the neutral ones have a bit higher fraction error. But that corresponds to a much smaller dipoles in absolute value, whereas the negative and positive clusters have much higher errors. So basically in absolute terms, it's like 0.01 divide or smaller errors that we get using single linear regression with this equivariant ace. And with that, I just wanted to show one more thing that we can do use these dipoles, for example, as the long range part of a potential, plug them into the, you know, Coulomb blow of dipoles, and then parametrize an ace for the short range part of a residual. And then we can run reactive molecular dynamics of, for example, of water clusters using proper short range and long range included. So here you can see like a proton kind of migrating around in a cluster. So to conclude, atomic cluster expansion is this very systematic framework to create efficient symmetric representations of high body order. And then we can generalize this and it can be invariant, but also equivariant. And we can also include continuous embeddings into atomic cluster expansion. And so we've, most of this work is actually in a preprint that we will upload later this week online. And this was a work jointly done with the group of Boris, especially investigating the connections to Necrip. And I would like to thank everyone. And so we'll, just to point out next Wednesday, there will be a much longer talk, like an hour and a half that we will give with Ilias on this. And so I think I'll just give the floor to him. And maybe we can take questions together. Thank you. Thank you very much. And now floor to Ilias for the second part of the joint talk. And as David said, the questions will be taken at the end of all of this. So in 10, 20, 15 minutes.