 Good morning everybody, I hope that you covered from yesterday's pizza and that you are here not only physically, but also mentally. Today we start a new series of lectures on non-discriminate disorder systems by Federico Ricci Terseghi who is a world-renowned expert in this topic and also in several interdisciplinary applications of what he's going to say. So applications of physics, I mean what he's going to tell you about the physics of this system to many things, to satisfy these family problems, to biological networks and many other things. So this is the reason why we are particularly pleased that he accepted to give this lecture. So we give a two-hour lecture, then a break, then another two-hour lecture, which is also some intermediate break. After his lectures, so we should more or less end up around one, we will have the group photo. So we will take the picture of all of us. So please come and do two things properly because this will stay true. So then there are no other announcements for the moment, so Federico please. Thanks Andrea. Okay, so good morning to everybody and sorry for my late arrival, but okay. I had problems at the beginning of the week. I recovered and so finally I'm here with you. So I thanks also Matteo Marci that gave the first two lessons and so it taught you some basic statistical mechanics and the revised models, which is always very good to know and my lecture will use those contents. So what is the aim of this series of lectures? I would like to give you a wide overview on disorder systems and glassy models. Both from the equilibrium point of view, so standard statistical mechanics, thermodynamics and some dynamical properties. So what these disordered models, glassy models do when you put these models out of equilibrium? For example, because you change the parameters of the model, the environment, so the temperature or the external field. And so the system starts to evolve in a complex landscape and the aim will be to give you some flavor of what's happening and trying to make to the best of our knowledge a connection between thermodynamics and other equilibrium dynamics. So I will try to convince you that in some models we do have a quite good understanding of what is the final outcome of the dynamics and how can we relate this to some thermodynamic properties. So you can do a standard statistical mechanics computation and guess what will be the very long time behavior of a system put out of equilibrium which is actually definitely not trivial. Indeed, at the same time I will try to show you that there are many situations where this connection is not still understood and so there are still many, many open questions that I will try to point out also for your interest eventually in the future to study these problems. And so the final part of this lecture I will focus mostly on application of the tools like replica method and cavity method that I will try to explain you during the lectures to some problems coming from mainly theoretical computer science, optimization problems, constraint satisfaction problems. And I will try to convince you that the tools that hopefully you learn from these lectures can be very useful also when you apply to completely different problems that are priori they shouldn't be described by statistical mechanics. In a sense statistical mechanics has been invented for describing a large number of atom spins that you find in real materials while these other problems have been invented by humans and have a completely different origin. Still the fact that these other problems are well described by a system of a very large number of variables interacting variables. This allows you to use essentially the same tools and to reach an understanding which is definitely much deeper than what was known before. And so I will try to convince you that phase transitions also in these other completely different class of models of problems is helpful and can provide you an insight of what will actually happen when you try to solve this problem by finding solutions or try to sample configuration in a given model or try to do inference in a noisy data set. So there are different problems that at the end they fall in the same category of system of a large number of interacting variables for which you can use statistical mechanics tools but not the standard one. Those that we developed in the last decades for disordered models because usually in real problems you have noise so life is not so nice as the Q device model is much more noisy. And so we have to learn how to deal with this disorder. Okay, so the subject is very good and unfortunately I lost the first two lessons so I hope I can keep everything in the remaining lectures. Obviously I'm here for you so if you don't understand something ask, so don't be afraid. I'm here to teach you and so if there is something you don't understand don't be shy and ask. I haven't written lecture notes first of all because I'm very slow in writing and also because very smart colleagues already did a very good job in the past so I don't think it's very useful to redo the same job eventually coping part of something which has been already written. So I will mainly follow some already written lecture notes by Francesco Zamponi for the first part of the course or for the first part of the lectures and eventually for the second part I will give you some more material. So just to be concrete let me immediately write so the lecture notes by Francesco Zamponi that we mainly follow in this first part of the course you can download it from archive so this is something that is worth having at hand and also for the spherical PSP model which will be the main subject of the first part of the course because it's the model that we solved better the other two materials that you can always free download from archive which are very useful and I will follow in preparing my lecture notes and one is again lecture notes by Cavagna which has been then written so these are Cavagna lecture notes actually written in collaboration with Castellani Andrea was lecturing these are for the spherical PSP model and you can find always on archive sorry this is old style con mat 05, 05, 032 and always on the spherical PSP model if you want to have a quick summary this is something that most people in the community when you don't remember some numbers about the PSP model you want to have it at hand there's a very short notes which contains everything by Alembarra so always on the spherical PSP model you have some quick but useful notes by Alembarra that you can find always on archive 97, 01, 051 so these are the, with this material we will do most of the first part of the course of the lectures and then when we move to the application to optimization problem constraint satisfaction problem I will give you more material even in the lecture notes of Francesco there is a general introduction also to that part but eventually I will give you more material depending on how we go with the schedule because that is the last part of the course I'm still unable to foresee when we arrive there this is the first time I give this lecture so I have to adapt okay so let's start so what do we mean by disorder systems so which disorder system we are interested in the first let me say the prototypical problem I have in mind are spin glasses what I mean by spin glass I mean a model of spins so I start always from the Hamiltonian a model is completely determined by an Hamiltonian I will always use the canonical ensemble so for me studying a statistical mechanics model is studying a model defined by an Hamiltonian eventually disordered Hamiltonian in the canonical ensemble and trying to understand the thermo-nomical properties of this model and spin glasses are model of interacting spins where the sum may run over all pairs of spins if you want or the red just not to repeat twice the same interaction where the couplings are in some sense disordered and we'll discuss in more detail how why and what is the kind of disorder in the couplings we are interested in so this kind of model is the most important disorder model for similar reason even for an historical reason is the first one introduced by Edwards and Anderson in 75 and so they introduced the finite dimensional version so the model defined on a three dimensional cubic lattice so the couplings were non-zero only between cubic lattice and the aim was to describe a real material so spin glasses do exist in nature it's not really an invented model as we do many times when we want to study something exotic we invent a model, no, these materials do exist they have many different realizations of spin glasses but the most common, at least the one that inspired the Edwards and Anderson model at that time are metallic alloys with magnetic impurities so you take a metallic alloy with magnetic impurities and these impurities interact with an interaction called R-K-K-Y interaction don't ask me the names of these four guys, it's very difficult that is oscillating in space so the interaction let me say the coupling as a function of the distance goes like the cosine of 2 KF R divided by R to the cube so it decays but has oscillation where KF is the Fermi way better and so depending on the distance the interaction may be positive or negative and so when you produce the alloy and the magnetic impurities take position in a random position but then fix position you have one spin glass sample so one spin glass sample is one, actually if you look there of this size typically one magnetic alloy where the magnetic impurities are fixed and so the couplings, this one are quench in the metallurgic sense so they are quench, they are blocked fixed for that sample so having one sample of a spin glass means having a system of interacting spins the couplings are random but fixed once for all and you won't understand the behaviour of the system so why these models raise so much interest such that we are still studying these models after 40 years now, 42 years well this is because essentially this randomness and actually it's not really the randomness required but I will show you that the randomness produces essentially the outcome of this quench disorder is to have many states where I put this in between quotes in the sense that I will try to make more precise in the following what I mean by a state especially because in finite dimensional system defining that state is still an open problem we are still discussing how to define states in disordered system in finite dimension so this is why after introducing real spin glasses we moved to the mean field version where states can be well defined so in the mean field version we removed the quotes so the guru having so many states is that one of the main consequences of having so many states is to have very slow dynamics I will try to convince you by solving some kind of dynamics but anyhow it should be I don't want to say obvious but it should be quite natural that if you have many ways you can order the system so you have many states so you have many local orders you can easily create a situation where the system gets trapped in a configuration where different parts of the system have different kind of orders and then moving the system adapting the order in order to decrease the energy as you do in a standard relaxation dynamics may become very difficult so this is why many states is likely to produce a slow dynamics and in particular there are some systems where this phenomenon of the slow dynamics is so strong that you may have an increase of the relaxation time which is really very very fast these are called glassy forming liquids so the slow dynamics is a really characteristic feature of some system called glass forming liquids which are systems where essentially you have as a function of the temperature the viscosity which is a measure of how slow is the system diverges in a very abrupt way so they become extremely slow and indeed I will try to convince you that some kind of spin glasses they had a good model for this glassy forming liquids so what we are usually called just to make the contrast with spin glasses we call these structural glasses so if you heard the word structural glasses I mean a system which doesn't have a quench disorder but the disorder is self-induced and you may have a drastic evidence of this drawing of the dynamics by measuring the viscosity of any relaxation time in the system so this is a very interesting field of application of the techniques for solving disorder systems that I will illustrate to you but the tools from spin glasses that we are going to discuss which are mainly replica method, cavity method well the complexity which is a measure of how many states you have I will define all these concepts but all these tools and concepts let me say which we developed in the spin glasses in the last 40 years they have very interesting they can be applied also to completely different contexts like indeed constraint satisfaction problems optimization problems constraint satisfaction problems or even to some other well inference problems also I know that Floran Giacola gave a very nice lecture last year if I remember correctly about inference problems but also in... so these are all let me say human invented problems and you can find very useful the application of these tools to biological problems like for example neural networks neural networks is a typical example where ideas, concepts and tools from spin glasses they found a very nice application and obviously neural networks are not, let me say really a real biological data this is a model for describing data and also the very recent for example deep networks in artificial intelligence we hope that is... we hope to understand more why these deep networks work thanks to these tools at the moment if you look I'm an outsider in that field, I have to admit but if you look from outside you have the impression that everything is work because these are very smart people and they use a huge technological power and they have some key ideas that makes this neural network, deep neural network work but I think that there is a lot of space and a lot of work to do from the principal point of view so try to really understand the key feature of these models of these systems that they in practice work and I think that again all the tools and the concept that we develop in spin glasses may be very useful if you want to really deep understand deep networks deep neural networks and finally let me also comment that many of the results that has been derived by statistical mechanics of disorder system has been proved rigorously by mathematicians this has been really a big challenge for mathematicians essentially they have been able to prove important result in fully connected models I will define now different models and now they are proving important result also for diluted models essentially for the moment they are reproducing in a rigorous way all the result that you can derive very easily by standard tools from statistical mechanics of disorder system but this is a good news both for physicists and for mathematicians because mathematicians means they are able to prove rigorous theorems on more and more complex systems which is definitely non-trivial and for us it's a good news that what we derive with our tools is true, is rigorously true because we typically base our result on some assumptions that are very reasonable and so we use them but it's not really a proof so when someone say no, what you prove is proven I'm much, much happier okay, I think I will leave the definition of concentration problem from when we will when we will discuss it should you sign this? okay, so let me now yes sorry of this, no this is just an example of a completely different, no somehow different system so-called glass forming liquids these are liquids that when you cool them down they have a so this log of eta, so this is a very drastic increase of viscosity and these systems are still very much studied for many reasons they are interesting also from an experimental point of view but they really have some cooperative phenomena which is incredible in the sense that in a tiny temperature range they may increase the viscosity by, say, 10 order of magnitude so something really is a cooperative phenomenon but incredibly strongly cooperative phenomenon and the meaning of this I will try to be much more precise in the following but just to say that the simplest sunglasses can describe also this phenomenon so I will show you a situation where you can understand this structural arrest so this extreme slowing down by means of spin models, okay these are not spin models, these are liquids particles interacting in a more and more dense way or at lower and lower temperature you can reproduce this phenomenon also in spin models of the kind that we are going to we are going to study so this just to give you a wide overview of a different situation where the models that we are going to study are interesting, so just to make you interested in this kind of models okay so let me come back to the to the simplest spin glass model whose Hamiltonian is this one I will call it simple because then we will make some generalization of this model I put a minus sign here just to put a minus sign which for random cappings would not be really needed but if you then take the ferromagnetic version, so the Curie-Weiss model where all the cappings are positive you need to have a minus sign here otherwise it is not really the Curie-Weiss model and the simplest spin glass model for me is variables s i are one-dimensional variables being either is invariable so this plus or minus one is invariable or eventually we are going to use spherical variables by spherical variables I mean variables s i taking values in on the reals with the constraint that the sum over i of s i square is equal to n the number of spins, the number of variables so these are the two kind of variables that we are going to use but it is worth saying in here that we are restricting to the simplest case because already the simplest case is very interesting and show a very complex behavior but you can obviously generalize these two for example spins belonging to having m components and so these are usually called OM models you can fix the norm just to these are called OM models and in particular for m equal to you have the so-called x y model and in the components model you have Heisenberg variables so these are called vector spin glasses these are usually easing spin glasses these are vector spin glass because the variables now are vectors either in with two components on a circle or three components on a sphere and so just to say that we are not going to discuss these models you first have to understand the simplest spin glass before moving to the more complicated one and also because actually these models have been studied very few well, especially because we were already the simplest spin glass took so much time to understand that we are moving now to these vector spin glasses they show a richer behavior but in these lecture notes I will not deal with this with this generalization let me discuss now the couplings, ok, so how do we take the couplings? this is the quench disorder so in these couplings I'm also essentially the couplings also determines the topology of the interacting network what I mean by this I mean that couplings may be different from zero so couplings the couplings which are different from zero they form the interacting networks so here the sum is over all possible couplings so if you set all the couplings different from zero you have what we call the fully connected network all variables interact with all variables so you may have, for example fully connected networks fully connected models if j and j are different from zero for any pairs i and j so you have to figure out models where all the variables this n equals 5 all variables interact with all variables in this case obviously you have to scale the couplings with system size otherwise you don't get a good thermodynamic limit because now each variable feels a bias field so a local field which is the sum over all the variables but the sum is over a number of variables which is diverging and if you don't scale the couplings with an inverse power of n you don't get a good thermodynamic limit so in general what you have to do you have to take couplings in the fully connected models that scale like the mean value that will be of order 1 over n by the over bar i mean average over the disorder distribution so these are quench disorder eventually we generate, I will discuss this in a while we generate different samples each sample has a given realization of couplings and so you can take the average over the ensemble of couplings over the couplings distribution in a good thermodynamic limit you need the mean value of the couplings to scale as 1 over n also if the couplings are random so they take both positive and negative values and the mean value is zero in that case you need that the square of the couplings must be of order 1 over n so this is what we have seen in the critical device model in the critical device model all couplings are the same intensity which is j and j is 1 over n otherwise you don't get a sensible limit in the thermodynamic limit if you take couplings which are both positive and negative so the mean value of the coupling is zero then you have to look at the second moment and the second moment must scale in this way so each coupling here means that each coupling is ordered 1 over square root of n so it goes to zero but much slower than in the critical device model so the fully connected models are those that we are going to study first because they are the simplest to solve the second topology we are interested in is what are co-diluted models or actually models defined on a random graph you find different names in the literature usually they are called diluted models or models defined on a random graph did someone ever see random graphs how many of you have studied random graphs okay some eventually we will move to the diluted case I will make you a reminder of what a random graph is but for the moment let me just say that in these models each variable is connected to a number of so this variable si is connected to a number di which is called the degree the degree of the interacting graph which is of order one so while in the fully connected models each variable interacts with all other variables so here the degree is trivial is m-1 each one interacts with all the other variables here you want to build the interacting graph such that each variable interacts only with a finite number of variables but in order to be able to solve the models you want to choose your neighbors at random because if you choose in a particular regular way like on a tridimensional cubic lattice each spin interacts with six other spins but they are chosen with a very specific rule in that case you are not able to do analytic computation while in this case you can do analytic computation if the neighbors are chosen at random still the fact that your degree is finite is not diverging with n you can take couplings over the one so this imply that your couplings can be well let me write the in this formalism the couplings are the vast majority are zero so the couplings are zero with a probability I choose alpha over n and a number which is of order one now with probability alpha over n so you see I can still use the same Hamiltonian but now the topology is completely different in the sense that yes couplings are random and the distribution of the couplings such that you have a delta in zero which holds all the weight so almost all the couplings are zero this is why you produce a diluted model the few which are non-zero and there are few because these scales like the probability scales like one over n so this probably goes to zero in the thermodynamic limit the very few which are non-zero then you can assign a a coupling intensity which is of order one and get a good thermodynamic limit so you see these two models are quite different but still what is good is that you will see that okay these models are more difficult to solve are those that we are going to solve in the second part of the lecture but still you can solve and for many aspects they are similar to the fully connected version so we have in mind the solution of the fully connected version which is easier to get and essentially we try to see how many and which of the features that we get from the solution of the fully connected model they also appear in a diluted model and finally you have the final dimensional version final this is the dimensional okay I call it final dimensional because these two models are usually termed infinite dimensional models it's not completely clear that this topology corresponds to a final dimensional say hyper cubic lattice when you send the dimension of the embedding space to infinity still you can prove so there are ways you can take an hyper cubic lattice make it diluted such that it keeps say always four neighbors then you send the dimension of the equilibrium space where you embed the hyper cubic lattice to infinity and you can prove that many many properties of that diluted hyper cubic hyper cubic lattice in the dimension converts to the random graph so it's not completely obvious but these models they do corresponds to the infinite dimensional version of final dimensional models so this version of the final dimensional model you just have the the only couplings which are different from zero are those so the only which are different from zero are those where i and j are at a euclidean distance one essentially you put a non-zero coupling only between nearest neighbor on any so you embed the spins in a finite dimensional space you arrange them in the way you like more a cubic lattice on a com lattice triangular lattice and then you put the couplings only between first or eventually second neighbors so these are the finite dimensional models yes but you can prove that in order to embed that you need a dimension of the space that scale within so in the thermodynamic limit there is no finite dimensional space where you can embed the random graph such that all the interacting spins are at a distance say or order one, not scaling within so you need really an infinite dimensional space in order to embed that graph okay so these are the the three topologies that I will discuss we will start with the fully connected models because as I was saying for these models you can actually do analytics so let me say pro and cons of these three categories of topologies there are also intermediate categories like long range model where you can bury the interacting range in order to move from finite dimensional to infinite dimensional models but okay this will be another story will be much too much complicated so let me keep to these three well defined interacting topologies so what are the pros and cons so here we can do everything analytically but they are really the most unrealistic models the interactions scale within this doesn't happen in nature in nature the interaction between two variables may depend on the distance typically but not scaling with the number of particles that are in your experiment so if you take any measure in this room or in a smaller room the interacting strength between two particles is the same doesn't depend on the size of the room you are taking the measure so these are the most unrealistic models so we are happy we can solve to move in that direction which is the most realistic one moreover another problem of fully connected models that if you do numerical simulation they are very very heavy why? because essentially in any numerical simulation the cost of the numerical simulation even only computing the energy so you want to compute the energy of a configuration the cost of the numerical simulation is proportional to the number of terms that enter in the Hamiltonian and if you have a sum of all pairs the number of terms in the Hamiltonian goes like n squared so in this model if you want to do a numerical simulation the cost let me write here the cost of a numerical simulation of a Monte Carlo simulation is of order n squared so it is very large so these are good for analytics bad for numerics and also very unrealistic these are extreme cases very realistic if you are able to solve this model maybe you get the Nobel Prize if you solve this model nobody give you the Nobel Prize because it is much more realistic the problem is that you cannot do analytics on this model we can solve only few very low dimensional models and with no disorder essentially so when you put this order these models you have to do numerics essentially all what we know from finite dimensional models is from very large scale numerical simulation and at the moment we have some special purpose computer built just for studying spin glasses and just to give you an idea of which kind of system we can simulate we can in finite dimensional version of this model so we can simulate we can measure the thermodynamics of systems up to sizes 32 to the cube which if you compare with the fermaticism model is ridiculously small this because when you do we still have to discuss how to do the average over the disorder you typically need to simulate many samples and because of the slow dynamics each sample maybe takes say one week to thermalize and so the Monte Carlo simulation of these models is really painful and so okay we do it because it's our job but if I can find an intermediate regime where everything works better I would be very happy and this is the situation like more is the one where I mostly work in the last decades because on one side thanks to the fact that you are on a random graph I will show you you can write analytic expression still they are more complete than in fully connected models I'll say you need computer even for solving the analytic equation okay so is an analytic solution computer aided but if you do Monte Carlo simulation here the Monte Carlo cost is linear why well because most of the interaction are new so the number of non-zero interaction in this Hamiltonian is over the n each variable interacts only with a finite number of variables so I think that this intermediate regime is the is the best situation if you want to compare analytics with numerics because you can run numerics in a linear time which is definitely mandatory otherwise you cannot go to large sizes and at the same time you can get some result from the cavity method that I will explain you okay so these are three very different topologies and depending on what you want to do you can choose one of the other so we start from fully connected models then we move to diluted models moreover let me say that these diluted models are those that naturally arise from human application like constraint satisfaction optimization problems is typical of this type even when you study a biological network is really unlikely that in nature one entity interacts with all the others there is no physical space so the best that you can have is you interact with six and in order to solve them you say six randomly chosen not six accurately chosen like in a regular topology and this is the best you can do in order to be close to real situation okay yes good question here I'm suppose that you generate one non-graph of degree six by chance you can generate even a three-dimensional cubic lattice but that is exponentially rare actually so we are interested in typical realization of the disorder I will discuss now how to take the average over the disorder which is the average over the couplings which in practice means also the average over the topology of real realization so if you generate a random graph it's true that a finite-dimensional graph belongs to that ensemble but with a probability it's so small that you are not interested in anyhow it's true that when you generate the random graph you can ask what is the degree distribution for example one can generate what are called Erdos-Reyney random graphs we will discuss this a little bit more in detail in the next lecture where each link is independent and the degree distribution so the probability distribution of this quantity is Poisson but you can also choose to have what is called random regular graph which is the the random so it is the graph with the largest entropy in the ensemble of graphs where each variable has the same degree generate this kind of graph is not so easy because you say each variable must have degree two is easy loops you take loops and loops each variable has degree two but we already take degree three think exercise think a way of generating a random graph where each variable has degree three it's a good exercise to think about just to understand that even sampling an ensemble of graphs is not so easy if you put strong constraint and having all the vertices with the same degree is a strong constraint so you are trying to sample from an ensemble graph with a strong constraint but and these are still all graphs that behaves more or less the same but you can also take distribution of the degrees which have a completely different physical behavior for example you can take a distribution of the degrees which decays as a power law distributed what happening all those networks like all those networks that are much more studied in the what is called network theory where people is much more interested in networks where the degree distribution is very very broad so both erdos when in random graphs and obviously random regular graphs have a very narrow degree distribution so degrees are either all the same in the random regular or very small so they are very concentrated in the erdos when but you can take the distribution where the degree is distributed with a very long tail and these produce a completely different physical they may have a new physics in that I will be mostly interested in random graphs where the degree is not only one but is also concentrated so I don't want to enter the realm of network theory where they treat these other models and there is gone I don't know if we want to make a five minutes break now or do we want to make a five minutes break now and then since this first lecture is very penny okay so five minutes break but five minutes otherwise then we take a longer break at 11 so let me do the case with p equal 3 which is simple to understand is the sum of i, j, k ordered i, j, k of j, i, j, k s, i, s, j, s, k so it's a multi spin interacting model and again you can consider different topologies for this model the fully connected topology we take all the caplings different from zero so all j are different from zero all j are different from zero but now you have to scale them let me consider only models where caplings are zero mean so we just are interested in the variance in order to get a good thermodynamic limit you need to have the variance that scales like the usual way we put this p factorial over 2 and p minus 1 this is the standard normalization for the caplings so you see for p equal 2 you recover the 1 over n that we already saw in the simple spin glass but for p interactions is the number of terms now is n to the p when you take the sum of the squares because these are random numbers you want to sum of the square each term must be n to the p minus 1 such that you multiply by the number of term which is n to the p you get n so the amiltonian is extensive is over the n again you can take the diluted the random graph version in this case the capling will be mostly zero and those which are non-zero over the 1 with probability that now must scale like alpha over n p minus 1 again you have n to the p triplets p people and you want to keep order n and so you want to keep a fraction 1 over n to the p minus 1 and zero otherwise finite dimension version we don't know essentially there is still no model these are no problem so mark as open problem open problem I will mark as open problem something that is still we don't know how to do to find a finite dimensional version of these mean field models that show the same feature because ok taking a finite dimensional version where spins interacting is easy you take a triangular lattice and you make all triplets of spins belonging to a triangle interact but that model doesn't have the physics of the mean field version so find a finite dimensional model that as the visitor I will show you in the mean field version of this p spin model is still an open problem maybe it doesn't exist but even proving that it doesn't exist is an achievement so why this class of models for p larger than 2 is strongly different from the simple spin class model I showed you before so the p equal to k again here spins can be easy spins spherical spins as before essentially because when you consider the local field on a variable the quantity that in particular enters when you solve the dynamics now I just catch something that we will discuss in more detail but if you take Langevin dynamics for these spin variables you have something like the derivative respect to time is minus the derivative the Hamiltonian plus some noise this noise okay say Gaussian noise or whatever I'm not interested in this part I'm interested in this part now this part here is something which goes like the sum of say some indices let me do the example for for well of many sigma how many p minus 1 sigma okay let me just keep everything very schematic so what this means is that this term for p equal to the simple model is linear for p larger than 2 is nonlinear so this term is for p equal to is a linear term for p larger than 2 and this makes a huge difference essentially what you can prove I will show you next because we are going to solve these Langevin dynamics for this p spin model with spherical variables and I will show you that for p equal to the model undergoes a dynamical transition so the system doesn't relax to 0 so the correlation between very between two times very far apart in time doesn't go to 0 at the same point where you have a static transition so let me say that in this linear case the dynamical transition where the system doesn't relax anymore to 0 coincide with the static transition that we are going to compute while in this case for nonlinear interaction what you will find is that the dynamic transition is higher than the static transition and the kind of the kind of dynamical transition in this case is what resembles the phenomenology of glass forming liquids so p spin models are a very good and simple to study this is the magic models of glass forming liquids because they show features similar to what is called the not coupling theory approximation to glass forming liquids so an approximation which is considered a good approximation for those models essentially is the temperature where the correlation doesn't decay anymore to 0 so you keep memory of the initial configuration all times so essentially is the ergodicity breaking so below td there is ergodicity breaking you don't forget your initial configuration and what is interesting in this model is that it happens before the thermodynamic transition so while in principle you don't have any singularity in the free energy there is no singularity in the free energy the free energy is mostly if you are very good in solving thermodynamics you would say at this point nothing happens no, there is a dynamical transition so we are going to try to understand why because this is really relevant for all the lecture and for all the thesis that we are going to study since we are going to ok, so let me just say that even this model can be generalized to including many kind of interactions I don't think we are going to discuss in very much detail but since I will point out some open problems in models of this kind where you use more than one kind of interaction so maybe p interaction and s interaction maybe three interaction and three body interaction and four body interaction at the same time let me just say that this model can be easily generalized in a form that you just sum over many possible type p interaction and so a further generalization of this model is this one you take the sum over all possible p values let me say larger than one actually you can put also the p equal one interaction which is the external field but for simplicity we don't put p larger than one then you have a coefficient cp and then here you put this Hamiltonian and since I am considering only couplings of zero mean so symmetric couplings let me forget about the minus sign so we put essentially all this term here so the sum of i1 smaller than ip j i1 ip s i1 s ip so while this is called the pure model this is usually called the mix model for obvious reason so pure model is just one kind of interaction mix model more than one kind of interaction and a convenient way of keeping all this coefficient just to understand very quickly what is the Hamiltonian is the following the reason why I am doing this is that because for spherical variables we are going to solve the dynamics for the mix model because I will show you the solution of the dynamics for the mix model and even the thermodynamic solution you can work it out for the mix model very easily so obviously in this case again couplings of type p must scale in the same way but with a different power of n so the variance of each kind of coupling scales with a different power of n and in order to keep track of all this coefficient you can just look at the average between the two Hamiltonian having the same couplings average over the couplings and this since couplings are random is quite easy to derive I leave you as an exercise that is equal to one half which is this one half here and then the sum over p of cp square because we are taking the square and in the two summation when you take the average only the terms where the same j appears have a non-zero value and so here you have q to the p where q is defined as one over n sigma i tau i and we call this f of q so essentially in one function f of q we have all the information about the Hamiltonian so when we work with mixed p-speed models I can say that I am working with a mixed p-speed model having this function f of q and what I mean is a mixed p-speed model with these terms and the coefficient that can be obtained from the Taylor series expansion of f of q for a pure model obviously cp is equal to one just for one value and so f of q is just one half q to the p but in general it can have different terms but in particular you can have for example a model you can mix models of say q squared plus q with a power p larger than 2 and these models still behaves mostly as a q squared model so like the simple spin glass while if you mix models say q to the cube plus q to the 4 now the model is more complicated but essentially it works like a p-speed model with p larger than 2 but it's likely more complicated so we are going to work mainly with pure p-speed model for most of the lectures and then I will show you some problems some open problems that we have when we work with mixed models so all the nice results that I will show you for a pure model which seems that we have understood everything then you go move to the mixed model it doesn't work so this is an open problem for you I will show you so I introduce essentially most of the models that I will use I will discuss in the next lecture so before starting doing computation let me spend few words on what we have to do with the quench disorder ok so we have an Hamiltonian that contains disorder in some sense we have to average over this disorder and so the question is how to average on the disorder which means on the j so both the topological part so some j's are 0 some are non-zero and really the interaction of the j's since the Hamiltonian each sample has its own disorder coupling you can always define a partition function which depends on the specific couplings of that sample which is nothing but the sum over the spins of exponential of minus beta h with that disorder ok the statistical mechanics essentially you don't have the dependence on j so you compute one partition function and your work is done ok here not because for each sample you compute a different partition function now we have to take the average over the disorder we have to take the average over j and and since the Hamiltonian is usually linear in j the simplest thing to do is to take the average over the partition function and this called the annealed average so the annealed average corresponds to compute zj average over the j's this is very easy because suppose that couplings are Gaussians so let's take Gaussian couplings usually in all these models the specific distribution of the couplings is not really relevant usually once you fix the first and the second moment and the coupling distribution is well behaved so it doesn't have very fat tails it can be plus or minus two values a Gaussian whatever you want as long as it decays fast enough all the physics is determined by the first two moments so once you fix the first moment which is maybe zero because we are interested in symmetric couplings the second moment because you have some rules like in the definition of the PSP model where you have a specific rule for the second moment in order to have a good well behaved thermodynamic limit then the specific choice of the distribution of the couplings is mostly irrelevant so for simplicity let's take a Gaussian distribution so since the Hamiltonian is linear in h this is very simple because each coupling is independent so doing this integral it amounts to let me work in the simplest spin glass model but as you can imagine the PSP model is practically the same you can take the integral over all the all the couplings with the Gaussian distribution of the couplings with the right variance so here you have to put some J the variance well let me write it explicitly since the variance is 1 over n you have to put here oh the coupling well let's write it this way you have to ask ok no sorry let me write this way which is much easier this is the distribution of the couplings the variance is 1 over n so you can put the n in order to make all the expression and then you have the sum ok so you see in this case the integral over the couplings is very easy ok and so you can compute this z the problem is that with this z is that this anneal average will change why for the following reason the partition functions are exponential in n so there are huge random variables having maybe huge fluctuation and so what happens is that when you take the average over all possible coupling realization you may have that the mean value is not representative of the typical value what I mean by typical typical is the I will make an example which is much cleaner so suppose that zj is a random variable and I will make a very trivial but simple and illuminating example it takes just two values a value say exponential of n with probability say 1 minus exponential of minus 3n and another value exponential of 10 times n with the complementary probability ok so you see that these are what we call the typical samples typical samples are those that take the vast majority of the ensemble measure so the vast majority of the probability so if you take a randomly chosen sample with very very high probability you will take one of these samples with probability 1 minus exponential of minus 3n however if you compute the annealed average what you get is that essentially this term which is much much larger will compensate the fact that the probability is much much smaller will dominate the average indeed what you get is exponential of n times this number which is practically 1 so times 1 plus this term times the probability and you see that this goes like exponential of 7n so you get a mean value for the partition function which essentially gives you information because what you get from this then you take the log so the free energy is 7 and 7 is neither the free energy of typical samples which is 1 neither the free energy of a typical sample which is 10 is an average in between makes no sense why this is happening because these variables are very huge and their fluctuation is very large so the annealed average can be used only if you can check that partition functions don't depend too much on the sample so if all partition functions are close to the mean value then the mean value is representative in order to check this you can for example compute the second moment which tells you how far from the mean value the typical partition function is if the second moment so you can compute the second moment and if the second moment is much larger to the first moment square then you don't have to use the annealed approximation because it means that these random variables have huge fluctuation and taking the mean doesn't tells you anything about the the typical values okay so if you study the annealed approximation is fine at a very high temperature at a very high temperature the specific couplings are not really relevant and so each sample looks more or less the same in this way you can use the annealed approximation which is much faster and you are safe otherwise you have to do the quench average what do we mean by the quench average there is that you first compute the free energy for each sample so the free energy of a given sample is minus let me forget about the beta factor is the log of Z okay you should have a beta factor but so if you want there is one over beta here so depending on if you want to normalize a factor n but it is not that relevant I am interested in the dependence on the couplings J now this F first of all is of order n no longer exponential in n and what I call the quench average is the average of the log of Z okay so now I want to define the average of F the quench average of Z there of the log of Z since you take the log now fluctuations are much reduced indeed in general you can prove that the average of the log of a random variable provides you the the typical value but it is much closer than the mean value so it is much much better to take the average of the log moreover there is one more several advantages of taking this average the first one is that if you take a very large system you can prove infinite dimension and you can make good arguments also in international models the free energy becomes self averaging self averaging means that the probability distribution of F gets concentrated around the mean value so F of J is self averaging which means that it is very close to the mean value to the mean value of F of J so the distribution of the free energy shrinks in the thermoamic limit so this is for N very very large so it becomes very very close infinite dimension this it can be proved by just with the same techniques by which you prove the existence of the thermoamic limit if the thermoamic limit exist the limiting free energy exist and you can say you can make the same argument even in the presence of disorder to prove that essentially if you take a very large disorder example you can split a small subsystem each one with its own disorder and so the free energy of the whole system is the average of the free energy of the whole subsystem each subsystem has its own disorder so it's like say that the free energy of the large system is equal to the average over the disorder of the free energy of the small system now if you take both the large system the small system to infinity you prove that the free energy of a very large system is equal to the average over the disorder so this essentially the argument and so this means that it is self-average so essentially you can prove that all the thermoamic relation for example if you want to compute the the entropy you can compute the entropy on a given sample by taking f is u minus ts so it's minus the derivative of f of j with respect to t so this thermoamic relation on a given sample migrates also on the average because now I can take the average here and so I can say that the derivative with respect to the temperature of the quench average is equal to the quench entropy so once you take the average of the free energy then you can use all the thermoamic relation so we want to take this average but taking this average is much more complicated because now you cannot take you have to do the integral of the way of log of z which is a complicated object and so here is where we use the replica trick that the replica trick is very simple it's just a moment to say that the log of z is the limit of n going to 0 of z to the n minus 1 divided by n this limit was known since 14th century I think so and if you want to write in a different way this is also the limit of n going to 0 of the derivative with respect to n of z to the n now you take the average and this is the replica trick so what we are going to do when we want to take the quench average actually we are going to study a new partition function that is replicated n times n is like the partition function of n independent systems then we will take the average over the disorder and this n independent system will get coupled I will show you in detail and so we have to study the thermodynamics of a system of n couple copies in the limit when the number of copies goes to 0 and this is the the cool part we have to take a a continuation because for n finite this is the partition function of n couple c's and a replicated system typical no it is only because you have huge fluctuations so it is like if you look at the you can take any probability distribution which have a fat tail so take a variable random variable x to be distribution which has a very fat tail you call this the typical the model but if you compute the mean the mean is here now you take the log of y or the x you do the same plot and you will see that the plot changes like this you take the this point moves here now you take the average of log well this will be more or less here so the average of log is much closer to the maximum rather than the average of x when x has a fat tail the typical example is the log normal variable you take log x which is normal well of vice versa you can build very easily a variable such that its logarithm is well behaved but not the the original variable the probability distribution is very wrong it's not really the way you it's not really the fact that how it depends on j it's the fact that z is exponentially in N so look I will make a specific example you take a spin glass so the Hamiltonian has whole terms where j is j as we said is order 1 over a square root of n this configuration you look at it is more or less random and the Hamiltonian the minimum of the Hamiltonian maybe is around minus 1 so minus n now suppose that by chance you generate a sample where all the couplings are positive this configuration has a much lower energy because you can satisfy all the couplings the integral energy will be the mean number of couplings per spin which is 3 so what happens is that typically this object here is exponential of n because the energy is roughly 1 so it's exponential of beta n the minimum energy is roughly minus 1 but by chance a really rare event you get one sample where all the couplings are positive and the partition function becomes exponential of 3 times beta n that event is atypical but will dominate the average and so you are not able to see the physics of the typical sample because this is covered by the physics of the atypical samples and so if you do this you will say ah no the model is nice and it's not so much frustrated because I compute the mean energy and the mean energy goes to 3 so it's not even so much frustrated because what you are measuring is only taking information from the atypical non-frustrated samples so if you do the nil average typically you get information from the unfrustrated samples which is what you are not interested in because you really want the typical sample which is the frustrated one ok and since we were discussing frustration let me end this introduction with a discussion of frustration what is frustration and and why is so important understand that this introduction is quite long but I think that all these are concepts that for people working in these fields are obvious but I think that when I was young if I had someone telling me all these concepts so let's put this question is the disorder really a key ingredient in order to get the complex behaviour of spin glasses actually the answer is no so we are going to study disorderly models because these are the simplest models that show a complex behaviour so a behaviour the definition of complex is out of the scope of this course and attains more to philosophy but one of the the typical sentences that people use is that if you take the sum of the parts the behaviour is more complex than the single unit and this is what happens in these strong interacting disordered models but the question is in order to get this complex behaviour should I really need disorder the answer is no what you really need is frustration so frustration is more important than disorder what I mean by frustration is the impossibility of satisfying all the interaction at the same time so the typical example that people do in order to show frustration you take three spins if you put all positive couplings there are actually two configurations that satisfy all the couplings all spin up and all spin down you take the same three spins and you put all negative couplings and you no longer have any configuration that satisfy all the three couplings all the three interactions at the same time so these are non frustrated systems these are frustrated systems so what changes in so consider in this case in spin glasses frustration is directly induced by disorder so in this case the true role of the disorder is not of being disordered is to induce frustration but if I can induce frustration even without disorder I would get the same complex behaviour that we are going to discuss and this is actually what happens in glass forming liquids or in many other systems where you don't have a priori disorder so if you put colloids which are particles in a solution in the initial configuration can be in the noise from outside but there is no quench disorder so there are no couplings that you put quench and you don't touch any more during the evolution of the system so in that case frustration is self induced so is frustration the real property that makes the system complex in the sense that you can have thanks to the frustration in different ways to satisfy the try to satisfy the interaction even in this very simple case you notice because if you enumerate the 8 configuration of these 3 spins what you get here is that in energy this 8 configuration they have put here the 8 configuration this is the energy you have 2 configurations which have very low energies actually here and here of energy minus 3 and then the remaining 6 have energy which is positive so you have 6 configurations here so clearly the system will prefer to go in one of these configurations but now if you reverse all the couplings you are reversing the sign of the energy and so what you get in this situation you have 2 very unsatisfying configurations where you put everybody parallel and so all the couplings are violated the energy is 3 and then you have 6 configurations of energy minus 1 so what you notice in a frustrated system in a system of 3 spins which is really the simplest you can build you have many more minima and of higher energy so in a non-frustrated system you have few minima very deep in a frustrated system you have many minima and not so deep because obviously if one minima is very deep it takes all the weight and so this is what produces the complex behavior when you have many configurations which are equally likely but very different in terms of spin configuration locally you can take different configurations which are equally good but they don't match and so this is what generates a lot of frustration because having many local minima it generates competition between different configurations so the typical picture energy landscape picture that we draw in order to keep a visual image of what's happening in non-frustrated models you have something like this so you have two minima this crew device model and if you do any dynamics any reasonable dynamics the system will flow down here or here so this is not so complex but if you go to spin glasses you don't have so low energy minima you are much higher energy level and you have something like this it's not very different from what I plot here for a tree spin system here two minima very deep minima here many minima at a higher energy level when you take many frustrating interacting pairs of spins what you get is something like this so here you expect everything to be smooth and here you expect everything to be much more complex much more complicated also and let me say that actually in real life and in real application the interesting situation is in between and when by this first of all this picture is too simplified in the sense that I can show you models so in principle you can satisfy all the interaction and still whatever you do the dynamics you are not able to reach the ground state and so actually the more realistic models are a mixture of these two and they look like this these models are really the hardest to tackle because on one side if you do the thermodynamics the thermodynamics is dominated by this minima these are the ferromagnetic minima they exist yes actually this is in the thermodynamic limit so you have a huge number of interactions I made a very trivial example just for so consider that you take a system of n spins you have n squared interactions so you can have a number of triangles which is n cubed so you really have all the frustration that you want your order induces frustration but the way I claim is that there are models this for example is a ferromagnetic model a random graph is ferromagnetic so if you put all the spins aligned you get this point of this point depending well let me do just one minima because I put three ball interactions so there is just one minima so here all the spins are equal to one so if you start the dynamics aligning all the spins so with an order at start you go here but I can assure you that and if you want to program a Monte Carlo code just ask me the details that if you start from a random configuration this is a ferromagnetic model on a random graph actually is a three spin interacting ferromagnetic model on a random graph any random start will bring you here okay so even in a model that thermodynamically has no frustration as a just ferromagnetic transition dynamically is much more dominated by these states so your aim is to compute the physics of this not the physics of this because the physics of these states is what determines the behavior of the dynamics so if you're interested in understanding what's happening to this system when you start from a random configuration and you try to look for the minimum better you understand this part of the phase diagram rather than this minimum which is practically irrelevant for the dynamics this may seem let me say a model I built in order to show you this but actually is the situation that happened most in inference problems in inference problems you usually have one signal that you want to recover plus noise this is the signal and this part is the uninformative part and trying to do signal recovery if you're writing a biasing form essentially amounts to minimize energy function which is disordered because of the noise and try to do signal recovery is exactly studied in this Pinglas model where you have many uninformative minima that try to traps you and there is just one minimum much deeper because it's the signal that you have to find so finding this minimum is like doing signal recovery or finding structure in a data set mining or any of this nice IT application that they are very cool now essentially it amounts to study a Pinglas model where you do have a minimum but you also have a lot of local minima at a higher energy that try to traps you so this picture is the one you have to have in mind when you solve these models so frustration and even self-induced frustration is the key ingredient anyhow we are going to study disordered models because at least we know how to do the average so we know how to do the computation and we solve the disordered models but keep in mind that the same behavior can happen also in models which are not disordered at all and they self-induced frustration during the dynamics even if there are still 5 minutes to 11 since I am going to change and start doing computation I prefer to do now the break with you do 15 minutes break and then come back at 10 past 11 ok see you then