 is devoted for that purpose. So the last talk of this afternoon session on quantum information will be given by Glenn Bigan-Beng from the University of Innsbruck. And yeah, there is Professor Wage who is raising the end. Does he want to say something? I'm going to another meeting. OK, OK. So we get the last talk from Glenn Bigan-Beng right now. So whenever you are ready, you can start your part. OK. I'm ready. Can you see the slides? Yes, we can see your slides. There's just some object on the right that's covering a bit of your slides. It's like the palette, the paint palette. The paint palette? Yeah, it says Colori on the right. I don't see if I can. That's a bit of an issue because I don't see this object. Let me try again. So let me try again. So yeah, it's on the right then in your right corner. Yeah. Oh, yeah, exactly. Close that and that'll be fine. Yeah, thanks. OK. So thank you to the organizer for the possibility to talk. And thank you for organizing this very nice conference. And I'll be talking about the problem of finding the ground state given a certain Hamiltonian. This is closely related to what Estelle discussed before. And this kind of problem in general can be solved using both quantum and classical resources. So solving this kind of problem would have various applications in various branches of science, such as quantum chemistry, computer science, machine learning, and physics. And various groups have already started using quantum device to tackle at least small instances of relevant problems. So one of the standard approaches to find the ground state of a given Hamiltonian is via quantum annealing. So in this case, one takes the target Hamiltonian. And here for the rest of the talk, we will assume that we are working on a Hilbert space of spin 1 1⁄2. So next one adds an Hamiltonian to generate a driving term, which is a composition of the two Hamiltonians. And they are interpolated via a schedule S of t. And this schedule interpolates from time 0, where the Hamiltonian is the initial driving Hamiltonian to the final time where the Hamiltonian is the target Hamiltonian. And the system is led to evolve under this time dependent Hamiltonian, and we collect the final state. So according to the other voting theorem, if the evolution is slow enough, the system will move slowly from the ground set of the initial Hamiltonian to the ground set of the target Hamiltonian. So this is convenient because the ground set of the initial Hamiltonian is usually easy to prepare. But there is a caveat. So the evolution in particular has to be extremely slow or slower, where the spectral gap of the Hamiltonian of the evolution Hamiltonian vanishes, or it's close to zero. So this poses an issue that it's difficult to choose correctly the schedule S of t because we do not know where we have to slow down because we don't have the knowledge of where the vanishing gap is. And usually the standard choice is using a linear schedule for any other guess. But these are simple choices, but not optimal. And in general, it's a problem to obtain optimal schedules without having information about the spectrum of the problem. So here I will consider this problem and we tackle this problem by first discretizing the continuous schedule. So using discretization, the schedule becomes, instead of a continuous function, a stair function. And this leads to the following state, which is obtain applying fixed Hamiltonians for certain steps. So now the schedule becomes, instead of a continuous function, we have only a set of two p parameters where p is the number of steps that we have used for discretization. So the problem can be further digitalized by using a trotter decomposition. In this case, one that composes the single evolution operator of each step in two parts, one for each of the Hamiltonians involved in the quantum annealing algorithm. And essentially, according to the trotter decomposition, one can, instead, apply these Hamiltonians individually for a certain lag of time. So in the language of the schedules that I introduced before, this is equivalent to applying a schedule which has only values 1 and 0, which is called bang-bang schedule, usually. And the time lags for which the Hamiltonian is Hz are now on what parametrized the schedule. So now the problem of optimizing the schedule is a problem of optimizing these parameters. And this discretization is also convenient because such schedules can then be applied easily and studied easily on digital hardware, quantum hardware. And usually they involve an application of few qubit gates. So then when we have optimized these parameters, in 2014, Fahri proposed an algorithm called quantum approximate optimization algorithm in which you take the state you obtain. So what is written here is the general state you obtain when you apply the gates prescribed or retorted to the composition. But now the parameters we use gamma and beta are free. And we use a classical algorithm to optimize them. Essentially, we apply, we create the state corresponding to a given set of variational parameters within a quantum device. Then we measure the expectation value of the energy. And with the classical optimization algorithm, we minimize its expectation values. This results in optimal values for the parameters, which results in a schedule which give a state close to the ground, to the desired ground state. So this is a technically a good technique to optimize the variational parameters. And indeed, what happens is that the equivalent sets powerful enough to describe actually optimal solutions. So the solution, this optimal solution can always be written in terms of these bank schedules. However, these bank schedules are usually very rough and irregular. And it is not clear whether you would recover something similar to the schedule you started from when digitalizing. So in the first part of the talk, I'll discuss how to obtain a technique to obtain these schedules via an iterative optimization of the quantum algorithm. And this leads to, when applicable, leads to an efficient search of these parameters. So it's convenient to use it if possible. So I'll start by discussing the transverse field is in chain. So this model has already been touched by telling the previous talk. And it's particularly convenient to benchmark and study for initial benchmark and studies of algorithms. Later on in the talk, I'll discuss possible generalization for the model. So the transverse field is in chain. It's essentially a ring where the spins are connected and interact. And here with an equal coupling. And here we want to find the ground state of HZ written here in red. And again, we use the algorithm I described before and we produce the auxiliary Hamiltonian. So in general, we can look for a general target for which is we can look for the ground state of a general target, which is a composition of both the driving Hamiltonian and when we apply the QAO algorithm and we have a schedule, it's good to have in mind a parameter that can tell us how good is the schedule. So we consider the rescale the residual energy here written, which essentially it's zero when we have achieved exactly the ground state and one when we are maximally far from the ground state in energy. So when this parameter goes to zero, the algorithm is working correctly. So one, we use an iterative, we propose an iterative construction for these schedules. And we test it on this problem. So we start from a low value of the steps. So here P is equal to and we start from the so-called linear schedule here in gray. Then we optimize the parameters. And again, as you mentioned that we can invert between relation between the parameters gamma and beta to obtain a schedule S as we did in wide digitalizing. So we obtain these parameters and we get the optimal. And here the optimal are the blue squares. Then we go to a higher value of P and we start from a solution which is given by interpolating the solution, the parameters we got at P equal to. So here you see this blue dashed line interpolates the previous solution. And we optimize starting from this point and we find the orange curve. And then we can proceed again iteratively to P equal 8 and to for higher value of P. So what we see is that in doing this local minimal search, the schedule is assuming it's taking the shape where it's getting flatter and slow where the gap is closing. So for this problem, there is a vanishing gap here in the middle. And the problem, so the algorithm actually is able to find where the gap is and construct this optimal schedule without a priori having information about where the gap was located or how or any special information about the problem. So it is interesting to study the performance of such an optimization scheme for the schedule. And we compare it with optimizing the parameters better when starting from a naive random initialization. And we find here, we studied a number of iterations that the classical algorithm needs to find the optimal. And with the random initialization is scale of the square of the depth of the circuit while with an iterative optimization that we just described as a square. So there is a quartic speedup when using this kind of algorithm in efficiency and a more efficient search for these parameters. And then the parameters, we look at the parameters we obtain. So the quality of the state we obtain with the final parameters. And this can be studied with the residual energy that we introduced before. And we compare these parameters with the parameters obtained via the usual standard linear digitized quantum annealing. And in this case, we find that the linear standard digitized quantum annealing scales as a scale as one over the square root of the computational resources while our order depth of the circuit while this solution instead scales as one over the depth of the circuit. So there is again, there is a quadratic improvement also within the precision of the final state obtained with the algorithm with respect to the linear digitized quantum annealing. So as I described, this approach is quite convenient and can lead to an efficient search. However, we found on our discuss later that it's not always applicable and cannot be used for all problems. So to generalize, to tackle more general newtonians, we try to study the problem using reinforcement learning to optimize these schedules. So we view the QAOA schedule optimization as a reinforcement learning process where we have an agent who can apply these gates we describe and then measure the states. So the agent applies the gates, measures the states, and then decides what to do based on what it measures. So the policy, it's the set of rules that the agent uses to decide what to do based on what is measured. And in this case, there are like, I don't have time to describe, but there are standard algorithms in machine learning which allow you to train efficiently an agent. So let's say an artificial intelligence will take the best choices for this kind of problem. So the agent will, at the end of the training, which would last a certain number of epochs, so a certain number of time where we repeat this process for him to learn, will learn a policy and will then try to apply this policy to find the parameters gamma and beta. So in this case, we have to specify for the problem which observables we give the agent. So what is his input or how he decides what to do? And we use as observables the expectation values of the Hamiltonian involved in the query answers. So one technicality which is important is that this approach requires an extra number of measurement because it requires also intermediate measurement for the agent to decide the next move. You have five more minutes. Yes. Thank you. So we look again with this approach at the transverse is in chain. And what we see is that by training the agent for more time, so here you see the first panel, it's a system where the agent has to be trained for more or less 100 epochs and the second for 1,000. And here the blue lines and the red lines correspond to reinforcement learning solution obtained for the schedule or reinforcement learning solution obtained plus a local optimization. So the iterative solution, which is the square, which are the squares, are recovered by the reinforcement learning agent in this problem. And next we consider the fully connected spin model, so where now all the spins are connected. So from a point of view of physically if this model has a longer interaction and somehow we find that it's more complicated to optimize the parameters in this kind of model. So indeed, one can show that the iterative optimization does not perform well in this model and one has to resort to other kind of initialization of the parameters. In this case, we just used a random initialization again, which is the blue line and initialization from the linear digital parameters, which is the orange line. So we apply to this problem again, reinforcement learning. And we see that contrary to what happens with the iterative optimization, reinforcement learning is still able to improve the solution of the problem and to find the optimal smooth schedule for the parameters of M, which are convenient because then for this you can invert the digitalization process. So I'll just summarize what I've discussed. I've discussed two possible ways of optimizing schedules. We digitize schedules. One is an iterative optimization of the schedules, which provides an efficient search. But however, we find that it fails for certain class of problems. And the other one is using reinforcement learning, which allows us to tackle more general militant as some advantages that I did not discuss here, which are transferable, that it can be transferred to larger systems. But we still have a lot of questions on the reinforcement learning or how the reinforcement learning work for this kind of task. So one issue is the problem of measurements. So I would like to reduce the number of measurements. And we have in part, I personally address this with the ability to transfer the policy to larger system. Another issue is robustness to noise. And the next issue will be to characterize the type of solution we obtain now because it's not clear what kind of strategy the solution are following because in the previous case we did not see any slowing down close to the machine gap which we were expecting before. So that will be all. And thank you for your attention. And if you have any question, please. Thank you, Glenn. And thank you for the nice talk. And also for sticking to the time that was allocated. So now I guess the floor is open for anyone who may have a question. So feel free to unmute yourself and ask any question you may have to Glenn. Good afternoon. Yes. I have a question for Enoch, the previous speaker. Can you wait until we get to the discussion? Then the phone will be open right now. It's mainly for Glenn. Yeah. I have just a clarification question. So Glenn, I mean, I'm of course completely out of the field but so the fully connected spin model that you showed Yeah. That is a transverse ising model but with all spins interact. Yes. That's it. So it's the same thing. Okay. Yes, yes. So, yes. Yes. Do we look at that model? And so the, so what's the challenge in doing that versus the transverse spin model? So it's, first of all, one of the first challenge. So when realizing an experiment will be the fact that one needs to couple spins which are far from each other. But this is not the challenge we are tackling now. So the challenge we are tackling now when realizing this kind of, when starting this kind of problem it's that for the, it will seem that in this problem with language interaction, the landscape is too rough and the iterative optimization fails to provide a good solution. So that was a testing ground to see whether by replacing the iterative optimization with a more sophisticated algorithm. In this case, we use reinforcement learning. We were still able to find the solution we're interested in. So we also looked at other different models. So also with disorder, but yeah, I wanted to present a model where one of where the iterative optimization was failing and the learning was working. Usually when the iterative optimization works, the reinforcement learning will also reproduce the solution. But sometimes we cannot use the iterative. I have a, if I can, Steve, a curiosity about the reinforcement learning. Is there a explicit length scale learning that goes on into the reinforcement learning? Or is this, is that something that's automatically built in? So if you have, let's say you have a system with correlations on different length scales. Is this something that is automatically built into the reinforcement learning? So somehow the reinforcement learning has success to the expectation value of the two Hamiltonian, HX and HZ. So in the case, let's see. In the case, if I can maybe. So in the case of the fully connected spin model, it will have access to HZ, which comprehends correlation between spin which are far from each other. Okay, so information about correlation is passed to the agent via the expectation value of HZ. Okay, okay, and I guess that's very important for the, for the learning. Okay, I see, I see, I see, okay, okay, thanks. Okay, so I think you can stop sharing your slides and you can start our discussion with.