 So maybe we should get started. It's my pleasure to welcome Fred Chong. He's a professor at the University of Chicago. So he's been looking at quantum computing for as long as I know as a grad student as for more than 20 years I believe. But now he's leading a high profile expeditions program at the University of Chicago involving many leading scientists in quantum computing. Yeah, so we are very excited to have him. Without much further ado, please take over Fred. Yeah, thanks for inviting me. I didn't pass this, but hopefully we'll share it just fine. So if you feel free to ask questions by unmuting yourself, you can also type your questions on the chat window and then I'll try to pose it to Fred if that's more convenient for you. Great. So as Satish mentioned, I've actually been in this quantum computation space for about 20 years now. And I'd say, and we'll see today that it's a really exciting time for quantum computing because we have real machines for the first time really, non-trivial size. But what I'm gonna talk about today is maybe a little different than most quantum talks that you might hear. I'm gonna talk about a computer systems view of quantum computing and how that's important to really get the most efficiency out of the machines that we have today and in the near future. And we're really gonna look at essentially building software and systems that go beyond the abstraction of a typical quantum machine and really look at the physics of those machines to optimize the computation. Now this work is part of, Satish mentioned an NSF expedition in computing called Epic. And we have a team of people which reflects the sort of vertically integrated work that we do from theorists at MIT, including Aaron Harrow, Peter Schor, and Ike Twann to computer scientists at the University of Chicago and experimentalists at Princeton, Duke and Chicago. So let me just start out with, why as a computer scientist and computer architect, I find quantum computation so exciting over all these years and really the main thing is that it's the only technology that we know that can fundamentally change what is computable, right? It's the only technology that can give us a doubling or an exponential scaling of computation potentially depending upon the algorithm with the number of devices that we have. And this will give us the ability to solve currently intractable problems in chemistry, simulation and optimization to list a few examples which could lead to nice benefits in things like better materials, photo takes, even things like understanding nitrogen fixation better. As an architect, it's also interesting to think about quantum computing because as Moore's law and dennyrd scaling slow down, we have a new technology that gives us some forms of scaling perhaps in specialized domains. And also the study of quantum computing has led to more insights in classical computing. Previously we've seen insights in chemistry, physics and cryptography and as we move forward, it's really sort of a healthy competition between classical algorithms and quantum algorithms in the computing space. So why now, why are things so exciting right now? Well, what we have now is, as I mentioned for the first time, machines of non-trivial size that have been built in primarily industry. And as John Preskill would say, who is a fairly proponent of quantum computing for Caltech, you know, now is a privileged time in the history of science and technology. And he's coined this term that we call NISC, which is noisy intermediate scale quantum. And this both talks about sort of the potential of where we are right now and the challenge, right? The potential is that we're now at intermediate scale, which means that we're at a non-trivial scale where we can actually compute things that can't be simulated classically or at least at the moment are very difficult to simulate classically. And so there's definitely new science that's going to happen, right? But it's also that these machines are noisy, right? If we had machines that were 53 perfect quantum bits, right, we would already be computing lots of really exciting things. But the challenge is that these are noisy qubits and we have to deal with that noise in those errors and try to mitigate them and do something useful even in the presence of those challenges. And a lot of what I'll talk about today is really about addressing that challenge. So another way to look at this is we look at the machines as they have progressed over time. And here, what I've shown is the number of physical quantum bits in a machine on the x-axis on a log scale. And the average error on a two qubit operation on these machines also on a log scale. And as we move from the left bottom coordinate to the right, the machines are improving, right? And in fact, the machines have been improving exponentially over time, right? Which is great. But there's still a huge gap between the algorithms we wanna run and the machines that we have, right? And really the goal of, you know, our project and other projects in this space is to close this gap, right? And the Epic project has this specific stated goal of developing algorithms, software and hardware essentially co-designing them to close the gap between algorithms devices by two to three orders of magnitude. And this two to three orders of magnitude translates to about 10 to 20 years of hardware progress. Now this sounds ambitious, but in fact, it's very attainable. In fact, in the last three years of our project, we've actually come up with many optimizations, each about two to 10x in increase in efficiency up to about 10,000x. Now it should be a little more precise actually. By efficiency, what I mean is either using fewer qubits for computation, using fewer gates or operations for computation or tolerating more air for computation, right? So those are the three things that essentially make up that gap that we're trying to close. And in fact, you know, from these optimizations, we've had about 70 or so papers, five of which received best paper, six patents, a startup we spun out and we're really pushing this idea of honor computer systems as a discipline. In fact, we just came out with a textbook synthesis lecture and this spring we'll have an edX course using this material. And in fact, the techniques that I'll talk about today, many of them have already been integrated into software in Google's software systems. All right, now just to give you an idea of the kinds of things that we do, here's a sort of a pretty long list of selected things which range from modifying the algorithms to for example, in this case, use fewer trials or measurements, changing the way that we compile algorithms to down to instead of gates, but actually to analog pulses, adaptively mapping and scheduling algorithms for the variation in machines, going from binary to ternary logic, looking at two and a half B architectures and various other kinds of noise mitigations in the system. And you can see on the right, the outcomes are fairly substantial and the idea is combining these outcomes will give us that two to three orders of magnitude and in some cases even up to five orders of magnitude in efficiency. We also have fairly substantial efforts in sort of creating an ecosystem to promote this idea of quantum computer systems research from industry relations to open source software tutorials to our textbook to even sort of K-12 efforts. So what I'm going to do today in our sort of somewhat limited time is give you six examples of how a computer systems approach can give us this increased efficiency and close this gap between algorithms. So that's gonna go by pretty fast so feel free to stop me and ask any questions. All right, so let's start, right? So one of the first things that we did is we looked at how to map algorithms to machines, you know, it's sort of a typical mapping and scheduling problem, but doing it in a way that's much more adaptive than typical, right? So in fact, quantum hardware varies day by day, right? And these graphs that I'm gonna show here come from IBM's publicly available machines. And what they do is they actually calibrate the data, they calibrate the machines every day, actually twice a day and they publish that data once a day. So if you look at the top right here, what I show here is on the x-axis different days and on the y-axis, a measure of how good a qubit is, okay? Where higher is better. And what you can see is that day to day, the quality of a qubit varies and even the worst qubit is not always the same one in a particular machine, although mostly it's that blue qubit, right? Also, if you look at the operations between the different qubits, right? In this case, something called a controlled knot between different qubits, you can see different pairs also vary in their gate error rate where higher is worse, right? Day to day, right? And also the worst pair is not always the same. So this variation, it turns out is very important. And what we can do is we can target not only a specific machine, but a specific day of conditions for that machine and compile right before we run. So here on the left, I have an example of a quantum circuit. If you haven't seen one before, each of these lines represents a qubit and time goes from left to right, right? So the P0 to P3 are different qubits. The boxes X and H are single qubit operations. They perform an operation on a particular qubit and the dark circles with the little plus sign, those are what are called a controlled knot. That's a two qubit operation where the dot is the control and the plus is a knot operation on the qubit that it's sitting on. Okay, so we take these gates and we map them to a machine and you can see two examples on the right, right? That's a machine is sort of a ladder apology. It's a real IBM machine. So on the top, you can see that IBM software maps the qubits P0 to P3. So they're all close together, right? So it's minimizing communication. However, it's unaware of sort of the poor qubits, right, the bad qubits, which we've shown in sort of gray circles and the bad links or bad operations between qubits, which are in the red axis and it happens to have selected some of those. Now, what we do is we use two flows, either heuristics or it's a peace solver to actually place those qubits close together, but yet avoiding the red axis and the gray circles, right? Pretty simple approach works really well. It's actually up to about 28x better in reliability, about 3x on average. This approach works so well that it's now integrated into IBM's Qiskit software. And in fact, it has basically changed how the industry compiles to their machines, right? Fred, is it trivial to know which qubits are bad and which connections are bad? They run a calibration routine and it's pretty expensive, right? It's actually probably not so hard to know what's bad. What they do is they run a calibration routine to try to make them as good as they can, right, to adjust for the errors. To characterize the errors. And that's why they only run the calibration routine once or twice a day because it's expensive. There are some questions about, is it more dynamic than that? Should we be running too much calibration? We're often, those are somewhat open issues, but this gets sort of, you know, this is the most of the benefit, right? We do something like this once a day. So this was great, but let's go a little further in breaking our abstraction here, right? Now, in this next example, we're going to look at compiling not to this typical gate model, which looks like essentially machine assembly instructions, but we're actually gonna compile directly to the underlying analog control pulses of the machine. What does this mean, right? So once again, on the left, we take a small part, a quantum circuit, similar to the example I showed before, and in the top flow, which is this conventional flow, that would be translated into these quantum gates which look like assembly language instructions. And then what happens is each one of those assembly language instructions gets translated into a pulse sequence to implement that on either, for example, the super-defecting or trapezoid machine. The super-defecting machine is microwave pulses on a trapezoid machine, it's laser pulses. And that looks like this picture on the top right. Now, instead, what we're gonna do is we're gonna take that function, consider it a three or four input, however many qubits it is, function, and it's just an input and an output. So it's a start state and an end state. And we're actually gonna use a gradient, the sense solver, to find the shortest analog solution to get from the start state to the end state. And what we get is actually this picture on the bottom right, which you can see is considerably shorter and simpler. And it ends up that this is about two to 10x faster than the traditional approach. And you can think of this picture on the right as the conventional approach would get go from the start state to the end state through this series of somewhat indirect gray arrows, right? Whereas our approach goes directly from the start to the end. So it has a lot of potential, but there's a challenge. It could take hours or even a day to compile a program before it can run it because this gradient, the sense solver is quite expensive and the dimensionality of a problem can be quite high. In fact, we have to do this for blocks of a program, not the entire program because it's intractable to do this for the entire program. So, you know, if we only have to compile once and then run the program many times, that's fine. But it turns out this compilation time is a serious problem for an important class of algorithms that alternate between classical computation in the outer loop and quantum computation as a kernel in the inner loop, right? So if you have to iterate an algorithm, a kernel say thousands or even millions of times, if you had, and the input, so this compilation is input dependent. If the input changes to that kernel, you have to recompile each time, taking hours each time over millions of iterations is not a particularly attractive solution. So, it turns out that looking at this problem, we're gonna come up with that approach of partial compilation. And the particular problem is actually is important. It's something called a variational quantum algorithm. And what we have here is quantum and classical hardware working together, where the classical hardware controls the quantum hardware, right? And a good example is, for example, looking at finding the lowest energy state of a particular molecular compound, looking at which electron configurations give you the lowest energy state. And the way that's done in this kind of algorithm is you have an outer loop that essentially guesses those electron positions and an inner loop where the quantum machine can very quickly evaluate what the energy is of that particular region. So in this particular example, which we call a variational quantum eigen solver, it turns out that we can find blocks of code or blocks of the circuit where only a small parameter changes. And it turns out it says this rotational angle parameter, what you see is theta one and theta two and theta three here. And so most of the computation is fixed. So we can do a partial compilation and then essentially fix it up based upon this changing theta. And the result is we can still maintain about a 2X speedup in the implementation of the program with these optimized pulses that is about 10 to 80 times faster than previous ones, right? And the key to this entire body of work, right? Was to break the abstraction of compiling to machine instructions and target pulses directly. And this has been a really exciting line of research that will continue. And I think is going to be an industry trend also targeting pulses. Sorry, could I ask a question? Sure. Can you say something about the complexity of the compilation process with the number of qubits? I would expect it to be exponential. I mean, it's a heuristic, right? It's a gradient descent. At some level, you get to the point where it's not just complexity, but the gradient descent solver will get lost if there are too many dimensions in the space, right? And so we restrict our approach to the three or four qubits at a time in the program. Can you give me an idea of what kind of problems I can solve with three to four qubits? I just don't have any intuition. You can't solve a lot of problems. We string together these blocks into a program of many qubits, right? So basically, a lot of our work is actually finding, being clever about picking which sub blocks, which things to group together in the circuit and compile this way. And then we string them all together. Does that make sense? Yeah, thanks a lot. Yeah, sure. Yeah, great, great questions since I'm doing this in very quick summary. Definitely ask clarifying questions. Okay, so now we're gonna move to another technique, which is a little different. This is a little bit more like quantum memory management. So what we're gonna do is we're gonna try to save qubits by reusing them, right? A typical quantum program would just allocate more and more qubits as you need them. And it turns out actually quantum programs use a lot of temporary qubits, which we call Encila. And in a classical program, you would reuse those bits, right? After you're done with them in a particular computation. But there's an interesting trade-off here in that in quantum programs, qubits are essentially entangled with the result and you have to uncompute or undo that relationship. So there's a big computational cost to this notion of quantum memory management or garbage collection essentially. So what we're gonna look at is the cost of this uncompute and whether it's worthwhile. And it's really interesting. It's actually very integrated with the notion of mapping and scheduling quantum computations. So there's an interesting sort of decision that we have to make. And it comes from this, if we look at a function here on the left, this is a quantum circuit, which has a nested circuit in it, right? So this U of G is the outer circuit. Inside there's a module like in your code, it would essentially be a subroutine or a function. And every time you call this, you're going to this U of G, you're gonna also call this U of F, right? And there are two ways of going about how we tried to reuse some of the qubits. So the red qubits are the ones that are temporary qubits and we might reuse. So one way that we can do this is what we call the lazy way. We can uncompute this on the right there. You can see the mirrored version of the function. And when we get to those little blue circles on the right, we get to use those qubits again, right? So now we've uncomputed it. And so during those red lines are when those qubits are unavailable. But we can do the more expensive thing, which is we can get some of those red lines, those red qubits back earlier if we more aggressively uncompute. So we take that U of F minus one on the left there and we can do that early and get that qubit back. But a crazy thing happens. We have to recompute it later in order to uncompute the parent function, right? So there's like a recompute of the uncompute. And so there's quite a lot more computation in the top option than the bottom option, but you get more qubits back early. So there's a trade-off between being lazy and eager. So it turns out that of course a hybrid approach of deciding when to be lazy or eager is important. And also it turns out deciding when to uncompute based upon how close the qubits are to where you need them is also important. And what happens is that surprisingly, so we designed this algorithm for future fault-tolerant quantum computers where we have a lot of qubits, like when we have maybe thousands or even millions of qubits. And when gates are much more reliable. So we expected that this mechanism would work when we could afford to do a lot of computation to do uncompute. It turns out that this approach works great for NIST machines, which was really surprising. It actually is 50% more accurate or more reliable to use this approach than not on machines that are slightly better than current, which is slightly bigger. So it turns out that even though we have to invest more gates that cause errors, it's cheaper to uncompute sometimes than to find a free qubit somewhere in the machine and move it to where you need it, right? So on these IBM machines, it turns out when you need to move something, it's done through a swap gate. So here, if I need to go from a gate or qubit 50 to qubit 24, so at qubit 24, I need a free qubit. So I move it a free qubit from 50 to 24. All those red arrows are actually swap gates. Alternatively, let's say that I could do some uncomputation and make qubit 24 free or usable again, which is that sort of little green circle, recycle arrow there, right? It turns out that sometimes that is much cheaper than communication, which was really surprising to us, actually. And in future machines, when gates become more or inexpensive, it actually can be 10x better in reliability. So this was an interesting result that actually we had been working with for quite some time before we worked it out. We knew that some form upon a memory management would be useful. So let me move on. So a fourth example is an even sort of greater departure from our normal abstraction. So we're actually gonna look at ternary computation instead of binary computation. So cute trits instead of qubits. And it turns out that another good use of having extra space in our machine is once again, these temporary bits are in silhouette. So what we're gonna do is instead of allocating new qubits to be temporary bits, we're gonna sort of borrow the third state in a device. So the idea of ternary logic, of course, is not new. It's been around classically for quite some time. But it actually tends to make quite a lot more sense in the quantum case. So in classical machines, right? You basically have a finite amount of signal, right? Between zero and half a volt, let's say. And then you break that up into, say, two logic levels and you have noise margins. But if you have three logic levels, you're basically squeezing your noise margins, right? And so you have less, you're breaking up the same amount of signal into more logic levels. But in a quantum machine, it actually turns out that we use these energy levels. And normally we just use the lowest two energy levels and we ignore the higher ones. And so there's actually more signal to be kept, right? And so there's some useful sort of physical property there that we can get three levels. Now, there is a disadvantage that the third level is a little more prone to noise or error. And that we have to evaluate from this one. And so really sort of skipping forward to the results, it turns out that if we use this third level and we cleverly design our circuits to exploit the third level, we get about a 70X reduction in the number of qubits that we in some circuits. There's been quite a lot of interest from different hardware platforms and vendors. Not surprisingly, people want to get the most out of the devices they build and sort of using an extra state and giving us more information to work with and computation is nice. And we used IBM's pulse level interface to actually do a Qtrit experiment. And on the right here, what you see is we're taking a single qubit and we're moving it between three different states in a simple sort of three state counter, right? And we're able to essentially fool the machine into using three states since the machine isn't really designed for it. And it's exciting that fundamentally, the technologies are able to do this three state manipulation, but the machine, the systems still need to be a little bit more optimized for doing. Okay, one more. This is some of our most recent work actually. And this is interesting because what we're gonna do here is we're gonna look at some newer technology, which is essentially quantum memory built from a resident cavity sitting on what's typically used as a qubit called a transponder. And it turns out that this gives us sort of a two and a half D topology or architecture, which it turns out is very good for the leading error correction code, which is called a surface code. And so on the top left here is an example of a surface code, okay? A surface code is a code which you can see that's four tiles here. So each of these tiles is a logical qubit. It's designed to work on a 2D near neighbor topology. He uses only near neighbor operations. And it's very efficient. It's very good at accommodating high levels of air with a small number of physical qubits. So these surface code tiles normally look like this. They're on the left. They're built in a big 2D array. And on the right, we're gonna use this new technology where we have a 2D array at the top and each one of those qubits, which is what we call a transmont, has a resident cavities hanging off of it. And each cavity essentially is like a 10-bit memory, right? So the cavity resonates in different modes and each mode can store a qubit. And it turns out you get random access to all of those modes and you can do some nice things like you can have two layers of these logical qubits and you can make them interact in parallel. So this is the first attempt to essentially virtualize air corrected logical qubits on a machine. And it turns out it works great. There's some cost, you're essentially serializing some of the computation, but it turns out that these cavities can be quite fast. And so, and that coupled with this ability to do random access and essentially have less communication, it makes this an extremely efficient architecture. And one of the things is this thing called a transversal CNOT, which is this parallel two qubit operation that's logical between two logical qubits. And it's about six X faster than how a surface code would normally do a two qubit gate. And there's the sort of cool thing we can do. We can actually squeeze this topology and double some of the qubits. It serializes computation by factor of two, but also gives us essentially a factor of two in compression of the architecture. And so when we look at this architecture versus a baseline, which is only 2D, we can see that something called the threshold is very similar. There's a slight decrease in the threshold, right? Which is, but it's basically similar. So we're comparing the picture on the right, the graph on the right, which is the baseline versus various options of our system. And what you're seeing here is a very typical quantum computing plot. What you have on the bottom is the error rate on a device becoming worse and worse, right? From the left to the right. And that that's a physical error rate. And the logical error rate getting better, right? From on the y-axis. And what you see here is each of these error correction codes, right, will get worse, right? So each of these colors is a different strength error correction code, right? So D equals three is the weakest code. D equals 11 is the strongest code. It means that you have a bigger and bigger 2D tile. So you're encoding in more physical qubits. And the threshold is when these lines cross. And what it basically means is that by increasing the code distance, you can make the system better, right? So you've now essentially, you're winning more because you have more computation to do, to do error correction. So that's more gates, but the correction is strong enough that you actually gain reliability. And so that's this point at which the technology has to be better than this threshold in order for your error correction code to buy you something. Yes. Francesca is joining us from CMU where she is a senior PhD student. And the topic for today is going to be hash tables. Before we get started, I'll value it. Thank you. Good question. Another talk. Seth, could you mute yourself? I did not come up with a little old buckets for hash. Can you use some sort of functionality to find what's that particular code? Can you mute yourself? Seth, could you mute yourself? Yeah. Sorry about that. So that's a little bit more detailed than some of our other things I did, this essentially shows you that we can make the system much more compact about 10X more compact 20X actually with a little bit of cost in how reliable the devices would be. So the exciting part of this is not just the compression and the speedup, but actually this bottom line here which is we could build a prototype with just 10 or 11 transmons and nine cavities which could implement 10 logical qubits, right? So this architecture and this scheme actually gives us a target of a physical machine prototype that would be very exciting and very near term to build which would give us a fault-tolerant quantum machine with 10 qubits that would last a really long time. So this is my last example. Now this example, I also quite like this example. This is an example of how physics-aware software can make it so that you can build a simpler machine which has the performance or reliability of a much more complex machine, okay? So what we're gonna do is we're gonna look at this phenomenon of crosstalk which is when you have two gates near each other causes errors and we're going to try to tune the machine to avoid this kind of phenomenon. So here's a picture of an IBM machine and these two black sort of rectangles are two quantum gates, two qubit gates that you want to do in parallel. If you do these two gates in parallel because they're right next to each other and have this link between one and six next to each other you will get error that's 10x worse than if you did not do them in parallel, right? And so that's crosstalk caused by resonance between the qubits as you're trying to do these to operate these. So that's bad and we want to avoid that. Now there are a few ways to avoid crosstalk and this picture, this quad chart gives us some of the design options, okay? So in this work, what we're going to look at is having qubits that are implemented that can change their resonant frequency. It's what we call a tunable qubit but we're going to have the links between the qubits are going to be fixed. So they're not going to be tunable. Essentially they'll always be working. Now there are some other options. The IBM machine is fixed qubit and fixed coupler, okay? So nothing is tunable, it's simpler to build but we can see that roughly speaking it's about 13x worse if you don't have tunable qubits. Now the Google machine is both tunable qubit and tunable coupler and has quite good crosstalk avoidance capability because you can essentially turn off the link between two gates that you don't want to have crosstalk, right? You can disconnect them by turning off the coupler. But it turns out if you're smart about how you assign frequencies, you can avoid that. And this is important because a tunable coupler is basically as hard to build as a qubit, okay? So a coupler, a fixed coupler is basically a capacitor. A tunable coupler is basically a qubit. And so one thing you might notice is that many years ago, what three years ago Google announced something like a 79 qubit machine and then two years later they announced a 53 qubit machine and every, and you're thinking how did they go from 79 to 53, right? Well, that 53 qubit machine is really something like 150 qubit machine because every link in that machine is actually essentially a qubit, right? In a 2D mesh. So their previous machine didn't have tunable couplers, their new machine has tunable couplers and performs quite well. Okay, so the big sort of conclusion from this work is if we're smart, we'll be able to build a simpler machine that has good crosstalk avoidance. And the way we're gonna do that is first in the top row, we're going to assign frequencies to the qubits that are essentially disjoint with the neighbors, right? And so that's simple, right? We do a simple graph coloring and we assign frequencies such that no two qubits sitting next to each other have the same frequency. And then we do something a little bit harder. We take the computation and we look at every epic of the computation in the dependency graph. So essentially we look at all the parallel operations that have to happen in a given time step and we build a crosstalk graph which essentially looks at all the potential crosstalk that can happen. And then we do graph coloring again to assign frequency so we avoid crosstalk between those. Now, what's important is we have to dynamically tune the couplers or not the couplers. We have to dynamically choose what the interaction frequencies are to implement gates for each application, for each time step for a machine. So it has to be very tailored. But if we do that, we can avoid crosstalk. And here in this picture, our baseline is the blue line which is sort of the IBM like machine. We are, our best case is the red line. You can see it outperforms the blue line on average by about 13X. And then you can think of the more complex Google like machine as the green line. Here, higher is better. This is a program success rate. And you can see that the red lines are comparable to the green line but our machine is arguably much simpler to build. So that's the conclusion of this crosstalk mitigation work. And what I wanna do is sort of wrap up with this idea that if you look at the software flow of the crosstalk mitigation work, it was really about building a very vertically integrated domain specific software stack that takes us from the program level to a physics-aware view of the hardware level, right? And this is really the notion of quantum computer systems design that we're trying to advocate and proliferate here, which is you need to have designers that are trained in both system software, architecture, and also devices at some level and be able to do this sort of vertically integrated optimization. And really, this is analogous to trends in classical systems, right? There's really a renaissance of classical systems of vertically integrated domain specific software stacks. Now, one more thing I want to bring up, let me see, do I have time? Yes, it's just an open problem, right? I'm not gonna go into too much detail here, but one thing that we're, I've talked mostly about optimization and efficiency. One thing that I haven't talked a lot about, which is also a big part of our expedition, is verification, right? And really, the big question is, how do I know if my quantum program is correct when I run it on a machine? We have this sort of ridiculous bootstrapping problem, right, like a forklift on a forklift, right? We have quantum hardware that will be building that will be barely reliable, right? We're really pushing the limits of technology to scale these things. And then we'll have quantum software that will be untested at scale because by definition, we're trying to build quantum machines that can't be simulated by classical machines. And so we can't really do our sort of traditional tricks of software testing and software test factors, right? So we really need a new methodology. Now, we've had some work, interesting work in essentially quantum assertions, which is essentially tests in program while we're executing on a machine, which is interesting, but costs gates. A really nice, you know, match here would be to use more formal methods, right? So, you know, typically, I think as an industry we're a little lazy about formal methods in the sense that we figure it's easier just to do testing. But in this case, we really can't do testing very well. And so formal methods to verify, you know, our programs and our compilation are really important. And really we've made progress mostly on verifying the compilation, right? The classical compiler that transforms the quantum program, we can verify that at some level. Fundamentally, there's a really, you know, tough question here is, can we verify the correctness of quantum algorithms or quantum programs that we implement themselves? And, you know, fundamentally, we can't verify any property in tractable time that's too close to the computation itself, right? Because if we do that, then we've actually just disproven that that algorithm has any exponential advantage over classical, right? And the fundamental question is, can we find properties of programs that are useful for checking correctness that aren't too close to the actual result of that program, right? And that's really unknown right now. I mean, we have some ideas, but, you know, that's a really interesting problem and actually has implications in addition to the practical implications of how do we check programs, it has implications for, you know, what is the space of quantum supremacy or quantum advantage, you know, which algorithms are gonna be useful? Okay, so I'm going to not go over this example in detail, but we do have some work where we did some SMT-based verification of a compiler, in fact, the IBM compiler. And the result was that we actually found four bucks in the Qiskit compiler, which is an open source compiler that has, you know, community contributions, right? And so we've actually built a automated method to try to verify those contributions. And so this is pretty exciting. But I will say that what I've talked about most of today, which is optimization for efficiency and breaking abstraction, is it somewhat indirect opposition to verification, right? So this system that we built relies upon the modularity of the IBM compiler to verify each layer, right? In a tractable way. And so as we break abstraction and break layers, we're gonna make it harder to verify them. In fact, for example, we don't really know how to verify pulse level implementations of programs, right? Without actually simulating the program or simulating the machine. So Fred, can I ask a quick question? So you haven't talked much about what type of applications are suitable for these quantum systems today. So some of these verification problems are probably dependent on the type of problems you're trying to solve, right? So for example, if you're trying to do SMT solver, then you can think about verification. But if I'm trying to run an ML inference, I'm not sure what verification means in that context. That's true. And in fact, yeah, so verification of the compiler is fine because we know how to do that, right? But verification of the algorithm or the implementation algorithm, that's quite difficult. In fact, another approach that I would advocate is that we're looking at is provably correct synthesis of an algorithm from its mathematical specification, right? So proving the equivalence of an algorithm specification and a program is definitely hard, right? Or proving that it's correct. But proving that the transformations, it's similar to this approach of an algorithm to an implementation are correct is probably easier. So I'm actually a big proponent of program synthesis as a method of getting verified correctly. I see. So when you're talking about verification, you're talking about the middle layers, like compilers and translators and schedulers, not the application itself. Well, we would like to verify the application, but I think that's a really hard problem. So I think a practical workaround are doing it layer by layer in this way, right? We haven't done that even for classical computers, so. Yeah, and this assumes that, you know, you have an algorithm that is sort of mathematically proven to be correct, right? And then you want an implementation that matches the algorithm. But I think a synthesis method along those approaches is probably the best way to go. So this brings up sort of my last point, which is, you know, I talked about this gap between algorithms and machines, right, and how to fill that gap. But there's sort of another interesting gap, which is between, you know, how should we design software in the short term and how should we design software in the long term, right? And so right now we're looking at a lot of specialization and breaking abstraction to get the most out of these short term machines of 100 to 1,000 qubits. How much of that will persist when we get to 100,000 or a million qubits and we get to fault tolerant machines, right, that are much more reliable? And that, you know, this is very much a similar evolution, right, in classical machines from the 1950s to now, right? And so, you know, my intuition is actually that it's a little hard to imagine that a future in which qubits are really free, you know, they're always going to be somewhat expensive, and they're always probably going to have some degree of error and noise, right? And so I think that some of the things we do, the most important things that we do right now will persist into the future, at least at the system level software in terms of dealing with the physics of these machines. And in fact, even in classical machines, we now see some backsliding of abstraction because, you know, we have device variability as things scale, right? And we have things like energy constraints. So, but this is a really interesting question because, you know, 20 years ago, I was working on the right side of this before we had machines and at a very high level of abstraction. And, you know, for the last, you know, three years or five years, we've been working very much on the left side of this, right? And so the question is, where will this go from the next five to 10 years, right? But I will argue also that the work on the left is very important in the next five to 10 years to actually get us over sort of the valley of death for quantum machines because we have to show that they can be useful in some way, even to get to the right side of this picture. Okay, so let me sum up, right? You know, I hope that I've conveyed some excitement that quantum computing is at a really historic time, right? And, you know, really my main message is that a computer systems view is really critical, you know, to get us to where we want to be, to greatly accelerate the progress in this field, to get us to a practical point where we're actually able to compute useful things. And it's also the greatest shortage in the community and in industry to have people who are trained in this, you know, vertically integrated way to really get these, you know, really large efficiency gains that arguably are easier to do through software and co-design than by improving physical hardware, which of course also needs to happen, right? So, you know, I went through a lot of this stuff today really quickly, but we have a very extensive website for Epic, which you can go to, which has all of these things that I talked about and more, as well as, you know, tons of videos and tutorials and our textbook and our blog. So I hope this has gotten you a little bit excited about this area. And, you know, I am happy to take questions and today and, you know, going into the future if this interests you.