 Good afternoon everyone, not a good afternoon. He's the only one person who would like a response to me All right. Well, good afternoon. I'm Eckhard Groll in case you don't know me I'm the William E. and Florence E. Perry ahead of the school of mechanical engineering as well as the rally professor of mechanical engineering I'm excited to see so many of you here today for our Purdue Engineering Distinguished Lecture It's great to see you would like to give you a little bit of an introduction to what this is all about but beginning in 2018 the Purdue Engineering Distinguished Lecture series invites world-renowned faculty and professionals to Purdue Engineering to encourage sort-provoking conversations and ideas with faculty and students regarding the grand challenges that we're facing and the opportunities in their fields Beside presenting a lecture to a broad audience of faculty and graduate and undergraduate students They engage in an interactive panel with faculty and students as well. So after the lecture We will actually have a panel discussion and I invite all of you for the panel discussion as well So with that the formal introduction of today's speaker will be done by Arvin Rahman He is the dean of engineering and he said I should keep the introduction very Short, but I would like to mention that Arvin and I have been colleagues for many years And I've known well and I'm so grateful for him to lead the college So with that Arvin come on up and do the introduction. Thank you Good afternoon warm welcome to everyone who's here in person and to everyone watching this online as we stream this event Yeah, it is my honor to introduce today's distinguished lecturer dr. Karen Wilcox dr. Wilcox is the director of the tinsley Odin Institute for computational engineering and sciences at UT Austin where she also serves as the associate vice president for research and She's also concurrently professor of aerospace engineering and engineering mechanics there at UT Austin While she does that she also finds time To be an external faculty at the Santa Fe Institute Now dr. Wilcox is Internationally renowned for her work on reduced-order modeling and scalable computational methods and her research has had tremendous impact on the world in particular It has been incorporated into aircraft systems design and into environmental policy decision-making She has been the recipient of numerous recognitions She's a fellow of AI double a fellow of Siam fellow of the US Association of Computational Mechanics and most recently member of the National Academy of Engineering Would you please join me in welcoming dr. Karen Wilcox to the stage All right, thank you very much for the kind introduction I Did not think to unlock my computer before all that happens and there we go All right, it's really wonderful to be here to see some old friends Sing up some younger friends and also to meet new new people new faculty And to talk with some of the some of the students I was trying to remember bill when I was last at Purdue was so long ago that I can't even remember But it's been at least a decade. I think so all right, so today. I'm really happy to Share with you some thoughts and some glimpse of some of the work that we've been doing over the past years With the long title here from reduced-order modeling to scientific machine learning how computational science is enabling the design of next-generation engineering systems and I really want to through the talk touch on these four things I'm going to start off really motivating the work that we do through the lens of digital twins Which I'm sure is a phrase as a topic that many of you have been hearing Thinking about what the excitement the potential is around digital twins But also what some of the challenges are around digital twins Particularly when it comes to the complex systems that we as engineers think about then I'm going to dive in and Tell you just a little bit about some of the reduced-order modeling work that we're doing and particularly the method that we call Operator inference and I do have to warn you there is going to be a little bit of linear algebra ahead But really just try to convey you to you some of those building blocks that are so and so essential to Realizing the potential of of digital twins Okay, so let's let's start out. So I have to ask is there anyone here who has not heard the phrase digital twins? Yes, I've started to ask the question in the negative to get the more meaningful data collection Of course, this is a hot topic. It's something that we're hearing about in a number a number of Venues when I think about digital twins and particularly when I communicate what is a digital twin to an audience of non-experts I often like to draw on the Apollo example and for those of you haven't seen it There's a blog from somebody at Siemens that kind of recounts the story of the Apollo 13 crisis and it's kind of it's I guess fitting that we're sitting here in the Armstrong building to talk about this So Apollo 13, I think we're all familiar with that. Thanks to Hollywood where the spacecraft suffered a malfunction up in space became damaged became crippled and Of course, the challenge was trying to figure out what to do and how to bring the astronauts back safely to earth So the story goes that during the Apollo program NASA when they launched a spacecraft up in space would also keep a physical simulator on the ground in Houston And so during this Apollo 13 crisis again The story goes that NASA were able to take the data From the physical spacecraft stuck up in space Feed it into the simulators and these were physical simulators These weren't digital simulators back back then 1970 but physical feed the data and adapt the simulators so now the simulator is mimicking what's going on and it's physical counterpart and Run the what-if scenarios that ultimately inform the decision-making that brought the astronauts back home safely to earth So I really like this example Not because this is a digital twin again It was a physical simulator because this example shows the power of what happens when you take models and Today we have very powerful mathematical models that are encoded as digital simulators You take models and you combine that with data from a physical counterpart and not just data That's collected once but data that is collected on an ongoing basis That is reflecting the changing nature nature of a physical system You put those together and then what that gives you is a very powerful way to drive decision-making So if we fast-forward today and we think about this the sort of paradigm of models and data and Driving better decisions. Let me give you another completely different example To think about and this is this is something that I have become Very interested in through some of our collaborative work at the Odin Institute So radiotherapy regimens for high-grade gliomas. So this is planning treatment for cancer patients cancer patients and in particular in this example these high-grade gliomas As a form of brain cancer So what we're looking at here and I always and I know I'm being recorded give the caveat You're going to hear an aerospace engineers explanation of oncology, but I'm speaking to mechanical and aerospace engineers, so that's okay What we're looking at here is what's called the standard of care Which is sort of the standards treatment that is given to patients today So on day zero the blue dot that you see there. That's The MRI the imaging that's taken The the tumor you can see they're depicted in the image then on day 20 the radiotherapy starts and See do I have a pointer on this thing? Maybe not So you see there with the purple bars that represents two grays per day of radiotherapy delivered and we Delivered two days on Monday Tuesday Wednesday Thursday Friday five days a week and then nothing on Saturday and Sunday And then two grays per day for the next week And so then you can see weeks one two three four five and six for a total of 60 grays That's the end of the standard of care regimen and Then at some later time There's an assessment for what is called the time for progression Which is a prediction of the time at which the tumor will come back at a certain level And then a follow-up evaluation and so again, this is the standard of care This is what is delivered to all patients So now I want you to sort of think about that Apollo example think about the notion of a digital twin and Think about how one could move from this one size fits all approach to treating cancer to To now bringing together models data this decision-making and moving towards patient-specific care So my group my group of aerospace engineers and computational scientists have been collaborating with the oncologist at UT Austin and at MD Anderson to To really think about how digital twin technology could make a difference in this kind of application and what you see now depicted on the second line here is You know a notion of how a digital twin Could could work in the situation. So again day zero. We have an image at day 20 We have another image for the first week We start with the standard of care where we're delivering two grays per day and then at the end of that That first week there's another image taken Which is now data that can be fed into the mathematical models that are describing The tumor in its evolution and being used to calibrate those mathematical models To a patient-specific model to think about this idea of models And we need the mathematical models that in this case and by the way and bill you may be interested to know this the models of tumors Reaction diffusion equations. So they're actually very sort of familiar although the physics the biology is very different. They're the kinds of Partial differential equations that we see in other aspects of engineering Combining that together with patient-specific data and really putting it all together and driving towards This notion of a digital twin that can drive decision-making So whether you're thinking about aerospace systems aircraft spacecraft whether you're thinking about civil systems buildings bridges and infrastructure Or whether you're thinking about medicine again That's idea of combining models and data Through the process of data simulation, which is mathematically how we referred to putting the two together to Issue predictions and drive decisions. I think is very very powerful and that really is is at the heart of what is a digital twin and You know, I believe strongly that a digital twin builds on modeling and simulation, but really goes beyond it And goes beyond it in the way. I've described where we have models We have data and we have this bi-directional interaction between the two So that was one of the next graphics that that was coming up And then the the next one switch I'll skip over in the interest of time was really really just to highlight and show you some of the work That's going on at the Odin Institute the incredible interests and digital twins across engineering across Climate sustainability the natural sciences and across medicine and while all of these areas are really exciting When I look at the medicine field, this is where I see just incredible opportunities To bring the kinds of simulation and computational techniques that engineers have been using To to really influence things in medicine. All right, so we're going to go over this one a little quickly The elements of a digital twin and again, this was fleshing up some of the the things that we have going on at the Odin Institute and manufacturing aircraft spacecraft Thinking about climate change really at a global scale with high fidelity simulations in this case This is Antarctica and that's Greenland and the interaction with the Atlantic currents and then in Medicine where we have two big centers within the Institute one focused on computational oncology and the Wilson Center Which focuses on cardiovascular? work so Okay, so the potential of digital twins very very exciting and Sometimes I think that if you were to open the newspaper or to pick up the brochure from a simulation company You might think that digital twins are a done deal that that we really can create them That's not the case and I want to spend just the next say five minutes Talking about some of the challenges of digital twins despite this great promise We're still not there yet, and I want to be very clear this cancer example I gave it's really Aspirational to think about having a digital twin for every cancer patient that could provide these optimized treatments in the way That I described this is a vision But we're not there yet. So why are we not there yet? Well, so the first thing is if I were to ask you and when I talk to people and they we start talking about digital twins to Think about conceptualizing a digital twin. What would you put in the middle of it? anyone What word would come to mind? What would you? Model oh good bill Bill got the right answer so Bill said model It's probably because of his background, but for many people they immediately go to the data they think about the digital twin with the data being core and You know really having piles and piles of data and then Probably having some kind of a numerical model wrapped around it probably a machine learning model that will go to work on all these on all the data that we have with some kind of analytics and drive decision-making and this may be a Good model for a digital twin and in some applications And in fact maybe in some of the most successful applications that we've seen of digital twins particularly in the aerospace industry Where we've seen digital twins driving predictive maintenance for let's say aircraft engines This is the situation we have where there is tons and tons of data being generated by the engine as it's flying and the Decisions to be made are the decisions about when to make maintenance. These are relatively simple decisions That fall well within the realm of the data that that are being collected across the fleet So that that way of thinking about a digital twin is absolutely valid for some problems But there are many many problems for which that is not That is not enough and that's because as many of you I am sure know it is really difficult to do predictive modeling for complex systems And what do I mean? We discussed this at breakfast this word predictive a predictive model is one that can predict Can predict the future can predict conditions that I haven't been seen but not just predict them predict them and get them right or predict them and Have some knowledge of just how valid are those predictions how good are those predictions? How much confidence do we have in those predictions as we we go to make decisions and again? It's really really hard to do predictive modeling for complex systems for all the reasons you see here these systems are Multiple cross multiple scales Despite the fact that we now have exascale computing. We cannot even begin to resolve All the scales in a human or even in a complex material in an engineering system Multiscale multi physics. We're talking about high-dimensional parameter spaces There are computational constraints Sometimes we need to do these sort of the data assimilation in in real time There are limited data. This is a really really important one to emphasize and actually if you take one or two things Home with you today after my talk. This is one of them. Yes We live in the age of big data But as engineers and also in the medical world we can almost never measure Exactly what it is. We want to know We are almost always taking indirect measurements of something else and then trying to infer what it is We want to know and not only are those measurements indirect They're usually sparse on an engineering system. They're expensive. They add weight to your aircraft They add costs. They consume power. They do not come for free and yes The data may be big as measured in gigabytes, but it is almost always Limited especially when you're talking about predicting the future or predicting conditions that you haven't seen so yes, the data are big but the data are limited and then finally Verification validation on certainty quantification are so important if we really want our modeling to be predictive so these are all challenges and You know, I really land on the statement that big data alone is not enough in some cases It's enough but for many of the problems that we as engineers care about big data are not enough so we really have to think about our digital twins as Bringing in something more and that something more is the models the models that encode physical principles Conservation of mass momentum and energy and all the other pieces that that we know to hold So let me give you another way to think about conceptualizing a digital twin, which is at the core Bilstall the thunder it's a mathematical model and that mathematical model encodes the domain knowledge even if it's imperfect domain knowledge it encodes what we know about these systems and Again, I really want to emphasize that this notion of a mathematical model This is what has let engineers predict the future as and make predictions about the structure in this building and What loads it can take and how it needs to be designed to have the levels of safety and reliability that we have in the built world around us Those are mathematical models encoded in in computation So this is not a new concept a mathematical model encoded in Numerics of course with data data fed in through the lens of models And all of that wrapped around with with decision-making Okay, so two ways to think about a digital twin and again I don't want to suggest that digital twins cannot be data-centric they absolutely can but I do want to emphasize that for many of maybe the the High-impact applications of digital twins and engineering and a medicine of in particular We need the models to be at the center of it all Okay, so that's wonderful. We say go off and do it, but there's a big catch and That big catch is that these mathematical models which by the way in many situations come to us in the form of partial differential equations What are partial differential equations? They're equations that describe quantities that evolve in space and time and of course that's not all of it There are other kinds of mathematical models Discrete phenomena that are that are important, but just as one example These physics-based models are powerful They let us predict the future but the catch is that they are really really expensive to solve because by the time you have discretized this mathematical model and turned it into a simulation capability you might have something like this example of simulation a combustion simulation of a Rocket engine a simulation that at full-scale the scale of a full engine might take weeks or even months on a supercomputer and You don't want to do just one simulation You want to do many many many simulations and you want to have a digital twin That is assimilating data and updating these simulations And so the expense of these physics-based models is a is a real barrier to Having them at the core of of a digital twin Okay, so that leads us into reduced-order modeling which really is a critical enabler for accelerating predictive computations We need to make those predictive computations. These are the mathematical models that we know are imperfect But we trust them to give us predictions that can drive decisions that can be validated and can be drive decisions We need to make them faster So that these mathematical models can be the core of a digital twin or even can be just the core of of engineering design So what is what is reduced-order modeling? reduced-order modeling is an old is an old topic it's a field that's been around for several decades and The general idea is shown here, and I'm showing to the students here. I hope you Sort of recognize your state space form a good old x dot equal a x plus BU right? This is sort of a schematic of that on the left is a high fidelity physics-based simulation by the time you've taken your governing Equations whether their partial differential equations or whatever they are and discretize them You're going to end up with some kind of a large system to solve and that system will have millions or maybe even billions of degrees of freedom because again, we're talking about discretization in space and ultimately in time and again that That simulation might take minutes hours days Maybe weeks maybe months to really solve at the scale of a full system where we're targeting for digital twins So the idea is can we come up with a reduced-order model there on the right? The reduced-order model is first of all much smaller in dimension So instead of having millions of degrees of freedom it has maybe tens or hundreds and importantly instead of taking hours or days to solve it it takes seconds maybe fractions of of seconds and Of course we can come up with reduced-order models But the big question is can we say something about those reduced-order models and how good they are? How predictive they are so that we could actually start to use them with confidence in our design or in our design task or our digital twins start task A broader class of problems is surrogate modeling right so surrogate modeling is the process of coming up with approximate models Of which I would say machine learning approaches are one way to come up with surrogates reduced-order models tend to follow a particular Process that process is first of all to generate training data Meaning that we have to solve or somehow query this large high-fidelity system to generate some kind of representative data in the reduced-order modeling world We refer to those as snapshots So these are going to be solutions pressure fields temperature fields velocity fields that represent the the system at high-fidelity The second step is to identify some kind of structure And the structure that we're looking for is some kind of structure that is going to be amenable To this compact low-dimensional representation the reduced model Many of the methods not all of them But many of the methods will look for a low-dimensional basis that Describes a low-dimensional subspace so the geometric picture in your head is here's that rocket Combustion simulation there are millions of degrees of freedom. That's a million dimensional state space What we're looking for is a low-dimensional subspace Maybe a hundred dimensional subspace that cuts through that vast High-dimensional state space and as a subspace on which the dynamic the the dominant dynamics really evolve So identifying structure and then the third step is the reduction step where we go back to the physics And I wrote here partial differential equations because that's what we deal with we go back to the physics we take the physics physics governing equations and Project them mathematically onto the low-dimensional subspace and I'll show that and a little bit of math in a second Again, I just want to pause here and note that if I had stopped at step two We could have been talking about a machine learning method right generate training data and then identify Structure sort of fit some kind of an input output map Where I think reduced order modeling and black box machine learning really deviate is in this third step Where we go back to the physics and even though now we are talking about an approximate model We are injecting the governing equations back in through their projection onto this low-dimensional subspace And by the way when you do that if you do it in Responsible ways and you are working with nice classes of systems particularly linear systems You can come up with reduced order models with rigorous guarantees Error bounds in some cases error estimators and other cases that again give you a sense when you use the reduced order model to make Predictions for conditions that you didn't see in training just how good is it going to be now? We can't do that for all systems particularly in the nonlinear case But in some cases we can get those rigorous guarantees and why are there guarantees? It's again because the physics is is injected Well, there is no network and there is not necessarily a network Let's let's do the linear algebra then maybe we can see if your question is answered. Okay, so Again start with a physics based model do not try to read the partial differential equations and this let your name is professor Anderson So just again to make it more concrete the governing equations for that reacting flow That you saw It's the Navier stokes equations conservation of mass for minimum energy and then the governing conservation laws for the chemical species and again People who are expert in these things work their magic and turn this mathematical model Into a numerical model that is solved with high performance computing And just to make things kind of clean and a little bit more concrete To the to the question that was just asked I'm going to work with the form that you maybe can see there at the bottom of the slide Which is x dot equal ax plus bu plus f of x and you so sort of the standard linear state space form with all the nonlinear terms just just grouped there and in in f and What is x here? These are now the discretized state So we're solving for the pressures the temperatures the velocities the chemical species But now instead of being infinite dimensional fields. These are discretized Over some kind of grid and so there are millions of degrees of freedom in this the system We can write it down nicely on this PowerPoint slide But this is a massive system and of course there are all kinds of complexities that go into that discretization Okay, but what's what's the kind of the point of showing you these equations? So I want you to think first of all just think about a linear system So a big linear system time time-dependent system x dot equal ax plus bu and again X is a very big. Let's think million dimensional state vector So let's think about those steps of reduced order modeling We said we would look for a low-dimensional subspace in other words We would approximate our very high-dimensional state x in a low-dimensional basis In a low-dimensional subspace represented by the basis V so V here is Got dimension millions of rows, but it's only got our comet columns It says it's got a hundred degrees of freedom and now the x-hats are the modal coordinates They're the expressions of my unknowns in this low-dimensional basis So here's that approximation we can substitute it into the governing equations Now we have millions of equations to satisfy We have only a handful of degrees of freedom So we are not going to to be able to satisfy all those equations. We have a residual This is the projection step and again just keeping things very simple for those who are not familiar with these methods what does projection mean it means imposing an orthogonality condition and It takes the form you see there The end of the story and the thing that is important is that the reduced-order model which you see at the bottom x hat dot equals a hat x hat plus B hat u Has the same form as the system we started with x dot equal x plus V u Now instead of solving for a million States we're going to solve for just r of them tens or hundreds and The matrices a hat and b hat are nothing but the projections You can see them over there on the right. They're the projections of the big matrices onto the low-dimensional subspace So that's a linear linear system And just hold that thought the reduced-order model if you derive the model through projection onto a linear subspace in this way The reduced model in the linear case has the same form as the one we started with Okay, the same is true if we were to take a quadratic model So now x dot equal x plus V u and now let's introduce a quadratic term What does it mean quadratic term think of Navier-Stokes u to u dx? It's quadratic in velocity Right or a term that involves pressure and velocity is quadratic So we're going to introduce quadratic terms which again show up because they are given to us by the governing laws of nature if you were to go through the same thought process Proximate the state in a low-dimensional subspace project you would get a reduced-order model that now is operating in this low-dimensional subspace, but it has the same form as a system we started with Okay, so why why do I emphasize this preservation of form? I'm writing it as linear algebra Because that's the way I like to think of it But the fact that we have an a matrix we have a linear term the fact that we may have that H Operator the quadratic term. That's not a coincidence. That's coming because of the governing laws of nature, right? When you write down conservation of mass You have a linear equation when you write down conservation of momentum Depending on the system that you're working with you'll go to have linear terms. You might have linear diffusion terms new D by D second derivative of velocity. It's linear in velocity You might have quadratic terms. You might have cubic Yes, you may have non-linear terms and that's a topic of another talk But the structure in the model is there because of the physics and so if your approximation preserves the structure Your approximation is doing something that is aligned with the physics And I think this is a really central idea as we start to move towards surrogate models that embed physical structure embed physical constraints And and move towards being being predicted Okay, so that's all all great and you'd say oh, this is wonderful Why is the whole world not using reduced sort of models because Karen mentioned that this is an old field that's been around for decades and You know the answer is that reduced sort of models have had some impact in some Some areas and some areas of industry But they have not seen anything close to the adoption that we're seeing with things like machine learning and you could ask Why is that and I think the main reason is they're just too difficult to implement right? I'm talking about a matrices and h's and higher-order terms and now I'm talking about going into a code And by the way, if you're bowing that code is a Fortran code that has been started to be written decades ago That is thousands and thousands of lines of code and someone's now telling you you need to find these matrices and these operators And then you need to project them. It's just just too complicated And you know this well because we battled with it with a with an Air Force code over over that so that then brings us to The work that we've been doing in operator inference, which is to say we said we're trying to come up with approximate models We have all this beautiful theory from classical model reduction That says if you start with a physics based model if you start with what you know and go from there There's a lot of structure you can embed and by embedding structure you can then Have theory and guarantees and predictive power that comes along with it But on the other side you have machine learning methods that are so attractive Because all you have to do is run those giant codes and collect data and then set the machine learning to work on it You maybe need to know how to how to run the code But you certainly don't need to go into the source code and extract all these operators that Non-intrusive nature of machine learning. I think is one of the reasons that has seen so much so much adoption So we asked the question. Can we can we try to get the best of both worlds? Can we bring the theory in the perspective of model reduction and put that together with a non-intrusive view? that comes from machine learning and Learn predictive reduce models and so the answer is yes, we can do that and that's the method that we call Call operator inference Where again, we go back to this observation that says if you are going to get the reduced models through projection and Today I'm talking about projection on a linear subspace. It preserves the structure So said another way if I know the governing equations I'm solving and I also should make a comment There are classes of problems where we don't know the governing equations and the job is to discover the physics And that is a really important class of problems, especially in biological applications I as an aerospace engineer are talking about problems Well, I don't need to discover the physics because I know what it is I might models may be imperfect, but I believe in conservation of mass momentum and energy I have to say I sometimes tell my children reminds me of Homer Simpson saying in this law We obey the second law law of thermodynamics if you guys have seen that episode, okay So in situations where we have a physical model that we trust enough and it's just too expensive We know what the physics are. We don't have to discover them and through this lens of Classical model reduction. We know the form of the reduced models. We should be looking for so Again, I really want to emphasize this point the form of the model that we're looking for is going to be inspired by the theory But we are not going to derive the models by the theory by that sort of Intrusive projection based approach instead what we're going to do is grab the data and machine learning style We are going to learn we're going to solve an inverse problem to learn the operators Are they structured form of a reduced model where the structured form of the model was presented to us by the by the physics? Okay, so that's the operator inference. You're going to start with a physics based model You're going to use this lens of projection. You're going to write down on a piece of paper I am looking for a reduced model that has this form the form dictated by the physics and then you're going to pick up a Machine learning viewpoint and say can I take data now from training data from these simulations? Or by the way, we could also be taking data from experimental simulations throwing it in the pot and learning The reduced sort of model which is describing the evolution of the coordinates on this reduced sort of subspace And as with all great things in life, it's all just linear algebra. There's something else I tell my kids all the time my kids think I think I'm a little crazy That's right, they're required to love me. That's right It's all just linear algebra this notion of learning the operators amounts to the Optimization problem that you see here. So the first thing and for the example here we are learning a reduced sort of model that has quadratic form and Again, I said it was a topic for another talk. It turns out there are many many non-linear Partial differential equations that you can actually write in quadratic form through clever manipulations of the variables But let's think of this example of quadratic form So we've got to solve an optimization problem to find the operators the matrices a hat b hat and h hat those are the matrices the operators of the reduced sort of model we are Driving it with data that comes from the simulation and you see these these x hat here these x hats here This is the data from the high-fidelity model, but projected onto the the low-dimensional man of the low-dimensional Subspace which is a very easy matrix vector multiply operation And what are we doing here? We are minimizing the residual Which is that in other words is we're taking the data from high-fidelity code with Substituting it into the form of the reduced model and then we're saying minimize the the sort of the fit to that model form and by Posing it in this minimum residual sense You can see that the things in blue are the things we know those are the data and the things we are solving for The a hat the h hat and the b hat. They're all linear So this is nothing but a linear least least squares problem It's a linearly squares problem at the at the end of the day now, of course There are many important tales that I'm glossing over in a talk like this regularization is really important this question of ill-posedness of this problem Benjamin Peer stopper my former postdoc and actually the original paper there was in 2016 with Benjamin had some really nice More recent theoretical work where he shows provable recovery Of the intrusive reduced model. Why is that important? Remember I said that those intrusive methods come with With a bunch of theory and so if you can recover the intrusive reduced model you can Get a window into some of those those guarantees Okay, a few things to emphasize. It's entirely non-intrusive All you have to do is generate the snapshots from high-fidelity simulation or from an experiment So start with the data set compute the proper orthogonal decomposition basis That's the way we identified this low-dimensional subspace. That is nothing but SVD SVD at big scale, but it's nothing but SVD on this set of snapshots Compute the low-dimensional representation. That's nothing but a matrix vector multiply and then solve the linear least squares problem To come up with the reduced operators You never have to touch anything about the high-fidelity model or the experiment that you are That you're trying to approximate So really is trying again to get the best of both worlds now, of course, we are giving up some of the theory To go non-intrusive like this, but what's I think really important is that we are keeping the physics We are we are we are embedding embedding the physics So to the young people out there before you reach for your neural network, you need to ask yourself Do you know more about the system you're trying to approximate? Is there a structure that may be more amenable to the physics and may give you more guarantees? Then the last note to mention is that everything because it's all linear algebra everything here is very very scalable and my former postdoc unit Farcast has been working on a distributed Algorithm of this where you can take now very very large-scale snapshots distribute them across processes and Do all of these computations at really really large scale so we really could start to learn reduce sort of models At the at the at the largest scale and you'll see a little bit of this in the results Okay, so in the last the last few minutes I Want to show you just one example and this is actually work that we've been doing under an Air Force Center of Excellence where Bill is a collaborator and also under another contract working closely with Air Force Research Lab These results in this work is unit Farcast's work unit has been a postdoc in my group And he has just started as an assistant professor at Virginia Tech So you can see the the setup here. So we're in this case. We're modeling a rotating detonation rocket engine So the high-fidelity model here is alias simulations of the reactive viscous 3d Navier-Stokes equations And these are horribly horribly horribly expensive simulations And I really want to emphasize that we UT Austin we don't even touch the code AFRL is doing all the simulations all the CFD simulations and Generating the data and we are interacting with them at the level of the data So it really is truly non-intrusive and I also want to emphasize that as much as you know We would love to do intrusive reduce models. It's just not practical practical to create reduce sort of models with this kind of complexity of of Code especially in commercial and legacy codes Okay, so there are those those governing equations again So let's walk through the steps. So the first step again is generate the training data and again We're not even doing this part of it the AFRL is doing this part of it Again, just to give you a sense of the scale of this problem the alias simulation We can simulate what they are simulating for two milliseconds And by the way, the desire is to be able to simulate for much longer, but the computations are just too expensive the full Engine has a hundred thirty six million degrees of freedom and the spatial Discretization and then you can see there just some of the the CPU hours That it takes to generate two milliseconds two milliseconds worth of data Turns out the and this by the way is a another sort of really interesting thing I've come to appreciate all this effort and energy goes into running these huge Simulations these huge simulations But then the data are so big that people can't store them and so they don't even store all the data that comes out the data that is actually available are dramatically down sampled both in space and in time and I think this in itself is a really sort of interesting question for those of us who are Working in these areas, so we end up having 501 Snapshots, so that's 501 time instance And it's also down sampled in space to a four million degree of freedom grid right there are 18 We use transformations 18 state variables in our model. So pressure temperature velocity is chemical species, so it's 18 times four 72 million degrees of freedom even in the down sampled and we have 500 data points and Yes, you could fit a neural network to this data But you really would have to stop and think about how many degrees of freedom you have to fit and How much information you really have to fit it and whether you would have anything that would be Predictive in any sense. Maybe by the time you've done training your hyper parameters Okay, so we have our training data snapshot matrix is 76 million by We use 375 Snapshots for training. We're going to keep the rest back to see how the the method goes We now come in compute the POD basis again at this point. It's linear algebra Again, there's a really important details about scaling Especially in a problem with so many different physical quantities you have to be very careful We compute the singular value that define the singular values and the singular vectors define that low dimensional subspace The singular values are an indication to you of just how reduced your reduced order model can be I always tell my students don't ever walk into my office without a plot of singular values in your hand The singular values tell you so much about the complexity of the problem the complexity of the data set You're working with and just how much you could expect to to reduce it You know, it has been working all of this in parallel and front era, which is the NSF leadership machine It's the the plus a supercomputer on an academic campus in the country at UT Austin. He's scaled all this up to To more than a thousand processing units and it's very scalable Okay, so we've computed the low dimensional subspace now we can infer the reduced model again at this point It's nothing but but linear algebra. So let me show you just a few Sort of snapshots of what it is we can do and Again, if you take anything away from this talk the next sentence I'm gonna say you should take away which is there is no magic Despite what you might read in the newspapers or even some academic papers. There is no magic you do not get to go from 76 million degrees of freedom down to 24 degrees of freedom and get perfect solutions And if anybody tells you they can do that you should become very very skeptical The reduced sort of model has 24 degrees of freedom. It is completely decoupled from the CFD It is going to run in a fraction of a second Really really fast the CFD even down sampled 76 million degrees of freedom. You saw how how expensive that is You are not going to get everything. There is no magic. What you're looking at here are three different implications in this engine one close to the injectors one within the detonation region and one downstream of the The detonation region. We're looking at pressure traces with the CFD in black and the reduced sort of model in orange And what you see is the reduced sort of model kind of gets it, right? It gets the right frequency it sort of gets the right amplitude it gets the bulk the course behavior Really pretty amazingly. Wow But it does not get the details Does not get the high frequency either in time or space as you would expect going from 76 million down to 24 degrees of freedom And I should say the top row there is at the end of the training horizon and the bottom three rows are Now moving out into the simulations well beyond the training regime It's kind of hard to visualize these kinds of results But I'll say I think this problem is really at a scale size and complexity of physics That goes beyond what people have been able to do with reduced sort of modeling and getting this kind of Predictive performance just a few more ways again. It's very very hard to visualize This is another really important area of research is Visualization of all the incredible stuff that we are computing What you're looking at here now Time is going from left to right so training in the first column beyond training in the last three columns on the top the CFD sort of full colors solutions of pressure and then Picking out a narrow band of pressure with those red contours the full CFD and the reduced sort of model And again, this is to emphasize there is no magic the reduced sort of model is not going to get it perfectly But what you're seeing is that the reduced sort of model is doing a really good job of predicting Kind of the the large-scale the general dynamics of what's going on and again We're working very closely with the Air Force to understand what is a designer looking for what would a designer want to see? Being predicted well in a reduced sort of model so that this reduced sort of model could be useful in running many many simulations and supportive design As opposed to one high-fidelity simulation that would take would take weeks So we're feeling really happy happy with these these results Okay, let me oh, I've got one minute just to just to close up So again just to summarize our take on operator inference again given a physical Natural system with non governing equations and a set of data and fur reduced model the end of the day It is all linear algebra. It is all linear algebra. There's some of course non linearity in the physics We use the physics to define the structured form of the model We use the theory and the lens of projection base model reduction to cast the inference in reduced coordinates We use inverse theory to analyze the structure of the inverse problem to understand What writes how big of a reduced sort of model can we solve? What guarantees can we make about this inference problem? and then we use numerical linear algebra to us to achieve efficient scalable algorithms and I would encourage you all to not take away Operator inference, but take away this general principle that as we start talking about Machine learning and how we combine physics-based modeling and machine learning to make sure that we're not just slapping physics into a loss function as a penalty term and hoping that somehow the neural net is Matching the physics but to really think about the physics and what structure that defines and bringing the physics in as a central part of the models that we bring Of course, there are a range of use cases There are many cases and to go back to what I said earlier where you have tons of data And we're black box machine learning is going to get you to what you need There are other cases where you can really afford to go a more mathematical route to go reduce water modeling We're really been asking the questions as to how we can can land in the middle And then lastly, I just want to conclude and just pop up a couple references So Boris and Benjamin who are now both faculty members both former postdocs and worked on a number of the work that I showed here. We recently just authored a paper an annual review of fluid mechanics that talks about these operator inference problems It just came out last month Unit there. I mentioned him in his work with the combustion Is in the final stages of submitting a journal paper as soon as his his advisor gets around to the final round of edits? I didn't have time to talk about it today, but I talked I emphasize the structure that's preserved through the linear subspace We have some I think really exciting work together with Rudy Healan who's a postdoc in my group and Looking at quadratic manifold so that you have a richer approximation Space and it turns out that many of the things that I said about preserving structure and knowing the form of the reduced model Carry over if you work into a if you move from a linear subspace into a quadratic manifold So again before you reach for the neural network. There's so much fertile ground In the middle there with more structured yet powerful approximations and then lastly again. It was hard to know what to talk about I'm really excited about the work that we're doing in digital twins We've been working with probabilistic graphical models really emphasizing the role of uncertainty quantification in digital twins and Anubhan in particular has been leading the charge on our collaboration with the oncologists where we have taken the foundational methods that we developed for Structural health monitoring digital twin for an unmanned aerial vehicle and brought them over to that cancer setting that I motivated in the in the beginning All right, I'd also like to thank the funding sources and then a very last plea which is If you haven't seen this my son is on a campaign You must feel sorry for my son because you've already heard that he is subjected to all these that's I don't know Conditioning about linear algebra. We are almost at a million views on the TED talk So if you haven't watched the TED talk, please watch it But in all in all seriousness again, I think a really important thing that we as engineers as computational scientists as people Who come from a background of physics based modeling have to do is to make what we do accessible to the public and talk about the importance of Models together with data so that it's not just a I'm machine learning all the time But it's a I am machine learning together with the incredibly powerful rich base We have of physics based simulation optimization design and control and I really tried to do that in that in that TED talk Okay, so with that I want to thank you all for coming and for listening to me talk when there were no slides And I guess we'll have time just for a couple questions. Thank you Hello, thank you for answering my question I just had a question about the advantages of using digital data versus the physical experimental data to actually base your The reduced order modeling on and then eventually the digital twin because I noticed when we were talking about the rotational Detonation engine. We only got a couple of microseconds of data versus is it certain like the The amount of like you would get more time using the physical simulation Versus less fidelity on the You know, I think I think that the surface of all is a real opportunity space which is simulation data and experimental data and Maybe you have a sense at least in the way we're approaching the problem there is no reason why we couldn't put the two together and I think this is this is a very good question to ask for this example the simulations are so expensive and you have such a Tiny window in through the simulation data, but if you were to talk to an experimentalist you would Hear from them that in this particular example the experiments are expensive and you know the kinds of data that you can collect are extremely limited as well and You know, one of the nice things about the simulation data is that you get the full full state, right? We get pressure and temperature everywhere and again talk to an experimentalist. Yes, you can have things like PID Particle image velocity Techniques that give you fields, but they're not going to give you sort of this clean data everywhere like you'll get from a simulation Now, of course the experiment is the real world except that it also isn't the real world It's also a surrogate of the real world the simulation is probably even less the real world So there are real strengths pros and cons to each and I think moving towards figuring out How you can leverage both simulation and experimental data and maybe even close the loop so that you're Understanding of these very expensive simulations and these very expensive Experiments, which are the ones that you should run to best inform The kinds of models or ultimately the kinds of decisions you want to make Yeah, we're very we're very interested in also using experimental data But again, I guess this is my naivety as a computational person I just sort of assume that they measured pressure and temperature and then you start reading the papers and they're measuring things that I have never heard of And then I'll ask, you know how and I don't even want to start saying them because I'm probably going to pronounce the words wrong But these luminescence How like how does that relate to temperature and then it turns out it's like a very complicated relationship? And so you would have to build that in so We often think of experiments of being measurements of just what we want they almost never are But this in itself is a really interesting opportunity And now the experimentalist can correct me I Thank you for the talk again So in the example that you gave the full-order model had 76 million degrees of freedom and the reduced one had 24 and I think that's sort of an extreme case, right? But I was wondering how the accuracy of the reduced order models scale with the order yeah, so One thing I sort of skipped over I was going fast at that point is Because we only had we had 500 snapshots and we use 375 snapshots for training we and Again, I'm sorry. I'm sorry if they're machine learning people here and you're feeling attacked but We don't believe that you can fit more parameters than you have data points because that's an undetermined system And yes, you can do it and you can get an answer. That's not real and So our size of the reduced model is actually limited by the amount of data that we have We could fit bigger reduced models, but again, it's not real You don't have the data to inform and we're forgetting we're fitting quadratic reduced models So we need you know 24 degrees of freedom for the linear term and then something that scales with 24 squared For the quadratic term, which is why we come up against that limit of 24 So we are in the low data regime for this kind of application Of course if you had more data you could fit a bigger reduced model and you sort of asking the question How does the accuracy? scale You can you can get more and more, but even though it's an extreme example I think it's a really good example of what you get with reduced modeling Which is you do not get the fine-scale detail You're not going to get all the and if you were to dive into those CFD pictures and see the very fine Scale detail of what's going on and the pressure and the temperature fields You don't you're not going to get that and if you do get it You probably have a reduced model that's overfit and will not predict what goes on later But what you can get are these sort of lower Larger spatial structures. You can get the frequencies. You can get the amplitudes You can get enough for it to be useful in Some kind of a setting particularly when combined maybe with real-time data in a digital twin So I know I'm not really exactly answering your question But I think it is more than just making the reduced sort of models big and again We could we could probably fit the data exactly But you would be looking at fits by hyper parameters You wouldn't actually be looking at a reduced sort of model And I think this is something to be very very careful with with neural networks is What you're looking at maybe one more question I guess I'm going to ask about the linear algebra problem, right? So you when you solve for the a heart b hat you assume you have a unique solution because it's linear or quadratic What happens in general in solid mechanics is that this a and b matrix are going to depend on x2 Right there. You don't know if you have a unique solution, right or not Yeah, so so I didn't I didn't really talk today at all about non-linear problems other than the ones that have the particular quadratic form So, you know first to say something I said earlier Which is actually turns out that there are clever tricks you can play with the variables that you use in the reduced model That allow you to write many many systems in quadratic form Even though the partial like the the governing equations are not and that's because we can we can Sort of play games with what we're representing in the reduced model state So there are many cases transformations that can get around that And then there are other cases where you cannot get around it or with or getting around it Causes you to end up with algebraic equations and all the things that create a mess And these are some of the things that we are looking at You can also sort of go through the notional approach that I Layed out, but then you don't get this nice clean linearly squares problem at the end you end up with a non-linear problem and then you could still Solve an optimization problem to find a reduced model. You might not have a unique solution You might not be able to guarantee So yes absolutely and for sure there are classes of and I should also say there are classes of problems That are not amenable to approximation in a low-dimensional subspace or a low-dimensional manifold like I've described today and Those are and there are classes of problems for which a neural network Approximation that maps many to many in a very sort of compositional and hierarchical way is the right approximation Technique, but there are also many classes of problems that are amenable to these kinds of approaches And so we shouldn't we shouldn't reach for the neural network before we have thought through linear approximation quadratic metaphors Wonderful. Thank you very much again for giving us a great presentation and being here today. So help me. Please. Thank you