 Hello, good morning or good afternoon, depending on where you are. My name is Pedro Ardueno. I am a professor at the University of Washington, and I am a member of the COGGI committee, and I will be today the moderator for this webinar by David McKellan. So let me first introduce a little bit COGGI, is the Committee on Geological and Geotechnical Engineering. And this is one of the standing committees of the National Academies of Science, Engineering and Medicine Board on Earth Science and Resources. COGGI was established as the focal point within the National Academies for Government, Industry and Academia on technical and public policy issues related to the Earth processes and materials, soil and rock mechanics, responsible human development and mitigation of natural and human hazards. If you have questions about COGGI, please contact Samantha Maxino at the National Academies. She is the staff director of this committee. This webinar is part, actually, of a quarterly webinar series that is produced by COGGI through the support of the National Science Foundation. And for your benefit, actually, the webinar and others will be posted on YouTube. The announcement will be sent out when this is available. First, I would like to thank Samantha Maxino, Kearney Devain, Sarah Hidrich and Mandy Enriquez for helping organize and produce this webinar. If this happens, it's because they are behind the scenes working on this. Dr. Marty McCann will assist me with fielding questions from participants following the presentation. For this, the audience can submit their questions anytime using the Q&A tab on the Zoom panel on their screens. Dr. McCann will pose as many questions as possible and as time permits. First, a little bit of a disclaimer. Any opinions, conclusions or recommendations expressed by Dr. McCallan or anyone during this webinar are those of the individuals and do not represent conclusions or recommendations of the National Academies of Science, Engineering or Medicine. So, enough of COGGI, now let's introduce our speaker, David McCallan. So, Professor McCallan, and we are very fortunate to have him, is the director of the Center for Civil Engineering Earthquake Research at the University of Nevada, Reno, UNR, and a senior scientist in the Energy Science Divisions at Lawrence Berkeley National Laboratory. In his capacity as center director, he's responsible for oversight and management of the state-of-the-art experimental earthquake simulation shake table laboratory at the University of Nevada. Professor McCallan maintains an active research program and is currently the bestigator for the U.S. Department of Energy funded EQSIM Advanced Earthquake Simulation Project jointly executed by Lawrence Berkeley Lab, Lawrence Livermore Lab and the University of Nevada. Dr. McCallan's research interests span from high-performance simulation on earthquake processes to advanced sensor and communication systems for structural health monitoring. We are really happy to have you, David, here today at this webinar. And now I offer the floor to you. Thank you. Wonderful, Pedro. Thank you so much for the kind introduction. So, let me ask the two most important questions for our webinar. Can you hear me and can you see me? Very good. It looks like that's the affirmative. It's great to be with you today, and it's even greater to represent the Earthquake SIM team. We have a really nice team from the University of Nevada, Lawrence Berkeley National Laboratory and Lawrence Livermore National Laboratory. It's really a combined effort, and I'll explain more about that team as I move through this because they deserve a lot of the credit for the work that I'm about to show. Secondly, I want to acknowledge our sponsor, the U.S. Department of Energy Exascale Computing Project. I'll talk about that project and how important it is to moving my performance computing forward. Not just for this project, but writ large in the nation. I think it's quite important. So I'd like to go through that as well. So let me begin. I'm having trouble advancing my slide. Ah, there we go. So let me begin by sort of framing what does this regional scale simulation nomenclature refer to. And I'm really going to talk today about something that I think has been subject to growing interest over about the last decade. And I think that interest is really accelerating. And that's the notion, as you can see here, if you have a region of interest where there is an earthquake fault and there's infrastructure and a large domain could be hundreds of kilometers large, it would be a very compelling and I think undeniably compelling idea. If you could simulate the earthquake processes in that domain, the entire earthquake processes on a large computer. And I think for a couple of reasons. Number one, as we all know, earthquake science as much as it progressed is fraught with uncertainties. It's a very, very difficult proposition to be able to predict the earthquake emotions from a future earthquake with high fidelity and get the site specific motion. So number one, wouldn't it be compelling if we have a three-dimensional model that we could demonstrably show, compute and demonstrably show the ability to, number one, predict the relationships between various processes, parameters like the fault rupture and the location of the epicenter, how the fault ruptures and so forth with the resulting ground motions so that we could really draw those sort of cause and effects between earthquake parameters and observations. That would be very, very empowering on the one hand. And that's sort of in the vein of, there was a famous quote by Richard Hamming who was a really, really famous early computational scientist. And he said the purpose of computing is insight, not numbers. And so the first application I think of these high performance computing capabilities that would be very, very powerful and compelling will be just understanding their earthquake processes more than we do today. We have observational data for ground motions that we use to inform our empirical ground motion prediction models. That data tends to be quite sparse. It's quite sparse now and it's going to continue to be quite sparse going into the future. So just first and foremost, it will be wonderful if we can develop these, these models that we could run on big computers that could allow us to draw inferences between parameters and their response on the one hand. Secondly, imagine if those models were truly predictive from the standpoint of being able to numerically represent and quantitatively represent ground motions and infrastructure response. That would be a big deal. The barrier and the threshold for that is a little higher because not only do you have to demonstrate the relationships, but you have to demonstrate that quantitatively you can predict those quantities. But both of those objectives would be highly desirable going forward and has the potential to significantly reduce the uncertainties that we have in earthquake science and engineering today if those objectives were successful. So we had a peer workshop on this topic about three weeks ago and there were over 250 participants and 40 presenters from around the world. And so the work in this area is really accelerating and major progress is being made. I'm going to talk about one specific activity today, but there's a lot going on out there both in the earth science and in the engineering domains and it's really a growing area. I think you can lump research and developments in this area into two, I guess three buckets. Number one is the computational model development. These are undeniably large models. They tax even the biggest computers that we have these days. So developing efficient computational models is really, really important and that's the first bullet on the bottom. Number two, code verification. And this is often skipped over. These are complex models. They often couple physics and they often couple the subsurface and the above surface geophysics instructors. So code verification is crucially important and that's really making sure that the code is computing what we think it should be computing. These would be the models that are embedded in that particular code. And then finally the real difficult burden is code validation. And that's really the element of ensuring that the predictions that are being computed in the simulations are representative of the actual physics that are occurring in nature. And that is a steep burden, but all three of these are quite important. I am going to focus on the first bullet there today, the computational model development and tell you what we've been doing over about the last three years for EQ sale. You know, all of this effort really revolves around our ability to compute. And so I want to make a couple important points about the remarkable progress in high performance computing and then tell you what's coming down the pike in about a year or two within the next year through the U.S. Department of Energy and in fact as a major motivator at this work. What I'm showing in this plot on the vertical axis is a high performance computer and you can just think of this simply as flux floating point operations per second. And you can see that those things go up as a long scale by orders of magnitude on the right is the year and the little red dots represent the world's fastest computer in a given year as judged by this organization that evaluates the top 500 computers. What do we see here immediately? A couple of things. Number one, as I've noted in the title, this curve is up into the right. You know, for the last few decades it has just been this tremendous increase in computational power and every time one thinks that maybe we've hit a ceiling in terms of machine architecture and so forth, really, really bright hardware and computer scientists figure out another way. And so you can see as these dots progress we've had almost, you know, continuous progression for the last few decades. Number two, what you see is that, you know, there's a lot of people in this game these days. You see computers from Japan that dominate for two or three years. You see computers from the U.S. that dominate for two or three years. And you see computers from China that dominate for two or three years. So there's a real healthy competition for the world's fastest computer that really drives this. Another point I would make is if you see the U.S. machines, the U.S. Department of Energy is everywhere. And as many of you know, the U.S. Department of Energy is really the pointy here in terms of high-performance computing development in the United States. So what about the Exascale Computing Project? That's really DOE's integrated program to develop the next generation of high-performance computers. And they're aiming for the blue box in the upper right-hand corner. And if you look at what that is numerically, that is a computer that can calculate a billion, billion floating point operations per second. And I think it's not in my lexicon, but I'm informed that's a quintillion. So that may be a new word for you. But this is a tremendous advancement in high-performance computing that has been led by a number of organizations. So for DOE, within the next year, the first Exaflop or beyond Exaflop computers are coming online. One is the Aurora computer that will go in Argonne National Laboratory. And the other is the Frontier computer that will go in Oak Ridge National Laboratory. Both of these will be Exaflop or beyond machines. And DOE has been planning and developing these machines for about the past four or five years. And the first hardware, interestingly, for these machines, the first nodes have begun to arrive and people that are working on ECP are getting access to those. So in anticipation of those machines, the DOE Exaflop computing project has really been developing the systems and getting ready for applications on those machines so that when the machines arrive, they can be taken full advantage of. And there are three parallel components of the Exascale program. Number one, an ECP is Software Technology Development, the box on the left. And that is really developing the software stack, all of the I.O., all of data compression, all of the software that can really help applications run efficiently on this new generation of what will be new or graphical process unit-based machines. The second part of the DOE Exascale program is Applications Development, the box in red. And DOE selected at the time 24, that's grown a little bit, but 24 applications through a competitive process that would be science-based and really demand and advance a particular field by utilization and exercising these machines. And we happen to be one of those that was selected. And I'll mention the details of the EQ Sim project in a moment. And then finally, the Exaflop computers that are really going to arrive in 2022, about a year from now and be available. And so all of this, you know, three, four, five years of work has been vectoring towards being ready to exploit these very, very powerful new machines. So what is the EQ Sim framework? So we proposed sort of the audacious goal of simulating an integrated fashion all the way from in a large region of interest, and you'll see what our region of interest demonstration project is, simulating all the way from the fault rupture, propagating those ways through a terribly heterogeneous earth, and then interacting those ways with infrastructure to determine how that infrastructure responds. So it's really an integrated what we refer to as end-to-end vision. We have to have geophysics as part of that. We have to have engineering as part of that. And really two key issues that we propose to explore through these very advanced simulations is, number one, how do earthquake ground motions actually vary across the region? And how does this impact risk to infrastructure? Number two, how do complex, realistic incident ground motion waveforms actually interact with a particular facility? So by that, we mean we would like to be able to predict and simulate ground motions for a particular fault over a particular region, for example, the San Francisco Bay Area, and understand the time and spatial evolution of those ground motions and what the site-specific ground motions look like in every location within that domain. And so that is really one of the ultimate objectives of our development of this particular framework. Number two, and we think this is important from the engineering standpoint, how do complex, and I'll say, realistic incident ground motion waveforms actually interact with the facility. And so as I'm sure many of you know, through the years we have developed idealized and simplified models to look at this complex interface between the subsurface and the surface. We often do 1D site response calculations that's only truly consistent with an assumption of vertically propagating shear and compressional waves, and it puts pure translational motion into a structure when we do that type of analysis. But in reality, for many sites we know and have intuition that the site response is truly three-dimensional. We are subjected to a particular site to surface waves, to inclined body waves. The ground can have translations plus rotations. Through the years, people have looked into these particularly complicating aspects with simplified models, but we think we can do a much more comprehensive job at evaluating some of these things. So those are the two things that we look for in the development of this model that we can actually address at the end of the day. I'll just note this is a terribly complex problem. In many facets it's multidisciplinary. We have to go all the way from representing a complex earthquake source that ruptures in a complex fashion. We have to worry about simulating seismic waves that are propagating through a terribly heterogeneous geologic structure. And then finally as engineers we have to worry about the convergence of these waves on our engineered system and how those things interact. So this problem is large. We often have to represent hundreds of kilometers of extent. On the other hand, it's multi-scale because we have to understand how things behave on a fraction, maybe a hundredth of a kilometer, and it's regional and character. So this is both a disciplined and a computational challenge of the highest order. So you really need a village to solve this problem appropriately. And so we have assembled a multidisciplinary team that you see here. We have people that are steeped in engineering mechanics, both structural and geotechnical engineering. We have people that are steeped in applied math and numerical methods because we have to have the most efficient algorithms as we run these codes at this scale, the scale you're going to see in a few moments. And we have to have computer scientists that can really help us run in the most efficient fashion on these very, very large-scale architectures, and particularly the new architectures that are coming down the pike. And of course we need seismologists. And so I don't have time to introduce all of these people, but I want to point out in particular Anders Peterson who leads the SW4 development, the geophysics code. I want to point out Arben Patarpe and Artie Rogers of Livermore that do a lot of the analysis and set up the analysis for the seismology project. And I want to put a special star by Hujun and Romesh of LB&L at Livermore Lab because they really help us make sure we stay on the up-and-up in the computer science realm. We also have a great cadre of postdocs and graduate students from the University of Nevada and also from Berkeley. And honestly, they get to do all the fun stuff in this project. We are developing the software. They get to exercise the software and look at what the software is telling us. We really have a broad and deep team and I always say it kind of takes a village to develop and perform these kind of calculations. So let me talk about our exascale project in one more detail. DOE doesn't just hand every DOE exascale application of power money and say go do good things. We have to at the start of the project define a challenge problem and we have to have a clear objective of where we want to get to in that challenge problem as a result of execution on these exascale platforms. And so I'm just going to share with you here how for EQSIM, we defined our challenge problem and what our exascale goals are. So we decided to use the San Francisco Bay Area that you see on the left is sort of our numerical laboratory and we want to do regional scale simulations. So we picked a domain that is shown by this bounding black box and that encompasses the Hayward Fault where we're doing some of our simulations and it encompasses the entire urban area. That's about 120 kilometers by 80 kilometers by 30 kilometers deep. In terms of our objectives for EQSIM and where we want to get to that shown in the plot on the right and what I'm showing in this plot is the simulation of one earthquake realization of a Hayward Fault rupture in that San Francisco domain. The vertical axis is the execution time on the computer platform to run that simulation and the right is the frequency that we're going to resolve. So historically we've been right up here in the left hand corner and that's sort of where we are at the start of this project. We might be able to do historically maybe a two-hurt simulation or a three-hurt simulation of those ground motions and maybe run on 25 hours of computer time with the fastest computers that were available. So we have a pose that we will do a 5x and 5x increase and so we would like to increase the frequency resolution of our regional scale simulations by 5x. That is we'd like to be able to simulate to 10 hertz so that we encompass more frequencies of engineering interest and we'd like to be able to do that fast. We'd like to be able to do that kind of calculation in maybe five hours at the end of the project and the reason for that is we have to explore the parameter space. One heroic calculation of an earthquake simulation doesn't help us a lot other than demonstrate the computability. This is a big challenge. We knew going into this we were going to have to have advanced algorithms, optimization of our codes on the hardware and of course utilization of exoscale platforms. The formula I'm showing on the bottom shows a little bit of the character of this computational challenge. If you think of a model like this the computational effort is proportional to the volume of the model how big a domain we're calculating the earthquake duration. So for the Hayward fault we're running 90 seconds of earthquake duration. And then it can be shown that the computational effort varies as the frequency resolved divided by the minimum shareway velocity resolved raised to the fourth power. And what does that mean? That means it makes it very very difficult to climb this frequency mountain of increasing the frequency of these simulations. That's really a really really big challenge. Additionally it means it's hard to resolve soft surface sediments that have very very low shareway velocities because that increases your computational effort tremendously. And so you'll see in a lot of these traditional models they've often imposed a v-s of s being cut off and only gone down to maybe 500 meters per second. But this sort of is our definition of the exoscale problem. I'll return to this curve so sort of keep it in mind when I show you what our actual advancements and performances have been. So what have we been doing then over the last three or four years? Over the last three or four years we've been advancing tremendously the SW4 geophysics code. And this is a fourth order in time and space code originally developed by Peterson at Lawrence Livermore National Laboratory. And we've been making a large number of improvements and advancements in that code. I'll go through a few of them. But you know if you think of a three-dimensional domain we now have the ability to impose some of the latest characterizations of stochastic kinematic rupture models. We have very efficient super-grade silent boundaries. We can represent surface topography. We can include spatially correlated stochastic fine-scale geology when we move to higher frequencies. And we've got some very, very important recent improvements in our meshing of that domain that prove absolutely critical to moving forward our calculation. So I'll mention those in a little bit more detail on subsequent slides. So we've done a lot in enhancing our physics for the fault rupture model. And I think this is an area where the interaction between earthquake engineers and seismologists has really paid off in terms of making some tweaks and improvements to the Graves and Patarka rupture model. We've done a lot of additions in terms of including things like a deterministic rupture patch. We've done a lot of checks of realism in terms of the validation by looking at, with respect to the rupture models what do our predicted ground motions look like vis-a-vis observations. So that's been a significant part of the effort. In terms of computations we have really had to redo the IO throughout our framework. And when we get up to running two and 300 billion zone problems the IO can just eat you alive. And anything that is inefficiently done in terms of reading large data and or writing large data out becomes immediately apparent. So we have translated all of our IO to what's called the HDF-5 tool. That's a tool for handling very, very large data sets efficiently that's being developed as part of the Exascale S&T project. So we've really made use of that to increase our IO and we'll see some examples of that. And then finally when you're computing these models you really have to optimize your model structure to match the properties of the earth. And if you think of the earth properties the earth starts off with soft properties near the surface and gets stiffer. And as a result of that you tend to have short wavelength waves in the near surface soft layers and long wavelength waves at depth. You'd really like to exactly match your computational grid to accommodate that type of variation. So we have recently taken the SW-4 grid which is a combination of a regular Cartesian grid at depth and a curvilinear grid to represent surface topography at the surface and we've optimized that gridding to really try to get optimally eight grid points for wavelength throughout that domain. It turns out that that was an extremely important optimization of the computational framework and I'll show examples of that. And then finally we've had to really optimize the code and get ready for execution of our framework on GPU based computers. So we use MPI tasks and we divide this very very large domain which currently can be up to 300 to 400 billion grid points and we divide that into pencil domains that we distributed across a massively parallel computer like the Summit Computer at Oak Ridge. So a lot of work has gone into that of optimization as well. What I'm going to show you here is a simulation of a Hayward Fault Rupture in the San Francisco Bay Area and I'm going to show you the most recent benchmark results from last year and this is a 63 billion grid point simulation. This calculation is now to 10 hertz so we have achieved the 10 hertz goal in this simulation and you can see the complexity of the waves. What I think is important is to compare where we were in FY19 in this table with where we are in FY20 primarily as a result of this improved gridding. So in FY19 the simulation took 203 billion grid points and it took 20 hours to run. In FY20 we advanced to 63 billion grid points and it ran in 7 hours. So we had almost a factor of 3 increase both in compute time and decrease in grid points. So this is just I think emblematic of the kind of effort you have to do in developments to really make these codes sing and get optimal performance of these codes. So I showed you our objective up here in goal a few slides back. Let me just show you our progress in this particular slide and so we started here on the Cori computer at Lawrence Berkeley National Laboratory and we began optimizations of our algorithms that got us down to D. We jump to the world, then the world's fastest computer summit. It upgrades the GP based machine in FY19. That got us here and you can see we achieved 10 hertz in about 20 hours then we progressed all the way down to here with that algorithm development. So we've gone from here to here in the last three years and I only have to qualify this by saying that we did this with the VCMS min of 500 meters per second. We want to get lower than that and so our next next task is to try to maintain this level of performance but drive these of this min down to maybe 200 or 250 meters per second to accommodate the soft sediments that are prevailing in areas like the bay margins of the San Francisco Bay Area. So we've made great progress but we have more to do. Let me speak quickly to the coupling which is so important. I've focused on the ground motion so far but I mentioned early on our desire to really understand with intimate detail the coupling between ground motions and infrastructure at the subsurface surface interface. So we have adopted two types of coupling between geophysics and our engineering models and what I'm showing you here are local engineering models separate from SW4 and just think this could be open seas or any other code, fund element code and we have an option for weak coupling where we can just take the surface motions that we get in our simulation and apply those directly to the structure like we've always done in fixed base analysis uniformly and of course that is predicated on the assumption of vertically propagating waves. On the other hand to represent this complexity of interaction we have drawn upon the domain reduction method originally developed by Bialac, Carnegie Mellon to allow us to rigorously couple a soil structure model with the geophysics model and if you're not familiar with the DRM method in essence develop surface tractions on this boundary that then can be applied to this model that allows one to just run the submodel and do a soil structure interaction but also to represent the full breadth and complexity of complex incident seismic waves. So the domain reduction method is very effective for our operational workflow because it allows you to take this conceptually coupled model and turn it into a two step process. Do the geophysics simulation, save all these motions near the surface and then subsequently apply these to the domain of interest and so we have implemented and tested this. I will also note that one of the challenges in these models to represent soft and maybe nonlinear geotechnical layers so the DRM method can be used for a local soil structure model but in our view and we're testing it can also be used to represent a near surface geotechnical layer as well so it's a very powerful technique to implement in this workflow. Don't have a lot of time to go through this in too much detail but we have embraced the DRM method to make that work in this large scale workflow we've had to develop an interpolation scheme and ours is spline function based that allows us to interpolate between the grid point nodal values of the large scale SW4 model and the local model engineering model so we've developed that we've also had to compress the data sets coming out of these very very large SW4 models because they can be literally tens of terabytes of data so we have used a ZFP compression algorithm to really make this work so we can compress those near surface details of the geophysics model and extract those upon demand. This has led to a workflow QCM that looks something like this we have a regional geophysics model we can run these very very high fidelity simulations we can save a volume of compressed data from the geophysics model on our computers that's compressed in literally thousands of stations and then we can come back at our leisure as engineers and fetch that motion to either run fixed base models or to run fully coupled models that we would like to navigate. So I'm going to show you a graphical example of what this completed process looks like in the upper left hand corner you're going to see the fault rupturing and the waves propagating you're going to see a trace in the lower left hand corner at the location of that red box where the structure resides and then in the right hand side you're going to see the time sink response of a soil structure system that is getting the appropriate incident waveforms capturing surface waves and inclined body waves into that system and if I can run this again you can maybe just focus on the structure you can see that that DRM boundary is really working hard in the near field because not only do we have dynamic displacements we have got those permanent static displacement associated with fault offset. What you're seeing in the color scheme on the right is a visualization engine that we've generated to show the damage potential in that building is a function of peak interstory drift where green is a linear elastic response yellow is mild nonlinearity orange is significant nonlinearity and red is extreme nonlinearity and so this is an example of the type of capability that we can now execute having linked all these systems together. In the few minutes I have left I'd like to just go through an application example sort of in the end that illustrates a little bit of in my view the power of this methodology and I'm going to revert back to the animation I showed on the first slide ahead of the offset. This is a simulation of a relatively simple sedimentary basin and rock below that looks like so the cross-section AB you can see that this is the fault rupture and we've imparted the rupture with a graze-patarca rupture model starting here you can see the seismic waves propagating away. As many of you know the data on ground motion and infrastructure response in the near field is very, very sparse. You can sort of look on one hand almost and count the number of records that's a bit of exaggeration but not too far of a stretch of really good data on response in the near field. So we have gone through and generated this system to try to A get a very, very high fidelity look at ground motions and structure response in the near field and so first of all we take these results and we scrutinize the simulation model results both qualitatively and quantitatively and so you can see the rupture progressing here so the first thing we do is just look at these ground motions and ask ourselves do they look like ground motions we would expect in the near field and I think with this example looking at point A, B and C the answer is yes we tend to see a nice fling step associated with the fault offset and then in the fault normal direction we tend to see a nice fault normal pulse due to directivity effects so things qualitatively look like they're working pretty well. We've also done an extensive comparison of ground motions we would predict with this model and there are literally thousands of motions we generate in this domain and compare those with the real records to try to get an understanding of whether those compare well and in fact they tend to compare well and we'll have a couple of publications out on this that I'll reference one can look at in detail. Then what we can do is we can go in and we can look at the response and in this case using the weak coupling fixed base and again look at the damage potential in these highly non-linear building models and you can see it really allows us to look at the evolution in the near field and how this damage progresses as you move out at different locations away from the fault and so these are things that we can look at and visualize and quantify and so what do we see with these models that we haven't seen before because we don't have enough data well number one I'm looking down now in plan view of that fault and I'm showing color contours of the peak interstory drift of a 20 store building located at each one of these grid points so we can understand the distribution of damage and I'm showing you here a little dotted box showing a box about 10 kilometers from the fault so what do we see we see a tremendous distribution and variability of building response in the near field number one and if you look in this box and you look at the max and min locations of peak interstory drift you see that things can vary the peak interstory drift in the building by up to a factor of 10 or more and so we see in the near field a tremendous variability and infrastructure response and in fact this has been observed in a lot of earthquakes where people have done post-earthquake inspection and very very similar buildings that are near an earthquake fault sometimes equidistant have very great differences in variability so this is allowing us to look very carefully at what sort of variability we have with high fidelity we also compare our results again with the data that does exist and what I'm showing you here in this left plot is the building response in terms of peak interstory drift at all almost 4,000 locations within that 10-story or the 10 kilometer domain and then we've taken the sparse data set that does exist for near field records within 10 kilometers and analyze the building with those as well and you can see that the trends and distribution between this high fidelity data set and the motions that we get from actual records in the peak interstory drift are pretty darn good agreement both in terms of the median drift 1.3 versus 1.15 as well as the distribution and so although we're getting new information by doing these types of analyses and new insight we're also trying to scrutinize and compare with data at every step to ensure that our models have a degree of realism and we also see in the near field for our tall 40-story buildings and our 20-story buildings we see that there tends to be a predominance of damage in the upper stories of the buildings and these are modern well-designed buildings contemporary buildings and we see a lot of high mode or if you want to think of it of wave propagation of the building whiplash effects in the taller upper part of the building so at this particular site and we do these planar building models we see the damage tends to be extreme in the upper part of the building and this is a plot of the peak interstory drift on the other hand in the lower story buildings we tend to see a lot more uniform distribution of damage throughout the height of the building in areas of very, very high response and so this is just a very quick example in the time permitting of the types of insight we can get by looking at these high fidelity simulations and if you have more interest in this there's the two papers out in the May 21st spectra that go through and talk about this particular simulation in much more detail so let me, I think I'm about out of time let me just end up with a couple of observations of where we're going and I know this is a whirlwind tour so our goal at the end of the day is to have an unprecedented compute engine for routine regional simulations. We want to transform the ability to do these types of regional simulations into something that's not heroic but routine so we can do multiple of these simulations in pursuit of informing hazard and risk so we like for example in the San Francisco Bay Area to be able to look at multiple fault rupture realizations, multiple realizations of our geology and compute in fast high frequency simulations and that's really why we wanted to achieve both high frequency to do these simulations at frequencies relevant to engineering structures but speed so that we can be able and capable of doing a large number of these simulations and really generating a large number of realizations and I think we're well on the path when we get to exoskeleton platforms of being able to accomplish this based on the work so far. Finally I'll end with the thought that one would really like to be able to get these types of simulations and data into the hands of a much broader swath of practitioners and researchers and not everybody is going to have the fastest computer in the world setting in their desktop or having access to it so we're talking with Pierre about a way of actually translating these motions into a form that the broader community can get their hands on and so where we're at on this is something that looks like this. One can envision computing very, very large number of earthquake scenarios compressing that data and then having a criteria for deciding whether that data is representative and accurate and so Pierre has a project with Dr. Patrona and Dr. Abramson where they're developing a four-part acceptance criteria for synthetically generated motions so we would apply that criteria and then store data for example on a peer archive in the data format that is compressed that we showed you and then we would fetch that data and the tool that we've developed to fetch that data so that people could access and utilize that data for actual application and so this in our view very briefly is a roadmap for moving from you know 400 billion zone calculations to getting something into the hands of a much broader swath of people and so it's an important thought process as we move forward as well so Pedro that's a whirlwind tour and I think I have run just a bit over so I would be happy to end and entertain any questions that might have bubbled up from there Wow Wow this was a good presentation I really really like it so you know we had a strong audience here which is very diverse and it comes from the geophysical world to the people that that use these motions on the daily basis for a small building or for a tall building or for a dam or a landslide so I'm trying to join a couple of the questions that maybe are related and the first one I want to ask you is about this connection with the seismology group and the acceptance of all this type of work and you have talked enough about verification and validation but I was in other meetings also where they said well but the problem is that you have the faults wrong we know much more about the faults we have many more faults now and this is the one that seems to be dominating these motions or people starting to talk about the origin of earthquakes all also talking about the difficulties in scattering, non-linearities that are much more than the frequency content that we can model what do you see that we are going in this connection with the seismology that's my question so look I'm not a seismologist but I've worked with seismologists for many many years so I would make the following comment as we try to understand some of these complexities more even more the motivation for having computational models that we can use to explore the space and understand how these various representations impact the observations and the ground motions as well as the structural response so I think you know the ability to compute these high fidelity simulations really opens up the aperture of the type of phenomena that we can embed in these models and test against observation so I think there certainly will be new discoveries out there as seismology progresses and studying the subsurface we know is difficult but if we can observe those phenomena and create what we think is a representative model then we can test those theories with simulation so that my answer would be you know a tool like this you know kind of going back to Hamings with the purpose of computing his insight can allow us to look at some of these things and evaluate some of these things numerically that you just can't do observationally we can't get sensors where we'd like to have sensors everywhere we can't have a dense array we can't have sensors all over on the subsurface to really get high fidelity but we can get some data but simulation really offers us another tool to go out and simulate these motions and aid our understanding and assist our understanding so I guess that would be the way I would answer that question so you really believe that because I had a question here by John Vidal for example that says do you really believe that the distributions of intrinsic and scattering attenuation as well as the near surface velocity that are not originating can be resolved at the fine scale that you are trying to to do because from that group of people they say this is way too difficult to do we don't see this possibility you think it's possible I think that you know we are there's no question that our ability to compute is outstripping our ability to characterize the geology at this point right the question is from an engineering standpoint I'm speaking as an engineer what do you need to do to skin this cat I didn't have time to go into this today but we have two two options in our tool set that we have employed and are working on to try to improve some of these geologic models in terms of capturing and scattering one is a spatially correlated geologic stochastic geologic structure and Arvin Patarka has been working on this for some time so to get that scattering one option is to embed stochastic geologic representations look at the sensitivity of that we're also developing a full waveform inversion tool kit as part of this using our forward simulations to allow us to do inversions and we're applying in the Bay Area now inversions of waveforms to improve our geologic velocity structure so I think there are avenues that you can pursue but my answer would be again let's pursue this through simulation space with these high frequency simulations and test the existence and the reliability of implementing some of these methodologies to see if it improves our calculations and gets us to where we need to get and I don't think we know the limitations of where we can get to frequency wise at this point and from an engineering standpoint engineers may view this differently the seismologists might engineers might not need to get wiggle for wiggle waveform matching they might want to get a spectra that is representative and so the burden of proof if you will and the burden of utility may be different from the seismology community than it is for the engineering community in that sense for that problem so let me jump now to the other extreme the seismology now let's go to the engineers yes because and you have over the presentation I think that you have tried to emphasize this but I think that it's good to reemphasize a little bit more who are the customers here who are the customers of all this work and how are they going to be using this maybe in your couple of last slides you have some of that do you see them doing 3D simulations of buildings using DRM or where are we in that respect do we need to go until 2030 or I can do right now a 3D simulation on my desktop using a bubble of soil well I think I think that will be a different answer depending on the engineer and depending on the criticality of the structure honestly right so we have designed this workflow and this is part of our you know I'm a pragmatic engineer at heart this is you know part of our desire to have an outcome that can help inform the practicing community so that's exactly why we've developed this workflow that I've articulated where all of this stuff up here really down to here occurs with a relatively specialized community that can execute these scale of models with the appropriate hardware and understanding the numerics to be able to do that and so if we can get this done and store these in a very very efficient fashion and I know this was a fast presentation to catch some of that nuance but but we see that we need to store this data in a very efficient compressed form you're not going to store this in the same way that you store pure NGA east and west data right now you have to store this in compressed form then there's an option for the practicing engineer to decide how they want to use these motions and that can go all the way from selecting a synthetic ground motion or motions for multiple fault rupture scenarios at a given site at a point on the air surface and using that in traditional models of whatever degree of sophistication they want to use all the way up to using the DRM where they want to account and understand for the appropriate coupling between you know the incident complex incident motions and the structure of question so this might be something with the nuclear power plant for example that has an embedded structure and in my mind you certainly would like to at least explore the options of understanding what complex ways do and look like and do to a particular facility like that and so I think the answer is there's many flavors of the use and I think if this is cast in the appropriate form the selection of the use case is left to the engineer who's doing the use that's my view I don't think we ought to force them into doing a DRM I don't think we ought to force them into doing a 3D model I think we ought to have flexibility so that they can use whichever flavor they like this is a nonlinear model a nonlinear planar model of a 40 story building that's quite commensurate with what one might use in engineering practice if they're doing nonlinear analysis I think they can gain a lot by utilizing synthetic ground motions to understand the response of that model particularly in the near forward regime so I think the answer is you know leave it to the user to decide so I have two questions there one is do you see new type of users that are not engineers for example after the New Zealand earthquakes I see that has happened also in other places all the insurance companies become the drivers for some of these things do you see some other I can imagine some other groups being involved could be political it could be social do you see anything coming in that end? I think the answer is certainly yes I think that's a possibility right if you're an emergency planner you'd probably kind of like to know hotspots where you might have to worry about after an earthquake if you're PG&E and you have substations sprinkled throughout the bay area you'd like to know the site specific characteristics of the risk at your site I think that goes without saying so those are all notional but they would be compelling notions and the question is how far can high performance computing get us towards those goals of really accurate site specific motions and now on the other end of the question I had there is as we produce more and more and more data I have the feeling that we are producing even more and more data and more and more data producing even more and more data who owns and who stores this data not today I am talking the next 10 years where do you see this going who is going to be managing who is going to pay for this? I think that's a good question and at the recent peer workshop we had a brainstorming session on this very very topic and I think there's a number you have to think about who would execute this in the future the national labs have the computer horsepower to do these types of calculations on a routine basis and store the data so that's maybe one part of the solution universities and other research organizations that can inform this type of model so maybe that's another part of the solution agencies that are stakeholders if you think of the San Francisco Bay Area you think of PG&E, you think of BART you think of East Bay Mug, you think of Caltrans one might envision a subscriber service where people might be participants to co-support some sort of effort to do these types of regional scale simulations that could inform the entire community so I think the methodology for storing and saving these motions and making them accessible is clear I think the roadmap to how we would ultimately support operationally these types of calculations is less clear at this stage and I think that's an important thing to discuss because that as these simulations progress we really need to have a roadmap a strategic roadmap with how to handle this information going forward so I think that's something that really ought to be talked about now you know with a vision for the future in my view Yeah and I see a lot of talks in other fields also who owns the date what is the role of the universities that are generating a lot of these things or national labs and the problem the funding is an important There's an operational cadence that's associated with such models that is not necessarily copacetic to a university research environment in my view there is some deliberate Q&A and operational activities that would have to be part of this and so it's hard to pick a single organization and say that organization is it but I can think of a number of organizations that could feed this and make compelling contributions to this I have several other questions here we have only maybe four minutes to go but I have several questions that are specific like for example can you handle liquefaction which has a shear wave velocity that is very very low as you can imagine can you model uncertainty quantification in building parameters or the engineering demand parameters can you model a soft size using your tool the question is geared towards now they want to use this tool they want to use it to say give me the tool what are your thoughts where are we now and where are we going that's kind of a final question that you can expand as much as you want great so let me speak to this for a moment I mentioned that one of the greatest challenges is to represent soft mirror surface soils where you can just screw up your time step and your element size and so forth so in my mind we ought to be expansive about how we think about the domain reduction method as I show in this slide if you can still see it and I think we could extend the traditional notion of the domain reduction method to help encompass a soil domain or a boundary where we may want to look at some of these processes and so forth I don't see a path to including all of these multiplicity of nonlinear soil models within the global geophysics model I think that's a fool's errand I just don't see that happening because there's just between essentially linear elastic model and all the efficiencies you have in the geophysics code mucking it up with all of this engineering mess to me sounds like a fool's errand but I can envision why you can allow for coupling of these models in a very efficient way so that you can represent some of this near surface nonlinear behavior that is so gnarly as a second fate as a step to analysis much in the way that we handle an individual structure and soil island in the domain reduction method now so we're in fact exploring this with one of our postdocs right now to try to extend the DRM method to a larger domain where we can envision encompassing a layer near surface nonlinear soil and representing the response of that soil so we're studying that a little bit right now this is great well we have several more questions that actually we are going to pass to you David so that you can look at them and maybe we can find a way to answer some of them or we can find a process I want to thank everybody that attended this webinar I think it was great your participation the level of engagement is very good I want to thank you David for a super presentation and before I leave I have to make the disclaimer that any opinions, conclusions or recommendations expressed today by anyone during the webinar are those of the individuals and do not represent conclusions or recommendations of the national talents of science engineering and medicine so with that, again I want to thank you David for a great presentation and thank everybody for attending thank you very much for the opportunity I really appreciate it there was a lot of ground to cover so I hope I didn't go too fast but there was a lot to talk about thank you very much and the presentation is going to be available in YouTube for everybody that wants to look at it again thank you have a good one bye