 It's my pleasure to welcome Caroline Reynolds today for our plenary talk. Caroline is the lead scientist of the probabilistic prediction research office in the marine metrology division at NRL Monterey, California. And Caroline will be talking about the US Navy's extended range prediction system during. Thanks again, Caroline. Thank you very much, Anish. Anish and Judith, I just want to say I think you guys are doing a fantastic job running the show and also I've learned a lot from all the speakers at the tutorial and the workshop and I've been really impressed by the insightful questions that the students are asking. It's been a real pleasure to be a part of this. So on behalf of a large group of folks at the Naval Research Lab that are working on this global coupled system, I'm privileged to talk about the US Navy's extended range prediction system. So I'll talk a little bit about, I'll start talking about motivation and description of why the Navy needs extended range forecasting. I'll give a little bit of background on high resolution ocean modeling, why we need that, then talk about Navy ESPC performance and research, including trying to identify what we're not happy with at this point and what we're trying to improve and how we're going to go about that. So many of you have probably seen this image or something very similar to it where it's showing the different time ranges of environmental forecasts and then the different sectors and what they need on these different time ranges. And for the Department of Defense, that includes things like for S2S forecasting includes things like ship routing and prepositioning, planning, humanitarian assistance, and managing force deployment. So to try to meet those needs at Naval Research Lab, we're creating an earth system prediction capability or global coupled system by coupling together our current standalone systems. And those include the Los Alamos SICE model for CIS, the NAVGEM atmospheric model, high-com ocean model, and then eventually coupling to WaveWatch 3. And it's a big team effort, as I've mentioned before, different divisions at NRL as well as working with the NOAA earth system modeling framework group. We're using the earth system modeling framework to couple the systems together and that's in the hopes of making future upgrades easier to do, easier to sub in new models. We've been participating in the NOAA SubEx experiment and that's been a really great way for us and to look at others interrogating the performance of our systems, trying to figure out what we could approve upon. And we're very happy that we had version one of the system go operational about a year ago with our operational partners, Fleet Numerical Meteorology and Oceanography Center. And they're running 16 member 45 day forecast once per week right now. And we're also working on improvements to that system and V2 is scheduled for hopefully operational transition in FY23. So this is a schematic giving the specifics of the different components of the system for V1 that's operational right now and then the upgrades that we're planning for V2 and I've highlighted those upgrades in green. And I won't go over this in too much detail. I'll just mention that if you want to read about the specifics of Navy SPC version one, there's an overview article by Neil Barton at all that references right there. And I'll also mention that I'm going to be talking about the ensemble, the long range ensemble configuration of the system. We also have a deterministic version that has even higher resolution, but that's not yet operational. I won't really be focusing on that at this point. One thing I do want to mention here and I've highlighted in red is that we have really high resolution in the ocean and the sea ice components. Even in the ensemble, we're about nine kilometers into the equator and it goes to less than four kilometers near the poles. And that I think is something that makes our system our global coupled operational system as to our system fairly unique is that high resolution in the ocean ice. And the reason we do that, of course, is that the Navy needs high resolution in the ocean and ice, not just for the impact that it could have on the atmosphere, but we're actually operating in those realms. But the Navy isn't the only sector that is interested in getting the details of, say, the subsurface ocean correct or the ice correct. Of course, we have the Arctic ship routes opening up now with the shrinking of the Arctic ice. And we also can see that, for example, if you're interested in marine ecosystems or you're interested in, you know, oil spill disaster response, you really need to have a good handle on what's going on in a subsurface ocean. So this is a bit of eye candy that I got from Eric Chassanet at FSU and it's showing surface currents, an animation of surface currents produced by a 150th degree high calm ocean model of the Atlantic. It's a two year simulation. And you see all these, this beautiful structure, you see the instability waves in the tropics, you see the meanders of the Gulf Stream, you see these eddies that are generated in the Gulf of Mexico. And of course, those can have big impact on tropical cyclones. If they happen to go over a warm core eddy, then they can intensify before they hit land. So basically, what I want to note here is that we see a lot of energy on fairly small scales in the ocean. But I also want to note that the evolution is fairly slow. In fact, in this animation, a month goes by in about three seconds, right? So things to think about in the ocean, you need a lot of high resolution, but also the time to completion constraints are not as severe. And also the time, the predictability time horizons can be longer in the ocean than in the atmosphere. So why do we see such energetics on high resolution in the ocean? Well, one reason is because the respirators of deformation is quite a bit smaller in the ocean than it is in the atmosphere. Rospiraeus of deformation being a function not only of f, of course, that increases as you go towards the pole away from the equator, but also a function of buoyancy and stability. And so this is an image taken from Halberg 2013, showing the resolution that you would need to resolve the Rospiraeus of deformation with two delta X. And you could see it goes from fairly coarse at the equator to quite fine at the higher latitudes. It also becomes quite fine when you're operating in these shallow shelf regions off the coasts. So we need high resolution to capture basically the ocean eddies, which can be about this the weather of the ocean. They can be about, you know, a factor of 10 smaller than the typical atmospheric weather systems. That's not to say there isn't good reasons to go to high resolution in the atmosphere as well. Of course, one of them being to get away from having to parameterize convection, but at this point I'm justifying the need for high resolution in the ocean if you want to get the interior ocean correct. There's another reason that you want high resolution in the ocean. And this is akin to what Tim Palmer was talking about earlier in the week about the need for high resolution in the atmosphere to get the orography right. You have something similar in the ocean where you need to get the bathymetry, right? So the like the subsurface orography, right? And one reason to get that correct is because when you have, you know, the astronomical tides moving back and forth over subsurface or bathymetric features, they can generate internal tides at these tidal frequencies. And I'm just showing an example here of cross sections of temperature and salinity from a model simulation taken in this cross section between Luzon and Taiwan. And these are the, basically these are the phenomena that you can see in these beautiful pictures from NASA looking at, you know, sun, sun glint off of the ocean surface. And you can see these internal tides here. So another reason for high resolution is to get the bathymetry right so you can get the internal tides, right? And depending on where you are, those internal tides can be very important. This is just showing results from Arbic looking at wave number spectrum from large scale to small scale. Two regions on top, Corosio is where the low frequency, basically the non tidal motions dominate. And then at the bottom, it's near Hawaii where high frequency or tidal motions dominate. So Arbic here is pointing out that the sea surface height, I should say that this is spectropower and sea surface height, that the variance that you see in sea surface height, some regions can be dominated by tides. And this is important consideration for data simulation because oftentimes we're using if you're using altimeter information to initialize your ocean model, you have to deal with this in some way, either filtering them out or explicitly assimilating them. So it's a concern. There's other reasons that it appears that we need high resolution in the ocean. And an example here is looking at the Gulf Stream, specifically looking at eddy kinetic energy cross sections across the Gulf Stream, comparing observations from moorings along a cross section at 55 West. I should say this is from a very nice overview paper by Hewitt et al that came out in 2017. And the modeling results are adapted from Chassane and Gilles article. And so we're comparing the eddy kinetic energy cross section in the ocean with depth on the left with what we see, the same cross sections from the model, an ocean model run at different resolutions, really high resolution at top 150th degree down to 112th degree at the bottom. And you could see fairly big differences in the amount of eddy kinetic energy that is penetrating into the deep ocean. And the very high resolution ocean model is the one that does the best job at capturing this deep penetration of eddy kinetic energy from the Gulf Stream and into the deep ocean. We also see reflections of using high resolution at the surface when we look at the Gulf Stream. There's a lot of studies that have been done on why models may have a difficult time capturing aspects of the Gulf Stream. Specifically, they can have a hard time capturing when the Gulf Stream separates from North America and how far it propagates into the North Atlantic. This is just looking out at one example here by Marsochi where they have the errors of models for the Gulf Stream SST from coarse resolution one degree at the top down to 112th degree at the bottom. And they find that 112th degree ocean is substantially better than the, it still has problems, but it's substantially better than the coarse resolution at capturing the extension of the Gulf Stream into the North Atlantic. And this is important not just for people who are interested in the ocean, but if you're interested in the atmosphere. A very nice article by Robert Svitart and Balmesada that recently came out in GRL shows that correcting North Atlantic SST biases in the region of the Gulf Stream can result in improvements to weakly mean atmospheric anomalies not just over the North Atlantic itself, but also downstream over Europe and Northern Africa. So that's the background material. I hope I've convinced you that there's a utility in high resolution ocean. And right now I'm going to start talking about Navy ESPC performance and some of the issues that we're seeing and how we're going to try to get around some of those issues. I'll be specifically talking about results with a fairly coarse resolution 37 kilometer atmospheric component, 112th degree ocean ice forecast, 60 day forecast that have been run once per week for the year 2017. I want to mention that the initial states have been derived from parallel update cycles using random observations. So basically this is an ensemble of data assimilations methodology and we'll also be not just looking at NAPGEM, comparing the system performance to our standalone system and other systems, but we're also going to be looking at the impact of say using a quarter degree ocean and sea ice versus a 12th degree ocean and sea ice because that's a lot cheaper kind of see what the impact is from that. And also talk a little bit about how we're considering methods to account for model uncertainty. Also want to mention that the data assimilation system is what we call weekly coupled. So the standalone systems are using their own update cycle data assimilation methods. So if the atmosphere that's a 4d bar, it's actually a hybrid 4d bar for the ocean and sea ice it's a 3d bar, but the background forecast that we're using in those systems is the coupled system. So that's is from the coupled system. So that's what we're calling a weekly coupled DA system. So just take a look at the performance for the Madden-Julian oscillation as we know as we've learned. This is really important for S2S predictability. So this is just showing Navy ESPC performance, the black curve compared to the performance from other systems. The anomaly correlation, this is for the RMM index is in the top plot and then the RMSE is in the bottom plot. What we see is that for the anomaly correlation, we're doing a good job. We're not quite as good as ECMWF of course, but we're I'd say in the mix for getting the patterns right. But where we fall short is that the RMSE, we're not quite as good because our model is too strong. We have a positive bias. Unlike most other models, we have an MJ that's too energetic. We also note, I'm not showing that here, but we know that our ensemble forecast for the MJ are under dispersive. So these are two issues that we're trying to deal with trying to improve as we move from version one to version two. I'll touch on those a little bit later. There are a lot of metrics that we look in the ocean performance. We look at, you know, how the ocean is simulating currents, salinity, temperature. We look at metrics that are important for acoustics. I'm just showing one example here where we're looking at the depth of particular isotherms as a function of forecast time, function of lead time out to 60 days. So we're looking at the RMSE and the bias and the ensemble standard deviation for the 26, 20, and 15 degree isotherms. This is against observations. And if we compare the solid black line, which is the RMSE of the ensemble mean to what we would get if we were just using a forecast from a climatology, which is the dash black line, we could see that we actually don't hit that climatological, hit those climatological values of errors way out until, you know, later in the forecast. This is kind of touching on this fact that the motions in the ocean are a bit slower. And so we kind of pushing out that timeframe for valid prediction beyond what we see in the atmosphere where things are moving a lot faster. So that's good. But at the same time, if we compare our ensemble mean RMSE, which is the solid line with the ensemble standard deviation of the ensemble, which is in red, we can see there's quite a lot of daylight between those two curves. So that is telling us that like we've noted for the MJO and other forecast metrics, our ensemble forecasts are under dispersion. So improving ensemble design is a top priority for us. So before I talk about how we're trying to improve our ensembles, I just wanted to mention a little bit about throwing two slides here about using a lower resolution ocean and ice component as compared to 12 degree ocean ice component, because it's a lot cheaper to use quarter degree systems. And so here I'm just showing briar scores for 15% ice concentration that's typically used as ice, as the, as where the ice edge is. And so I'm showing briar scores as a function of forecast time for the Arctic on the left and Antarctic on the right. And this maroon curve is what we get from the Navy as PC. And climatology, it's a bit hard to see. It's this light, like lilac colored curve that goes straight across scale for then using climatology. And we're comparing results with the quarter degree forecast at top with the 12th degree forecast on the bottom. And what we see is that we can move this measure, you know, how long we have scale, how long we can be climatology, we're moving it about five days to the right when we use the high resolution versus the lower resolution. So we are seeing gains gains in things like ice edge prediction from high resolution. We're also interested in, you know, atmosphere ocean interactions, how that changes with the changing the resolution of the ocean. So in the study that was led by Sergei Frolov recently came out in monthly weather review, we looked at the correlation, trying to understand the coupling between the atmosphere and the ocean by looking at correlations between different atmospheric and ocean fields. And here I'm showing correlations between SST anomalies and surface wind speed anomalies, where you see the red colors, that means that there's a positive correlation between SST anomalies and wind speed anomalies. That's typically considered indicative of the atmosphere responding to the ocean. And where you see blue, that means there's a negative correlation. That's typically indicative of the ocean responding to the atmosphere. And so we compare what we see when we have the high resolution on the left with the lower resolution on the right. We see that the, this leads to weaker positive correlations and stronger negative correlations. So basically, when you go to low resolution ocean, the ocean becomes less dominant and the atmosphere becomes more dominant in this measure of coupling. And so we would expect to see some impact on our atmospheric forecast. So we do, although it's fairly small. And so this is showing results from Justin McClay. He has an extensive ensemble scorecard for the atmosphere looking at all sorts of metrics and fields and regions. And this is just trying to consolidate that information into one graph here as a function of forecast time. And so where you see positive numbers here, it means that the one 12th degree ocean is helping the atmospheric ensemble where you see negative, it means it's hurting it. So in general, you know, mostly the impacts are small. We do see at long lead times a significant nice improvement from one 12th degree in terms of the bias, which is the green curve, where we see some degradation. That's in terms of variance of the, I should say, or the, the relationship, what we expect to see the relationship between the ensemble mean error and the ensembles spread or ensemble variance. So somehow that's being degraded a bit. And we're not exactly sure why that's happening. I wish I could give you an answer on that, but that's something that's still being investigated. So some improvements in some fields we're seeing from the 12th degree ocean over a quarter degree ocean. So now I just very briefly before the end of my talk, I wanted to talk a little bit about how we're trying to address some of the issues that we, some of the deficiencies that we see in version one. Primarily, we know that our ensembles both in the ocean and the atmosphere are under dispersive. And so one way to, we're trying to correct that is through adapting something called analysis correction based additive inflation. It's a technique that's outlined by Will Crawford in overview paper. And it's based on earlier ideas earlier introduced by Bowler at all and Piccolo at all. Basically we're using an archive of analysis correction. So using information from our data simulation update cycle to, to calculate both a mean correction that, that, you know, that's supposed to correct the biases in the forecast as it moves forward. So it's a time mean term. And then we also have a stochastic component to address this ensemble uncertainty, this ensemble spread deficiency. So there's more details in Crawford at all on this technique, just to briefly show you some results that we're learning. Yep. Two more minutes. Two more minutes. Okay. Got it. Okay. So the, I just wanted to show that the Navy is PC with a sigh, we see if we look in terms of global metric at the impact of the bias on total precipitable water, for example, we see a really nice reduction in terms of a global average. So just showing on the top as a function of forecast time, the control forecasts is in blue, a sigh is in orange, the global average of the absolute value, the bias is on the left and the mean absolute error of the ensemble mean is on the right. And so on a global average, we see, you know, things look great, a really nice reduction in these error metrics. When we look at a plan view plot that's on the bottom, what we're showing here, this is for day 14, we see the blue area is showing where we get improvements from a sigh and the orange is showing where we see degradations. And so we see that we have a lot of areas where we see blue, but some areas where we see degradations. And this tends to be where the bias changes in lead time and or we are assuming that there are state dependent biases. So we're trying, we're currently working on ways to try to refine this technique so that we can get rid of some of the troubling issues while keeping the benefits. And I'm going to skip over this. We're also looked at the impact on the MGO. I'm going to skip over this because of just trying to get things done on time. And I just briefly mentioned that we're also looking at stochastic forcing in the ocean and we can control that stochastic forcing as a function of depth in the ocean and we're working on calibrating that and then looking at the impacts of that. Very briefly, I'll mention other upgrades that we're planning for V2 includes that we're going to be incorporating ocean ties into our ensemble configuration. We're going to be having a one-way coupling to WaveWatch 3. Eventually we'll move to two-way coupling but the first implementation would just be one-way coupling and then an extension to the middle atmosphere that we've heard from speakers yesterday talking about how doing a better job with the stratosphere should improve the troposphere. So we're going to be moving towards that as well. And then just to wrap up, we have the operational forecast went forward in 2020. We're expecting a lot of improvements, hopefully with version 2 that's scheduled for 2023. And then looking further out to FY26 or FY27, replacing Navgem, which is an old model with Neptune, which is a next-generation model that has things like capabilities like static and adaptive mesh refinements. And that will allow us for kilometer scale atmosphere ocean coupled forecasts. And with that, thank you for your time and just put up my last slide. Thanks. Thank you, Carolyn. It was really comprehensive overview of the Navy prediction system. Thank you again. So Judith, you have a question on the chat. Hi, Caroline. Really nice talk. And one thing Caroline and I share is we are always interested in model errors and how to correct for them. And I think Asai is a really nice method to do this. In terms of improving forecast scale, my question is, have you found that helpful in learning about model shortcomings from a physical standpoint for either missing processes or wrongly represented processes? And have you been able to improve the model based on those Asai tendencies? Or do you have any ideas how to do it? Right. So yes and no. So we haven't yet improved the model based on these. But we are trying to use the information that we get from Asai to learn about model deficiencies. In particular, using process-based diagnostics and looking at how Asai might be changing some of the process-based diagnostics. In particular, looking at MJO and what may be leading to mean state, what terms may be leading to mean state biases in the moisture and moisture gradients and how that changes with Asai. So we are trying to move in the direction of learning. As you know, it's one thing to identify model biases. It's quite another to try to figure out what's causing them, what's the root cause. And so we're moving in that direction. But as a good example of, aha, we found this and we fixed it. We don't have that yet. But hopefully we'll get there. Thanks, Jonathan. Thanks, Caroline, for this point. Highln, you have a question next? Yeah. Thanks for the nice talk, Caroline. Yeah, your system with a very high ocean resolution is very impressive. So my question is, I'm not sure the MJO scale you show is that based on a refocussed or that's on the real time forecast. And also, you didn't mention much about your refocussed. So is the data dissemination and also the ensemble members are the same as the real-time forecast in the real forecast? Right. So this is actually based on a year of the operational system forecasts. So just here. Is the bias corrected? This is not bias corrected. And this is one issue that we're working on right now in that we're trying to put in place, because we have the 20-year sub-X system that is slightly different from the operational system. So the results here, you're seeing are slightly different from sub-X in a couple of ways. The model has changed a little bit. But I think the big difference is that this is a 16-member ensemble as opposed to what we have in sub-X is a four-member lagged ensemble. So starting the forecast Thursday through Sunday, one forecast a day. So the issue of post-processing is a really big one that we're trying to tackle. And we're hoping to get something started that would be producing a reforcast on the fly similar to what ECMWF does at this point. But we don't have that at this point. I'm hoping to continue with that. Great. Thanks, Harlan. Thanks, Caroline. Andy Robertson has the next question in the chat. Hi, Caroline. Great talk. I was intrigued by your nice slide on impact of ocean resolution on the atmosphere. And you showed that in that slide, it's more blue over the Indian Ocean at a lower resolution than at higher resolution. So does that mean that the ocean's responding to the atmosphere less when you have a higher resolution? Any comments on why that might be? Yeah, I think it's the other way. I always find this tricky, Andy. But from what I understand and just thinking about when we see blue, the negative correlation thinking that's indicative of the ocean responding to the atmosphere because you think, well, you have high winds and that brings up cold water. High winds brings up subsurface water, deep into the mix layer, et cetera. So you would get a negative correlation ocean responding to the atmosphere. And we see that that ocean responding to the atmosphere actually gets stronger. So the blue is gets stronger when we go to a lower resolution on the right. And we also see that the reds tend to get weaker. So I think in general... So I was thinking that at higher resolution, then there's less blue. So I was wondering why is that over the Indian Ocean, in particular, that that is happening? Yeah, so I don't have a good answer for you other than we seem to be kind of moving the balance away from atmosphere dominating. We've seen you moving the balance towards more dominant atmosphere. So that might be important for a sort of monsoonal studies there, where normally we think the ocean is responding to the atmosphere. But it seems if you go to higher resolution, that it's less the case. Yes. Yes. Yes. That's right. That's right. And the physical mechanisms for how this is occurring, we don't know. But that does seem... The general finding is that you go to a lower resolution ocean, that the atmosphere becomes more dominant and that the impact of the ocean becomes more important when you go to finer resolution in the ocean. But the physical mechanisms, I don't really have a great feel for what's happening there. Thanks. Good question. Thanks, Andrew. And thanks again, Caroline, for a great talking, for the discussion as well. Okay.