 Okay. I think we'll get started. So hello everyone and thank you for joining us today for the Strategic Studies Panel in the Order and Disorder Navigating Global Crisis Series. You may have noticed, if you're expecting, if you've read the chair lineup that I'm not a handsome Italian, but I shall endeavor to fill Dr. Paltolano's position to the best of my ability despite my lack of cooking knowledge and skills. Alessio, I expect that money shortly for that sort of comment. Just some brief housekeeping rules before we start proper. This is being recorded and is live streamed. And if you could please for the moment direct your questions to the Q&A box down below where you identify yourself and just direct who your question is at. That would be great. Thank you. Please feel free to ask questions throughout the entire process. You don't need to wait till the end and we'll draw upon that in the wider Q&A session. If you'd like to join this conversation online, please feel free to do so on social media under the hashtag Order and Disorder and you'll also find it with the at school of security studies. Alright, so we'll go into the quick bios, and then we'll lead on from there. Joining me today is Dr. Kenneth Payne, Francesca Goretti and Ivanka Berseshka. Dr. Payne consulted after having consulted for the UK, US and Dutch governments and being a NATO research fellow and a visiting fellow to Oxford University. His research areas include political psychology international relations and strategic studies and AI, which leads me nicely on to his very recently published book. I, I wore bought the dawn of intelligence conflict. By the way, there will be a book launch for that on the 17th of June for which there should be a link in the chat. And he will be looking at us AI and techno optimism in the IR, and he will go first. We also have Francesca Goretti, who is a fellow lever home scholar, so I'll try not to be biased with the Center for Grand Strategy as part of the interrogating visions of the post Western world project, and she is looking at the securitization of Chinese FDI in the EU for a PhD. She's a research researcher at AI for which she's contributed to recent publications, one of which is including 5G in Italy, and she's also contributed to publications with the EU and China Research Center. One being the fourth technological revolution and how that shaped Europe's views of China. She'll be going second. We have Ivanka, who as a MacArthur funded research associate at the Center for Science and Security Studies. She is looking at how disruptive technologies can affect nuclear risks. Part of how she does this is through strategic war gaming, which is linked quite nicely with her position as a founder and co director of the king's war gaming network. Prior to her academic life, she was in the position of managing the Federation of American scientists into the scenario assessment of Iran's nuclear capability. Work of which has been cited by multiple news organizations, including Reuters, AP and the Washington Press. So those are, that's our lineup and it's an excellent lineup. If I do say so myself, if Dr. Payne could please take the stage to look at his paper, which is again looking at AI and techno optimism in the IR. Hello everybody and I'm just gonna start my video and I've got control of it and just gonna put my PowerPoints up. Can you see those and why can't I? There we go. Hopefully you can all see those. I'm just looking at the participant list here and there's quite a few people I know and there's also quite a few people. So good to see you first of all and there's also quite a few people who know quite a bit about AI as well. So, so forgive me if I'm teaching you how to suck eggs. I've got 15, 10, 15 minutes. So hopefully I will run to time and we can have some questions afterwards from people who probably know at least as much about this as I do, if not more. As Will just mentioned there, I've got a book launch next week if you want to hear me wanging on at greater length about artificial intelligence and warfare, there's your opportunity to do it and said book. If you haven't bought a copy yet, please do. And that will be available there at a healthy discount. I think I can't remember exactly what but I think it's about 15 quid. Okay, so artificial intelligence and techno optimism in the UK integrated review a title I thought up on the spur of a moment when I was pinged to come talk to the conference and then I had to sit down and think two days ago. Well, what did I actually mean when I was talking about techno optimism. There's certainly a lot of optimism about AI and some of it coming from the, from the top of the British military at the moment is very much the buzzword. Well, here are the three of the four big grand from ours is at the top of the British military the chief of the air staff went Winston, we're at a turning point. He said in an interview last week technology is going to bring both mass and sophistication will return to that question mass, a little bit later on when I talk about why it's such a big deal for the UK. The commanding general of UK strategic command in his big talk to the Rishi conference couple of weeks ago now said the AI of all technologies is the one ring to rule them all and he then went on about it at length. He's he's an enthusiast, all these top figures by the way are currently vying for the, for the very top post chief of the defense staff. And the commanding general UK strike on me doesn't command aircraft carriers or aircraft or the army writ large so his USP is the enablers, one of which is AI so that might explain perhaps a little bit of the enthusiasm there. Chief of the general staff the man in charge of the army reckoned in a speech last week that AI has the potential to render legacy systems obsolete and to transform warfare so overall a lot of love a lot of optimism for AI. And some of that fed through into the integrated review which majored, as many of you will know on on technology. And here's what it said, a bit of a mouthful. AI enabled autonomous capabilities will be essential to defense modernization accelerating decision making in tempo, extending the range persistence and mass of our capabilities, removing personnel from harm's way by undertaking dull dirty and dangerous tasks and delivering significant efficiency and affordability gains. That's a lot of things that they're hoping the AI will do, spanning a whole, the whole gamut of activity in defense. Like a lot of other areas of the integrated review, actual detail was on what's going to happen was pretty thin on the ground relative to previous defense reviews. But there was some detail in the IR and around the bazaars as to what's coming. So, the announcement of a Center for Defense AI to centralize and coordinate and drive through change I guess the standing up of an AI ethics committee in the in the MOD to oversee the ethical dimension of AI, whether that's things like, you know, biased algorithms or whether it's things like lethal autonomous weapons. Not clear I imagine but both of those and other things to a unified career management structure. So one of the interesting things about AI is it's a broad purpose towards the decision making tool, not a particular weapon system so none of the three services can claim unique ownership of it in the same way that you can an aircraft carrier or an intercepting fighter, air supremacy fighter, perhaps. And so the skills required and needed across all the services and that's one of the reasons that the strategic command is so heavily into AI. The unified career management structure I think this is the beginning of what's going to be a much larger process at the moment, it's going to relate to cyber activities I think in future it'll do more things. And one of those to be AI I wouldn't be at all surprised to see greater scope for people to shift between between services, if they're doing AI decision making type of work. And then the final thing extensive experimentation and exercising where at a point where there's been with increasing volume and and frenzy of hyperbole, lots of comment about what AI might do for defense. I think we've arrived or fast arriving at the point where the armchair naval gazes like me have to take a little bit of a backseat to the practical business of testing out what works and what doesn't work. And that's where the military is at the moment just starting to do increasingly large scale experimentation with some AI technologies. So what is happening now I think there's a there's a confluence of factors, some of which relate to the UK and some of which relate to AI more broadly and and to military culture more broadly around the world for the UK. In particular, we're at a point where our armed forces are getting ever physically smaller and ever more expensive defense inflation as exemplified most pointedly by the F 35 program where the current buyers at 48 aircraft originally intended to be somewhere around 150. So there are costs there are financial pressures on defense, and also pressures in finding people with the right skills you want to career in defense as well so that's particularly UK reason. There's also another particularly UK reason which Brexit and the almost willful disengagement if you like from thinking about the EU or not so many times the EU gets a mention in the integrated review not many I would I would guess, but there's certainly the aspiration for Britain to play a global role, more independently minded global role, and also to pivot to Asia, and the carrier task group on its way to the Far East now is a symbol of that. Another British reason everybody else is doing it so we should to the concept of military isomorphism AI is in defense is big news in America Russia and China. Famously Putin, the quotes everybody wheels out Putin says whoever masters AI will will control the world. And most people look at that and think he was thinking about defense. The Cummings factors of course Dominic Cummings was a big enthusiast for AI and drove through a lot of this change and I remember a story in the Sunday Times a little while ago that said that when the integrated review was being drawn up. The army's proposals were dispatched back to the army several times with you know the note in the margin saying, come back with more AI, and one of the key drivers of that was was Dominic Cummings factor that's now obviously diminished in importance. And then the last reason AI is perceived to and I believe will deliver combat effectiveness in certain circumstances, although there's much to be settled about what those circumstances are so a range of factors help us understand why AI is a big deal now. Let's just have a look briefly at a couple of examples of AI doing its thing. This is the alpha dogfight from just before Christmas dog fighting Charles run by DARPA. And the human pilot there is being thrashed by an AI from a company that nobody ever heard of before called Heron, Heron AI, Heron actually beat Lockheed Martin on the way to winning this contest in the knockout rounds, you know AI versus AI. And then in the final beta, a top gun pilot five nil. So in certain circumstances, AI can make something of a contribution to to the tactical business of combat. This is is is the test bed for deep mind when it's looking at some AI in complex strategy games where more thinking is needed more forward planning is needed more creativity and imagination perhaps human type attributes or attributes that we associate with humans less with AI. And this is Starcraft to a game called Starcraft to and increasingly what we're seeing is these sorts of games open AI is one called Dota. These sorts of games being used to test out the concept of human machine teaming. And I think human machine teaming is a big area for AI in defense, understanding how algorithms and humans interact complement each other compensate for each other's relative weaknesses. So in Dota two, it turns out that the best combination in team matchup is not a team with all humans, or a team of all machines, but a combination of the two, such that the machine is doing the unerringly accurate quick shooting. And the human is bringing to the party the ability to look ahead and plan complex strategic moves. So it's not just the tactical dogfighting bit where AI excels. It's also thinking about how how AI works in combination with human minds to produce better effects of human machine teaming key area for AI. That's the and rule ghost will skip that for reasons of time. Here's another application of AI in defense. This is the predict swarming drones. I'm sure whether you can see it on the resolution, but out the pods under those FAA teams is coming hundreds even thousands of micro drones that can then be coordinated and act as a swarm. Another tactical activity where AI is making a difference. Okay, so why techno optimism per se in general. I think there's a real danger when we're talking about an exciting new technology like those things we've just seen as AI informed things we've just seen to slip into technological determinism. So my idea that the technology is somehow out there fully formed and that it's going to have a massive impact on defense. There's always a sense anyway in strategic studies I find of presentism where people are trying to say something new and make a name for themselves by latching on to the, to the latest thing. So in that sense of technological determinism with the presentism and you underplay the continuities in strategic studies the extent to which there are long standing traditions that inform how countries organized to fight war. And then the bottom one that that sense of war there's new paper out on on technology and or the Journal of Strategic Studies that prompted me to put that there worth checking out. And optimism, you know the Cummings enthusiasm for the enthusiasm with the chiefs meets reality. Well I think we've got some particularly British challenges ahead and some more broad challenges ahead when it comes to adopting AI particularly British challenges I guess is our point of our organizational culture in the UK. It used to be known British armed forces used to be known as the borrowers by the by the Americans for our propensity for for pinching bits of kit of them and using it to our advantage. And that points to the you know making men do approach that we've we've sometimes adopted. It's a little unfair we are one of the largest militaries in the world one of the best resourced militaries in the world, but there is still a gap I think between the hyperbole on one hand, and the scale of activity on the other so I mentioned drone swarming squatting in the very small scale affair. If you look at the land warfare trials of autonomous robots, it's still a very small scale affair there's a large gap to what's being done, and where the breathless talk is. I think that organizational culture as well in the source of procurement decisions that are taking place now with respect to tempest which is still got a cockpit, or with respect to the loyal wingman concept, where you know the appropriate application of AI in the area is seen as a servant of a human pilot in an F 35 rather than the thing in and of itself, not withstanding what we saw in that dog fighting trial. Organizational culture I mean militaries are conservative and traditional for very good reasons it tends to work. And if it works why change it, and they're hierarchical as well which tends to impede radical change or can do. But those things work against rapid adoption of new technologies, the Cummings effect now gone of course or at least diminished with the departure of Dominic Cummings. As I alluded to earlier there's a susceptibility to revolutionaries and snake oil which comes back to bite you when people can't deliver on what they've promised more broad challenges about the adoption of AI. I wouldn't dwell on any of these in any particular detail except to say that we're not in an AI winter, you know the history of AI has been a burst of enthusiasm and excitement about what's possible, followed by a winter so called as the reality sets in when AI can't deliver on what it's promised and I don't think we're by any means in a winter, but there are some big challenges about the sort of AI that we're seeing now and what and what what its potential is going to be for the future and of all those things there I'd point to the problems of getting an autonomous car on the road, where is your AI car. The answer is that there are also people on the road and AI is not very good at reading what other people intend to do. So on the list the problem of bias and you can see that in terms of AI being used in predictive policing or in human resourcing. If data sets of bias, then the output of the AI algorithm that you've trained it on bias so at fairness and and unbiased algorithms are a big problem with the current approach to AI. I'll leave the rest for reasons of time. Okay so much optimism, many challenges some of them British some of them to do with the technology. The panel asked us to focus on what the implications for international order are and I've put some on the screen there is probably worth concluding with those. The bottom line there is the important one the integrated review hasn't really on the face of it grappled in any depth with with those problems listed above. The first one is that arms control for AI is extremely unlikely in my view, notwithstanding that there's a very vocal group of people who would like to see rigorous arms control, particularly when it comes to lethal autonomous weapons systems. It's unlikely because there'll be a lack of agreement about exactly what it is you're regulating, and it will be really difficult to police any agreement any arms control agreement that you put in place defecting from the regime and the jargon is very easy. So that's one implication for international order we're likely to see as a result in absent arms control we're likely to see in my view something of an arms race as as all sides scrabble to understand and instrumentalize the technology. That means that we're likely to see changes in the balance of power, some strong states will carry on being strong states in an era with well developed defense say I United States the UK Israel, all have very well developed militaries and very good science and innovation sectors and a reasonable prospect of linking the two together. China I bracket there as well to a lesser extent. It's not clear to me that China has the same capacity for innovation as does America it's also not clear to me that AI fits well with Chinese strategic culture and military culture which doesn't really emphasize mission command. So, some states will do better than others the biggest loser among the great powers I perceive to be Russia, and I can pick that up in Q&A. Third point, the utility of force AI is likely to change in my view. Some existing attitudes and norms about when force should and shouldn't be used. And you can start to see that already with respect to cyber activities what's the appropriate response to to being hacked or to ransomware. You'll see some more of that as AI starts to be an increasingly prominent part of cyber warfare. You see it in the use of robotic and unmanned systems. There's an appropriate response when somebody fishes your unmanned submersible out of the Pacific and cast it off home to figure out how it all works, as has already happened. So different different approaches to the utility of force. And I would include there, what are the prospects for the actual use of force, if you can be reasonably sure that it's not going to involve exposing your own people to risk you more likely to use force. I think the answer is yes, and the obvious precursor would be the use of drones over the tribal areas of Pakistan. And then the last one I think AI, as it as it filters through into defense is going to see something of a change in the nature of the state and I would include liberal and authoritarian states alike there. There's always been a long standing relationship between the state to the government the governed and technology is sort of three way relationship. And I think the introduction of AI is is already starting to change that. And the two examples I put there what's the appropriate balance between the public and private sphere, particularly in democratic individualized western states how much should the public sphere about you, what's happening to your GP data this week, for example. And secondly, a change in the nature of the relationship between citizens and their armed forces, as there are fewer warriors, and as force that is increasingly created and carried out by robots already a very small proportion of Western societies are involved in armed forces my favorite stat is that 0.13% of Britain's serving uniform, and of that small number, and even smaller number are employed in in close So that already tenuous relationship between society writ large and its armed forces is going to be stretched even further as you automate our force. The integrated review, whilst it's hugely optimistic about AI hasn't in my view started to grapple with those issues. There you go that's my little cancer through I hope there's something in there to prompt some thoughts and feel free to buy my book as I say over to you. Thank you Dr pain for that so yes just a reminder book launches on the 17th June link is in the description and also reminder that Q&A feel free to put questions in the box throughout the session and we will get to those at the end. So thank you for that next up is Francesca and she will be looking at how the UK should be shaping its China policy off the back of the integrated review so take it away. Thank you. Thank you will. Let me share my presentation and definitely not trying to say to the government what it should do but I'm trying to provide an analysis of what has been so far and where it has moved with the integrated review. And that is why the title is shaping UK's China policy filling the vacuum and we get to the vacuum very shortly. Before I need to give a little bit of a context nothing new. Most of the people who've looked at UK China policy know about these. We start with the golden year at the so called golden year doing the government of that the Cameron was particularly push forward to buy George Osborne. This view of UK China sort of a special positive relationship that picked in 2015 with Xi Jinping state visit to the UK where both you know wish for a great collaborative future for the two countries. China was particularly eyeing London as financial hub before you know the internationalization of the remedy but also for other reason real estate investments and you know also technological advancement. But then this era ended quite abruptly in 2016 when after the Brexit referendum camera resigned after which the question mark was what is UK's China policy now because that era that sort of golden era of the relationship. And then we made government we entered the so called vacuum, which is not of course a phase that has been named as such on purpose is just literally that while during the camera government the so called countries of prosperity is where the ones that were more vocal and had more leverage over the UK's China policy, once the camera governments transitions out. Other ministries also start to cover a more important role and of course because the resume is by minister the office in particular with his own concerns. And that is why surely after she becomes prime minister you have a review of the project of the construction of the input points see the sorry the Chinese involvement in the construction of input points see that afterwards gets quite lighted and the launch of the National Security and After which there's also a sort of a step back from what was promised by Cameron and official involvement into the Chinese project the Belt and Road Initiative may it assures the population that there will be no official involvement of the UK into the country to try out. Of course, unofficially, there was still quite a lot of push for the economic relationship between the two to be very tight, also in light of what Brexit would mean for the UK and you know, the start into shape of sort of the global Britain. Again, however, Brexit negotiations and plans for Brexit kind of occupied most of the agenda of maze government. And so not much time was dedicated to understanding and shaping UK's China policy, which means we entered this vacuum. And it was rather unclear what the UK China policies was, and China of course in the meantime was becoming an increasingly important actor. And so you have to start thinking about what is at least your approach and position towards such an actor. Okay. And that is when I'm sorry, and when Johnson's government begins here we see the shift. Of course, the shift is not immediate. And it's not just due to the change of government and to Johnson in part it is due to that debate that had been started earlier with the maze government. And also the fact that to a certain degree, the Brexit matter was coming to an end, which I know is a bit of a strong statement, considering latest news but you know, let's put it that way for the moment. Other elements became extremely important in shaping the UK's position, you know, the position of the Trump administration, the debate on Huawei and of course these two are particularly linked if we think to developments in mid 2020s. There was the acceleration post but the pandemic of COVID-19, but also issues such as the introduction of the national security law in Hong Kong, and because of the historical relationship between Hong Kong and the UK displayed quite rolling, shaping the view of the UK towards China and to a lesser extent the issue of Xinjiang. So you see more critical voices towards China arising and becoming quite prominent. And of course the foundation of the China research group is a great example of this, led by Tom Tuggenha and has been steering the debate in a direction which is very much. I mean we could describe it as a bit oakish but it's more you know concerned about the UK being able to protect his own interests and security. And now we get to the integration review and I'll slow down the pace a little bit. I just wanted to get through the background quite quickly. Of course, through the years China has become an increasingly important element in the UK's foreign policy, because China has become an increasingly important element in everybody's foreign policy. But interestingly the content of the integrated review doesn't quite mirror the hype in the public debate. If you look at the public debate and the political debate it seems to stir much more towards a more negative position of the co-towards China, which is not necessarily mirror the integrated review, the integrated review is still rather balanced, at least this is my assessment. And interestingly the definition that one could get out of you know what is the UK's China approach is very similar to that adopted by the European Union in 2019. And I realize how contentious it could be that you know it might have been inspired by the approach taken by the European Union so I won't go as far as saying that, but it definitely resembles that approach, which is the so-called tribo-type approach. China is at the same time a partner, an economic competitor and a systemic rival. But it's not quite the same. And to a certain degree the UK's approach and the EU's approach is similar to what has been named the Blinken doctrine, which again recalls very much the so-called tribo-type approach, which is an approach that you know is multifaceted and leaves quite a lot of space of manoeuvre to other actors vis-à-vis China. And I think that within the integrated review this Sunday's paragraph, which I won't be reading quite well explains the part about the partner and the systemic rival. But there's a bit of a lack of economic competition, which is why in the previous slide economic competitor was in red. And the reason is because this element has always been a little bit difficult for the UK. The UK has always been a large advocate of open economy and balancing the defence of its own interests with leaving the economy open and free has always been a bit of a difficulty. And at the same time the element of economic competition for an economy such as that of the UK is an element that has come to the attention quite recently. Why? Because China historically has been a manufacturing-based economy which didn't present much of a competition for a service-based economy such as that of the UK. But of course as we go through the shift in China from a manufactured-based economy to a value-added and service-based economy, the competition with economy such as that of the UK becomes more visible and more felt within the UK as well. And those I'm wondering whether you know in the words that regard technology, we could see a bit of that economic competition mostly if we think about the mentioning of here adverse effect on prosperity. So not just about security but also about prosperity. This change and this attention to economic competition is not entirely there yet and is definitely more recent and less strong than for other actors, most certainly than for the US and the EU. There's also a larger security component compared to many European countries and definitely compared to Brussels and that is because well first of all the UK has interesting in the Pacific region, which are more evident than for the European Union, but also because there's a long-standing concern regarding military supply chains which has not always been about China. It was you know more brother, sorry it was brother and just regarding you know being aware of how those supply chains were composed. But recently it has become increasingly more about China and it has also been expanding beyond the military sphere to door use and further. So a lot of attention has always been paid to national to critical national infrastructure and I'm not saying that that is not the case in other countries. It's just it's just been very specific in the UK much more focused on the CNI than in other areas where for example, knowledge transfer was much more concerned rather than critical national infrastructure. There are also values and of course again here we should stop a little bit thinking about that I think about the change that also the Biden administration has brought to this conversation, bringing values back to the conversation. And therefore it becomes easier also for a country such as the UK to buy into that sort of these squares. But of course I think it kind of precedes the change in the US, and that is because of the introduction once again of the national security law in Hong Kong, and therefore understanding that you know values that are an important element of the UK's China policy. And in this regard also the contribution of the China research group, and that sort of narrative that that type of group advocates also played a role in shaping this part. So just to conclude the integration review field vacuum that was present, you know, before, I think that to a certain degree it did, but rather than describing how the UK should move forward is a description of the existing positions, and is a framing of positions that at the same time are contradictory, which is fine. It means that it's still very general and perhaps it's the first step. But I do think that when it comes to China because of quick changes and difficulties, general frameworks are perhaps more helpful than very precise policy because this is not always the case but of course the integration review was a big work. So I think in this case it applies because it leaves current and future governments quite a lot of space of maneuver when they come to make their decisions. The problem is, I don't think that not just the UK but any of the actors involved in understanding their China policy are kind of understanding how these contradictions play out one place, one next to the other. And I think this is one of the issues that we'll have to face very very soon. And then finally what I'm wondering is whether the UK's China policy is its own China policy or is an harmonization of the UK's China policies with that of the US and the EU. And my final question is that is a little bit of both. I think that I've shown that some elements are very much unique to the UK while others are more in harmony with the Allies. And I'll conclude here. Thank you. Thank you Francesca. Really good talk, really interesting. It's always nice to get an outside perspective, which is knowledgeable on China policy in general and sort of apply it to the UK where we can sometimes be stuck in our own circles or on our own echo chambers I should say. Up next is Ivanka Pashanka. And she is going to be looking as it says on the screen on reforming collective strategic capability so I won't take up any more of your time. Take it away. Well thank you very much for the kind introduction and for the opportunity to present my views here today. I should also add, in terms of background that I have my my backgrounds in both physics and international relations so I come from from those two disciplines and and my work at King spans both fundamental and applied research. So on the fundamental research side, we've been interested in methodology and epistemology questions, focusing on on strategic issues, in particular high consequence looking at high consequence problems on which real world data is scarce. And this interest has also translated into the development of new analytical methods for strategic analysis, incorporating strategic analytical wargaming but also decision analysis and risk analysis. So in, and in that capacity, I have been advising the Secretary of State's Office for Strategic Net Assessment and Challenge. So I should clarify that the views that I'll present today are personal views there and they're no way representative of that of the UK government and this is just my thinking and as an outsider. I will talk about five questions in this talk. Given that we have mostly an academic audience I will focus on what can we as academics be doing to enhance the UK's strategic capability. I'll provide a general overview of the problem and need and and highlight some of the key challenges. So the reason we're talking about reforming collective strategic capability was that this was this was one of the reform priorities identified in the integrated review. On screen you see the paragraph and bullets that describe what the authors of the review saw as these priorities. And then is in some of the other areas you know we can see that this is this description is quite general and in some ways vague. I will focus in my remarks on on this last bullet which is the methodology piece the you know what are the new sources of evidence what new techniques can we use to to address these these other questions. But I'll also talk a bit more broadly about collective strategic capability and what that means. So to to to elaborate on on the integrated reviews assessment of of those needs there at least five functional components to to strategic capability. The first two relate to situational awareness. So this is about you know how good are we at identifying challenges and opportunities that have strategic implications to the UK. And this this is these two are basically about diagnosis and prognosis of problems so so furthering our understanding of the problem, both long term but also short term. We're able to recognize the key triggers in real time. Having done that situational awareness. We're now looking to identify solutions options. So this is where strategy exploration comes in. We're not translating interest into strategic political and military objectives which which is a very hard problem, and identifying solutions and courses of action that would enable us to achieve these objectives in different contexts. The strategy development piece is then actually doing this in practice. So once you have the options identified actually doing this. The last component is is the implementation, which is the effective and efficient execution in in a timely manner and in coordination with other other parts of government. We also have what I would call the joint neighbors. So this is the people who needed level of strategic thinking of relevant domain expertise. And also the the methods that help us think about the emerging problems in in security environment. And of course, at the end being able to bring all of that together to deliver strategy development and policy development. So, so why is this important why is reforming collective strategic capability important. As the title of this panel says, you know that you know they're they're big implications for the regional and international order. This will have implications on how the UK government interprets unfolding events, how it translates national interests into strategic objectives. And also in what I would call theories of success. In different contexts, how it works together collectively to implement these theories of success and and ultimately how effective it is in shaping real world outcomes in accordance with with its interests and doing so either unilaterally, or with with allies and partners. So, why is why is there such a need why why why is this reforming this a priority. We're seeing two key drivers. One is external. And as my colleagues have described, there are fundamental and important changes to the security environment and the integrated review is is reacting to these changes. These include aggressive competition aggressive and persistent state competition. And also technological change, which is evolving at great speeds. And this creates new analytical requirements to understand the evolving challenges to deterrence and defense and to adapting the UK's strategy doctrine and capabilities. So we have. So we're looking at basically the new problem of major power competition and the technological and political and economic change that that follows that, which leads into a recognized internal need. So we are top down signal from from UK leadership that we need to change the way we do strategic planning, moving away from what what what is called the capability based planning, which is, you know, you say, you know, notionally we have capabilities to do a whole range of of things in in these notional scenarios to actually being able to answer the question of how well prepared are we in various contingencies facing a capable thinking adaptive adversely like Russia or a competitor like like China. So we see both that both that in turn external driver creating new analytical questions and then internally leadership saying you give me a new a new way to do this. So what has been done to address this problem. I'll talk about mostly developments within within the Ministry of Defense. So the short, short answer is, is quite a lot. The first point is, which is quite significant is the creation of a new office for strategic net assessment and challenge, which is described as challenging the accepted wisdom and way of doing things. This is what it what it's for. It's meant to serve as a central hub for strategic analysis within the head office at the MOD. And for ensuring that both strategy and and force development are threat are oriented around the threat and are driven by by by evidence. Now a part of that office is is something calls for teaching that assessment. And for some of you this, you know, rings bells and you go back to the Cold War Office for Net Assessment at the US Department of Defense, but the UK is thinking about this in in in ways that are both similar and different from from its US counterpart. So basically this this new net assessment function aims to answer the question of how how do we measure up against these competitors. What are we competing over and in what circumstances could competition transition to war so not looking just at UK capability but looking at how do we, how does that relate to to what what Russia China and and other actors are doing strategically and how does that affect the balance of a strategic advantage, meaning our ability to to advance strategic objectives in in different context and then and then finally there's support for for for doing this kind of strategic new strategic thinking at at the NATO level. So a lot has been done, but there are a number of organizational challenges, all of these relate to the ability to link and integrate different functions. So one one aspect of that is long term and short term net assessments. So you know you have somebody that is thinking about how are we measuring up against competitors over the next 10 years, and then somebody thinking about, you know, what does this look like in the next months and year and how to how do those link up and how do you iterate and update these kinds of assessments in a way that's systematic. Secondly, you know you have this assessment you've identified. You've diagnosed the problem, you think you know how this, what the trend is how this this might unfold. Well, so what do we do about it. And, and you have different teams working on different pieces of the problem and how do you how do you ensure that's done again in in a systematic way. You also have then the question of what does this mean for for capability development and for the for the for the technology and the stuff that we want to to build and and deploy. And then finally how do we do that, working with other government departments and also with with our with our allies and partners. And analytically, and this is this is where where this problem gets particularly exciting for for academics is that there is, you know you have external and internal needs, driving new analytical questions. And this creates the needs, the need for new concepts. So briefly give you some examples of of the kind of new ideas that are emerging from the Ministry of Defense and another government departments. And then I'll talk about some of the challenges and further work that that still needs to be done. So concepts, you know, these these include, you know, we've been thinking about modern deterrence multi domain cross domain deterrence. But now there's there's a need to think more holistically about both, not just the deterrence piece but also deterrence and defense and how that relates to strategic objectives and in general. So there's new thinking on a new development on organizing concepts in that regard, looking at theories of victory theories of success. And, and things like the balance of strategic advantage and how that relates to two opponents. There's also a recognition that we need new sources of evidence to be able to understand these these these kinds of dynamics over the long term. There's an interest in in war gaming, which is a new way of generating data on, particularly on phenomena which are on which real world data is lacking so you can't go and and look at the historical record to be able to understand how AI will will affect the balance of strategic advantage you need, you need a different approach to do that. And then finally there's there's new technologies like like AI and a big effort to to automate some of these these analytical processes. Now specifically in at the strategic level, the focus is on tools that will allow you to process qualitative data quickly and with in a distributed way. And I can talk more about that in in the Q&A. So the key analytical challenges that that I'm seeing is that there's still a lot has been done on on trying to develop new concepts that are better tailored to describing and explaining what we're currently seeing in in the strategic environment, but but we still need need need to do additional work. So part of this is that you know we're taking concepts that are come from a structural realist tradition like you know the balance of strategic advantage which are you know very much about how hard power, and we're trying to bring phenomena like competing strategic narratives or or hybrid warfare. And so bringing this bringing these things these things together is is is still a challenge. Scientifically sound methods, you know I talked about war gaming. I can talk a lot more about war gaming, but but the bottom line is that war gaming has been practice more as an art than a science by small companies of professionals to look at mostly tactical and operational level questions. So there's this big need to be able to do that effectively at the strategic level, and also do that in a way that is that is systematic and rigorous and that's where some of our work comes in. So the state of the art technologies. These critical and emerging technologies you know these will be some of the key enablers to be able to do all of all of this at scale. Again, for me the big questions here are around epistemology and having having a sound scientific base on how you can use AI. So finally some some thoughts on on what we could be doing in academia to advance this this capacity. Obviously the work on on new tools and techniques. Importantly demonstrating how these methods can be applied rigorously and ethically to answer different kinds of questions. Advancing knowledge on what these key phenomena with strategic impact look like what is there under underlying logic, making sure that those findings then then get to policymakers and and provide the needed challenge and further thinking. And finally there's the educational bit and just helping ensure that we have that next generation of strategic thinkers to ready and support this work into the future. So I just leave you with with my email and some links to publications that you can look at to further explore some of these issues and I look forward to your questions and and continue discussion. Thank you. That's great. Thank you, if you stay on with your video and if the other members of the panel could also flip their cameras on, please, we can get to the Q&A session. And hopefully some of the exciting stuff. Luckily we've had quite a bit of good engagement actually from the Q&A and at least every panelist has one question which is always really nice when that occurs. So I think we'll take it from the top actually because we're at 39 attendees. If the Q&A is want to actually free the mic and ask and articulate it that way just simply because there aren't too many of us. That is equally helpful or if you would like to put your hand up if you're not in the Q&A box to do a question that's also good. Emma Salisbury has asked a question and I don't know if you want to ask it free the microphone or not just because there's a smaller number of attendees. Her videos popped up. Don't know if she's there or not though she's popped off to make a cup of tea. Lovely. Hello. Yes, my question is for Ken. Where should the balance fall in the armed forces between adding AI centre training for existing personnel and focusing on hiring in talent from the civilian tech world and will other nations do this better than we will? You probably know the answer to that better than me. Hello Emma. Yeah, so I think I think I mean there is a skills shortage worldwide of on the on the technical side of AI. People command high salaries at the at the premiership end of the table and and it's an international workforce, which is partly why the US is very well placed because it's attractive to international workforce who want to come and work in California. So there is a skill shortage. There's a skill shortage within the military as well, but the military doesn't need high end code as the sort of going work for deep mind or open AI. It needs technologically adept people who are capable of thinking also about how the technology is going to be used and who understand what the opportunities and limitations of using AI are. There's a skill shortage internally. I think that's all of that's centralized career management for cyber and technology people that I mentioned is an attempt to is a recognition of that and an attempt to kind of tackle it, whether it will or not. I don't know there are a couple of things working against the military doing the doing these sorts of things tradition hierarchy, deference for rank, stove piping into one specialism or another. Whatever cat badge or trade you have. And, and then also this the individual service cultures. So, it's not like the military the armed forces are going to turn into Google with a flat hierarchy and sort of project teams thrown together overnight. But there are attempts to change the culture of how the military deals with tech and with AI in particular and point to the J hub as an example of that is reasonably small organization that exists in London in shortage, and designed to take the advantage of off the shelf or near off the shelf commercial technologies that can be employed for for military activities that's a good example of a change in existing culture designed to to reflect the new times. I think also actually I say not just the the technological part of your question like familiarity with AI and that, but also the changes that it means for how the military does its traditional things I mean what does it what what sort of skills are needed to lead in combat when large parts of your fighting power are autonomously created. You don't necessarily need the same sorts of skills and attributes that the military has looked out for for a long time. What happens to the RAF's culture as the fighter pilot slides eventually dragging his fingernails fingers fingers as they go slides into history. RAF has been an organization where the where the fighter pilot has traditionally been the you know the center of the action, and they've selected their senior leadership from fast jet pilots primarily. What happens is that as that changes to military culture so they need technologically minded people. Do they also need the sorts of skills that they've had for a long time. And then the last part of your question how how are we placed compared to other people. Pretty well actually I mean the other other large powers have got these problems to and in many respects are less well equipped to cope with any or all of them than we are. A loss of a lot of attention rightly drawn to China strategic competitor, potentially rival increasingly sophisticated modernizing its armed forces. But from a reasonably low base of capability and not clear that it's prevailing strategic culture fits as nicely with AI as does ours so challenges elsewhere as well so we shouldn't be too negative. That'd be my answer. Sorry to wang on a bit. That's great. Thank you Emma and thank you for the answer. Rachel Griffiths you have two questions one for Ivanka one for Francesca I don't know if you want to ask those via microphone or not. Hi good afternoon. So, so my first one really was within the, obviously within the the IR there's, there's quite a lot of references to the threats from Russia and also China, and noting your point Francesca about the lack of policy, many probably from policy with for China. Is this, do you think that this is actually deliberate. Because it would it would appear to be quite, in some respects quite a belligerent sort of posture to take. Should I respond or show Rachel. Rachel, could you also ask the question for Ivanka please just so we could get it all we don't sort of have the logistical nightmare that might ensue from a zoom meeting. And so, Ivanka, you mentioned about sort of basing threat based analysis as opposed to capability based analysis. And as someone that's particularly just almost totally focused on defense capability that some maybe slightly biased but that to me just looks like a generational return back to a process that we had previously and was moved away from probably I would suggest due to the the costing implications around. It's very difficult to cost threats, whereas it's very easy. It's a lot easier to actually cost capabilities. So, again, is that is that a return do you think, and for what reasons. That's great thank you Rachel. So, for just if you want to go first, and then Ivanka if you want to go second. Extra brief. Yes, I do think it is actually a voluntary decision that have keeping it quite wide quite flexible, because although they do recognize the threat. First of all they still have to keep economic ties with this actor. And secondly, they have the problem that everybody has they don't really know still how to respond. So I think that the flexibility is something that was put in there voluntarily. Yeah, that is everything I have to say. Well, thank you. Thank you, Rachel very much for your question and the. We are going back to to the future and in some ways because of this revived problem of major power competition, which which now looks quite quite differently. And this is this is exactly why the priority of reforming strategic collective capability is so important, because there's the temptation to just go back and dust off the old approaches that that we have. So, you know where we're returning to functions and processes like net assessment, and the challenges is now making sure that they're modernized that they're tailored to to the new strategic environment so you're not. So I'm just trying to to out compete. You're also looking at where your theories of success come are conflict in a line with with a whole range of actors. So not just not just one power. And in general, making sure that the concepts and methods that we are using aren't, you know, just the Cold War concepts and we're not going back to that. So this threat based planning is as it was one done was done. That's great. Thank you for the very interesting questions, Rachel, and thank you for the good answers there Francesca and Ivanka. There's an anonymous question so I think I'll just ask that rather than go fishing for the person who asked it, and it's directed towards Francesca. So put that you just had some really interesting points and I'm asked if you could speculate I'll put as I'm sure you haven't got a one to one psychic link with the with the chairman. But could you sort of speculate what you think the CCP's views might be towards the UK both in terms of opportunities and challenges. Thank you. First of all, it'd be great to have a one to one, you know, link with the chairman, but it's becoming increasingly difficult to have links with anybody really in China at the moment. Again, this is a bit of a speculation. And of course, I haven't studied in depth at the spot and I'm sure there are great studies out there that could give you a better answer than I can. So the conclusion at the moment is that so earlier China viewed the UK as first of all a great financial hub a great opportunity for investments and also a great entry point into the European market but also the balancing act within, for example, protection for protectionist push coming for example from France, right. So which is why China was very much against Brexit they didn't like the idea it would have made their foreign policy, even so more more confused than what it is towards Europe more confused than what it is now. And also the other thing they were a bit afraid of was, you know, the UK becoming even closer with the US once he had left the European Union. So what is their view now that the UK has left the European Union there are still these beliefs that you know they have this that the UK has this very important component of prioritizing economic ties. But they've seen their sort of, you know, quote unquote worst fears materializing after Hong Kong with you know the the BNO and also in regard to our whole discourse about value based foreign policy protection against foreign direct investments which I mean we're getting there it took quite a while but eventually we're getting to the adoption of that mechanism. And again, I think the most important perhaps change was when the UK banned Huawei last year. So they have seen the realization sorry the materialization of what they feared it would have happened. And they think that the UK therefore is not as much as a positive counterpart as it used to be in this center where it's not that dissimilar from the rest of from how China views the rest sorry from how China views the European Union. So it's not that far away from that perception. Big question and I don't have an answer to this one is whether China perceives them to be savageable you know whether whether they think that you know the relationship can be savaged and indeed it can be brought back to a more positive line. And again, I asked this myself as well so I don't quite have an answer to that part of the question. Great thank you Fran I would say probably stay tuned for the next couple of decades and hopefully you'll be part of that vanguard of academics who might be able to shed some light on that in the future. So we have another Q&A and this one's directed towards Dr. Payne. And it's from Hugo Rosman. If you'd like to ask it by a microphone, take it away, possibly stepped up for a cup of tea as well. I know I have been guilty of doing it once or twice. Perhaps I'll articulate it then and we can just sort of carry on via that route. So Dr. Payne you touched upon public private cooperation and how essentially the question is how should the UK mod and do approach industry engagement from the outset of setting up a defence AI centre and he puts bearing in mind that other countries are very different relationships with the private sector. Yeah, thank you. Thanks Hugo as well. Who is here as far as whom he's wrestling with the tech. It's the worst thing about Zoom conferences that awkward pause whilst people try and unmute themselves furiously. So I think it's not a brand spanking new problem is it? There's always been a problem of how the government relates to technology companies working on defence related issues. And every country has its own blend its own particular strategic culture and whether US has enjoyed fabulous success, especially in respect of the relationship between defence and AI has been in that that some unique blend of venture capital University departments, DARPA slash Pentagon funding CIA funding and congressional support for regional state based activities. So there's a unique American culture. There's a unique British culture and a distinct Russian Chinese culture and every one of them wrestles with the same sorts of question that comes up with their own answers. I think in the UK, we've had some success in building an indigenous high tech defence sector and I think there are reasons to be positive that we can do that as we move into the AI era as well. However, I think there are some non traditional businesses that aren't in the habit of working with defence that are at the leading edge of AI research, and that's the challenge for government is how best taps into that expertise. It's starting to happen with with companies like Amazon, Palantir, and so on. But there are some companies right at the cutting edge deep mind, for example, is quite explicit that it won't work on defence related projects. And Google has walked away from defence related projects, including ones like Project Maven that the Brits are involved in that's using AI to to analyze video stream captured from by drones. So how do you work with people who don't want to work with you who are producing tech that is going to be extremely useful in the cutting edge, especially when you know is that the cutting edge and the cutting edge is really important with AI. And I think that's that's the problem that the British government faces. There's still a lot of talk in the run up to the integrated review about the possibility of state aid for strategic industries to secure an indigenous production capability and development capability. I don't know that that's going to be the case with AI more likely to be the case in related fields like quantum computing, where you might want to ensure whether there's where there's no financial pet immediate financial pay off from the product but you nonetheless want to ensure that the basic research gets done. So it's a challenge, but the short answer is it's a challenge for everybody as well and I'd rather be in our position than some other people. I hope that helps you. Excellent. Thank you. And sorry to hear about the audio problems. It plagues us all as Dr. Payne noted so no harm there. I'm going to interject and take some liberty out as the chair to ask Ivanka a question just so we can spread them out a bit before going back to from a query I had and apologies. It's coming from outside of your area of expertise so it may seem a bit basic for you but something that interests me especially based off how Whitehall works in the integrated review. You talked about or listed some things that Ministry of Defense has been working on to reform collective strategic capability and you also pointed out that some of the issues or challenges obstacles would be working with other departments in the wider UK. Have you had any thoughts on where best a national version of this could be placed in Whitehall or whether you believe it should be a bit more decentralized and it should just be departments having their individual sort of versions of these. I was just wondering if you could sort of extrapolate a bit more on that topic please. Thank you for the question. I mean, my view is that to be able to do strategic net assessment, effectively, and not just focus on the military problem but also focus on be able to understand non military aspects and and how what that looks like over the long term. We need to be, you know this needs to be a whole government approach from the start. Certainly where that's best placed is in the National Security Secretariat. And I would, I would expect that if net assessment is able to deliver up, you know, new methods and insights that are of interest to senior leaders that similar kind of capability will be replicated, probably at higher levels. That's, that's my guess. Thank you for that. It's an interesting, at least as I'm a historian so coming from sort of diplomatic history and then sort of the finicky anecdotes that come from from practitioners both old and new so I'm very interested to see how that happens in. So it's sort of the connectivity between your research and mine and sort of all the chaos that could possibly come out of that which is interesting for us as academics so thank you for that. Really looking forward to see how that happens and fingers crossed it will happen in a positive light and not a chaotic one. So there's next question is for Francesca and it's from G Leslie again. Leslie if you want to ask via microphone. I will sort of articulate this question and that would be great thank you. Yeah, okay. I was wondering whether or not we heard this recent illusion to a slight change in maybe the Chinese approach to diplomatic language, maybe a move away from war for a diplomacy. Yes, you, the United Kingdom has a unique obligation to Hong Kong. Through the sign of British declaration that it should take this opportunity to if it's not going to affect anything, at least it would great it could shame China in the world community and embarrass it by its tool duty to uphold the sign of British declaration of the fact that has been breached and clearly to the Vienna convention by taking China to the International Court of Justice, making it plain where it has actually broke deliberately broken international law, so that this supposed change in China that it's going to be a little bit nicer is going to be shown up for what it is that they have supposedly no interest in abiding by international law, and clearly also doing the same thing by upholding the rights of the Newton Newton ease agreement in having its treaty with China abrogated and its land 200 square miles worth being occupied by China with very little notice being taken by the international community that that too could be taken to the International Court of Justice or somebody to sit in a Security Council of the General Assembly, which might well be to other Asian states, maybe sticking up themselves a bit more in the South China Sea, if, if somebody is prepared to start the ball rolling in abrogation of China it's duty of obligations and it's all of the law of sea etc. It might actually have some effect, at least embarrassment maybe more than that. What are the responses of your panel. Great question. Thank you. Sorry, we all did you want to say something. And I was just going to say that's a big question so good luck with that. Huge question. I'll try to tackle it the best I can. The first point I would like to make and I think that is something you also mentioned is the fact that you know Hong Kong is lost. There's very little that you know can be done to bring back Hong Kong to his pre 2020 status. And this I think is important because anything that will be done would be mainly symbolic which it doesn't doesn't mean it's not important, but it would be symbolic. What does this mean, it means that you know if you do, if the UK decides to take such an approach. So by being fully aware that is escalating the situation. So, and I don't think that at the moment, the UK is interested in escalating the situation which is why it opted for the option of granting BNO to Hong Kong citizens rather than you know, bringing Hong Kong to the International Court of Justice. The other reason is that China has a truck record of largely ignoring any resolution from international courts that has been taken in his regard and of course the most notable is that there's a lot of China see from the court in the Hague, sure is different that was an arbitration court this one would be the Court of Justice but again, knowing that type of behavior and truck record from China means that if you do that type of action you're fully aware that China is not going to comply, is going to respond very negatively. And at the same time, I'm not entirely convinced that other powers within the region would follow suit. So I think the position of the UK, I'm actually quite skeptical because they have to balance their position much more than any other European country has, they are in the region so they're more directly affected. And the reason why I think the UK wouldn't do something like this is that maybe I'm cynical, maybe I'm skeptical, terrible person, but economic ties and economic advantages that still are there in keeping a good relationship with China are still an important element. And escalating the relationship between the two like that would endanger such a such a tie. So I'm not saying that it wouldn't be, you know, theoretically the right thing to do. To keep China up to its, you know, international agreements and make it face when it breaches international law I think is something that should be done. I'm not quite sure whether you know the UK government or any other actor is up for the challenge at the moment. I think this is the best possible answer I can give to such a difficult question and I'm sure that many people would disagree with what I've just said. Do you not think at least that it should be put out there, the occupation of so much of the Putanese territory, that it hasn't hit world news headlines or come out of the mouths of any major international actors I find quite extraordinary. Sure. I mean, if you're talking about opening a debate in this regard, the short, I mean, of course, thankfully we still live in a part of the world where public opinion and free debates are allowed. But again, I'm not quite sure it would be consequential. Sorry, Leslie, we're going to have to, we're going to have to cut that flow. There is a very interesting flow and hopefully will be stuff we'll look at more, but I just want to get one more question out from from the Q&A before we finish. Thanks again for that Fran. And I wouldn't worry about people disagreeing. I think every person has their own view on China. So there's probably it's kind of like grand strategy for every book on it. There's a different definition. That's the same with China. I'm going to combine the last two questions because I think we can just about get away with that. One from the anonymous and one from Michael Minor, who can't unfortunately talk through the microphone. Michael asks basically to the entire panel where China sits basically in all of your spaces or in the space of the integrated review for the UK, which Francesca probably touched upon the closest due to the question. Speaking ahead. How do you see this competition folding in all of your spaces for the UK and NATO and I think also Dr Payne that that will that will touch upon the other question which is, you talked about how the China is coming from a different space organizationally when it comes to AI and as and implied it was behind the China, at least according to this question. Could you expand a bit more on that probably when you're answering your section of Michael's sort of larger question, which is an excellent question to end on, especially for the integrated review which is looking 10 years ahead. So, if we go Dr Payne first and then Francesca and then Ivanka. Thank you. Yeah, great. Thank you. I'll be reasonably brief reasons of time so I think the first thing to say is that you never know until it comes to a punch up power ultimately is revealed in the moment says Laurie Friedman and he's he's right. You never know how your kits and your concepts are going to perform until they put to the test against somebody else's. The reasons to to think China is well placed for AI it's got huge scale huge defense budget, and that's growing and rapidly. It's got lots of data to hand, particularly data internally gathered social data internally gathered that it can to train its AI and to figure out how it best works, got lots of computer scientists, many of them trained in in Western universities. It's got a permissive approach to ethics and more permissive approach to ethics to what you can do with that AI and that tech and that data. And that's true when it comes to surveillance. It's true when it comes to things like using AI in conjunction with biotechnology. It's got a permissive ethical environment. And it's been pretty good over the years at espionage to acquire ideas that it doesn't necessarily have. So it's got some advantages, but the disadvantages are profound. And they are I think economic fragility I'm not so sure that China is as well placed as we sometimes think to continue its economic miracle. The experience is reasonably high levels of corruption and centralization, both of which military against innovation. It doesn't have the same degree of soft power for the third parties looking on as does America still push up and as does Europe, and so it can't necessarily attract the best non Chinese national people working on the technology in the same way that American California and elsewhere can. And I think the story of China when it comes to defense technologies, maybe even more broadly, has been one of extremely good emulation, less good innovation. And so for those reasons, I think I'd be quite quietly confident that Britain is well placed relative to China. Thank you, Dr. Payne, if we could move on to friend Jessica. Fantastic. Well, I think I've explained quite a bit how China plays into what I was talking about. But if I can just say something about NATO, well, we've just heard that China is going to be quite high on the agenda in the NATO summit that is coming up. So China is going to feature increasingly prominently in that agenda is going to become even more important in the in the coming years. And although I'm not saying this happily, I do think that the competition side of the relationship will become much more present. And one of those people who's not 100% sure that the positive positioning towards cooperation in the climate sphere, for example, is actually going to be positive and an area of collaboration. I think that that too that there's going to be competition and friction. So I hope I'm wrong for all of us, but that is my assessment. That's great. Thank you, Fran. I look forward to the future article on climate competition as opposed to climate cooperation. Ivanka. So I'm glad you you ended on this question because it's a perfect net assessment question. So you know this is the reform and strategic capability is is ongoing exactly because of the need to be able to answer those kinds of hard questions with with evidence. And as Ken said, you know you never know until your, your, your power is tested in real world. Well the proposition is that there are ways of knowing war gaming is one and so that's why there's this interest of new sources of evidence new ways to test solutions to better understand the problem in a safe to fail environment. That's great. Thank you all. Like Ivanka said, an excellent question to finish up on. The integrated review is pushed out for 10 years and theoretically 20 in terms of the more conceptual stuff so it was a very good sort of future question to look at. So that's it for this panel strategic strategic studies panel for the wider conference thank you all for joining us. I believe the recording is on YouTube if you'd like to go over it again. Any last minute notifications again I would just reiterate, as in the chat 17th of June is the book launch for Dr Payne's book and watch this space for exciting new stuff coming from both Francesca and Ivanka in the future. I know I certainly will particularly based off their papers. Thank you all thank you for joining us and I hope you have a nice evening or if you're from Massachusetts like Michael have a nice morning. Thank you.