 Welcome, everyone, and thanks for joining us today. I'm Laura Azege, your course lead for the MITx MicroMaster in SEM program here at MIT CDL. I'm very happy to be co-hosting this live event with Mr. Kellen Betz. He's also a course lead here at the MicroMaster program, and you may know him already. And today, we are very, very fortunate to have here with us Dr. Milena Janiewicz, research scientist at MIT CDL, and she's leading the supply chain design initiative. Welcome, Milena. Thank you. Thank you for having me. We are excited to learn from you today. First, as you know, probably if you've been here before, we'd like to know more about our audience. So I want to kick off this event with a poll. So I'm very grateful for having Jan helping us with the poll today. We want to learn why you're here today. We want to know if you're here just for network design purposes. If you're very excited about that topic, if you want to learn more about the MIT CDL research initiatives, if you are more into technology and you want to learn how to use it on supply chain or if you're all about optimization models, which is a topic we really love here at the MicroMaster program. So while we let that populate, let's go to Kellen and the agenda for the session. Awesome. Thank you, Lara, and hi, everyone. So for the next 15 minutes or so, Milena will provide some context on supply chain design, network design, as a strategic decision and share some examples of trade-offs involved in this process. Milena will also show us technology, how combining optimization, simulation, and machine learning methods enable visual analytics to better understand the end-to-end impact of our decisions. And then Lara and I will ask some questions after she's done with her presentation. We are prepared and we'll definitely save time at the end of the event as well for your questions. And so please keep those questions in mind and get them ready. And please use that webinar Q&A function, that little button on the bottom that says Q&A. And please use that Q&A function for those questions. And also be sure that you're logged in with a name. You won't be reading any anonymous questions. And so please be prepared to participate. We look forward to seeing your questions. And with that, maybe we can enter a poll here and take a look at some of those results and share those results there. Awesome. So the question was, why are you here today? It looks like majority of you want to see how visual analytics can help me improve supply chain performance. That's awesome. I'm also a majority of you are more than half of you. I want to learn more about supply chain network design in general. That's great. I'm also a similar result for I want to know more about using technology for decision making. And that's awesome as well. And good to see many of you in our micro-masters programs who don't miss any of our events. And as always, always fun to see that as well. Awesome. Anna Molina, if you have any thoughts on that first poll result or not. Yeah. So interesting results. I hope I'll be able to give you at least some of the answers to some of these hopes that you are all having. I'll touch upon the visual analytics and network design in general. And then I would also love to hear your questions at the end to see if there's anything we've missed and that you want us to cover. So yeah, interesting to have such a motivated audience. Happy to be here. Awesome. Well, with that in mind, Molina, are you ready to kick things off with your slides there? Yes, yes, yes. Let's do it. So let me share my screen. All right. So you should all be seeing my screen. Perfect. So yeah, I will present in the next 15 minutes or so some of the work that we do in the supply chain design initiative around visual analytics in supply chain network design. So just say a quick reminder of what do we mean when we say supply chain design? As probably most of you know, in supply chain and management, we have various levels of planning, strategic, tactical, operational. When we talk about supply chain design, we are really at that strategic planning level and we are making decisions such as configuration of our network, number of facilities, what suppliers to use, how to coordinate flows between facilities, et cetera, et cetera, et cetera. So typically when you think about these strategic decisions, they have a very high return on investment and therefore I'd say supply chain design is probably one of the most critical areas of supply chain planning. Now, when we design a network, we basically need to consider some trade-offs. Obviously, I cannot list all of them here, but just to give you an idea of the type of things that we need to consider, here I took one of the decisions that is, I'd say, a very critical and important decision to make, which is how many facilities we have in our network and I've analyzed, okay, how does that decision on the number of facilities impact various areas of performance? So on the left-hand side, you can see that, as I am increasing my number of facilities, my inventory costs are increasing, but non-linear fashion, I can see a slightly different trend in terms of transportation costs. So when you talk about inventory, when you talk about transportation costs, we are here really in the realm of typical things that we consider in supply chain plan. However, if you look at the right-hand side, you can also see two additional metrics. One of them is the response time and one of them is the market share that are slightly less traditional and that are not considered as frequently in supply chain topics. So as I am increasing the number of facilities, my response time is decreasing and then in some specific industries, for example, if you think about e-commerce or e-fulfillment that will, of course, drive market share and ultimately drive the revenue. So ultimately, if I take this very simple decision, which is the number of facilities, I can see that both on the logistics side and on the business side, there's going to be various impacts on the performance and so what I'm basically trying to do is design a network in such a way that it is going to balance between the ability of the network to capture the revenue and the ability to keep our logistics costs under control. And so it can become a quite complex problem. To solve that problem, there's a variety of things that we can do and first thing that we can do is obviously use various analytical tools. So I think that you've talked about optimization models quite a bit in the courses that are within the MicroMasters but there's actually various types of analytical approaches that we can use. I'd say the most basic one is using some descriptive analytics where we're basically looking at data and trying to figure out some trends and patterns based on what happens. So for example, try to analyze how the demand shifts and how that will impact our network. We can also use predictive analytics where this is typically in the realm of machine learning where I'm trying to basically anticipate what will happen. So basically I'm looking at historical data, for example, and trying to derive what will be the future of the demand and then we can use prescriptive analytics which are the focus of operation research methods and optimization models are a great example of this type of analytics. So we basically have this toolbox of things that we can do. We have all these various type of analytical tools that we can use either separately or combine them together. And now the question comes, okay, how do we select the best tool? How do we select the best decision support for our specific problem? Now there one basic premise between the supply chain design initiative that we just kicked off at MIT CTL is that analytics are only going to be useful if they can actually help us drive better decisions in organizations. And when you think about it, if I take a given model, let's say an optimization model and I get the results of that model while solving a model is not the same as solving the problem, right? And so if, for example, the results of my model are not understood or accepted by decision makers if they don't drive actual change in the business environment, well then that model is actually not going to be very helpful. And so what we do in the supply chain design initiative is we basically try to create ways for humans, for decision makers in the organizations to interact with our models to gain better understanding of what's driving these decisions, build consensus and basically allow a higher transparency between the model and the solutions that are being proposed. Few things that we do here, one of them is visual analytics. So visual analytics at the most basic level serves to analyze different data and to create a better understanding, common knowledge among the different decision makers on what are some of the issues on some of the problems. And that is especially important if you take into account supply chain design where we saw there are impacts on costs but there was also going to be impacts on revenue on market share, et cetera, et cetera. So if I consider this problem and if I say, well there are basically impacts that go beyond what we typically consider in supply chain management I actually need a way to engage with a much broader audience and that audience is going to be stakeholders from finance stakeholders from marketing from sales department that can also be important decision makers in these supply chain design initiatives. So that visual analytics basically allows to kind of create a shared situational awareness to share knowledge and basically create an agreement of what the problem is. The second thing that we do is we use what we call human in the loop analytics which is basically allowing interfaces between our models, quantitative models and the decision makers through these interactive interfaces. So in the example that you can see here on the slide you can see one screenshot from a visual analytics tool that we have developed and in this specific case it turned out that indeed the proximity to the customer was a big driver of market share. And so the company basically was trying to understand how many warehouses they need to locate throughout the US in order to capture most of the demand but also keep the costs under control. However, they did not have a clear idea on what the relationship between that proximity of the demand and the market share would be. And so what we did there, we incorporated into the analytical tool this type of slider, the demand slider where basically decision makers can define these different functions and see how sensitive the model is, see how sensitive the results are to the type of function that they have defined. So this basically allows you to understand if the type of solution that you get is robust or is very sensitive to the input steps you are considering. Now to give you one example of how we perform this type of visual analytics in a very concrete case. I will take an example of a project I worked on recently where we had one global manufacturer that was basically launching a new product in a new market and they were trying to decide between multiple channel strategies. And so they were considering free channel strategies, home delivery, traditional or direct transportation where basically they would buy best a number of steps and these are shown here on the chart. And so what turned out is that in their decision to basically design their channel strategies there were a number of elements that were important. Some of those elements were quantitative elements such as cost and inventory and those are the things that we can typically model and simulate through our different tools but then there were additional elements for example customer service level, customer satisfaction there were more of the qualitative nature. And so what we did is we basically made a tool that allows to simulate the performance of these different scenarios and also that allows us to make trade-offs between these different dimensions of performance that can be qualitative and quantitative. So as a first step, if I go into the simulation part again, you can see there's going to be a visual app where I can build my scenarios define different parameters, et cetera. And ultimately when I perform that exercise I end up with something like this. I have my free scenarios here and I have my four KPIs here and then I basically get a number which gives me the performance of each scenario according to each KPI. Now in this specific case, what you can observe is that none of the scenarios is all green or all red. So for example, my traditional scenario has the best performance in terms of cost. My home delivery scenario has the best performance in terms of service. Now the question becomes, how do I decide? How do I make trade-offs between cost and service level? How do I choose when I am facing a ambiguous situation such as this one? Now here, the approach that we have put in place is basically to allow decision makers to define the weights that they give to the different scenarios. And so in our visual analytic tool, we will have in addition to a pain that allows us to build different scenarios and define the parameters, we will have something that will allow us basically to adjust the weight that we give to different metrics. And so I can say, for example, my cost is the most important, I will give it 70% and then everything else, 10% of importance in my final decision making. And then we basically engage with the company, we engage with the stakeholders and say, okay, what is your assumption on what those weights should be? And this is a purely subjective discussion. And how does that basically impact the recommendation that we get out of our system? And so we ran a bunch of different simulations here. And so what we did here is we said, okay, there's really two things that are kind of key in our decision making, and that's going to be the trade off between the cost and the service level. And so we basically ran this exercise by varying the cost, the weight of the cost between 100% and 0% and same per service level between 0% and 100%. So it kind of ended up with a grid that gives us various ways of prioritizing different objectives. Now, in this specific case, when we ran that, what we were able to obtain is for each of these scenarios, a kind of a composite score, a final score now takes into account all of the different weights. And what you can see here is that in this specific case, basically unless I am in this very extreme areas where I only value cost or only value service level, the moment I am in the middle and I'm considering a mix between these different objectives, I have a direct path that is the most recommended one. So my model is actually not that sensitive to the type of inputs that I need to put in here in terms of the weights. And so with a great level of confidence, I can say that the direct path is actually our preferred path. In other cases, we will get a much more sensitive model. And so, for example, we will then need to really pay more attention in, okay, are we putting 20 or 25% on one criteria or the other? But in this specific case, it didn't really matter. So as a summary of this very quick presentation, so we have some trade-offs that we need to consider when we are designing our supply chains. We have a lot of different analytical approaches here I've talked about simulation optimization, machine learning at the beginning, but there's others that can act to support the decision-making. However, we believe that creating intuitive interfaces to these analytical approaches is really what is going to help us drive decisions in organizations. And there I've mentioned the role of visual analytics, the role of human-in-the-loop analytics. Ultimately, it comes down to saying that the quantitative models and the analytical approaches are decision support tools and not decision-making tools. When I'm facing a complex strategic problem, such as supply chain design, that's not something we can simply outsource to a machine. There's always going to be a human intervention that is needed to account for all the things that we need to incorporate in our decisions. So with that, I would, if you would like to learn more about the approaches that we put forward in our supply chain design initiative, I would invite you to go on our website, so scdesign.mit.edu. And on that website, you'll find a summary of what we do, but also you will be able to download a white paper that we just published. And that white paper was built with collaboration, so of Koopa, who is a provider of artificial intelligence and has extensive knowledge in supply chain design. And so we really set together and collaborated on kind of putting together what we see are some of the most common challenges that companies are facing and what would be some lines of action, what would be some opportunities to address in the future years in terms of supply chain design. So if you go on the website, you scroll down, you can see, you can download the white paper. And if there is anything there that is of interest to you, please feel free to reach out and share your thoughts and ideas with us. And that will be all from my site. And now, I believe. Also, Milena, thank you. Thank you. Yeah, thank you for that. It was short, but it was very insightful. So thank you for all that you have shared with our audience. I find it very interesting this topic about the human-in-the-loop interfaces. We talk a lot about technology and how important it's becoming and when it comes to decision making. But the expertise and the knowledge of those experts that we have in every field will definitely bring the best insight out of any model we create. So it's very important that you bring this to us and that we share that with the audience because at the very end of the day, like as you said, and also Dr. Kabliss says in some of our lessons, people is the one that's making the decision. Like not the models, people do make decisions. So that's very important. Thank you for bringing that to the table. And one of the challenges and now to like go a little bit beyond what you just said, but touching on those topics, one of the challenges that company face a lot when creating those optimization models or any decision-making model relates to the number of variables you have or to consider when analyzing these trade-offs or the constraints they will have, how much of that complexity should they add on their models so that they are actually bringing good decisions because you can incorporate as many things as you want, but that doesn't mean you will drive a better result out of a model. So would you share some more about that on what you've learned on your research? Yeah, yeah, absolutely. So indeed, I think one of the biggest challenges that we find when establishing optimization models or other models as well is how do we make choices? How do we make right choices to reflect the business decisions that are at hand? And when you think about it, when you design an optimization model or any other decision support tool, there's a number of things you can do. So first you can define the scope of your model. So I can say, well, I'm only looking at a regional level at one region, one country, or even one city, or I want to have an end-to-end supply chain design model where I look from the supplier to the supplier to the customer, my customer. So that's the first choice that we need to make. The second choice is going to be relevant to how granularly are we basically describing each of the components of that model. So for example, how granular am I in the description of my flows or the facilities or the processes? One example there could be, I can say, well, I can just take my customers and look at it in a given city and consider that to be a single point. Or I can consider a model that incorporates routing decisions towards each individual customer in my city. And there I can have hundreds of vehicles performing hundreds of routes per day. And I would need to incorporate that as well. And so ultimately there's a trade-off there. So if I have a very large scope of my model, I'm probably not going to be able to be as granular. And that's for two reasons. The first reason is going to be the solvability because we still don't have computers that can solve a full end-to-end supply chain design model with that level of granularity, but also for practical reasons because simply it will require us to gather too much data and wouldn't be practical to drive the decisions. So then the question comes, okay, I can have a wide scope or a narrow scope. I can be very precise in terms of transportation, inventory, production planning, et cetera, et cetera. So how do I decide what should be part of my model and still keep it within a range that is feasible and that I can implement in the time that I have at hand? Now there, I'd say what I've seen is that a lot of companies start by using kind of generic approaches where we say, okay, no, a typical supply chain design model is looking at transportation costs, inventory, et cetera, et cetera. And that's good if you have a generic supply chain, but if you really want to use your supply chain as a source of value creation there, it really comes down to identifying what are for your company, the main drivers of value, the main drivers of cost and really focus your model on those areas. So if I'm in the pharma space, I will probably have very high cost of inventory. I will probably have very high service levels that need to be insured. And so that's something that needs to be translated in the features of my model. If I'm in the e-commerce space, there probably I really want to make sure that my last mile part is going to be properly modeled. But it's really coming back to, okay, how can supply chain design drive value and how can we best design our model to reflect that for our organization? So that will be my advice. Thank you, Melinda. That's great advice. I really love what you just ended there on, you know, kind of focusing on the business instead of using more generic models, really using a model that's focusing on your industry or your particular network design and utilizing some of that business understanding and some of that business context, right? And just kind of a generic model. Maybe that also kind of then goes to the concept of, you know, that requires a lot of human intelligence, if you will, you know, human understanding of that supply chain. But something that you also mentioned briefly in your presentation was the concept of machine learning and AI, you know, so that computational intelligence or that artificial intelligence, if you will. So I'm wondering if you can maybe just touch on a little bit more on the use of and how AI comes into this context, you know, for that predictive and that prescriptive stages, how AI and machine learning come into these models and visual analytic app that you built. Yeah, yeah, absolutely. So I would say the most immediate application of machine learning and AI in supply chain design is going to be on the model inputs. So when we design a given model, that model is going to take in a large number of inputs and those inputs can be, for example, relevant to the demand. It can be operational metrics. It can be the costs of different elements, et cetera, et cetera. So there's going to be a very, very large number of inputs, number of parameters that we are considering. And what we see typically is that traditionally, companies have used very approximate values of those parameters. So for example, you know, if I'm designing a regional supply chain and I need to consider the costs between a point A and B, I will simply look at my historical data and say, well, you know, on average last year, it has cost me, I don't know, that many dollars per pound to connect to those two points and I will use that as an input to my model and then design my model with that in mind. Now, what we've seen is that in a lot of cases, if you take that very simplistic approach where you just basically approximate in such a rough way, some of these input parameters, that can lead to huge errors in those parameters. So typically, you know, one project that we worked on, we looked at that approximate approach where we take the average and then we compared it to the real cost. And we could see the differences were up to 60% of the real costs. And so obviously if you are working with this type of inputs, the recommendations that you're going to get outside from your models are not going to be very valid because you're not putting accurate assumptions. And so I would say the first role of machine learning or AI is really making sure that the inputs that we're considering are correct. And here, for example, in the case that I've just mentioned, what we did for that company is we build a machine learning model where we looked at a bunch of different features such as origin, destination, GDP of a country, where we depart GDP of the country where we arrive, the weight, the carrier, the time of the year, et cetera, et cetera, and came up with a much more granular description of the freight cost. And then we use that as an input to our optimization model. And that obviously yields much better results in the long-term. So I would say that's the first application. Now, some other application that I don't think I'll have time to touch upon today are basically replacing a part of your model with an AI component. And that can sometimes allow us to decrease the complexity. And then the third one would be basically using AI to analyze the range of solutions that we have obtained. And so trying to understand basically better what's driving the results of our optimization model if using AI. So there's a number of options. I think it's a really interesting area to explore. It's just starting to develop. And I'm really looking forward to see what comes out in the next few years. Thank you, Milena. And I will like to switch gears to touch a little bit on a topic that I know many of us in the audience could be interested in. And it's thinking on the sign-in for sustainability. So you mentioned the tools we can use. You mentioned the trade-offs we have, variables we can consider constraints or levels of complexity. And you touch upon the service level as one of the qualitative aspects you include in the model. I would love to know if you can use the same tool and the same approach to design for a more sustainable network. Can we optimize on those terms? And if you have seen that done, what would you recommend or suggest? Okay, yeah, thank you. So yeah, I think there's definitely a few ways in which you can approach that problem. So the first approach would be something similar to what I've just described where we have a limited set of alternatives that we consider. So in this case, we had three different options for our channel strategy. And we have a number of different dimensions of performance and those could be expressed through a quantitative or qualitative KPIs. And so I can have no cost service level, environmental impact, social impact, et cetera, et cetera. And then there we basically draw from the multi-quartier decision making field and what we do is we basically analyze how does the weighting of the different option impact the recommended solution that we get. So that's kind of one way. And that's very much in line with the example that I've shown. Now, the second approach is to basically embed different objectives in our optimization problem. And so there rather than solving for a given solution and then assessing it according to multiple dimensions, we are basically embedding in our objective function the different performance dimensions. And so there I can, for example, have now an objective function that will have one component that is cost, one component that is environmental impact and one component that is a service level or something like that. When we end up with that type of objective function, I would say there's basically two ways in which we can solve for it. The first one would be to say, okay, I can bring everything to some common measure, some common unit. So for example, that common unit would be cost. And I can say, well, I can convert one ton of CO2 into a dollar amount, or I can convert a percent of service level into revenue and therefore into a dollar amount, or I can use a different measure of utility or something like that. And in that case, if we knew that, what we basically end up is with just slightly more complex objective function, but where we now still have a single objective that we are solving for and that now reflects a multitude of different subcomponents that we have predefined. So that's the first approach. Now the second approach is to basically not convert everything to a single unit. And that can be either because we don't know how to do it because it's sometimes not exactly clear-cut what is the translation between a social impact and a cost or service level and decision revenue, or it can happen because we don't want to do it because we want to keep those trade-offs explicit. And there we will basically come into the area of multi-objective optimization where we are now explicitly in our objective function considering various dimensions that now are not at the same level. And that kind of allows us to keep those trade-offs more explicit. And I think from decision-making perspective, it makes sense because now the company is the one that is deciding, okay, how much value actually I'm putting at my environmental impact versus cost and not just using a standard that they might not be aligned with to start with. That's awesome. That's definitely an area that I'm very interested in is incorporating sustainability and some of these concepts. And so to have those two different strategies, either multi-objective, if you will, or trying to convert it all into a single objective function, I think that's a fascinating concept. And I'm kind of trying to keep an eye on time here. We're getting pretty close to wanting to pull in maybe some questions from the audience and maybe we have time for one more question here, Laura. I think we have prepared and maybe to build you off a little bit on what you just mentioned, where you kind of may have these two different strategies. And so the one question might be, well, then how do you evaluate the effectiveness of each of these different strategies or more just more generally, how do you evaluate the success or the effectiveness of a model? And so are there KPIs or are there strategies that you have in mind where you used to approach this concept of evaluating the effectiveness of a different strategy or a different model? Yeah, I mean, I would say the most elementary way of evaluating the model would be to say, well, I'm looking at my expected performance, I'm looking at my actual realized performance and I'm trying to assess what is the gap and I'm trying to basically see, okay, how accurately was I able to model my business problem at hand. However, I would be careful with evaluating the quality of the models as such because we know that models are always going to be wrong. We know that they are always a very simplified version of the reality. And so those KPIs should serve more to basically monitor the success of the model within our supply chain design process. So the objective is not to develop the best model, the objective is really to put in place a process that allows us to monitor how successful our modeling was, what areas should we focus on and reiterate on that and basically have a continuous loop where we are continuously learning from the implementation and allowing to adjust our assumptions, adjust our scope, adjust our modeling approach or adjust simply the environment in which we are operating. And so I would say that the success is more going to be in that adaptability of the supply chain design process over time rather than coming up with one model that is going to serve us until the end of times and be completely accurate. And by the time we develop it, it's probably already going to be obsolete and we'll have to renew it. So it's more about that continuous design process that we need to be put in place. Thank you, Milena for bringing that. And you're already answering some of the questions we have in the Q&A feature because we have this on how often should I review or revisit my network design. So thank you for bringing that. Before going to the Q&A with the audience, I would like if Chen can help us launch the second and last poll just to see where we are, what you have learned today from Milena's insight. So while we let that populate and so to learn what are you taking away from this event, I would like to go to Tarun Kumar's question. So I'm summarizing it because it brings a lot of different topics but I wanted to know based on Tarun's question, how do you treat outliers and how accurate a model can be when we have situations like the pandemic or a trade war? How do you incorporate or not the outliers or how to decide treating them? Yeah, that's a really good question. Let's think just about incorporating risk and uncertainty, for example. So I have two different types of uncertainty and risk I consider. I have my business as usual where I say, well, my demand is going to vary between 80% and 130% over this period. And I'm going to put in my play in place a model that is going to account for all of those different scenarios that's going to account for the probability of those scenarios. And basically I'm now looking at a wave of maximizing the expected value over a range of scenarios which are all within a given business as usual range, let's say. So that will be like one approach to treating where we don't have these outliers. Now, what happens if we have global pandemic what happens if you have a very high disruption, et cetera. So the problem with that approach is that if I take the same approach and I say, well, I have 0.01 probability that I will have this catastrophic event happening, that is basically going to get lost in my solution. The probability is just so low that I'm not going to actually account for that in the type of solution that I'm getting even though the impact might be catastrophic. And so there I would say what I would combine what I would do is I would combine the previous approach with more of a scenario planning approach where you're explicitly looking at some of these catastrophic scenarios and you're trying to understand how robust your model is to those specific scenarios. And there it really comes down to how do you define those scenarios, who defines those scenarios and what should be the ones that should be incorporated. And what might happen is that you might end up with a model that is slightly less cost effective in your business as usual range but that is maybe much more robust in terms of those very high disruptions. But you almost need two different ways of looking at the problem to be able to come up with the recommendation that will be robust against two types of uncertainties and solutions. That's a great answer. I love that answer just kind of reinforces this concept of bringing in the human loop in the modeling process but also just the focus on the analytics serving the purpose of decision making and really bringing in that human side of it. Because these models often like machine learning models for example are often built on historical data and if something is really low probability event historically it's not going to predict that to be very high probability in the future either. And so you got to bring in that human intuition and that human insight. So that's a fascinating answer. So maybe if we could just enter a poll real quick here and we'll take a look at those results and then we'll pick maybe one or two questions here to wrap things up. So again, the question was and what was the most interesting part of today's session for you. And so thank you again for everyone who answered. We have lots of answers to our poll here so appreciate you providing the answer to our polls but it looks like many of you are interested in expanding your knowledge just generally in your network design which is awesome to see understanding how technology can enhance collaboration which is awesome to see as well, it's very interesting. Anna Molina, if you have any thoughts on some of those results of that poll there. Yeah, yeah. I think this is, I think understanding how technology can enhance collaboration and supply chain that's a very, it's a very wide topic I think we could probably spend not one hour but several days discussing that. Again, I'd love for those of you who get a chance if you can download the white paper and read some of the things we've put there and come back with any thoughts you have. And I would love to hear your feedback and learn from your experience working in company and organizations that are actually performing this, how does this relate to you and what are some of the things you find are interesting or are missing. So I invite you to do that. Thank you. Thank you, Milena. We do have one more question if you are okay with it. We are on time but we don't wanna leave the audience without sharing some of their questions. They wanna learn about how to design for reverse logistics. Do you have any recommendation on including reverse logistics? Do you plan for it in advance or you just incorporate afterwards when you need it? What is it that you learn so far on that? Yeah, I think it's a very interesting question. It's a fascinating topic and it will really very much depend on the type of reverse logistics system that you are putting place. I think one of the key differences in the reverse logistics systems to our typical supply chain design is that we are now incorporating a much larger number of stakeholders and actors that are often outside of the organizational boundaries. So if I'm for example, taking reverse logistics for plastic bottles, et cetera, I will have obviously the company that is ultimately going to be using those but then I will have a network of processing facilities located throughout the country that is now under the public governance. And so I think how do we establish that interface between those different supply chains that are actually connected but often governed by different bodies? That is one of the most interesting topics to me. In other cases, a company will have a much more control over its entire supply chain including the reverse logistics. And so there the question is basically how do you just extend the current model to incorporate these reverse flows? But I think depending on the type of situation you're in, you will get a very different, you will need to have a very different approach. So yeah. That's fascinating. Is there a different area that I'm very interested in as well as reverse logistics? I think there's lots of opportunity there. And we're running very short on time but maybe Laura, if you have one time for one more question, it's probably a pretty quick question. And it really kind of summarizes there's a number of questions in here. I'm just kind of grouping it all together. It should be pretty quick but a number of learners seem to be very hands-on. They're very interested in what tools you're using. So what software you're using for some of these new models and the visual side of the models as well. I don't know maybe if you have any tips or tools that you might be able to share. Sure, sure, sure. So I mean, basically the tools that we develop, there's always going to be a front-end part which is basically develop on a framework that is developed by the computational and visual education lab, the cave lab at MIT CTL. So they've developed basically a framework that allows us to visualize all kind of network data and other things as well and that we are heavily using in our projects. And then on the back-end side, it's going to be typically Python and then solvers that can even open solvers or Ruby or whatever you want to use but basically those would be some of the tools that we're using. Thank you, Milena. So we invite everyone to check the cave lab and the supply chain design initiative at MIT CTL website. You will find it there if you look for it. We appreciate your time, Milena. With us today, we learned a lot and we are sure our audience has taken a lot of great insights out of this event. We want to remind everyone that this is a webinar series. So it's not only the first live event for SC2X and SC4X, but it's also a webinar series. So we look forward to having you all joining us. Kilin and myself will be hosting those too. Coming next in also technology and connecting with network design. I don't know, Kilin, if you have any final words for Milena and our audience. Yeah, Milena, thank you very much for your time. I appreciate you sharing your insights and thank you everyone in the audience for participating in our event. We have lots of questions, you have way more questions we have time to answer. I appreciate you sharing your questions and your participation in our polls. So thank you, everybody. Thank you so much. Thank you, Milena. We'll have you again soon, I hope. Thank you. It's been great. Thank you, everyone. Bye-bye.