 Hello Welcome to the SC forex live event We're very excited to have you our guest today is dr. Daniel Merchan And we're going to talk about applications of machine learning and supply chain management Before I introduce our honored guests. I just want to give you a couple of sort of house cleaning notes for the course So I'm your course lead Dave Correll You've gotten many emails from me and you've got one last night Reminding you that for verified students your week seven graded assignment is proctored The whole thing is proctored and that could be sort of a tricky thing to do On your end as the learner So we really encourage you to take the practice proctored exam to read those many pages of detailed instructions And to make sure that you can execute the exam in the proctored settings We're doing this because this is the last course in the sequence for you And if you're going on to take the CFX that will also be a proctored experience So what we're hoping for is you can try it now get used to it. It's honestly frustrating So we understand that for you So please try it now when the consequences are perhaps not as high and then you can understand it before you go Through the experience with the CFX. I hope the midterm went well for all of you I've been watching the scores as they come in and it has been going really well So I'm really really happy for you guys Okay, the main event We're very honored to have dr. Daniel merchant here with us Daniel recently completed his PhD Working here at MIT with the mega city logistics lab He has quite a bit of experience Particularly as it relates to big data and working with companies to solve logistical problems So it's really an honor to have you here Daniel. Thank you so much. Happy to be here. Thank you for having me. I Wonder if to start could you tell the viewers a little bit about your experience in this world of supply chain and big data? And in some of the projects that you work on sure sure so as David said I am I'd recently finished my PhD here at the Center for Transportation Logistics Working on how basically we can help companies use data and use analytics to innovate last mile delivery operation That has been the space in which I have focused E-commerce is pretty exciting at this point and also in general last mile operations in large metropolitan areas offer many Opportunities for innovation. So my my research has focused on how can we bring these new sources of data that we now have available? How can we bring classical methods that we have been using always in logistics like optimization models along with machine learning methods to come up with better decisions for Logistics operations excellent and for that we've I think we've personally have worked with companies in Latin America, India in the United States and even though the context might differ but Also, there are some challenges that are quite quite similar across these geographies and and I think that both data and in these new Machine learning models can can can offer an opportunity to tackle some of those problems Great, but when you work with these companies, are they all part of your own dissertation project or is this a cross variety of projects? It's generally across a variety of projects, of course As a PhD student you want to align as much as you can your research with the work you do with the companies as part of the research project CTL engages with And but but it doesn't have to be the case So nevertheless, even though if even though a project might not be part of your dissertation You still use some of the insights you learn from that project You still may use some of the data you got for that project to strengthen your work. And that was my case my primary case study within my dissertation was the operation of a large e-commerce platform in Latin America is in Brazil specifically They were interested in redesigning the last mile operation in the city of Sao Paulo. It's a very large They serve around 15,000 customers on a daily basis. So they they had a complex network and they needed to run They wanted to redesign their operations to support future growth and to make sure that their offers their operations were as efficient as possible however, very similar where I was exposed to very similar problems with other companies in other cities and That's where I got the opportunity to Validate some of my ideas from my work in these other projects. So it's a bit of both I guess you can you have case studies that are a Big portion of your dissertation, but at the same time you have other projects You're exposed to other cases that help you strengthen your your ideas and test your methods Excellent, but before we start on your dissertation many of our learners have Undergraduate or maybe master's degrees or thinking about master's degrees some even thinking about PhDs Could you tell the learners just a little bit about your undergraduate and graduate education before your doctoral degree? Yeah, absolutely. So I'm I'm originally from Ecuador the southern part of Ecuador my town is called Cuenca grew up in Quito got my undergrad at my home university in Ecuador University at San Francisco in industrial engineering and that's how I had my first Exposure to the logistics world through one of my undergrad classes Then came to the United States. I did my first master's at Texas A&M University of Masters in industrial engineering Then went back to Ecuador Worked primarily in an academic environment. So I was basically teaching undergraduate courses in logistics And that's how I became interested in pursuing a PhD. It was very Involved let's say in the academic world Was teaching a lot but not doing so much research So I wanted to develop that profile and that I When I was in Ecuador, I learned about the skill network MIT was CTL and MIT were launching the skill network And they were beginning their efforts in in Latin America. So I say well, this is a perfect opportunity I really wanted a PhD that look both at what happens in the developed markets but also in the emerging markets because they all offer very interesting logistics problems and It took some time to prepare of course to come to MIT but after couple of years of preparing exams GREs tough holes and strengthening my profile I joined MIT in 2013 first as a master student and now I recently completed my PhD We're very glad to have you. Thank you so much Maybe now would be a good time to tell the learners a little bit about your dissertation project So as I was saying before this case study was in Brazil and What this company the challenge this company had they had already a multi-tier distribution at work basically when we say multi-tier and maybe you have seen this already in previous classes or in previous live events I think Matias was here a few few weeks ago and basically when we refer to multi-tier distribution we mean this Urban distribution networks that combine large facilities large fulfillment nodes with smaller satellite facilities spread through all the air the city to Facilitate transcription or as a temporary storage locations so the company was basically interested in understanding to which extent their current network was efficient and Also planned for future growth So we engage in a project which we help them Basically redesigned their operations and this redesigned until Understanding whether the current location of the facilities was the optimal one and they should look into other possibilities Understand also the allocation of different service areas different neighborhoods within the city to those facilities and finally the vehicle choice They had made whether in downtown Sao Paulo. They should use the same vehicle as probably they use in a residential neighborhood So that was kind of like the decisions they were facing. So this is a classic optimization problems This is where you basically bring your location models or your location routing models and and try to solve this problem However, as we were testing classical methods that we've used in logistics, we'd notice and and if we talk about optimization Usually you're trying to minimize the cost or or maximize some sort of performance measurement And what we notice is that there was an opportunity to improve or cost predictions To then better inform the optimization model So in in this setting for instance Part of what we want to optimize is the cost to serve a neighborhood and that cost serve a neighborhood is based as a function of the Distance or the time it takes you to actually operate in the neighborhood and that might be driven by the local traffic conditions The road network you have available. So it's a quite a quite a complex Scenario and what we notice is that classical methods were failing to provide a good prediction as on how much it would cost to serve Those different neighborhoods So that's where we brought machine learning methods to help us improve or prediction of The time it would take to serve the area the distance would be required to serve the area and ultimately the cost Estimates of serving that region and with those better estimates now We go back to the optimization models and we Basically redesigned a network or we run our analysis to understand. What is the optimal network design? So I think this is this is an example of how I think machine learning can have a role in Decisions in supply chain management is by actually providing us better ways to predict either demand or cost or distances that we then use as inputs to the other decision support systems such as simulation models or optimization models that Ultimately help us make better decisions Can you tell the learners a little more about the predictions? Sure the models how you were getting the predictions where they set neighborhoods or how the How would how was it determined? The category for the prediction that makes sense. Yes, so Those of you that have taken se1x. Maybe remember Chris's lecture on routing Approximation right so they use a very simple analytical expression to calibrate to try to approximate how much time how much well the distance Do you will need or the expected distance? You will need to serve a given area what we notice is that Those approximations might work well in some settings But especially in cities where you have these Levels of complexity because of traffic because of the road network properties one-way streets obstacles and so on and so forth Those approximations were underperforming However through new sources of data Google Maps for instance or open-street maps so basically by bringing new layers of information to these classic models We can significantly improve the prediction So I mean technically this is a supervised learning problem because we we will use Routes to train the algorithms will use routes or near-optimum routes with their corresponding Distance estimates to train the algorithm But the key part of that prediction was that we were able to bring new sources of data that were probably not being used before Because they were not available now. It's for instance very easy to To extract real distance estimates or time travel estimates from Google Maps This was hard to do five years ago now. It's becoming easier So I think the key part of of improving that prediction in any neighborhood was that we could use Data that was specific from that area or from that region within the city to actually calibrate a predictive to Tailor the predictive performance of the algorithm of the to the local conditions of that of that area excellent Could you tell the learners a little about? The data collection and perhaps the data cleaning work that went into doing that kind of project. So we we That's always the fun part Data cleaning data processing is always fun because it takes quite some effort in the sense that Depending on the data source you have available. You might need to spend a little time or a lot of time cleaning it I I think we use in that particular project three sources of information one was company data and Usually company data is well-formatted. Well structure and that was relatively easy to process and we use company data to predict To approximate or predict the demand in a neighborhood. So I we use past orders to Predict how much or what would be the intensity of demand in a given zone But then to bring road network data and traffic information. We use Google Maps Google the Google API is quite easy to use now and it offers High-quality data. So there's very little cleaning involved However, if you want to bring road network information like basically how many one-way streets you have in your neighborhood or stuff like that Open a street map is a great data source. However, it's crowdsourced. So It's not always Fully accurate. So there might be some data cleaning that has to take place I think generally these data sources that we need are becoming More reliable and more accessible but the still the data cleaning process is is is important and I think the key the key aspect of the work we did in Sao Paulo was that we were able to triangulate not just company data Not just transactions and orders and and and even GPS traces from the company But we also brought information from the context from the city the traffic the road network And the combination of those two is what I think really drives an improvement on the prediction models we develop Excellent I see we already have a question from a live viewer and just to remind any of you that are watching now You can participate in this discussion through Slido So you have that link in the message from me and on the course page So please let us know if you have any questions or ideas So Jesus Madrid asks Daniel to what extent and how was simulation used in this research? Great question. Thanks. So I was mentioning before that we managed to combine Basically machine learning with optimization models, but you can think of it exactly the same way if you want to combine machine learning with simulation at the end Simulation is a tool then we do which we try to replicate the system to understand how it performs And you can still use machine learning to come up with better Parameters better estimates of the cost of the time or the distance that eventually feed your simulation model So we have done some simulation analysis in last mile operations because sometimes our corporate partners are interested in Finding an optimal network design, but sometimes they are interested in understanding how this their current network Perform on their different demand scenarios, and that's where we use simulation. So we simulate let's say the operations on their normal Demand patterns or the usual demand patterns, but then we simulate also when they have a peak season like Christmas or Black Friday and Actually what and the logic is the same, right? So we use machine learning to improve our predictions to improve our estimates and our parameters That eventually feed and and inform either the optimization or simulation models It really depends on what you want to do it really depends on whether you're more interested in understanding the performance of your system or you're interested in in a Network in a in coming up with an optimal or near optimal network design Usually in practice companies do both and you can leverage the same tools and the same data to achieve both both both goals Thank you for the question. Thank you for the answer my pleasure You mentioned when you were talking about this project that you use the machine learning techniques to come up with better predictions You have a sense by how much? The machine learning predictions outperformed the classical predictions that did not include these new data sources. Yes so it really depends varies by by Based on certain conditions, but on average we found that by using these improved models We were able to bring the error down by approximately 20 25 percent so the improved prediction still has a Error level of between 10 and 15 percent depending on on again those conditions, but it's About 25 25 percent better than what we were doing before So it's it's a significant gain and I think The key part is not so much the algorithm itself is at least in my in my example In other cases the algorithm might be playing a significant role in that improvement I think in this particular case was that we were able to use better data and I think those two pieces are very important when we do machine learning studies and machine learning projects is not just about The the availability of no algorithms because you might be using maybe a classical linear regression or a classic logistic regression But just by using better data you can improve your prediction So it really depends on on the specific case Just out of curiosity, could you tell us about the prediction model what models you used? sure so We use a variety, but we actually always start and that's something we do in every single project We try to use a combination of models So if we were predicting let's say travel distances or travel times just to give an example We this is probably a supervised learning problem in which we can try regression analysis, but you can also try random forest and I It's hard to say whether one is better than the other before trying them, right? so this usually in practice what you do is you you bring a set of models and even Different variations of the same model. You have a basic logistics linear regression for this is and then you can try a polynomial regression And or you can try a lasso regression so different versions of a classical of research regression problem And other models such as random forest and what we usually do at least in our team We tried all of them and then we said we says which one gives us better performance But also is all which one gives us Better insights right so helps us understand the correlations and the relationships between the variables we're using to predict and ultimately the Measurement that we are interested in predicting That's great. And could you speak a little bit for About how you choose which model is better than others So there's sort of classic measures of model performance, but it sounds like maybe there's some other things that you consider too And I guess I asked because I know for our learners for example in your mid-term exam We asked you to look at area under the curve and then tell us what was best and move on But maybe it's more detailed in a real project. How do you? So what we do is measures so what the way we usually go by this is If you have several models you want to you want to compare You have to be of course the first thing you look at is the R square as a measure of Model accuracy how good your model is But you have to be careful because especially if you have a large data set There's always a risk of overfeeding the model. So basically you are Modeling the noise and not so much the signal So that's why you usually good idea to break down your data set into a training and a testing data So you train your model with the proportion of your data set And then you assess the performance your model on a data set that has not been used for training So that's a way to control for overfeeding and then you usually monitor the R square both for the training and the testing data set so that you make sure you don't Overfit the data although there are other techniques to also help you not to overfit, but usually Splitting your data and training and testing. It's the first very first step and I think our square if we're talking about Predicting a numerical value the R square is usually the measurement You used to compare if we're predicting a category and usually the classification accuracy But it's some some measure of model accuracy is what usually you used to compare different models against each other You know, we've been asking so many specific questions I know you brought some slides to share with our learners. Would this be a good time to yeah, absolutely so that I Brought a couple of slides so that we can also talk about the different types of machine learning models that can be used because Perhaps the most intuitive one is we can say well, I want to prove I want to predict demand actually demand prediction is a Very it's a You critical decision supply chain managers have to do so and that's where there are some Opportunities for machine learning and we can discuss that in a minute, but we have the world of supervised machine learning, which is how We predict a number or a category based on Data that we have observed But there's also the world of unsupervised machine learning and the easiest way to think about it is What if we have a data set with multiple variables? But you don't have a classification and you want to come up with a classification and you can think of this as I want to create a customer classification Based on several parameters that have I've never combined them before and the more parameters You have the more difficult that classification becomes. It's less intuitive So in unsupervised learning what we usually do is we build machine learning algorithms to help us understand the structure of the data And ultimately derive classifications and I have an example in the screen. This is Part of the research we have done with my colleagues at the mega city logistics lab Imagine that you are an urban logistics planner or you are a supply chain planner in a company And you're trying to understand a city and it's actually a screenshot from Keto the capital of Ecuador That's where my home is now But the idea is that so from from a logistics standpoint You might want to classify the city and maybe the classification that you get from The city itself the neighborhoods is not as important to you because the doesn't include the variables that are relevant for logistics So in this project we actually use K-means clustering one of the unsupervised machine learning methods to help us group the different areas of the city each one of those Dots you see in the map is a square kilometer and That's how we segment the city and we want to try to understand which of those square kilometers are similar have similar characteristics and then we can group them together we can cluster them together and which of those are unique and To give you a sense of the ones in purple those segments share a The the several variables specifically these are where the manufacturing district of the city is concentrated so by bringing data from basically census data or Data from economic activity plus other variables such as the road network data such as whether my car where my customers are So we're basically bringing to this analysis different layers of data different variables that we think are relevant for logistics This is of course one example, but in these analysis We thought well what are the variables that drive last mile operations that make it more difficult or more complex? And that's of course population density the density of the retail establishment the retail dynamics of the sector But also the availability and the type of road network you have it there and by but of course This is a completely different worlds population density road network information It was not intuitive for us to see how then we classify this together So we use machine learning for that and actually came up with this nice classification of the city into stereotypical zones That we can then tailor Logistics decisions to those areas so we can then tailor either policies or logistics strategies to each one of those segments in In in keto and for instance you see some of those clusters are larger. Maybe those are the residential areas Whereas for instance the blue cluster that kind of spreads through the city And those are different residential areas that are both in the northern and the southern part of the city Whereas you have a few segments that are a few clusters that are just made out of three segments Very small ones and those mean that means that those are very unique source And actually that happens to be downtown keto and also the entertainment district So those zones share very those have very unique characteristics that is actually where most of the The density of demand for let's say restaurants or retail establishment concentrates So by using classroom analysis, we now are able to segment any seat in the world based on these variables And then start tailoring or decisions either from private sector in terms of tailoring strategies Or from the public sector in terms of table lowering Policy measurements This is a great example. Thank you so much and you said that you spent some time growing up in the city Yes, so when you looked at this map, do the colors correspond to your understanding of the city? Absolutely. Yes, absolutely. So and and that's a great point because and We can use machine learning, but that doesn't replace human judgment, right? And it's always important once you run the algorithms to validate and to double check and actually Make sure that the the the results that you reserve are consistent What would you expect at least I would have expected Downtown keto to be unique and and that's what we see However, it was also harder to think of what are the other categories that we will find So that's a combination of judgment of subject matter expertise local knowledge Along with the use of machine learning methods to actually come up with this classification And and this is one example another common example of uh, maybe not in the logistics world, but another common example of Of unsupervised machine learning is basically what you guys are doing now We were on youtube and youtube use these recommendation systems So does amazon when we buy them out and buy something online spotify all this Web platforms that now give you recommendations of things you might like They're using unsupervised machine learning to say well They based on your search profile and the things you've been watching you might like this new video or this new book, right? So this is another example Probably that we're a bit more familiar with and actually the truth is that we are using machine learning every day in different apps and different tools that we use It's part of our daily labs Absolutely, and I think that's a great example We talked a little bit about in lecture of the finding common products or that might be Interesting to a consumer and I think it ties to this and that if I look at what's recommended to me It might not be the same answer that I would give if you were to say hey, Dave, what books do you read? I might give books that make me sound smart. Yeah, the algorithm would know no day This is what we've seen and so we get maybe a Surprising but sometimes truer picture of what the data really suggests I wonder did you see anything like that in this city where maybe you would think Gosh, I thought this was going to be Its own residential area, but perhaps it's a little more like something else. Do you find those kind of counter-intuitive discoveries that are supported by data through this technique? Yes, one example was actually the manufacturing district So if we look at the city and based on You know our expectation and our local knowledge We really thought that we will see a similar manufacturing cluster in the southern part of the city But even though there are some manufacturing facilities there It's not as concentrated as it is in the north and actually the classification suggested that that felt really more like a residential with some commercial manufacturing activity, but the predominant let's say dynamic in that zone was Or the predominant land use sorry was a residential purposes. So yes, we found things that we We weren't expecting but at the same time we also find for confirm some Results that were based on our intuition and our understanding of the city One more question on this could you maybe walk the viewers through and this could be just hypothetical Now that you've found these different groups and you're doing it for the purposes of logistical decisions What kind of logistical decisions would be different for servicing say the yellow dots compared to the purple dots, right? so actually, this is a project we we finished or recently In collaboration with a large Development agency and local governments in cities Keto Bogota and Lima and our idea was Based on this profiles, how can we inform policymaking and one example is that if we look for instance the two clusters the two most relevant clusters downtown Keto in brown and the the other one the In entertainment district in red the smallest ones These are both dense zones very important from a logistic standpoint. They have a very large retail density but stores in downtown usually operate from 8 a.m To 5 p.m regular business hours whereas in the entertainment district the restaurants operate maybe from lunch To midnight So once once you capture those differences then you can tailor policy because part of urban freight policy generally entails restricting access Within certain periods of time. So this type of analysis can help them understand. Well, maybe this given based on this profile, then it probably doesn't make sense to Establish a access restriction Let's say around noon in the entertainment district because that's where most of the activity is taking place so this can actually inform The characteristics of both retail dynamics logistics operations in the zones and then for instance help us better understand better tailor access restrictions or even the development of infrastructure parking is usually very limited and by by understanding these patterns we can Not necessarily dedicate parking full-time to freight operations, but we can say well within this time window These spaces that are usually used by residents now can be used by trucks just within these two or three time windows So that you know, we can facilitate logistics operations in these in these areas. Oh, that's great. Thank you I think there's another question Oh, there is and we have a great one just a reminder for those of you watching You can interact with daniel and myself through slido and it looks like we have a question Great. Uh, this is from hawn. I says congratulations on earning your phd Uh, he points out rightly that we used orange for machine learning Can you talk about what you use? Uh, it's a software package So, um, orange is a great tool. Um, I like it a lot because it kind of has the applications ready to use However, sometimes when you're dealing with new problems and with large data, you might want to have some more modeling flexibility So in my case, I use python as my main coding environment and python has a set of great, uh, machine learning libraries specifically a skill learn Um, uh, it has all the supervised and unsupervised machine learning models Already implemented kind of ready to use So the python is a very important part of Our research and the good thing about python is that it's not just machine learning specific. Actually python is a general Coding environment and it allows us to combine in the same code Machine learning models with optimization models with simulation models Something that it's for instance difficult to do if you're using orange because you cannot really embed an optimization model within the orange platform so, um, that is that is the tool we use and and I think an important part of it is that This is a it's open source. So python is open source. So that's why we like it We don't have to pay for it and in general the data science community is uh advocating more and more for the use of tools like python or are This is free open source environments For machine learning. So there's that that is usually the tool set we have And of course on top of that sometimes we use Tableau to visualize results tableau is a great tool. It's not free. It's not open source Um, I think python is getting better at visualizing things. So as python gets better We'll probably transition even our visualizations to uh to python It's a great answer and thank you for that great question Uh, follow up. I'm gonna skip ahead on some of the questions But one of the things I wanted to make sure I asked daniel Uh on behalf of the learners who are watching is I think many of our students will be inspired by The work that you do and the project that you've done What are sort of the skill sets that you think learners should develop now to be able to perhaps go into a graduate program Or even a consulting position and do projects like the ones that you're sharing with us, right? So I'll answer that question in in two parts. I think one is what learners can can do. Um, and the other is what organizations Should be doing or can do I think learners The best thing they can do is they continue doing what they are doing now Leveraging the resources they have now available tools such as edX kursera to Take machine learning courses and uh ai related And this is actually a very large Variety of options out there all the way from the course that teach you the implementation aspect of it It just if you just want to learn how to implement a model in python and code it and run it There are tools there are courses for that um There are courses for if you want to learn more of the the math behind the models like the the framers the analytical framers and in a bit a bit Bit of more of a theoretical understanding of the problems you also have Courses for that actually there's one very popular course in kursera by stanford on deep learning that You know, it's it's it's out there anyone can take it. So I think that's um One way to go actually I have myself Throughout my phd leverage some of the python courses available on edX to learn those skills that I couldn't find in a regular class So, um, I think that is an important piece and especially supply chain managers. I think they need to become more and more confident comfortable and familiar with analytics and I think sc4x is part of that vision um, and the reason is that If we want to and that's where I get to the second part of my answers If you think this from an organizational perspective and how companies can leverage this potential It's really not so much about the tools the tools are secondary. It's about the teams you put together it's about the people you put together to work on these problems and based on subject matter experts suggest that you usually need to have and we have witnessed this in practice That you need to you usually need to have a combination of three skills For tackling these problems and for basically leveraging the potential of data analytics in organizations First one is analytics. Basically, you need people that understand deep learning that understand neural networks that understand all this Basically the math behind the these models Because they are actually the ones basically building these analysis you need computer scientists That's the other set of skills you need to have And because they then they are the ones Processing the data cleaning the data and implementing and scaling the models that the data scientists develop And the third I think the third role that you need in those teams and they need to work together, of course Is actually the are the subject matter experts in this case The supply to managers the logisticians And that's how we actually make sure by having those three roles in a group And if you rarely have all that in one person You always need teams you always need teams because people have Skills and you when you combine those is where you get the most of the value out of it And I think that especially for supply to managers They are the bridge between what the analytics and the computer science do and then the decision makers within the organization They some people have used the term data translators This is just just the idea that I am someone that is very familiar with the business. I know the business. I know logistics I I know the operational aspects, but i'm also familiar with the analytics and I can translate what the analytics team is doing To what the organization needs to decide upon And so that that role is very important. I think that role has to be Taken over by the subject matter expert by the logistics people However, they need to be comfortable with Analytical ones, maybe they are not the ones that are going to be building a deep deep learning algorithm But they should understand it and they should be familiar enough so that they can extract the value out of it And connect that to the needs of the organization That's excellent. Thank you so much. And as Daniel said, I think our course can hopefully be a part of that for our learners Those of you that want to go deeper into python and to coding these things yourselves Daniel just gave a lot of great resources there and hopefully on our side We can get everyone comfortable with some of these tools so that you can be the translator that daniel talked about It was just meeting with a partner of ours recently who said that same story, you know We we have all of this data. We're getting all this great analysis, but we need people who can Translate it to the pictures and the problems that our company faces. Yes, absolutely. And I think Usually translating data means telling a story right telling translating complex analysis Into decisions actionable decisions is usually entails Telling a story about the data and then translating the data that the data or those results into something managers can use to make a decision and I think I just wanted to add that Visualization is an important part of that of that puzzle Because the visualization I think is the way how we can convey complex analysis and complex models How can we translate those into actionable and understandable decisions? especially for members of the organizations that are not as familiar with the analytics as the others are and Is ctl we have been increasingly interested in Understanding how to better visualize and communicate those complex analysis to decision makers I think that's a part that shouldn't be underestimated communication visualization storytelling the very very important piece of the data translator role That's a great point and it's it's an art in itself You know, sometimes you see something visual that is beautiful, but I can't understand it Very complicated. I know there's power there, but I don't understand it And sometimes you see something like what you prepared with your map here that I think is A perfect representation of some pretty complex ideas Looks like we have another question This is from edmar who asked about using econometric tools combined with machine learning tools Yes, we have and Actually, some of the tools are very similar, right? So you can use regression analysis From a machine learning angle, which is when you're more interested in predicting rather than understanding relationship, which is at least as far as I understand one of the Motivations behind using econometric tools So I've used econometric modeling in my own research to basically understand the correlations for instance between Road network efficiency and the properties of the road network the amount of one-way streets you have available for instance in the area However, in machine learning, we're a bit more interested in Being more accurate in our predictions And that's what that comes usually at the expense of interpretability. Whereas where econometric models are are Are basically strong at So in what at least the way we approach is because also understanding this relationship is important even though your interest is mostly about Better predicting demand and trying to be as accurate as you can It's always useful to have this This is a better understanding of the relationship So we usually start with a simple model a simple regression to try to understand the correlations And then building on that result we get into the more advanced machine learning models that probably are less interpretable But are Better at predicting more complex relationships. So we we use a combination of both and I think that's usually a good a good idea So that you have both You kind of cover both ends Great question Edmar. Thank you and great answer I think it speaks to we're in a really neat place as people who are interested in this topic and that there's So much free software out there and people can learn so much about python coding to bring those models together in a way that Would have been difficult Years ago when you had sort of paid for packages that didn't necessarily have all of that capability Yep, absolutely. This might be a nice time to look at our word cloud So we asked you viewers to tell us Which parts of supply chain management do you think could benefit the most from machine learning applications? and so bigger words mean More people thought that so I might ask our resident expert to comment on some of these words and just say maybe do you see this is Supervised or unsupervised do you think about certain types of machine learning models when you think about some of these issues? What what jumps out at you here? demand forecasting and I mean I Agree 100% with you guys on on on this um, I remember when I when I came to MIT First class with professor Sheffi and dr. Chris Caplis Very first course, which is actually a replica of se1x Very first course on logistics week one demand forecast. So the demand forecasting is, you know fundamental problem in supply chain management and actually I think that's where machine learning can play a role We have used some methods and classic methods Exponential smoothing we have used even regression analysis to predict demand I think the opportunity now is because we have more data and we have slightly better algorithms or we have Made improvements of the algorithm and we because we also have computing power more computing power more storage capabilities That's where we can really Build more advanced demand forecasting methods and actually there's a very nice example It was came up in the economies a few weeks ago. It's a german retailer is actually used is using deep learning Basically advanced neural networks to better forecast demand and the reason is that they found that because they were not Basically being good at it. They were underestimating the demand and sometimes that led to delayed shipping Orders to customers or they just meant that they would have if you place an order and you have five different items You will send five different shipments and of course from As a customer you just want to have one delivery. You don't want to have five or maybe two but not not five for sure So by using deep learning and by moving away from just maybe one or two variables used to predict They are now using 200 variables in their deep learning framework all the way from past sales to weather data to web searches and You cannot do that with a basic linear regression. So that's where you need deep learning and But by using the learning they've asked you to improve significantly their forecast accuracy and If you can predict better your demand then you can plan your capacity or preposition your inventory So many decisions that depend upon your ability to predict demand. So I personally Believe that demand forecasting is an area where machine learning can play a big big role In helping us helping us better predict those parameters And then on top of that we also we will improve all the other decisions Surrounding our ability to predict demand, which is a fundamental problem in logistics But of course demand forecasting is I don't think it's the only one many many other applications. I'm not sure how much time we have left but We can talk of this for a little while Several I think there are several areas where machine learning can play a role. Oh, absolutely Looks like we have another question. I want to make sure we get to all the viewer questions we can Chris asks are there new dynamics in demand that make machine learning techniques more suitable than traditional techniques Yes and no, I I'm not sure if this these are dynamics in demand It is true that maybe because of e-commerce Maybe customer expectations are becoming more diverse and therefore becomes harder to predict I think that is might might be part of of of the argument But I think the the the other reason is that we just now have Better tools. So we have more data. We have classic and new methods And we have more computing resources to leverage all those data sets to make better predictions. So As I basically the example that I gave from my thesis The the methods are still the classic one. We're just improving them with better data So I think that even though there might be some changes in the dynamics of the demand patterns I think the key part of why we're talking about now about machine learning and all these algorithms is because We have finally enough data to train them and to make them Useful for predicting and we have enough computing resources to actually leverage efficiently leverage these tools I think we are getting close to the end. Let me ask you one more sort of philosophical question Because I know that's your favorite topic thinking about How do experience managers interact with machine learning now? Is it um sort of replacing our decisions or is it informing them? So could you sort of describe the human computer interaction on logistical decisions today and then predict for our viewers How it will be different 50 years from now as the technology advances, right? So, um, I think the first thing that I'll like to start by saying is that I don't think there's going to be such a thing as general artificial intelligence which Robots will take over The things we do. However, there's going to be definitely Changes in the roles the different roles people now have and will have in the future and I I think I I believe that What managers are doing now and will continue to do is just use data to better inform the decisions but in machine learning and all these new technologies to better inform the decision not necessarily replacing their experience their intuition and their judgment is just um bringing some more information And different angles at a given problem so that the decisions is more are more robust. So I think Machine learning analytics are meant to complement not replace Human judgment However, there will be some roles and this is something that's already happening in warehousing a couple of years ago Robots in warehouses. We're just basically will train them or will program them to do specific tasks Now we see more and more but but because of the combination of robotics and AI We see more and more robots taking over some of the tasks that humans usually have been doing But those tasks are usually the ones that are more physically taxing or the ones that are more tedious and as and then Humans are basically taking over the supervision of those robots the Mantons of those robots or the control of those robots. So I think that we will see more of a Coexistence of humans and robots in the workplace The the role of humans will definitely change and that's something well Both companies but also the educational institution the education institution need to think about well If based on these trends, what are the new skills that people in logistics should have so that they are Still part of the of the economy And but I don't think there's going to be such a thing as fully replacing the workforce It's just there would be some tasks That are more suitable for robots and others even for instance things that require Precision or things that require some level of improvisation humans will still be Much better at at least for the foreseeable future So it's it's more about finding what what what activities are better suited for robots and for automation And what others are better suited for for humans, but both will coexist At least that's what I I believe I 100% agree in what you said Had me thinking about Your work that you've shown us and thinking about how Having that better information can actually help a human better innovate because if we Understand something at a deeper level than we did before the human might get a new idea and who knows how that happens But it's a human experience Before we go we have one more viewer who has a question that I want to make sure that we get to It looks like this viewer is interested in the work that you've done on classifying cities By demand and wonders how Particularly in the u.s. City of washington dc Could they get information about demand in different areas? Do you have any advice? Well in our case we had access to company data, so that it was Let's say help us Come up with those estimates if you don't have access to corporate information Probably the the best thing to do and we have done that in the past and actually in some of the The studies I discussed earlier. We use population as a proxy of demand. So it's not You know there are some there's some error there in in that prediction But usually population density, especially if you talk in in retailing settings population density is usually a good proxy of the demand and in the united states at least there is a the population information is is usually available and it's It's Good good resolution. It's up to date. So I would recommend Looking for some population density or sensor sources to To cover that gap Great. Well, thank you so much danie. We are so proud of you here and excited for your completion of your degree Thank you so much. Thank you for spending some time with with our learners today. No, my pleasure All right Bye guys. Bye. Bye guys