 If you can still hear me, I guess, we can kick off the discussions. Thank you all for dialing in. First disclaimer, this is the first time I am having a poster session in a virtual conference. So we're going to be a little bit improvising here. It's definitely different than just standing next to a real poster and having people passing by. On my side, I am part of the mathematical modeling and artificial intelligence team of KPMG Switzerland. And I'm based in Zurich. And well, this session is about the way we approach optimization problems. As I was mentioning before, in the past few years, Python has matured a lot, both from an probabilistic programming perspective, but also for what concerns the integration and the interfaces to solvers such as, for example, grubby, but also open source solvers. Both grubby py and pyomoc would do a good job. I think that from an application perspective, our methodological approach is coming definitely something viable. There's a lot less glue code that needs to be put together to have a model up and running. So I think this is probably a reasonable way to approach the Caesar science problems in a context of a high uncertainty. Do you have any question or shall I just skim through the content of this poster? All right, I saw a sign of feedback, but are you able to also speak? Or have you been cut off? Yes, can you hear me now? I can hear you, yes. Yeah, because I have to activate the microphone. I didn't want to make confusion with voice, so I silenced it. It's okay, that's helpful. I still need to get some feedback from time to time if I'm still there, if I'm going in the right direction, so please don't be shy. Okay, thank you. Yeah, so I think the feedback was if it's okay if we go through the content of this session and then perhaps if you have some questions. Anytime 53 to interrupt me. All right, so the idea is that we often work in on optimization problems and optimization problems can be, for example, well find the optimal price for a portfolio of products. In this particular use case, we have helped a company, an industrial distributor, a company that purchases industrial goods in Asia and resells them in Europe, find the optimal price. And the definition of the opening price, there was a price that was maximizing the revenues after the application of transportation and storage costs. And interestingly enough, we started from the bottom, and we have built what is called a company value drivers. We tried from a modeling perspective to break down their target metric, or North Star metric, I was also called in some other context, into its components. The optimal price is the result of the calculation of the revenues and the calculation of the optimization cost of the transportation costs. The transportation costs are based on a shipping strategy. The shipping strategy itself depends on some optimization parameters that include volume, how much you would like to transport when there are valid times with transportation and storage costs. In this setup, the client had to approach different manufacturers that had different factories, then goods had to be shipped close to the cost, and from the cost had to be shipped to by both to Europe, to distribution centers and from distribution centers to the client. And the question was, well, how do we deal with uncertainty? And one way to deal with uncertainty is that when you're running an optimization problem, you have to factor in information from distribution of the optimization parameters. And one way to define the estimate of the distribution is by having a generative model. The generative model takes in some input. Input can be internal data, such as, for example, historical sales data or information about the marketing promotional strategy, but also external data. Micro-economic, for example, the price of competitors or micro-economic. You can have some macro-economic indicators, for example, unemployment. You can have some information about commodity prices. And there, here, what you build is a generative model that speeds out the distribution. In this case, over volume, you might have a second model that focuses on the transportation and the storage costs. From a business perspective, this helps because stakeholders that come from the business can have a look at the structure of this model and somehow understand how things have been calculated. They understand the revenues. They understand what drivers led to the revenues. They understand the transportation costs, what drivers led to transportation costs. And this is one of the first advantage. The second advantage is that you're not bound to distributions that are easy to handle analytically. You can have the classical Gushan, but what you can have is you can have a partial distribution. And then you're able also to have, for example, a catastrophe and rare events model in build in your model. We said that the optimization parameters are produced with probabilistic programming. I don't know if you are familiar with the concept of probabilistic programming. But the main takeaway is that what you want to have is a generative model that takes in priors in a vision sense about the drivers. You can use some data and combine them to produce a distribution of the optimization parameters. I guess one of the interesting thing from a Python perspective is that the frameworks have matured a lot. You can have a PMC. You can have Pyro. You can have eWords. They come with pros and cons, but depending on your needs and modeling preferences, it's really a lot easier to set up a probabilistic programming. A probabilistic program for the key optimization parameters. I guess also that the topic is really interesting at the moment because I feel that variational inference made a big step forward. From a computational perspective, it's an existing algorithms at the moment are a lot more efficient than what we had five, six years ago. Debugging is a lot easier. Modern frameworks come with all the tools you need to follow the traces in the probabilistic program so that you are able to see what happens under the hood and debug things. And once you have the optimization parameters, what you can do is you can take your results. For example, you can get an approximation and an estimation of the key optimization parameters. And you can plug the data into a... In this case, what we had was a mixed integer linear program. And from there, you can calculate an optimal solution. For example, how we should transport goods from point A to point B. And you can have either a robust or a stochastic programming setup. I think that from an implementation perspective, stochastic programming with this methodological approach comes out right out of the box. It's also if depending on the number of variables and the complexity of your optimization problem, you might also want to recode your problem as a robust optimization problem by defining the deterministic equivalent of your problem. And one of the frequently asked questions there is, well, do you really need all these probabilistic programming to write your deterministic equivalent? My answer generally is yes, you need to get there. You need to get the feeling for, for example, the first moments of an optimization parameter so that then you can encode it as a deterministic equivalent using, for example, an elliptic formulation. And then when it comes to the optimal solution, I guess the things probably you realize that rather quickly are relatively straightforward because what you have is the transportation cost, given the volume estimated at a given price, you have the price, you have the volume, you have a transportation cost and there's not really a lot more than that in this current use case. I guess the exciting part is really the combination of probabilistic programming and the robust or stochastic programming. Any questions so far? No for me, Mattia. Good. Did I scheme to the content too fast? No, it's okay, it's okay. Are you working on these topics yourself, in your daily business or daily life? I personally work in a quantitative financial world. So not exactly that, but I'm interested in also this topic that it is more related to industrial world, I guess. Well, I started off my career as a quote in an investment bank and I transitioned out of finance and went to consulting. Fine. Are you Italian? Yeah, you heard it from the accent, I guess. Yes, yes. Although I am currently based in Zurich, probably I saw that. Yes, yes. We have a good team here. We also have a few Italians in the team. If Italians at the moment are everywhere, we are spreading out really fast. Yes, it's true. On your side, when it comes to your daily life, I guess, you are solving a lot of optimization problems in the Markovitz sense, right? Not so much because in actual financial quantitative strategies, optimization doesn't work as expected or as the books said, say or write, because a financial world is characterized by a lot of unexpected events in a fat-tailed magnitude. So optimization is not so cheap, optimization from an economic point of view, because when you optimize, for example, an active strategy in the markets, if inputs are not what you put in the model, the results can change drastically. So we work personally in a more robust sense than I optimize the sense. Right? Well, yeah, I think it's very interesting the conversation. Yeah, well, I think one thing that is interesting is that you can get the optimization parameters by Monte Carlo. Yes. By inference. And in both setups, it's fine to get to use fat-tailed distributions. Yes, yes. There are, there was a couple of use cases where we have, in the generative model, we have estimated an optimization parameter by having also a couple in the middle. Okay. So what you have, you have your priors here, and then you have a couple of models here, and then we were getting what we were interested in here, that then we were in turn feeding into the optimization. Yes. I think the methodological approach is really quite flexible. My question is, when you were talking about robust, what do you mean? Because when I, when I, when you would say robust optimization, I really think about robust optimization, but maybe in your domain. Yes, yes. Because in my domain, robust means essentially that the capital allocation, so the capital that is invested in the current day in the markets is a function of the current volatility of the markets. So robust, it means more, more a sort of risk management, risk management dependency or risk management function in the sense that you have to, you have to follow the estimated risk of markets, the estimated potential volatility of markets. So the model has to change in the, in this sense. So robust means a risk management optimized tool, not an outcome optimized tool, but a risk, a risk metric optimized tool. Okay, I understand, I understand. Yeah, it's very interesting. It's, and then why I guess what, yeah, and then it's a capital allocation on different assets. Yes. For this reason, the first I said to you the fact that the Markowitz framework is not used anymore in in applied finance because it focuses more on expected returns, but the estimation of expected returns is the confidence interval are very big compared to the risk metrics. So we focus more on the risk measures risk metrics than the expected returns. Okay, it's very, very interesting. The only thing I didn't remember what it's called about the last thing I did in between the field that was implementing some entropy portfolio location. It was based on the work of Meucci, but that pretty much as far as the things. I think it was even 10 years ago. Yeah, wow, time flies. Is there somebody else in the Boston? Yeah, I have a question. Please. Yes, about your destruction of the model, was it like a, did you design it in a hierarchical manner. It wasn't really going to your combination of the demand estimation and the macroeconomic model or just if you could describe the structure a bit more. Yes. In this particular setup, what we have is really a modular architecture where we have two models. One is the demand estimation where we want to estimate demand for a product at a given price point. So we want to draw the price quantity curve. And that particular model in this case was an ensemble of trees. Alongside that, we had a model that was looking only at the macroeconomic indicators, and it was trying to estimate the transportation and storage costs. And that model was another ensemble. And the output there was on one hand we had as an optimization parameter we had the volume that was the output of the blue box. And then we had the macroeconomic model that was producing transportation and storage costs. We have the tree that in in the optimization, which is the pink purple box here. We also know that the optimization parameter in the mixed integer linear program were actually estimated distributions of the volume of lead times transportation and storage. And I think that all these are helps when it comes to understanding the structure of the model and breaking out the things from a business perspective is easier, because there were also different department different subject matter experts involved. Yeah. Actually, if I think your question is quite interesting and as a, as a team, we will travel around and we will work with different companies. We see that in across different industries from from retail to chemicals to pharma. We see that many companies have started applying time series analysis to them to forecast the things transitioning from pencil and pen or Excel to to have some some models. The companies are a little bit more sophisticated. They were trying to neural networks but the less the messages companies are now having a lot of models built by different functions. You could have for example, functions from the commercial department that work on revenue forecasting, then you have manufacturing but focusing on cost of manufacturing. You have like tons of models that are spitting out on a daily basis, tons of forecast with different purposes and goals. So the way we see probabilistic programming is also a way to combine existing models into one big model that builds on a value driver street. This is what articulates from a north star metric. In this case, we were trying to optimize the price as revenue is less cost but most of the companies are primarily interested in, for example, earnings, and then earnings can be broken down as as part PNL items. PNL items can be further broken down. And in there, in that value driver street with probabilistic programming, you can combine existing generative or discriminative models. Okay. Makes sense. Yeah, it makes sense. And yeah, I'm a bit new to this bachelor student in data science. So when you said the ensemble model or ensemble structure. What exactly does that entail? Yeah, so what what we had he was whether it was either random forest or some causing or random forest, for example, gradient boosted trees. Okay. The idea is that you can have a one regression tree, but then you can have plenty of them and each tree contributes his own forecast and then the model combines all these are over forecast produced by the regression trees into one final forecast. Okay. All right, yeah. Okay, that makes sense to me. Thanks. Yeah. I mean, the random forest or tree ensembles are really, let's say the Swiss army knife of the data scientist in the industry. Yeah. Okay. Thank you. Thanks. Thanks for asking, asking questions. Anybody else. Yes, I have another question. Sorry. No, that's, that's fine. And it's good that you feel the awkward silence. I just said that it's the first time I do this poster session in a visual conference so I don't really know how this goes. When you're like having a proper poster session in the physical you have your thing where people come to grab coffee and it's easy to chit chat. I just, you know, I don't want to have been, I have impressions sometimes. Yeah, I got disconnected or something. Yeah, sorry. No, it's fine. Yeah, I'm also a bit like new to this online video chat. I normally don't do a haven't done too much video conferencing. I guess we have to get used to it. But yeah, I have a question about the drivers, because I'm, I'm also currently working on an internship and I'm also applying Bayesian modeling as well, Bayesian machine learning. Yep. But it's more like in marketing and sales context. So I was curious as to how do you identify your key drivers. It was mostly through like domain knowledge or if you used like a, you say a Cartesian, like Cartesian grid search or how you determine the most important or most salient drivers. Yeah, it's, it's a very it's a mix of both. Let's say, let's say that over over them over the years as a team, we have put together in a repository, a time series data store. Where we keep track. I'm talking now about external drivers. So for example, price intelligence, micro economic data from for example, sector views or micro economic indices, which you can pull from for example, the Bloomberg or Reuters, or capital like you so we have a number of data, data sources, we consolidate the time series in a time series data store. We have a lot of them. And with that, but we have a good of starting point. And then, when, when, when we approach a new client, we have we organize a workshop and we get the different, different subject matter experts from different areas of the company. So what do you think drives your business. And then if we don't have that particular driver, we try either to find it or to proxy. And then what what you end up having is, is that it's a data store that contains tons of external drivers, then you have to connect that to their, if you to their earpiece system. And what you do you pull out the things like this historical sales data. At least most of our clients are with SAP. And then we have our data extractor and the thing goes relatively smooth. And then probably you're going to have also other, other silos in a company where you can find information about for example market. And then once you have all these, the question is, from a modeling perspective, how do you read out the signals that don't matter, or from a modeling perspective, there is also the question of, if you have signals and signals don't matter or signals go integrate or correlated does this work. So you have to understand a little bit the model you're using and the hands of these situations. Okay. I think what maybe if if you're looking at the side of the topic, one other thing that I think it's very interesting from my perspective is the featureization of marketing and promotional data. So if I just to spend a couple of words of these without going too much in detail, I wonder what I see is that most of the companies, sort of like know when they send, for example, or start promotional campaigns online. But they don't really know from a quantitative perspective how they're doing it. So what what what I've been doing for in a number of projects is helping clients, for example, go through their email campaigns or tweeters or LinkedIn and a lot and extract features out of that. One feature, for example, could be how many characters, then they are like a matrix with describe the complexity of the language. You can see online that they, how they are analyzing and comparing the performance or one of them, the complexity of the language used by different US presidents from Trump to JFK. And then what you can you're going to build also then features around the promotional campaign. Yeah, I've seen that also, like, there's some initiative in the company I'm working to do that. But the problem is there's like not enough data scientists. Yeah, they should hire a consultant. Yeah, but yeah, it's, it's pretty exciting but I've also noticed the research itself, it's still a bit. I mean, at least what I've, I've seen is still a bit immature in terms of how it's exactly to quantify the impact of these new marketing outreach, you know, for attribution models and things like that. Yeah, attribution models are, well, the one attribution model are at the moment still an alchemy. I think see many companies that, for example, sort of like a random guess with the attribution strategy that they have in place. First touch, a second touch and touch. Yeah, from a modeling perspective, I think, if you look at the fly machine, definitely a field but it's arrived before for innovation. Yeah. And yeah. Oh, sorry, go ahead. No, I want the, I thought I heard somebody saying something of the background. No, never mind. Sorry. Anyway, any, any other question? Hello, yes. We are actually doing something very similar in our company but from the buyer side, let's say because if I understand correctly, this is for the selling site, like you produce a product and then to put the optimal price on the product eventually, right? Well, in this specific case, we had a man in the middle. It was a company that was buying from the manufacturer and then selling as a distributor to a company that would in turn resell it. Yeah, it was and let's say rather not vertically constituted industry but we have used a very similar modeling approach in, for example, fashion and the luxury where it's a company that manufactures and then sells B2C with their own stores and B2B to stores that go to them in turn resell. Yeah. Because what we noticed is very often there is specific demands actually when, yeah, you have to get suppliers or so for your current production but then only the microeconomic model is the variable input. And I was wondering the next step in the optimization, how you use this because I missed that part of the presentation a bit with the stochastic programming during the optimization step. Is it kind of an iterative process or how do you really take into the model for optimizing these Bayesian outputs of the machine learning from the step before? Yeah, good point. So I guess you're asking what happens in the purple pink box. Yeah, because we use actually the same solver I think and yeah I didn't know that this is possible or how to do that. This is a very good question and I think, I mean on my side I hope I'm very interested about the topic so I'm happy, I would be happy also to connect after this poster session. But there are two different ways to go. So what you have is, as I understand that you have a solver that does for example, this has the capacity to solve mixed integer linear programs. Yes. We work a lot with Gruby. I guess you have with Gruby. If you're not with Gruby, you're with C-Plex or we are with something open source. Then once you're there, so you can start with a standard mixed integer linear program. But now what you have is for your optimization parameters, you don't have your point estimate, but you have a distribution. So one way to do, to solve the problem is to encode the problem with robust optimization where you write a deterministic equivalent of your MIP program taking into account the information that you can extract. For example, the first and second moments from the distribution. So the idea of robust optimization is something that comes from engineering internationally. Imagine you have to build a bridge. It has to be able to load the 10 cars at the same time. How do you optimize? One optimization parameter is like how much weight goes on the bridge. And the robust optimization says, well, you have an uncertainty set. And the uncertainty set is how much I expect these 10 cars to weight plus a safety. The stochastic programming approach is for every, you can, for example, run a Monte Carlo simulation. So you are approximating by Monte Carlo your distribution and then you say, sorry, from an optimization perspective, you want your MIP program to satisfy the constraint for all the points in the data set. And if you like, you can set it up as in a way where constraints can also be satisfied from a soft sense, soft constraints or improbability sense. Yes. So you have a chance of breaking the constraints. But then since it's from a Monte Carlo, hopefully the most important ones you will most of the time not violate because there's so many of these samples. Well, what you want to say, in the robust optimization perspective, what you have is a constraint, which is a hard constraint on the uncertainty set. In stochastic programming, you can use your data points. And then you can rewrite your data points from the simulation. And then you can encode a program where you can use the information on the fully characterized probability distribution so that you can recode your mixed integer program in a way where you can have, for example, you can tune your utility function on one hand. And on the other hand side, you can say you can also by having information of fully characterized probability distribution set a constraint in a way which is satisfied, soft to satisfy. So the constraints holds 80% of the time. Yeah, yeah. That sounds very interesting. Thanks a lot. Thanks. Yeah, it's a, it's a quite, quite a broad, broad subject. Yeah. I'm not sure I covered everything in a very rigorous way. There's some hand waving the way I presented. I just was trying to communicate the, let's say, the big picture while trying not to choke. But, but if you like, this is something that I'm very quite passionate about. So we can touch base after the after the session. Yes, gladly. Thanks a lot. My pal is here. Um, any other, any, any other, any other question. I also questions on the discord. I'm going to look at them later. I make you. Hi. We say here. Yeah, hi, hi, how are you doing. Yeah, fine. How are you. I'm fine. Fine. It's been quite a quite an interesting poster session so thanks for dialing by the way. Yeah, not to be competitive but another track is very quiet while you are buzzing with the questions. Another poster track is quiet. Good. Good. Well, no, I do really appreciate the questions because no, it's been a very good, good audience. Yeah, quick question. I was going through the references were provided at the right bottom corner. Yeah. Found somewhere that your company helps in organizing the sports events. How is this possible? Can you, can you add some color to your, to your question like how would the, how would the problem setup look like? Not sure. There was a reference PDF file to study. Use case, scheduling of sporting events. Yep. KPMG has strong credentials in the scheduling of sporting events in Switzerland and internationally. Yes, this is, yeah, we have been doing sports schedules for quite a while. My, my question was, was about what, we have a few minutes I have to stay also longer in the channel if this is possible, but what would you like, would you like me to give you like an idea of how we structure model, or are you interested in some specific No, this is the first time I heard that machine learning can be used for scheduling the events. Just an overview. I'm not a programmer. I'm from electronics perspective. Okay, yeah, I understand the question. So generally, most of the schedules that we have tackled. The scheduling problem was an operation research problem. So what we're talking about the mixed integer linear program. We are also by the way, we also presented at the scientific conference about how we do schedule optimization with the mixed integer linear program and five or six months ago. Perhaps if you like after after this session I can, I can send you the material. Yeah, please. That is the question. And where does a machine learning come into play. I think there in the sports industry we have a very similar setup, where we use a generative models to estimate the probability distributions of a key optimization parameters. For example, one of the classic use cases that we had with sports organization is they would like to understand demand better web demand is a number of viewers for an event when the competition is streaming and the number of attendance that will be able to join physically. So what what we do is we take a historical data about the competition about the event sales data for example streaming data with the partners, such as for example, I don't know the YouTube's equivalent and what what we do there is we build a model that says hey, this is what we expect them on to be. And then based on that and based on other constraints such as the availability of a location, the cost of rent a location because often that also is is varying. Yeah, we have a mixed integer program. Mixed integer linear program that then produces the answer. It really makes sense. Very happy means this is the first time where I heard how to use machine learning before organizing something. Yeah, it's a very, yeah, no, I understand it. So it's something relatively new. The classical approach is a deterministic mixed integer linear program. And one, one other element which is interesting is that, well, most of the time you end up having a not just one solution, but you end up having several solutions several optimal solution on the frontier. And this is because when when it comes to scheduling one of the biggest challenges is that you generate is a multi objective organization. You have KPIs like for example, travel, but you also have KPIs related to the fairness. And the getting into fairness is a very tricky problem. Fairness can be quantified in different ways. For example, if you have an event where the competition sees home and away games, what you want to have is a schedule where as much as possible you have a partner home away, home away, home away. But now with the sports federations are looking at also other factors, such as for example, the estimation of the impact of travel on the players on the athletes. And this is again a machine learning topic where you start from certain information about the traveling and you try also to the sports team now are really getting into the quantified self perspective. So they are using devices to track physically physiological data about the players. And based on that, and information about the traveling schedules, as federations asking for example, what is the impact of, for example, the jet lag. All right. So this is where we stand with respect to sport foundations. If you ping me on even on LinkedIn, if possible, I'd be happy to share some material with you. I will definitely do I took you off the track from your posters. You gave a very pretty good insight. Very happy. Thanks. Thanks. I definitely would like to learn about the mathematics. It's most often we use machine learning with algorithms which are pre written completely defined and all our mathematics will be helpful. I'll get to learn from you. Sounds good. Happy to stay in touch. I like these events because we are also an opportunity to network. Yes. Cool stuff. Cool. I think I'm, I'm out of time but nobody is is pinging me to stop. So if I, if you guys, if you guys or girls or ladies or have other questions. Just off the record and on a lighter note. This is the best session what I have seen from the morning. You can interact, right? Others were little passive. Yeah, look, it's, yeah, there's, yeah, there's, there's a interaction element which is, which is interesting. It's all other top tracks are one sided means questions will be collected in the charts and answered at the end. We have a regular talks. We have a regular talks as well, but this, this time I wanted to try out something new, something new, which is being there and presenting on a poster for Thanks. See you. Yeah. Hello. Yeah, hi. Hi. So where can we find any material about your, I would say your presentation, like, even like concrete, what concrete examples. Do you have like, like, I would say like project to share about the topic, for example. Um, I have put something super high level here. And if you click, you can see something, something more. But and, but I also have been given some talks and I have presentations and material online. I have quite some stuff on my, on my LinkedIn. And, but I can also share the things with you on the discord channel, perhaps. Okay, yeah, would be great. So I was given a chat about talk about kind of like this very same model setup and use case at the grubby days. And I believe that they have the grubby as put something online. Yeah, this is my, my full talk about these are busy use case, but I'm going to share the link with the video as well. Okay, thanks a lot. Thank you. Thank you. And yeah, look guys connect with me happy to stay in touch. Share ideas. It doesn't really have to be on the bar stuff though. So if you have just just run on questions. Okay, I think we're all going to add the only Indian Adobe. Perfect. Perfect. Thanks. Thank you. Thanks for the presentation. My, my pleasure, my pleasure. Bye. Actually, I think I'm out of time, but I'm going to, I'm going to be reachable on this card. Yeah, thank you. And enjoy the event. Bye bye.