 Thank you and welcome everyone today I want to give you a little introduction to integer programming and the idea for this talk was actually born when we as Program committee for the pike on pie data this year in Berlin Had to actually solve the problem How do we make a great agenda for the conference and how we can model this and what is optimal? so Yeah, just a bit about myself. My name is Florian. I'm a mathematician mathematician by profession Also, this is the reason for the talk. I really do like mathematical modeling So translating stuff into math everything around analytics data warehousing on so on I'm a huge fan of the Python data stack, right? And that's Europe Python here and I'm also an open-source contributor to myself So just a few words about inovac so my employer we are an innovation quality-driven IT project house We offer basically everything from the IT infrastructure mostly in the cloud to application development back and front end But also the analytics part so that's the department I work in and if you ever have an interesting project come and talk to us so a Structure this talk in the way that I'm going to start really high level and abstract and gonna go and get we are Getting more and more concrete as we go So the field we are dealing with it actually the field of operations research and some of you might know this The subject from from University for those who don't know the objective of operations research is the development and application of methods to support the optimal decision-making Process and this in various domains Manufacturing finance, but can be many more and Typical problem classes in this fields and this is not an exhaustive list just some examples are Problems like assignment allocation problems like the problem. I just talked about like you have a schedule You have some talks and you want to assign the talks into the schedule to get some kind of agenda and you have given some constraints like room sizes and so on and But also many other problems like a transportation problem Imagine that you have some plans and they need goods from supplier to build something And now you have a certain cost of each supplier delivering goods to your plans And somehow you want to make sure that every plan gets the right amount of goods But at the same time you want to Minimize your cost of transportation other problems include like shortest path problem imagine you have a graph and Source and a sink and you ask the question How do I get from a to be from the source to the sink? Given that all your edges have a certain cost and you want to go the path of minimum costs Another one is maximum flow problems imagine that you have some computer network and you are asking yourself So what's the maximum number of? packages I can send through the network or through a pipe system. So those are typical problem classes and They can all be solved with linear programming or integer programming and so what's the definition of this it's Looks like really simple. So we have some some objective functions some cost function where we want to Minimize over a given variable x. So this is our decision variable and we have some vector of costs That is multiplied with x and we have some additional constraints this case given by some matrix a time x is lesser or equal than some vector B and Our acts are our acts is just the n-dimensional Real vector. So this looks really simple this mathematical formulation but turns out that the typical algorithms you use like the simplex algorithm is cubic in in its complexity on average even exponential number of steps that you need in the worst case and There exists some methods which do it in and to the power of 2.5 Which is better, but still not so good, but at least for typical kinds of problems It's it's gonna be fast enough But now I said that only some of those problems are linear problems Many of the things like shortest path you would need actually an additional constraint And this is the constraint here that some of your Some of your components in your vector they need to be integer and Although we are adding additional constraint and you might think okay This should make the problem somehow easier. It makes the problem really really hard. So the complexity is now NP hard and If you have many components many integer components or even if all your decision variables are Have this integer constraint then The problem grows really dramatically and So what you might think okay, why not just drop this additional constraint and this is what's called linear relaxation This will in practice get you no result like if you think you could just solve it and Then round to the nearest integer Values this will not help Future so additional Special cases are that I just want to mention so we talked about the linear case sometimes is also good to have a quadratic cost functional and in the case of That we're going to talk later about that all our Decision variables are integer and it's no longer called integer mixed integer linear program, but just integer linear program or just integer programming and Quite often it's not only integer. It's just binary So you need to decide if it's zero or one and then it's called zero one linear program and If you wonder why this is called programming. It's looks like math, right? This was a kind of marketing trick that George Danzig who invented this This linear programming so he coined it programming because back in the 1940s So during war times Programming was more used for term for planning and structuring things and getting to a goal so this sounded more like Practical and that was the way at the time to to get money to be concrete and Rather practical than just abstract math So no talk about integer programming and linear programming without the kind of hello world of integer programming, which is the Knapsack problem and I think this Example shows Really nicely how you do the mathematical modeling how you come from them from a concrete Use case in the real world to some mathematical formulation to solve it so Imagine that you have a backpack and you have a set of items and now you these need to decide which kind of items you want to take with you and each item has a weight and some some value and Of course, you can only carry so much weight so you have some kind of constrain what the maximum weight is of all those items and On the other hand, you want to maximize the value of those items that you want to take with you so this kind of the our our objective to Maximize the value is translated into our objective function So this time we maximize and we just multiply our decision variable, which is zero or One with the the value depending on if we decided to take it with us or leave it there and the same goes for our constrain that kind of selects those weights which we decided to put into our backpack So this looks really easy enough in practice It's kind of the high art of translating a problem like this into the mathematical formulation so beforehand I I stated this that Solving those problems is NP hard. So what do we do now in practice? So they're typically or basically two common Heuristics the cutting plane method and the branch and bound methods, which are often combined as branch and cut methods And I just want to give you an idea how those methods work They are heuristics. So you have no guarantee, but in practice they Yeah Help you to get the job done and solve the problems so the the main idea of the cutting plane method is that you do the Relaxation so you just forget about the additional constrain that it needs to be integer You solve this problem for instance using the simplex algorithm or an interior point And maybe you are lucky and the solution is integral mostly it will not be so what you do is now You add an additional constraint that excludes the Non-integer solution but none of the interior points and then you repeat and repeat and repeat and this gets you Eventually to the right solution. So it's actually much easier to see this in a small example. So in in a 2d 2d problem is quite easy to to visualize. So we have our Objective function that we want to want to maximize x1 plus x2. We have constraints which Can be interpreted. It's just as those lines and additionally one wouldn't be positive So all our feasible points. So that's the bright blue dots here all those points fulfill our constraints and The red one would be the solution just by looking at it and the dotted line This is like the easel client. So on this line The objective function has a constant value and now what you can do graphically is that along the normal To this easel client. You just move it in this direction and up to the point where it's still in the In the domain of the feasible points. So If we would now wouldn't know the solution we could do the relaxation so this means if we move this up, so this would be our solution to and 2.5 which is not integral We add an additional constraint if we could for instance say we want to cut the way Everything up here So we add this constraint then we solve again. So this time this would be our solution using the simplex algorithm for instance and Again, we are not integral. So we add one more constraint and you can directly see in this case we have then found Solution which is integral using the simplex by just adding more and more constraints and now the the art of those Horristic is finding the right constraints because if you add wrong constraints in the end you will Yeah increase the problem more and more and you need too many iterations and then You will have Exponential complexity the other method branch and bound I just want to give the idea how it works It's a kind of smart trick. So we solve again the relaxed LP problem Then we decide for one of the components which is not integral and we add two problems out of it So one by saying by adding an additional constraint that x i is smaller equal than the floor so the lower bound of our of our solution Component and the other one larger or equal than the ceiling and this gets us in the end some graph if you do this over and over and You can eliminate some of the branch branches by exploiting this inequality So of course a feasible solution is always smaller than an optimal solution so just by definition if you maximize and The solution of a relaxed problem of a relaxation Will always be larger or equal than the optimal solution that means whenever somewhere in your graph you found a feasible solution and The relaxed solution of one of the sub problems is smaller than you know, okay I don't need to look in those kind of sub branches anymore because I will not find something that is better and And so how can we now apply this to conference scheduling? Imagine you have a number of talks So some some speakers sent in talks you accept it then and then you already define some schedule where you say, okay There's a morning session afternoon session. There are a certain number of rooms and maybe a certain number of days And now our task is to cite which talk should go into which slot So Now the real question is so do we have any constraints and can we state those constraints? There are like really easy constraints for instance each talk must be assigned exactly once right? So no one wants to give a talk two times at the same conference then There's also a constraint that Each time slot room combination can only be occupied by one talk at most Would be kind of strange two speakers talking at the same time And then the length of the time slots like we had we had 30 minutes 45 minutes 90 minutes slots must match the length of the talk if there's certain tutorials that have a part one and two days should be consecutive and Those are like that. They're really the hard constraints now talking about the objective Of course, you always have some speaker that gives you certain kind of preferences that they would like to speak on a certain day and Of course The also the keynote speakers and so on so you will have certain preferences one could put those into Constraints but then it might happen that you have no solution at all So it's good to have this in an objective then there's the popularity of the talk that you want to Be reflected in the room size So also, I think it was the same for your python, but also for pycon PID We had a voting beforehand Who wants to see or what how you like the different talks just by looking at the abstract and the title and This way we already had the information that certain talks are more popular and than others And then of course a more popular talk should go into a more popular room By the same information It also is interesting that you could optimize that if a lot of people want to see two talks Then those two talks should not be at the same time just in different rooms, right? So there should be one after another so that people have the chance to see both talks Same idea is with tracks. So we had different tracks like pie data pie con and also subtracts like pie data data handling and if one session Which are like three slots if they have the same track or maybe even subtract Then the likelihood that people don't need to change the room as often is really high so this is also something you might want to optimize for and Since this is kind of we have now multi objectives right five Parts and somehow we need to bring them in an order. What is more important for us then? the others so there We decided to go for one is more important than two more important than three And to get in the end to some final solution There are also techniques to solve for them at the same time to get more like a whole space of solution This is the goal of a multi objective Optimization and there will be also talk think even after this talk directly about this and So let's say we have stated now those Constrains and the objectives and so on How do we now? Get this into into Python and how do we solve it like really practically so first of all there's there's the separation into frameworks which help you to Formulate a problem and they will in the end give you some standardized format basically the Mathematical formulation that I showed before and this can then be fed into a solver so you can take whatever you want from the first and They are all compatible with the with the with the different kinds of solvers So since I'm a huge fan of open source. I decided for pyomo. I heard that pulp is also a really cool tool Then there's a MPL or ample is maybe the most well-known but commercial one and from the different solvers Hikes is quite new so I Some colleague recommended this one. So I opted to use this one So how does it now all look in? in in Python Just to give you a little idea How you then formulate a problem like this so we we define our model So this was the park on the pie data schedule. We have different kind of sets so to help Express our constraints and our objectives later on so we have the set of all talks Then we have another dimension and another index set for the days So Monday Tuesday Wednesday and so on then we had different sessions two in the morning two in the afternoon and Each set each session is comprised of of up to three slots So first second and third then there were I think it was like five rooms and Then we had a different track names like pycon pie data and the subtract So now in pyomo, you have the the parameters so the parameters are the things you you give into your problem But there are no no variables. So they are depending on your Yeah, and how you formulate a problem like the preference that for some Speakers and those are the code names of the talks and they stand for a speaker Let's say they really like to give the talk on Monday Then we set the parameter two to one for this talk and and and speaker so to make two to To express that there's a certain preference for this slot and zero In case It's it's not one of the Monday slots and so on this example for Monday morning So I stripped the code down. It was way more complex than than this and By this you can then define for instance the preferences for the parameters and those will The preferences as parameters and those will in the end then end up After pyomo does its trick and for and combine everything into this mathematical formulation as Things in the in the cost functional In the in the C we saw before this cost vector So then of course we have the decision variable So this is what we are interested in in the end, right? So we have a variable over all those different Indices that we just said so you already see here, but it is quite high-dimensional and the domain is binary So this is just zero or one so you can imagine so if for a given index set here We have a one We know that that talk will happen at the given day at a given the session at the slot at the given room Then what you always need is a lot of auxiliary variables like To say if a talk happens at a certain room So we see we have only the two indices also for parallel tracks. It's everything except of the room and You know if you would Sum over this Dimension for instance, then you get the sum of talks at the same time so now The really interesting and really one of the hardest part is like defining now constraints so we talked about the one constraint that we only want to have one talk at most for each room time slot combination and There we just We can just sum in our decision variable over all the talks for for a given day Session slot and room and if this sum is smaller equal than one then This is fulfilled. So then we know there was assigned only like one talk at most and you see that already here we are adding a lot of constraints and This is a quite a simple one then here to Give the constraint at a talk length actually fits the the slot length We multiply the decision variable which will be a zero in many cases with the slot length except of the one where It is assigned to the slot then it this Multiplication will lead to the slot length and this should be equal to the talk length. So this looks a lot Maybe much more complicated than maybe you would Formulate this and why is it like this? Why does it need to be formulated like this? Because we have the linearity constraint, right? We have to make really sure that we don't do things like Multiplying variables and so on so this is like the hard part formulating things like this now talking to the talking about the objective in case of the The preferences we can just multiply the preferences with a with a with a schedule and to get the first term the same as with the room capacity then for the corn currencies it's it's it's similar and Also for the other terms and now since we know what kind of the the range of value of each term is we can just express for instance this Fact that the preference is way more important than the other things by multiplying it with a high number So this is one way to kind of push in this fact that we first want to fulfill the preferences and only then The penalty for not hitting the right room capacity should be fulfilled Yeah, and in the end it looks so this is a visualization of one of the results we had Looks kind of unimpressive, but in the end you then have Which talk is assigned to what and this ran for 48 hours, and it was still not completely Solved, but you know how far away you are from the from the right? Answer so we took this solution. So if you wonder about the NIN so some of the slots we just wanted to be empty so those this is just normal and Yeah, if you're interested in how we did it in detail, so we also Release the package called Python is where you also have a pre talks clients Whatever to organize community events and one of the notebooks holds this Formulation so you might now say hey sounds easy enough. I want to go for it and There's one final remark I want to say because for me it was a really brain twist to get into this To to to fulfill the the linearity terms because for instance If you want to say that something should not deviate so much from something else You would normally take something like a root mean squared error But of course this is not linear then you think okay why not take the absolute value? But also this is not linear, but you can linearize it. So how do you linearize it? Let's say you wanted to have the absolute value of x in your objective function You could just say well, I express I add a additional Auxiliary variables t1 and t2 and express it like this so what we have like graphically So this is our x and this is now this t1 plus t2 And Now just flipping it in The objective so putting now t1 plus t2 in the objective will lead to what you actually Wanda wanted to in the first place you introduced additional variables additional constraints and a lot of Designing and modeling something like this is about coming up with With tricks like this and there's the big M method and and many more and many more so Yeah, it's really interesting so the main takeaways. We learned that mix integer linear programming is part of operation research We talked about the optimal decision and the knapsack problem. We learned that those problems are Or have NP hard complexity Are NP hard and you can solve them with branch and cut in a in a heuristic way that there's pa yomo and also Hicks to formulate into turns to solve the problem and Actually hardest part is the linearity with this I want to start check out my my want to end Check out my references. I took a lot of inspiration from other talks and Also lectures and with this I want to say thank you