 Next up is Stefan Scherfke. He works at the Office Institute in Oldenburg and is a researcher there. And he works on simulating the smart grid. He's also one of the authors of SimPy, which he is going to talk about now. So please give a warm round of applause for Stefan. Hello, and thanks for being here. So planning big events like EuroPython is no easy task and requires lots of planning. And the organizer team, this he has done an amazing job so far. And to test your assumptions that you have to make beforehand, to test them you can build a model of the conference of all the attendees and speakers and then run a simulation on that and see how your plans work out. And if you want to do that in Python you can use SimPy. This is a discrete event simulation library. And SimPy can of course not only be used for planning conferences, but also for all other kinds of planning problems. For example, production planning or hospital process planning. It has been used to simulate multiprocessor real-time systems. I've been using it to simulate electric vehicles and their charging strategies. And last year I've heard of a guy who used it to simulate the spreading of smartphone viruses via Bluetooth. So you can do really interesting things with SimPy. SimPy is by now relatively old. It was created in 2012 by Klaus Müller and Tony Vigno. And it was designed as a process-based discrete event simulation framework. That means that you model the actors that do something or they change their state over time as processes. And these processes schedule events at a discrete point in time to indicate a state change or an action that they want to perform. And in 2008, Anjaliensdorf, a friend and colleague of mine, and I started to contribute to SimPy. And in 2011, we became the project maintainers because Tony and Klaus wanted to retreat from the project. And the currently active version is SimPy3, which we released in 2013 together with the shiny new logo. And it's only been in the recent years that I realized that SimPy has lots of similarities between other asynchronous frameworks like as I think I owe from Guido or Twisted or Tornado. What we call an environment in SimPy is usually called an event loop in the other frameworks. Our processes are very similar to tasks or coroutines. And events in SimPy are really like a future or promise. And finally, we have resources in SimPy and the resources very much like a semaphore. But in contrast to the other frameworks, we have multiple types of environments, events, and resources. And since SimPy is so similar to other asynchronous frameworks on here, so why not put asynchronous networking on top of it? So we created a SimPy IO with that you can use network communication in your simulation and you can simulate network communication, which is not possible with the other frameworks, but more on that later. The core idea of SimPy is to use generator functions to model processes. Who of you does not know how Python generators work? Nobody, nice. So I'll just skip over it. One note, usually people say working on other frameworks that the concept of using generators for asynchronous coroutines is a new concept, but SimPy used it from the beginning so it's not really that new but like 12 years old. So here's a simple example how a SimPy simulation can look like. You start by importing SimPy after doing pip install SimPy, of course. Then we'll define a simple clock process which gets in reference to the environment, the event loop, because in SimPy we use explicit event loops and no global object because global objects are always not that good and may introduce hard to find bugs. Then we have a name and a tick and our clock will repeat printing the current simulation time and then waiting for some time. To start our simulation we create our event loop, our environment, then we start two processes, a slow clock and a fast clock. The fast clock will print the time every half simulation time step and the slow clock will print at every simulation step and then we can run our simulation until we reach simulation time two and there you can see the output. Apart from the normal environment that performs the simulation as fast as possible we also have a real-time environment and that can be synchronized through the wall clock time to realize real-time simulations and that can be useful if you have hardware in the loop or if you have human interaction with your simulation or if you just want to analyze the real-time behavior of an optimization algorithm, for example. And of course you can create more environments if you have specialized requirements. So I said before SimPy has several event types. The most important event type for simulations is the time-out event because it lets time pass. So here's an example for a conference speaker which will hold a talk at a certain point in time and it starts by yielding a time-out to wait until we start off the talk and then it will yield another time-out event after 30 minutes to indicate the end of her talk. So this is really the basic event type in SimPy. And processes in SimPy are events too. That is very useful if you want to model process hierarchies. For example, here we have again a simplified speaker that will just yield a time-out for the end of her talk and then return a hand-out and we now have a moderator process that will spawn speaker processes and wait for the speaker process to finish the talk and after that start another speaker process until the session finishes. So in SimPy 3 you can use processes like normal events, start them, wait for them to finish and then they can also pass a return value that you can print for example. Now not all speakers may just talk exactly 30 minutes. Some speakers may talk slower or faster so there need to be some way for the moderator to interrupt a slow speaker and SimPy also provides a concept for that. Here our speaker talks for randomly between 25 and 35 minutes and can be interrupted and our moderator now starts again a speaker process but then yields its own time-out for 30 minutes and if a speaker may talk longer than 30 minutes like in that example the moderator just interrupts the speaker and stops the current talk. But this example has a problem if our speaker spoke less than 30 minutes the interrupt would fail so we should fix that and we can do that by condition events that are also new in SimPy 3. With a condition event you can say I'd like to wait for that event or that event or for that event and that event and that event so you can concatenate events with logical operators. We have again the same speaker process as in the last example but our moderator now yields a condition event that says okay I'd like to wait until the speaker process has finished or until a time-out of 30 minutes have occurred and then you get results back the results is a dictionary that contains all events that have been triggered and if our speaker process is not in the results it has not finished yet so we interrupt it. And instead of the pipe as logical or you can also use the end character to say I want to wait for two events to occur both. That's the event markers on the timeline. So another way for processes to interact with each other are shared resources. SimPy has three categories of resources the first one is called resources and the main tab of that is also called resource and as I said before it's very much like a semaphore the main resource has two slots there very reducible at the slide currently so our resource here has two slots and we have a queue in red where processes queue up and the first two processes can then just go and acquire the resource and the third process has to wait until the slot becomes available again and then use that slot. There are specialized types of resources one is the priority resource where processes in the queue are sorted by priority and another specialization is the preemptive resource where processes, important processes can kick existing users out of the resource. The third, a second category of resources is called restore. The store is similar to the resource but instead of storing process tokens it can store arbitrary Python objects and it has two queues one for process that want to put something in and one for processes that want to get something out so in that example the process puts an object into our store and another process can then grab that object and get it out of the store. We also have a filter process where processes can define a filter function to only get certain types of objects from the store for example if you store like machines in the store you can say I only want to use machines that are currently functioning at not broken for example. And finally there is a container a container stores a discreet or continuous amount of some homogeneous abstract matter so it can be used to store like apples or water for example and in that case a process may put two liters of water into our container another process wants to get three liters out but has to wait until another process puts two more liters in so that it can get the amount the requested amount out of the container. So these are the three types of resources that we have in Zimpi extended with other types of resources if you need them. And now that we know the building blocks of which you can compose a simulation I'd like to show you a more a little bit more complex example of a conference attendee how you can model that with Zimpi restart of course by importing stuff and then doing some configuration in our world we have a continuous flow of talk sessions and each session consists of three talks and each talk is 30 minutes long and after three talks we'll have a break of 15 minutes and our attendees will also receive an environment again they will have a name and they start with an initial knowledge of zero and they are not hungry in the beginning and during the talks they gain more knowledge but also get more hungry and the hungrier they are the less knowledge they gain because you know if you're hungry you just cannot keep your attention up so we start with a while loop that loops over our sessions and within that while loop there's a flow loop for each talk one iteration and we get a random amount of new knowledge and we get more hungry and then we yield a time out for the end of our talk and after the three sessions we just print how much knowledge we've gained and how hungry we are now and then we go to the buffet grab some random amount of food and reduce our hunger and wait again until the break is over to go to the next session and reprint a little status message again starting with simulation is easy we just create an environment start five attendee processes and run the simulation for a few hours and the output of that may look like that you can see that the four attendees gain different amount of knowledge and they get hungry and after the break when they've eaten something they are not hungry anymore and can gain more knowledge as you can see in the second block so that was already relatively interesting but we didn't gain any new insights about organizing our conference so we should maybe model the buffet in more detail to see whether our buffet is big enough or if people have to queue up and starve to death so that's not what we want so let's add a refit to our simulation we have two new configuration variables one is for the duration of eating that is three minutes and our buffet will have one slot so only one person can get food at a time and the others have to queue up and the signature of our attendee process now also has a reference to the buffet the part of for visiting talks is the same as in the last example but now the part for getting to the buffet gets more interesting we create a request for our buffet here in the context manager with a risk statement and using the risk statement we don't have to release our buffet when we are done so you should always use risk if you don't use risk to request a resource you have to explicitly call resource release when you are done to avoid that we use the risk statement and then we yield a condition event to wait either until we get to the buffet or until 12 minutes of the break have passed because if we wait more than 12 minutes we won't have enough time to eat our food and it makes no sense to wait any longer and when we get resumed we check how long we've waited and then check whether we made it to the buffet if yes we grab some food again and yield an event for the duration of our eating process print a nice status message again and we are done and if we had bad luck and didn't get to the buffet we get a penalty of one hunger because we could just stare at the food but don't eat anything and yeah it's so sad so and then we print a status message and wait until the break is over and when we start our simulation it looks like in the last example but we also create a buffet resource here with the capacity of one and the output now looks again similar like in the last example but you can see that attendee number four didn't make it to the buffet couldn't reduce its hunger and thus gains the least knowledge in the coming session compared to all other attendees so it yeah and it's really hungry after the conference so a very unhappy attendee and we know that we should add more slots to our buffet in order to make every attendee happy and I think the organizer Kim Sissy has made a great job with the buffet food delicious and enough slots so yeah okay that's the example now I'd like to get back to Simpa IO event-driven networking library on top of SimPy and reset you can for example model multi-agent systems and these multi-agent system can be distributed over multiple systems or processors and use network communication CCP sockets to communicate with each other but you can also just simulate the communication between your agents and for example look how they behave if there's a big latency between messages drop so this is very very cool for evaluating your distributed algorithms and SimPy also implements web sockets so you can add a web visualization for your simulation to make some nice graphics in the browser for example and SimPy is still in a very early stage it's already workable but doesn't have a lot of documentation yet and the APA may still change but roughly using SimPy looks like that you just yield instead of a time out a send event or a message receive event in order to say SimPy okay I'd like to rate until I got a new message or I got my message is sent so it's not that difficult and I've already used it in a production system and it's really cool and relatively fast yeah our plans for the future of course keeping the community happy helping everyone who has questions implementing all the features that we want if they make sense of course we also have some minor optimizations in the pipeline for some speed improvements we are continuously working on our documentation and currently re-discussing adding some helper functions for helping you monitoring your simulation especially if you want to monitor the usage of resources but there's nothing decided yet so some final notes I hope that I could show you that SimPy is really easy to use and flexible SimPy has lots of documentation I'm really proud of our documentation because we have a tutorial, we have lots of examples, we have topical guides and an API reference we have also a mailing list and of course there's Stack Overflow where you can also ask questions and we have quite an extensive test suite that makes sure everything works as expected and that can also be used as a reference of how things should work yeah, SimPy is written in pure Python and it supports Python 2.7 and 3.2 and above and it also has no external dependency so installing it is really very very easy and fast since it's pure Python we support all kinds of other interpreters like Jison or PyPy for example PyPy can give you huge performance improvements if you have a longer running simulation and there's also a port of SimPy 2.0 so you can get an additional performance improvements by compiling SimPy 2.0 to C and since a few months there's also a project called SimSharp which has ported SimPy 2.0 C.Sharp which is also kind of cool if people focus on another language so yeah, and there's SimPy.0 that you can use for adding software communication to your simulation so that's all, go to Bitbucket or simpy.readthedocs.org to grab it and yeah, thanks for being here so if you have any questions you may also raise your arm in the field of computational chemistry researchers use packages like Tinker, Games, Gromax and others to do molecular dynamic simulations or Monte Carlo simulations of chemical or biochemical processes do you know about the applications or uses of SimPy for computational chemistry or if there are people that use this package for computational chemistry no, not directly a colleague of mine uses Monte Carlo simulation and he doesn't use SimPy for that because if you have just single processes that don't really interact with other processes or don't use resources that need SimPy so in that case you may be better off with just using a Y-Lubria process update at state after each time step or delta of time I think you may use you could use SimPy for that but maybe it's just too much and you can just use plain Python standard library to do that and if you want to use a continuous simulation with differential equations SimPy also really offer anything to help your reset so the main focus of SimPy really is discrete event simulation or discrete time simulation where processes interact somehow with each other or use shared resources that's where SimPy really helps you Hi and first thanks for the talk and you're working on this package I like it I have a question about interdependent events what's the best way to reschedule them if needed for instance a server is sending a file over a limited bandwidth and then it starts sending another file and of course this delays the arrival times of other files so what's the best way to do it so you want to know if I already scheduled an event and read for that event and now I'd like to wait for another event yes and something happens that probably cancels that old event or just delays it or whatever there are several ways to do it one way to do this is interrupting a process that waits for an event and then tell them okay please wait for another event or that is the process that waits for the event itself cannot do that by itself so it has to really wait for the interrupt and if the process itself wants to become active it should do something like yielding a condition event where it says okay I'd like to wait for that event and if it takes too long then I'd like to for something else and for that you can use a condition event so either condition events or interrupts by another process or maybe just refactor your simulation and just implement another process called a network cable which just handles all these delays by itself because it has reference to all the events and can just say okay cancel event and just create a new event further in the future you could do that by implementing a new type of event but once the process yields until the event occurs and then it gets resumed the only exception to that is if it gets interrupted by the event loop but if you create a special event type where the condition under which the event triggers can be changed depending on whether one thing happens or another thing happens then you can also achieve your goal so it should be possible okay thanks I got actually two questions the first one do you have some tools to modulate statistical distributions like points on something or am I supposed to use something like SIPU and the second one do you have some built-in visualization tools or should I use my... could you please speak a little bit louder louder okay so okay once more do you have some built-in tools for modeling these great distributions statistical distributions or am I supposed to use SIPU because SIPU is like real big package and if you try to implement it on the platform where you can't install that it's like quite a huge thing and it has a lot of binary dependencies and stuff it could be quite useful if I had some basic basic bunch of statistical distributions to model timeouts do you understand what I mean it doesn't have built-in any distributions but you can use the random distributions provided by the standard library or by SIPI so SIPI is really just a small package and tries to focus on the modeling of processes and event loop and everything that goes beyond that it's left to libraries that are better in that than we are so why should we rebuild distribution, random distributions if they are already very well done in SIPI or NumPy or Python standard library okay I understand, so the answer is no okay and I think you got the same policy for visualization or something you've got the same policy for visualization, I mean it would be quite cool to see how your system develops with time and such stuff SIPI used to have some kind of visualization built in but it really looked not that good as MatCloud Lib or D3.js in the browser so we just said we exclude that from SIPI and don't maintain it anymore and collecting data from a simulation is relatively easy, you just pass for example lists or NumPy arrays into your processes and then append data and after your simulation is done you just pass these lists or arrays to your visualization like MatCloud Lib and then let it do the visualization so that's the better approach again, thank you any more questions no, then thank you again Stefan?