 Hello everyone, thanks, and a few words about me. So my name is Florian Wilhelm and yeah, I'm a data scientist at Inovix Inovix is an IT project house in Germany and Yeah, its focus is on digital transformation So we have projects in the fields of big data engineering and data science But also web mobile and so on in my spare time I like to contribute to the pie data stack and I also am the creator of pie scaffold and I really found it cool that this that pie scaffold was actually mentioned a few times in Some of the talks so short hand up have who of you have has used pie scaffold just to Okay, only a few. Okay, but to the actual talk so the outline is first give a small motivation and about decorative thinking about the concept behind this then I Show some examples in Python how you can actually apply this way of thinking and in the end I conclude with a little math riddle. So I want likes riddles. Hopefully so Let's assume you just move to another city and after all the moving business you want to to Make a little housewarming party and invite your friends so let's say you have two friends two best friends Alice and Bob and you Know that Alice she's really good with tech and so on so you just send her an email with your new address Saying okay, please come over. I want to make a little housewarming party and be there at that time at that date And you know she will be alright She will use Google Maps or whatever to figure out how to get to your to your new address and To be on time But let's say with good old Bob. It's a bit. It's a different kind of story So he's let's say barely able to to write emails. So you send him not only an email with your With a new address and place and date you also make the whole trip planning for him Let's say you say okay it would be cool a best if you take that train from your city to to to to my new city and Then leave the main station take this tram and the dead bus and finally you will arrive on time So with this example we directly see so we Interacted completely differently with Alice and Bob so with Alice we more or less stated what we want of her We declared what we want of her and with Bob We not only told him that he should be at the housewarming party We also Defined how he should actually accomplish the task, but it's not always that easy to tell the how From the what and the how clearly is more an imperative way and the what is more in declarative way so To see this Let's take a look again. What our actual task was in this case in this example So we were we our task was to do a to plan like a housewarming dinner So we have different tasks like to clean up the mess find an easy recipe to cook for your friends Put the last remaining boxes in the basement by maybe more beer and one of this only one part of this was to invite Your friends so this is the kind of abstraction layer We are dealing with if we want to to invite to to make a housewarming party And we see that if we have given this abstraction layer so with Alice we stayed actually on that abstraction layer and With Bob, it's a completely different kind of story. We went down to actually planning a trip for him So clearly be left this this kind of layer of making of planning a housewarming party and This is an important aspect actually when talking about declarative thinking that one has to think about the level of abstraction and where our Tasks actually reside and and we know this from many there many other examples like if you want to take a picture for instance then Your kind of abstraction layer is that you you know how to to Handle a camera and you don't care about all the inner workings of how actually the the photo is created and To be a bit more techie I mean we are more like tech related conference, right? So maybe you remember like five or six years ago There was this big hype about how to map reduce and everyone was so like hyped about this. Yeah, it's so cool It's just too easy building blocks map reduce and you can do basically everything with it But then later people realized that wait, so most of the tasks are actually more like query tasks So more like SQL or SQL related tasks So having realized this people realized, okay So but to do something like like like varies to do this in in in map reduce It's a lot of work and it's actually not stating the problem anymore on that on that level So people came up with things like hive and pig and and you name it so this is also an example that the The layer of abstraction is really important for this So again to sum this a little bit up So I hope I can wait the idea behind this imperative and declarative thinking in a way that imperative is more like specializing on the how and You can see this if you over specify things like when we talked to Bob We said okay, you should end at a train and stuff like this which is completely unrelated to the actual task of giving a housewarming party and detailed instructions and declarative is the more the what and this is also related if we think about programming and Also about design patterns. There was one talk about Python and design patterns, which I found really cool And one of them is separation of concerns and the single level of abstraction And those are also also highly connected to declarative thinking and programming Meaning that you want to keep you want to modularize your code You want to make sure that you're old that one part of your code is actually dealing only with one concern and When you're calling when you when you write a function That you stay with inside that function on the same level of abstraction not going up and down All the layers, but this of course as I said depends on the level of abstraction you're dealing with right now so one One note about abstraction. There's the so-called law of leaky abstractions by Paul ski So this is actually more like an observation. So Americans tend to to call everything a law But this is more like an observation So he said that all non-trivial abstractions to some degree are leaky and so from my experience, this is absolutely true, so I mean many of you have surely used SQL and Maybe have also tuned SQL and you might have then realized that if you start with a slow query You can sometimes do a lot of equivalent Transformation of that query and you end up having the same result But the the actual query is some magnitude faster than before and this is clearly leaky abstraction because actually SQL should kind of Cover the the complexity below and So that you can just state what you want to what you want to have But sometimes it's then better to know what's going on Beneath to to optimize something and this is where as even a language as old and as a good as SQL so good for this one specific task is It's leaky. So this is something one should be aware of when talking about Yeah, when speaking about abstractions so having now the idea of Decorative thinking I want to give some small Yeah, some small examples in Python So some quite easy examples so let's say our task is to get a list of squared numbers from one to ten and imperative and Maybe something someone coming from see your Java would maybe do is you would create an empty list and you would just say Just loop over it in a pant the squared numbers of one two ten and This clearly is imperative because we and we are I mean our task is just saying we want some some kind of list and By here we are over specifying. We're giving an order of how we want to do it and A more declarative approach would surely be just to use list comprehension as you surely all know and love list Comprehension because they're so easy to use they're much more readable and actually we are staying more on the level of our task So we're basically Transforming the task of getting a list of squared numbers from one to ten directly to to the syntax and this has many advantages So one could think that this could also be paralyzed. I mean, it's not paralyzed right now and see Python But I mean it could be possibly so someone could say, okay, we changed the code in in the back So under the this abstraction layer and do it in parallel. So this is also an advantage That it could be done besides being more readable Another example would be let's say you want to write some kind of dispatcher So given some argument you want to call different functions on some value and an imperative way of doing this would be just like you check if arc is option a then you call function a With the value and so on and if it's not the case you call default and also here we are giving an order and we are going down to Much lower level actually so what would be better is to use a dictionary here So some way more complex data type and that data type a dictionary is actually more like a mathematical map Being so you map one thing to another and I mean this is what we want to do if we want to write a dispatcher like this So basically with the help of a dictionary you can just say, okay, we map One argument to a certain function and then we can just use the get Method of the dictionary to do this. So this would be way more declarative and Even if we had a lot of options, this would even be faster because in the back It's it's implemented with some kind of hash map algorithm so maybe a little bit more complicated Example let's say you've wrote a paper your paper is called B And you want to see if someone stole from your paper so you want to check your paper against the paper a and The question is how would you do this? So of course the really naive approach would be you take each sentence from your paper B And check all the sentences from paper a and see if they're equivalent So this would be an algorithm if you do it like this with the complexity of o n squared or to the power of two so quite Yeah, quite a huge complexity and if you think a little bit more about the problem is see that actually Good abstraction layer could be set theory so given that Python has also a set data type you could then think of a better way to actually Formulate your problem if you would just want to compare the sentences and just put all the sentences of work a and work B into dictionaries and then finding The the intersection is just as simple as this and this is also a nice example where like if you think a little bit more in Advanced about what kind of problem you're dealing with and if you find the right abstraction layer for it then it's actually much easier to write code and Yeah, it's actually Way more readable and in this case also way faster because also this will use some hashing magic so another thing you Experience quite often in the in the Python ecosystem are configuration files I mean it was also mentioned I think in the in the first lightning talk and in other talks that in Python a lot of libraries like things and also set up by setup tools they use They use Python modules for configuration and this is quite bad I think because a Python module it's kind of hard to check Right what happens in there everything can happen. It's it's a touring complete way to do something as simple as a configuration and For instance in IDE So it was also mentioned in in in the pie charm tutorial of a pie charm talk that an IDE like Like pie charm could not check what kind of requirements you put into set up high if you put it there directly It's much easier to pass something like a requirements txt txt so I think the takeaway here is here is that it's way easier to To use for configuration a way way declarative approach is to use something like yaml some some markup language or tumble for instance That is less powerful So you have to kind of think more about your actual problem how you want to set up the structure of your configuration But you earn a lot with it. It's easier to check it and Yeah, it's it's way better and this was also one of the reasons why I chose I used for my projects PBR which is in pie scaffold because there you can instead of using a setup pie You can use a setup dot CFG to do the actual configuration in a more in a more declarative way Okay, so coming from some like really simple Python example Let's Talk about a little math riddle so You might know the the German newspaper at each side So the time and they have this little math riddles called local I which which is some kind of mixture word between Logic and Lorelei and I think Lorelei is some kind of mayor mate that attracts you and then kills you so this is kind of riddles you start and then you can't You can't stop until you finally Yeah, solve them or yeah or gave up and Yeah, so this is a riddle and It's like crosswords, but with numbers and we see here that That for instance, we have to make sure that it did should some of Of a is horizontal C and that C is a prime number and so on and one of my friends Sent me this and or showed me this and said, okay How would you solve this with the help of a PC over the help of Python and this got me thinking and Then I remembered like yeah back in the days at university someone mentioned Prologue and declarative programming and Yeah, logical programming so I thought okay, maybe I looked this up again and see what I can do and See if I can solve this staying on the same level of Of abstraction so that I basically just want to state what's in the riddle and in the end come up with a solution so How many of you know Prologue? Maybe oh, okay quite a lot quite a lot so data log is actually just a subset of Prologue and It's then it's easier. So it's used to be able to reason more about Data log it's declarative and logical programming language. It's used to query deductive databases or databases where you just specify facts and certain rules and then you can Ask this kind of knowledge database For certain facts if it can't be deduced from whatever you've given before and it actually has quite many use cases. So Microsoft is doing a lot of research in this regarding security Network security that checking some kind of firewall rules for this then it's also used for data integration information extraction Networking program analysis and of course, yeah, you can do it for you also use it for for silly riddles and There's a nice Python library actually called pi data log where you can just use it and run it As a as a Python module and I just want to like show a little bit How one would go about this and how it is possible to to stay like really at the level of this Yeah, of the Conditions in the in the riddle. So we saw that one of those conditions was that We that some numbers are squared numbers So how would you how would you specify this with the help of pi data log? so you would just say that that squared X is So X is squared if you can take the square root of it and if it's still an integer and You read the leftmost Smaller than equals sign just as an if so the operator is overloaded and it's best to just read it as an if and You can see it's just like you're just stating it. So and this is squared is now some kind of predicate for X and It's some kind of of set of some you by this you're defining some set, but this is all you need to know and The same goes for divisible if X is divisible by Y and we can also check only for the remainder It's divisible if you do this and Then it's equal to zero. So maybe it's something more interesting. How would you check if a number is prime? We know that two is a prime number. So the plus operator is overloaded We so we therefore just check if our state that two is a prime number as a fact and For other numbers and three and for the other numbers we use a rule Stating more or less that all numbers larger than three which are odd numbers or not This is the tilde so the curly one not divisible by two and if there's not any other factor between three and The square root of X and this we defined by this by the second last line With some kind of recursion so in the last line We're basically saying so if y plus two is smaller than Some some upper brown so some the square root and then we check all the numbers and in just a few lines We we can state this lazily for all numbers and since most of our Conditions are defined on numbers and we are actually looking for the digits We of course need to have some kind of mapping between digits and the actual numbers and this is really easy So the digits the number of two digits is just ten times the first digits plus on and then plus the second and Then we can just recursively Define this for Yeah, for all Digits for different number of digits here again, so sorry here at that point We would be now able with some more rules and so on to actually just write down one condition after another of the riddle and Here is a part where the leaky abstraction bites us so since what happens below is that what What pie data like does it? Generates a whole list of solution candidates and starts eliminating them and so on and if you start with the wrong kind of Condition first then it might happen that the list of candidates gets too too large at certain points and Then you can run out of a into out of memory error or Into a door just a runtime is just too long. So This is then some kind of leaky because actually it should play no role if we want to stay on this level so one thing to to deal with this is to define certain parts or like this partitions this colored petitions and Yeah, using this petitions in now in case of this riddle helps to keep the number of solutions low at all Time and then you just combine them. So just to give an Example how this looks like for the upper left corner. So for the blue corner This is this large enough to be read. So the first line a little larger so This was see and we wanted it the prime numbers the two prime numbers Sorry, the two digits form a prime numbers. So in Pido beta in pi data log We can just easily state that a two is a digit ranging from one to ten because it's the start of a number So it can't be zero to ten or sorry one to nine and not zero to nine and The same goes for a three and then we we just can use The prime check the prime rule that we have to defined And by this we can for all Yeah for all conditions in our riddle We can just write them down and actually it's just like as you see here like each line in the box above is More or less one of the conditions in in the riddle and it's really staying at this abstraction level It's using no loops or over specification It's just a declarative way of specifying a riddle How would we grab it this now? It's really as easy. It's just saying Just like print and the riddle and the riddle is a combination of all those four boxes Which I have left out here But you can look this up a later show a link and you just say okay now Give me the the numbers the digits for that fulfill all the constraints. I have given you and Then it will just come up with a solution and not only The solution so if there were more than one solution it would would print all solutions So we know now and not only one solution we are also sure that it's a unique solution which also a really nice thing and Yeah, if you follow the the URL below then you can also get the whole source code and I think this so for me this was a really nice experience to to work with a logical programming language and Yeah, maybe some of you get interested in this and want to want to try this pie data log out or maybe prologue directly and Yeah, I want to conclude that there are many other applications we're actually declarative thinking and Programming becomes much more important. So in one of the talks also Nick. So OS was Was mentioned and this is one operation system where you have a declarative way of stating how you want your system to be It's not like app get install and moving from one state another you have a declarative way And this has gained quite some traction over the last year. Also A tool for the framework like tensorflow. I mean they they included just the layer Submodule where you can now easily define like in a curious style of way Your your network because if you want to build a network if you're on that layer You're not interested in really all the details how you do the matrix multiplication so on But it has it now provides like two layers like this layer Submodule where you can just define layers and of course also the low level parts with Where you can do like almost everything that's possible with a linear algebra and others Yeah, and there's for instance also Luigi, which is kind of a competitor to to airflow and This is also using a more declarative approach so to to make a small summary of the talk so What are the advantages of? declarative way of programming and of thinking because actually it's it's more like thinking so in in most cases you can actually improve the readability of your of your code your You're more like stating what you want of course and Keep it then to some other parts of the code to do this it also reduces the number of errors most of the time because If you for instance if you write some query in SQL if you would do this really with the help of Some map reduce or if you would Try to to mimic this with your own code You would surely end up having a lot of error and strange corner cases in many cases. It's also increased performance because I mean as a user you stay at the same level of This at one level of abstraction and what happens below is then can be programmed by some domain expert like yeah, I mean in case of SQL it would be the relational algebra and so on and Yeah, and you're following the separation of concern. So the the big takeaway message here is So it's kind of my definition for me How I think that declarative programming is that declarative programming for me means Finding the right abstraction layer describing the problem and then using this layer So thanks for your attention and yeah, I'm welcome to hear questions if we have enough time Okay, we have about two minutes or so not much so I would take one question. So who has a question That saves us time. Okay, there's one Hi, you said that the datalog is a subset of prologue. Yeah, what is missing? I mean what? What are the future or it's kind of those desert in prologue this breaking operator where you can kind of stop the computation and This is missing. I think this is one of the the most important thing which is missing in in data log But I'm also I'm not an expert in this. Yeah, but uh, yeah Thanks. Thanks All right, that was it. So give him a hand and Remember