 Okay. Thanks a lot for coming. I'm very excited to be here today. The stock I would like to talk about something I like a lot in computer science which is algorithms and in particular graph algorithms that are, graphs are great data structures and almost useful for every problem you solve in your everyday code. Okay. So the title says traversal mesas. Mesas means graph. So in the stock we are going to talk about mes traversals related problems. And of course mesas are the everyday data structure for the role-playing gamers here today. I'm quite sure that many of you have played tangents and dragons and stuff like that. Okay. So this stock seems to be, it wants to be an each Icarus guide for getting out from the engines. Okay. The second part of the title is algorithmic adventures. Algorithms are the most important and durable part of computer science. And they may be studied and analyzed in a way which is language-independent, machine-independent. But in this talk we are going to talk about Python, right? So we're trying to analyze the algorithm, the algorithms and the algorithmic parts in the slides from a very pythonic standpoint. This is my goal for this talk. And by the way, this is not supposed to be an algorithmic class, right? So this is, I will provide some very introductory stuff. I'll try to brief recap something that I'm quite sure that you're already aware of. Okay. I'm sorry. Yeah. Actually you're right. Is it okay? Yeah. Sorry for that. This is the first time I used the IPython notebook to make presentations. So it's quite a mess. Okay. So let me briefly recap some basics for algorithm analysis. We need a technique to compare the efficiency of algorithm without worrying about the implementation, well, the machines and the language. So we have two different techniques. We have the ROM model computation. We have the asymptotic analysis of the worst case complexity. In other words, the no-win scenario. So we're interested in the worst case here. This is useless. In the worst case. And we have the ROM model computation. The ROM model of computation simply tells us that each simple operation, such as plus, minus, if, of course, takes exactly one time step. Loops and functions are not considered simple operations. And each memory access takes exactly one time step. Thus we have, we may have as many memory as we want. And the ROM model takes no notice on whether an item is in cache or in disk. So always on disk. So all the details are right behind the, all the stuff. But we don't think about it. Okay. Of course, we have different notations to come through to analyze the complexity of algorithm. We have the worst case complexity. So the big O, also known as the big O notation, the tether notation. So the average case complexity and the best case complexity. The big O notation originally was the big Omicron, actually. And tells the upper bound, the asymptotic upper bound for a function and the complexity of an algorithm. For instance, we say that the time of our algorithm belongs to the O to the N square. This means that the, for large enough value of N, which is the size of the input of our algorithm, the time complexity, the asymptotic complexity will be bounded by the N square function. Okay. This is just a very brief recap for all this stuff. This is just to introduce the some topics that we will talk about later. And just for your information, there are infinite dominance relations. This table shows you the different, the relations among the different, some of the well-known different asymptotic computation from the constant time to the factorial, which is the heaviest, the heaviest one. Okay. So we'll talk about graphs. Graphs are one of the unifying teams in computer science. Their graphs are a great data structure, a powerful mental model, and graphs can represent almost all kinds of structure and systems. You may think about transportation networks, social networks. Nowadays, social network analysis is a great research field and very active research field. Dependency networks, for instance, you might think to the graph representing the dependencies of a piece of code or the dependencies of a software package that should be installed on your machine. Graphs come, comes in different flavors with different explosivity, of course. In a nutshell, the graph terminology, we wrote the graph as G as a top of V and E. V is the set of vertices, E is the set of edges, vertices within an edge between them are adjacent. The edge is then incident in this vertices. We may have a subgraphs, so G prime, which is B prime, E prime is a graph where B prime is a subset of B and E prime is all the edges incidents in the nodes of V prime. We have the notion of path, so we have a path that is a subgraph where edges connect the nodes in a sequence. And finally, we have a cycle, which is like a path, but with a difference that the last edge links the first node in the path to the last node to the first one. Graphs, as I said, come in different flavors. We may have directed or undirected graphs. In the figure, you may see that the edges helps and the notion of directions. There are unweighted or weighted graphs where there is a weight on the edges. There may be simple or non- simple graphs. The non-simple part here means that there are some edges that may require some special handling in our algorithm. For instance, in this case, self loops or loops in general. We will see another example of this in a few minutes. And sometimes there will be edges that may be incident multiple times in the same node. In that case, that will be a multi-graph, not graph, actually. Okay. So we may have cyclic versus acyclic graphs. An instance, for instance, a cyclic graph is a tree. Trees are connected, acyclic and undirected graphs. We may have the well-known and so-called DAG, which stands for directed acyclic graphs. And with DAGs, we may apply topological sorting and stuff like that. These are all concepts from very basic introductory algorithmic class. And finally, we may have a labelled versus labelled graphs. Where there is a label on nodes, so our own vertices on the graph. Graphs may be sparse or maybe dense, which refers to the number of edges in the graphs. And last but not least, graphs may be explicit or implicit. This is what I was referring before when I told you that graphs are a great data structure and a powerful mental model. Because sometimes when you have to deal with a problem, you have to solve such kind of problem, you may think to the problem as a graph problem, so you may represent your data, your problem as a graph, and you may apply graph algorithms to your problem. So here we go to the graph representation. So finally, we start to see some code in action. One of the most common way to represent graphs is the adhesion salists and something like that. One of the adhesion salists are one of the most intuitive ways to implement graphs. And for each node, we can access a list or a set, as we will see in a few minutes, of its neighbors. So this is the idea and this is the graph we are going to use as an example in the next slides. Okay? So we have nodes labeled from A to H. And for simplicity, we assume that we number all the nodes from A to H, assigning a number that goes from 0 to 7, so N minus 1. In Python, we want to implement a adhesion sunset, which is, in this case, we instantiate the number of nodes from A to H. And then we create the structure N, which is a list of adhesion sets. Okay? Before, just a mention, before Python 2.7 or Python 3, you would use the set constructor here. But now you may use the, this expression, and you may use the set constructor to create sets of literals, right? So in the brackets, you may instantiate sets of literals. Okay. The name N has been used since in graph theory, N of V, which where V is a vertex, stands for the neighborhood of node V or vertex V, right? So in this case, we may write LEN of F, which is 3 in this case. F is the node, okay? Which is the degree of the node. And then we may also test for the neighborhood membership, B in NDA with this simple instruction, and it returns true, okay? So this is for neighborhood membership. And we will try to discuss on the different complexity, depending on the different implementation of the structure we're going to use. Okay. So in the first example, we use a set. In this other example, we try to modify it a little bit, and we use an additions list, so we replace the sets with the list. Okay. So actually the code doesn't reflect this. Sorry for that. I didn't realize it before. Sorry. The braces should be substituted, of course, with square brackets, okay? Sorry for that. In this case, again, we have the same expressivity, okay? So we may apply again LEN of N of F, which is 3, again, so the degree of the node, the number of node incidents in that graph, and we may test for membership, for neighborhood membership. So B in NDA returns true. But this time, neighborhood membership takes theta of N of V, which is the vertex. In case of sets, in the average case, the neighborhood membership is constant time, okay? So this is just the first different of these two very simple implementations. This can be problematic in case of dance graphs or bead graphs, okay? We could leverage, for instance, we could use these implementations or sort additions lists. We may sort the additions lists, and we may rely on the sorting, on the data, or sort the data to apply a binary search in order to test for the membership, okay? In that case, we may reduce the complexity to log of the size of the neighborhood. But in the general case, if the graph is big, maybe an implementation based on sets, on the average case, would fit better, okay? This is just the first thing. This is just an hint for more complex problems. Yet another trick. We can substitute the lists or the sets with a dictionary, okay? So this time we implement the adhesion sequences with dictionaries. This is a sort of sparse weighted graph representation. In this example, we also embed in the graph representation the weight, okay? So we increase the explosivity of our graph. We also, with this kind of representation, we are also able to represent the weights of the edges, okay? So in case of weighted graphs, this is a very intuitive way to represent graphs. Again, we are able to test for membership to calculate the degree of the node with exactly the same syntax as before. But now we are also able to get the weight for a particular hedge. For instance, if we are interested in getting the weight for the edges connecting node A to B, the weight is 2, okay? And this is how we handle this. Okay. Very fair enough. A more flexible approach. Actually, Python in its documentation suggests this a good way to implement an implementation pattern for graphs. If you're interested, please take a look at the link on the slide. But basically, the idea is to represent a graph in a very dynamic and flexible way by means of Python dictionary, okay? So we embed the nodes, I mean, of course, considering the adhesion C structure, okay? So you embed the nodes as keys of a dictionary and every value associated in the dictionary is a list of other keys in order to get the corresponding edge in the structure, okay? And this implementation is quite similar to what happens in the well-known network X library, okay? Network X is the reference Python library for graph algorithms. Everything I'm going to talk about here today is already implemented in network X, okay? So my goal here is not to show how all this stuff is made in network X, but it's just to reason, just to get more details on how you could handle all this stuff in Python and what are the difference of every possible decisions you made on graphs, okay? Again, following the implementation pattern, we may implement a dictionary of adhesion C sets, okay? This is a very simple example of how you can implement a dictionary of adhesion C sets, okay? So this is a very simple implementation. We drop the weights information, and we rely on the fact that we're dealing with nodes labeled with single letters, so we create a set from the string, so it's very simple, okay? So this implementation is very, very similar to the former, to the first one, but drops the weights information, and again, it relies on a dictionary with sets for adhesion Cs, okay? And with this structure, we are allowed to test for membership for a neighborhood membership, which is still a constant time, and we may calculate the degree of a node. The other well-known structure to represent graphs is the adhesion C matrix, and in the adhesion C matrix we list all the neighbors for each node, okay? So we want to store a value indicating the presence or the absence of the edge in the graph connecting to nodes, which is through our force, for instance, zero or one, one over zero, respectively, okay? And we may try to implement an adhesion C matrix with nested lists. We may argue that this is not actually a matrix, this is a list of the lists, okay? Just implementation details about Python, just to sort of proof of concept, okay? So with this structure, we are able to implement, we are able to test for membership, as usual, we're able to calculate the degree of the nodes, and of course you may substitute this homemade implementation of the matrix using, for instance, NumPy arrays, okay? So which is, of course, even more efficient, and if we will have time, I would like to talk about the SciPy graph implementation already in the library of SciPy, okay? The properties of the adhesion C matrix, the adhesion C matrix is a great structure, okay? And I thought to talk about the differences of the adhesion C sets, the adhesion C lists in general and the adhesion C matrix, because in algorithmic class, algorithm class in the university, sometimes there should be some misconceptions on the use of the two different structures, okay? So on the one hand, we have the adhesion lists where we are interested in storing only the information related to the existing edges between vertices. In the adhesion C matrix, we represent all the information, okay? So sometimes there should be a misconception that you want to use one structure when you have very small graph or the other one when you have a bigger graph, okay? But we'll talk about this in a few slides. Adhesion C matrix F has some very interesting properties. The diagonal of the matrix is equals to an array of zero, okay? So as long as we don't allow self-loops, so we're dealing with CO dot graph, the diagonal is always zero. In case of undirected graphs, the matrix is symmetric, okay? So if we're dealing with undirected graphs, we may leverage some very efficient matrix representation for triangular matrix. Extending the adhesion C matrices to allow for edge weights is very trivial. Instead of storing truth values, zero, one, we may store the values of the weights. Let's make an example. In this case, we're going to represent the weighted adhesion C matrix and we store, we use this syntax in the first line to store the infinite weight, okay? So in this case, if you want to test for the membership here, we should check if the values corresponding to A and B is less than infinite, which is in this case true, but it is not true in case of C and E. So there's no edge connecting C and D, or in other words, the weight of the edges is infinite, okay? And finally, we may calculate the degree by summing on the row of the matrix. In this case, we apply minus one at the end because we want to remove the diagonal element, okay? So in case of A node, the degree is five, which is the number of elements in the row except for the diagonal, okay? And finally, some consideration about the efficient graph representation. This is a great book, okay? This is from Jeremy Spinered. It's called efficient graph representation. He says in this book that there is a classical misconception. In general, after taking an algorithm class, we may end up considering that there are two methods representing the graph in a computer science, adhesion symmetries and adhesion lists. It is faster to work with the adhesion symmetries, but they use more spaces. On the other hand, you may choose adhesion lists in case the space resources is the most important to you. There are many interesting ways to represent the graphs, actually. Adhesion symmetries and adhesion lists are just two ways to represent graphs, not the only two ways, okay? For instance, you may have adhesion sets, edge lists, or incident matrix, okay? Which is something a little bit different from the adhesion symmetrics. There are specialized representation for different graph types. For instance, trees are specialized versions of graphs and they're represented in a very different way from general purpose graphs. Interval graphs is another very interesting example. The rule of thumb that he suggests is decide on the asymptotic overall complexity and decide which is the structure that best works for your particular problem and for your particular data, okay? So this is just a very interesting consideration about this. Okay. So here we are to the traversal of meses, okay? So we have a meses, okay? A partial traversal of a typical role-playing dungeon. You may think to the rooms as the edge of a graph and you may think, sorry, to as in the nodes of graph and you may think to the doors connecting the different rooms as the edges of the graph, okay? The traversal tree is defined by your track, so by the track you apply while you're traversing the maze. The fringe or the frontier or the traversal queue consists to the neighboring rooms, okay? The remaining darkened rooms here in the maze are the nodes that have not been discovered yet, okay? So this is a very simple and trivial example of classical problem represented by means of a graph structure, okay? In this case the graph is implicit. We have no explicit graph in this representation. For the following slides I'll be using the dictionary with addition sets for graph representation, okay? One of the most famous traversal algorithm for meses is the one by Trimo. And Lucas in 1891 is Recreation Mathematique, Burke explains this argument in this way. I'm trying to quote this one. To completely traversal the passages of a library twice from any initial point, simply follow the rules posted by Trimo making H entry to or exit from an intersection. These rules may be summarized as follows. When possible avoid passing an intersection, you are already visited or avoid taking passages you have already traversed. Is this not a product approach which also applies in everyday life? Okay, so the basic idea here is to get the information, every time we traverse an edge, so a connection between two nodes, we store entry or exit information. In this case we avoid taking the same steps and the same ways more than once, okay? The basic idea is backtracking. So the idea is start walking in any direction you want and then backtrack whenever you came to a dead end or an intersection that you are already walked through to avoid cycles. This is the basic idea and we will see in details this one. Okay, so sorry for the formatting. This is maybe due to the font size. I don't know. No, sorry for that. We may have this function in Python which is the arbitrary walk, okay? We assume as said before, we assume graph represented as a dictionary of decency sets and we have an input S node which is the starting point, okay? So we select the random starting point or the point where we are now. We have two structures, the P and the Q structures, P is the predecessors, Q is the sets of nodes to visit. We select randomly, we add to the set Q to the starting point and we iterate while the set Q has elements in sight. We pick one element in the Q completely arbitrary, okay? By using the Q.pop method and for every node which is in the, which is incident to the U node but has not been visited yet, we select add this node to the Q and then we store the predecessor, okay? So in this case, we walk through the graph one step by step, okay? Actually, this algorithm will traverse the single connected component, okay? So we will find the connected components of a graph starting from the given node. P is the predecessor's tree. If we would like to get all the connected components in graph, we may iterate on the every node in a graph and we call the walk function and we simply update the structure scene which is a set of visited nodes, okay? So far so good? Okay. Well, thank you. The time complexity of this algorithm is theta of E plus B, so this, the, which is the complexity of almost every traversal algorithm, okay? Just another detail about this, the important part here is that when we select the nodes we want to visit, we use, we rely on the pop method here on the set which picks randomly an element in the set, okay? So this is why this is called an arbitrary walk. But if we want to go deep, so we want to apply the so-called deft-first traversal, we might use another trick. So instead of picking randomly the element in the structure, we might use a structure that has a precise, a predefined protocol to get the elements. In this case, in the case of deft-first traversal is the last in first out protocol, okay? This is a recursive implementation but in Python we may, or in general we may, every, always translate a recursive implementation to an iterative one. In this case, we rely on the list structure, okay? So every time we end up to a node which is, which is new, we extend the fringe to the set of incident nodes to the new one, okay? In walk implementation we use set, okay? So we would risk having the same nodes scheduled for more than one visit, okay? So this is why the deft-first traversal is more efficient in the general case. In DFS we use the LIFOQs, so every node is scheduled for visit only once. If you want to generalize this traversal, we might write this function which is traverse and the type of traversal we're going to make depends on the structure we're going to use for the fringe, okay? In general this is a set which is very similar to the arbitrary walk already shown a few slides ago. If we use the list, so the Python list, this will end up to be deft-first traversal. Using yield allows for lazy iterations so we may create a generator in Python which is very, very useful and it's more efficient for dance and make graphs, okay? And that's it for, yeah, thank you. And that's it for mazes, okay? But what about the infinite mazes and unweighted shortest part? So far the over-eager behavior of the deft-first traversal hasn't been unweighted a problem. Actually the problem of the deft-first traversal or the deft-first search is that it may run for ages and never get back, okay? So in case of very, very big graphs, the deft-first traversal is not a good choice, okay? In fact in case of visiting or crawling the graph of the web, the deft-first traversal is not the solution, okay? In general the very, the solution is another kind of traversal which is the breadth-first search that I'm going to talk about now. And what if we would like to follow the shortest path in traversing the mazes? So in this case the shortest path means the minimum number of intersections, okay? The first attempt is the iterative deepening deft-first search. I'm not going into the details of this algorithm. The basic idea is that this algorithm applies the deft-first search traversal many times with different deft limits, okay? But this algorithm led to another well-known algorithm which is the breadth-first search. The breadth-first search is very similar to the iterative DFS, so the iterative version of the deft-first search, but it uses a different structure, okay? In this case the breadth-first search traversal uses a structure which is a queue and that follows the FIFO, so first-come-first-served, which is the first-in-first-out, in other words. And we may implement this in Python using the DQ structure, which is in the standard library. This is very simple and very trivial to implement. The DQ actually corresponds to a double-linked queue in Python, as you may know, and in this case we rely on the pop-left function to get the element we want to start visiting, start traversing, and the pop-left is constant time because of the implementation of the DQ structure. The complexity of the overall breadth-first search is always, again, theta of the dimension of edges plus the dimension of vertices, and it can be demonstrated that the BFS calculates always a correct answer. In case of weighted graphs, another well-known problem in computer science and in algorithms in general is the shortest path, but this time the shortest path means the path with the minimum cost, sorry. As this comic says, I would like to have a pillow talk, so we will talk about the dystra algorithm, not the Bellman for it, as in this XKCD comic. Okay. The dystra shortest path on directed as a graph relies on the relax function. The basic idea of the relax function is to propagate the knowledge about the shortest path one step at a time. We look for an improvement to the currently known distance to V by trying to take the shortcut through U, which is the vertex we want to test. If this shortcut, if it is a shortcut, so if we pass through the U node cost less than the already completed shortest path, we will follow that direction, and we test this one step at a time. Okay. So, yeah. The dystra algorithm is the shortest path algorithm. Python exploits the eep queue structure, which is a priority queue that allows us to get the nodes. Every time we add a node to the structure, we are guaranteed to respect the epify relation, so we add the nodes in order. This is a priority queue, so this is an order queue, and we get the element with epop, which returns the element with the least priority. Okay. The difference in the relations between the dystra algorithm, breadth-first search algorithm, is that dystra allows assigning distances other than one for each step. But meanwhile, the breadth-first search basically just expands the search by one step, which happens to have the effect of finding the smallest number of steps it takes to get to any given node from any source. Okay. The last example here is the shortest path for all pairs, because dystra is shortest path, also known as shortest path single source, so you have a node, a target node you want to optimize, and you start looking for the shortest path considering that single node. Okay. Another very interesting problem is the shortest path for all pairs, okay, so we're interested in getting the shortest path for every possible traversal on our graph. This is usually solved by a well-known algorithm, which is the flight virtual algorithm, that relies on an optimization technique, which is called dynamic programming, also known as don't repeat yourself. In mathematics, the dynamic programming is a method for solving complex problems by breaking them down into simpler sub-problems. It is applicable to problems exhibiting the properties of overlapping sub-problems and optimal substructures. So the basic idea is memoization. Memoization means cache the value of already solved problems, instead of recomputing every time the same sub-problems. We may create a function for memoization in Python in a very elegant way. I like this a lot. We may create a memo decorator, which explains the wraps function from fun tools, which wraps is a sort of shortcut for a partial function, which is partial. And in this case, we create this decorator. In this decorator, we create a cache, which is the dictionary that stores the values of sub-problems. Okay. Let's say it in action. The flight virtual algorithm is a problem that may rely on the dynamic programming technique. Because the basic idea is let us consider the DUBK, be the length of the shortest path that exists from node U to node B. If you're only allowed to use the first K nodes in your graph, right? You can decompose the problem as it follows. The distance from node U to node B using the K nodes in your path corresponds to the minimum to the distance from U to V without taking that node. Or the cost of the path that goes from U to K and K to V. Maybe this is more clear in this loop. We might consider whether or not to include the node K in the shortest path. So we calculate the cost of the path that passes through the K nodes, starting from U and ending to V. If this is a not convenient cost, so if the cost of not taking that node is less than taking that node, we simply disregard that node. Otherwise, we include that node, the node K, in the shortest path. We may implement the Floyd-Borsch algorithm in Python, considering for instance the addition, symmetric, and the distance only solution. In this case, we deep copy the graphs and updates in place the weights of the distances. But we may also exploit the memorization reps. Sorry for the formatting is very awful. And we may recursively apply the D function, which is the function that calculates the distance from nodes using the memorization. So if we end up to distance from node U, V, and K are really calculated, the memorization functions avoid to calculate it again and returns the value, thanks to the decorator presented before. So that's it. Some references, Python algorithms. This is a very introductory book for this kind of stuff. The algorithm design manual is not actually related to Python, but it's a very good reference manual. And that's it. Thank you very much for your kind attention. Okay. Thank you very much, Valero. Any questions? We still have time. It's lunch break. So please feel free to ask questions. Okay. So I think it was a very compact introduction. Thank you very much. I could say...