 All right great. Thanks for coming and I hope you all can hear me. Luckily we've got this wonderful mic My name is Lee Shang. I'll be talking about code doffing with Python So a little bit about the company I work for it's Yelp We connect people with great local businesses So we have an app and a mobile website to help you find interesting local businesses to work with and a little bit About myself is I'm an engineer at Yelp building distributed systems primarily with Kafka And previously I've been at Amazon a startup called we be data and Dropbox and I intern at Starfleet a little bit That's a data scientist. You'll get the pun in a moment if you think about it So a little few caveats. I'm not actually a golf player. So my analogies may fall a little bit flat I'm not talking about traditional code doffing in the way of Maximizing every single character that you use to produce a particular result I find that that tends to generate unreadable code the way that I'm going to try to express code doffing is a way That generates more readable code while being more concise And finally because we're talking about code. I use a lot of monospace types. So if that hurts your eyes I'm sorry. That's just kind of what we're dealing with in this presentation So for me, I'm defining code doffing as minimizing the number of strokes in each block of code And I'll define what a stroke in terms of what I'm saying is later and my thesis is that Concise code requires less cognitive load to understand because we spend a lot more time reading code than we actually do writing it And so spending a little bit of time to be more concise Enables the people that look at the code later to and especially yourselves when you look at your own code to understand What it is and I find that a great exercise when you're looking at concise code is to say can I read this out loud using? using English or whatever my native language is very simply And oops and finally like to quote William Shakespeare Right like gravity is the soul of wit to kind of like continue along the lines of shorter things is better And I sound a bit like a snake oil salesman and what I'm trying to say is when you shorten your code you'll increase your clarity and To of course because this is snake oil. I have harm no pythons in the making of this talk So what are strokes? I kind of define them as plus one for any keywords variable names and operators so I want to promote the idea of using longer variable names rather than single character ones and to be more expressive using the language and then I don't count Whitespace dots commas parentheses quotes colons and I'll give more examples of this visually later But I just want to kind of explain where I'm coming from And these I believe are all the kind of like structural things that kind of separate out the individual blocks of information So kind of ultimately what we're talking about is counting the units of information present in a particular block of code So why even bother doing something like this? Python has this wonderful Easter egg called import this and it will tell you about the philosophy of Python So I've selected a few lines here that I thought are particularly relevant to this this topic Beautiful is better than ugly simple is better than complex and if the implementation is easy to explain it may be a good idea So if you all seen code like this It's kind of a little thing where you pull out the address field from this my contact stick You you set an unknown as your default and then that way if it actually exists in there You can replace the unknown with a particular thing That took me a long time to explain what this actually does So Python has this wonderful feature where you can just say I want to get this Attribute out of a dict and give it a pass it a default value if it's not there So this is way shorter. I can describe exactly what it is in like eight words And to go back to my strokes metaphor, I can count them all up So two male as a variable is one though my my equal sign is an operator. That's the second one if you work that out, that's about 12 strokes and then my kind of get using a default thing is Only six strokes at the end of the day so that like numerically I can think about exactly how much I'm making my code shorter and have a framework for doing this So for the visual learners to kind of like see where this information is actually going The final set the final line is kind of really where the the nuts and bolts is going on like I have two male I'm setting it to the address field of the dict of my contact this if statement is kind of extraneous like that Actually just goes away entirely and the default in that first line that unknown like that's what's get pulled in gets pulled into the final result So if visually this kind of shows like where those those lines are going away or those those key words And so along the lines of a simple get like we can also just initialize a dict in the same way like we've kind of seen code like this this little block of code takes this item in this count variable which I'm trying to in this count dick and Kind of every time you run into a particular item you want to increment the count by one And so you have to initialize it by zero if you haven't seen it before So you might wrap this little block of code inside of a loop or something like that And this is also similarly like a giant block of code hard to describe Python has this Collection called a default dict which lets you set your initial value for a dictionary You import this default dict and then you can just you tell it what you want the default value to be in this particular case I want it to be an end switch defaults to zero But you can give it you can give it a more complex object if that's what you need in your particular use case And then you can just use the object as though it were automatically initialized with the thing that you want So I can just start incrementing up by one whenever I encounter an item so this is much much shorter and You see like I'm moving from 18 strokes down to 13 and that's including this import at the top Which you probably only see once per file. So if you do this multiple times it actually saves you a lot of work So another thing I want to talk about is cleaning up resources So in a lot of times we'll open up network sockets will open files We'll be doing a lot of a lot of things with resources and when we open in this particular case We're opening files reading them out line by line printing them out and then finally we have to close it at the end Why even bother closing this up like Python has a memory manager that Deals with the variables and stuff and memory that we allocate closes them up when they fall out of scope Why shouldn't we do this with resources? Well, it turns out that we actually can Python has this feature called a context manager that lets you Kind of define the particular thing that you want to do using this with keyword Set it to the variable and once it exits that block Python will close the thing automatically So this saves you one stroke But at the end of the day if you look that closed statement is really the only thing that you're saving And I've got a room right here because that's the automatic cleanup tool Now that didn't seem like it saved you that much But I think where it really starts to come into play is when you have to deal with exceptions So in this kind of contrived example, I'm opening up my in file and raising exception Just to do some stuff and I have to wrap it in a try finally block to make sure that my code my my file gets closed With the context manager it just happens automatically like it knows how to catch the exception Run the run the closing portion of the piece of code and then re-raise the exception so that something upstream can actually handle it So I don't have to do anything else It's already handled automatically and this is much more natural because this is how I'm thinking about the code and the business logic That I'm trying to run So to do one of these context managers so simple right like you've got this wonderful arcane block of code You implement these double underscore enter and the double underscore exit methods And so this particular example I took off of the Python Example where you kind of poorly add an HTML tag to a print statement So in this sort of thing we've got the enter method which prints out the opening tag with the name that you pass into this thing And then on the exit when you exit the block it prints out the closing tag So you can print out a bunch of stuff in the middle and I'll be wrapped in this HTML tags This is a lot of boilerplate right I think we can do a little bit better Python has this built-in decorator called a context manager with the idea that a lot of these things follow the path of a function execution You run a few you you run a few lines of code in the beginning You give control back to the like thing that's within the block that you want to run and then you'd run some stuff afterwards So in this particular example if I decorate this this method called tag with a context manager I can print out the header tag Yield control back to the main function that I'm like or the block that I'm running this from and then print out my closing tag So this is way way shorter And I think way easier to understand compared to this like enter exit thing that I had before and Then this even gives me enough space to print out an example at the end that shows how this can be used I have no I tagged my h1 header tag with the foo stuff printed out And when the result when I run this the result will be that it prints out foo wrapped in those tags. That's really great So next I want to talk a little bit about functions so functions I'm sure we've all heard of them. They are really that scary What they do is they take something as input and then they do some processing on it and then they produce some output So in this particular example, I want to say that like I have this cook function I'm taking in some corn and then when I run the cook function on it. I can produce popcorn This particular construct example doesn't do any of the error checking to make sure that I actually passed in corn to Produce the result, but it kind of shows like this important output sort of thing Python has this feature called a lambda which is another way of defining a function without giving it a name And this will come in to play in a little bit When I talk about some of the other functional programming aspects, but it's an important concept to to understand So in this particular case, I've got lambda food Which is the thing that's being passed in and the result is this popcorn So it's equivalent to the function above except I don't give it a name But it's a it's an object of Python can deal with and you can pass this around just like anything else like any other Variable that's set to an inch or something like that So this can be very power a very powerful concept to utilize with functions that act on other functions So when people talk about functional programming, they're talking about three major methods, which I'm going to go through real quick We have the map filter and reduce so the first one is map Which takes one of those functions like the cook one that I mentioned before as well as a list of items So you take map you run the list of items and you have the function and what map returns is a list of Items where that function has been run on every single element of that of that list So as in this particular example if I run map with a list of corn a cow and an egg With the cook function, I'm returning the output with cook run on all of them So the result is a list of Popcorn a hamburger and a fried egg pretty simple, right? So the next kind of functional programming thing that there is is called filter and what this does is it also takes a list of items and It takes a function that returns a Boolean and it returns a list of items for which that Boolean about evaluates to true So in this particular case, I've got the pop the list of popcorn hamburger and fried egg before and the is vegetarian function And then my output list is a popcorn and a fried egg Assuming that you think that eggs are vegetarian. That's a matter for debate and outside of the scope of this talk And finally, we have the reduce function which takes a list of items again And it takes a aggregator function, which is a function that takes in two inputs and produces one output So in this particular case, we have popcorn and a fried egg as the list for reduce You pass to eat function and the result is of course poop And I'm not really not that clever I stole this tweet from someone else, but I thought it was a good way to kind of describe the three main functions of functional programming So that seemed a little academic Let's get into some examples and I find that examples are very well done with Goofus and Galant which are part of my childhood in terms of Kind of describing a good kid and a bad kid and how they interact with the world Goofus is the bad kid that does poor reckless things and in this example He runs around with scissors pointed up so that when he trips and falls he stabs one of his eyes out and Galant is the good kid Who walks around very carefully with scissors pointed down so that when he falls he doesn't hurt himself too badly So Goofus and Galant explore functions Goofus thinks very iteratively he focuses on how to compute the result He thinks about the computations that he's trying to make and ultimately he loops over data to do the things that he's trying to do Galant on the other hand thinks functionally about the end result that he's trying to produce and he focuses on what the result is And so using kind of this mathematical function oriented thing He composes the functions in order to produce the result that he's looking for So let's talk about how how they can use map Goofus in this example is trying to compute a Given a given this nums array to compute a new list that has a double version of all of these numbers And so what happens is Goofus iterates over this array and has this result called double nums and appends Number times two to every single when you encounter every single number This is pretty straightforward reasonable block of code. It's actually very clear. I would say Now with functional programming what you can do is you have this map function and you want to pass it in a lambda which takes this What did I even type here? Okay, that's a typo on the slide It should be x and I don't know why this got turned into xbox But imagine that there is an x there. Maybe I was playing video games or something while making this slide So So I'm passing it a function with lambda of x which is that thing that we saw before And this function let this lambda transforms x to x times two and I pass it in the same numbers living I run the map function which will produce the double of the results and I have to tell Python that I want a list at the end And that's an implementation detail that I might not go into So this is a little bit shorter fits on one line Possibly a little bit clearer if you're starting to think in that sort of functional transformational form And now we've got reduce so we've got this like let's say I want to compute the total sum of all of the numbers in the list I have my total equals zero I'm iterating continuously over the list or at least Goofis does And then kind of adds this thing to the total and this takes 10 strokes as well So Galant has this accumulator function which is this lambda x comma y taking in two variables And then the result is x plus y and he continues to run this over the nums thing He runs reduce and he gets the total in the same sort of way So this is also 10 strokes it's about the same thing but it's kind of shorter compact fits on one line Which is great because I have a widescreen monitor and this is a widescreen And finally filters Goofis iterates over the numbers he runs this for loop again We've got this result accumulator thing which is a list and we say if this number mod two equals zero We append this to the result 16 strokes Galant can just run this filter method using a lambda that says that do lambda x mod two equals zero Which is that same little if statement returns true when it's even runs it over the nums list And then runs the filter thing over it and that produces that same over only even thing with 12 strokes So that's a bit quite a bit shorter. So this is I think a distinct advantage here for functional programming Yay But wait there's more So now that functional programming seemed a little bit arcane in some ways I would say like took a little bit to introduce and a little bit to explain Python has this concept called a comprehension which can be used in a lot of these cases And it's a much more natural way of constructing these lists and dicks as well So we have this kind of result We've seen this pattern in many of the examples that I was just talking about before we have this kind of results list that we're trying to produce We want to iterate over everything in a collection of things We run a condition on it and then we kind of append some sort of transformation on this particular thing This is kind of the basic pattern that like the previous three examples that I was talking about was doing And this is kind of 14 strokes at the end of the day Now not all of the examples use all of these features, but they're all kind of there So list comprehension is a way to define a list using this kind of bracket notation Which is what we say with a list And we say I want to do this transformation on the item for all the items in the thing If this condition is true like that was very natural for me to read out loud right Result is the list of the transformed item for the item in things If the condition is true on that thing Very very clean and easy to articulate Whereas the previous one if I were trying to describe it, it's a little bit harder I couldn't quite do it on stage And this is actually a little bit shorter right, this is 12 strokes rather than 14 So back to that like visual sort of thing that I think is kind of shows you where this information is going is We still are setting this result list And we're kind of creating a list at the end We want to have this for item in things that kind of moves to the middle We still have that if statement which is the filter thing and that goes to the end And then this transformation moves towards the beginning So this is like we're removing this result dot append thing in our in our list sort of thing And this is where those like two strokes go to So let's kind of go through those functional examples again And like let's see if we can do this with a list comprehension So before when I was talking about producing the doubled version of every element in a particular list We've got this map of lambda with and this one actually has the right thing without the Xbox This takes 10 strokes Now with Billy Maze's list comprehension We just say double nums is the list of x times 2 for every x and nums Very simple to articulate in plain language And this is super easy to understand right Filtering works in much the same way Galant has this thing where I'm going to filter for only even numbers within this list of nums 12 strokes And Billy Maze uses the comprehension We have only evens x for x and nums if x mod 2 is equal to 0 Now this is actually a little bit longer So my metaphor kind of breaks down a little bit But like which one is easier to understand I would say the second one is much easier to understand So don't listen to everything that I say But think about what you're trying to do And what your goals are And better reduces We've got this total accumulator And we have this like lambda x and we have mod this to do But then think about it like all of these functions When I'm saying a list comprehension my result is a list This total thing is really just producing a single number And the Shamwell guy who's the Billy Maze's chief competitor Realizes that there is in fact just a sum function in Python And you can just call that on a list And it will produce the result that you want to do So why not use that instead Because it really just does the thing that you want So think about these built-ins And I teased a little bit about with dicks And dick comprehensions work in the same way That list comprehensions do So instead of doing x for x whatever whatever You do for x and y The two like the key and the value for the particular list So here's a new example of Goofus Trying to create a mapping of integer values From 0 to 26 with the letters that they correspond to So this has, you know, character 97 Is the ASCII value for a So I add 0 to a and I'll get that like a value So my result will be a dick with 0 to a 1 to b and so on and so forth all the way up to 25 to z And this is 17 strokes So when I use the dick comprehension I have nums to letters which is the same variable And I have x which is the number Numeric index that I have And then the care 97 plus x function Which turns that number into a particular string And basically this works in the same sort of way That the list comprehension does You can use the if filter as well So if you understand list comprehensions You can do this with a dick as well And this is 14 strokes so it's much shorter And I like it So where can conciseness help? I would say they're helpful in presentations like this The more code that I can fit onto a slide The easier it is for it to read And with widescreen monitors like you can fit a lot more And also on the screen when I'm actually coding Like I can type a lot more I can see a lot and get a better picture For the code that I'm interacting with At any given point in time And finally like if you're ever interviewing I think that on a whiteboard Being able to write code very concisely Will give you a lot more ability To fit more code on your whiteboard As an example Quick whiteboarding tip If you start coding in the middle of a whiteboard Like naturally your eyes are drawn there You're essentially dealing with this font size A 26 by 6 screen If you're coding on a screen You would start at the upper left So why don't you do that with a whiteboard You start from there And your screen is way, way bigger So that lets you do a lot more I'm sure that we've all encountered that instance Where we've been coding on a whiteboard And then we're trying to juggle around And we run into the edge And we draw all kinds of arrows Really convoluted and hard to follow So start at the top left on a whiteboard The same way that you start on a piece of paper And a piece of code So final takeaways I'd say that stroke reduction Which is making code more concise Reduces the cognitive load to understand code We read code a lot more than we write it And therefore like the less we have to The less energy we have to spend To understand the code that we've written The better we are and like Better able to understand like The broader picture of how things are And Python has a lot of great features For enabling us to do more with less And I think this is a big part of learning And discovering all the different techniques And kind of mastering the art of Python And finally with some operation That I talked about For a lot of common things If you think that this should be really, really obvious There actually is probably a built-in Or a library So you should search and try to find what that is Because it will do exactly what You would expect it to do And hopefully cover a lot of those edge cases That you don't want to deal with So let somebody else have dealt with them first And like not have to worry about them later So finally a quote by Blaise Pascal I apologize for the length of this presentation But I didn't have enough time to make it any shorter And a little bit about Yelp We're hiring, so come see us at our booth And here's our wonderful social media links We've got a Facebook thing With our engineering stuff We've got Twitter Where we advertise our engineering blog Where we talk about a lot of the cool stuff That we do with Python As well as other things And we've got GitHub Which of course is a social network these days And that's the end of my talk Okay, we have five minutes So I could take some questions If someone has a question, please Raise your hand No Ah, there are two So sometimes it's nice to Observe a mechanism For making some code shorter And more concise And you're really impressed with it And you show somebody and they say I don't understand that It's not clear I preferred it the long way Because it's easy for my mind To read and comprehend You sort of want to keep it Because you're so proud of it But you sort of sometimes have to let go Do you have anything to say about that? I think it's always a delicate balancing act Between clarity and conciseness It's definitely something that I encounter Especially when I'm reviewing code Written by other people If it takes me longer to think about it It's something... For example, the first time That I saw this comprehension I was actually like This is really complicated This is really confusing And this is very arcane But as I kind of grew And learned about it And I asked those questions I'm like, oh, this is actually more clear As a particular thing And I've kind of promoted On the teams that I'm on This is a much more concise way Of doing this And because it's such a common thing to do For less common things Like producing lists Or I don't know I'd struggle to think of a less common Like if you're trying to open up A network soccer or something Where there's a lot more Like non-straightforward kinds of things The explicitness and explaining Exactly what you want to do Comments are kind of The best way of getting around it And I think it's really about Understanding what the team That you're working on is comfortable with And which kind of conciseness techniques Are best adopted there Does that kind of make sense? Thank you very much for your talk I really like the golf metaphor Do you know what the 19th hole Stands for? Hole 19? Not so much What's that? It's the place where we get food So it's doubly perfect That you have to talk right before lunch I should have thought about that When I was creating these slides But I didn't know why Slot was right before lunch So everybody off to the 19th hole Right? Okay, any more questions? Oh, yeah Great So one of the downsides Of doing these sorts of things Is that sometimes They make debugging harder Because there are cases Where you want to actually have the array Which doesn't actually materialize When you do all these list comprehensions So do you have any recommendations And how to deal with that? Not often I think sometimes If I'm debugging it's Yeah, I think the thing That I kind of wave my hands around In the functional programming example Where they produce this Where you have to wrap it in a list It actually produces a generator Which lets you produce Like the individual items In that particular thing So if you're kind of like Iterating over that You can produce the element Look at each element within there And kind of use that As a debugging tool When you generate these comprehensions Or something like that Straight off the bat Sometimes it's worth just Kind of outputting them Or inspecting them in a debugger Sorry, if that doesn't Quite answer your question Debugging is a harder Sort of concept That really depends on The environment And the editor Stuff that you're doing So maybe go to some of the PyCharm talks And I'm sure they'll Give you some better advice on that Alright, one last one Anyone? Nope, well Then give them a hand