 So, this talk is, as was introduced, the idea is that to show some interesting pieces of Ruby Core and start a library, to hopefully use simple but not simplistic examples. And the main problem with examples is that the people who write them are usually so much into them that they make them very interesting to themselves but not really readable. So please, if there's anything that is too complicated or that you would like me to explain a bit more or stop at some slide, just shout at me and I will stop or go back to a slide. So, my name is Peter. I have no clue how to pronounce my nickname, so I'm totally fine with people mispronouncing my last name. I work at the University. I'm the CSO of Ruby and Race Consultancy and I'm currently in JavaScript recently. We do Ember. So, if you want any help with that, please let me know. And yeah, and at the university, when you work at a public university, at your department you usually have some servers, and because you cannot really employ dedicated DevOps people, you will find just the least incompetent person on your team to run those servers, which for some reason happens to be me, and I'm so clueless when it comes to a puppet or a chef that I decided to use Ansible, and I orchestrate our servers with Ansible, which is apparently the right term to say. Ansible is a two-year-old application that basically lets you do this. It basically lets you describe how a server is supposed to work. This is for one of our servers, and the main idea behind it is you write a script how the server is supposed to behave at the end, and it makes sure it behaves like this. So, you can rerun it. It's important in that if there are some packages already installed, it keeps them installed, and so on, and this is the typical output, which basically means refresh the package cache, upgrade packages if there is any bash package on your system, probably upgraded if it's older than an hour, and install all of the funny packages that you're supposed to use. And this is somewhat of a bland UI in that it basically prints out those things with the stars and so on and so forth, but if you happen to have installed on your system a 15-year-old program that is called Cousey, and that's exactly what it says, then suddenly, without any changes to Ansible, the output at Ansible starts to look like this. You have lovely ASCII, ASCII, ASCII, ART, ASCII, ART, Couse saying all of those things. And first, this is surprisingly funny, especially if it's 3 in the morning and you have yet another bash upgrade because last week was really, really fun. And the second thing is, if you have this environmental variable set random, Ansible Couse selection random, obviously, what could be a better name, then the cow is actually randomized, which means that every single server gets a different output, so if you have all of your different servers in different tabs and keep them rewriting the upgrades every hour, for example, then you can see that the different servers are being upgraded with the different cow images. So Cousey, as I said, I think not only the initial release was 15 years ago, I think it's still the version that we all use, the way it works is basically a Cousey program that you pass a string and it says that string that you pass to it. And if you want to have a different cow or bunny in this case, you just pass the dash F flag that takes the name of a cow file, and that cow file is used as a template to render it. As you can see in this case, when you are using shell, it's really important to get all the quotation marks right, because if we got the quotation marks wrong, or if there was a command inside, hiding inside of this string, then we might voluntarily run that command. So can we call Cousey from Ruby? This is a Ruby conference, so let's actually get some Ruby on the slides. Yes, of course, there is a system method that is defined on the kernel module, which means it's available everywhere, that takes a string and runs that string, executes that command. In this case, there is a DAG that says attack surface intensifies. The problem with this is that if you construct this string like this, like you take the text from so outside source, hopefully it's your own data, but maybe it's some user-submitted data, you might want to run it like this, and it will still work. The thing is, what if we make the text a little bit funnier? Let's just add a semicolon and some random characters at the end of it. What will happen if we run it? I'm not sure, I'm not pasting the output because I didn't set up a virtual environment just to test this one out, but basically, the thing is you should really be careful about sanitizing your input, or you should know that the system takes not only a single string to run, but it actually can take any number of strings you want, and the first one will be the command that you're supposed to run, and all the others will be parameters, and if you do it like this, then this is what it expands to, obviously, like call say dash f, these are all separate strings, and this actually works, like system is clever enough to sanitize or escape the last part and not let you wreak havoc to your own files. The problem with the system command, though, is that it actually runs it in a sub-show, so there is no way of capturing the output of the system call. If you try to assign it to anything and then print it, this is just an echo, then the echo will actually still run, you will still get the output from the echo, and the result of the system method is just a boolean that says whether that was a successful run or not, just a boolean that says true or false, whether the command succeeded. So can we capture the output? Is there a way in Ruby to capture the output? Well, there is. There is a kernel back-tick method that can be used to construct something like this, so just a string inside of back-ticks. It looks like a special Ruby syntax, but it's not. It's just like the Ruby parser finds that there's a matching back-tick and just takes this all, turns it into a string and passes it to the back-tick kernel method. So we can, for example, redefine it. And the problem with this is that this is exactly the same problem as with the original system call that we use with just a single string. I don't know whether you know what Bash Profile is. It's basically a file with subsequent commands that set up your Bash environment, and Bash History is a file that means, for example, the last 100 or the last 1,000 commands that you run in Bash. Source is a Bash command that runs all of the lines in that file. So if you source Bash Profile, it will set up your Bash Profile. If instead you source Bash History, you will run your last 1,000 commands. Probably one of them is rm-rf-star, and it runs them in the context of the directory you are now. So if you have this, the first thing you might notice is that the quote is missing those two parts that are in the back-ticks. And the second part is there are those two lines that basically say you're super lucky because your default shell is SH, which doesn't have the source command. If your default shell was Bash, first, you have had a lot of fun last week. Second, you would have even more fun because you would be the person that says, what happened to my system today? The thing is whatever is in your Bash History, it's you know best, right? So is there a way to make back-ticks, which are super useful because back-ticks allow us to get the output of the command into a variable. So back to our Ruby, is there a way to make them safe? And one of the standard libraries is the Shell Words library that is 16 years old now. And what it can do is it can escape a text in a shell-safe manner. So we can pass the text to escape method for the Shell Words module, and then we can pass it to the back-ticks and it will actually work properly. So if you ever do any kind of passing strings to Shell, especially if they are user-provided, first you shouldn't do this, even if it's just, you know, somebody uploads a file and we want to preserve the name of that file before we mangle it through image magic. Be very, very cautious before you pass the name of the file to your image magic instance. If you have to do this, if you have to preserve the name, use Shell Words to escape it. But the thing with POSIX Shells is that not only the output, the standard output is what is interesting. There are also things like standard error, standard input, which you might want to feed into the command. And for this, there is, again, a standard library. This one is a bit younger. It's only 15 years old. And one of the methods that, once you require Open3, you have the Open3 module, and one of the methods is Capture3, which allows you to manipulate all of the... So it returns the output, standard output, standard error, it returns a status object that tells you how it went. That is not a simple boolean, but it's actually a status object. And if you want to, one of the parameters you can pass is the standard input. So this works. You get a very nice caller that says something. And the thing is that it also works like the system method. It's that if you pass those as separate strings, then this will actually escape them properly, and we won't get the contents of our Etsy password file concatenated to the output. We will actually just get it into this parameter properly escaped. And this shows also that the status is an actual status object that you can send the success method, and it will tell you that this is true. So it's not a boolean, it's an object that you can also, for example, ask for the numeric value of the output. So if there are different kinds of errors, they're usually in bash or in POSIX compliant programs. They are usually signified by different output error numbers. Okay. But the thing with start out input, start out output and start out error is that they are really streams, and it would be cool if we could operate them like on any other I.O. object in Ruby, and we can also still do this without reaching for any kind of external gem. There is a P open three method that we pass what we want to as the command, and we pass it a block with four parameters. The first one is standard input stream, the second standard output, the third standard error, and the fourth is a thread that will run this command. So this runs the command in a separate thread. So first you get the control back much sooner, and you get actually stream objects. So you can put this into the stream's separate lines, then you can close the input stream. At that point, COSE will say, okay, so this is everything that people wanted me to make the COSE, and then you can read from the output stream, and also the thread is actually where the code executes. So you can wait for the thread's value, and this is the success object. So this will make the Darth Vader koala say it can be a little bit difficult to secure the connections, it's decided using JSON-P and MD5. And again, as you can see, the status object is returned, and it has the status method that returns it . Okay, but if we have different kind of terminals, usually the default terminal that you open is probably 24 by 80 characters or 25 lines by 80 columns. But people can make their terminals more narrow, and it would be nice if we didn't print very wide COSE into narrow terminals because then they would get wrapped. Is there a way to peek at the co-files, see what the templates are, and maybe find the most narrow co-file that we can use. So first, let's peek at a single co-file. For this, you could just use the regular approach where we keep the paths to the files in strings, and then we use the file open stuff to manipulate them. But there is the path name library that is surprisingly young, it's only 11 years old, but it's there, and it's my really pet peeve that people should use it much more. There is a great article from Arne that describes all of the nice things about the path name that is linked at the end of the stock, but just to show you some things that the path name objects can do, first there's a glob method that you can pass a glob that says, basically, give me all of the paths to all of the co-files, and if we get the first one, first we have an object that represents a path, but it's not a string, so it represents both the path and to some extent the file itself. We can actually do path.read on the first object, and we can see that it looks like this. This, if you are unlucky to recognize it on the first side, is a pearl file, and it's a template for one of the co-says that basically has some commands and then has a bit of pearl code that is a heardock, which is those long strings that have a starting and ending identifier and between those, they're basically a long string. So, first, we shouldn't really try to parse or even mangle pearl files with Ruby because that's obviously wrong, so let's do it. Let's start by dropping the dollar variables first. Well, it's super simple. We just gsub from anything that starts with a dollar and is followed by any characters or an underscore, and if we do so, it starts to look surprisingly like an elephant that is eaten by a snake, which obviously is one of the co-files that you can use and that your server can communicate with you by using this and transible saying, like, I just eaten an elephant and also upgraded bash again. So, can we keep only the heardock? Can we only print the heardock? Is there a Ruby construct that would allow us to print only the stuff between these identifiers? Does anybody know what's the most obscure Ruby, like core Ruby feature that could allow us to do this? There is a feature in Ruby that every single time a new Ruby release like 1.8, 1.9, 2.0, 2.1 is proposed is let's drop the flip-flops. Let's not have flip-flops anymore. There are things from you know, old time past, it's from cargo-culting pearl basically, and there's always somebody who says, no, I have a use case for flip-flops that really is good enough to keep them. I was trying to come up with the use case, as you can see there's some preparation people have to do to make flip-flops work in your mind again, but I actually came up with the use case and it's basically this. If we want to print all the lines between this heardock like this delimiter and this delimiter, we can use the flip-flop operator, which basically is, says, put the line and remove all the pearl variables if it's one of the lines between a line that contains EO and the line that is EO something by itself. I learned the hard way that not all of the heardocks in Kause are EOC some of them are EOF technically, this identifier can be anything, so you really shouldn't try to mangle pearl files with Ruby, but this is enough in practice, which is one of the things we learned that good solutions that work 90% of the time actually work. Great solutions that work 100% of the time but are never completed are not as good. So the way flip-flop works is basically you put it in an if conditional, it looks like a range, but it's not, and it basically will do this it won't do this up until this is met, then once this is true it will keep running this on a line, in this case line-by-line basis up until this is true. Once this is true, it will stop running this. Again, as I said, a drink or two is highly recommended before you try to wrap your mind around the flip-flops. Okay. Can we find the best co-file for narrow terminals? Can we actually also show that Ruby can be used in a functional way? And can we do this by still handling profiles with Ruby? Again, please don't do this. So let's because the flip-flop operator is somewhat gnarly, it's good to move it away to some place, ideally some other file. So let's put a lambda that takes a single argument, which is in this case a line, and put this whole flip-flop thing into that lambda. And that lambda basically says if we run this lambda over and over again, this happens, this gets executed, and it will again keep being executed until this is true. So this is basically a lambda that if you execute it over a collection, it will the moment this is true in that collection, it starts returning up until this is true. So let's now take all of the co-files, let's map them by the red lines method, which basically means turn an array into an array of arrays of lines from lines, from every element of this map will be an array of lines from that file. And then let's run this lambda on each of those arrays. Let's drop the nils, so now we have an array of arrays of lines that are only from inside of the HeroDoc. And then let's find the path name that has this returning the smallest number, and this returns the size of the longest line. So basically for every HeroDoc like this, translated into an array of lines, we find the longest line with the site that's the number of characters is the width of the co-file, and then we find the most narrow co-file in our system. And it happens to be run, which is really, really thin, so it will be really, really thin. And it will fit even if you have like 15 terminals, very narrow ones. And of course, what I'm trying to show here indirectly is that you can chain, this is all one call that is basically take a globe, then call them up on it, then call them up on it, then call them in by on it. This will return an array of lines, and put S when it's called an array of lines just with a new line, so concatenating them will actually create a string that looks like this. So you can do functional programming in Ruby, Arna tomorrow will tell you why probably you shouldn't. If we have this, it's very tempting to also find the widest co in our system, and all that is needed is replacing this minby by a maxby, and if we do this, we see that this turtle is the absolutely widest co-file that we have. Okay. So this is all cool, right? If we have a console that is wide enough, we can create, for example, a dragon co-file that says the real meaning of Halloween gets lost in all the costumes and marketing, most people don't even know Jesus for the dragon, let alone why. The problem is when we do this and we see it, you can see that the text is wrapped in 40 characters, because that's what callsators by default. It would be cool if we could use all of the width of our console and just wrap it to just wrap the text to be as wide as possible, but thanks to this may be shorter. We can do this, there is a dash W flag to co-say, we just need to find the width of the console. Again, this is possible without any gems. There is a IOConsol library that seems like it got into Ruby core like the distribution only four years ago. I'm not sure whether it was imported from elsewhere. I wasn't able to figure it out, but it seems that the first commit is from four years ago. And what we can do once we require IOConsol is the IO module gets a console method that has a wind-size method that returns in a very good basic old Ruby manner, just an array with two numbers and you're expected to know that the first number is the number of rows and the second number is the number of columns. So if we now take the last number from this array, which is the number of columns, subtract four, so we have some space in the borders of the balloon and the spaces, then we have, we can print, we can wrap the text to whatever size the console we currently have. So the last thing I want to show from the user-related or the system-related stuff in this library is an example why the libraries in this library are very different sometimes. This is an example of using Etsy library. This is a quote that I really, really hope is made up, but I've seen enough of systems to be skirted that it might be true. Please just another password to create your account, the password you have tempted is already used by user Jenkins. I can see how people end up with this kind of solution, like, okay, I hashed your password, but there's another hash like this and it belongs to Jenkins, so please use a different one because we don't want to have two hashes that look the same. Okay, so let's say we have a message, but depending on whether the user is in the admin group or not, we want to use a different co-file. And we can do this. The Etsy library is, again, 16 years old. There is a very nice and nicely named get-login method that returns the currently logged user, but there's also this wonderfully named get-the-grnum method that takes a string and returns an object that represents a group entry. The thing is, you cannot check if you have a user, you cannot ask the Etsy module what groups the user is in. But if you have a group, you can ask what users are in that group. If you know how POSIX works, it stems down to basically parsing two text files and the group file actually says who belongs there. So the get-grnum method returns a group object that has a wonderful mem method that you have to memorize that returns members. These are not actual user objects, they are actually strings. So if that array includes our user, let's use the tags co-file. If not, let's use the www1. And this example also shows that as many stuff in the library, if you require shell words, it will silently extend all of the strings with a new method. Because that's what was considered good practice back in 16 years ago, probably. Okay, this makes very nice tags saying this thing. To recap, all of the shell stuff, if you want to run something that is, you're not interested in its I.O., then just use kernel system, but please use separate arguments. If you want to have only simple I.O., use the back tick back, please, please, please, satinize. What's the word? Sanitize. Sanitize the input with shell words. If you want to have input, output, error streams, and separate thread, just use open three. If you want to have even higher control, there's a PTY standard library that is 15 years old and the shell standard library that is 13 years old, and they are both also enable you to run this. Please use path name everywhere. If you have any kind of strings, please use path name. It will fall back to be a good replacement for a string in most of the contents where you use I.O. console. The last part of this talk is about my favorite standard library. It's really confusing that it's a library of libraries. Let's say the COSI was really an expensive operation. What can we do to not really stress our system when we want to generate a lot of cows and we want to generate it for a very long time? We could cache it. The cache could get large. This is not a web application, so we won't get the process killed anyway in a couple of minutes. We can have a long-running process in which case the cache growing large might be a problem. What can be simpler than cache invalidation? Maybe naming things is simpler, but cache invalidation is trivial. Let's try to do this. We want to have this API. We want to have a co-cache object that we can generate. We want to have a funny text and we want, for example, a DAC to say this funny text. If we had two different texts, like Node.JK or TTY Hools, then we want our system to work like this. First we want to, the first time we ask for, in this case Node.JK, we want this to tell us on the debug output that it generates this cow and then prints a DAC, obviously. The next time we have a new library, we want to have a cache. We don't have a debug line, but if we reuse some of the text, we want to see that we have the output, but we don't have the debug line. Basically we want to create a cache that actually shows us that it works as a cache. We could do it like this. We could say, okay, I know all of the libraries. There is a forwardable that is 13 years ago, because we did it late in night. Because we know we are really on top and we know in Ruby 2.0 you have this percentage I construct that lets you construct an array of symbols. If you want an array of symbols, let's use percent I. Our team will love it. The next person who tries to read this will come to us and say, thank you for enlightening me. I will now use percent I to show that we really know Ruby and we will alias method fetch to that square brackets. And then we will have a couple of private methods, one that generates the cow using a logger. Logger is also something that comes from a library. It is 11 years old and gives you a logger new. You pass it a stream object and it will print into that stream object whenever you call logger. And we can also show that we want to return a logger here but we also need to adjust it so we will use tap which is this fancy method that made its way from Rails to Ruby core. So first when it was only in Rails it was absolutely the worst idea ever but then it was translated to Ruby core and everybody said okay, that's actually very useful things. This is one way of implementing it. The other way of doing it is to make things simpler and your code will be read many more times than it is being written. So let's actually do it a bit simpler. Let's create a very simple hash. So this hash worked in a way that it was initialized with a block and the block is called whenever there is a missing key. So we basically offloaded all of the missing cow generation to this hash and whenever there is a missing key we can make it much more explicit and simpler. Let's say that our fetch method will try to fetch this from the hash and if it's missing it will generate it. As you can see the forwardable library is gone, the delegate method is gone, the alias method is gone. It's maybe less fancy, maybe less. I've been doing this for many, many years but actually much more readable. Please try whether being explicit is not sometimes actually simpler and better. So we have this, we run this, it actually works. The problem is as I said this cache will grow forever. If there was a way to expire it easily, for example whenever we actually need some memory. So if there was a way to make those cows automatically expire whenever the garbage collector is run because we run out of memory. There is. The weak ref is my favourite standard library library and once we require it, it's one of the oldest ones, it's 16 years old and it allows us to wrap any object into a weak ref. And the way it works, the object once wrapped in a weak ref still has all the same API. The weak ref will forward all of the calls that are made to weak ref to that object below it but it while still keeping a reference to that object and the question for you is how it does it it still keeps a reference to the object but it will let the garbage collector garbage collect that object whenever it needs to. So if we do this, whenever we have a cache miss, we generate a new cow and save it but wrapped in a weak ref, then we can try to run this which is fetch and generate the first one, fetch and generate the second one. This one shouldn't need generating but then we run the garbage collector, so this call should regenerate the cow for the string because now our cache should be busted and empty. And we try to run it and this actually means it works. It generates the first cow, prints it, generates the second one, prints it, takes this one from the cache so there's no debug line that says it generates it but then it runs the garbage collector and tries to fetch this cow which is now missing and the weak ref says there's an invalid reference probably recycled. Probably the object that has this cow that was wrapped in the weak ref got recycled by that garbage collector call. And of course we need to handle this and the way to handle this is instead of basically saying that if there is a cow with that key just return it and we check whether it's there and whether I think it might be the only method that weak ref adds is a weak ref alive method that basically checks whether its object actually exists so if it's in the hash cache and the reference is alive it wasn't garbage collected then return it otherwise so if it's missing or it got recycled, regenerate it, wrap it in the other weak reference and save it into the hash and it works now the first one is generated second one is generated this is pulled out of the cache here the garbage collector runs and here again we need to regenerate the cow and have it here so the questions for you is weak ref is a library written in Ruby how is it possible that the library written in pure Ruby that is not a C extension can work like this have a reference to another object keep the reference somehow and at the same time let the garbage collector recycle that object and the way garbage collector works if you don't know is that it recycles objects that are not referenced by anything else so how is it possible that weak ref keeps a reference to another object but lets that object be garbage collected to wrap up the talk the title of the stock is really really bad but I was asked by the organizers at some point could you please really really provide the title for the stock so I decided I will just reuse those words and mingle them behind recognition but this those words strong opinions weekly held by Paul Safo who at the time was director for Institutes for the Future at Palo Alto I think is in my opinion a very good guiding motto for thinking a lot of things he coined it when he thought about predicting what will happen and his reasoning was try to have gut feeling predictions if you can't make any better predictions go with them but be very open about adjusting your opinions when the facts change or new facts arrive so do have strong opinions but hold them weekly be open to changing your opinions and the thing I try to convey in this talk is that there are cases where you might want to use standard libraries and actually they might be good enough it's good to know what Ruby brings you out of the box it's good to know how you can use Ruby it's good to know that you know Triska I can never pronounce it properly Triska I can pronounce it wrong but when the facts are personally upset that although 13 untrusts appear to succeed and reinforce their beliefs the state doesn't stick 13 untrusted subsequently returns false similarly logicians and cynics alike must re-jace in despair that true untrust untrusted is perpetually false these are probably Ruby calls that you might have not seen before that makes the talk look much better than it really was, but also because what I want to convey is that there are a lot of things in Ruby Core that are there for a reason that might not be very often used, but the reason why they are there might be interesting to look into. So your homework, please try to figure out how WeCraft can possibly work, how it is possible that a Ruby library creates objects that can hold references to other objects that can be garbage collected. Once you figure it out or want to cheat, go to this website. There are wonderful posts on Stano Library. One of them explains how WeCraft works. There are earnest posts at SitePoint. One of them explains how PathName works and all of its APIs. There is this Ruby Stano Library account that is run by some very lazy person that hasn't been posting for a long time, but hopefully if we can all shame them into reviving this account, then we can possibly probably bask in the glory of Ruby stuff again. Of course, probably that person can also take suggestions. And there was a very nice presentation stop reinventing the wheel that had a very distanced view of Stano Library, reasons why you can use certain libraries, why you probably shouldn't use others and go for the gems. There is a series of blog posts, those and ways to start sub-processes in Ruby. That is, as it says, three blog posts with all of those things, if that was the part that picked your interest in this talk. And finally, if you have a spar $9 a month or you can cut one coffee a week from your life, please, please consider subscribing to Abdi Grimm's Ruby Tapas. These are screencasts that appear twice a week, five-minute long screencasts that shows very nifty things from Ruby Stano Library, some very nice gems. And Abdi also created the Cowsays website that is basically a web UI for the Cowsay program, but also is open source so we can see how it works. I would like to thank you very much. The slides are here with all the links clickable. This is me online and I hope you enjoyed this talk. Thank you.