 And the first workshop, Martin Pavlachek, a quality engineer in Opelstik team in Red Hat, who is interested in, you see, those guys don't read the red sign on the door, okay? Just prepared for your workshop. So I'm going to just invite them, and some of you will try to guess. We are full, I'm sorry, Martin, I'm sorry. What's in there? He's interested in virtualization in Selenium and Janga. He's playing bass guitar and do climbing, so let's welcome the speaker. Thank you for your warm welcome. My name is Martin Pavlachek, and today I'd like to show you pretty cool stuff, what you can do in terminal with Python, if you maybe don't know. Let's jump into the real-life task. Imagine some video on YouTube, and you decide to download it. So you have several ways how you can do that. There exist some web services that can do this for you, or you can install an add-on to your browser, but it's still not so cool way how you can do that. Another task. Imagine some JSON like this, and you want to read the structure. Yeah, it's completely fine for scripts, machines, and this stuff, but it's not so comfortable for real humans. So how can Python can help us in such situations? There exists a package which is called PyTube that you can use in your scripts. Or if I would go back to the JSON example, there exists a module which is called JSON.tool, which actually works as a stream filter, and whatever puts in before it formats and prints it pretty nicely. So what we are going to do today, this session is considered as a workshop. So I'm going to show you a lot of demos that I've prepared for you, but also you can try all these stuff on your own. I will show you that it's in the next slide. In order to use these tools, I mean install them, I will do short introduction about pip, pipi, and what's virtual and if you maybe didn't know. I have a question for you. Please raise your hands. Who knows Python? I mean knows how to write the whole world. Ooh, it's better than I expected. Thank you. But for others, I have good news. You don't have to know the language to be able to use these tools, so no worries. That's why this session wouldn't be about the Python as a language. So I'm not going to tell anything about decorators, generators, and other Python-specific stuff. So you can try it on your own. If you visit this application, you will find that all materials for this session. When you finish the cloning, there's a script which creates a virtual N for you, and you can invoke it by this comment. In case you will have some problems with that, there are several, or at least a few, use these digraphs that you can use if you will have some problems. If you need, you will be seeking results first. If you can download it from the app. You will see. Press the button marker. If you are using Python installation, it has to be downloaded or installed into the system package manager. If you are using Fedora, CentOS, or REL, just type dnf install python-virtual-env. To enter the virtual-env, just source file which is in bin folder and activate. We are ready to play something. Welcome to my sandbox. I have prepared demos of several categories. The first of them would be about manipulation with files. You probably know zip archives, right? But you maybe didn't know that there exists a Python module which you can use for creating archives, listing files in there. And also, because it's just Python library, you can use the same functions in your scripts. So let's just shortly take a look on it. I have some examples here. So some modules can be invoked by python-m which will, let's say, execute the module itself as it works in this example. So python-m is a file. And I don't remember all parameters. So let's list it. So it's in there. But we can also do that the same by python. So let's open ipython which is interactive, more interactive version of python. Import the module, open an archive and see what's in there. And also get some more information about just one of the files that would be python.png. For example, get size after compression or before. Another one which is module which is called fileCMP. It stands for file comparison. It actually works as well for comparing directories. So let's try that. And you can see that there is one file which is called .ng which is contained in the both directories. Okay. We can do the same in the python itself. So let's import the module, do the div and let's take a look what's just in the b directory. This is a content and this is common. In the report we just saw it in previous from the command line. Look for some text and image filters or tools. As I show you on the just beginning, there exists a tool or package which is built in which is json.tool. You can use it as a filter. I mean it works for standard depot and standard output but also you can pass files that should work. I have another question. Who of you knows where is... There exists some algorithm which is called root13. Do you have any idea where this can be used or some application or some place where you notice this? For example, there exists one kind of outdoor game about finding stuff outside. Okay. It's geocaching. Because when you visit some page of the cache, there is a clue that should help you to find the actual location of the box. And this clue is encoded into this algorithm. What's nice? It's symmetric. So, hi-all would be translated into uvnyy but you can use it twice and you will get the same input. Markdown. You probably noticed that a lot of GitHub repos are using readme files which looks pretty nice but actually the others of the repo actually don't write HTML into that files. In the most cases, it's written in Markdown language. And Python provides a package which converts this Markdown syntax into HTML. So the input, for example, kind of looks like this. There are headlines lists and you can turn it into the HTML which looks like this. So this is the output. But also there is a package which you can use for inversion conversation that tries to parse HTML file and put it into the Markdown back. The name is Markdownify. Some leftovers. And actually this package doesn't provide command line API so it's necessary to use the Python. In this example, I just open the output which is example.html, read all lines and by Markdownify call just turn it into the Markdown back. So this is the input and this is the output in the Markdown. It doesn't look exactly the same but it's pretty similar. It was nice. Actually we can test it. I mean we can use Python again and use Markdown command which should turn this back into HTML. So let's see the differences. It doesn't look exactly the same but I think it's still pretty useful. There is only differences in rendering lists. One liner. This is my favorite. You probably know command which is called NL, number lines. But it's not so flexible as you sometimes you need. One liner module allows you to put an expression that you want to apply to the each line of the input. So this is ordinary output of NL and let's try to mimic the same output. There is NR variable. It stands for the number of the line and the underscore which is actually the line of the input. Let's look at another example. Because it's... Yeah, also sometimes you need or you want to print the filename of the input. That's why there exists FN variable which contains this name. And because it's ordinary Python expression you can, for example, use the upper method and print everything in uppercase. But we can still go further. The one liner module also contains ability or have ability to import other modules. In this example I will use regular expression substitution so I will import the module and turns every number into the number as a word. So now you can see that port number and all constants are substituted. Pigments. Probably once sometimes colorize some source code. This utility is pretty handy for such situations. It's a pretty clever tool because it will adjust the output to the formatting. What can went wrong? So if I specify output as PNG it will create the picture. You can also the output can be also order HTML page or you can use different color theme and also add numbering of lines which looks like this. But there is also an option that you can unfortunately I can't show you this example. But there is one feature that will display non-visible characters, bytes, dots and other characters. So that was Pigments. MgDef is pretty handy if you have two pretty similar pictures it's not so obvious why they are different or if there is there any difference. So if I show you the first and the second you probably didn't notice anything. But if I mgdef it displays pretty nicely where are differences. Also you can specify background color so it can look a little bit better. Also you can use file as an output and have it as an order folder file. Let's take a look for something interesting for the Python as a language. You probably know PDB, Python Debugger which is integrated in... which is built in. You also know that there exists a pretty nice command line or text-based Debugger which is called PyUDB. I have some example here. This is just pretty easy example. On this line I put a breakpoint. You can use cursor arrows or VIM movements to look around. If you press N it will do one step or one. It will jump into the next line. If I like to go inside press S like a step and you can go further. On the right side there is variable list and what's pretty nice you can also see the structure of each variable. Okay, so go next. Something is wrong because exception was raised. So let's hit E and it seems that I pass something wrong into this method. I don't want to continue because I need to edit the code. So I will press Q as a quit which immediately exits the Debugger. So let's fix it. Also there is nice feature which is pretty common in Debuggers but this one also you have to do the same which is run into the cursor. So move your cursor whenever you want. Press T and the execution would be stuck at this place. Also when the method returns something you will see it on the right side. This exception generated on STD out just about the Python Debugger. If I remove the back point it wouldn't be there. So that was another Debugger that you can use. Notebook is pretty pretty nice tool. Actually it is WebServer. It provides a lot of features. One of that I like the most is terminal. It doesn't look well. So I am in the browser but I still do everything as in order terminal. That's one thing. Another is the Python Notebook. This web environment lets you interactively execute some parts of your code. Also you can run off these cells at once. So if you hit Shift Enter it will execute this place. You can see that it really works for each execution. You can also put several lines in the code into the cell and run it once. But you can also it also contains auto-completition. It's pretty neat. You can also edit any other folder any other file. It works for a lot of other languages and such things. So this was Python Notebook. And what if you need create some data for testing purposes. In the case of CSV you can use a random data module which provides data gen functionality. According to spec file it will define the amount of lines of this file. The spec file can look like this. You can for the integer columns you can use ranges minimum on maximum. You can also specify a date also to delta between generated lines. And it's also possible to pass your own format extreme here. Or for some other cases you can use one of these choices which would be randomly picked up. What's pretty nice you can also pass a regular expression and this tool will define some random names. So let's take a look at it. This parameter stands for I want to create just 10 lines. It creates new file and this is the output. You can see that for example these pseudo names are generated according to regapps not all of them has the same length. I can draw it several times and also the output chains. Or you can fake others. I mean create some random names but this is much more clever tool because fakefactory also considers your national environment which can be sometimes pretty handy. First of all I will show the python version. Usage is pretty obvious. Whenever you invoke name it will pass you will get the new name or the new string. At this computer I have US environment. Also the names and the rest looks like that. But what if I want to generate three check addresses? You can just pass nationality specification and it works pretty nice. What if I would like to generate 10 check names? Did you find yourself there? Because yesterday it generates name of one one friend. I mean his name and surname as well. But it's pretty nice you can also try to for example use Bulgarian environment. I mean I'm not completely able to read this. No way. You can see nationalities you can use. So it's really nice. At this point I would like to do a short break. You can take a coffee or just walk around and we can be back in let's say 10 minutes. Yes. Please grab a coffee. Leave the bag for the phone. You can get some river if you want. Please be inside. Thank you. Good. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Yes. Let's see how it goes. Wait a minute, can I... I'm glad I managed to pull it out. It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... It's... There's Hephaxac. No. It's natural. It's yours personally. They put it in their squishies with... carrots, too. Could you introduce yourself? Why are you doing that? I wanna tell you something personal about me. In the past, when I was... I was at a meeting... Or I remember... I was in a meeting with... I don't know how to say it, it's just a little bit of a problem. It's just that in the future there will be some questions, I don't really know how to say it, because I think it's the most difficult example there. I don't know how to say it, I don't know how to say it, I don't know how to say it, I don't know how to say it. I will ask everyone to come home. It works? Yeah, it works. So what's about? You wanted to say something? Yeah, it's 40, but I'm over half of the presentation. So we will finish sooner? Yeah, and I also think that a lot of people are not back yet. Ah, now? Yeah. So I don't... Are there screams in the hall? We can... We can... Thanks for coming back, and let's continue on the other part. It should be about presentation. You can use OpenOffice Impress or another tool, but also there exists a package which will let you create presentation as interactive HTML page. The name is PythonPyDown. Beware, there also exists a package which is just called PyDown, but it's due to different stuff, so don't do the mistake as I did. So let's take a look how it works. The usage is pretty obvious. This is an example of how the slides can look like. It's an ordinary markdown syntax, but there is an exclamation mark slide which is a divider about content of each slide. So we can run it. And it will generate HTML folder with index HTML inside it. And this is the output. You can use your arrow keys to navigate between the slides. It also supports pretty nice code highlighting in different languages. I think it also uses pigments that I showed you in some previous example. It also contains some pretty nice animations. If you want. And that's it. Another way how you can do the presentation from markdown is called ODPDown, which turns your markdown file into open-declared format. Actually, I use this tool for creating this presentation because it's much more easier, at least for me, to maintain the content in GitHub or in Git in plaintext form instead of ODP binary. So how it works. Let's have some slides, such like this. For this tool, there is no slide divider as I showed you in previous example, but it splits slides according to headlines. So this is, let's say, its chapter. And in the second level, headline would be content. I also have some template for this purpose. Let's have some first slide, second, and something for the end. But I would like to insert my markdown slides somewhere in between. So how it can be done. ODPDown understands several parameters. The most important are content master. The template contains several master pages, which would be used for first headlines and the second level headlines. So the main one, I call it chapter because it's name in the template, inside the template, and the content. Actually, I can show it again. Two master slides. This would be content chapter, content last slide, and this would be chapter. But I don't want to insert everything to the end of the template. I want to put it in the third position of the whole presentation. So that's why I am passing dash p, and the number where these markdown slides would be inserted. So let's run it, and let's take a look at what it looks like. So we have first slide from the template, second slide from the template, and this is a content from the markdown file, this one as well, and the last one was also contained in the template. You can combine some handcrafted slides with the others which you can maintain in the markdown file. Another pretty... I love this tool. It's called Play It Again Some. There are several other variations of this tool. I think one of them is Do It Live, but this one I found a little bit more handy. Actually, it does recording of your terminal session, and you can record it everywhere you want, again. The command that the package will install into your system is PS, and, for example, record something. It seems that it does nothing, but actually it's not true. It drops me into the sub-shell, and from this point every keystroke is recorded and the output as well. So I can print a date, or something with a lot of typos. Oh, no. Just hide. To stop the recording, just leave the terminal. I mean, just type exit, and let's play it again, and now the funny thing will happen. I can press any key on my keyboard, for example, just spacebar all the time, but actually it will show you just the recorded characters. So I hit enter, and actually this... I screwed it up. And continue typing. You can see it also record my typos, I mean backspaces. And I'm still clicking on the same button in my keyboard all the time, and just wait for the enter. There is also pretty nice feature, which is called AutoType and AutoWayPoint. AutoType is actually timeout between each keystroke, and AutoWayPoint is timeout or just wait for hit enter. Sure. It depends on what you want, because there exists also live something parameter, which is not... You can... You don't know about it from the help, but yeah, I can show you the example of our... It also captures the output, but with the parameter, it will execute the command by the time you are ready to execute the game. It will execute the command by the time you are replaying the session. So, if I decide to just use 50 milliseconds per each stroke, and 704 each enter, it looks like a video. Presentation... Because you are writing... fast. Yeah, actually you got me. I'm using this tool in several examples, but not all the time. Just a little bit difficult, because it's so easy to do a lot of typos here. I want to do you to enjoy this session with my typos. But yeah, back to your question. For example, let's see the day here, and do the same. So, yes. Record. Session. Date. If I would record it, the date is in past. Actually, I don't remember the parameter, but I know that there exists a live replay. So, date. As you can see, the date, the output is the same. So, yes. You can use recorded output, and you can also use it for the live output, which it depends on your own. Yeah, I've also forgotten. So, there's no HTML player for this. It's done by the same author. This is an example from the author. It's not done by me. It's still Firefox, and works the same. Let's move to some interesting system utilities. All of you are using keyboard. But there are much handy tools for manipulating with the content. I really like module, which is called copy and paste. Just just pip install copy paste and use it. I'm, for example, going to copy this section paste. Just put it in the output. And I can also use standard input as the content of the keyboard. Sometimes pretty handy. By iNotify it can be useful when you design some application, which, for example, have to watch some directory if there are some changes. For example, the user changes a config file or some new file will appear that. If I create some new file, the event would be raised. In your Python script you can create a handler that will do something on this event. Something for internet. Simple HTTP server pretty simple tool in case you have some folder and you want to share it, for example, on your local network to your friend. The usage is pretty obvious, just called the module. By default it works on port 8000 and this is the content. If I, for example, create a new folder and refresh the page and the content will appear. Definitely. So right now if I know your IP I can double-fire on the same Wi-Fi? It depends on your firewall configuration. By default I think it wouldn't be possible at least on this machine. If the port would be open it's shared to anybody on the same network. Don't let it this is a contest for you. Do any of you have any idea where or for which purpose you can use Donut by these days? You know that it's pretty old. Pardon me? There is also also let's say it's a toy which is more for fun than for real work. Say it a lot? It was a long time ago in the store wars. Yeah, exactly. Yeah, it's an ASCII art version of this movie. I'm not going to play the whole movie here. It takes more than half an hour at least. Yeah, it works. There was a new episode on cinema by last, I don't know, months or two. Another thing that I already mentioned in just the beginning is PyTube which is utility that you can use for downloading videos from the internet. Some action. Let's postpone it to afterwards because it will take some time. You can use the tool and with just passing the URL of some video. Hit enter and you don't have to worry about resolution or format. The tool server is more like a tool where you can use when you need to debug something. For example, you have to you are looking for some response 400. I mean, HTTP status. I do apologize. This will not work at this moment. But you can try it on your own. SIGAL is pretty handy tool if you have some bunch of photos and want to share it to others as the HTML gallery. So run SIGAL in it which creates some default configuration. The imported line is here, source which is the name for the pictures. I mean the content. SIGAL built will create all static HTML files and there is also built-in web server for presentation. It also contains full screen mode. I just press F like a full screen. What's also nice is that it takes load exposition if it's available. I have some few things for your terminal. PyChalk will let you in your Python scripts create colorful output. So the output looks like this and the usage is pretty easy. You can also specify some formatting options as a sickness understanding. RAD it can be called as a console highlighter. It also contains interactive mode for creating new rules. For example, highlighting hex numbers which will match this pattern. You can listed it. I already have prepared a rule which is name lock for this purpose. So it works this way. Or you can also use it for highlighting your locks. This is some simple input. It also can look like this because of these rules. The last package that I'd like to show you is ConwayCPU which is actually implementation of Game of Life that you maybe know. Actually, I would consider this like a screen saver. Yeah, I will need another terminal for this. It looks like a mess. But if I do some load on the processor it starts changing a lot. Because according to load the new seeds will appear there. It's also it can be distinguished by different color. If I kill it it starts slow down. No, it's not random. There is algorithm why some pixels are appear and disappear. It's not so. It looks like a random chaos but actually it isn't. During work on this topic wasn't heard even killed any zoologist or snake. I promise. So, remember this python will not bite you. Python can be your friend. Thank you for your attention. Actually, I didn't notice that but I think that on PiPi there can be some package for this. Actually, on PiPi there is all the thousands of packages and it's impossible to just browse all of them. But here, the rest is also pretty famous format and I hope that there would be something for that. How did you find all these packages? How did you find all these packages? Actually, I try that. I spend a lot of time by that. Actually, a lot of packages I found on Twitter of the PiPi and also I do recommend to piomtw which stands for Python module of the week, I think. There is a lot of document examples for built-ins especially and there are several other resources that I've used but there is not all of them. Do you have any other questions? When you mentioned PiPi do you also know that you talked about how to build and how to do so? I'm sure that there are other tools that does the same. I'd like to show just one example in Python that I've noticed. Good note. I didn't know that. Thank you. If there wouldn't be any other questions thank you again for your attention and enjoy it. Thank you. Yes. Independent packages some of them are also incorporated into Python 3 but the most tools that I've met is still from Python 2.7. Yeah? No, no, no. Sorry, sorry, sorry. Okay, cool. I just leave. This is our props. So, I really, how... Yes, like... No, please. Play like this. Yeah, please. We're going to play.