 Haj, imam Peter Sonec, in tukaj imam tukaj prav, kako izgleda syslogenji v Pythonu. Imam z Hungrij, imam in evangelist v jednoj identitve, syslogenji je upstream developer, imam syslogenji, pakaging, sopport in v vkasi. Prosto imam tukaj prav. Oh, OK. Prosto imam tukaj prav syslogenji, in imam tukaj prav. Prosto imam tukaj prav syslogenji v Pythonu in veš simple source code izgledaj, in in Pythonu v syslogenji. Čak je v logi? To je rekočen v eventi, kako v ssh logi in tukaj prav, in tukaj prav, in v vrlo directory. Syslogenji v in s logi v dima v stavnih fokusov, in vsej vsej vsej vsej, vsej vsej vsej, in vsej vsej vsej, vsej vsej, vsej vsej, vsej, vsej, vsej, vsej, požodno tukaj prav in v sriznovu sodate. Kad sem tukaj prav trgasbil syslogenji z peterv 320 data collection. There are a wide variety of platform-specific sources that support it like devlog, zjurnal, sunstreams. As a central collector, it knows the legacy and the new system protocol over UDP, TCP and encrypted connections. But the absolute Jolly Joker source is the Python source. This way, you can add easily just about anything like an HTTP server to collect log messages or fetch messages from Amazon CloudVetch using the Python API or adding a Kafka source without really too much effort. The next role is processing. You can classify, normalize and structure log messages with built-in parsers. You can add log messages, and you don't have to think about falsifying messages here, but for example, anonymization is required by many different compliance regulations. You can reformat messages. For example, if your destination needs JSON, you can turn your messages to JSON, you can enrich messages using JIP or add additional fields on each content. And the Python parser can do practically all of the above I've mentioned, and you can also use it to enrich log messages from external databases and also do filtering, which brings us immediately to our next topic, filtering, which has two methods. One is discarding surplus log messages, and the other one is message routing, so you can make sure that, for example, all authentication-related messages reach your log analysis systems. It can be based on many possibilities like message content, parameters, different macros, using comparisons, and filtering functions, and best of all, any of these can be combined using Boolean operators to make really complex filtering. Finally, you have to store your log messages somewhere. Traditionally, syslog messages were stored locally into flat files or sent to a central location using the syslog protocol and saved to flat files over there. Over the times, many different destinations were added, like SQL, different big data destinations, like HEDU, plastic search, and now we also have Python destinations, so you can easily write support for just about anything. Personally, I did an MQTT destination, which already used in production in many places. Let's talk a few words about log messages. If you look at a typical log message, you see that it's a date, a host name, and some text after it, like this SSH login message. If you look at the text part, you see that it's an almost complete English sentence with some variable parts in it. It's pretty easy to be read by a human, but you will have quite tough time if you want to create alerts and reports automatically from these kind of messages. The solution for this is to use structured logging, in which case events are represented as name value pairs. For example, the SSH login I've shown you in the previous slide can be described as an application name, a user name, and a source IP address. SSH has name value pairs right from the beginning, the date, facility, priority, and so on. And parsers can turn many unstructured and some structured data also into name value pairs. And the Python bindings support name value pairs as well. So, before going deeper into Python, a tricky question, without looking into the next slide, what do you think, which is the most used SSH version? Some pointers, our most popular distribution has 3.5, the current version is 3.19, what do you think? Well, close. It's 1.6, I don't think there are any other computers running in this number, so it's 1.6, and over 100,000 Kindle devices. So, back to more serious topics, configuring syslogang. So, my initial advice is don't panic. Configuring syslogang looks difficult task at first, but it's really simple. It has a PyPy model where there are many different building blocks, like sources, where you collect messages, destinations, where you store messages, filters, parsers, and so on. And these building blocks are connected together using log statements. Here you can see a quick example. Oops. It starts with a version number. You can include extra other configurations. You can comment your configuration, define some global options, so you don't have to specify each time how many lines to write together, but configure it once, and you can override it later on. Then here are the different building blocks I mentioned, a source, a destination, a filter, these define varlog messages, and you connect all of these together in log statements, the source, the filter, and the destination. So, how Python is coming here? The Python bindings always have two parts, a configuration part and the code part. Usually the configuration part has just a single mandatory option, the class name, what you call from CystoGangy, but you can also pass many parameters and also pass parameters to your Python code, so you don't have to hard code everything into Python, but have generic Python code, which you parameter from your CystoGangy configuration. The code itself can be stored in the CystoGangy configuration, in a Python block, or stored in external files. Like you can put it just anywhere in a script and call it, or create the proper Python module and use that. Which Python versions are supported? It really depends on, at the time, compiling CystoGangy, so it supports one version at a time, it can be compiled with 2.7 or 3.4 or later. In any packages I maintain, I compile it with Python 3. Let's start with the Python destination, even if it's logically at the end of the chain, but this was the first one implemented within CystoGangy. On the Python side, the only method you have to implement is send, which is actually sending your message to somewhere, and you can pass name-value pairs from CystoGangy to Python in 2 ways. If you don't define what to send to Python in the configuration, then there is an object containing all of the different name-value pairs from CystoGangy, where you can define what values to send to the destination, and in that case, all those name-value pairs are in a dict. There are many optional possibilities in configuring the Python destination. You can define things like this buffer, so no messages are lost if the Python destination is busy, and if you want to write real robust code, then you don't use only a single send method, but do proper initialization, then you need which are called when CystoGangy started or reloaded, and use open and close, which are called from CystoGangy when CystoGangy started, reloaded, or when sending the message failed. Here is a very simple Python destination. It's practically a five-destination, re-implemented in Python. As you can see, here is just a single class option in the Python destination, the same as here in the Python code. We have a log statement, which connects the default source to the Python destination, and here is the Python code inline. The CystoGangy configuration, as you can see, the class name is the same as defined up there, and we have a single send method, which has all of the code, the filename hardcoded, picking a single name value per through the message object, writing it into the disk, and closing the file. Of course, if you want to do something productionally, then you better use all of the optional methods I mentioned in the previous slide, but this also works, and good something to get started with. The next one is the Python parser, where in the Python code only the parse method is mandatory, and here name value pairs are available only through an object, and here you can create new name value pairs, using this object. In the sample code on the next slide, I will parse this log message. This is generated by logCan. That's a tool for CystoGangy for testing and benchmarking, and it has this beautiful message format. Here you can see the configuration part for the Python parser. Here is the parser defined, as you can see the class name is here, and here we also have an optional thing. The regular expression sparsing the message on the previous slide is passed here as an object, passed here as an option to the Python code, and here is a log statement, which starts with a network collector, the Python parser and the destination file, where we also use a message template with the name value pairs extracted from the previous log message by using the Python code. And here is the actual Python code using this, importing the regular expression library and the class we defined also in the configuration. Here we also use the init method, where we initialize the regular expression parser and also start the counter. And showing this as the Python code is started together with Cystogenji and it's running constantly as long as Cystogenji is running or reloading. So you can create variables, create a counter like this, and you can refer to it on later log messages. So if Python is not started each time a new message arrived. And here is the code part. The init method is usually empty, but I put it here, and here is the parser part. We pick the message from the log message object, run the regular expression parser in it, create new name value pairs to the log message object, also create a counter name value pair and increase the counter here. Finally, we also have a Python source. This was released just recently, probably two months ago or three. It has very few optional configuration possibilities like time zone handling, name value pairs are handled here also through an object, and it has two modes. You can write a complete server application for receiving, collecting messages, or you can use it as a fetcher and it handles the event loop. In my example, I used the server part, and the server has two mandatory methods, the run method and the request exit. The later one is necessary to quit from our own event loop. Here is the configuration part of the Python, a simple Python source. Here we define the request name and pass a few dummy options, file destination and log statement which combines the two together of the Python source and the destination file. And here is the actual Python code for the Python source. This source simply generates log messages like crazy as long as syslogang is running, so it's an easy way to fill up your hard drive quickly. I tested just for numbers, syslogang, on my laptop syslogang can handle about 900,000 messages, and with this Python source it handled about 300,000 messages. So it's still quite a nice number. To use the Python source you need to import a few classes from syslogang and the class needs to descend it from the log source class. In the initialization part we just simply print the options I defined and set the exit variable to false. This is used by the request exit method and here in the event loop of my Python code to exit once syslogang is reloaded or stopped. And here in this part we generate log messages just a simple text message and send it to syslogang. And here is a preview which was merged just recently into syslogang and will be available in the next release. This way debugging Python code in syslogang will be easier. As by using the logout method of syslogang you can create log messages from the Python code for syslogang appearing on the internal source so you can follow what's going on in your Python code as log message within syslogang or if you start it in the foreground then these messages are printed on your terminal window. These were quite simple examples but there are more complex examples on my blog, an MQTT destination, a few parsers and then HTTP source you can check these out for more complex examples. If you want to learn more about syslogang and try it out either just syslogang or the Python bindings you can find more information on syslogang.org The source code is on GitHub and we also have an issue tracker there if you run into any problems and you can ask questions on our mailing list or on jitter and we have probably two minutes for questions if you have any. What I've mentioned here is a collector so it can pull messages from there I guess if it has an API it's not yet implemented but it shouldn't be difficult to implement it. It's open source and actually the earlier Python bindings are already in distributions but if it's too early syslogang in the distros then there are third-party package repositories and with Python bindings enabled so you can test it. Any questions? Ok, thank you.