 Okay, again, welcome to this 10 o'clock session on this Wednesday with us is Markus Zappke-Gründemann, active member of the jungle community in Germany, and he will tell us something about multi-language documentation in Swings. Yeah, morning everyone. I hope everyone can hear me. Yeah, okay, great. Yeah, so nice you all came to hear the talk. Hope we can tell you something new. Yeah, yeah, as Fabian already introduced, I'm member of the German jungle community and Yeah, I'm doing software development since now nearly 15 years, started with Pearl, then doing some PHP now mostly I'm doing Python jungle stuff and also involved with the open data community in Germany and I also do a lot of training with people, basically Django training and I have a company called Transcode and I'm a board member of the German jungle Association, so if you have anything about Django in German, you can also talk to me And you can find me on the internet on a kind link day or at kind link on Twitter Yeah, so some basics first Short introduction of Swings. So who is already using Swings here in the room? Okay, so nearly half of the people. Okay, so introduction is good. Yeah So Swings is a Python documentation generator, which means it is written in Python It's not only for Python. You can use Swings for any project you want to. So Yeah, any language you want to you can even document stuff which has nothing to do with programming at all if you want to and The markup language is called restructured text Which is something which was if I remember correctly was invented by the Python community It is something comparable to markdown which many people know Already and The interesting thing is that you have a lot of output formats So you buy your restructured text documents and you can create a lot of output formats as you can see here like HTML Latech and of course PDF EPUB text in for manual pages plain text. So you also have the opportunity to Display and your documentation properly on different devices. And yeah, Swings can be found on Swings. That's dr Rock also the Swings documentation Yeah, and of course as the title of the talks says I'm talking about Internationalization, so it's often referred to as I 18 and so because it has 18 letters and first that is I last that is And so this is often the abbreviation for internationalization, especially if you look at module names in programming and The idea about I 18 n is that you Can translate These strings inside your software without having to change your software all the time Because I mean if you if you will have your software and inside all the different Texts in the different languages and you always have to switch there in between while writing your regular code This would really be a mess and of course you also you need a like transparent system to exchange the Messages on the language of the messages the people see that use your software and The most or the best maybe best known tool for that is a new get text, which is open source software And this is also used in Swings to create the Transations So in a simple get text example looks like this There was a bit missing on the left side, but Yeah, I hope that's not that critical. Yeah, so It's simply that you create a translation catalogue object This example the catalogue is called example and it's in a directory called locale and Foreback through is simply For Changing the way of exception you get if it's not existing so it's not important at the moment and So what what is missing here on the left side of the screen is that there is an underscore there So underscore equals to get text. This is usually done everywhere so underscore is used as a alias for the get text function so that you have to write less a Code for the internationalization and then you can see this print statement always look on the bright side of life and and you can see that the string that is inside the print statement or function caller is Again inside this function call which uses this underscore, which is an ad as of tea get text and this way you and Collect this string for translation So your source language would be English in this case and you would have English as the source of your translations And then you can create inside this locale directory Different example catalogs for each language we translate this English string to something else so this is how Get text basically works you collect the source strings and then in the end you translate them to You're like target language Of course this is what happened it's happening behind the scenes what the swing is doing for you So you don't have to do anything like that if you want to translate your documentation just for Short explanation of technical background Yeah, but but why use get text for translated documentation Why not simply copying all my fights to a new directory like from EN to I don't know DE or GP and Simply translating all the text documents again bits of documentation again, and then I have the documentation in a different language Yeah, so first problem is and if the documentation is updated in the regular language So the instrumentation is updated and you have a hard time on Finding which stuff has been changed because you can't easily differ between them because all your content is different Because your content is a different language Especially if you if you have a language like Japanese which has totally different characters like Latin languages Second problem then is if you leave something out Because maybe you don't have the manpower though more to translate it at the time or you don't know what is the proper translation for For this thing Then it's simply gone or you have to copy and paste this paragraph to your documentation in English and remember that you Have to translate it and if you use get text Get X where we place all strings that haven't been translated Automatically with the original string. So if you don't translate a single string or paragraph And you build your translated documentation you will still see the original string Another thing or advantage is that your markup is not obligated because if you think of a document With any markup like with such a text or marked on whatever HTML and you copy this whole document And you translate it you copy all the markup with it and if the markup is also changed because the documentation evolves and The way it is displayed is also Changed with the next release of the documentation. You have to do this as well in your translation But you only wanted to do translation and markup. So you again have another problem and Another problem of yeah, not using tools like get text is that you exclude people because For this usually you have to use tools like git or mql which have to yeah And they have to understand how all of this works. Maybe do a pull request then somewhere to get your translation merged in and so forth and Yeah, sometimes people do translations I'm not familiar with this enough to get And involved in this process and so if you can use professional translation tools The is much more easier for the translators to translate something for your project and these Professional translation tools also offer you opportunities you normally don't have with regular text editors like These tools can for example show you oh It seems like that This paragraph has been translated before and here I have a text which is 90% the same the translation 90% the same like Similar to the text that has there been before so you can simply then get these 90% similar translation from a translation cache Put it in and simply change a few words. So also your translation of already existing and evolving a documentation is faster Okay, yeah, so how does it all work with swings now what I explained and this is an illustration from the swings documentation and Yeah, and this shows how Documentation or internationalized documentation is built in swings. So you see on the far left the RST files Which are and then built into PoT files, which are um Templates so Po is the extension for the source files of get text and PoT is in a template for that So it's the source language and then you translate these PoT files here to like put list mentioned But there are also other tools to a Po file And the Po file is then again I'm Transferred into oh, yeah Compiled into a mo file and the mo file is a binary version of your Po file Which is faster on access so you can can get all these translation strings faster and in the end then The RST files are taken again together with the MO files And then the all the original strings are replaced by the translator strings and you have your Translated documentation if you want to do this and you have to do a few things first thing is If you haven't done this in your so usually you have a doc directory inside the doc directory You have a conf file and swings where you configure how swings works and First thing you have to do you have to say which is my source language like the main language of my documentation in this case it's English and You have to say okay, where do all my Translations go like with the other catalogs with the different other languages and in this case it's the directory locale and then there's also a fairly new configuration value getx compact and if you set this to true and your documentation has several subfolders or Documents in a subfolder are collected into a single catalog if you set this to forwards you get a catalog for each file So because if you have a documentation which consists of 50 different files and five different folders You may only want to have five catalogs instead of 50 because it makes translation easier So we have different opportunities here So to do the I'm Translation in a easier way there is an extension or plug-in for swings called swings INTL and Yeah, you can install this via pip so that it is available in addition to swings itself and This will then help you to deal with all the translation files because it is possible to use swings itself and some get-txt tools to create the PO files and MO files and all the stuff, but this is a very nasty way and Also complicated way of doing this and swings INTL makes this all easier so Yeah, of swings INTL is installed first thing you do is make get-txt which are Yeah, chords the built-in swings command to create all these POT files So it extracts all the strings from the RST files and put them into these catalogs and then You say Swings INTL update minus L which is the language. So the new language I want like German and where I should Where am I? Locate and where I may be POT files like this build locale directory So they are make get-txt stores them in this directory and so these swings INTL update Command will fetch all these POT files make get-txt created and create new German Catalogs for that of course, it will be empty because This command is only like copying them and preparing them so you can insert the German translation strings So the next thing you have to do is translate the documentation I will talk in a moment about the details for that and then you After you translated everything you can say swings INTL build and then it will create all these MO files that had been shown the image before and then in the end you can again say say Make HTML which creates for example the HTML a documentation you can also say make PDF Lartech or something else make EPUB and With the swings opts Options you can say that the language should be something different. So you override the language in the conf py file to create a German HTML instead of an English one because English is your default language and now you want something different So this is the four commands you need to You create the catalogs and then translate them and build a different Documentation language Yeah, so what you can use for translation one two, of course there are also others, but what is very Helpful at least to me is trans effects so trans effects is a web service, so It's not a software you download. It's more like a website you use and It's free for open source projects But if you use commercial projects you have to pay them of course because yeah, they also need to have some income and Trans effects really works like a professional translation tool like which features I said like show me similar strings I had before and all the other things and it's also very Nice tool because you can collaborate with many other people on the same Translation and people can work together to translate all the stuff online It has trans effects as a also a command line tool trans fixed line to pip install it you say TX in it and to username and password so we said you can that the Transfects line can talk to the platform and Then you can use Sphinx INTL together with trans effects because Sphinx INTL has trans effects support and as you can see are You first run this update TX. So TX is the abbreviation for trans effects update TX config resources which Yeah prepares Trans effects so that it can be used together with your Sphinx documentation Normally you have to do a lot of steps here manually and Sphinx INTL is doing everything for you here and Then you can say TX push minus S And this is a little bit similar to like git or mercurial. So you push Your stuff to trans effects and minus S means sources So I push my English catalogs to trans effects so that other people then can create translations and translated to German Spanish Japanese whatever If translations has been have been translated on trans effects I Can pull them again with TX pull and I can explicitly say which languages I want for example here again I want to put the German translation Some people created on trans effects and then and again say okay like before Sphinx INTL built Sphinx make HTML for German And now I have my Like German full translation and can see everything in German. Yeah, so this was like the yeah the workflow you usually use and During the years, I'm already working with Sphinx INTN I discovered a few things that are helpful in the everyday work was internationalized documentation So if you Use code inside your documentation here's a snippet of template code for example, you should be Should be careful and always use English inside these code examples because otherwise If I would for example use German here and something would be translated to Japanese The Japanese people would have a hard time understanding what the content of the template really is about There have been requests on the Sphinx mailing list to also translate code But this hasn't been decided on how to deal with that And if this is possible because at the moment only like the The the written text is extracted from the Sphinx document and not the code of course Another thing is there are different ways to handle URLs and RST you can use the Sphinx above which Puts you are directly into the text and you can use the Sphinx below where an alias is used If you use the alias version UL gets lost because it is not longer part of the catalogue and especially if this sentence is translated the name Python sources would be translated to Python query code or something similar in another language And so the name of the reference is lost and the URL can't be inserted So always put the URL inside your Documentation so that translate a CZ URL and can translate this properly um Yeah, other thing is you often want to refer to external URLs and Often these URLs are referred to referring to documentation, which is versioned like you see it's referring to Django documentation for 1.5 and if you then Switch your documentation to refer to what Django 1.6 you may want to I don't want to go to all of your documentation and change it And especially not all the translators want to go over all of their translated stuff to simply update the version number And they you can use a feature of Sphinx So the thing about the dictionary X links is something from the Sphinx configuration And they you define these aliases and then as you can see below in your text You can simply use these aliases as prefixes for then the rest of the URL And so you like magically can construct these URLs and only have a single point where you define the prefix on the left side Yeah, and how to handle special cases There is a if config construction in Sphinx we always can check against the current language and so you can have an every documentation like special cases For example, here's a link to the German community And of course you want to have this part of the documentation only in the German language and not on the other languages So you can always have like some kind of special part of your documentation only for a specific language last tip is how to Run link check stuff for example There is a link check target and also the link check target and many other targets except these language switch and link check simply goes over all your documentation and checks the URLs and Of course you want to check URL for every language. And so this is also possible with all the different languages Yeah, so what is still missing? For example a translation setting is missing so that you can build or translation with one command at the moment you have to Execute all the steps for every single language and if you have a documentation of 500 languages is a lot of work You have to do and so of course you want a translation settings Like to see language with your original language and translations for other languages and you can simply build all of them together and oh another thing that is missing is a landing page for the HTML version because Yeah, you need something like a page where you refer to all the different languages if someone Comes to your page with our HTML documentation is Another optimization could be you do use getx compact to create a single catalog and instead of many catalogs and The rest is not that important. I think we use rest of the time for at least a few questions before time is up So thanks for listening and please ask a few questions Okay, Q&A. There's one microphone over there. Yeah Hi, thanks for the talk So if you have several source files and documentations and you have choose to have several catalog files But then you have small words That occur In every file do you have to translate them in every catalog file or are they shared within one sphinx? Oh, yeah. Yeah, I Was thought about that maybe excluding an example slide where I show how this catalog looks like so we should do that Yeah, so I'm with swings the The always a full paragraph is extracted. So you every paragraph is a single message So you never have single words that can be translated. This is only true for the index So in swings you can have an index of keywords Where you then which is in the end of the book which can link them to different parts of documentation And these index words are translated separately But all the other stuff is simply translated per paragraph I hope this was your question Okay Yeah, any other questions. I mean, yeah So I'm available all the day here and I'm here to Saturday and on the sprint on Saturday We will also have a sprint on swings So if you have any ideas on how to optimize stuff or I don't know have any answer questions You can always come to us on Saturday and talk to us Yeah, about improving swings would be happy to hear from you Yeah, okay. Thank you very much and