 Atomic? Atomic? How do you mean atomic? No, it's the atomic. Atomic? Not the atomic. Do they have atomic stickers today? Yes, yes, yes. Which commune are you working for? Which commune are you working for? The biggest, you know. So... So you're your ownership? No. We're not actually... Ah, okay. So... Let's start. I'll introduce you. Hello. Can everyone hear me at the back? Okay. Welcome to the systems engineering track at DevCon US. Thank you for being here. Before we get started, I have a couple of announcements to make. The first one is that we are having a party in the evening. It'll be at the beach, at the BU beach. So if you wish to attend, you'll need to pick up your ticket from the registration desk. And the second thing is that we have the Red Hat CTO speaking tomorrow at the DevCon keynote. That's at 9.30 in the morning in the big auditorium. The one that you see from the window here. So please attend that. And now let's get started. The next talk is lessons learned by migrating Python 2 projects to Python 3 by Jun Fu. And I'll hand it over to him. Hi. So... Sung, you're all being with me. So my name is Jun Fu Wen from the Red Hat Company, located in Beijing. So today I want to present one topic lessons learned by migrating Python 2 projects to Python 3. Because in the recent years we are working on projects migrating so we want to show some experience about that. So here's the agenda today. So first thing I will have a very simple introduction about our current project is our Cardo.VT and the Tiffany Award. Secondly, I will show some protein strategy. Third, because we have a lot of other tools to send easy stuff for you. So I will introduce some of them and nonetheless we are to use them. Finally, during the migration we meet a lot of common time-on issues. So I want to show some experience about it. So the project, our Cardo.VT will execute the organization the data space. Its main purpose is to serve as an automated independent testing framework for the world development force. So the Tiffany Award project is the perceived to send organization functions such as LibWord, LibGaffers, and LVSP for some kind of things. So all these projects are in person 2007 before the migration. Actually, this project is a dependency with the other, so we just put in both of them together. So we are aware of the reasons why we need the motivation. The second one of mine is because on presence link we have, for example, we have faster performance than present tool. Let's have a synthetic indicating light on presence 3.6 it can serve in CPU 10% and the memory footprint can save 30%. Also on presence link there are a bunch of backfists especially for some 6 CVE fix and also there are nearly newly introduced advanced features such as single IO and also on presence link there are better type definition. And the first one is the unified long int integer and the integer. So its privacy cannot persuade you to migrate to presence 3. I think the last one is the concrete reason because at the end of in the end of 2012 2010-20 the present tool will be end of life. So motivation strategy roughly there are 3 of them to convert. That means that you convert your whole source code from presence 2 to presence 3 just completely stick it back to presence 2. So this is the dramatic strategy and the second is just create separate branches for presence 2 and presence 3 that means that there are 2 set of code one is a speak to presence 2 and the other is speak to presence 3. So if you have one feature or one new feature you should put the code back to 2 different branch so it's your W maintenance effort. The third is the code is this that means that you only use one the same set of code base and it can support presence 2 and 3 for all projects because there are busy needs so we just use the numbers with strategy. So specific for our projects this project we need to put in all the modules and test case to presence 3 is the not less than 3.6 version and after putting we needed to keep the compatibility with presence 2.7. So all code change we just push it to the mass branch. So at the beginning we are trying to create a separate branch to check in the porting effort but later we realize there are huge gap between the porting branch and the mass branch so it will a lot of effort to multiply like multiply graph to the mass branch. The other reason is that we also have a sign new feature integrate to the mass branch so we just push all the we don't try to create new branch so we just push all the changes back to the mass branch and with all the code change need to be validates on both presence 2 or presence 3 environment. So we set up a sign we already are creating testing including unit testing or integrate testing use the check in the pipeline and my good example my good idea is to use some tools for example presence 2 have one tool to pilot with option it can help you to locate the scope of changes required. That means you are giving your tools to activate whatever you need to do the migration or do the porting so you can use do some planning how many efforts or how many resources you need to use during the migration so we use the continuous integration to divide the porting efforts into the three phases in the first one we adjust some very common issues that easy to fix for example as automated tools can be fixed and in the phase 2 we just fix the manual issues that automated tool cannot fix that and remove all some dependence the final phase phase 3 we just set up some to more troubleshooting so the main problem here we find some issues we just fix these issues and we just start with each part I just mentioned in the phase 2 we just try to fix some with the overview solution and we use increment integration that means we use the separate call request to check only one for six for all the issues so if we find some issues it is easier for us load back and just mentioned that we set up some continuous pipeline in all production so each day we monitor results and if we finish it we quickly fix that so the fundamental idea is we try to don't break anything when porting but it's ideal situation actually we have cut a few but because we finalize it earlier so it quickly fix that so because the magnesium is quite hard is time consuming and ever consuming so now since we should know that there are a lot of tools to design easy stuff for you so I just list some tools here and I will analyze where they should be fitting in so tool 3 is the first list lab that make share the code between the pattern 2 and pattern 3 and it's actually is the Python platform and it will do source to source transformation so it's first we will read pattern 2 code and apply a serious fixer to test them into the validate pattern 2 code tool 3 is have underline Japanese lab loading is lab tool 3 this lab is the kind of which set of fixers that will be kind almost of your code also my opinion is that flexible and generic lab learning so it is easier for you to write your own fixer such as visualization or modernization is just based on this package so you may wonder how to use that so you can by default to the 3d tool you have a long set of predict for the different fixer so your sign flag can be used for example dash f will be explicitly speak fine what kind of fix you need for example this command is my issue and the dictionary has key issue and I want to apply this function fix and I just apply this to the single pattern file and this command also can be apply order fix and also dash w flag can overwrite the changes back to your pattern file so this another tool is filterize filterize is customized to this script and it is dependent on the package this package is the compatibility layer between pattern 2 and pattern 3 and it allow you to use single clean pattern 2 code based to sports cross 2 and 3 so filterize is the try to make your code to pattern 3 so code style is pattern 3 but you provide some fix because when you code back to the pattern 3 some code may be not work on pattern 2 so they will apply some fix make them work on the pattern 2 so you can long filterize command in two states so cg1 is just your combination code to the pattern 3 and state 2 try to apply some fix because when you code back to pattern 3 they may not work in the pattern 2 so cg2 will apply some fix based on your current pattern 3 code style 6 6 is a last comparably double loading it's widely used in the open stack they also use the 6th module to try to make the compatibility and it's very simple it's very simple for wrapping all the difference between pattern 2 and pattern 3 so it's intended to sports code phase that work from first 2 and 3 so how you do that actually in the 6th they define a lot of classes functions, attributes and when the 6th, when you import a service module or package which will automatically detect what pattern environment you're longing and it will point to the specific class or functions or attributes under your pattern environment so and the 6th is only one pattern file so it's finally copied into your project so the final tool is the modernize the modernize is the harness the lap 2.3 it also is the script and it depends on more than this lap loading so it's very simple to use that is to make pattern 2 code more modern with intention of eventually pulling over to the pattern 3 so modernize is just made of your code to be modern but the code phase is still angled to pattern 2 so how can we use that it's very simple to use so QPL may wonder how can I choose the figurelize or modernize figurelize I just mentioned that the code phase is something like pattern 3 so modernize is just code phase is still pattern 2 so it depends on your project if you eventually want to migration your code to pattern 3 so I highly recommend you use the modernize tool because the code style is pattern 3 now the final part is common techniques issues we account during the migration so I just categorized those issues with several categories number one is the essential syntax it is called obviously and the center is bound means that on pattern 3 there are a lot of modules classes or function attributes are dropped and the center is in the name is that in pattern 3 there are quite a few modules just changing their module name to combine with final name conventions something is we will organize that there are a lot of standard levels to distribute to different modules to make it more easier or consistent to be maintained or to be used so to the developer most challenging part should be there are a lot of difference between the tags and the final name changes so then in the following I will adjust that so essential syntax difference we are print print and exact now it is the function instead of a statement there are slightly changes in the expression handling you should use the keywords now on pattern 3 module class attributes are removed but you still can use as the parameter in construction so this extension also have slightly changes you need to enclose your parameter in a length basis some complex functions such as map, filter, chip, knowledge and detail it instead of list in most cases it is the work but if you apply index on the because it is written in iterator so if you apply an index on the iterator it should be has some issues so after iterates there are small changes you need to put small letter O among them and on pattern 3 we only express it relatively important for example in this package there are two modules class 1, class 2 so you want to on pattern 2 instead of class 2 you want to import class 5 you just use this syntax but on pattern 3 you need to use this one it only supports the interpolated import on pattern 3 there are unified long integer and integer are the one so the surface we are not spot it so on pattern 3 is a spot real float division so it is a particular dangerous because we are not slow syntax and the goals are notice so just be careful for the division and for additionally iterated keys, iterated values and integer items as key what is function has to be removed is function to replace that is the solution make them compatible on both of Python and decision keys, decision values no different dynamic view and instead of list so most case it works but if you apply some index on the dynamic view it will restore ILO next in addition in pattern 2 we have machines next but on pattern 2 we needed to double underscore next so make it compatible you just needed to define Alice for this and then use the building function next for unicode it was in first rate default you store is unicode so this function is removed you can use this to make it compatible you can define it by yourself this label is removed you needed to from collection package import left and low import actually you need to input and on the pattern 2 this one is low import and not import so actually low import is the natural import and the import is removed so you can use the 6th movie import left X length is the natural length actually and the previous length is removed and apply the function also is removed you can use the function and the list star before the list parameter and file also removed use the time encoded to implement by yourself commands module removed you needed to use sub-process and stream module middle and functions in the store on pattern 2 but on the first rate they are totally removed so you needed to just apply the store building functions something is in the name so just mention as on first rate there are a lot of modules that you need to comply with the final conditions so this sign of the list on highlight so for example so how to make them compatible you can use the trial is a simple item to deal with this and there are more for example peak building is also have a similar solution for that something is in the organize just mention as some module are just the organize to expect some levels to make it more easily consistent views so each special in the url lab url lab 2 and url lab 2 parcel those modules are heavily organized and these modules have distributed to some on pattern 3 there is parent module is url lab and on this parent module there are some sub modules url lab dot url lab parcel url lab dot url request and url dot i-load so try to make them compatibility you need to use trial or expect import i-load this is not only way to you can use this movie package it will be 100% for you so it's just one alternative solution and it's all applied to htpsurl, htps parcel but i lost also the distributed to different packages in this way building in function load in turn load function to remove to different packages so try to use trial expect link i-load to let make link compatible so now we can use the last part is the test and find out the difference so it's what you might want as it's the most trained part because they could take a look at this table so for test stream on pattern 3 we use stern to represent the light and on pattern 2 we use unicode for the final data on pattern 3 we use bytes but on pattern 2 there are two, it's bytes and stern bytes are only the Alice of stern on pattern 2 so pattern 2 and pattern 3 confusing your stern to many differences for pattern 3 there means the test stream pattern 2 is the final stream so it's the huge difference so this kind of change actually brings on a lot of challenges so one lesson we learned in the last try to avoid with text and finally sequence so we have this stern here for example we match a lot of issues kind of those issues for example concat bytes to stern if you combine bytes and stern now we are slow the arrow on the pattern 3 concat uses stream pattern on bytes like objects in the likelihood of exertion so pattern is the stream and double x is the bytes, if you use that on pattern 3 it's a slow arrow but in pattern 2 it works to convert bytes object to the stern implicitly for example we want to replace the stern here with bytes so it will slow the arrow so during all migration we match a lot of issues so we just try to manifest that just remember you need to decode bytes and encode text so how can convert them bytes and text so you need to use encode functions or decode functions so you just if you want to make sure that you use the bytes you just use the decode function and you want to be bytes you use the encoding so don't do it in words since that those conversion your common use very heavily so it's a good practice just define them into your series module so it's style for example you want to text and this is your input parameter and this is the encoding so first you can draw to that whether it's it's the binary data if it's just decoding it's decoding default encoding coding and even though we just written that to bytes it's quite simple to use the previous one and the to send decoding so it's just some utility we use because there are some confusing use the style in the pattern 2 or 3 so also it's impact your API definition even your so just watch out be careful your API definition so you're needed to realize you want to accept the parameter in text or finally don't try to miss them because it's harder to keep code working if you want to accept first of the parameter in pattern 2 it works but obviously it doesn't so make sure that the API that can access text parameter can also work with the Unicode you know that of Python 2 if your API or function works on pattern 2 so probably you need to input send special Unicode code to validate that because on pattern 3 the default string is the Unicode there are much larger little details and those that work with finally must work with Python so on pattern 2 finally your API you just use Stir for that so Stir means that it could be text but finally so if it probably has some issues on pattern 3 so you need to just on pattern 3 you need to test with some bypass data and make sure that it works so just restrict in whether you're passing text or binary data that means that we recommend that you use the prefix that's B or U sounds like this the prefix to just distinguish your parameter also it's a way to make your code look really more human clearly let people know that not just your ambiguous Stir so there are some useful resources here so this is I think provided by a left hand they have summarized how to make a project in a conservative way not so dramatic also there are a lot for analysts for the industry new features so that's it so any questions okay thank you thank you so I have another minor announcement to make due to the weather forecast the location for the party has been changed it will be in the Ziscan lounge so where lunch is available that's where it will be so thank you that's a question