 Good morning. I am Junichi Wakawa and welcome to everyone who has there to wake up in the morning after the long sound session and come back to talk on shared libraries. Usually I am a Debian developer who maintains audio-related packages and I'm interested in that kind of thing but also I've deviated from there to QA and other things including pBuilder and library packaging guide. I would like this both session to be where I kind of finalize the shared library packages guide which I have been playing with like for the last three years and I would like to present you as tutorial on shared library packaging. Well shared library packaging has been for the last century it was a cabal mastery where only the cabals could do and a newbie developer could not ever do any shared library packaging because you had something very cable and that's going to change with the shared library packaging guide which emerged in this century and we have a century of liberated shared library packaging. Okay so who I am I'm mainly doing audio and distributed computing and Debian quality maintenance and some localization Japanese and I am not a hardcore shared library package maintenanceer but I do maintain a few shared library packages and I hope that and I have been picking up on shared library topics for like last three years on things that were discussed in Debian Devel and I would like to convey to you the results. Okay so that was the last century it was the cable forging shared libraries there are people who have just randomly building packages and they said when asked how do you create shared libraries master I would just say hey you're too young just go on something else there and in the past shared library packages were very much named in an ad hoc basis there were there was not much very very much rules in naming them they were most were named leave something zero without any regards to whether the surname was zero or something and that kind of had to change when with the introduction of GNOME on GTK where they had to change the surname so often and they made the surname so long and complex that we had to have some kind of system to maintain them okay and I would like to talk about the background knowledge I hope everyone knows this by now but source code is built by GCC how many people knew that great okay so we are on the same ground and GCC outputs some kind of assembly and gas processes to make an object an object is processed by LD to create an executable file and in this we have this operating system called Debian GNU Linux which supports shared libraries and did you know that great okay so we have an executable but that is not actually complete we need the shared libraries linked to the executable to be run at runtime so what happens is that there's the funky thing called ld.so which collects all the shared libraries when executable is started and it kind of relocates everything and you have a complete executable file and you have a binary running so the problem and background of the problem is that you don't really know what shared libraries are linked to and what kind of symbols are loaded until you actually run the program so that's the lovely thing about shared libraries okay and I would like to mention a little bit about shared library package guide and its history it was started in 2002 as a response to go back you're too young to share package shared libraries and well there were many problems and I was watching Debian box this mailing list and there were many problems happening like missing symbols in shared library who this binary crashes when shared library package operates that kind of problems and I collected references to the bug reports and kind of categorized them into a list and added comments on how to avoid the situation and that's how shared library packaging guide evolved into the current shape and it has been going on for the last three years because there are things which change and I hope it's being useful to few people how many people actually read shared library packaging guide oh great thank you how many people actually used it okay nice how many people actually hate it oh okay that's a trick question okay and in the three years that with the shared library package guide there are some events like we had a large larger scale introduction of virgin symbols into Debian. Virgin symbols wasn't very much used at the beginning it was only probably only used in glibc and I think it's coming mainline that debugging symbols in separate objects is become mainline I don't know and people introduce pre-linking so there are a few things that change while library packaging guide is being evolving and it's not actually catching up yet and the problems with shared libraries can be summarized as keeping a stable API and using dl open and the implications of dynamically loading shared libraries and coexisting multiple versions because when you have multiple versions of shared library you have to know which version to load and which version actually works with the shared library and I think the remaining things that needs to be done with the library packaging guide is to the there needs to be some policy documentation so reflect things back to policy on how to name development packages uh how many people know that it's actually you don't really have really have a rule for naming development library packages because there there actually isn't set rule and you don't really have a guideline except for the library packaging guide and it would be nice if there was a dev helper like two for shared libraries how many people is maintaining shared library and thinking think that dev helper does not do enough work for me okay because dev helper doesn't really do too much work it it does some but not everything and I hacked a tool called dshulips which does more work but it's not going into dev helper yet or maybe never and currently how many people is maintaining shared library using version symbols okay how many don't you think that you have to do too much typing and updating yeah and there's too many to do so I think because it's something routine routine work you have you have to do every time so it should be simplified and that's the topics if there are any questions that regarding to the topics I would like to jump into a tutorial on making a shared library because I don't think there are many tutorials on how to make shared libraries using auto-conform to make unliberal and this is a very short tutorial and if you want to make a shared library it's very nice and I will distribute table of this example code it's very small so it's you'll be nice to see if you have configure.ac start with these components if you're doing auto-conform you have to start with the ac init and well you probably want some of them the automate initialization and you will need to add the ac probe lib2 to add check for lib2 and that's it you have some something's missing from the below but that's generated by default so it's not much problem and if you have makefile.am you have libaut libraries which is a libshared.la you specify the name of the lib2 archive file which is in the extension of .la and that will by lib2 that's the name you give to lib2 and lib2 will create the .so files and .a files and if you specify to automate that libaut libraries is libshared.la that means automate will try to make libshared.la and you specify the flags you give to the automate on the second line libshared.la fd flags and you specify the versioning information the obvious and basic version you can start off with is 0 colon 0 colon 0 and that will create a share library with .so.0.0.0 and you probably specify the include directories and you probably have a header file to specify as your prototype for the library and you have the libshared sources list which is the list of sources for the share library how many people are used to automate okay only about half so you'll be useful to say that when it says libshared you can replace with your share library name it's not something stuck and you have this make file .am and you may run automate and autoconf and you make a configure script and make files there are a few things that are controversial in this make file and that is the use of release and version info release and version info makes different kinds of soul names and version info is more of a conservative one and release gives a very different approach that it makes a different soul name for every release you give and if you're doing some very new project it might be easier to just use release because soul name grows so often and you really don't bother with compatibility and for version info the first number is the most important so be careful to change it when you're actually changing the share library interface and when you have the source code you can have the source code write your functions which are going to be shared as a share library on the header file should define the declarations okay any questions on the autoconf automate that was all I have for the autoconf and automate upstream source code yes just a nitpick really your example of autoconf .ac file it uses the old syntax it works perfectly but there's a newer syntax that works a bit better but oh yes just that it is a new syntax and I want you to know the new syntax rather than the old syntax I'll talk to you later then okay yeah I'll repeat if you thanks for the tutorial that I would have liked to have had it when I started building shared like and maintaining shared libraries and at least now people are not going into the same try to find things that I went through and a question about using dash release I've never seen it so I'm interested especially because I maintain very fast changing libraries but if that hooks the soul name to the version of the library doesn't doesn't that mean that the library binary package has to go through new every time I upload a new version yes it does mean that okay and that's the problem people try to avoid it by not changing the package name but that's not really a good solution so we actually have a problem oh a problem I really care about I stopped making share libraries because of this problem and I do all static and it's a nightmare anyway yes static is currently a better solution because we have a overhead at ftp master for changing package names so it's not that's a problem with the debian infrastructure and how we organize things so the thing is that for new emerging libraries which changes interfaces one recommended action is to use a release option and change the soul name every release so that you would change the package binary binary package name and require every package to be rebuilt against the new shared library but that means that it will it is more work for ftp master and that is one problem I'm seeing us and I'm not sure how to solve that yes and so Brian has suggested there was a tool called i-check by usfield nice and that can be useful a bhc yes the question was that if I could explain the three numbers in the versioning and they have some meaning but I don't really much care about them so I don't really know the first one is some major version of something and the other is a age and something to specify that you have compatible interfaces for several versions but I'm not quite sure if you're really able to maintain that kind of detailed information if you're striving to work out what kind of a bi is stable or not so it's not really practical or useful in my opinion it's too difficult yes someone was doing a really neat work utility that detected all obey changes automatically so we could use that with an automated degree of certainly the first name is the the first number is the full abi number the second is the minor revision and the third is the last time you broke the the abi it subtracts them in a weird way and generates the user increase only when things break backwards numbers we are using for ld.so it works really well after you read about 20 pages of money for about how to use that okay so that was a question answered okay so I've discovered that there is a abi checker and that's nice to hear oh it's not finished so it's not unfinished abi checker which could appear very soon okay okay I will explain about these slips to which I have been I have written as an example implementation of what is written in shared library packaging guide I think it's used by a few people not many but in it breaks sometimes but it's mostly working the goal of these show lips is to facilitate the use of the output of object dump on a binary the elf headers of the binary the of the shared library has enough information so that we can extract the debian control the required information for debian control to be extracted from a shared library and really it should be automated in that in such a way that uh debian control doesn't have to be sort of uh synchronized by hand to what the shared library shared library itself is so it's one way of going and what it does is that it looks at the s o file and with a set magic it creates a shared library package name and the development package name and checks for debian control so that the package names match and it also generates a list of from the list in the needed section of the shared library package you can find that the shared library package which this needs and create construct a list of development packages it should probably depend on and creates a debilits depends just like the show lips depends um it's been maintained for three years and can i advertise it now and the guts of the debilits these show lips is that you have a you check the debian control is matching what it says on the dot s o file and it will also do the moving around the files which you will usually do manually in your dh install or paper scripts so and it will create the list of dependencies on debilits depends so the development library dependency which you might be doing manually is somewhat automated there are weaknesses in dish lips in that it depends on the shared library packages and development library package being named in a consistent manner so it will be better if we had a policy on how to name them okay um is there any question that dish lips for all of the shared library packages that i've had anything to do with over the years which is mostly exclusively stuff related to the x window system i've gotten into the habit after what ben collins did with g-libc of adding a dash dbg or debug version of the shared library which is unstripped and placed in the in user lib debug does dish lips have support for helping people do that automatically as well um actually it doesn't have support for that currently and i'm working on it and i'm working on the the split debug symbols oh well yeah we should yeah we should be able to have those with gdb now shouldn't we yes so the nature of debug debug library packages will change yes and i was thinking of because debug packages different packages i i was thinking we might be better just dump it into dev packages because it will create too many packages for shared library yes i know the archive maintainers are concerned about what would happen if everybody shipped an unstripped debugging shared library so they haven't given me too much trouble but but i mean you know if everybody did it uh you know gnomon kde things would start to become pretty large so do you know what the status of the split debugging symbol support is generally no i have no idea does anyone know about the state status that's that's the kind of thing i think we should you know try to get moving on early in the edge cycle so that we can have a chance of having something you know complete by the time we release no okay okay thank you okay yep what's the merit of using object dump instead of ldd um as opposed to ldd object dump gives you the s o name field which i want and for the needed things instead of ldd i'm using object dump because it will miss the indirect dependencies and i don't want everything to be there but ldd print out the uh the exact path name for the current build environment so i think a deep package shall leave devs using uses uh ldd so it's nice way how about this okay so um the idea is to use ldd for the full path and find the package names which contain the full path but the actually what they point to is the name of the runtime shared library packages not the development packages so we don't the main problem is that we don't really know which development packages is connected to the shared library packages they may be created from the same source package but we don't really know and we don't have a naming rule which is enforced to mechanically determine so that's one problem and it's not improved by using ldd over object dump yes well for consistency's sake shouldn't we have that anyway we should try and have you know a simple rule that somebody can find the name of the the dev package i i don't think there's any good reason for not yeah having the dev package have a different name to the to the library unfortunately i think most people do this but we have one particular legacy package which gives us a bit of trouble and that's called uh libc because it's named different things on different architectures and uh it it can be kind of tricky to deal with well i think you have it would be a good time after more than 10 years of devian development you might be a time to agree on the name of the development library packages we've been going on for more than 10 years yeah maybe time yeah um yeah i want to talk about version symbols uh you have to write a script like that and maintain it yourself uh it's not too bad that uh do you really have to do it yeah the comment was that uh okay well uh official support on lib2 would help sending things upstream and version symbols is the kind of thing you want to be equal everywhere so upstream is the really the real place to have it done i had upstream often widely used uh libraries just as sheels as sheels s a s l tell me that they would accept version symbols upstream only if fliptoe supported it suddenly okay but i really would like if it's there's like a tool i mean it looks like we can really automate it generating this script so might be nice to just hack on some script well i could from looking at that now i could imagine about 10 line shell script to create it so it should be simple yeah okay um yeah we had problem with the gnome had initially had problems with having multiple development versions of shell libraries and having they solved the problem with the having pkg config scripts to find the path to the exact version of the shell libraries how many people actually have the mastery of writing pkg config scripts okay not many it should be simple looking at the examples you can find in userlib package config.pc but i don't really know how to write them so it'd be very nice if you could sort of give me some clue i can include in library packaging like that might help on some pointer so another thing i would like to note of as in shell libraries that the internationalization is slightly different because you need to use the get text instead of get text and that's because uh you're a shell library you can be called from any program and that any program may set any domain when you're calling so you have to set your own domain when you're calling your calling get text to get the text how many people actually don't have a clue what get text is okay don't okay so please please it would be a very much of an advantage for everyone if you learn to use get text i tried to test get text and get text dies but yesterday it didn't work very well so i'm not sure as a future of my work i'm considering of and still maintain continue to maintain share library packaging guide maybe add it to the debian doc and it is written in debi doc book xml and it is a guide but it is also a torture test for xml two chains like xml editors and yes and if you have any problem please let me know so that i can include it to share library packaging guide it's getting like a nice portal for people working with share library so we can get information there it's kind of central information and as with debian it's very big it's very there's everything's open everyone communicates but we don't know how to get to the information so i i like the way that share library packaging guide is somehow well i'm doing it but it came on like this and if you have the motivation it would be very nice if you could do it for your own pet projects so that people will benefit from it i mean debian yes i had a comment yes first of all i found me the packaging the library packaging guide to be very useful and enlightening and i was wondering if to increase visibility why not include it as part of like the new maintenance guide or some other you know documentation i think we have too many little pieces of documentation everywhere you know that are hard to find sometimes so it would be better to like have like one maybe debian developers guide or something that covers everything yeah or the developers reference to you know some more visible location okay um i think it's already referred to in developers reference so it's it's good enough yeah i was about to make the same comment in a different way i was about to ask you yes is lead packaging guide easy to translate uh do you consider converting into p o format um you know xml is not easy to translate i consider the reason i'm not i've not translated into japanese yet is that i don't have a good working in japanese toolchain to translate it uh to get japanese text into pdf and other kind of thing so and it's a usual format like dog book uh like debian dog so it's same and for p o format i don't like working with p o formats with a large documentation because they kind of reorder things yeah so some translators also don't use a p o format for xml translation but it may help to to maintain actually more easily but possible for a call for translation and make it more visible actually it's a part of making it more visible if you include it in developers reference it will be more visible and it will be translated actually so it's about the same comment then okay or at least try to get it into the debian dog uh infrastructure which is already available actually there's already a directory in debian dog infrastructure there's no make file because i haven't figured out how to build it i haven't got around to it yet i guess um if you're going to visit franc's talk uh which is going to happen as next um about the website uh round table it might be possible to discuss this sort of problem there okay that's great thank you any further questions okay okay um so the time is up almost almost up okay um thank you for hearing this talk um i would really like this kind of FAQ kind of policy kind of document to be more maintained in debian it's very useful everywhere so everyone hack on but write documents also please thank you thank you very much