 What is it? It's okay. I see something. Next topic. Okay. 200,000. That's what I'm talking about. It's less T-Dev than P-O-Files. There should be multiple P-O-Files in one P-O-File. Yeah. It's only available in the same package. And they are the same look-out. And the P-O-Files are also included in my code. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. Yeah. All right. Alright. So P-O-Depth Consul can go into the T-Dev, too. Yeah. Yeah. So P-O-Depth Consul can be on to Dirt one of the things that Mark was talking about the other day, rather than having a t-dev for every language, was the concept of deep package filters to allow mDebian to only install or mJuzers to only install the languages they wanted and then use the t-dev purely as a way of updating or adding new translations. So you would only have a t-dev, you would hopefully only have a handful of t-devs which would be so the translation team could upload any language without having to do an NMU or a full package and the idea would be that those would ultimately go to upstream or otherwise incorporate it into the main package and then no longer be a t-dev but be supporting them in property which I believe gets around the NMU master worry about lots of packages but still gives translation teams what they're looking for. I'm trying to think what is the balance between new translations and updating existing translations and probably new translations isn't, it isn't fifty-fifty users, just to think, there's probably more work. It's probably more about updating translation, moreover we are not that encouraging people in Debian to update translation for upstream software. Because it should be done upstream, this is what I have ever pushed. So it's mostly about updating translation or adding translation for Debian native languages. So it means way less translation. This is what we would target. But if it happens, the t-devs, whatever way it happens, we would try to update or add translation mostly. But presumably there's the translations for DevConf as well. Which covers a lot of... I counted the other days, it's about six hundred fifty packages with the DevConf. But presumably the translations for DevConf are slightly tricky with the t-dev because often that's a package install time. But that can be easily done by modifying d-package or apt to install the t-dev before the package. And then DevConf is got the strings available when it actually comes to install the real package. That's a sequencing thing. You can do a pre-demo with the translation. Yeah, or you can just set the d-package, or just check to see it or apt to check to see it as a t-dev. Because the code I've written should do the updates. Try to make sure that is written in a way that could be put into apt. And the code I've written to actually generate the t-dev to try to make that based roughly on the Perl code that is using DevAlbum. So they should go that way if you like in time. But the t-devs, even if they were playing packages they should never have burst-inscripts or actual dependencies at that level. So they can always be pushed to the front of the power that's implemented. They can clearly be pushed to the front of the power without breaking anything. The main part is that t-dev should really have no impact on the whole dependency tree hopefully. Or it should impact as we can. They will only continue. Yeah. Now in Debian those are going to be architecture at all. In mDebian we're going to reduce some of the CPU overhead by making them architecture anyway. To do that in Debian it's even more on scale. The small CPU overhead or the small batch of the machine that size is not a problem for Debian. Is it actually really noticeable to do end-in swapping and stuff on a translated text? When you're loading X then yes. Just to come back a moment, we started up with different aspects. We took you out of the embedded versus translation updates and so on. For the embedded scenario, supposing you had some filtering system that worked, do you think that's enough or do you think that it's still bad to have to download all the data? That's our problem. The system where you actually download all .dev and then manage the installation on the box is fine for Debian. I would probably still, partly because we're changing the architecture of the .dev anyway. We're going to be building the .dev for ourselves. We might as well then actually make sure that the .devs are exactly what we need and only install on the device the .devs that we need and take out all the calls from the actual package. In that case if Debian had implemented it with the filter method could you then use that information to filter that information out of the .dev when you're rebuilding the .dev this way? Yes. So it's not just the Debian people. For the people we've already got who have to find download Debian on the slow network links and got a bit, not need all 8 DVDs. It would also be nice to actually split out as much as possible. I'd rather say let's move all the translations out of the data. It could be interesting as well for the CDD people who are building, say, a distribution for one country. Absolutely. If I put some numbers on it for example, a typical Debian box will have half a gigabyte of files under user share local. And probably the most that even frequent multilingual people will tend to only have five or six local set up. The biggest local on the box is French and that's on one box that would be nine megabyte. So you go from half a gig to about 40 meg. It's a huge difference on the entire actual bandwidth requirements for Debian. And also on the disk space it's all obviously filters all fixed there. I noticed myself from doing tests of the DVD installs that actually I can't install in the limited disk space I've set up for a few years ago. I can't do a desktop levy install anymore because it overflows two gates. And the vast majority of that I can make it work by how long would the install look. So the background is doing a recursive OEM on the content and you use a share local. Yeah exactly, you could lose 300 megabyte over the night and not even notice it. But filters covers half of that and then it covers the disk space usually actually. Yeah, it's just the fibers. The downloading stuff. How do you deal with that with loading the size of the archive in terms of the number of bits? If we ignore how you would implement it from a moment, which is a lot to do, but if you think of actually the CD or so on problem, there you could think of having, I mean from a user's point of view or for a stable distribution you could have a language pack idea. You could in principle have a big package rather than 200,000 packages. I want to get the French translations. Obviously the problem for us is because we don't build stable in one go we build all the packages and we want to have... So the question is how can you get to that for a distribution? I don't know if you could easily get to it. Possibly, I mean the other alternative is if we do go for a package here, we group together related languages. You know, we could do the Western Europe or South America or whatever. Sure, you would at least improve things. But you could do that with meta packages, you wouldn't have to choose a server. Yeah. But we'll change a little bit of T-notes and the servers will put together. And that could be easily controlled as well. CDDs is what I would make it say or we want this set of translations. So okay, so what kind of workflow are we looking at for this in the end? I know, you and I have had this discussion before, we're not with everybody here. What kind of workflow do people think could work? I'll dive in with the stuff that we've discussed before. I mean, we can have containers uploading normally to the archive. We can have tools that automatically go through looking for translations that are needed, pick out translations that are already available for the upstream source and build a massive big database somewhere which will help us track exactly what we've done and of course we've got people already doing this. How do we then generate the T-debs? Do we generate them automatically depending on the changes that have gone into Pootle? Do we have the translation teams explicitly going through periodically doing a new update? I guess we don't need to have them tied specifically so that the T-debs are exactly version-aligned with sort of packages. So they'll look like NMUs or BitNMUs or something. Alternatively, we could even go so far as if we are happy to have some kind of automation then each time the translation is updated and reviewed with the automatic ability to magic com and drop it into the archive. Yes, we can. I mean, if we can get to that point, what's the feeling with the FTP folks? Obviously, your opinion here matters as well. I think anything which is going to involve multiple hundreds of thousands of files is going to pose a problem. Do we have to mirror every T-deb on every mirror? Yeah, that doesn't matter because even if we start doing exclusives we can't take it to actually build a file that's still available to transfer. If we exclude that from various mirrors then we have to change our tools and the ways that they can access the T-debs somewhere else so that we can meet to our special knowledge about accessing the T-debs. If we're looking at maybe having the T-debs in a separate area of the archive again, we'll still need to have this discussion. That's part of the way that I've played at the moment is so that you have... When you download packages GZ for the T-debs, it's very, very small because it only relates to that one locale boot. So you have a PT locale boot and that package is GZ and it contains PT, PT VR and whatever else comes out of the PT. So by the actual package locale selection you do as a user, that determines how many... how much data you have to download Of course that helps for the user Every time we make it easier on one end we make it more common in the other end. Is it worth actually setting up an extra, say, locale.dev.org archive tree and then we can place it up by the difference so en.locale.dev.org es.locale.dev.org would work that way. Then that way, for example, the various country mirrors could significantly reduce the amount of locale they go through. We need to improve the support. We need to add the support into the tools at the bottom left. And if we do move all the translations out to T-debs then it means everybody, say, one team of Spanish translations will have to be pushed, pulling forward. But if I move the lang update C++ code into act if the act maintainer is willing to take that on then act will automatically understand that T-debs, it gets from a different source. And what I'm doing, because of the embedded history of the program it tries to do it all, if it can, in RAM so that it generates a tiny bit of sources because that needs a file to generate. And then it downloads it and it downloads on to a type of air system like that to get the packages to GTA looks at what T-debs it was and it stores them towards every way. So it works as a possibility if you actually have a completely separate section of what you think of it if you must ensure aggregation. We're already looking at a data section to go alongside the archive at some point. Is it worth having an actual locale section? It takes a lot of effort on the main neural network. Was that just the question if you really want to have a million external files around? One of my concerns as well is that you took a box by Gluck where basically we need to install all the translations into a shell server. What is the overhead of typing app to get installed random package in a multiple or do we need to install initially? I mean how many files are you going to have to fetch to do an install? One extra one per source package per language or per language group? Yes, per language group. The T-debs themselves are tiny. That's part of the problem in some ways is that they are such small parts that it makes it unreasonable to load them into a load group. When you actually look at the section in half of the archive don't forget that you're also reducing the size of the main archive by a lot of alphabets because they come out of the source package but they are finally packaged. They will be in the source package but they will be coming out of the bind with a new group of T-debs. Okay, so we have one more thing to start with the Gluck header part where it will be that we need to have some source files around or we need to send them back to us. We need to make sure we don't throw away the source one always so that we'll cancel it. Sure. Is there not a risk if you're pushing every single translation that is in English to a separate mirror network that won't be established that all of a sudden languages, non-English languages become second class citizens or do you think that the mirror network will? If you're saying you have to carry this new archive mirror you have to explicitly take some action to set it up and you have to carry a million new files. Because you mean within just the mirrors themselves? If we have local.debian.org that has all of the T-debs on it and we don't put them in the main archive that any existing mirror needs to take a possible action in order to start sharing the languages whereas with the splitting out of the architectures it was a step to exclude what you got from all the ones you excluded and this is the other way around and our archive will no longer have any translations in it if we push everything to T-debs and there's a risk there. If we push the people first and we do have obviously the central archive will have thousands of C-names and stuff in place to point to the various different locales anyway because of course we will need the master copy anyway it's then just pushing them out to the mirrors I mean I'm still expecting us a lot of the mirrors are still going to keep a lot of it anyway but of course by splitting out by making it easier for people to just pick up the smaller number of locales that they care about it's not a million files. It doesn't really save that much space splitting them out I have no idea how much but maybe half a gig but that's largely taxed I assume what does it compare like with Preston we wouldn't end up splitting out all of the languages in size which is well, then is the effort of having it all separate worth of rather than just putting them in the packages and having t-dabs as a way of updating our object one thing that kind of we are going to reduce the size of the main archive but we're not going to actually reduce the number of files and we're not going to reduce the size of packages GCC which is what I'm sure you're going to like run out of memory with already say I only have one locale I'm going to have now this number of files in main and this number of files in main FR and the same size packages file above so apps dynamic and math will have twice as much chance to work it can be a completely separate process doesn't have to try and map them at the same time well it must because they have to be typed by source package version and so apps is the one that has to make that decision that's the difference then because without using the source version dependency in the t-dev we can do that separation in memory if you need that source version dependency then you have to which you do you have to have don't you know what happens if you massively change the localization for a package good text is actually very intelligent because you get text at the moment t-devs only support get text alright get text by far the way the fact of standard the fact of standard but some big upstream software are not using get text most of the office the major ones actually do not use get text is there any technical reason to have some calendar use get text no it could just be a top or whatever localization files as long as the programs can deal with you know so as long as you can separate out there's no other way to do it they have to look out actors in effect most of those will just become t-devs but they would disappear from the way out of that I wonder if we really need to have some t-devs split in their language just one t-dev per package would also be a nice option I think we don't need then something on install time to sort out the language that's in the t-dev we don't want to have but you have an easier way of updating the t-dev and it would be a number that we can support in the archive directly leaving out all the problems we have with this split but if you're just having it for the updates then it's not clear you necessarily even need to have it for every package but you could just have it when there is an update the ideal way would be that we have support for every package but it's only used in case we need to update the package out of band the update should be done with a maintainer usually but shortly before our release like in freestime t-devs directly could be uploaded and if we know that in advance we can create some procedure for t-devs to be updated without uploading a complete new source tab and on install time we want to also which package are you which package package you have all the normal languages like you have currently then you need to duplicate d-package to be able to support classes where you can sort out I don't want this language and update translations when the package is posted you upload a t-dev and the d-package now says a t-dev package instead of entry or something download the t-dev t-dev type I like this but this of course goes back to the problem I can't remember who there is an inherent trade we either have millions of files or we don't do the reducing bandwidth thing I think this brings a good balance at least with the DNS proposal we speed out translation from I refogulate to be sure I understood where we speed out the translation from the source package in a t-dev for all languages right and then we can be able to update that stuff and then the package could be batch or change or whatever to only extract the French so that the user configures the system for French and then only French files are so this brings this is more or less okay for the archive not a big answer this is quite easy to implement in the tools and a good way to work for the translator this does does not completely solve the problem of bandwidth or the light but this is already a good problem so by keeping a careful track of the metadata I mean we can have the t-dev packages file track not only the number of t-devs and also the track which languages are contained inside each one so at that point yes you can then pick out very quickly for example if there was a newer French translation that you can pick out from one file you don't necessarily need to go out all of it so it also allows the MW folks to specifically split them so they do only how on the bandwidth type of question I think it's probably not realistic to say it's never been really a focus of WN installed to reduce bandwidth anyway so I think maybe we shouldn't worry too much about that but it would be nice if we supported the infrastructure so I think we have if we could actually have that infrastructure like at the tent the support for it there in the WN versions of the system then any derivative, whether it's MWN or some localised WN or some country or whatever where they want to split it out there also wouldn't be any stoppers would be to a stable release on a different server splitting all the translations to a stable and dumping them over there so we could do that we could do a stable just do a stable release we could do specific individual CDs or whatever of all the translations if you're saying this I guess you need to make that work you would probably want to do some processing of the depths which I think sounds a bit nasty but I don't think there's any problem with doing that yeah that's the install type of question if you want to receive the CDs then you could process the depths you actually put on the CD to rip out stuff actually in that case do we want to have translations in the normal depth files or do we move them all out to one T-Dep file that sort of package common FTP masterfills it is EBL-based they are normally in the depth and only if you get updates they are in the T-Dep but that means on cases if you have an update you don't have translations twice once in the depth and once in the T-Dep it would be the ideal view but I can also let this we have a T-Dep for every package I think it would be better to have a T-Dep per source packet because otherwise the logic for splitting them out is going to need the depth and the T-Dep that's another thing that point you can get your updates so if I split from the body start and I stay split re-open out the split for various different purposes but essentially we have the split at the point where the package is first built is anyone writing this down? that was the last one do you have a bleeper in you? if you start off with the idea of having a T-Dep per package or per source packet but what I was going to say though if you have it at that level you don't in principle even need it's like you can treat it like you treat the DSE and whatever that it's it's something that's there automatically you know the location you don't actually need to be adding this explicitly into you don't need to have involved packages by all of them you sort of do what you want to allow the translator to do and stuff like that you don't need a translation exactly that's what I was getting at at this level you don't need it the question is then what's the minimal addition that you actually need then to allow you to find these updates as well I see what could happen is that in the main previews we only have binary, actually tank shortcuts but also binary, translation and there will be lists of translation which need to be in the T-depth in some kind of control file there will be some kind of packages files where you can then select on languages and package and what I was thinking was for the packages file we could just add a translation for an optional translation version and we develop a convention whereby translation updates are like binary updates up to B what we make them plus T1, plus T2 and we add an optional translation version into packages which that just tells you where you need to go to find the latest T-depth and do the do the translation files like for get text post after they have reliable version numbers internally abandoned per language or not what the the the workload is put on to get text so the get text has the string passed to it by the binary it goes to the compiled file and says oh that doesn't match I'll use the English no, in terms of if you update one language we want some way to know ideally again what language has actually changed or obviously you can do a Diff you can do it like you have the language as in that you can then say 1.0 or something that's what I'm asking do we have at the moment versions for these or do we need to introduce versioning version numbers as well I don't, I mean I guess at the moment the translations just change, we don't have any version identifier for them I think of Chinese up these French read downloads that's a waste of all time I don't see how you do with that to summarise something about if we are going this way what needs to be modified the split would be made at build time so the packages themselves have to be changed progressively to be a very simple thing to put the code I've written into DevHelper and then DevHelper will just generate the Diff as part of normal WNL it's about 80% and we're sure that 20% that are receiving Diff it's just basically where's my own show the other option is to add it into the end part like a build package so at the very end of the D-Patch build package it's harder to control there if you've got something that behaves slightly differently to everything else in terms of the actual, I mean Endevin's got 1800 TDAs I tried actually doing I tried to describe this I tried what you do in the build but if you are going to, what we do is we drop a lot of the control information we don't need the description from the package in the TDA to the D-Patch so all those kind of changes actually make it very difficult to do it half way through the build it's easier to do it at the end of the build when nothing else cares or if you need a problem to control file or the Devin files file and then you can I was more thinking there's two separate parts the selecting one is to go in the TDA and then there's the actual building in the TDA and the building in the TDA part needs to be taught you're building a TDA just we only want this but the selecting the files part needs I think should probably for guest text or man pages you can imagine doing it even as processing on the dead file I'm not saying that's how you should implement it but it is technically possible the question is so on the one hand you want to have it you want things to happen as automatically as possible for the majority of package like this but on the other hand clearly we also need a system which will support open office and whatever else makes up its own system the open office methods are fairly rigorous and they are consistent for what they do so they could be automated probably as easily as they can just that I haven't needed to do so in MDN obviously one more question do we have a default work file included in the packages or remove all of the text now so is a package going to be useful without a TDA you just put text that's not I mean that's not that's fine just verify that before we come in you won't have packages that suddenly appear with no text guess what I mean I don't think we shouldn't be automatically privileging in English the C is there but again any deviant system install even if you're in New York you should be getting the English translations because they actually for some packages give you better stuff than C or whatever you can make sure it happens the devs would be available in the city so when a guy is doing a fringed installation it's a short one of course you have to list the things that need to be changed to deviant cd to make sure that we of course the tdevs go on the cd before the devs that they match with that kind of thing we've already done some of the work it will probably affect the installer probably yes yes because they're going to need to change that so I think this is something we need to have the tools for for lenny plus one and completely implement lenny plus two I would say because we don't need to say that yeah well that takes quite a lot of work to do right here so what resources will we have we've already done much to it the work now is going to be merging my and deviant scripts into the relevant deviant we're going to get them people on board to detect that tdevs are special and specific we have to do a lot of archive changes anyway my other question as well is we obviously people is quite important part of this we're actually generating the translations in the first place what's the side question how well is that working at the moment what not very well why not what it's doing basically that's also true we also need a strong server now we need this we need a strong server we need to be more scalable it has been designed to accommodate small translation one or two projects a few languages and then in deviant we're bringing a huge number of files we're thinking about DDTP which means thousands of files if we do it with the profile so here also at some moment we will hook up but we will hook it up probably on the IT and people side when the TDAP stuff will be usable we will try to set up something for the translator to be able to commit translation in a BCS and SVM stuff no need for very very fancy stuff anyway so that's the most important and after we will hook up some fancy web stuff but that's not a requirement actually so that can be not at the end but during the process as long as the tools become available we will have a few packages switching to this TDAP stuff then we will be able I think to figure out progressively what to do because you do need more of a server for people to please my audience at some moment anyway we might need some resources for Pudur currently it's nice to have the one we have because it's hosted in Extremadura it's a nice way also for Extremadura to give back so I think this is in terms of public relation it's a good thing so no Cesar is not here so he already told me that if you need a more strong server ask me we have two people to ask can I ask the GP we sent me would it hurt and backup at some moment we need some backup also development server something to be able to hack on without what we know about which is at some moment because the number of files basically is pretty at some moment we know that Pudur will probably not be able to scale alone so we maybe need to speed up things and if we do that we will need some admin people I think this goes we have some direction to go I don't know if it's possible to summarize that Matt already implemented some stuff everything works I've just got some brief notes that we're planning one TDep for source package to use some form of deep package classical for splitting the batch at its stall time the fact that the TDep will then be split we will need some form of script for splitting them for mdev and other users possibly for stable releases plus T for binary translator some form of translation version in the packages file to help keep systems up to date we're ealing for Lenny plus 2 because of the chain effectively because of the changes in the tool chains we need to talk to deep package people about the class support the app to people about whatever we talk to them about the DI people so that Anna can do the right thing and how it will affect DI we need to look at Dev help and talk to Joey H about what he thinks the best way to be doing that and effectively master needs to fix that that's just a more general comment that's not related to that almost all of these things I see addresses areas where there are IT people except maybe apps yeah apps can be maintained by MVL and Otavio mostly but in a low-professional so it might need someone to provide the patches I do, the only hindrance from my perspective is that I find hacking on that quite difficult doesn't everybody? the way I've actually implemented it is to use some of the libGlib stuff to actually work with that I'm not sure whether that should depend on libGlib I just need to work in a different way than getting the list of supportive accounts if nothing else you've at least got a prototype that can help to explain to people how you think it should work and the main thing is that there has to be another talk about not relying entirely on a call to a C function to get the list of supportive accounts, maybe we could have a file and etc. that is generated by the IP version for it but don't you already have an Etsy locale.gen that's something that's actually defined as being a system-wide thing so that will always as you've already added a locales package to Cyprus but I mean that's from the locales package and in specific to get out of the locale there is a list in the locale.gen and they are built in the presence of groups except if you use the locale.gen I was just wondering if we could use the list of supportive accounts let me tune in the future because if we are able to provide an engine list locale, maybe you can package on the locale and extract them from the tarbol because the local in the tarbol are smaller than the the generated locale in the tarbol are smaller than the source. Tarbol is an arbitrary concept so for those who don't know this is something said by the upstream Kalypsy Tarbol is an outdated so it keeps using CVS we didn't talk about the impact on the release or stable release managers point of view there is no real impact thank you would it be possible with the TDEMS to get the TDEMS updated later and release psychomety translator so they could continue translating later that would be Christian yes I would get more work this is a good thing it means that we can just update translations with no chances that we are going to wait and work with boundaries due to that minor tool chain issues that would help in some way because at the moment you can update translation during the freeze process but it means we really need to update the GZR which is always the case of two we have something like 10 minutes first so one of the the goal we had was to continue this work actually during the next manual session I talked already about this there is a work session planned in November with you have one for MDEMS in September I'll also be there for the November session plan to have FTP master people and QA I think and I told to Cesar and he is okay to host instead of having one in October merge we had the idea of an I-18 session anyway merge the I-18 session with no more than people to be able to fit Cesar requirements and merge all the things kind of big extraordinary meeting with FTP masters for the QA and I-18 unfortunately we could focus this I-18 and work on this T-Deb stuff I think and we could always get it if we could get a D-package oh a D-package I don't want to take you anywhere to touch people can check back in the morning Raphael or Gilem Frank is doing that much I don't know how to do that Gilem okay, Extremadura okay so that would be I think the way to move on on this topic so yeah I think we covered what I have in mind the question for NVIDIA Tivian I think you are also streaming out the contents of user share though yes, yes so we won't do that in Tivian you will only have to change for example the developer something like that I'm working on a patch for our dev hub that will support dev build options no docs because it's quite common for dev build options to break policy in many ways you wouldn't use dev build options for anything else so we've got options for stopping stripping and normalization of both break policy anyway so dev build options is inherently breaking policy so we can actually hopefully get Joey to implement this but for our side which just takes out everything and uses jet dock and just compresses the cover that's what I'm working on so that's the time because hopefully we just didn't need to review the archive but any way we will okay so that I think we went over this whole topic so I had which need to conclude all that stuff so thank you everybody for coming and it was a very good discussion and very nice thank you thank you thank you thank you thank you