 I'm Johannes, this is my friend Tobias, I hope we know him as a local organiser as well. We're here for the dark table team celebrating 10 years of existence. Yeah! Yeah, thanks everyone for getting up reasonably early for us, we promised it would be cake and we will deliver this promise in case you stay here with us until the end. This is, I think, the most ill-prepared talk I ever gave in my life, so you have to excuse that. It doesn't look like it should be after the worst. Exactly. So, we have a couple of slides, we want to step through that, we have maybe a couple of thoughts that we'd like to share about the community, about history, how did you join this project. If you have any comments, feel free to interrupt us and ask in-between because it will not throw us off our schedule if there is one that we postpone this. Since there is a break right after this, I think we might as well jump in and start with the slides and drag it in whatever length it turns out to be. So, contents of this talk. At first, we'd like to introduce what is dark table because, yeah, I keep meeting people who ask me to start what. Sorry, what are you doing? So, just, we're all on the same page with that. And we'll have a very quick historic overview. I thought I had a couple of very cool screenshots, like the first window of dark table that ever opened, which looked very terrible, but I didn't find it anymore. So, we'll have a couple of different ones and then, yeah, we'd like to focus on some organizational aspects because, yeah, ten years is the big thing. It's time to reconsider what we've done. Let's think about organization. How did we manage this? And, in fact, we did not. We're just a bunch of kids. We have no social skills. We just do whatever is fun to us at the time and it has worked out reasonably well so far. So, we should talk about this. The first was dark table. This is just a couple of random screenshots. I found it in my hard drive. This is the light table mode. So, this is the one aspect that we focus on less. This is the digital asset management aspect. So, you have all your images in a row. Could be thousands. You can device workflows here to reasonably quickly select a couple of them and then you would go and develop them one by one. This is a screenshot where I was developing a open VDB point cloud thing to match a reference picture of the sun. It's weirdly colored. The sun doesn't actually look like that. But I wanted the render to look exactly like that. And so, this is one use case for dark table. It may not be super obvious to photographers, I think. So, yeah. That's the edit mode. And you probably all have seen that. We have a bunch of key features. What's your favorite feature? I have features with keys, I guess. With keys. Anyone know that? You can type colon at any time and then grow the eye style. Like at least colon Q was really reliable. The other side. You can... We should have made this an April school joke. We have space in April. Yes. We have that. So, that's the two key features that the two of us use every day. Other than that, there's the non-destructive photography workflow, I suppose. Yeah. I mean, dark table, you don't really see if you work. It's just the same for the background and the database and also you need to cycle up to files. And you just close that when you're done and when you're over it again, you can start over or continue working with that. And I think that's the top spot. You don't have to think about anything like that. Just close it four-way and the next day or the next month can keep working on the images. Yeah. The other thing that sets it apart from other war developers is probably the local edits. It's pretty sophisticated. You can do hand-drawn things. You can have a semi-scripted interface, the breadth ranges of values in the input and then just works only on that. Like, automatically, a computer mask for you from, say, just the sky colors or bright colors and stuff like that. It can be really useful, but also daunting to combine that. I have to say, I don't fully understand how you use this. So, if anyone wants to start, I'm glad to hear your workflows. One more thing is, I'm the GPU Acceleration. We've had GPU Acceleration for the longest time now. It's based on OpenCL and to tell the truth. I'm not sure for how much longer we'll have this library around vendor support seems to be going away at some point. They're not very enthusiastic about it. And also, it's going faster, but I believe we're still kind of CPU bound there. So, in the future, this would be one point to work on, I think. Yeah, I said the digital asset management part is not our strongest part. There are people who much prefer other programs for that. Still, the database scales up to tens of thousands of images easily every so often we get feature requests or bug reports waiting for people to have half a million of images in their database. They are just really slowing down if I'm searching for tags or something and then we add an index or something. So, this can be spelled out if you have problems with that. And then, yeah, there's tons and tons of features I never dreamt about. And one thing I really want to point out is that we have this modular architecture. We might get back to that in the graphs later on, which enables us to just, maybe we never do that, but we could. Just delete a C file. It would not build a DSO. We would not load it back in, and then parts of this thing is just gone. Even just delete the shared library once it's installed and it's just lacking there. If you want to make it lightweight. Yeah, you can just delete stuff or copy stuff in. That works. And so the core interface is reasonably stable. We'll see that in a graph later on. But yes, the most important part I think about Dante was the community. Like, I can write a little bit of code. You can do the same, but we have so much more in this program. There's stuff in it. I have no idea how it works. And it's just because there's so many people supporting us in there. So a good community working on it. And yeah, that always surprises me how that really works. So let's have a look at that with a quick historic overview. The first commit, it looks terrible because it comes from SBN, right? So 10 years ago I thought SBN would be more than 10. So it's almost exactly 10 years ago, which is important. It could have been snowing. It could have been snowing. And I did not bring the first picture of the buoy, but I did bring a couple of ditch images on the way. Because I think this is fun. And most of you developers probably know that if you're working on images every so often there comes one out that looks totally not what it should. But it's fun to work on. This one here is one of my favorites because it looks like our missing image skull in a way. The plastic pyramid and the dressings we used broke and just came out like that on my screen from the front. The other thing here looks like a feature. It's like a good taste filter or something. This is what it looked like opening it. And the technical explanation is there where the eyes catch highlights. You can see it in a very bright pixel. It feels like a bra. Yeah, exactly. It just produced a not a number and not a number propagates, right? It's viral. It's like local contrast it just feels the whole lawyer is black. So we did not censor anything. We are not looking at your pictures. Mostly. So who does start a table? I said we have this community and I said we're not really managing it but what does that look like? We have a ton of contributors this very large group and very many people caring about this thing so I thought for today we could do a retrospective and do some data mining on the git repository. What we regularly do with the git repositories we just take who had commits since the last major release and we put those in an authors file and there's a contributor credits thing we have a release workflow thing that just goes through git and says okay, this guy needs to be in the eval dialogue. So that's interesting but it doesn't say anything about the code it doesn't tell you who owns this line of code or where is this a good line of code or comes another contributor and needs to fix it like at least the indentation or maybe it's just broken and then after a year later we just remove the whole thing. So what I want to look at instead here is this git playing base output of a shell script which of course you all don't understand by now but then there was Kippin's talk the other day and he showed us a tool that visualizes this kind of thing much nicer and so forget the script I did run it it is interesting but we actually have graphical plots and I want to show you though we're visual people and I was too lazy to visualize so first plot interrupt me and tell me whatever you think about it if you want that's the survival plot so that's 100% is first commit or I don't know first year probably in zero and then it goes down over the years like how many lines of the initial lines of code make it till today I think this is very interesting to look at because that here is probably my first prototype with a terrible glue and everything I was not there anymore but it also somehow tells us that there is about 50% chance to get over 2 years and once you make these 2 years if you code it's probably staying yeah and then it doesn't go down as much as you would think and this is really just the crappy initial prototype that filled in holes with like stuff called that does something to remove it afterwards and after that it's kind of stable just the same thing just added over time so every one of those blocks is the lines of code introduced in the here you can see at the bottom and then you can see it's history through time so as it goes down and you can see to the right the recent push for a new buoy and other years contribution from back here it's a steep increase you can see this bike you can also clearly see the original old code stayed for a very long time up until 2015 then suddenly we are not sure that happened a few things for when we moved some libraries we banned it we also ran a code form once or twice which changed its ownership because I ran it so it touched half the code I was over half the code so that might also be a little misleading but I guess you can factor that out of these graphs but we didn't because of reasons yeah but libraries maybe because you mentioned Libraw we were based on Ufraw then a home made Lib Ufraw which is no such thing we just had that and then we had Libraw as well and they removed some feature it was the same story but different thing this is why we are living here now so this is about authors which is something I find very interesting as well so this is what you can do as your one man army, Memphis is me over time it's relatively stable but here in the total complexity of the thing it doesn't really matter at all like the whole thing we need everybody else see this blue thing on top that's others, that's everybody else who does a little patch here and there so yeah it's a massive number of supporting people in here I mean, don't know if you have the actual number in there, it's over 140 people something like that I can't be 100 in my T1 not 190 from people in the midst I find that impressive I don't know about game people some people just one line of code fixing a typo and read me a file or something like that we have this old school workflow I see, we're not there, you do that over weeks again and then it's hard to get your one line of code into dark table yeah it's also a little misleading because this also contains the user manual and the translations which, especially the .po files are big like lines of code files they're huge so I guess half of the people you see in here are translations first one, alexander or Tatica some of them they have maybe one or two or three lines of code changed in their life but still they're in here this the tikka is this big thing it's just translations it's translation code alexander I saw this red thing it's smaller lines of code than Spanish user manual is big user manual is the big one because it's translated it was really for the first stretch of translation so it was hard but she also translated user manual and really did all the user manual itself and a lot of code so it's half of the singlish it doesn't matter in this league alexander's push for Russian translation really early on also meant that there's more people going into the project so all of that is very important work this is what you just said about the file types let me see if I can find the .po in here yeah it's the big the big thing on top but the biggest part is translations and user manual a bit of functionality too mostly you use a translation you can see files so the actual code and the great part here the hand files so our API is quite stable the hand files don't grow but the implementation changes a lot that's what I meant with the module that you signed her here on so the module interface in the core is kind of stable, that's good and then the modules is where all the work is made I mean you can also see what's good in the right button you can read a lot of that if you know what you're looking for any cats in here conclusion from this part of the talk I was surprised to see how long code lasts we don't replace it or delete it, not unnecessarily and even considering that there's this code format that would change ownership very grossly you don't really see that looking at Hendrik's code, Hendrik has done a ton of work on dark table in the early years he hasn't since January 2016 still 50% of his code is in there it's functional, it's working so yeah I don't know what the conclusions are to be drawn from that either everybody just writes perfect code here you don't need to touch it and somehow don't and the other thing is there may be a slight possibility that we don't even reconsider the old technical debt we just keep it tied it up so yeah the number of contributors I just said is around just shy of 200, 191 what I counted and they're all from everywhere around the world like we have time zones, nationalities, backgrounds and complete mind sets and as we've seen in the full site they told us we have people who test it who suggest feature improvements who translate it into weird languages a couple of techies like people who know their technical aspects, they can do some programming or they're just into signal processing or they know photography or they really are color management just technical people who don't necessarily need to write code but tell us to fix the things and some photographers how do we organize it? I promise we will talk a little bit about our stringent management issues exactly, so there's a norm that our tape was like a duocracy whoever does it the key is under the door mat exactly, yeah be reasonable about it and do whatever you want because we're all grown ups yeah, this works mostly, but it works surprisingly well we're not going to say so yeah, this is a spare time project for us, we want this to be fun we don't enjoy managing people so if we have any rule at all we never keep anyone from doing code if they're enjoying themselves there's a couple of quality assurance things here and there, we want this to be reasonably fast and so on, so we have in our history rejected a handful of patches mostly because we didn't have time to review them or rewrite them in a way that we wanted but yeah, if somebody steps up and says I enjoy doing this, I want to fix this because I think the rest is cool but this needs work then yes please, which works to an extent, right because you can't just do something and then leave it half broken and disappear you need responsible people as well and this is where this fine person comes into play I think, mostly you and we have worldly as well we have a couple of super responsible like senior people who just show up and are reasonable and grown up about things and clean up after the rest and have this this really precise way of making things work we have to maintain it that is the big problem of a non-destructive workflow you have your picture, you have your XMP file and you come back after five years you expect this thing to produce the very same output and we promise that we try to stick to that and whatever crappy module is released with the release we get we are less strict about this we promise to support this why the long dark table is around so you can just open your old picture and it should look pixel-identical I mean, we have broken the two pictures in my database don't say that we will just wash the pictures and we broke these bronzes a few times like in slide face there is like an improved notion that we think nobody is going to rely on this workflow on rounding out the important 10th picture that doesn't count this is also being a responsible thing because there is a trade-off of keeping legacy code and having to maintain it in the future and yeah, this is the second aspect you need people like that what did we learn from this approach of not managing the project I was surprised how quickly we gained features initially it was really just me doing a little bit but then everybody came and did super cool things and it was very fast to develop that goes in terms of community and features of the program or any push or something like that for some good reason maybe it was at the right time for a closer contact there wasn't much wrong therapy was closed source at the time this was started, so this was the whole reason why dark tape exists thanks for not open sourcing earlier I guess yeah, nowadays we have features I would have never thought of I could never have been like a commercial manager and stand there and you give me $30 million and I go and think of those features I just would not have had this idea I'm not a pro photographer I don't do dodging and burning the paint we can do this in dark tape I don't know how well it works but there are a ton of features like that that I would not have paid a developer for to do it they just step up out of the crowd as a community I think this is amazing I really learned to love about open sourcing the community same for languages we have languages I didn't even know and yes, we're coming back to that we have a ton of technical depth because of this approach we also piled up trap code because we're soft about this if you have fun doing this and this is a cool feature we want to feature exactly whatever come on let's merge it we'll worry about this later and then we don't there is a ton of code that's less than optimal in terms of readability, maintainability it's useful we have a ton of these modules in darkroom in development modules some of them are very obvious you need them like dropping or something like that some of them are really obscure we have perceptual night vision no that was cool it's cool but it's useless but it's fun I remember this because in the very same here there was a sikraf paper coming out after we had that and they did exactly that ok cool do you think the audience at sikraf did you know that no as I said we try to finance some things we would never do such a thing so what else we learn the non-destructive means we cannot really deprecate things we do but you can't easily throw it out so this means after starting from uofraw code we have 10 years of uofraw style pipeline that's good I mean it works it works and it works especially if you have a ton of modules and you can just switch off one half and switch off the other half or considering the number it's more like to switch off 10% and switch on the other 10% but yeah it also means switching off all modules without deprecating them you're piling up stuff above stuff which is useless and gets more useless it keeps growing so we could have a whole other session what do we think we would never ever use again with the entire table how do we do this because we want to have fun lesson learned if you want to have fun you need a community of compatible people turns out it's quite many of them you have to just talk get along, have the same idea of what you want from this project and then the management involved in that I think this is really the number one most important thing that you talk and get along and you have people that have a similar but enough mindset to do this together I have to say, I have to thank lipographics meeting for that because we met we didn't meet the first time we met a lot of users and co-developers at LGM the awful lot that you know if you have met someone talk to them, have a few years to marry together and if you only see each other for these three, four, five days the rest of the year you still get along very nicely because you know each other, you know how to reach his IOC messages and know when he's killing and when he's killing even more you know we're never serious we never so I have to thank this community even the people not contributing to Darktable for having the whole bigger community and bringing us together because I learned about Darktable at LGM in Brussels from Alexander a talk he gave and I think this is a place to say thank you to you all I very much agree I want to point out another thing that's different to commercial projects as you say we need to in the best case we need like five days a year to be honest I'm more like every three years or five and we have a completely different set of people because of that we're not sitting in the same office every day which means we do not have to get along every day which means there's different ways to advance your code like we don't have scrum we don't do sprints I'm terrible at all of those things professionally as well but here in open source it is good to have an anti-social work star in your community like super narcissist asshole person who just knows his shit knows how to code and you have to get along with him for some time and I don't have a problem with this kind of people and I could sit with them in an office as well I'm just saying in the community or bigger team at work this can turn out to be a problem in open source I don't know about that because it's normal or just know we learn how to like people and then that still works that advances code like somebody can push for stuff if he has fun doing it and yes do it we'll take that code because we know we have the genitors after that right we also have the responsible people hopefully it works after some critical mass of community of course who step up and maintain code after like super coders disappear and I think this is something you can do in open source as a community and sitting together in an office traditionally in a paid for job I would not enjoy this so much I think so output and conclusions I'm not sure I have any I'm not very good at managing people or projects or something if anyone has ideas or directions you think we should be going this way yeah come on and talk to us I think we're pretty open about any of these suggestions and yeah all that's left to say for us I think a huge thanks and to all the supporters what you just said about LGM I think there's this big wall of support every way here not only from a community that's directly related to dark table and actually contributes to colder translations but just yeah the sense of we're working on the same thing we would all do a lot of exchange ideas or we like what we're doing yeah this is good this has been fun and I'm looking forward to the next thing yes