 Yes, there are slides. You were surprised there It was totally totally magic I'll stand here so that we actually have a microphone perfect First of all who here knows what fabricator is. Could I see hands? Great. So mostly these are people who know about fabricator Fabricator is the software behind developer.blender.org I'm gonna talk a little bit about what's good about it what we want to save Well, we really do not want to save and why we are having to go from something entirely different in the first place I would also like to see hands of who here enjoys the Wi-Fi. Is it working for you guys? Yeah, perfect because I'm also doing that kind of stuff So if there's any problems with Wi-Fi or networking or you need anything on the network side, please Try and to get into contact with me and I'll try to set you up Right, let's start So first things first fabricator, what is it if you look at fabricator? It's a forge It's what a lot of people call a source forge But you know source forge but net is already taken but that's basically what it is It's a it's an organization a tool around a repository of software So it deals with codes it deals with teams it deals with projects It tries to make it that you can work with multiple people on a project do commits do reviews do quality Control over what you're doing. So what is fabricator? That's what it does It lives on developer.blender.org most people here know that Why well why? Basically fabricator got created in a kind of intermezzo moment between source forage a number of other types of forges that came up And the rise of git and github and github clones. So fabricator is kind of in this shady area Where it does a couple of cool things that the others didn't yet, but it didn't figure out the magic Bullet that a lot of others seem to have replicated Not of not all of what we're doing now in github or gitlab or whatever is a perfect workflow But it is better than what fabricator came up with in a number of ways not all of them So this is not the fabricator kicking conference. That's that's that's to that tomorrow But there are definitely some things that we we want to get rid of in fabricator Um nice thing is it's CVS agnostic ish in design It can work with git and SVN, but in theory it could work with other types of CVS is I have not seen implementations of this But it's possible. It's fact flexibly configurable If you have any kind of Organization or any kind of project that you'd like to work on and you put a fabricator in Probably some people can drag some applications together and make things work in the way that works for your team So that's kind of cool That is also the flip side when you look at a fabricator instance. You kind of never know exactly how somebody put it together It can be completely uniquely weird for the people that you You meet running a fabricator instance So with that in mind our blender fabricator use is not necessarily exactly the same as everybody else's fabricator use which also complicates migration and moving to something else later At some point which I'll touch up on later It's extensible. There's a lot of applications. These are the extra things that you can do What's better and it was open source is open source There are some weird features or quirks The way that it works Internally is very complicated touched on some of that later One of the biggest things, of course, most people will know but if you don't it is end of life It is a Facebook project It was hosted on the forge. I believe it was called ph orge And basically they said look we're closing shop. We're stopping development of software. We're stopping hosting of any projects You will have to do something else. That's the message that basically kicked that off And it meant that blender was on a platform, which technically worked There's no technical reason why we couldn't keep on using it but it would mean that Security patches if there's anything wrong with the product with with it We would own those problems directly and have to develop have to put developer time into that Slowly collapsing community because of course if you throw this type of bomb people say, yeah Well, I'm moving so you kind of don't have people to ask things about anymore. No new features get suggested or integrated There is now a new development group that's kind of taking what fabricator had and Is making a community supported version? But I think a year before that kind of took shape and the shape is still not very clear So might or might not sink or float and the holdoff Meant that we were not able to actually say let's let's implement a couple of cool features in our use of developer the blender The org that we'd really like because we don't know how long we're gonna be staying on this. So that's damaging to two things. So I Came into Employment of blender at the beginning of this year Who am I? I'm an ISP guy. I worked at two or three Dutch ISPs depending on how you count Do a lot with Linux nerds? I Built a render cluster for Blender, so that's just a cluster around Sindel time They needed the render hardware. I was a nerd. They found the nerd and That became a project and the project became an obsession and the obsession became a render cluster Now basically I'm employed as both. Sorry not both because there's three things So I am sysapp meaning I am network guy for DC and the studio in in parts And I'm also doing DevOps. So the DevOps part is mainly our build bot infrastructure Which is building our our our products and making sure that people can test against versions and of course it's part of the Getty Transition which has become the Getty transition is now also part of that And yes to get give you an idea of when I started co-actual cables were a thing still then So problem statement we have hundreds of developers spread around the globe. They're not necessarily in one's place It is a big project We have one large repository which is as Projects go quite large indeed. It's it's almost a gig of just assets that you pull in just for the git Stuff that doesn't include documentation. That doesn't include static libraries There's all kinds of our stuff spread around that also is part of just blender core and then there's stuff like you know Small hobby things like flamenco and whatever what I what have you We have a problem or the opportunity to get rid of SVN SVN is running for the documentation translations That's something that we need to get rid of at some point. It was on the roadmap already Exploring git LFS, but SVN is still with us. So that's a challenge. We will have to tackle at the same time and There's another thing which is very important for the whole selection or the whole Problem statement, which is if we were a commercial company, we would call github and say hey give us one of those boxes that you sell and we'll hung it up somewhere and We will pay you good dollars blah blah blah, and we will keep it in-house blah. That's not who we are We're not going to do that. That's absolutely out We have a number of things that we find really really important, which is everything we do We do because of the grace of the community and we want to be able to give back to the community Which means that even the technical solutions that we choose we would like to have as an example for others to reuse later so Others need to be able to benefit We want to have our own infra running as much as possible. This is not possible or even useful for everything like you know Hosting a top-level blender domain for example might be a bit excessive, but you know other things like a git source code So source code hosting system is kind of you know core to what we do So this is one of the things that we really want to keep on site not have it run somewhere for three or whatever night really on site in our own hands and We prefer open-source solutions over other types of solutions obviously So some of the things that we decided upon quickly is look We're not gonna try and find something that can do SVM. It's the pool of selection is just too small We're not gonna find something that's gonna do that. So SVN is gonna drop off the one-and-a-half list Change is inevitable regarding workflow. This touches upon something I'll say later But basically change on our workflow is related to the fact that the way that people use git In the typical github git lab Etc workflows means you make a branch or feature branch or you make even a clone and you pick make a brand So you do pull pull request from there. This is kind of not what fabricator Needs you to do Which is which is a way that we can't keep on doing that's kind of inevitable But it doesn't mean that we need to do the whole github workflow The way that github says it should be or whatever because this is an open field and I'll touch that upon that later Get workflow based is What a lot of devs Want or no meaning we get a lot of people coming in To the to the development Community who asked like where where's your git thing, right? And then we do have a git repository But we have developer to blender org fabricator as the way to interact with code now It has a couple of really good features But archonist is something that it's interesting to explain to people So just having something that you can actually say look look at a generic github Manual or just git manual you'll probably find out how things work. It's kind of handy Planning process changes are inevitable. So this is kind of regarding work boards The way that we Handle projects and teams that's also going to change These are things that we said look these are things that we were daring to touch and daring to kind of change on the way to a different thing and One of the things that we will have to also let go or know that we will have to let go is the loose coupling of what the CVS in this case git does and Forge knows what I mean with this is Fabricator allows you to send in a patch and you don't have to tell where the patch came from It doesn't even have to be you know a blender repo that doesn't have changes that we don't know It's just a patch and you don't even have to tell it What kind of commit hash belong to the patch? To make it so it's like here here is something apply it at some point and see if it fits and then gets better Later when you decide hey your patch is approved. We're gonna you know apply it and commit it Then theoretically there is of course a commit hash in git This is optional in fabricator. So there are applied accepted Committed hash a committed patches in blender where we do not know the commit hash for in fabricator It's an optional field. You do not have to supply it. This is not something that most other forges Allow permit or think is even a good idea This is also something which will basically need to start fixing to go to any kind of our forge Why So git is the dominant player most people most people don't use SVN anymore so much So we're just gonna touch on git Fabricators pulse requests are very loosely coupled. I just touched on that and get up is the dominant player if you like it or not And it has defined a lot of expectations about what people You know want to see on a on a forge So how did we start in on the 9th of February? I did one of the first presentations inside of the studio for just you know the local available developers For the organization like hey, I'm gonna stick time into this. This is what I think we can do Which means we'll start talking with developers I've caused doing a lot of jitzy talks with a fair number of people that Were very closely related to committing things directly into master in in in the github repo This does not mean that I got all developers because they're as I said, there's quite a few But I concentrated on a number who had either strong opinions about certain workflows or strong knowledge about workflows or Strong knowledge about challenges that we will be facing If and I'll come to this later if you are here and you are a developer and you would like to be heard about things This is also why the development the dev attic is here We can meet and we will be able to talk about all the topics that have not been touched yet Towards the end of the presentation it will be also more clear what these topics are There's a dev talk There's a series of dev talk topics which have a number of small threads about hey, this is where we are this is what we're doing and We were trying to focus on which options do we even have like if you go away from fabricator Where do you go to? There are many options a lot of them are Things that somebody developed but kind of got abandoned some things are used by one organization only GitLab is one of the big ones that is actually an open source solution available and there's also a Supporter tier or open source Open source project Offering that is not open source, which is really weird But basically it means that you can get the ultimate tier of GitLab project a product and Even run it yourself on your own infrastructure as long as you only do open source stuff with it So the product itself it would not be open source But the stuff that you do with it needs to be open source Now we looked at this for a long time And there's oh, there's a spelling mistake there also but One of the big things there is basically anything that we add any Extension that we create anything that we start modifying we cannot release as an open source thing So that kind of bites directly doesn't sit well The open source offering from GitLab has a lot of stuff going for it Super complete has a lot of stuff, but it has a couple of artificial limitations because there's a Commercial tier that they want to be able to sell which is fair but the limitations that we saw in the open source offering were of such a Magnitude that we said look we're always going to be you know having trouble doing our workflow with what the open source tool can do So theoretically and you can make you know patches against the open source version and commit them upstream But there's no way that those would actually be accepted by an organization that also sells these Features for money. So that became a big discussion like oh, we could be you know so quick maybe Probably they'll have support for this and that blah blah blah But it doesn't sit well with our with our ideals and there's something about this ideals things if they are optional Then they're not ideals so GitLab We did look at it. We did some soul searching. We decided these do not fit. Well There's a another project called pageur. I hope I pronounce it correctly or I'm butchering the the French language it Has nice features it has some kind of Really tight coupling with source and documentation so that if you do anything with the source the documentation creation Kind of flows out of it Automatically or at least you can do that if you do it the right way. It's used by Red Hat However, it seems to be a dead end in the sense that you don't get a sense that many people are either using it extending it or writing about it, but it's it's it's quite feature-complete and then there's githi githi is a Continuation or rewrite or an exploration of what the cogs project left behind When some other people said look we want to extend it a different way and that became githi or githia I've heard that it's githi. So we'll keep githi for this presentation It's relatively young in the sense that the the the amount of professionalization After they kind of split off took a while for them to settle but they now are quite up to speed and I'll touch upon that a little bit later So we did a test setup mostly with githi, but I installed a number of hours as well Looking at communities around the projects like do people use this what are the writing what their experiences, etc And we re-looked at our goals regarding our values and intentions. So that's the get lab part like hey We could but no we can't or we shouldn't Around July we said hey, let's let's stop You know saying oh we could or we should or maybe somebody will will magically commit something to github, which we can directly use But we said no, let's let's Go for githi. It has the best fit with with the things that we need and want But there are definitely a number of challenges that we need to Figure out. However, it's fully open source and it has goals that align with a lot of what we are also trying to achieve They are really an open source group. They really want to have the open source communities grow by having a good Solid Forge software available to people. They also also offer a support organization. So They are able to support people who want to do a small or large scale githi implementation inside an organization or just in general Including adding code and having that be released in the open source sphere as well They also are one of the few that had an curated extension repository Meaning they have a number of extensions available that allow you to interoperate with CI CD like bill bot or drone And it's quite well maintained So that gives you an idea of hey the base that these Extensions are programming against is at least stable enough that you don't you know implement something that might change Somewhere halfway which creates trust And it has familiar features that correspond to the well-known platforms meaning for better or worse Some people would call it a get a github clone but it presents things in a way that is familiar but is not the same as github and Allows you to find things if you're familiar with a github type of experience So that's done then we installed that we click play on tape and then we're done which is Sadly not how a migration of something like fabricator works and not at our scale So I call this of the work. I should have made it all capitals. Maybe Fabricator uses it. There's challenges and these are Some of the challenges are weird things and this is maybe why the talk is interesting for people Who might bump into this later fabricator uses a database per application now an application is something like Issues which they call manifests. That's an application differential is the thing where patches and pull requests live And they are a database each application is its own database. It's not its own table It's an its own database now for the people who are familiar with SQL works well within a database But it doesn't work between databases So you have a relational model of data that relates to each other because you have users that Committed something and leads to a pull request which might be a task But none of it Works with SQL you have to pull one in look at it and say oh I need to do another SQL statement to another Database to figure out what's going on Luckily, there's a API the conduit API which allows you to pull data out of fabricator in a semi semi-documented way Which is kind of convoluted There's a couple of interesting things to note email addresses are not accessible from fabricator This is by design meaning if you have a user You're not actually able to know who to mail when you your user is doing something bad Which would be great if there was a great in System messaging system, but that isn't there or didn't get implemented So your users are there they do stuff, but you're not able to communicate with them except in manifest task comments or something I don't know how they envision this, but it's It's by design. I can understand that especially if you're Facebook and you're harvesting email addresses You kind of want to say hello, we can't even get them out ourselves nor can you but we we did need this for things like Being able to port over our users and mail them like hey now stuff is going to happen somewhere else And a lot of other stuff is abstracted away is in the transaction. So if you have a manifest Manifest is a tasks issues and the whole comment thread underneath are transactions So you pull the You pull the task object and you say I want to see all the transactions of the task object You get a mince ton of Jason out there and some of it relates to an actual text comments Some are actions that happened like I closed open something some of them are null which Apparently is internal housekeeping stuff. It's not documented. You should ignore it according to all wisdom on the internet But that's that's a challenge, right? Like what do you do with this stuff? Patches and pull requests loosely a couple to get reality. This is something I touched on already You have no strong relations So if it is in fabricator doesn't mean that get knows about it or if it happened in git It's not necessarily fabricator things happen You have to verify this or you have to even invent data And we I'll touch upon some of this And we have the blender project Uses very specific way of doing certain things Which is not necessarily how fabricator was designed to use very simple example is if you have a project like blender You can make a sub project called blender 2.8 and blender 3 point 2 Which would be the logical way to do it because you have to blend a project and sub projects But it turns out that if you do it that way your project planning Workboard tool does not allow you to make a workboard specific to that sub project They can only do it on top level project So then you get all the design goals for all the projects and you can't focus on the blender 3.0 only Meaning blender the organization said look we're gonna make everything a top level project So we have top level projects blender and Blender 3.8 3.0. Sorry. This is not a pre announcement of a new release 2.8 3.0, etc. etc. So the way that things are organized is not necessarily something that is Directly translatable to what fabricator thinks an Organization should be doing meaning there are going to be some migration challenges that we have Uniquely to our situation That's why there is a blender fabricator Migration tool, which will have to make some assumptions based on the reality of how we put our Projects and our organization together in fabricator Which is not to say that that tool won't be useful for other people doing stuff But there will be stuff in there like hey, this is something we did specifically and are fixing this There's other stuff. So everything in fabricator is a fit fabricator ID Or an exact fit which is basically the transaction object that relates to a fit and All of these have a super long thing including users including everything So if you find anything out of a database, you have no clue what it is until you actually pull that object in and look at it and say Oh, it's it's my user object and it's aren't So that that's you know, not so handy But that's how it works return strings from come to get conduit can contain multiple null characters For the programmers under you this should be you know a big warning flag For the others a null character terminates a string So you have doubly terminated strings coming back which for for better or worse was not what I expected There is a user called none, which is his name There's a user called deleted user, which is his name or her name. I don't know There's a user called null, which apparently is also a name that somebody chose and there's the user none Which is actually not a user. It's a deleted user. So I was delete at the doing Debugging of this and I had one comment thread that actually had the user none and You and the none user in one and I was figuring trying to figure out like why why why am I hitting? Oh, it's that one It's that object because again everything has a fit and until you look at it. You don't actually know What's what's happening? Let's see there's non-utf out eight strings in utf 8 And some people use the title for the bug bug report thing So they just change titles all the time. You don't actually have a comment thread They just change the title every time they want to say something Really great stuff Some things were simple to do Authentication against blender ID done. Oh wow Really simple expert of data a fabricator is doable. We even got the emails out with SQL We have get T Has support for migrations of everything and they are able to dump it into a directory of YAML files Now this creates a great Opportunity because what if what if we magically don't dump something but just create an artificial directory of YAML files In the right way with an Avertool and then have it import by get T So we create something that was not created by get T, but get evil take There were a number of things that we needed to to implement Priorities were one there were no native priorities. So something could have a tag label Critical and not important at the same time. We are fixing that Flexible work boards. We want to have work boards that you can actually Create based on a query. So give me a word board that matches these specifics Complex import types. We needed the migration tooling had the option to import stuff as comments Meaning text but not events. So if somebody says to subscriber added or topic changed that was just, you know The only thing you could do was text. So we have to fix that And for that reason we got into touch with the with the get T people We became their second or third support customer depends a bit They they already had one at least and then we were the second or third depending on how they count. I don't know Code work is done on future the blender project a future Projects of blender.org not everything there represents the state of where we are at this point But that's where we where we do stuff We have a wonderful guy Matyranta working on that who is part of the get T organization and we have weekly Jetsy meetings to discuss project progress, etc They've been great on implementing the feature requests. These are the ones I touched upon already and badge support So the badges in fabricator where you can see if somebody's a studio support or something That's already implemented and available. They're doing operational support Understanding what the migration challenges are they give us information about hey, this is what we can expect to happen This is what it will do to a get repo, which is super large. Yes or no And they've actually done something interesting which is baseless patching matching So they took all the patches that we don't have a base for and said hey, we know the date. It was created So let's just do a git bisect and see if we can can match it in into the git repo at that point in time and if it matches great, that's now your commit hash This is not This is not foolproof because if you create a new file It will match any repo and it will commit cleanly no problem So we'll have to look at that but this is solved things up to 700 or so patches that we need to look at Documenting process for organizations with the same challenge is a big thing that we're doing So they are also helping with that making sure that we document like this is what fabricator does This is what get D does and this is how you could move from one to the other All features for that are includes Available for inclusion upstream. So if there's anything useful, which is generally useful like priority labels, etc will be included in the get T upstream and That long URL is Findable via the future the projects blended org also will lead you to a read me file Which leads to a wiki pages of that repo which documents a lot of the migration challenges So it discusses. This is what we did. This is what we saw. This is how you translate this type to a number I About five minutes ago. I got a queue saying hey you have two minutes left So this is the main most important one and then I'll cut off. Is that still doable? Perfect Tech feature wise we're kind of there Process project wise is something that I'd like to work on At this conference with developers and especially module owners, etc To kind of feel and get a feel of what their process is right now and what their ideal ideal Process would be in a more get hub like workflow So I would like to invite people who who are interested in that to get into touch with me Saturday I have a second talk, which is mostly an introduction into the features of get T and What we expect to be possible workflows that we want developers to consider This and this deck has not been written at all Literally not even the title has been written yet in in in press or anything meaning it's completely open So it's going to change its nature based on the interaction that we can have during the get the Blender conference And next to that once we have that done and we do know what the migration is going to look like what it means for people It will also mean something for the community at large that we have a lot of people who do bug reports who? interact with Blender via developer the blended or related things so we want to have them know that it's going to moves to something else and how it's going to work and where to find the information, etc the The ambition is to have this be at the end of the year Maybe be beginning of next year, but the end of the year is the ambition to see look we can switch over And the shape of the transition is also not completely determined We don't know if we're going to be able to pull everything or if for example translations will keep on SVN and tackle the get LFS migration later if we can and That is my next Talk, it's on Saturday. I don't know exactly the time the developer attic here is a good place to be If you want to talk to me or other developers about this if you see me running around and you'd like Hey, I would like to have an hour of your time Please do so and we'll sit down somewhere and we'll sketch something out We have another cool guy Jason from gumster I No pressure, but he's also doing a talk It's it's about bill chain thing So if you want to set up a bill chain for blender at your local studio or even let your local residents He will explain what challenges are and how you could approach this you can find me on blender chat My name is aren't Called the blender. I have some stuff deaf talk and I even have a YouTube channel now And my email is aren't it blenders with org This has been my TED talk and thank you for coming