 Hi everybody My name is Bastia Montaigne and I'm a Blender developer since about four years already and So this talk will be about Blender and Asset Management Because Asset Management has been kind of my baby dragon for ten months already I started working on it But December when I was three weeks working on the Institute here in Amsterdam So let's start with a few general definitions and concepts First what is an asset? So if you want to if you go to the dictionary Quoting in their definition an asset is any component model process or framework of value that can be leveraged or reused fine We know know what is an asset an asset is pretty much everything On the right part I'll try to lay out a few points that I think Are required for an asset it must be self-contained and yet it's usually compound That means when you are using a character as an asset for example It's made of geometry, materials, textures Probably skeleton, pose, etc. However, you do not want to have to go fetch everything every Elements of those separately. You want to consider your object or character as a single entity Must be reusable and shareable. I think this is quite common sense and It may be procedural. I mean pretty much any asset can be procedural on some point You are just linking for example a node tree of a material You can change the output of the node tree by changing input conditions But to what I refer here is procedural is more things like for example Presets or sets of settings There are not data in themselves. They are used to generate data. So and yet They can be considered as an asset So from that I think we can say that in better. We already have assets There's just better blocks. I mean you can link them, share them. They have dependency between them. So Next asset management. I didn't even try to put a definition from Web for that because it's just cover only you know financial things and all that. So just What do we Expect from asset management? It had to handle storage of assets or to Be it internally or using other tools. I mean you do not want to bother Where you are putting your data, what file names, what kind of data structures and all that It's handled by the asset management It has to feature advanced bruising of assets for example searching with tags or Category Maybe you can filter your asset by popularity or by pure priority So you know some kind of data you usually do not really need them to leak Directly so they can have a low priority and all the type of data or some specific objects that you keep using Very frequently in your workflow. You can give them highest priority. And so well, that's just idea I mean many different ways of handling bruising of assets, but It may keep history of changes. That's not really mandatory But it's usually when you're working on a big project. It's a very important part. So version control systems or whatever else but And last but not least it has to keep track of assets in the consumer blend file That means one you have linked that are currently in blender you have no real way to Control which version of the data you are using We want to be able to keep the link between The assets management system and the data inside the blender file to be able for example to fetch a new version Change the revision for example low poly high poly and so on so all this leads to One of the main idea behind the project that is asset engines because there are so many different ways of handling and managing assets We do don't want to enforce some specific process inside blender itself in blender core So we the idea is to delegate this to asset management. This asset management to what I call asset engines we are actually quite similar in principle to render engines for example and Well, the main part of this Presentation will be about that project so No, we are going to switch to the dirty details Let's first resume the core ideas behind that project and the implementation that has been done already We want to define an asset API and not an end set engine inside blender itself That mean blender does not have to care about how assets are stored What kind of metadata you store with your assets? It just want To have data in the end and everything else is asset engine task So this asset API has to be simple and yet flexible because as I already already said there's many different workflow possible for asset management We want to keep and build an existing library handling because well Linking is not a simple process. If you do not believe me just have a look at the code I mean, it's not cycles of course, but yet it has many Potential Corner cases and difficulties. So let's not reinvent the wheel and let's just keep what is already working rather well This will also allows us fail-safe behavior because if further for some reason The asset engines become an available or not working as long as the brand files remains available on your File system you can still open a blender file which has been using assets. It will just Link the data from the library as it is doing already No, so you just you lose of course some features related to asset management But you do not lose all your data you have linked in and for the same reason it also gives but give us Backward and forward compatibility. That means a blend file is created using assets engines should be Usable with all the versions of blender. We do not have asset engines or some who do not have the same asset engine install and so on and Finally it ensures us that opening a blend file remains rather efficient. We'll see why later and Yes, the for the same reasons Consequences of the first point an asset API should only manipulate metadata about assets and actual data linking remains A lower-level library code which is already existing mostly So Before working in the asset engine itself There are several things to be changed in blender the first one was making the file brother asset ready because in Previous code there was the file brother working was really tightly linked to how a Regular file system is Defined I mean it didn't it wasn't ready for things like tags or It wasn't efficient if you had to list tens of thousands of items It would just eat your memory and all that kind of problem So I had to split this in different steps and this has been done and Imagine master a few months ago, so it's available in the 2.76 release Then we have to fix the mixing linked data blocks issue Probably all knows what happens when you open a blend files, which is trying to use data from libraries who are not available you just lose them and Especially if you have you save your blend file, then you lose them definitively That's not cool So the idea instead is to generate some place holder data blocks that means There are data blocks in the same time that's one should you who should have been linked But they are just empty. However, they allow us to keep the reference to the data blocks from inside the libraries So once you have opened your blender file with missing data, you have to do those empty place holders You can keep editing your blend file. You can modify the Library path in the old liners you can save your band file reload and you will find again your data You won't have lose anything This has been made just a few days ago in master so you can try it already and the last point is relocating and reloading libraries on The fly that is without having to reload the whole blend file which has using them This is well in current state It's just a matter of being more user friendly, but for the we'll see later for the asset management system it's becoming quite kind of mandatory feature and It's a bit challenging. Well, it's not really challenging, but it touches areas in blender core, which are very Deep in the roots of blender. So we have to be careful with the chance here Many old code too I've been working on this in the idea we might branch available on the get repository of blender It's mostly also a matter of cleaning what we already have in code Who is managing data blocks and all those kind of things? No, let's see the data structures defined so far first we have the communication with the asset engines this is mostly used in file browser and probably outliner in future too and Quite obviously it's not stored in blender file. It's just live data but defining The common basis that all asset engines should be able to feature So we have an end entry which represents Classically the asset itself Each entry stores a list of variants who can be for example used for I or or poly Objects or you can have several variants for given materials, whatever And each variants stores a list of revisions, which are used of obviously to store the history of these changes This is kind of the last two Structures are kind of optional, but I think it allows us To represent pretty much every possible Data to some extent we'll see it's not stuck right now We can add or remove things as needed one of the important things is a UUID member Data which is used will be used by blender as a unique reference for the asset So when blender needs to communicate again to the asset manager We don't want to store too much data. We just store those those three UUIDs in every data blocks that has been linked by the From asset engines and the asset engine is expected to be able to find again the relevant informations about the assets so that Slide represents the change to the data structures Which has a core of blender. I don't know if every Probably then everyone did not know what is ID, but ID is really the kind of the parent of all data blocks structures That means it's really the basis of data in blender So I am trying to keep the changes in this area as limited as possible and the idea of the unique UUID and We also have changes to the library data blocks to Store references to the asset engines himself ID name and versions and the route path of the repository because of course an asset engine may handle several different repositories it so we have to keep a reference for that too Know the main part in some way because workflows. I mean it's our Simplistic descriptions of whole things are happening in the code So it's what's really tries the type of data. We want the API we are going to need to implement this and everything first That's a main the best defined part because mostly it has already been implemented in the In the branch in the asset engine branch It's Using assets that is bruising them and linking them inside blender The main editor is the file browser for that of course However, you may note that the third part of the schema the bottom one is actually implemented at the import and linking Operators level which means that it's available from pretty much everywhere inside blender so if you want to make customization and everything and I don't think we have really time to go in this schema step-by-step, but there's another point which is quite important here It's that asset engines where we probably have some who are working on the web or network or also Constraints which implies some delay so we have to To have a design which allows not immediate response. So on this schema That's dashed lines and the italic text Here to materialize the Asynchronous answers from the asset engine that is blender sends a query for the asset engines for example requesting the numbers of available entries and Then it will keep poking the asset engines regularly until it gets the final number That allows not locking the user interface for process who are a bit slow for some reason Then what happens when we save or load a blender files which is using data from a set engines? Well, when we are saving nothing new happens. I mean we just store the you the data blocks with the new UUID Data and that's all unloading Again for speed issues the main idea is to Not change the main part of the loading of the blend file We are just trying to read library existing libraries as usual and if something is missing we Generate our playholders and then the main idea would be to run a backend job after loading is done Which would go checking each data blocks and carrying the asset engines to see if Well, if it's missing to get again the data and if for example if Asset was set to always use the list of latest revision We would go querying the asset engines to ensure we have the latest revision or that kind of things So that's apart which implies the ability to reload on the fly the libraries data The next workflows are less defined because I didn't really start implementing them already so it's more a list of requirement about them The asset loaded asset management by that I mean In handling assets you are using already in your band file For example, you want you may want to select or reload different variants different revisions on the fly You also remove loaded assets or relocate to live a library for whatever reason this will happen in the old liner of course following the Blender way of doing things that are five brother and Mostly used to booze and load new data while the odd liner is used to manage loaded data and Yes, and so again about the variants and revisions the idea is to have some kind of common options like for example a default variant and latest revision Which will be kind of some special values and Again, this implies being able to reload data on the fly from libraries and Finally the last part is that you want to be able to edit your asset repositories That is add assets update remove replace edit some asset metadata like tag descriptions add new revisions and so on So the main problem here is that this will most likely very It will be very tightly related to the asset engine himself. I mean That's mostly asset engines tough to be down Blender should not Care too much about it. So We'll probably leave most of this to the asset engines the question Currently is whether we should have a basic support for very basic editing tasks in the asset engine API. I Do not really have an answer about that yet It all depends on other projects like for example finishing the UI the UI customization from Python because if you can define your own editor from Python that Then asset engines, we are may hardens could define their own editor if they want advanced and complex Repository management, of course, there are still many open topics and questions so the first one is kind of Anecdotic because Do we want to keep track of loaded? Data when we are actually appending you know Blender currently has two ways of linking data. You can just link them and There are you are still using data from the library file Or you can happen them appending is kind of very similar to Creating new data directly in the blender file because you have no more relation to the library once it's done Although Yeah, do we want to only use ideas that is data blocks or do we also want to handle All the file types. I am thinking here mostly about pictures sounds videos because You know, well, it's possible for an asset manager to just have database of Image and then go generating a blend files with image data blocks itself To allow blender to link from there, but it sounds a bit over complicated so idea would be to try to extend a bit the linking The linking process to be able to also load that kind of common files Common asset types well I'm not very fond of the idea the idea would be to have some kind of Let's say ghost data block who would be used by asset engines as hooks inside blender. That means blender wouldn't Know anything about that data type. It would just exist in his in his files and It would be used by the asset engine, but I think we can do without that kind of ugly tricks And like several data blocks as a single asset is a quite important topic Currently my main idea would be to use as a library that I block behind it because As you probably know Each time you are leaking some data from a library. You are also creating a library data block inside blender which referring which mostly keeps the File path to that data blocks so it would be could be a rather simple to extend a bit this and use those library data blocks as kind of Assets that are blocks actually. That's just an idea right now. It's not yet Define or and user interface, of course, but that's also related to other projects of the other 2.8 projects So that's it for the presentation You can also find on the web link on the bottom of the screen the written Version of this presentation So thanks everybody for your attention