 This is the D8 file management initiative core conversation, so hopefully you're in the right place. And if you like, I messed up the date that I put on my slide, so I have to kind of quick it fix it. So that's like a good representation of my Photoshop skills there. If you'd like to follow along with these slides, this is my first time trying with an HTML5 slide deck. So Dave does fancy stuff. So if you want to follow the link right there, if you're having trouble seeing this, and it should work on mobile too, fingers crossed. So a little bit about me. My name is Dave. This is my first time in Europe. I'm enjoying myself immensely. I'm a senior engineer at Palantir.net. I'm pretty easy to find because I use my real name everywhere. I have a pattern there. And I also am kind of famous for maintaining a lot of modules. And so how I kind of got started with file management is I started helping out with a media module at Palantir. And so, yeah, there's other things like American football. I had to add that for here. I have some cats, one you might see later in the presentation. And I'm going to be a new dad in October, and I just like sharing that. So I like to start talking about this from a media initiative. I know you're like, this is, wait, this is file management, but we have to take a step back. Because last summer, Aquia Palantir and the maintainer of the media module, Aaron Windborn, got together. And we had like a week-long sprint where we locked ourselves in the conference room. And we kind of decided that Drupal Core is starting these initiatives, like configuration management, that kind of thing. And we have this big like ecosystem of media modules. There's media, OMBED, YouTube, all the stuff that works with media. And it kind of lags behind some time, catches up, lags behind, catches up. So we wanted to start this like unofficial initiative for media. And so kind of what are the goals of this initiative? The first thing we did was kind of get a roadmap defined together for the media module. Not so much for everything else, but just kind of, you know, what cool stuff are we looking to add to media if you want to help out? And it's kind of community decided. We also have kind of a small media bug squad that goes on now inspired by views. So it just kind of helps control the issue queue. We have bi-weekly meetings just like core initiatives in our Drupal-media IRC channel, where we discuss what we're working on, where the things are kind of coming up, organizing for sprints, that kind of stuff. We also want to encourage new people to get involved, just to kind of help pump some life into it. By communicating so people can see, oh, people are working on it and stuff. And it's also kind of our commitment as maintainers that we're going to still be here and not go away, or someone will be here to take care of it. And not just like AQUA is working on it or Palantir is working on it now. It's kind of more community-owned. So one of the things that we decided there was that we wanted to... Well, this is I decided. We wanted to make a little subset of the media initiative called file management, specifically targeted for core. So it's the FMI initiative, which kind of looks like FML, which is one of my favorite abbreviations. So let's go a little bit more into this. This is highly focused more towards core. We want to fix some kind of what we consider bad assumptions that core makes about files. We want to provide a UI for adding, editing and deleting files, not just from a node form. We want you to be able to do it as like, these are the files on your site. And I'll show that a little bit later. We want files to be fieldable too. They're already entities now, but it makes a whole lot of sense. If you think of stuff like captions, thumbnails for video files, all of this kind of data can be fields on the files directly. So that anytime you use that file, it can be reused and they can use that data. We also want to add a little bit of a better file access API and add a metadata API. So things like image dimensions. We're considering metadata that we would want to support. Things like audio length, video length, that kind of stuff. We also want to kind of encourage support for HTML5 for audio and video. How exhaustive that will be, we'll see. And also encouraging kind of remote data with Drupal because right now with files, they're all kind of local. So how are we going to accomplish this? Right now, we kind of have a roadmap and a schedule for the file management initiative that kind of lists what we're trying to do. I'll give you kind of the shorter version of it. So the big part of how we're going to do this is we're working on it right now and it's called the file entity module. And it's available for Drupal 7. You can use it. It's in an unstable state. That's just my nice way of saying you shouldn't use this if you can't support it. But it's basically, we're putting everything we want to put into core in this module. So right now we're kind of working on completing those features, getting it finished off. And that also involves adding test coverage because we're not good developers with file entity. We haven't been adding unit tests. I know you can throw tomatoes later at me. We want to clean things up. There's some things that are using C-tools exportables that we're going to have to clean up and convert to CMI if we want to make this for Drupal 8. And then once we've got all this stuff ready, we basically want to merge this module with the file module in core. That's simply pretty much what our goal is. It's because C-tools kind of low. We could have been ambitious, but there's a reality that we only have so much time left. And so this is what we'd like to do. And we're thinking it's best accomplished by doing it in contrib first, kind of like Spark, and working to move that into core. And we have other major issues with core, some not quite as major, and those can pretty much be solved at any time, so they don't have to wait on file entity at all. So I mentioned bad assumptions in core. So kind of going through those, it assumes that all files are always local and always writable, but if you have things like Flickr or YouTube, which conceptually is a file, yes, those are remote and not necessarily writable. So we want core to be able to easily work with those. Right now we have to write our own stream wrapper for it, and we would like that stream wrapper to maybe be in core and assumptions about stream wrappers and image styles, generating an image style from Flickr. You probably want to store that image style locally on your system, but leave the file remote. So we want to enable that kind of thing. It also assumes that files will never be reused, and this is not a bad assumption for what we had, and we basically converted image fields into core, and if you wanted to use the same file, you just had to upload it twice, you can't really reference it before. There's not too much we're going to be doing about this with the file management, that's more of a media module issue, which we're probably going to be leaving in contrib for Drupal 8. It also assumes, so core tracks file usage, so every time you attach a file to a node, it says, okay, I use this file on this node one time, and if you were to remove the file from that node, it says, oh, this file is used zero times now, I'm going to delete it. Which is handy without the assumption that you're never going to reuse files ever again. So now if you wanted to reuse that file later, it's gone from your system, and you have to find it and re-upload it. So that's something we're going to look at. And another interesting thing is image fields are actually a special case. So core has file and image fields, and these are entity reference. And so when you have a node reference or a user reference, you don't have an article reference field that only references articles. You have a node reference or an entity reference that references a specific type. So we kind of have these two dual fields in core, just so that we could use different widgets. But why can't we just use a file field and use an image widget on top of it? Makes sense. So one of the things we've done already, a lot of this includes a few things in core. File delete was kind of a fun thing where if a file was in use, you couldn't actually delete a file. This is an API function. It shouldn't care about usage. It should just delete a file no matter what. So we fixed that. That's kind of a small win. We also helped convert file entities to actual objects and using PSR0, just kind of as the general core keeping up there. And we've almost finished with file entity. It's looking in a really good state. We've got just a couple major things left. So those kind of blockers are... We've got one in core, which is right now, file entities are owned by system module. We actually want to switch that to the file module because that'll help us merge in the file entity module. Hopefully that makes sense. We're almost there on a file access API in the file entity module. That should be complete hopefully tonight by tomorrow morning. We want to add the file metadata API. We're still kind of not sure on how it's going to be stored. Is it a key value? Is it a table with a image width column and image height column? Do you have to specify what columns there are or should core just have a table that you can store stuff in? We also know make file types configurable. So right now, and you'll see in the demo, they're kind of predetermined based on MIME type. So if you upload a PDF file, it's actually going to be called an application file type because the MIME type for PDF is application slash PDF. It works pretty well for, like, images and video because they all go to image and video, but people don't really consider PDFs applications. They're more documents. So we need to make sure that we make this configurable. So that people can use it how they want. What we may do is just ship with a default, like, plugin to determine what a file type is that works by MIME type, and it could be swapped out if people want it to be. That just may be the easy way. And of course, we have some accessibility issues of making sure we support Alt and Title tags on images at all times and documentation as well. There's not a whole lot of in-module documentation or there is a lot of code documentation, but not so much in the UI. And we may need to make sure that we have that. And some other tags that aren't necessarily required. The way that you configure file types and file displays is a little bit weird in the file entity module. I mean, once you get over, like, the big hump, it kind of helps, and you know how to configure it. But if we kind of go with this and put it into core as it is, I know there's going to be lots of people going to be like, what the hell is this? And they won't know how to configure it. So we want to take a look at it. Maybe we can make it easier. We'll see. And we also want to make this file usage system more optional, so that once it's easier for modules like media to not have to figure out how to not delete your files when core wants you to. So we'll figure that out. And a couple nice to have is that we would like to do, which is higher priority, but where there would be kind of nice flashy things. It'd be nice to be able to support drag and drop upload, like WordPress does, or multiple files. You can just drag multiple files, drag them and upload, or use the multiple file widget, some kind of multiple support. Right now it's just single only. It'd be nice to have a kind of standardized entity reference module in core. Even better, we'd be able to have offline entity forms. So if you attach a file to a node, you'd be able to edit the fields for that file right straight in there. That would be really nice, because right now we're having to put a modal up, and it doesn't really work very well. Related to that, it'd be nice to have a modal API in core for the media browser, but we can deal with that if it's not there, for now. And eventually it would be nice to have media in core. But like I said, we're still kind of working with the UI. We want to put in a lot of this foundation of file management in core. And so we can live with media being in contrab for Drupal 8, but we'd probably like to target that for Drupal 9 in core. And another great thing that would be interesting is so if you have a title field on an image, that title is always the same wherever that image is used, or caption. But what if you wanted to change the caption for an image when you embed it on a certain node? Right now we don't really support that. So being able to override fields on an entity when it's used on an instance basis, that's a lot of terms I know, but that would be really nice. Like say I want to attach this image, but I want to use a different caption for it here. So like how do we work that out? It'd be nice to have that. What kind of help do we need? It's kind of a typical initiative help. We're not familiar with simple test, because we're lacking documentation and tests, so we can just throw you at here, here's the functionality we need a test for. We can give you some guidelines for what it probably should do. Go write the test, that'd be really great. Because we were lazy and didn't do it before. I know. We have a couple architectural decisions. I mentioned the metadata API and how it should be stored. We have opinions on that. Go ahead and come and talk to me later. We also need patch writers. We have a couple issues in the queue. It would help if you're familiar with the file into the module or get familiar with it tonight. That way you can not have to be onboarded since we're sprinting tomorrow. We also just need patch testers. You don't have to have familiarity with too much. Just be able to see, apply a patch, see if it works, work around that kind of thing. Again, documentation. If you are familiar with the module and want to help just write some in-page documentation, that'd be great. When can you actually help? You can help out any time. That would be really great. We are actually having a code sprint tomorrow. If you're interested, you should come. I mentioned we have our bio-liquid meetings. Our next one is next Thursday at our 20 UTC which is like 3 p.m. central time if you're in the U.S. It'd be great if you can come meet us in IRC and chat if you want to get a way to help. Code sprints all day tomorrow. Yeah, sure. The sprint will be in the big keynote room in the Weston. They'll split it up and we'll have signs to go for each initiative and who's doing which sprint. We'll hopefully have a big file management sign. Again, we have IRC channel Drupal media. You can stop in there any time. I'm always there pretty much during the day. Here, I'll show you what the file module does right now and then after that we'll open up for questions. This is Drupal 7 not Drupal 8, I know. The basic parts of file entity, the first one we'll go into, is there's this add file straight from, like you do add content, so you hit add file and you can ignore the remote web. That's stuff added by media. Right now you just basically get a default upload file so I can choose a file and that is my cat, Rodney. He has his own Twitter account if you want to follow him. I'm going to hit submit. He did run for the DA, yes. Can he run next time? Nonhumans have not been allowed anymore. I uploaded the file and now I see that I've got a couple of fields on this image so I can just like you do with nodes, I can say this was taken by me and of course I have to tag everything with cats. Here, I can see basically all the files that are uploaded on my site. My new one that I've uploaded is right there at the top. I can see what type it is, image slash shape egg, what size it is, who uploaded it, etc. I can go back and edit it or cancel. I can delete it from here. Sorry, go ahead. Not currently, no. This would just go straight in the root files directory or what the default file system is. That's probably another thing we do need to look at. If you're uploading it from a field you can define where it goes. This is more just like I'm uploading a file to be used later. You can just delete the file from here. We're actually going to be adding a little usage tab to this file details so you can actually see where this file is being used to, which is kind of nice. You can also bulk edit so I can bulk delete if you have the multi-form module installed you can actually bulk edit. Maybe this is the thing that is better for if VBO gets into core too would be nice but here I can edit multiple images at once. But that probably won't make it over to core since it requires another contrib module. I'll go over to the file types. These are the predefined file type bundles. We have application audio image text and video. You can manage the fields just like normal. But the one thing right now you can't actually add a new file type that's what I was talking about. We wanted to make this customizable probably. The managed display is kind of as you would expect. The one thing that's a little odd is we have and this is kind of the confusing aspect is there's two separate things. There's managed display which is managing how the fields are displayed for this file and then there's managed file display which actually manages how the file itself is displayed because it's not actually a field it's just the raw entity. Because when we have a video file type it may be a YouTube it may be a local file it may be a Flickr video. So in the managed file display we have this kind of concept of fallback. So you manage, you allow several different types of way for this file to be output. So here with image I've got just raw image display with an image style. Or if it's an ombed image like from Flickr or another image service that's external it'll display it that way. And then you manage which order it tries them in. So it tries the first thing and displays it that way. It goes to the next one. So this is kind of a little bit confusing process. I mean it's really flexible is one thing. But is this the best thing to put into core? If you have ideas or alternate ideas I would really love to hear it because I don't have any ideas for this. So yeah, that's kind of just what this would be looking to add. We'd also be looking to add audio and video HTML5 formatters enabled by default. So if you uploaded an mb3 file you could actually play it with core without having to install any additional module. Or upload a video file if it's supported in your browser you could play it. It'd be kind of nice just to have that out of the box. But yeah, that's kind of just what a file actually does. It provides this UI for adding, editing, deleting, listing. Stuff like media can go in and add stuff onto this so it can add like a thumbnail listing which you can see more pictures of my cat here. And it does stuff like we saw in the add file screen so I could add a file from a remote source or from OMBanner, that kind of thing. One thing we're actually playing with recently is actually uploading an archive. I actually upload a zip of files and Drupal would copy it over extract it and add all the files that were in the archive for you. It's kind of nice. I don't consider that really multiple upload functionality but I guess I could technically. Yeah, you can zip your files up then it'll work. Sorry? Not right now. That's the question is are we supporting the multiple property on the upload form which is added by HTML5? And we're not right now that may be the best way that we add that multiple support but we'd love to have someone if they're interested in writing a patch for that for core. There's an open issue for that. Yeah. Go ahead over here. You mentioned a lot of things, decisions that kind of wrote math or schedule for making those decisions. We're going to make those decisions tomorrow. Any outstanding decisions, we can make them tomorrow with code sprint. So this HTML5 thing, there's actually a module for it. Which module are you talking about? Yeah, it exists as another module but we'd like it in core. Yeah, I've seen it but I would love to see a patch for core for it. Yes, again. And then assumes files will be somewhere where it's not because of some stupid stuff that happened that's upgraded that went wrong on this database or something that someone did. Okay, as it is right now. I don't really have a plan for how to work with that. If you have questions, you can come up with a mic too. That way I don't have to repeat them but I'll repeat them for you too. Go ahead. You mentioned on your nice to have list inline editing for file entity. Did you meant to use the inline editing the commerce guys doing in their kickstart version 2? That would be nice to have but again we would need to shift that into core to make it available. It may be something that we support with Contrib then for Drupal 8. It's definitely something that we would look at. It's been on the list of things that could possibly work for that. I've liked what I've seen with that module. Have you considered at all the ongoing file links and the problems that we have with moving sites around and the old references that we end up in HTML which I think is a Drupal file management problem which keeps on coming back. Is there a scope or is this out of scope for that particular problem? So there's two problems. One is actually getting the files in sync between servers. Because entities use UUIDs by default. So if we embed those in a syntax with WYSIWYG we'd actually reference them by UUID. We can't do much if the file doesn't exist but we can at least make sure that we're referencing the same thing. And what media does is it references a file by file ID which almost works the same and then generates the URL to it based on that, on the server that it lives on. So this is a hard URL in the text. It actually converts it on the fly. So I think that's probably a good solution. I know Sun has some ideas too on an inline API for how to use that as well. That's nice to hear. I don't have a question. I just wanted to point out something that has to do with something he said earlier in his slides about how you can help with his initiative. He mentioned all these, did a very good job and one of the first things up there was he wanted simple test writers. Now you may say to yourself I don't know how to write a simple test. That's also a solvable problem. So if tomorrow morning you want to help, if tomorrow you want to help Dave read write simple tests you can come in the morning to my sprint where we will teach you how to write a simple test and then the afternoon you can go and work on the media. That's a great system. All the other things that are up there too, aren't sure that you're confident in doing those things, come in the morning learn from us and then go help Dave. In the same area, in the keynote room area which will hopefully be divided into separate rooms so that when I talk loudly I do not disturb the people who are working on his and other initiatives in the morning. In Balsal AB which is the keynote round. So yeah, again if you're ready to dive in with helping out by all means come and find my table in the morning. But if you're not so sure find Jess. It would be a great way to get introduced and then come join me in the afternoon. Is there an idea about files having a language like if you have different PDFs and then you have different versions of that in different languages. Will you make multiple file entities and each has a language or should we use some kind of entity translation? I would hope we would want to use entity translation but the problem is that you would have to use it as well. So we're kind of getting into an inception kind of thing where you have a file field on a file. It supports languages out of the box. You can associate a file with a language but how to actually translate that may be a different issue. We don't really have a good answer for that right now. We'll do one more question and we'll call it a break. Should the file field really be an entity reference field, do you think? Yes, it should. It should just use a different widget. Thank you everyone for coming. I'll look forward to seeing you tomorrow, hopefully.