 preventing catastrophes with GNU Media Goblin by Ben, so please give Ben a warm welcome. Thank you. Hello there. I first met Media Goblin about four and a half years ago. My wife and I were sitting at a card table surrounded by cardboard boxes full of our worldly possessions eating Thai takeaway and if you've ever tried to eat anything off a card table you know how difficult that is rickety and wobbly and up and down but it's even harder when you're nine months pregnant and you're about to give birth in the next couple of days so we were eating our takeaway there discussing the exciting you know plans in a sense it's an exciting time and we we started talking about privacy and how we were gonna host photos and videos of our new little person and we camera to talking about Media Goblin so Media Goblin is a tool for hacker parents to post photos of their babies well not really not at all actually it's not it's not privacy focused at the moment currently it's it's it's a and it's not just for hacker parents it's it's intended to be for everyone it's it's a tool for artists and it's it's a tool for artists to be able to publish their their works it's a it's a media publishing system in a gallery sort of format so you take your work you upload into the gallery and other people can view it it's for painters for illustrators for musicians for 3d modelers for people who make guitar effects and want to be able to demonstrate them and for speakers so the idea is that you can collect and curate beautiful and interesting things based on whatever your understanding of beautiful and interesting might be it's a python web application and I'll have it running here on my laptop so I'll give you a little demo now we'll see how this works okay so this is media goblin it has a web interface it's I'm running this this application on my laptop here and this is the homepage of the media goblin site that I've installed here so I'll log in this is what you might see as an anonymous visitor I'll log in I have an account here so I've uploaded some things here already it's the the key difference between media goblin and other systems is that it doesn't do just images or just photos or just audio or just other things it does multiple types of media for the same author so you can upload all sorts of things here I'll show you for example so these there's some things here I'll show you in a second I haven't I'm gonna only upload a couple of things just to save some time so what we'll do here is as an author I'll add a piece of media now I'm a extremely passionate free software activist and developer but there are some things that are more important to me and one of those is cycling and this is a this is a bicycle well it's a tricycle actually that I own and I take my kids around in a little bit and move other things around in so what happened there was I just uploaded that piece of media now nothing actually happened because that's just been uploaded and media goblin has to do some work now behind the scenes to actually produce a version that you can look at might have different formats and different sizes and all this sort of stuff but if I refresh now that's been working in the background and it's now published on my media goblin site so this is the box bike that's really it's quite a fun piece of gear you know it's got a big box in the front you can put all you know put 80 kilos in the front and cut it around so it has the normal sorts of gallery type things so it has titles descriptions I didn't add a description as you saw there tagging you can modify the slugs you can specify a custom license media goblin is very respectful of your choice to choose you know it could be an all rights reserved type license or something more free culture friendly like CC by SA and you can set a default license to I'll flick back here and there's all sorts of other features like collection so you can collect certain types of media on the same related to you know Star Wars or whatever you like to do there's reporting features so that the terms of service on your site if there's a breach of the terms of service someone can report that that and the the moderator can connection that what else do we have we have comments so you can add a comment no okay I can't see it on my laptop so we've got a comment on the bottom so so that's images so that that's your flicker essentially so you know you know instead of having some web service to do that you can you can host yourself we we also do I showed you tagging there's a geo tagging feature so that if you have a camera that's that adds a GPS location to your photos it'll actually pop up a little map in that this spot here and you'll be able to see where it is on the map which is quite neat but as I mentioned going to meet a goblin is multimedia ready so it also does things like audio and I've uploaded one of those here this one is an alarm that my son and I recorded to put on to our phone to remind us to lock our trucks our chickens if you're not from Australia or New Zealand I suppose so we have backyard chickens and if we leave them out at night the foxes come and eat them so obviously we don't want that so we have this alarm on our phone and we've I uploaded here just a copy of it a nice wave form spectrograph there nice media player so I uploaded that in just a wave format and it's converted it to a web-friendly all this web M and all this format so I didn't have to worry about it it's a beautiful feeling to be a technical person who has done all this sort of media conversion at one point or another and tried to figure out formats and bit rates and all this stuff and just drag it on here and up it goes it's lovely and in addition to audio it does video I'll show you something it's next in the line here so so that would be your sound cloud essentially so if you're a musician or speaker I don't know whatever you might be it does PDF so as someone who speaks occasionally I like to put some slides up online this is a great tool because we've got the PDF.js feature in here so you can go through slides so you can put up both the video of the talk and the slides and as I mentioned I said that would be your you know slide share something like that and we have video of course so this is a video of the Free Software Foundation's 30th birthday celebration which is actually quite fun so you should have a couple years old but still and sometimes really nice animation it's got a good good strong story but what happens when the tools we use are obeying someone else a tool you control I won't play at all but that this is the most compelling use case for me personally the the video drag it up and it's their format because things like you know videos my my kids I don't want to put on YouTube and I want to be able to share them with my family and this sort of stuff but I also use this for talk videos and stuff as well that I that I don't have such a strong privacy concern about so do you think that's alright is that useful yeah well I'll show you one more thing this is a this I think is the coolest feature so far one of my friends wrote this game he targeted at 14 year old kids I guess the kids this is a 3d model so what you might print in a 3d printer so he wrote this game called cargo it's about a snail a post snail who delivers letters in blender actually it's quite cool so I'll show you that's let's just start uploading I'll show you the way this works architecturally so I have a process running there that runs the web process this is a secondary process it's running in the background on a system called salary which you might have seen if you do web applications that's salary that's media goblin talking to blender blender the 3d modeling program so salary is just asking blender to do some conversions of that format which is interesting what does that look like okay so here's the snail his cargo the character of our of our story and there's a from that stl file here's a render of cargo the snail from the top or whatever his perspective so we have a couple of pre-rendered shots of cargo here's cargo we can zoom in have a good look at cargo that's any good this was this was done by Ava Palacek for a bounty she's one of the core contributors to media goblin but this was done for a bounty for 3d printer by the lolzbot team they put in a put in a bounty for this who could provide a feature comparable to thing of us and Ava came up with this in a couple of days which is truly amazing this this sort of you know complex talking to blender here's your beautiful experience of uploading a 3d file there it is lovely so back to slides there we go cool okay so so that's media goblin the web application that you've seen it's a GNU project so it embodies all the for the philosophy of the GNU project being that it's it's really important that people have right to control their own use of computing in their lives so it's a JPL licensed which is a great license to choose for a server-based application it now supports Python 3 of the last as of the last release and you might you might be quite familiar with the if you dive into the code you'd probably see some similarities if you're writing Django it has it's not Django but it it does has similar models dot pi and you know routes URL dot pi type thing it's Postgres SQL alchemy salary to do the back-end processing G streamer does all the audio video transcoding which is a fantastic framework and handles all sorts of media types brilliantly it's entirely pluggable so you can write your own authentication media types themes that sort of stuff and it has this other interesting feature where it has an API that's compatible with the pump I owe protocol I'll show you that actually there it is okay so this is pumper pumper is a client for the pump I owe social network which is a second-generation free software social network that came about but the kit the author of status net which is now going to social wrote pump I owe as a successor to status net and just to be clear this this social networking client knows nothing about media goblin it's never it doesn't know media goblin exists as a as a thing but it talks this thing called activity streams Jason activity streams and now what I've what I've done is I've just connected this to my media goblin server so what we're actually doing is we're not implementing a new social network but we're using an existing social network and we're able to interact with this media goblin server so I can go to this photo of yeah so I might go I'll comment on this bike to why not oops and if I flip back over to media goblin we should see two comments yeah so this is really key in that we're not reinventing the wheel here we can provide full client features but we don't have to build our own social network because that's not really a feasible thing to do so the idea here is that you'll run media goblin here as I am on my laptop on a server that you control that's the point of this you're not using someone else's centralized system you're using your own so this is Jason activity streams which is the client to serve a protocol which I just showed you with pumper it doesn't really matter what the details are here it's a Jason blob that says Chris posted an image to of this goblin in this case we know who Chris is because it's a client to serve a protocol so it knows who the client is because it's talking to them has a whole lot of other cool features like command line upload which I find really useful when I'm uploading a whole lot of meet-up videos or recordings you can just specify on the command line does bulk upload from CSV if you were say running a conference and you can actually configure it to read to avoid retranscoting stuff if you've already done your transcoding for your conference videos or something like that it's all queued up so there's a nice little system where you can reprocess things if the transcoding doesn't work properly so YouTube or flicker or slideshare or something like that might be right for you that I'm meeting up and might not be the right choice for you is so to say if you're comfortable with that sort of hands-off approach and you really just don't want to think about it because you've got other things to think about which is totally fine you don't care where the data lives I think that's that's fine you should totally choose that because I mean as an activist I think the most important thing is that you live by your own values and not mine that's that's a really important part of having a comfortable happy life I think is living by your own values but on the other hand we need to acknowledge that we are entrusting these enormous corporations with our precious cultural works and these organizations have very different values to us we need to acknowledge that the sacrifices that we make including things like controlling which country things can be seen from censoring unfavorable media things like New Zealand's political cartoonist Malcolm Evans his cartoons on the Israeli Palestine conflict being censored and say you were a punk rock group making music about Vladimir Putin's involvement with the church in his election there really is no no free speech without independent media publishing so we just need to be a little bit careful I think the other thing to be aware of is that we're these sort of sweeping DCMA copyright takedown things where essentially I mean you might be aware if you if you're on YouTube and you kind of have Sony as a keyword in your video there's a good chance that these auto takedown features will just take you down and it's on your own as to actually fight this this process and make sure your your video is reinstated there's also a subtle effect of these automated popularity rankings so things like you know the stuff that comes up on the front page these tend to strip the the the the diverse and unique flavors out of some of our what we see and we get this sort of bland mainstream culture I think that's that's a sad thing and what will people say in or Python's 25 years old now what will people say in 25 years about what happened at Kiwi Picon will they say I wonder what Kiwi Picon was like I wish I could get onto that video but it looks like it's gone I mean I wouldn't be the first person to have gone back to have a look at a video a technical video and found that that site no longer exists these large corporations aren't good custodians of our cultural works and that's not to say they're doing it deliberately or there's anything malicious in there it's just that they're values are different and it's not to say it'll be a big bang like this it may just be the slow erosion of videos that weren't popular or videos that the author never didn't hang around to fight the DC may take down that occurred 10 years after they they published the video but the aim of media goblins not to replace enormous hosting websites with enormous hosting websites the aim of media goblin is to replace is to allow allow everyone to scale down to a node for their organization or their household or their community the idea is for it to be a non technical media publishing system where you can essentially pull this thing out of the box and stick it into the wall and start dragging your media onto it that's what we're really aiming for here and now is the right time for this sort of stuff because we have affordable low power hardware we have in many cases fiber to the home or fiber to the node so we can actually afford to start putting stuff in our own households under our own control and when you control the data you can control the terms of service which is great you can make a site that actually respect your code of conduct rather than someone else's so media goblin is a young but very ambitious project and the whole as I mentioned the idea is not to make it just for sys admins or you know hackers like me to be able to use we want everyone to be able to use this system we at the moment it's distributed which means that you can run your own instance of it but it's not federated yet and I'll explain what federation is in a second we've just released well not just but a little while back 0.9 was released and that has all the plumbing for this federation feature which I'm going to tell you about but the full user interface for federation is due inversion 1.0 which which is coming up sometime in the future it's underactive development at the moment so federation this is the the modern social experience the the stuff we really want to be able to like comment on run our own instance of media goblin and someone else is running their instance of media goblin we don't need an account on every single server we have this feature where we can comment or subscribe to someone let's an example say so this person here might like the media of this person and they subscribe to this this person so their servers they send a message to their server their home server consider it like email essentially so when I say home I don't mean a server that you're running in your home I mean the server that you belong to sort of you have an attachment to so this person sends a message through their client to their their server to say I'm I would like to follow in the in the language of activity streams this person over here that server makes a note of the following activity sends it on to this server over here and that server will probably send a message also to that person to say that they've been followed and then that server now knows to pass any messages back when there's new media showing up similar things work for comments so once once there's a following relationship this server knows that it has to notify back this way when there's when a comment is posted or new media is posted or stuff like that so the key takeaway here is that Federation is a server to server feature so we have the client to server feature already Federation is server to server it's messaging between servers and storage of different states across the across the network so activity streams is the way we do this both for the client to server and this server to server Federation this is based on the pump IO social network originally and this is a similar Jason blob that you don't really need to understand but gives you the idea essentially so Jason activity streams is a Jason serialization of this activity and it's saying here something like Chris the person posted an image to his followers and that message gets passed around the network essentially so it's both a command and the log language so this is sent as sent to Chris's home server as a command and it gets passed around the network as a log language as well it's as I mentioned it's a it's a an existing protocol so we're not reinventing the wheel here one of the other quite ambitious goals of the media goblin project is to standardize some of this social web stuff so this is a photo of Chris with the glasses and the above above the lady with the purple jumper Chris Webber core developer Jessica Tallin core developer and Evan prodromo up in the top left Chris and Jessica are media goblin developers and Evans the original pump IO author so they're working in the W3C social working group to try and get activity streams to the successor to Jason activity streams through as well as another standard called activity pub which is the protocol for how these social networks work so activity streams to is the new and improved serialization of these activities and activity pub is the the social network protocol so how how social networks will work and they've spent a whole lot of time looking at different social networks and trying to really bring something to the table it's really solid and they're working super hard and trying to get an implementation out so they can get to recommendation status which is quite challenging and they've quite time constrained and they're working really hard so go them I really hope they can bring it together because they're their tendency with the W3C finishes the end of this year so ambitious and progressive work how can you help you could write a plug-in you could as a developer you could write your own plug-in to do themes authentication a different media format that you like or a feature you might like like geocoding you can come and sprint with us I'll be sprinting tomorrow on me to go so feel free to come and join us I'd love to help you get a get a patch in and it's quite accessible and easy to do in in a day I think but the more important thing sometimes than contributions I mean we really value contributions is money media goblin has some really great developers and we'd love it if they didn't have to go and do contracting to to to pay their pay their bills we'd love it if these developers could work full-time on media goblin and build these great features for example the Federation feature Jessica is working full-time currently based on the the funding from the last campaign so that that's an example of how how your money is being put to good use by the media goblin project so media goblin is a it's we call it fiscal sponsorship fiscal sponsor is the FSF the US charity they they do the administration and the accounting and the paperwork for media goblin so that the developers can get on with the actual interesting work so I'd strongly encourage you to support by making a donation to media goblin and also becoming an associate member of the FSF if you're not already while I'm here I'll also give a plug for the software Freedom Conservancy they do amazing work in the free software space also a couple little projects you might have heard of G event Calicia Mercurial Piper sugar Swig build but busy but get homebrew inkscape QEM you sandbar Selenium wine they're the ones I thought you might know there's a whole lot of other projects they these projects have all chosen to share their administration rather than spend their valuable developer time on it so they they use software freedom Conservancy to do that administration which is a really great thing so please support them so media goblin is media hosting for artists so if you're a pixel artist you write research papers you do meetup recordings podcasts stop motion video you talk at conferences you take photos of food and or you're growing your very own hack a baby media goblin is for you so go make something amazing and host it on media goblin thank you all right we have time for like a couple quick questions yeah how easy is it to do backups as easy as is on any system I guess backups media goblin doesn't inherently handle backups like most systems it's your responsibility as as the person running the system to do do backups yes I was wondering does media goblin have much support for metadata being stored alongside files yeah so any plug-in can add whatever metadata it needs to the database so photos for example have you know location time the type of camera all that sort of stuff 3d models have the volume the size like the the center of the object what sort of stuff is that the type of metadata you're talking about oh yeah yep yep that could certainly be added so last question so I'm a consulting software engineer and I do user experience development as well so I'm always interested to talk to interesting people about interesting projects so feel free to get in touch if you if you ever looking to talk to the media files gets served with a streaming protocol was straight-up HTTP good question normally you would run media goblin behind a proxy such as nginx or Apache it would probably depend on how you configure that proxy as to whether that was available yeah all right thank you very much so we've got the closing ceremony