 Hello and welcome. Okay, I'm too loud. Sorry for that. At least you're awake Welcome to the next talk at EMF camp and with me on stage Is Matthew to talk to you about our latest and best hope for secure federated Communication, please give a warm round of applause to Matthew Thank you. Hi everybody. Can you hear me? Wonderful perfect brilliant Thank you for coming along to hear a little bit about matrix I guess before I get too deep into it I should probably find out who knows what matrix is already and who doesn't show of hands for people who already One with matrix actually not that many people about okay about 10 11 people So I'm afraid the next bit is gonna be really boring for you And that we'll have to go over some of the basics and to bring everybody else up to speed First of all, I'm Matthew. I'm the tech founder of matrix We started doing this about four years ago to build a whole new decentralized communications network For the internet and the idea to be end-to-end encrypted and completely decentralized So that no single power can have any authority over the network So if you're talking to somebody using a different system on a different server, the conversation is completely Replicated over those two servers and this is the important thing that everybody needs to understand about matrix that IRC or XMPP or SIP or whatever has not done The matrix is basically very similar to get it is a entirely decentralized system And if I fire up a server on my Raspberry Pi and start talking to other people Even if it might be I don't know AT&T or a government or something the copy of the conversations on my little node are mine and The data is replicated with cryptographic integrity over the other servers So there is no single point of control at all. So I mean that's basically the spoiler Main headline here of how we are liberating communication and giving power of it back to the people so in practice Matrix it's an open network and we're doing secure decentralized real-time comms And that's for any kind of communication the typical thing we waste all our life doing is instant messaging interoperable chat But one can also use it for VoIP and you can use it for VR and AR communication You could use it for world data and geometry and VR and AR You can use it for IOT similar to the previous talk if you wanted to have a completely open Fabric which you can publish and subscribe to real-time any old data And in fact one of the early demos we did with matrix was to hook up a car and its ODP to port Intermatrix have it rattle around the network and then do visualizations in a totally different system built on top of it And so why are we doing this? Well, it's a bit of a crazy moonshot to build a global decentralized and cryptic comms network Which is really as ambitious as the web Except optimized for real-time communication because the web was always meant to be read write It was always meant to be as easy to publish as to consume and it all went a bit wrong back in the web 1.0 days and nowadays it's still pretty wrong and if I want to publish stuff on the web I have many different ways of doing it whether it's Facebook or FTP or Twitter or Mastodon But there is no kind of Common language and other than possibly DAV and DAV failed to go and publish data out there So with matrix we just want to say hey, I've got a little bit of JSON data I want to publish it to somebody else in real time here is a simple HTTP API that is standardized for doing that I guess activity pub and activity stream have come along in the last couple of years in a similar space But we are coming at it Almost from a simpler. Well, I'll show you the API's you can judge how complicated it is By the way, if anybody has any questions or thoughts, please just interrupt because it'll be a lot more interesting than me Just going blah blah blah blah for the next half an hour Also, if anybody wants to steer things in a particular direction like what about spam or What about encryption or what about VR then again just yell it out and we can Steer the talk as you might wish or not I mean practically speaking matrix looks like this or doesn't look like this more accurately because this is where we are I'm today with all of these some kind of silos like slack or telegram get a NIC network Or even an application like github and the point of matrix is to be a glue that some connects them all together a Decentralized network of servers connected in a full mesh topology today at least and you have these blue bridges Which take you through to other systems you have native clients living on the matrix side And so you can use it in the bridging model So that somebody on free node can go and talk through to somebody on a given slack by bridging it through matrix Or alternatively you can just ignore the outside and have a native matrix client like riot Talking to I know another native matrix client say like see glass sitting on another server But the unusual thing here is as I said earlier that the conversations are replicated over all of the servers So there isn't any single server that can go down and take out the conversation There is any there isn't a single server which acts as a single point of failure Instead, it's just like cloning a get repository over all of the different nodes on the network made sense Yeah, perfect. So just to ram it down again. No single party owns the convos. They're shared over everybody Some people are probably saying, ah, how is this different to x and pp? Why have you reinvented x and pp? You're a horrible person. You're going to hell anybody's thinking that No, well one person is wondering why we reinvent We vaguely wondering why we might have reinvented x and pp and the answer from my perspective is that we haven't a completely different Philosophers and architectures in x and pp. It is a strictly federated some protocol Based around routing blobs of XML over a network and it does a very good job of it. Whereas matrix Primary building block is conversation history. It's not messages. We're synchronizing the history of chat rooms or whatever The data is across these servers. So if anything, it's more like couch DB or Cassandra We're one of these eventually consistent object databases. The fact that you can do pub sub with it is obviously fundamental But in the end or it's a lot more like it that we're going and replicating the history around the place Group conversation is the first-class citizen. You do not get one-to-one messages in matrix You get rooms now for room has two people in it Then you call it a one-to-one conversation But there is no distinction between a DM and the PM or a group conversation otherwise at the protocol level Also end-to-end encryption We designed from the outset it took us a few years to implement it and technically it's still in late beta now But it's still there as a kind of fundamental assumption that sometime real soon now We will turn it on compulsorily for all private communication on matrix Also, we use HTTP and Jason as the baseline API You can use other transports too if you're allergic to HTTP and you hate Jason You do not have to use it in matrix However, the lowest common denominator Stupid simple thing that we provide is a web API so that to send a message is an HTTP But and to receive a message is an HTTP get and that's it now It's inefficient, but anybody can do it. You can literally send messages and receive messages using curl And finally our big difference to X and PP is that we're all about defragmentation and bridging We're not trying to build another ivory tower or another sort of castle in the sky Which is going to be the one true communication network instead We're completely pragmatic and just trying to bridge to everybody else provide some glue in between them and try to Defragment the silos which exists today now I'm not gonna bang on too much about the problems inherent in silos communication because I'm assuming I'm preaching to the Choir and everybody agrees that it's a pretty bad thing that Humanity has become helplessly dependent on Facebook and WhatsApp and Twitter as their ways of communicating so let's just take it as a given perhaps that Isn't good idea to empower people to control their own communication pick their communication provider be able to migrate between their communication providers and have a completely open standard open source non-proprietary Non-encumbered model and for doing so. I don't know did anybody see the thing about better slack a few days ago One person two three four, okay So some guy went and wrote a browser extension to make slack cyclists and he called it better slack and I forget the features, but they were pretty cute and he had to reverse engineer the minified Slack source code in order to do fun stuff with it and he announced it on Hacker news And it got like 800 upvotes and everybody thought it was the best thing ever and two days later slap legal Go and shut him down completely For daring to reverse engineer their code and undermine the integrity of their products and blah blah blah blah blah I'm happy to say that he's now looking at matrix and right But it was this classic example of the problems of working on a proprietary system where they suddenly enforce their terms and Conditions on you and you're screwed even if you've done something really useful and really nice It's I kind of wish you'd been building on an open source platform and an open standard in the first place and now he is so Architecturally on matrix you get home servers these guys feel and look a little bit like email servers You run them on your VPS you find a provider who can provide them to you And they contain your accounts and your conversation history the clients are very thin as I said It's just plain old HTTP talking back and forth here and it's a very asymmetric protocol the clients are really stupid and simple to write Literally, it's a curl one liner the servers on the other hand are having to do this eventually consistent object database replication thing to go and Replicate the conversations all the way around the place so they turn out to be quite hard almost like a database implementation Then you've got application services, which are like clients with super user access to the server so they can masquerade users They can masquerade rooms. They can act as bridges. They could act as know any kind of filtering or And they're logging or whatever service and then finally we have identity servers Which are kind of the odd one out because we haven't solved identity yet in matrix What we do is to identify people based on their email address or their phone number or anything else And then we map it through to their matrix ID Which is intended to be opaque at the moment the that lookup directory sits in these Logically centralized services. We're hoping somebody is going to fix the problem of decentralized identity And we'll just move over to using it when that exact happens But until then it's kind of the weak part of the architecture This is the fun bit in terms of where things are at today our main deliverable is the matrix spec which is a Increasingly scary single big document and one of the things which we do differently to say XMPP or others Is that we just have a spec? It's not a bunch of extensions. It's not a bunch of optional add-ons and chapters It's one big document that gives you everything that you could want to do. It's got VoIP It's got end-to-end encryption. It's got read receipts typing notifications all this and sort of thing and the advantage of having all in one place It's that if you have a server and you say it speaks matrix 0.4 It does you don't get the fragmentation. You don't get that. Oh, I'm not sure where from my clients implemented end-to-end encryption yet Etc. If it's a compliant client We have a baseline that defines precisely what you need to do to speak matrix historically, this is being quite Unloved should we say because In practice trying to get the right juggle between doing reference implementations and then making sure the spec is really really good It's a nightmare because it at least doubles the amount of work in order to Also get consensus from the community that we're adding in a new feature into the spec and all of this kind of logistical thing But the good news is that in the last couple of months We've actually been working almost full-time on the spec and it's got a lot lot better And so if you have ever had the misfortune of looking at the spec in the past and thought wow, this is a bit shanky Now you can Look at it as of last night and see stuff. Hopefully looking a lot better Now talk about it in a bit on the implementation side at server side We have a Python server called synapse which was the original prototype and Irritatingly it's still the main one out there. It sucks in many ways. It's a resource hog We haven't optimized it as much as we would like but in practice it works well enough On the matrix org server we have about 30 40 files and some concurrent users at any given point so it is possible to go and scale up to that sort of level and that's over a couple of million accounts and Then we have dendrite, which is the sexy next-gen server now dendrite's written in go and it uses for its own Microservice architecture that goes and scales out completely horizontally it shamelessly rips off how hangouts works inside and Should in theory be amazing other than the fact that we've had to Take people off it to focus on synapse because there are so many synapses out there and it's become quite mature That we need to keep supporting it and it's been this classic second system syndrome a bit like Netscape Versus Mozilla or whatever back in the day where eventually we will move over to dendrite And it's going to be incredible, but we keep having keeping delayed by keeping the current stuff propped up That said synapse is usable today. It just uses more RAM than you would hope or expect We also have application services and bridges through to loads of different Platforms whether it's slack or get it or telegram or whatsapp or iMessage or basically anything out there either we or the community of Written a bridge and the purple stuff is the community the green stuff is the core team on the client side We give completely separate stacks on Android iOS and JavaScript The lowest level is the web API wrapper the next one up is the UX UI components And the one above that are applications themselves written on top of it and the team also writes the riot app Which is a kind of flagship app because one of the things we think went wrong in the past is Protocol open protocols often don't have a flagship application My XNPP has never had until in some ways recently on Android you have conversations But before them especially cross-platform there hasn't been the obvious XNPP client. Everybody should be using So on matrix we wanted to build one and we called it riot and it sits on top The cool thing about having totally native stacks Here is that if you want to embed into Android or iOS or the web You've got many many different layers and much to go into it at and you're not having to use react native You're not having to react use some kind of device abstraction thing the bad news is that for us We end up doing triple the work that we should do in any sane world because we have to maintain completely separate code bases overall three platforms Enough talking. Let's actually look at it. Oh, yes Oh, so your microphone is winging its way to you on on angel wings a A question Yeah, sorry to interrupt. I didn't see anything about embedded platforms like microcontrollers and things Is is that just because it's straight HTTP and you don't really have to or I don't know do you have to comment on that? So on the server side It's not well suited to embedded platforms because the servers today are chunky the next-gen ones should be much better client side You can absolutely use them on embedded platforms and in fact, there's an ESP 80 to whatever that processor is base platform In matrix and it's like 20 lines of code because luckily it has an HTTP client that works And let's face it most things have an HTTP client these days So it's not perfect because it's HTTP, but it's certainly good enough to control a de fruit some LEDs I know a guy who has his entire house lit by matrix And and that kind of thing but it's more of the proof-of-concept face That's right And can you see this this is my personal right web? sitting in one of our dev rooms in synapse dev It's worth noting that matrix scales quite well. I'm in like 1400 different conversations here and I don't know if I get into a room like matrix HQ. That's got 1,600 people in it at points The rooms have gone up to 20 30,000 people so relative to something like slack We're just in a whole different order of magnitudes of Scale and complexity we have things like read receipts hello world and Typing notifications, of course. Hopefully I have enough internet connectivity to send a message and Hopefully you'll see that people's faces start to tetris grid down the right-hand side as they read my message Why doesn't slack have read receipts like seriously? This is basic stuff here and it's incredibly useful to have this Rich presence effectively of saying precisely and what's going on? What else can I show you in here? A good thing to do might be to actually look at one of these messages So if you look at some foxes saying hello stream you can always look at view source and it shows you the underlying Jason and here It's pretty simple Typically, it would just be plain text In here is also given HTML because I happen to know that he runs his own client Which presumably isn't doing the optimization to only send plain text if there isn't any formatting and You get your timestamp you get some the room that it's in and the type of message and this is completely extensible You can put any Jason you like as long as you name space it to a given type into the room So a slightly more interesting thing might be I know is there an image or something? Reality gaps are asking to be heckled. This is very sad Oh, there we go. Oh, this is very better, isn't it? I hope everybody appreciates the inception that's going on here so if we look at this slide from the talk and View source on this then it's a lot more interesting We've got a URL of an image here using the MXC URI schema, which is a decentralized content repository that we haven't made tricks. It's very simple It's just a full mesh of HTTP Hits so that I can quit so the people basically sync over HTTP the message Sorry, the media of the message and it works well enough You have metadata of the width and height and size and the mime type, etc Perhaps something more interesting would be an end-to-end encrypted room if you go into something like mega on test this room has got 193 people in it and we use it for stress testing the end-to-end encryption Because there are thousands of devices in the room typically we see about 10 to 15 devices per an average per user in matrix and If we go and look at one of these guys the source itself has got the ciphertext in it and Then you and this is encrypted using mega on which is a ratchet that is led on top of ohm Which is a clone of the double ratchet that signal users So the way the encryption works is that you set up one-to-one ratchets Between all of the participants in this room So in this case, it's a full mesh over a thousand devices for the one-to-one ratchet But then over that you share the key data for the group ratchet called mega on so you only need to send one copy of the message It has one session and hopefully everybody is synchronized their keys such that they can decrypt that particular Message so it's a kind of hybrid between signal and a slightly more pragmatic and group ratchet-based approach And I'm waffling and running out of time. Let me show you a different client. Let's go and look at another one So what I've got one here called neko, which I built last night, which is hopefully still gonna work Yep, yep, yep sure it will be enlarged like that Okay, so this is neko starting from scratch. That's the entire thing booting up on a test account with like hundreds of Conversations going on it's written in QT by the community led by a guy called Mugex in Greece and is a fully native QT app it has end-to-end encryption in it as of a few months ago and It's looks a lot like telegram as you can see and it's just really nice I mean obviously I use riot because I work on riot but an increasing number of people are now using this as a good native client also excitingly from my side is Sea glass, which is a new client and for Mac OS which actually uses the iOS SDK let me just fire up a copy here, and I just have to build it. No, there we go and See glass. Yeah, it's built on top of the iOS SDK, but otherwise there's a completely native Cocoa app GPL all open source, of course It's just logging in as the same user. We were looking at in neko It's not quite as fast to load as the other one all the logging flies past Is it running in the background? Oh, yeah, there it is perfect So this is what see glass looks like actually it's kind of similar to neko other than the fact. It's not QT It's a can confuse everybody and look at the neko room from sea glass or we can go and look at the sea glass room which might make more sense and You can see it's a completely native Mac and app looks really sexy on my Harvey. Oh We'll have you pronounce my half my Harvey my Harvey I don't know the the dark theme that you get on the new Mac OS But it's really fun to see these really good clients coming to light This one also does end-to-end encryption and thanks to piggybacking on the iOS SDK that we built for riot All of these projects would love contributors So if you are a QT hacker or a macOS hacker or whatever environment hacker go find a matrix client of your choice and make it amazing Meanwhile back on the slides I've demoed it to you What do you get all the stuff you would expect and the kitchen sink? I don't think there's anything obvious missing apart from reactions, which are coming real soon now And I think there was something else or edit editable messages both of them turn out to be harder than they should be I'm gonna skip how it works, but I'm running out of time the actual API looks like that if the font is more large enough For people to read you're literally doing an HTTP post if some JSON to a URL like that You if you're doing a web RTC call then again, it's a single HTTP hit to offer the media that you want to talk and You basically offer start the call offer some media the other guy answers it and then you're in the call So if you've ever had the misplacer of doing set or VoIP by any other technology This is basically the simplest possible approach you can use You can put some MIDI onto it. We did jamming over matrix It's not very good for the really low latency stuff like MIDI, but we recorded the jamming sessions into matrix and notated it which was fun Bridges look like this. You just got the bridge and the third party thing So people might know the xkcd where Randall said I have a hard time keeping track of which Chatsystems my friends use and we literally drew the bridges that exist for the existing things on this into matrix at the time and We gave a node stack with a JS layer and the nodes in order to bridge through to IRC or Slack or anything that live purple can speak And so you end up having three nodes bridge in the way or slack or like that and that should say slack not IRC You can even do it to things like our T devices and this was a really fun demo until we broke the drone where you would take the proprietary API that the drone spoke and Took its video feed and took its telemetry and its management It's actual fly-by-wire stuff and put it into matrix So you can take a completely generic matrix client do a video call to the drone and Then at that point you literally would type launch and up and down and left and right and the thing would fly around until it crashed into the audience What else do we have here and to end encryption I kind of glossed over that New stuff relatively are communities and widgets and so communities Allow us to filter these massive sets of rooms down to the ones you care about So it's a bit like Slack teams or discord servers I'm now just looking at one at rooms which are to do with the core matrix team I know wants to do a G-SOC or wants to do with GNOME or whatever it might happen to be Also widgets are really fun in that you can take Any old room trying to think of a good room to demo in Perhaps I'll just do one in matrix HQ So if you go into this you basically have an app store of bots and bridges and things that you can add in And a fun widget might be something well It could be any HTML page it could be Spotify could be a YouTube it could be Grafana, etc Jitsie is probably a good example But if I actually press that button there then it will go and embed to video conference as an HTML widget Into the conversation and for better or worse everybody in matrix HQ will suddenly see Hopefully the conference pop up like that in if their client supports widgets And it's just a way of kind of coordinating iFrames across the people, but it's surprisingly useful Let's see if anybody joins us. I hope they don't It's surprisingly useful in terms of basically building a dashboarding system into any chat room So if it was an ops room imagine that you had like Grafana and you had I know some other dashboard in there Oh, there is somebody else out there. Hello, whoever you are and look as if you're in your room But this room based on the ceiling We can go and try to crash the Jitsie and see what else it can do anyway Kill that now. Otherwise my laptop will probably run out of battery That's an example of a widget and I'm very nearly out of time. What else should I say? Community status. Oh, yeah, France has adopted matrix on a kind of state level They've gone and rolled it out across 15 of their ministries now. They're aiming for 35 by September end of September They've written their own matrix client, which is a very very cute kind of totally end-to-end encrypted approach for doing this and they've also Done a lot of interesting stuff with antivirus and the other enterprise these stuff You would expect if you were a government running on top of matrix if you happen to work for a government here And you want to not be using slack Mr. GDS Then please come and talk to me afterwards because we'd love to help you out Because it's a bit amusing that a UK based project like matrix has ended up being adopted by France, but not the UK Terms of growth over the last couple of years users keep on going up. Yay Traffic keeps on going up other than the encrypted traffic which has grown a lot more slowly because the UX sucks We're doing everything we can to fix the UX at the moment and there's good stuff coming very shortly Number of servers out there is actually now it's about five and a half thousand So not quite as big as the web, but hopefully we're getting there and the number keeps going up Matrix 1.0 we've been in beta now for four years We were trying to get a stable release of all APIs by the end of August as of 3 a.m This morning we got all of them out apart from the Federation API So this is a massive massive one previously. It was just the CS API As I said, we've been doing lots of work on the spec You can see that we first wrote the spec back in 2015 Then it kind of trundled along and then in the last couple of months We've been right back at it to try to get it to a 1.0 The server server stuff is nearly done. There's 26 issues left nine in review Lots of emphasis on security But at that point hopefully you all hear that matrix is out of beta and we're finally completely launched All new right on the horizon looking a lot less green a lot more lights like or discord hopefully a lot better Also on mobile looking like that Lazy loading is going to reduce RAM by a factor of five, which is nice We need to get end-to-end out of beta need to do other things which are missing and we need help Please run a matrix server. Please give us feedback follow us on Twitter and thank you very much Perfect timing. Thank you very much. Sorry for having to rush. We can do we can do a few questions Because I want to test this new tool So can we turn it up? Amazing. It's some kind of flying microphone. Yeah, this is really good Closer to the mouth. Yeah, I was wondering about I was wondering about Get and or get an issue tracking integration because I'm aware that I think githlab are looking starting to look at activity pub for that and I was wondering whether what the Yeah, what the capabilities of matrix in that space were and what's planned our capabilities kick-ass when it comes to Issue tracking integration So if I go into an internal room like this one where everybody's getting excited about the spec releases ID slash the exclamation market hub create Hello world then it will go and log in with my actual github credentials into the github and create that issue On the default project which happens to be right web as me as my actual thing and likewise you can go and webhook Everything in from github to so we live by it and it's not just github. It's got bit buckets and all sorts of other things, too If we keep if we keep it short or we can do two more We move it outside for further follow-ups and have you got any news about the key verification UI change? Yes Absolutely, so I completely skimmed over that slide But it was one of I think it was even the top thing of exiting a beta friend-to-end encryption first of all you got incremental key backups, which the PRs have almost landed which Makes it a lot harder to lose your keys then the new verification We've got QR codes and comparing the monics both of them are fully designed fully spax We have the UX workshops on Monday to actually lock down what it looks like and then we're going to implement it before we go mad I'm sorry that it sucks for so long, but we're going to fix it So the short question part works already the short answer part we work on last question, please Thank you very much for the talk. I was wondering if you could tell us your thoughts on machine-to-machine communication through matrix We want somebody to do it. We haven't been focused on it It could be used very much for that HTTP isn't great for it better transports will be amazing Please do an MQTT transport or a co-app transport Also, it helps if you need the encryption because otherwise it might be simpler to just use MQTT But if you need federation or encryption then use us Awesome. Thank you very much. Please give it another warm round of applause for Matthew