 All right. Hi everyone and welcome to this talk. Enter the matrix. My name is Brandon Obolivier. I'm a software engineer at Element and also a core team member at the matrix.org foundation. We'll get more into what those are a bit later in the talk. I'm also a core committer on Synapse, which is the reference matrix implementation, which is what we're going to install later in this in this talk and you can see at the bottom of the slide my contact details if you need to reach out to me after the talk. So I've been talking about what in these first minutes of presentation I've been talking about matrix, matrix and servers and so on. But there's probably people among you who don't know what matrix is. There's something we usually like to do at physical conferences, which is a show of hand and who's already heard about matrix who's using matrix who doesn't know matrix, but still uses it but uses it anywhere. And that's kind of gives us an idea of whether we can keep the boring part of the of the presentation obviously for obvious reasons we can do that here. So I'm just going to go ahead and explain to you briefly what is matrix. So matrix is an open network for secure decentralized and real-time communications. That can be used for in terapial in terapial chats. So between different clients, different servers, sometimes different chat platforms entirely. We'll get a bit more on that later. It can also be used for VoIP, for virtual reality and augmented reality. There's a nice demo that was done at first time a few years ago about that. And it can also be used for IOT. And this platform has a mission. This network has a mission. It's to create a global decentralized encrypted communication network that provides an open platform for real-time communication. And to fulfill that mission, that goal to work towards that, to work in that mission, that network, which is matrix, is defined by an open spec which comes with a bunch of very interesting features. You've got group messaging and one-to-one messaging. You've got end-to-end encryption using a library called OWM that implements the double-ratchet, well, signals, double-ratchet mechanics. You've got VoIP signaling for VLTC. You've got authentication search, 100 counts, content repository for media, files, etc. It can data to store basically random user data. And the most important of all, I think, is decentralized conversation history because in matrix, no single party owns your conversations. All the conversations are shared over all participants. What that means is if you're in a matrix room, which is a chatroom on matrix, no that room doesn't belong to one server, like the server who created it or the server of the admin of the room. It means that the room is a decentralized entity and all of its history is shared with all of the servers participating in that room. So all of those features are offered in four major APIs. You've got the client server API, the server-to-server API, which we also call the federation API. You've got the application service API that's used to create bridges or advance boats. You've got the identity server API as well, which is which allows you to create servers that can act as address books for email addresses or phone numbers that are associated to accounts so that you don't, you can search someone with their email address, for example. It all fits into that kind of distributed architecture, which you can see here and so you can see, for example, the identity servers, which are the one implementing the the identity service API, which I was talking about. You've got the application servers, which are bridges to other platforms or more advance boats or stuff like that. And then you've got basically server and clients, which are the which, so each client talked to the servers with the client server API and each server took to other server with the federation API or server-to-server API. So Matrix is very much for now on the client server architecture with federated servers but we're also working on peer-to-peer matrix so that you remove the need for servers entirely. The client service API, the client server API is like basically, like all Matrix APIs are just JSON of HTTP and so sending a message in Matrix is actually very simple. You can do that with just a simple curl command with so you can see here, you provide an access token that's provided at login or registration, and you just send the content of your message, so that's a text that says hello, and you send that to your server into a room, which is that decentralized entity that I was talking about, and you give it a type. So basically an event, well, the time length of a room in Matrix is organized by, well, in events, which have types and, well, each event is a blob of JSON with a type and contents, basically, and it returns an event ID. And so what I've shown here is how to send text message, but Matrix can do much more than that. And I mentioned IoT before, so for example, if you wanted to control a view light, you could send something looking like that. So that kind of event is not mentioned in the spec, is not described in the Matrix spec, but it is also something you can send because Matrix just accepts arbitrary events that you can define and process as you want, basically. So that's basically something very similar to the previous example, except you're going to have another type, which is arg.matrix.midi, and you're going to give it a very specific content, and it's also still going to return the same kind of payload, the same kind of response with an event ID in it. In terms of the other APIs, so they're all very similar in that they're all JSON of HTTP or JSON of HTTPS, generally. So you've got the server API that synchronizes messages and rims state between servers in real time, and so that also allows them to retrieve history from each other and query a bunch of things from each other. So that's basically the API that allows servers to communicate between each other. And you've also got the application services API, which is a kind of privileged version of the client server API, and can, so for example, they can perpet, they can perpet some users, they can create virtual users so that if you're in a room that is, for example, bridge to ISC, you're going to see other ISC people as if they were matrix users. That gives you a bit of a peek into the matrix, the general matrix ecosystem, which is, which is pretty, pretty wide and pretty, and very interesting to look into. So when you look at how matrix works in general from in terms of the interactions between clients and servers and how you can build that and what you've got in the, in the whole ecosystem, you've got a bunch of, a bunch of servers. So you can see here made a synapse and then try to synapse is the matrix server we're going to install a bit later in this talk. And then right, then right is another kind of a second, second generation home server for matrix that is being, that is still, still in the development and you've got all the abscesses and bridges. So that's very much the server side. On the client side, you've got some, well, a lot of, a lot of libraries for, for, for, for Android, for iOS, for JavaScript. So that's, those are the ones the foundation maintains, but you've got a whole lot more on the, on the community side. And you've got a lot of, and so you've got, and so you can use that to build the clients, for example, element, which is kind of the, one of the flagship clients for, for, for matrix. But you've got also a whole lot of, of other clients, you've got Quaternion and Niko and QT and C++. You've got Fractal, which is under the umbrella of the GNOME project, which is, and it's, and it's written in REST. You've got Thunderbird currently working on an integration of matrix into Thunderbirds, GOMEX, which is a T UI clients written in Go and Flutthichat, which is a very, very nice looking clients written with Flutter. And of course, and so to kind of harmonize all of that, those the client side and server side, you've got the specification, in this case, the client server API, the, in case of application services, it's also the application service API and stuff like that. So you've got clients available for every platform. And so you can see, you can see that on this, by clicking on this, on this link, you've got a whole lot of very interesting clients for, for a lot of platforms, very good looking as well. And if you want to build your own, you can use, you can use one of the many SDKs. So, official, so when I say officially, it's as the matrix dot org foundation, we maintain four of them, JavaScript one, React one, an iOS one and an Android one, the community is built a whole lot of other SDKs for other platforms. So you've got one go in Python, Java, Ruby, Rust, you've even got one for the Nintendo 3DS, one of our committee member built a client matrix file for the Nintendo 3DS, which was pretty well and, and made an SDK out of it. You can, so you can find, find them by going through this link, I'll send the, send the slides through, through the, the conference's website. So you can download them and check out and check it out yourself. In terms of home servers on the other side of the spec, in a way, you've got two official as in maintained by the community, the matrix dot org foundation. We've got Synapse, we've got dendrite and we've got a few, a few ones, maintained by community members. So concrete, which is written in Rust, construct in C++, and you've got a few, a few more. In terms of the global status of the community. So as of right now, we've got about 8 and the 19 million visible accounts on matrix, sending, sending a bit of a five million messages, messages per day, across over four million chat rooms. And so, and they're all distributed over 45,000 servers, approximately. And, you know, a lot of, a lot of bandwidth, a lot of, and a lot of projects building on matrix and also even companies building on matrix and governments using that as well. Currently, the French government is deploying a matrix based solution, as well as the German government and, and, and others that are more in the building in the conception phase. Right, so now that we've gone through that whole chunk of information, let's install the home server. But wait, what's the home server? So that you've had me use that a bit before we're going to explain, I'm going to explain now what is like what is what concretely is in matrix, you've got roughly three types of servers. You've got an identity server, which is, as I say, it's kind of address book of email addresses and phone numbers associated with matrix accounts, you've got application services, and which are bridges and more advanced books. And you've got home servers. Home server is basically going to be the home of a matrix account. It's going to implement this, both the client server API and the federation API. And that means that's a client that connects to, well, clients and both clients and servers are going to connect to a home server using their respective APIs to send and receive messages. Also, I just wanted to mind shed a bit here. When I say the home of the matrix account, that's how it's currently is. And we're also working on portable identities on on matrix, so that your main your home server, the home server you're signed up with doesn't have to be the only home server you can, you can use your account on it doesn't so that it doesn't have to be the only home of your of your matrix account. So the home server we're going to implement again is well, we're sorry, we're going to install is synapse. It's there. It's a reference implementation. It's open source on GitHub under the matrix dash org organization, just on the synapse repo. And the plan will be for us to install synapse using the official Arch Linux repo, configure synapse and also configure a reverse proxy at the end of that. So let's do this. So let me switch to my terminal, get that a bit bigger. And let's just start with pseudo tag man as matrix synapse. So we're going to install is going to install a bunch of dependencies as well. And we're just going to wait for this to finish. Right. So it's finished and it's done a bunch of installed us a bunch of things. So currently, so the current package on our next, which by the way, isn't made in by us, but by the very nice Arch Linux community, or the very nice Arch Linux trusted users. And thank you very much for providing that package. So that package doesn't generate a configuration just on the fly. So what it does, what it does is tell us that we're going to need to do that. So it first, first needs, first tells us to go to that directory. The slight issue is we can't access it as a friend. So I'm just going to, to show that it's very temporary. So that's, so that's, we can access it to, to, to do what it asks us to do. And so we're going to, so if we're, so we're going to first look at what that comment is. So the first, so on the first line, we, we see that we're using Python. So synapses, we didn't really mention it, but synapses written in Python, using the, using Twisted as it's, well, using the Twisted framework for, for networking and also a bit of a synchronous, but we're slowly migrating to, and I think we've almost entirely migrated to async away, to the async here, async.io Python call library. And so we're going to use Python to, to start synapse. And we're going to give it a few, a few more things. The first one is we're going to tell it to generate the config, because we don't have a configuration yet. So we, that's what we want to generate it. We're going to give it a config path. We're going to tell it that, yes, we're, we're happy to, to send it, to send it some usage stats, which are very, very, well, which are completely anonymized. It's just, so the, the point of that is, is like, it's not going to, your own server is not going to send us, for example, the complete list of members or whatever. It's just going to send us some very broad aggregated statistics so that we can make some decisions when planning the development of synapse. For example, it's going to tell us your Python version and your database version, so that we can know, for example, well, Python 3.5 got, well, rigid end of life last month. And, and so we're looking, we're looking at those graphs to see how much home servers in the wild are using Python 3.5 so that we can decide when we can drop support for it. So that's an example of how we use the stats. And we're going to use, and we're going to provide it with a server name. So the set, so in matrix, all of the, everything has an ID. So the user has an ID, the room has an ID, the group has an ID, an event has an ID. And in most of those IDs, you're going to use, for different reasons, you're going to use the server name, which is going to be the identifier of your server. So for example, if my server name is example.com, sorry, it has to be a domain name, a valid domain name. So if my server name is example.com, for example, my user IDs are going to look like at Brandon on example.com. So they're going to look a bit like that. So we're going to, so it has to be a server name, well, it has to be a domain name that you own. It doesn't have to be, we'll see a bit later that it doesn't have to be the name of the domain name where you're hosting your own server. So we're first going to cd into there. And then we're just going to, so I'm just going to open this very quickly, open gd very quickly to edit that. And we're going to, and we're going to change that into my domain name, the main name that I've got for this demo, which is arch.com2020.demo.tablity.bzh. And we're just going to, and we're just going to, to run that in all terminal. Tidix is very nicely asking me if I'm happy to run a Cd command. Yes, I am. And let's do this. And we can see, and it's telling us everything that's everything it's done, it's been doing. And so the main config file, the one we're going to, to look into is, is Hamsa, it's called Hamsa.demo. So let's, let's have a look here. That's right. So we can see that it's already pre-filled a bunch of, a bunch of things. So the server name was pre-filled from the command line arguments. All of the, all of the path and files and, and all that are pre-filled with the current path where, when you're running the command, which is why we see it into, into, into, into varlib synapse. So we're going to, sorry, I'm getting my notes here, so by time I don't overlook anything. So the first thing we're going to do, and that we're going to talk about is, is enable registration, which is right here. So by default, registration is disabled on synapse when you install it. That is because, that is basically for moderation purposes. We've got, in the past, we've, we've seen a lot of spam from big public servers on Matrix and, and we've seen people just basically crawling every Matrix server they could see to, to, to, to see if they had registration open and if they could create an account to spam other rooms. So just, so if you're setting up your just private server for you and your friends or you and your family, you don't need to enable it or you can just enable it just for the time so everyone can create their accounts. But if, yeah, my, my point is really, if you want to enable it, just make sure it's, make sure it's actually what you want to do. So we're going to enable it for this demo just so we can test our home server later. We're also going to, so we're also going to look at the database and actually I've got a slide on that. Synapse will use an SQL database because it's the easiest to, to, to set up and basically the easiest if you just want to set up Synapse and just start it up and see how it goes. We recommend Postgres for prediction though, for performance, for performance concerns. And in the future we're going, we're even going to deprecate SQLite for federation. So that means that you won't be, that in the future you won't be able to use federation. Well, you might not be able to use federation if you've got, if your server is using SQLite because we're getting a lot of support requests because, and performance if you report because of people using SQLite. There's, and there's some dark hair if you, if you needed to set up, to set up Postgres. So we're not going to change that right now just because we, we want a quick and easy config. And I think that's about everything we needed to do. Just, just a very quick, very quick notes on the, on the log config. The package that you install provides a file called logconfig.yaml which is not really, which is a bit outdated and not really usable with, with the Synapse. By default, when you run the generate config command, Synapse will automatically generate its own log in configuration. So that's which is so the file in which you configure how you want Synapse to log. By default, it's going to, to just log in this, in your current directory. Well, in the directory you were when you created that doc, that's, sorry, that, that configuration file. And you can obviously, obviously change that and, and define a lot of, a lot of stuff. Those are, this is a standard Python login configuration. So feel free to, to, to look at that if you want to, to see a bit more on, on how, on how to configure it. Right. So another, so last but not least. So we're just going to, so first we're going to start Synapse. And it started correctly. It's giving us a bit of a warning, but that's more of a, not really advanced, but not something I have time to, to touch on in this, in this presentation. So Synapse is, Synapse is running. It's running on, it's running currently, let me just go back up here. It's, it's running on one listener right now, which is on, which is a local host listener on thoughts, 808 with just pure, pure HTTP. The reason it's just, it's just on, just running on, on local host is because Synapse, well, the whole, the whole of Matrix now requires, requires TLS for Federation. And, and so up to, up till Matrix 1.0 in June 2019, we were using critical perspectives to validate TLS in Federation, which was, which was a project that's allowed you to say, allowed a server to say if that many servers trust that server certificates, I'm also going to trust it. From, starting from Matrix 1.0, we now require a valid TLS certificate. And, and, well, I, a certificate validated by a certification authority, such as let's, let's some trips. And, and so you usually want to do that with a reverse proxy. Just as a quick parenthesis here, Synapse does have admissible and it actually does have the configuration to, to, to provide, if I can, if I can find it. Yeah, it even has the configuration, yeah, here it is, to, to serve HTTPS itself with, with the TLS certificate and the TLS key. However, you, so it's, and it also has built in ACME support, which is using TLS's TX ACME library. The issue with that is that that's library as of right now only supports ACME v1, so the v1 version of, of the ACME protocol, which is currently being turned off by the centripet. I think it's only available currently for, for existing domains on existing accounts. There is some work in progress on Twisted site to support ACME v2, but the progress is, is slow. Twisted is an unprofit organization powered by volunteer who might not have the time to, to, to work on that. So currently we, we still have or ACME support, so CNAP still has ACME support, but we don't recommend using it for new servers because it's not going to work. Instead we, we suggest, we advise using a reverse proxy, which would reverse proxy the locate, the local host listener that I showed, that I showed you before. On this demo I'm using CADD, because I'm familiar with it and it's, it's got automatic TLS, so, but other, other options definitely work. So you, on here you've got a link to one of our docs on how to use a reverse proxy with CNAPs. So, and you've got a whole lot of examples on how to use it with Apache, HAProxy, and joinings, and so on. So we're going to use CADD2 and you can see, I'm going to zoom a bit, so you can see that on here, that every, every single config example has two, two hosts. The reason for that is because we're differentiating the client's port on which we're going to expose, that we're going to expose to clients, which usually is 443, so that clients can just connect to their home servers like any, like any HTTP API. And we've also got the HTTP, the federation ports, which is by default on 48, on 8448, and we'll see a bit later how we can, how we can change that. So, sorry, wrong terminal. So, yeah, let's do this. So on these bugs, I've already installed CADD, so I'm just going to, to change the config file, which is currently just Hello World, I've already kind of prepared that configuration. For now, we're still going, we're just going to use the same domain name for both, because we're not doing any kind of delegation. And we're, and so just that domain, that domain on 48, on 8448. And because we've got the doc handy, we can just reverse, we can just copy that. And we're going to restart CADD. Hopefully everything is going good. Yeah, it feels the same like it. And now if I go to my domain, ArchConf 20 demo, and I go into matrix, I should have. So, right, so now I think, I think the one I'm looking for is, yeah, it's this one. So, it works. Synapse is running. So, yeah, we've installed, so we've installed, we've installed matrix and we can, well, we've installed a home server, we've installed Synapse and we can use, and we can now use any matrix line to connect to it. So, I was planning to demo that, to demo that with an actual client, but I'm kind of running out of time. So, let's see very quickly what we can do to go a bit further in our setup. So, Synapse has something, it calls workers to, that allows big servers to scale horizontally. The reason behind that architecture is that Synapse is written in Python and therefore very roughly one process equals one CPU core. That's sometimes, the result of that is that sometimes Synapse gets CPU bound and it's starting in terms of CPU and so we're, and so workers are going to be just sub, well, processes, different processes that use different, different CPU core and that's going to handle part of the main processes workloads and you do the, and so they communicate, they're communicating between each other through Redis and then, and it progresses being rooted by reverse processing, basically. A good example on the usefulness of workers is syncing. So, in Matrix you, a client receives messages by, by syncing, so it's, well, what we call syncing is a long poll on an endpoint called slash sync, and so that's, that's how it's, it's going to get new messages, new data, new invites, new rooms and all that and that process can be a bit, a bit heavy because it's dealing with a lot of events and a lot of, potentially a lot of users so it's kind of the perfect example of something you can just afloat to another process with, with workers and well, synchrotrons as, as we call it, as we call them. Another completely different nice thing about, that you can do with your, with your server is using delegation. So, that would allow, for example, example.com to be hosted at matrix.example.com. What, and what I mean by that is that the actual synapse is hosted at example.com, but the server name you give it is, is example.com. Because it's something I, I kind of forget to mention, like why the server name needs to be a valid DNS name, it's because your, it's because your, well the other servers are going to use that name to, to, to try to reach your, your home server. Delegation can also allow the Federation port to be something else than the default one. So, for example, so the default one as I mentioned is for, is 8448 but you could change it to be 443 and basically have both client API and Federation API on the same port, which might be easy in terms of, in terms of routing and whatnot. There are a couple of methods to do, to do that. The, the most recommended one is just setting up a JSON file at a well-known URI on the delegating domain. So, you know, for example, the delegating domain would be example.com. So, an example of that is that on example.com for slash, well-known for slash matrix for slash server, you've got that JSON file that's being served with m.server, which contains the address, the, the host name and port of the, of the home server. So, that would example, that would allow home servers federated, federating with example.com to send the traffic through matrix, dot, matrix point, dot example.com on port 443 instead of example.com on port 8448. A couple other features that you might want you to look into, you've got turn, so setting up turn, turn server is, well, turn slash turn server is, is something that could be very useful if you want to do void behind the NATs. And, and we've got some kind of, so, and Synapse somewhat support, well, support turn servers in that you can give it a turn server that it can then provide to clients. And another useful thing to have is metrics and monitoring on your, on your Synapse with Prometheus and Prefena. You've got a whole lot of info here and we also provide the JSON for a graph and a dashboard that you can use. And that's it for me, thank you for listening to, thank you for listening to me, rambling about matrix and servers and if you've got questions I should be there in the Q&A. Thank you very much. Hey everybody, so first of all Brenton, thank you for your very nice talk. My name is Jonas or Diabonis as a Nick and we've got a couple of questions. So, first question concerns the future of matrix. So, the question is, when will Dendrite this next generation home server you've been talking about be ready? So, first of all thanks for having me been a very long time user of Ashnik and it's just a great pleasure to be here. About Dendrite, it's actually quite nice timing because just last week, I think it was on last Thursday, two days ago we literally released the first beta release of Dendrite. So, if you want to spin one up and test it, it's definitely the time. It's not prediction ready yet but it should be usable and modular some bugs and features but hopefully there's not too many of those. Awesome. Next question is, are identities portable yet? So, can you move your identity from one home server to another one? Not yet but it's something we really want to do and it's definitely it's definitely on the roadmap. I don't have a specific timeline for that. I know that we've we've got at least once a spec proposal open for discussion on that and I think Dendrite team has started to do some experimental implementations of those. So, it should arrive soonish. I think but I don't have any exact any precise time scale to give you. Cool. The next two questions are a bit similar so I'm going to group them together. So, the first question is, how do you think the packaging in first steps after installation of SNAPs can be improved? And the second probably related question is, how would the best practices installation differ from the demo you showed in this talk? So, there are a few things that I didn't do in the in the demo. Either by lack of time or one of them I think I just forgot to mention it. In that, so one thing very important in that demo I at some point set a schmo to 777 on one directory that's because the package by default sets that to 7.0.0s and you can't and you can't just studio SU into the Synapse user so you have to do that in order to access that directory to CD into it. But obviously afterwards you should read that that's schmo to 7.0.0 or depending on your security policy for your system. Another thing is as I mentioned as early as possible in the process I stopped using SQLite and as the database backends and switch to PostgresQL because just it gives much better performances for usage with something like Synapse. And if you're already using a server that's federated that's being in use and all that that's using SQLite you should check out on the Synapse repo. We've got a migration script that's works pretty well. And I think that's about and obviously you should I'm kind of reiterating here be very mindful about whether you whether you really want to open your your home server to public registration. I think in terms of packaging I don't really know myself I haven't been involved in packaging especially for Arch Linux and I don't really and especially as someone who who's kind of very familiar with Synapse obviously I don't really have any any improvement ideas but that's probably something we can but I'm definitely happy to discuss that offline well I think on after that after that Q&A session with people interested. Cool next questions concern the database. So first question and again group these together because I think they are related is matrix tested with alternative possible implementations like Cockroach DB or Ugobyte DB and the other question is could you also use a no SQL database instead? So currently Synapse supports only SQLite and Postgres and we would so we would actually want to to switch to move towards a world where we mainly slash only supports Postgres SQL. The reason for that is because supporting multiple multiple database engines actually creates a big maintenance overhead in terms of maintaining the software and the reason we trust Postgres SQL is because it's the best compromise in terms of popularity usability performances and so on. So we haven't tested as far as I know we haven't tested Synapse with other SQL engines and we don't have any plans for implement it over the database engine both SQL or no SQL. Okay, time for perhaps one last question. What's the current state of the matrix app service IRC? I think it's I mean it's it is working with the we've got some we've had some issues in the past with in terms of family of preferences when bridging to massive networks such as free notes because just the rate of events was was kind of slowing down some some servers. We're trying to we're trying to reduce that to reduce that that performance impact on the bridge that we host. Well, so that's the bridge that we have some matrix.org and the bridge that other people host can work better. So it's it's also good to be mindful of the fact that it only those performance issues only impact big bridges. So for example, if you're bridging to the whole of FreeNode, you're going to have those issues. But if you're if you're if you don't have if you want to set it up for a more personal usage or more small community usage, it's it works better for a perfect fine. Cool. So thanks again for your talk and for answering all these questions and see you all at the rest of the conference. Yeah, thanks for having me. Bye. Bye.