 Hello, can you hear me? I think you can. Welcome to my talk. And who am I? My name is Sishan Ali, as you can see in the slides and everything. I work for Redhead starting from this week. I worked at Redhead before for five years. And I've been trying to get back ever since. And I finally got back. And my background is mostly false, Linux desktop mostly, but other things too. And most of it has been in C. And some of it is in this obscure language called Walla. But lately, I've been more excited about Rust. And that's why I'm talking about Rust here. My hobbies are flying and cats. Well, it's not a hobby. But I have a cat, and I like cats. And what am I talking about today? It's about Rust combined with Gstreamer. And it's inspired by a talk by Sebastian Drogue. He's a Gstreamer hacker. Has been for a long time. And he's really huge on Rust. And he has made Gstreamer bindings for Rust. And made all the interoperability possible. So he's the actual hero here. I'm just presenting about his work mainly. And there's a lot to cover. So I will go very fast. If there's any problem, I can slow down. Just ask me questions, and I'll explain. So first of all, let's talk about Rust. So what's that all about? It's a system programming language, like C, C++, Go. But there is a lot of emphasis on both safety and efficiency at the same time. It's one of the first languages that emphasize that in this core design as a main thing. Both of these features are the same time. And it does it with the concept of zero-cost abstractions. So you have high-level abstractions, like iterators, and stuff like that that makes programming easy. But inside the implementation is always zero-cost. So you're doing the same thing you will do in C and C++, but you're using a high-level API, and it's easier. And it's also safer, not just easier. And another thing about safety is that the main way to do it is like it doesn't allow raw pointer access. And that means you won't have any DREF ring. You won't DREF any dangling pointers. You won't DREF any null pointers. And actually, there is no null pointers in Rust as you see later. But the thing is the world is not safe yet completely. So you need to access C libraries and other languages code from Rust and the other way around. So for that, you have something called unsafe code, where in Rust, you can specify the unsafe keyword and put braces around the code. And then you have a bit more access to pointers, and you can do a lot more pointer arithmetic and stuff. It's still not as unsafe as it would be in C and C++. But still, compared to normal Rust code, it's unsafe. But the good thing about that is that once you mark your unsafe code as unsafe, which Rust will make sure that you do, whenever there's something wrong, there's a memory corruption or anything, you know where it is. You can just go to the unsafe part and debug that part and just ignore the rest, because that is safe anyway. It has the concept of non-mutable state by default, which is, again, the same thing. If you have a problem with anything, it's most certainly in a mutable state. It can't be in a state that is not mutable. So you only suspect the mutable state, and then it gets easier to debug any issues. And it also has strict ownership semantics. So in C and C++, you would have access to a resource, and someone frees it, and you still have reference to it, and you have crashes and stuff like that because of that. And the reason is that it doesn't have the concept of ownership semantics. So there is no concept of owner at all, except for some C and C++ modern APIs that do it. But that's API. That's not through compiler. The compiler doesn't know anything about ownership. In many high-level languages, most of the modern one, it's done through garbage collector. So it makes it much more safer to access memory because you leave the freeing of memory to the garbage collector, and it does it. But it's not an efficient approach. Like, nowadays, really, really efficient garbage collectors. But still, it's using your users CPU and stuff. So it's one approach, but Rust doesn't do that. Rust doesn't take that approach because it wants things to be efficient. So as I said, ownership semantics, strict ownership semantics. So you have only one and one owner to a resource. And for example, this code, it won't compile because you pass the ownership. When you assign S1 to S2, you pass the ownership of the resource that S1 was pointing to, to S2. And S1 does not have a resource assigned to it anymore. So you can't use S1 anymore. But and also the same with functions. Like when you pass by value, you pass the ownership to that function, and you don't have the ownership anymore. So this code won't compile either. And there's one exception to this rule that is copy types. So small data types that are not very heavy, like integers, booleans, stuff like that, or characters. And they all implement a trait called copy. And that allows them to be passed around as copies. So in the previous example, if this wasn't a string, an integer, and I pass it to the sum function, it won't pass the ownership to the function, but just a copy. And you won't have any problem with that code. Like this code will compile if it was integer, not string. But like a lot of data types, most data types you will use, it won't be very efficient to implement copy trait for them. You will be copying a lot of memory for no reason. And you can't get by just by passing values only. So you need some ways to get around this restriction of Rust, but in a safe way. And one of the first ways is borrowing, which is called pass by reference in C++, where you borrow the value to another variable or another function temporarily. And then you get the value back when it's done. So this code will compile because you're just borrowing the value. But if it was a mutable borrow, S2 was mutable, then this won't work because you cannot have a mutable reference to a resource while another variable or something is pointing to the same resource. Because that is calling for catastrophe, memory problems. If one is mutating while the other one is still having read access or write access as well. But the problem is borrows is that it's temporary. You cannot keep it. If I borrow it to a function, and it wants to keep it in a struct, the same data, it can't do that because it's just temporary. It doesn't have access after the function is complete when it returns. So for that, we have other data types. You have something called RC, which stands for reference counting. And with this, you can have multiple access to the same resource. But since it's provided by Rust, the implementation ensures that it's done in a safe way. So it's not left to you, ensuring that there's no memory corruption or anything. And you just use this API to share the same resource between multiple users at the same time. And you can keep it in a struct. It's not temporary. But it's read only. If you want write access, I'll get to that later. But if you want write access, you use something called cell. You put this resource in the cell, and then you put that in the RC, and then you go on from there. But why does my presentation title Fearless Multimedia? The reason is something, a concept in Rust that exists called Fearless Concurrency. And it's because Rust ensures that most of the problems you have with multiple threading do not happen. And that applies very nicely to multimedia, as I'll show later. And we have the same, like RC I mentioned before, it can be used from the same thread. But it's not possible to use it for multiple threads because it's not atomic reference counting. So it's not thread safe. But it's much lighter than this type I'm talking about now, ARC. And if you are sharing data between the same threads, then you are better off using RC because it's more efficient. It doesn't do locking or anything like that. But if you want to share between threads, then you need something a bit heavier. And that's ARC, and it ensures that the reference counting is all atomic and stuff. It's not that heavy comparatively, but it's slightly heavier. And as I said, RC only gives you read-only access. And in a single thread, you will use RC with cell or something. And in the same way, in multiple threads case, you will have, you use something called mutex, where you have a data in it. And each time you want to have access to it, you lock it. Like just like usual mutexes, you will see in other programming languages. But then the way you pass it around between different threads is you put it in an ARC, and then you can use it from different threads. And that was like a really quick interaction to how Rust works and what Rust is about. As I said, I have a lot to cover, so I can't give you a really good intro. We come to now GStreamer. GStreamer is a multimedia framework. So if you want to make an audio player or a video player or anything to do with audio and video, GStreamer is a good framework to do that. It's LGPL. You can use it in proprietary software as well. So it makes it very popular among people who wants proprietary multimedia solutions based on open-source software. That's the concepts. The main basic concept of GStreamer, extremely simple. It has a concept of elements and pipelines. A pipeline is a combination of elements. It's a bit like Lego, so you put different elements together to achieve a particular goal. And the way you connect to different elements is something called pads. The darker blue things in this diagram, they are pads, and that's how you connect them. I'll talk a bit more about that later. And that was a really simple example, and it was abstract, but this is a real example of a very simple pipeline. If you want to play a video in org format with Orbus, Audio, and Theora video, this is a pipeline you can use. What you do is you have a file source, you connect it to a demuxer and demuxer, demultiplexes the audio and video parts, and then you connect it to different decoders, and then sync is the player, actually. And, for example, in this case, Audio Sync would be a pulse Audio Sync or something like that. And the thing is when you connect elements, you connect, actually, their pads. And pads have something called GST caps, which stands for capabilities, and capabilities tell you what they can handle, right? In case of a Orbus decoder, the Sync pad will say that I can handle Orbus data and nothing else. So you can't connect it to, for example, something that is giving you MP3 data. So with the help of caps on pads, you know which one can connect and which one can't connect. And that's how, in GStreamer, they accomplish auto connections. So there's these elements that look at these capabilities and then they connect on the fly, which one connect to each other, and yeah. And so you can easily automate all of this in one element that does this auto plugging. It's very much plugin-based, so all these elements that you will use they are provided by external plugins. And that's the beauty of it, like you can replace any element with another and you can have better capabilities in one element than the other and it will be higher ranked and it will connect automatically and stuff. And it's written in C, like most of GNOME infrastructure and libraries. And it's multi-threaded because it tries to utilize the as many CPUs you have, so it has to be multi-threaded. The GStreamer API, they have designed it so after a few iterations, so that apps don't have to usually care about threads. Sometimes they do, but most often they don't. But plugins, there's more times in plugins. If they're simple plugins, then simple are providing simple elements, then probably not. But if they are a bit complex, then they need to care about that there's threads and stuff. So yeah, the threads are important there. It is object-oriented using the GeoObject library. And why is Rust relevant in multimedia? First of all, parsing media formats is not safe. So especially from untrusted sources, there was even like a recent case of FLV decoder being used to take over a system in Ubuntu, I think it was, at least. So that those things happen. So, and if you do it in C, it's extremely unsafe. And multi-threading is extremely hard, especially in C and C++, they are really, really hard to get right. So that also brings Rust because Rust is, like it makes multi-threading really, really easy and safe. And mutability and ownership maps well, the concepts, how the GStreamer has these types and everything, they map really well with Rust. I have to go really fast now. GST, MiniObject, for example, this is just an example. It's, so they try to make things safe with API, but it's still C, so they cannot ensure, they cannot tell compiler that we want this. So like this GST, MiniObject, it's read only if your, it's a reference count is only one. And, but it, sorry, it's, you can write to it, you can modify it if it's only one reference count to it. But if you have multiple reference count, that means there are multiple owners of the same, so you cannot modify it at that time. But this assumes that you do reference counting and you increase and decrease the reference, which is not always the case. So you cannot ensure that with C, but with Rust, you can ensure these things. And avoid a lot of memory problems. As I said, in C, you can do so much wrong. And C is an archaic language, it's not just unsafe, it's really old and it shows, you know, when you, when you work with it, it's really hard to work with. So there is the GStreamer bindings, as I said, Sebastian Drogh wrote these. He's also writing a bunch of plugins in Rust based on these bindings and there's a lot of plugins already there. And I hope that it keeps growing the GStreamer plugins. A simple example here is this code. As I mentioned, in pads, you have capabilities and a single pad can have multiple capabilities. So that's why it's in plural caps and each capability is a separate structure in a cap. This code tries to get the first structure it has in it, the first capability. Then it removes the same one and then it sets a property on it. This code will compile fine equivalent code in C, but in Rust, as you will see, it won't compile because it doesn't let you do unsafe things. The first error you get is like, like the set method is not found. And the reason is that the first line, it's not giving you a structure, like C on C, you will get a structure or a null pointer. So if you don't check for null, you will have a crash. But in this case, you get something called in Rust option, which is a enum. So we can handle that now. So what we do is we say this enum is either some or none. So if it's some, that means you have a value, you have a valid value and you can use it then. But it was none, then you can't really use it. And that makes it very safe because it's not a null pointer that you can de-reference, but it's an enum that you need to parse. So we are parsing it here and we are saying if it's some, you give it to me in the structure and then you say remove, the rest is the same. But this won't work either because there's other issues. Like all the issues that exist in this code, but C compiler would not let, wouldn't tell you about those. And now we get a lot of errors. And the reason is that you're trying to modify something for which you didn't get a mutable reference. So this time we get mutable references. So first of all, capabilities, we get a mutable reference to that. And then we get a mutable reference to the structure. And then we try to do the exact same thing that we were trying to do in the first place. And then you get an error that you cannot borrow. When you did C to get mute structure, you borrowed also C as mutable reference. And in that scope, the C is already borrowed mutably. And as I said earlier, you cannot borrow the same thing, the same resource mutably multiple times. You can only borrow it once in the same scope. So that's why there is this error of you're borrowing mutably twice. And so we now solve the simultaneous mutability. You get a mutable caps. And then we remove first the structure. And then we get a mutable reference. And then we set anything on it. This code won't actually work because you removed the structure already. So you won't get a structure out of it. And you will get a none and it will not work. But it's safe. It doesn't work, but it's completely safe. And Gras compiler ensured that you did that. And now it will build just fine. It won't have any problems. In a closing note, I just want to say, try not to write new projects in CNC++. They are very unsafe languages. People think that they are really good at them and they won't make any mistakes and they will be fine. But no, nobody is perfect, nobody. So you will make mistakes and it can cost a lot of problems for your users, for your customers, then for you as well. It's a maintenance hell. So always consider Rust if you want to write something efficient instead of CNC++. That's it. Thanks. C, Sebastian Drogue, as I mentioned. What do you see? Do you see the question? Sorry, he asked, who wrote the Rust interface to the C APIs? What was the problem first? It was mostly one person, yeah. He used other people's work as well, like the people who did GTK-RS and stuff and they're collaborating a lot. And we have these every six months, Rust, GNOME, HackFest, and they're all there. So it depends like what do you mean by, because it's a moving target, you know, like there's always new APIs on C level so you have to cover that. I think it was six months or so that they had something that they could use and they can create plugins. It depends on how much time he used, like full time, you know, like he does other things too. Yeah, no, it's not true. Anyone else? You have to ask that the multimedia coding often uses the SSC or ABAE instruction set to clarify the recording and encoding and how does it play with the app? Can you look at that here? There is, I don't know if it's been, oh, sorry. So the question was that, do it as a GStreamer use SSC and all those intrinsics? And how does Rust, how do you do that in Rust, right? That's the question. Yes, GStreamer uses it, like all the hardware capabilities it possibly can. It depends on the plugin developer, right? How they develop it. But yeah, usually they try to. Rust language, I was about to get to that. So in Rust, they have like SSC support. Last I checked, it was about to be merged, but I think it has been merged already, the SSC support, at least in the nightly. So I'm pretty sure in the 2019 Rust, it will be major part of it, because a lot of people ask for it. Almost, I think, yeah. Sorry. The question was if there is 100% coverage of the CAPI in Rust, the answer is I'm not sure. I think there is like at least 99%. So yeah, cool. Thanks for coming.