 Hi everyone, my name is George and I'm a senior multimedia engineer at Colabora in this presentation I'm going to talk to you about transporting media through containers using pipe wire Now you may ask yourselves pipe wire is an audio service, right? No, in fact, it's not just that it's known as an audio service, but it's it's actually something that enables Processes to share any kind of media with each other and and share also devices with these processes So it it allows transporting audio and video Between any process in the system It is built so that it has support for containers right from the beginning and a built-in security system which we'll see in Overall, this is how How the pipe wire system looks like so there are there are applications all these green boxes at the top These are applications that connect to pipe wire crossing through the various security buyers be it containers or or in something else and They connect to pipe wire and they create So-called nodes which are objects that represent the streams of these applications and these nodes Then connect to each other and they form a pipeline that then allows media to flow from one application into another Let's see a simple example We have an application Let's say we have a system that needs to receive camera freed camera feed from the network for using a network camera and Then it needs to display this camera and also record its contents These are three distinct functions Something that receives the feed from the network something that displays the feed to the display and something that records on the disk Now without pipe wire This needs to be one application that handles all these three With pipe wire we can actually split this functionality into three different applications We can have one application that receives the content one that displays it and one that records the disk and This can be done without any additional overhead since transferring media through pipe wire is zero copy In a graph this looks like this We have three distinct applications the receiver the viewer and the recorder They connect to pipe wire and pipe wire acts as a bus It allows these applications to be linked together and allows media to flow from one to the other Now I said pipe wire is container ready In fact While this can be implemented with these three applications running natively on on the same host as pipe wire in fact It's not necessary. We can have these three applications in different containers And we can restrict the functionality that is provided to each of those So we have a system that looks like this We have a container a that runs the network camera receiver that has access to the network and to pipe wire And nothing else. It doesn't have access to the display and it doesn't have access to the storage And then we have a second container with the viewer that has access to the display and pipe wire But nothing more and finally we have container see with the camera recorder application. It has access to storage and that's all and All these three have to connect to pipe wire and they have to To open the socket to connect to pipe wire now these connections to pipe wire Are restricted connections and that means that these applications Because they are coming from different containers They do get access to pipe wire, but they cannot do much with that connection. They cannot really See the other clients that are connected by a choir and they cannot change settings or manipulate devices Or things like that. They can only do what they are intended to do and what we allow them to do now Let's see how this works in a typical Restricted sorry in a typical socket connection We have the application Sending a connection request to the application that has the socket in this case It's pipe wire and then pipe wire returns a file descriptor and then the application can freely send Data to this socket and communicate with that application in pipe wire This is the the most simple case like When we have application audio applications, for example in our desktop machines, they connect like that but when we are talking about Sunboxing we have Two different security models that can be applied the security model one Works like this the application Coming from the container has access to the socket, right? So it sends a connect event To that socket asking pipe wire to to connect then pipe wire it internally. It has an a module that applies permissions for that client Basically pipe wire internally tracks Every client separately and tracks the permissions of each client separately So every client can access different objects with different permissions inside pipe wire Even though it can send protocol messages The responses are basically filtered by pipe wire. So if for example the application is asking to Read an object that it doesn't have read access to then pipe wire will return that this object doesn't exist, for example So with all this logic is handled internally in pipe wire So the application connects and pipe wire applies initial permissions and then It returns a file descriptor to that application, but this file descriptor is initially blocked That means that the application Cannot really write to it or maybe it can write but it will not get any response from pipe wire yet Now the fourth step what happens next is that pipe wire notifies the session manager in In this case wire plumber it notifies that there is a new client and then wire plumber has Some logic to look at this client Inspect it see what kind of client it is and apply a set of permissions to that client Typically, this is to allow the client to actually see the pipe wire core object, which is the main Connection object and also to give it some permission to create streams Now on the sixth step pipe wire unblocks the connection after the permissions are applied pipe wire unblocks The file descriptor so now the application can freely talk to pipe wire and look at what objects there are But since we have applied permissions, it won't be able to list all the objects available But most likely it will be able to create a stream because that's what the application is for right so we have Applied permission that enables this application to create a stream So on the seventh step the application creates a stream and when it creates the stream Again, nothing happens in pipe wire pipe wire then instead notifies the session manager that there is a new stream available and then the new stream is Appears in the session managers logic and the session manager chooses. What to do with the stream if it is If it is allowed for this application to create this this kind of stream Then this new stream is indeed linked to a target. So it gets linked to another node That may be another application or maybe a device Otherwise if the application is not permitted to create this kind of stream Then an error is returned and the application is denied the request for that stream So then pipe wire sends back an error to the to the stream and the application bails out This is the first security model Now there is also a second security model, which is somewhat more advanced more secure and Involves a middleman, which we call the portal This portal is a separate application that sits between The container and the host barrier. So it has access The application has access to it through the container, but the actual process runs on the host Now the application is not able to access the pipe wire socket directly, so it sends an IPC call to the portal Asking for a connection to pipe wire this IPC call can be anything from it Simple socket connection or a debas call or something similar Then the portal connects to pipe wire and Again pipe wire applies initial permissions To that connection and returns a file descriptor, but indeed in that case it doesn't it's configured So that it doesn't block this file descriptor And then the sandbox portal Gets the connection It has access to most objects right in pipe wire at this point and Then it applies permissions based on what the application is because the The same container may have different applications that need different sets of permissions. So checks what the application is and Then it applies permissions It could do that by consulting a database for example a database with permissions per application or something like that That's typically what happens and then applies the permissions To pipe wire. So pipe wire now has a very specific set of permissions for this connection and now the file descriptor of this connection on the sixth step is Sent to the application and the application now has a connection which is restricted So far wire plumber has not been involved, right? So the whole initial permissions Setting is done by the portal Then the application creates a stream right and the stream goes to pipe wire and From from this point on it works like before so the wire plumber gets notified about this new stream and then it checks Is this client? allowed to create this kind of stream if yes, then it links the stream to the Target device or target application. If not, then it denies the stream and sends an error back This is a little bit nicer because it takes the logic of applying permissions away from the session manager and moves it into a separate process and it also Doesn't expose the socket of pipe wire directly in the container Instead it all it exposes the socket only to the portal and the portal is the only thing that gets exposed to the container But otherwise, it's pretty much the same now. I want to show you a demo In this demo I have Two web cameras on on a machine that are being shared to different applications in separate containers containers are built with podman and The applications are getting access to both cameras simultaneously As you will see, but they are not getting access to any other resources by a pipe wire Let's see that So I'm running this demo on my work machine on my laptop Where I have two USB cameras connected One is the internal one from the laptop and the other is a secondary camera external one As this is my work machine and I'm actually currently using it to record this presentation you will see in the WP control status Command line you will see that I have a bunch of applications connected here doing both audio and video. I have like four audio devices connected and two video capture devices and Currently I'm using OBS studio to capture this presentation so it's both capturing from my microphone and from GNOME shell, which is my desktop shell and it's Screencasting the content of my screen To OBS for capturing. So this is what it looks like in In the graph, this is the this is these are the nodes inside pipe wire and you can see I have GNOME shell connected to OBS and my Sound card connected also to OBS and While this is happening. I am going to start Two containers. So I have prepared a podman container of image That is based on on Debian unstable and I have added pipe wire and wire plumber in that image And I'm going to start it with a script which looks like this So it's it's starting this Pipeware Debian image that I created and it's binding the pipe wire and Wayland sockets to it So that it has access both to pipe wire to capture the video and to my Wayland display so that it can show the Graphical window output. So I've started this container Now I'm going to go to the MNT directory, which is where I have put my my scripts In in these I have mounted my scripts basically there. So I Have a Bunch of scripts here. Let's look at the camera first. So I have a camera one and camera two scripts that display The two different cameras respectively Let's look at their content camera one both scripts are To steamer pipelines Camera one is capturing with pipe wire source from devs video zero And the second one is capturing again with pipe wire source from dev video two So this is the first camera And let me start another container here and this is the other camera now. I have both both cameras Being used simultaneously from two different applications in two different containers. Okay, that's not Something new we could do that before Now let's try to do something More interesting. I have another script called mix which takes both cameras using two pipe wire source elements and Capturing them both and then rendering them in the same window using just streamer's compositor element that renders into a single Window output so both cameras are now being captured from this single application in the first container And I'm going to start the same script in the other container and You see how both Containers capture the same image exactly and they are perfectly synchronized as well and If you look at performance pipe wire itself is using only two to two point something 2.6 before Of CPU time while the just just the launch processes are using much more. These are doing The heavy lifting the rendering of the window while pipe wire is only capturing from those devices from video or 0 and video 2 and moving the buffers To these applications in the two different containers Without copying the buffers so it only transfers the file descriptors of the of the memory that it needs to share and Of course this 2% is not just the two The two video cameras because as I said, I have OBS studio running still so this is what it looks like I have all these GST launch processes. I actually I'm running two GST launch processes, but they appear as four streams here because Every pipe wire source element into streamer creates a new stream to pipe wire So it has a different node And that's why I'm seeing two nodes from the first application and two nodes from the second application capturing from these two cameras and then I still have my OBS here recording as usual now That shows the cameras, but it didn't Show you any security features. So how how do we know that this is secure? So first of all, I would like to compare WP control status, which I showed you before this is on the host running because I exited the container and this is WP control status in the container as you can see It doesn't have access to any of these objects. No client objects. No audio devices. No video devices even So it doesn't see what happens in the host. It doesn't have a clear picture of the graph Additionally, I Could look at WCLI LS for example, which shows a list of available objects and it only gives me two objects the pipe wire core and The client node factory which is used to construct Streams While on the host this would give me a huge list of objects Which are all the nodes and all the ports and all the clients Which have been created in pipe wire now I because I'm giving access to this Client node factory I'm giving access to that in the container my Clients here camera one camera two and mix are able to construct streams But if they are able to construct streams to capture video Maybe they should be able to capture audio as well or playback audio. Well, this is also restricted here And I have created two scripts to showcase this audio capture and audio playback the sage Audio capture is also just streamer pipeline using pipe wire source to capture audio this time And on your playback again pipe wire sync Playing back some test audio Audio playback to the sage shows an error. It says not permitted and audio captured to the sage again I just it says not permitted and the same happens with Standard pipe wire utilities like PW record that captures audio into a wave file So this is supposed to connect to the microphone and record something to this wave file, but it says not permitted And again PW play which plays back a wave file not permitted Now where this is coming from This is coming from the session manager from wire blumber which I have modified in order to To check If a process is coming from a container and then restrict its access So I am using the first security model that I showed you in the slides where The container has full access to the pipe wire socket, but the session manager restricts What the client can do with this socket? So in this script here, I have added this function container check that for every new node that connects It checks if the client is Coming from a different host so it compares the host name of the client versus the host name of wire blumber and Of course this check is not entirely secure in a production system. We would have to check something else not the host name Because this can be manipulated, but For this demo This was the simplest form of checking and this is why I've done it like that So if the host name doesn't match that means the client is coming from a container And then I'm also checking the media class of the stream And if it doesn't match stream input video, then I'm sending an error saying not permitted and this string here Is exactly what gets displayed in the command line over here so we saw this demo and Now I'm going to show you something else that we've built on a GL for For the instrument cluster demo On the instrument cluster demo we feature LXC containers That Run separate images of a GL so there is a host That runs pipe wire and There are two containers the instrument cluster container which takes care of displaying the instrument cluster and The IVI container which has the applications of the IVI system now the host is Taking care of managing the devices like the also devices for audio output and input But the the IVI that has the applications is the one that runs the policy Even though we have Pipe wire running on the host the actual policy it runs on the IVI And that means that we have a separate instance of wire plumber running on the IVI that manages all the applications that run in there and Allows them to securely access the host and the instrument cluster container uses Alsace So it doesn't go through pipe wire for audio output For stability and security reasons But every time there is a sound coming out of the instrument cluster it poses everything that goes on in the IVI So it sends a message through a through a separate IPC IPC protocol and causes the wire plumber running on the IVI to pose and Stop producing audio so that we can hear the audio from the instrument cluster And this is what it looks like in a graph So two containers instrument cluster and the other container is the IVI container and we have Two instances of wire plumber and one instance of pipe wire pipe wire runs on the host Then we have one instance of wire plumber on the host which Manages the devices. So this is the one that Actually discovers what kind of Alsace devices we have and makes them available to pipe wire And then the second instance of wire plumber in the IVI container is the one that has the policy So every stream that connects from the IVI goes Is notified into that instance of wire plumber and then this instance is the one who decides if the instance can be Allowed to connect or not While the instrument cluster application Uses a separate IPC demon pipe wire IC IPC. This was written specifically for this purpose and that When it sends a signal to pose then this pipe wire IC IPC basically Sends a signal to pipe wire and then that signal propagates to wire plumber and wire plumber poses everything and Disconnects devices and mute devices and so on So here I want to show you the interaction between the host container and the IVI container and for demo purposes I have created a virtual machine. That's running Debian in which I have copied the wire plumber configuration from AGL so that It has the exact same configuration as an AGL So with this configuration now there are two they're going to be two instances of wire plumber One instance, which is going to be running on the host container and it's going to be managing the alsa devices and Secondary instance, which is going to be running on the guest And it's going to be handling linking policy now let's start By entering the IVI guest first of all I want to show you that In WP control status right now, there is no wire plumber instance running and there are no devices listed Now to start with I'm going to start wire plumber in the container And I'm going to start it with some debug output so that you can see the activity And I'm going to start it with the policy configuration file So you see that it starts to register some Endpoints which are policy specific and I'm also going to go to the host I'm going to verify that there is something connected now There is wire plumber policy and we have all these endpoints registered these are coming from the IVI container you can see that the user running these instances would at IVI and I'm also going to start wire plumber now here in the host and again with some debug output so that you can see the activity and in this instance It's configured to only handle the devices so it starts And let me find another terminal So now if I look at WP control status, I can see that It the built-in audio device is also registered now It's it's here and I have another instance of wire plumber running Which is actually on this on on the host container now if I enter the guest again I Will try to play some audio from the guest and It works That was all I hope you enjoyed my talk. Thank you very much for attending