 Okay, welcome to my talk. I love that. Okay, thank you. Today I'm going to tell you how we're using valiant in IVS systems. I would like to give you a short overview of what I'm doing in my daily life or in my work. I would give a really high level of the graphics tech so you know which component I'm talking about and where we are placed in this tech. Then I would like you to explain why IVS system needs something different as a usual desktop system. And the main part should cover then the usage of Western as a valiant compositor in IVI. And then I would also like to talk about what's coming next, what we're planning to do and what we already have done and planning to upstream. So my name is Eugen Fiedrich. I'm working from ADIT for ADIT since 2010. I started handling graphics techs in the company and till today I'm doing the same job actually. I'm responsible for the complete graphics tech. It's a video end display, pipeline, GPU and all surrounding and related components. I would also like to give you all of you what you're doing in ADIT so you know why actually we need, why we choose the Western and valiant as a graphics system. ADIT was founded in 2003 from Bosch and Denso. From those big two companies we are getting out of requirements. We need to harmonize them and create one single platform which will cover all the requirements. The biggest part of the software we are getting delivered but still to implement the use cases we need to implement some additional components or enable features and modify the stacks a bit. Once we are done, actually it's never really done so it's a continuous integration writing and change request and so on. We're delivering the platform to other companies and they're building projects on top of it. For us it's very important to have something really generic here which will cover both requirements and also practically they are up to 10 projects using this at the same time. This is the overview of the graphics tech, very simplified. You can see on the upper layer of the applications they can use some frameworks. They will in this diagram they are all using valiant. They have to talk with valiant compositor which is Western in this case. There is OpenGL driver. You can see here that the compositor and the applications are able to use the same driver thanks to the valiant agile implementation. And the compositor is actually then responsible to give the final content, final results of all applications to the kernel monster driver and then to display. So I think this is and now I would like to talk a bit more of Western itself. The Western has a really good defined plugin infrastructure. You have to have three plugins at least. One plugin is a renderer which will use OpenGL driver to make a composition. Actually render the stuff in the compositor. There is a backend. Most of the time we are using the RAM backend but you could also use for example valiant backend which will output the final result to a different valiant compositor so creating nested compositing is quite easy and there is a shell. The shell is actually or the implementation of the shell defines the window management. It will provide some valiant protocols to the applications and they have to use it and talk with this protocol to compositor to then request how they want to be presented. There is no different process or component which will decide how to behave. So this is all implemented in the shell. And this is exactly the main difference what we don't like in AVI or what we cannot directly use in AVI because our customers has very different requirements very different ideas how the application would look like and the complete setup of the system. They could have screens with one application or five or three they want to have them slide away and flank in from different directions and they also asking fast for example to implement something which will ensure that the arrangement of application and the application content change will happen in the same vision. It's quite hard to achieve because the APIs are synchronously implemented there are some events and it's hard really to synchronize different protocols and APIs. Looking closer to our idea so how to tackle this problem we decided to implement a special shell which will provide the interface to outside so you could implement your window management outside of Western. It should be really only this it should be really a slim interface it is a slim interface which will allow you to control the composition but it will not take care of the actual rendering this stuff and it will also not take care of how to output this to the final display. At the end it actually could be not really a display it could be some virtualized output or even a different compositor. To implement this we have now the IVI shell which is living in Western itself IVI shell can load additional plugins those plugins will implement a variant protocol and this protocol can be used to control the applications. On the application side we also need to add something maybe it's not really the best idea if you're coming from desktop but this is what IVI was using for a very long time they always had some identifier for their applications and this IVI application protocol is really slim so it's small, it has one request and two events we need this to assign a unique ID to our variant surfaces and of course the drawback that application needs to be modified or some framework needs to be modified to implement this and to assign ID or to map the ID to the variant surface which in variant represents the application or the content of the application. Looking closer to the implementation as such how we relate this the Western is its own open source project the IVI shell is living inside Western so it is in the Western repository and the bluish plugins they are in a different open source project it's a Geneva project and ADAT is maintaining of this here we are developing the controller plugins and maintaining this IVI VM protocol to control the application IVI input protocol to control the input routing there are also some examples and VREPA or the lay manager protocol which will ensure compatibility to some defined IPI in Geneva for window management you could also see that here inside of IVI shell you could also have some HMI controller built in this is a different plugin which can also implement the video management if you don't want to use this and you know how the video management should look like there is only one use case or simple use case which can be implemented and which is really good to define then you can also do this with HMI controller which will then implement window management so what we can do with the IVI VM protocol it's quite simple we have three types of project we have a screen this screen is actually mapped to the output struct of Western which at the end represents a physical display most of the time on the screen we can put some layers layers are just logical object to group the surfaces you can scale layers, you can make it invisible you can define transparency and also change set order on the layers and we have the IVI surfaces which actually represents the application content and there are some few events to get notification about new object in the system about new surfaces new IDs if they are created and deleted so that the HMI controller which is sitting outside of Western can react on this and make it visible it is actually also not possible for application to be visible so application cannot request, make it visible this logic has to be implemented in the HMI controller which we will get to know application is available then I would like to talk about this tiny thing the IVI application protocol this was from this slide sorry I am spoiling my slides this one they had to modify the application of framework actually we would like to get rid of this long term and convince also our customers to follow this and the idea is to use a different protocol because some protocol needs to be used from application side we decided to focus on the XDG protocol which is also defined by the Western community and actually used also already in some distributions and even different variant compositors there is also a stable version of it then we think this is a good protocol to be used in the future also for IVI but still we need to identify our applications and we need to define the ID to be able to use the same IVI-VM protocol and we define now additional plugin for Western which is called ID agent this is work is going to be upstreamed in the next week maybe patches are already provided we are planning the generic idea is just to define how to assign the ID to the application on the HDG protocol you have already a lot of stuff which you can use for identify like a window name or application ID and in the ID agent you need to implement some logic which will take some of those attributes and will say ok this application appears then this is ID 5 you could also have this more complex and your ID agent could also communicate with some database and ask someone to get the ID from this but this is an up to implement of this ID agent this thing that at the end each project could have its own we could provide some basic implementation which will look up some configuration XML this is how it will look like then if this will be implemented and available we will still use the IVI shell we will still use the HMI controller the major change is actually only for application and the biggest benefit we will get because of the application frameworks something like Qt or OpenSceneGraph already have support for HDG but they also have support for IVI application protocol but it is not really good implemented and not really widely used it is definitely better to focus on HDG protocol in the application frameworks and with this ID agent implementation we are able to make applications visible without any modifications in framework or application site and of course we cannot force the customers to use the HDG shell protocol they can still use IVI application protocol it is still available, it is supported but I think long term this will be a good idea also because of the compatibility you can easily port, exchange and also reuse application or just try some application to run a new IVI system which are already running somewhere else the next use case we want to be able to manage with Western is distributed at Gemai this we are getting more and more requested to implement something which will realize the use case we have a different hardware different displays the content is generated on one place and is presented on another and there are also very different systems and all this stuff is quite complex and also the communications is very complex between those components at Gemai and there is actually no real standard way to implement this there are some components and some implementations available which will be able to help a lot for example Ramses which was also open source recently from BMW but we would also be able to support this use case a bit better and the idea is to implement another plugin for Western which should use FileZem FileZem is actually very similar to Valiant as such but there is one major change or difference it cannot carry file descriptors it can cross your device it can really communicate to outside it can use TCP IP sockets and get connected to a different system with Valiant sorry, FileZem the complete picture will look like this we will get additional FileZem transmitter this transmitter will actually implement one additional output object in Western and if you will be outside of Compositor if you will be on application side or in framework you will just see additional output you should know this output is virtualized and if you will put some content to it then device and transmitter will be notified and will start perform some action so it will stream this to a different device if you have virtualized system it could use some graphic sharing mechanism to share the content at notify different operating system the GMI controller is actually also a client of Western it will also get this virtual output in the same way so it can then use the IVIVM to assign application to this new created output and this is the idea but it is quite hard to implement it is a really genetic which can be used between Linux and Android or Android to Android and some proprietary operating systems which are not widely used in desktop but in automotive also to implement the compatibility mode to hyper-vise us it is not quite challenging but this is what we are planning to do next years and those are just a list of questions what I like not solved now and where we are getting requests again and again but for now there are no real solution let me say in the lower platform level and we are still thinking how to solve this how to implement this or maybe the final result will be it is not meaningful to be implemented in Western it should be done on the upper layers yeah, I think that is all for my talk do we have some questions? that is a good do you have a feeling of the latency now between what M remote displays and for example or what it means? depending on what you will do the Linux channel is very fast if you will use a very simple system without load in real test we could not recognize we should be very careful looking to a display to see where is the original content and where is the fake output content but it is again like it is idle system without any load that all use case will be a little bit hard but I cannot tell you the number in milliseconds but it is usable differently yeah the original design was the idea of single full screen only and one screen per device in modern cars that is quite out of fashion do you still believe that I will be a shell as it is designed to make sense could we use an XBG concept and just have a controller to stop the application to do whatever they want it creates a lot of side effects actually I don't think that IVA shell limits you to the number of displays it is not related if you have 5 outputs you can use them all it is difficult to manage them you have different size of screens different color depths it doesn't manage that we know it is difficult IVA shell will not limit you to do this what will happen if you use IVA shell then you will have differently arranged displays so in default behavior of fashion that it will extend the display you will have all the display in one big coordinate space if you will use IVA shell you will get this completely distinct so it will be different displays with different resolution you may not have one common visual space but you could arrange your layer and surface setup that it will look like one common space so it is I think especially for IVA you don't want to have extended displays you have to have separate displays the typical use case is to move for example the map from the big screen to the small screen under when you have another application starting out that's difficult you probably have to design the application in line with the hardware where your fashion and experience limit it it means the application has to get the knowledge of the real implementation of the graphics maybe we can talk later there is a way to realize this use case with the IVA-IVM protocol they can move your application to a different screen with the same application content, same surface because I didn't show this here what you can do, you can assign the same surface to different layers at the same time but you cannot assign the same layer to different screens this we didn't implement because of complexity but if you will assign your surface to different layers you can use the layer capabilities to scale this differently and assign this to different display with different resolution this would be...