 So, yeah, I'm David Ehrle, you may know me from previous LCAs, I've pretty much given similar talks. I work at Red Hat in Brisbane, I've been there quite a while now, unlike Keith, I'm not giving up graphics tomorrow or last week, but my talk today is about a technology that sort of caught me by surprise last year, sort of like mid-last year, called DisplayPort MST, which stands for Multi-Stream Transport, and it's pretty much involved in a whole load of devices that, when I initially saw this idea, I went, oh, you'd never use it for that, but well, everyone did. So, the two main things that this technology affects are 4K screens and lots of later laptop docs. How I actually started working on this was quite accidental. One of the local, I work in the Red Hat office in Brisbane, but most of my management and all that is done through the U.S., but the local IT guy in the office walked in one day and my cubicle was like, oh, someone from around here, and he goes, how come this laptop doesn't work with two monitors when I dock it? I don't know. Well, could you, could you have a look? So I said, I'll have a quick look at this, it can't be that hard, I've heard of a problem, silly. So he went, because they had actually bought some of these laptops, some of the docs, and tested one monitor plugged into the dock, and went, well, it's fine, we'll give it to users. And then someone plugged in two monitors into the dock and it just mirrored, and they couldn't, was like, why isn't this giving us two monitors? We've already given it to people, please fix. So I looked at the, went and dug into a little bit, figured out what was going on, and realized, I came back to myself, it's probably going to take about six months maybe, and he was like, what? We've got to get these out to people now. I was like, yeah, you pretty much have to have made a whole networking stack to get this working. I was like, okay. So I said, well, I'll start doing it now. I told my boss a couple of days later that I was actually starting on this project, and it would probably take the next six months and ruin all the other projects I was meant to be working on. But since it seemed important that we should have this stuff working, Red Hat has customers apparently, who knew. So yeah, keeping them happy is more important than doing my other projects that I like. No, Virgil, hence why the Virgil talk has not been given this year. Just a quick intro to the area, history of like display technologies. I'm not going too far back. I'm not talking, like, I'll start at VGA. No need to keep going further back. But VGA was like probably the most sort of successful of the display technologies that's been around. It was like analog based. It did everything for analog monitors. People were quite happy with it for many years. But things happened to go digital. There's no point in having an analog signal in your loop when everything is digital on either end, especially with flat panels. So along came DVI. DVI started out as those four little pins in the cross are actually a VGA port. And the rest has some pins of the VGA port and a whole set of digital channels. So it was like an evolution of VGA. You could get DVI that was just digital. You could get DVI that had digital and analog transports. And you could get DVI that was just analog, which was actually VGA. But I don't think anyone ever shipped that in a product. It was a standard. It just wasn't in the products. DVI was good also. It had sort of extra pins. So they put two digital channels into DVI. The second digital channel started being used for a thing called Jewelink DVI, which was only certain graphics cards would support this. And it was for big monitors. At the time, big monitors usually meant Apple, I think Apple 30 inch, Dell 30 inches, that sort of monitors used Jewelink DVI, 2560 by X, big numbers. For the time, not big numbers now. But DVI sort of got remodeled and put in a different shaped connector with only one link in it called HDMI. And that was done by a different consortium of people because it's the computer industry is not the same as the entertainment industry and everyone likes their own standard and everyone likes to charge money for different things. So HDMI came along. HDMI is one of those things where you have to pay the HDMI people for using their connector. Their cables cost $70,000 million. If you buy them with gold plating on either end, even though they're the same cables that cost $5. But it was a good scam and people are still running with it. It's good call. And around the same time as HDMI, DisplayPort started coming out probably a bit later, but not too much different. But DisplayPort was the sort of computer industry attempt to come up with a new standard. They went a lot further. HDMI is pretty much DVI with a different plug and then adding features to it and making it, you know, more stuff for TVs and stuff. DisplayPort was like throw everything out the window. And we'll sort of pretty much start again with a cool socket, cool plug. And so here's a nice picture of what the sort of way DisplayPort looks. You've got a receiver and transmitter. And between them, you've got a big, large link, main link. You've got an OxyH channel and a hot plug interrupt. The main link consists of four up to four channels. And the four channels can have different data rates on them, depending on what you need. So the original display link data rate was 162 megabits a second, along with 270 megabits a second. So each channel could do that. And then you could have four channels. So that was quite a good bit of bandwidth. The DisplayPort 1.2 spec has up that to 540 megabits, which is called 1.2 high bit rate. And there's another one coming in DisplayPort 1.3, which I'll mention later on. But the idea was, depending on what you plugged in, you didn't need to bring up all the channels. It's kind of like a sort of like PCI Express similar, like you have a number of lanes and you don't want to use all the lanes. So this was the same thing. You'd have some monitors would only need one channel, some monitors would only need one channel at the low bit rate. Some might need two channels at the low bit rate. There was also some silly things to spec. Didn't quite say like, well, if a monitor could do it with either two lanes at the low rate or one lane at the high rate, which one should I pick? And it seemed like some people picked one and some people picked the other. So there was actually some little incompatibilities even after they'd specified all of this. But the main link pretty much transports all the data. It's got like the video data in it. It's got audio data in it. And a few other little things at the time were, but mainly it's just video and audio in it. OxyH is a control channel. So it's just your standard way of signaling between the two devices to set everything up before the data starts flowing. I'll talk a bit more about what goes in there. And then there's just a hot plug interrupts so the receiver can ping the transmitter and say, wake up, I need you to talk to me and the transmitter will send a request. The OxyH channel is one megabit per second in DisplayPort 1.1. But in DisplayPort 1.2 they added a new thing called FastOxyH. Again, I'm not even sure how to support this. It's optional in the spec. But it can go at 675 megabits a second, megabytes a second. So the reason that is they're actually using that mode to put USB over. Because why wouldn't you? I'll get to something even stranger. Actually I probably haven't got a slide below and mention something stranger later. So yeah, the idea was that you can stick USB over this FastOxyH channel and you're life will be much happier because you'll have one cable to your monitor and all your USB will go over it and all your display and all your audio. So that was the original DisplayPort. That mode was called DisplayPort 1.1 is called the single stream transport. There's only really one stream going across it. It's going from your transmitter to your receiver and there's one of each. The things that came out on that you'd rather have a direct DisplayPort to DisplayPort connection. You could have a DisplayPort with a converter on the end. The converters would do then DisplayPort to VGA, DisplayPort to HDMI, whatever you wanted. You also had two classes of converter. There's a standard part of the standard called DisplayPort plus plus I think. Which some of the pins could swap into DVI HDMI compatibility mode. So you get a converter that just was very simple and it would just make it pump HDMI or DVI out. But it could only do single link. So if you had a dual link monitor you have to get an active converter and there was a bit of confusion around that area as well with our active converters and passive converters and some of the active converters needed power and some didn't. Kind of messy. The spec also specified a bunch of random things. I don't think I've ever actually seen like repeaters, replicators and switches. I've probably never actually seen them in the real world but they were all specified by the spec. So this OxyH channel is where most of the control channel, a lot of the fun things are done. This is pretty much, it's like a register space in the receiver that the transmitter can request read write out of. So there's a native OxyH transport and you basically have a register space that's I think it's about 64k now and you give it an offset and write a bunch of registers you give it an offset and read a bunch of registers. It also has a secondary I2C transport on top of it. So for compatibility with old monitors that needed like EDI, well not all monitors still use it but EDI, DDC, CI all of these other interfaces that run over I2C. Instead of trying to redo those all with OxyH they just made an I2C sort of simulation on top of the OxyH channel and all that sort of worked out pretty well. We had a handle on all of this last year and it was all going really well. But then along came multi-stream transport. So what is the multi-stream transport? They pretty much decided one transmitter, multiple receivers, they take that transport link that we had those that link that we've got, split it up into a number of individual blocks so there's 63 blocks on the link. You can assign those 63 blocks into different virtual channels and then you can point those virtual channels to some receiver block. So there it was, well we've got this large bandwidth link. I want to be able to run two, three, four X number of monitors and things off the other end of it. How this is implemented I'll get a little bit into it but I get a technical after this slide for a couple of slides just to show you how awesome I am but otherwise it's not too bad. But it's mainly there's they've sort of layered a whole messaging system on top of a bunch of the DPCD register space. So the DisplayPort register space has I think it's four, I think there are 48 byte buffers or something really silly that you write messages to go up and get replies back and then messages to go down and get replied back. So there's four of these buffers and you can send these types of messages up and down the the tree and you can send broadcast messages and a number of other things. Here's an example picture from the spec which I've nicely cleaned up and put it to a nicer picture. I hope you can read the line down the bottom you probably can't it's behind the things but I'll say it out anyway. But the idea if you look at this picture is you've got two source devices you've got five branch devices you've got like five syncs sitting all off them the numbers are they're called the port numbers so the source one looks out at zero port it sees branch two so the thing at the end is say if sync one relative to source two so source two wants to send a message to sync one it looks at it's zero port then branch one's second port and branch two's second port then branch tree's first port and that's how it sends a side band over the oxy-h it starts at the top and gives it a an address so every device in the tree has a GUID a global identifier it's a 128 bits yeah 128 bit number and nearly every device I've gotten you I've had to program that into it at start when I connect they generally don't have anything in them the spec says the only time it needs to be initialized is if you have a device that is both a display port device and a USB device some property of the USB device needs to show the GUID the same as some property of the display port device so you can match them up otherwise the OS has to assign random GUIDs to all the devices the main use for the GUID is to avoid loops you can actually plug these things back into each other and the whole idea is that you should detect that and not try and send messages the wrong way or have messages going around in circles each of these devices at the bottom you probably just see a 0.2.2.1 line that's called a relative address so you want to send a message to someone you need to know his relative address you stick stuff into a buffer with that address on the packet on it and off it goes to the guy whoever wants to sit at the end one of the things I have not built something that looks like this I don't have enough devices in fact I think everybody got like source one and maybe two other levels and even then that was quite hard and messy to get working in terms of payload and bandwidth management you have to manage the bandwidth as I said it's cut into 63 slots you can have up to 63 virtual channels probably not so useful to have 63 of them but that's how many you get the way it works is they have a number called a payload bandwidth number if you take the number for one lane at 162 we get a bandwidth number of three these are like just a for four lanes at 540 you've got a bandwidth number of 40 so the idea is every one of those 63 slots at this rate you need three pbn so you have three by 63 at one 162 at four 540 you got 40 by 63 so as an example this mode a 1920 mode by 1200 mode at 60 hertz with 30 bits per pixel this is the example given in the spec requires 689 pbn so you gotta say well that's not going to work at one lane at 162 and so how many I should have divided it out on the screen but yeah divide 689 by 40 in your head and you'll get the idea of how many of the 63 slots you're going to need and that is science for that monitor and the next monitor you need to do another chunk of slots and this worked really well if you hadn't got big monitors but I will get the why this is a problem later but that's that's sort of the basics of what mst is it's like a layer over display port that cuts up the display port transport into 63 bits unless you address individual devices on the end of it again those try those slots can be used mostly for video but there's also audio sideband stuff that you can stick over to do the audio transport yeah usb still goes over oxy-h just to give you a quick example of somebody what the protocol looks like I won't go into too much details on these are not that interesting unless I say I'm the only person outside of display port that has read them um and whoever reviewed the patches but uh link address so yeah there's an address discovery so you've got your resource something has plugged in you get a hot plugged interrupt you got to go find out who's there so link address pretty much is in a you send it out and wait for all the answers to come back and you find out how many things are in the tree um connection status notify and resource status notify there are things you receive from back up the tree to say something's changed so if further down the topology somebody unplugged a hub or a monitor gets turned off you'll get a connection that status notify to say something's changed reconfigure things to suit that change resource status notify is if the bandwidth requires something changes so maybe something something I actually don't really know when you could you know this is going to be maybe a hub get something else plugged in or it goes into a lower mode or something in that case you get a resource status notify saying I used to have this much bandwidth I've only got this much now or I used to have a small amount I have more now it lets it dynamically change uh enum patch resources is a way that you go okay I want to get a connection from my source to that sync how many pbn's can I get so what what what bandwidth exists in the system right now for me to put on that just to check that when I get a mode if I need 600 pbn's and there's only 400 that link I can't give that mode to the user so it just allows you to block cases that you know you can't fit into uh allocate payload is okay I think there's enough bandwidth reserve that bandwidth for my channel and clear payload id table is when oh oh something's happened bad everything reset just nuke the tables so they're they're pretty much the data set up in the the discovery stuff there's not a huge amount of messaging the sideband message protocols kind of messy there's like two different CRCs involved in it you got to split the messages up into small chunks and some of the messages are quite long the spec is kind of badly written like hard to sometimes interpret whether it should be big ending or little ending of exactly where things go and because I was writing this against hardware I had not really much knowledge of and I'd know examples for took me quite a while of just sticking numbers in hoping something came back eventually something did I think it was about two months but about that time the mst these are more like so you've got device down to three that's a display port device and you want to talk to it just like it was locally so you can do ask it to remote dpcd so you can look at its registers remote dpcd read and write you don't I don't tend to use them I don't think I've actually had to use them at all so far in this in way of written this remote i-squared c-read and remote i-squared c-write you want those because you want to be able to get the things like the edid display id all these you know things that happen over legacy i-squared c-buses so they're quite useful things to have use them a lot and then we've got power control so you can say turn yourself off turn yourself on because you can do that locally with the oxy-h channel as well so that's sort of like covers the bones of the spec the spec is not a public spec either which is a bit of a problem you have to be a member of visa I'll say no more about my membership status because I probably get in trouble but I have a copy of the spec and I am limited from the copy I had a copy of a draft copy of the spec looking somebody gave me a real one because the draft had quite substantial differences in the number of areas which confused me for a while but that's the sort of basics of it there's a bunch of code in the Linux kernel it's not actually a huge amount of code yet but there's a bunch of code in the kernel if you want to interest it in it join visa no don't actually just talk to me or talk to anyone else there it's definitely a it was an interesting project to start to take on from scratch I had no idea about it before the guy dropped the stuff on my desk and Nick it was it was quite interesting but so that was the problem why is this being used what was it for I originally heard of this spec the idea behind this a few years ago I say about two or three years ago in the play port first came out there was a promise of daisy chaining that was all we ever really heard about those oh you could plug a monitor into your computer and plug in another monitor into the monitor and it would make things easier and we had some customers who were like oh we've got like three monitors on our desk and we'd rather not have to run three cables down around and underneath things can we do at one that was the only sort of promise I'd really heard about it I hadn't really talked about how is this going to be used so that's why I was so surprised when this turned up in laptop docks because why it just seems like why this why does this make sense in laptop docks seems like a lot overkill and to be honest I still think it's overkill but hardware yeah wires it basically all it does is it reduces the number of wires going from the dock from the laptop to the dock from if you've got a vga port and a hdmi port on your dock or vj dvi whatever combinations they all require their own channels and wires this allows them to run essentially I'd say a display port set of wires so like six wires true and then they've got a smart device in the dock and then that smart device actually calling it smart it's probably not a great idea they have a they have a device that claims to be smart in the dock those things are again it's written by firmware guys so that explains the level of how smart it can be it locks up a lot and the only solution is to power everything off guess how much fun that was so but laptop top manufacturers love this because it saves them wires and that makes them happy so it showed up in pretty much everyone's I think all the the novos all the dels I've seen it in like three or four places now all the docks are using it and it kind of makes sense now you go okay well you've got the display port plugs in your dock and you've got your vga and dvi and they're all just one display port and then the from the laptop's point of view it's much simpler and the dock just has a little bit more smarts in it so it was a that was interesting and I've sort of accepted that's the way things are going to be from now on another area was called these things called mst hubs again I heard about these about three years ago but they have not we'd never been able to buy one we've been like can I get an mst hub oh people would announce them like AMD were big into the of trying to ship these a few years ago and you just couldn't get them but the idea behind these was if anyone's seen like AMD's or ifinity the putting six monitors onto one graphics card solution where you you know they end up with six mini display ports you know sockets on the back of a card it gets a bit cramped on the back of the card well the idea that AMD had was we'll just put one or two mini display port sockets on the back of the card and you can buy two of these little hubs and the little hubs will then break out to three display port adapters and you can plug in your six monitors it's one card that way that was it was a good idea but again these took so long to come to market eventually when I started looking at the things the docs I went oh well I may as well try and buy these and I actually managed to buy two different ones one of them is a later model that seems to be semi acceptable the other is actually it seems to be an older model because they've used two hubs one to get three ports they got two two port hubs and just sort of cascaded them so it's like yeah we obviously got the quick to market design but really crap but again you see these all over the place no you don't see them over I've got two so yeah I'm sure no one else in this room has ever seen one but maybe some day you will and they like work but yeah they were mainly an AMD type thing they're not really much you Nvidia are probably useful on as well in a way but AMD are doing it with six scanouts so they're crazy not much use so the other place these also showed up in was 4k monitors now this also took me by surprise because well monitors just a monitor but the problem that people were facing trying to get 4k monitors to market quicker than everyone else and get them into like a specific price was there was a there's a piece of hardware in the monitor that they couldn't get fast enough no what was the reason Intel chips well that was one reason yes one reason okay I'll give that reason the next time but they basically had there's two I'll give it in a second there's two in the monitor so they ended up putting there wasn't enough bandwidth to sort of like draw a full monitor off one clock off one scanout from the chip you needed they decided well we'll just use two so what they did was they split the monitor into two logical I suppose panels one on the left-hand side of the monitor and one on the right-hand side of the monitor and each of these are 1920 by 2160 high and then you add them together and you get 4k you're happy so well the other reason that Keith has just given I'll actually say it a little bit on it the bandwidth requirement for doing one monitor is over the clock limit for a number of Intel graphics chips and that would mean that you couldn't use these 4k monitors with a lot of hardware that people want to use them with so it made sense to try and just split it so there are the two sort of reasons the hardware has caught up you can get 4k so there's two types of 4k monitor on the market at the moment and this is why it gets annoying that's why 4k is going to be a bit of a pain support and there's single stream 4k monitors you get those from Acer and Samsung I think are the two ones I've seen the Samsung like as a 27 inch 4k monitor I got one when they plugged it into my laptop and it didn't work and then I plugged it into the laptop that I thought was identical but it was a model up from that and it worked fine so it was like oh clock limits you can't run this monitor on any laptop that's not heavy was essentially it was like pick up a brick it works pick up a laptop you actually want to use it doesn't work and some guy in my office had bought one of these to use with his laptop but he had like an x240 and he was like yeah it's never gonna work just go back to your old monitor and you can get the multi-stream versions the multi-streams are the double panel ones they come from I think Dell were first and I'm sure someone else is doing the same thing I just I've just got Dell because Dell gave me some for two months in May and it's still on my desk so they haven't asked for it back and after watching this I'm keeping it for a little while longer but so how does this work in terms of display port and MST what when you plug one of these in you end up getting two a hub with two ports with two connected devices on it there's a concept and MST of physical and logical ports so laptop docks would have physical ports hubs have physical ports these have two logical ports there's no real difference from the the point of view of the kernel but it's just it's information you get the port start at a higher number so you get two of these ports and they both report EDIDs and they both report EDIDs of 1920 by 2160 as the preferred mode and when you plug that into your standard Linux box generally what happens is it comes up with two monitors with like not only line number but it also gets them the wrong way around out of the box nearly always so it's quite frustrating but what happened at the same time was I plugged it into a Windows box and it did the same thing so I was like well some of these Windows drivers suck so they obviously hadn't caught up either so it was like at least who I'm doing just as crappy as Windows is a good starting line so what was required to make this better and it's still an ongoing process the idea was it's called monitor tiling they there's a protocol called display ID v1.3 again this is not part of the display port spec so someone had to lend me a copy of this as well it's a separate spec that again consortiums makes no sense it's a replacement for EDID that I think AMD may have started a few years ago and the idea was instead of just making EDID enhancing it let's just write a whole new display protocol and stick it on a different I2C address and make monitor manufacturers put it in and I wrote the spec and I started writing and then I started probing the I2C and going no there's nothing on this address on the I2C what's going on this display it says it needs display ID v1.3 and then I realized somebody actually went after a while had went oh nobody's gonna do this computer industry is insane but it's not that insane that they're going to replace this protocol that we already use so what they actually end up doing was we won't use all display ID but down in appendix like b or c there's like a this tiling specified extension and it's a display ID extension so what they did was they took the display ID extension wrapped it in an EDID extension and stuck it on the end of the EDID so it was hidden away down the end of the in an extension block so it's like why but don't ask it's like yeah there's a special EDID extension byte called display ID and then you have to open that up and pass the display ID protocol inside that and then you get the tiling information out the tiling information is IAM tile 0 it's an x and y value of where you are in the full setup how many monitors or how many tiles are in the setup what size your tile is and some information about bezels and stuff for those sort of cases where you like have a big six monitors with big bezels between them and you want to specify that sort of thing I currently ignore a lot of the sort of corner cases of this because it's painful and I have no bezels so and no one's giving me a monitor with bezels so if someone wants bezel support they know where to find me but uh this is this is protocol it wasn't that hard once I figured out where it was and what to do with it but we I've I've implemented most and I think I've just merged some of that into the kernel but that's the basics of protocol just to go on to where are we from like how does this Playport MST work with hardware side so you got your standard this is I'm pretty much x86 only I'm not really sure what's happening on ARM I don't think anyone supports on ARM yet Intel have MST Displayport MST support from Haswell onwards and it was all the Haswell laptops with the docs that were the first one to do this and cause all the pain then Intel make you make make these market differentiation things and they've got three streams of Haswell there's like ULX, ULT and normal so ULX can't do the high bit rate Displayport rate but it can do MST ULT I think can do the high bit rate but can't sustain the clock over it you can't use all of it just for one image you can split it up amongst things and then the normal one can do everything so I had a I've got a Lenovo T440S which is a ULT machine and I have a Lenovo T540P on my desk at work which is like a full Haswell so it could run the 4k monitor single stream at the high bit rate and work fine because it could clock out enough information but the pretty much the same family of laptops this one can't frustration cause the T540P has a numeric keypad so that implies it's a large brick you know it's why you need a numeric keypad don't know but it's a brick so yeah that was annoying so that's why MST monitors probably exist because Intel do things like that and restrict the clocks AMD have had actually support for this since Radeon HD 6000 series which goes back a few years it's Northern Islands was the family name back there and like I dug out a Cayman card I had I think I've had it maybe three years maybe four years could even be a little longer and stuck it into my desktop and wrote support for it and plugged two 4k monitors into it and they mostly worked I was like it's not perfect but for a couple of weeks work it was like oh wow that's really old hardware and it mostly just works probably because AMD did a lot of work on specifying DisplayPort MST Nvidia only got on board around G4 6000 time NVD9 I think is the family name but Kepler so many specifiers but yeah so from G4 6000 on I think pretty much Nvidia support it they all have different limitations Intel can only scan out three different things so no matter how many things you plug in you can only have three different images most radians can scan out six and most videos can do four so there's different ranges of how much things you can plug in at one time and get away with seeing again it's going to make purchasing decisions painful but that's why you buy support from Red Ash free ad time so driver status I first worked on the i915 for this because that was the laptop I had and that's what Red I needed for internally and we've so I had to get it working for the docks we merged it to the kernel for 317 I think it was we added I've been adding support for the tile stuff only just got merged it's in it's in DRM next right now yeah I don't think it's been merged yet it's in DRM next for the next merge window but I could be wrong I'm only a DRM maintainer so I even forget what I'm doing AMD I actually have patches sitting on my machine I'd be meaning to send them out but there's still a couple of small corner cases I'd like to fix but it mostly works it mostly works with the monitors when I've got one of the hubs plugged in and I plug in some crappy old Dell monitors I found under my desk they don't work very well at certain resolutions they work and some they don't so a bit of a pain Nuvo Ben Skag sits next to me I've given him the hubs he played around like he's got an implementation that works because he wants to clean it up as well and we had some I think he's waiting to me to get the radio unfinished so he can just copy what I did so we'll probably hopefully get that soon in terms of binary drivers FGRX and NVIDIA both support the MST stuff the tiling stuff is actually something that we're all working on not all I say all NVIDIA are interested in us working on it and nobody knows what FGRX does because we don't talk to those they don't talk to us but overall what does it mean for the rest of the stack so this kernel support is all sort of there now well the problem is that we try to hide these crazy monitors in the kernel everyone back I went to Chicago about six months ago talked to Key talked to Linus everyone's gonna hide this in the kernel hide this in the kernel it's crazy you can't let people see the fact that it's two monitors but I know Windows hasn't hidden this I know MacOS probably hasn't hidden this and then I went and tried to hide it and spent probably two weeks three weeks hacking away going I gotta hide this but there was no nice way to hide it I tried my best I think I tried twice on two separate occasions to go from scratch and do it but there's just so many when you've got a limited number of hardware resources and you tell a user space policy making thing at the end of the subenvironment that you have that many hardware resources and then you try to start to hide them after the fact it gets very confused it just doesn't understand when I say I've got you can scan out three things and then you try to scan out two things and then you take all its resources and it doesn't know what to do there's a number of other cases like page flipping and stuff is really hard to get right when you're hiding stuff in the kernel so you try to didn't work moved on try to get it fixed so what did I do expose the tiling property from the kernel to KMS so let user space know this is a tile monitor here's a property with its positioning and all that information in it I got Xorg to pass that tiling property true via RANDR so that desktop environments could find it I hacked up MOTR because I'm mainly working on GNOME because that's what Red Hat wanted I hacked up MOTR GNOME Control Center GNOME desktop package there's a few different places I just wanted to get a proof of concept out and I hacked it up and I was like oh this works quite well you can see the monitor in the control panel it looks like one monitor you can see it in MOTR things maximize the right size so that was a proof of concept so then I went back to talk to Keith Packard and we went over and we're just currently I'm currently reviewing what he proposed as so we still passed a tiling property true but then we allow the desktop environment to create like a monitor so applications don't need to think how do I create a monitor out of these individual pieces they just say there's a rectangle here and there's a rectangle here don't need to start worrying them about the fact that there's subdivisions within those at the hardware level unfortunately this means we're going to have to do fixes in like KDE SDL we'll probably need fixes SDL works SDL works oh I think SDL 2 might need some change in case you only think it's broken yeah obviously no I think SDL 2 will still need because they do strange things like setting a mode when you start games and things like that need to be fixed but there's a few things that it's very messy and I really wasn't happy in the end but it's the best we can do and hopefully we'll get it done soon myself and Keith just going to finish off the render stuff soon and then I'll start getting the desktop environments done and then I'll start wondering how to get it out to customers and other people so future fun other stuff that's sort of sitting out there that when I originally went five came on her started just turned up as well I was like oh 4k's I haven't even really gotten out yet Dell went we're going to make a 5k it's like Gillette add another razor we're good so it's like 5k's better than 4k or you want that six minute abs no so 5120 by 2880 that's a lot of pixels I haven't gotten one yet I'm going to ask my Dell friend they're like two and a half grand apparently so they are on sale but I don't know if you actually get them or they just started go we'll give it to you when we figure out how to make it and the problem with 5k was there's not enough bandwidth in a single DisplayPort 1.2 cable for a 5k monitor so even if you've got multi-stream transport it doesn't matter because there's not enough total bandwidth to do it so Dell monitor ticks two cables and you have to plug them into two separate DisplayPort ports on your machine and it requires four scanners so I know I'm making maybe I'm required two scanners I have to check that it would be really annoying for Intel if it required four but it could do yeah I think it's okay to do half oh yeah probably not probably not fast enough for Intel to do either a half on its own so the 5k monitors also made me have to redo a chunk of code because I hadn't really thought about it enough all of those tiling properties I was sending out have a group identifier at the start so you know if you've got two monitors plugged in these are from the same group these are from the same group I originally did that based off the DisplayPort they were plugged into and then went oh that's not going to work out so well because this can be plugged into all of your DisplayPorts so I had to go back and redo a bit of code for that and I realized the consequences of 5k the Apple 5k is interesting I think the iMac that they've made with 5k in it is they don't it's not cabled for a start so it's directly connected and they put a special display controller in it so it looks like an very large EDP panel from I understand so they've done something very hacky and it's only on a radio and they're not trying to ship a 5k standalone monitor yet because they understand it's not going to work very well Dell of course ship it and hope people buy it so what's coming up to solve that problem is DisplayPort 1.3 DisplayPort 1.3 adds HBR2 which is a 8.1 gigabytes per second protocol so just make it more faster so that's interesting that there's nothing with DisplayPort 1.3 in it yet as far as I know I haven't seen any hardware shipping with it from any vendor possibly AMD will come out at first but it'll be another six months I'd say so yeah 5k monitors probably won't get too successful before then but you never know another feature that's coming in DisplayPort 1.3 is a thing called FreeSync if anyone's heard of the Nvidia G-Sync I think it's the Nvidia call it's basically you variable synchronization so you don't instead of having you're sending a screen full of data you can send updates to pieces of the screen to the monitor and the monitor keeps track of them so you don't have to send it's not like you're scanning out the whole screen 60 times a second and giving it to the monitor you can just send it small blocks and update it updates on the monitor side again this is the Vezer standard I think it's part of the DisplayPort 1.3 but maybe a separate standard as well I unless one of these monitors and one of the cards that supports it falls into my lap I probably won't work on it but like usual things do fall out of the sky when you least expect them one other thing actually I haven't got on here but also I just noticed the other day is USB 3.1 is just being sort of announced and more information has come out about it and you know DisplayPort guys went and put USB over DisplayPort so the USB guys went well we're going to put DisplayPort over USB why not so that's what they did so you can now get USB 3.1 which is a brand new USB connector that's reversible and it will run DisplayPort over it so I'm not really sure what's going to happen there if you start doing you know you like USB and your USB and your DisplayPort it's just crazy so um I'm sure there's other things that will happen in the future that will equally surprise us I'm hoping the rounder work to do the monitor support will help us out because that actually allows for other things that people have wanted to do like display walls and stuff we don't really care about those but it's nice to enable other people to do things when you actually do stuff but it'll be interesting I sort of thought I wasn't really working much on display technologies this year I was wrong so that's my desk at the office and any questions the desk if anyone wants to know so I think oh this was a 4k single stream you can't see my mouse pointer can you can you see can I get it out there no can't do it but the one on the left well actually one of the very left is a really old crappy monitor but that's a 4k single stream monitor that one that's at a funny angle because it's on a weird mount is a HDMI old Dell HDMI monitor and then you've got the T540P the T440S a few Dell laptops sitting holding them up that's a 24 inch 4k monitor that's a 32 inch 4k monitor and the two that are lit up behind they're like old monitors and then that's an old laptop so that's where I have to sit in the middle of most days our office management don't they do off desk audits and they just leave our little corner alone so if anyone's any questions over in front of Mike how long did it take the cable testing guy to test the cables on your desk yeah so we have got we've got this guy come in and do tag and test every few weeks or every few months and he came in about two weeks ago and he walked into our corner and was just like oh probably should have taken it reserve the day because that's my side of that's two desks on my side and Ben's gag sits on the other side and he's got a desk and he's got we've got another desk with other stuff and he's got a desk just got Nvidia cards all over it so we actually have to help the guy so we sat there for like an hour or two just pulling cables out and testing them it was so Dave the multi-stream stuff is only in the new docks so if you've got like I think I'm going to confuse things I've got a Haswell that's on an old dock that was worked with a originally with a sandy bridge Dell but the dock connectors are the same so does that mean the dock wouldn't be using the multi-stream because it's too old to have yeah the dock requires smart so if it's a Haswell plugged into an old dock which I thought they'd change the connector sufficiently or maybe some laptop manufacturers that both set up Dell's latitude ones go back I think three generations so the same dock that worked with a a camera of Sandy Bridge Ivory Bridge still works with Haswell all right so they probably left all the pieces mostly works I've had the motherboard changed three times there's also another area that just reminds me there's another thing called of USB Docks which is a separate project which I also have done work but it's display link is the protocol technology company that does that and they don't tell us how they work and it's all encrypted and I've wasted a few months on that lately just because I don't like them but that's another project where because a lot of USB a lot of laptops are coming with this USB tree docks for the cheaper end and people really want them to work as well so we keep hearing these well intriguing talks every year of how you've managed to deal with a whole lot of craziness from a whole lot of different hardware vendors well after hardware has been released and available to the public do you think you'll ever get to the stage where vendors will give you hardware samples ahead of time so you can kind of deal with it maybe before release well this case like I was I wasn't even in this working in this area when it happened so I was like had been working on the virtualization stuff and I've been I wouldn't have actually wanted someone to give me in advance because I would hope someone else would have done the work if it had been available and you know I asked a few people like are you actually going to do this and it's like oh yeah it's on our list but it was like I only the only reason I actually started this was because red had had the hardware in-house and needed to give it to people in-house it wasn't even for customers at that point like we have customers now but at that point it was internal people need this and supporting the people that work with you is a good way to make money so on that topic we heard from Keith this morning when BigSkit but when Ben finally plays StarCrafts what's going to happen in terms of graphics development new vote development Ben replaced StarCraft with a Subaru and Preza WRX so we're not sure if Ben's going to survive another year to driving that thing hopefully but yeah he doesn't play StarCraft thankfully yes I think he played it for a while and then he realized that driving his car fast was much more important back to that oh I'm now time slicing between graphics and VRT as in I work on this mostly when I feel guilty that my monitor doesn't work properly and then I get tired I'll work on VRT and then I'll work on the reverse engineering project and then I'll get bored and work on old graphics cards because because you need a break any more questions thank you for talking as an appreciation a show of appreciation you can give this oh thank you