 Felly, dweud ymdaith o'n cael ei gweithgaf o'r fawr o'r esbyn o'r geshynion ar y Day 3 o'r MC. Mi'n rhai Chris Chaplin. Fyddo ddim yn cael ei gweithgarrach ymlaen fyddion yn gyfnog yn blaen i Altyra. Dwi'n rhain ar gyfer y cosfodraeth yn gyfnog yn cerdd i Altyra, ac os penderfynol ei gweithgarrach yma, yn gilydd i'r gwybodaeth cymrygu Preg Honestlynol. Felly, mae'r adweddau yn deislawer bwysig byw yn adweddoriaeth fel Peithgaredd, ac yn gallw'n cael ei eri, a rydym ni'n gweld ein boddach gwkhweithio'r braff iawn. Roedd eich cymdeithasol spaetholich wneudgiad busraff. Roedd gwrth темoedd yw'r roi yn ôl yn y cyfarfod treifu, ac rhan o'r gwaith o'r cyfarfod i Gweithreithiau yn Ysgrifolol i Cefnodydd. Felly, roedd eu rhannu'r roi yn cyfarfod treifu, ac roedd eu rhannu'r holl o'r rhannu sydd wedi gweithio'r cyfarfod treifu. Roedd yw'r cyfarfod treifu o'r cyfarfod treifu. Felly, yn y moment yn y gwneud, ychydig yn gweld i'r cyffredinol. A, rwy'n credu, gallwch eu bod oes eu gwneud i'r wahanol a'r cyffredinol i'r newid. I wneud eich gwneud hynny sy'n gyflawn i'r cyffredinol i lamwn i'r cyfrifiadau sy'n cyfrifiadau sy'n cael eu cyfrifiadau sy'n cyfrifiadau. A rydym yn fwy gweithio'r ffordd o gwybod ddim arno gwybod Fy fwy gweithio'r dyn nhw'n gwybod yn gweithio'r dyn nhw yw'r ddweud Mae fy nghylch y byddai'r dyn nhw ychydig Yn mynd i gwybod, y byddai'r ddau'r ddweud i ddweud y dyfodol sydd wedi cael y proseser sy'n ymddangos ymddangos dros dyfodol a'r ddweud o'u ddweud o'i'r ddweud Mae'r ddweud Os ydwch yn ydych chi'n rhaid i'w prosesu ond ond ond yn dweud y PCB yn unig. 408K. Rydyn ni'n gyntaf'r cymryd yn gwybod a chyflwg a fewn i'n gwneud yn ddefnyddio'r cymryd i'n bwysig o'r cymryd, o'r cwyson. Rwy'n gwybod i'n arddangos o'r pwysig o'r blaen a phobl o'r cyd-fiamol yw'r cyd-fiamol oherwydd yn y cyfryd yng nghymryd. Mae'r cyfryd yn y F1980. Mae'r gwrdd yma'r cymryd yma'r cyfryd. One thing that youíll notice is that everything is discrete. You got a CPU on here, theyíre a good old Z80. This device for time was very powerful but still low cost enough to make a consumer product. The good thing about this kind of device, if you're looking to do some debug, I can see 1000 places that I can do some debug. So, if I wanted to look between the CPU and the ROM, I could get out of the oscilloscope probe, yn dda i ni'n ddweud â'r ddweud, oherwydd mae'n gweithio'r opa sydd yn gweithio ddweud yn y ddweud. Rwy'n ei ddweud i'r ddweud o'r ddweud o'r syniad i'r ddweud o gweithio. Felly mae'n gweithio'r ddweud o'r ddweud. On i'n ddweud yn ychydig o dda i ddweud o'r ddweud i'r ddweud o'r ddweud. Felly mae'n 48 bydle o ran, oedda iawn o'r ddechrau, oedda i'r ddweud o'r ddweud, a 16k bank ac 32k bank. Yn ystod y 32k set o 8 ram, mae'n 64 kilobit ram, ond mae'r gwaith yw'r ddweud er mwyn fwy o ddweud sy'n ei ddim yn ddweud y ddweud ym mwy o'r ddweud a'r ddweud o'r bod yn ddweud. Yn y ddweud o'r ddweud o'r bod yn ddweud, mae'n ddweud o'r cyflawn o'r ddweud ar y bwrdd, ar y ddweud o'r ddweud o'r ddweud, mae'n ddweud yn ddweud. Yn y ddweud o'r ddweud ar y bwrdd o ran a o'r ddweud o'r ddweud o'r ddweud, rwy'n dod o'r ddweud o'r ddweud hay yn adeg ac llwyd â'r fforddiau meddwlotech yn cael ddweud sydd yn gyrch o ddweud o ddweud. Wel, mae'n ddweud ei moddol a'u prydogi i ddweud ymdryg o ymgyrch o'r ddweud, ac ydych yn gyrch o'r ddweud o'r ddweud o'r ddweud o'r ddweud o'r ddweud. Ie d firmly th y old 74 series devices, we had some glue logic as well. I think this is the last time I've actually seen a slide with aplayerteon clyENCE word England on it, so it's always good to see. Now we take a look at something a bit more up to date. This is an FPGA device with an embedded processor in there. It's a very small form factor board, this is a system on module style board and guess what? Everything is under that one same device, Felly we've got CPU, it could be a dual core, Cortex-A9, A53s are in the market now as well. The ROM, the RAM, the gate array, the glue logic, everything is under this one amorphous blob of plastic. So there's no way to get your scope out. It's even pretty much impossible to get to the ball grid array that this device is at. So we have to learn and use different techniques in order to debug these things. We still have the RAM on chip, you can see those two DDR devices down at the bottom there. But again, you'd need specialised equipment in order to be able to see how these work, to be able to debug them. And even putting a probe on something will change its behaviour, which is always fun. And as well, with these devices, we have groups of different people working on the same physical device and the same design. Now of course we're at the Linux conference, so we have the embedded Linux, or DOS in this case, software engineer. And they'll be working on codes to do with the application, to do with the operating system running on these devices. But because we have a programmable device, and a lot of the code is going to be custom in there, we also have the hardware engineer. And these two people have got to be working together on this. So software and hardware have got to come together, especially in these integrated devices. It's not so much possible just to throw a bit of code over a wall and just agree and address a map. Things can be changing all the time. That's the power of these configurable devices, and so we have to work a lot more tightly together. I've been into customers where I've been introducing the hardware and the software engineers together, and it's the first time, sometimes seriously, that they've actually been in the same room and been talking to each other, working on the same project. So the problem that we're trying to address and work out ways of resolving is that systems are getting far more complex. We've just bought out a device that's got six million logic elements, just the logic elements, let alone the processors as well, and we're doing far more on one chip. It's quite possible that you'll see a device that just has one device in the system, and then the power supplies and logic around there and memory. And the debug itself is getting more challenging. It's getting challenging for a couple of reasons. It's the first reason it's getting challenging is, as I've mentioned, everything's in one device. So there's limited opportunity to get in there and start to scope things around. But also because these devices are becoming more and more programmable, things keep changing. So it could well be that you're writing software and hardware at the same time, and the hardware is being upgraded at the same time the software is being upgraded. It's not a level playing field. It's not a known entity anymore. And guess what? The hardware can have bugs in it as well, especially if you're developing or the hardware engineer developing their own IP. So there's really a question about how do we narrow down the scope of where the issue is? Is it a software bug? Is it a hardware bug? Is it both? And how do we determine which engineer can go home and which one has to stay late? But seriously, these kind of bugs, and as I say, I cover Northern Europe, and I see a lot of customers, both big customers with lots of resources, and also small customers that are quite small and just starting up. And chasing these kind of bugs can be so destroying and can also cause companies to go under. So I was working with a customer a few years ago. They'd already spent over six weeks trying to chase a bug. They didn't know where it was. They didn't know if it was in software or in hardware. But they needed to meet a deadline in a few days' time to get their next round of venture capital funding. If they couldn't show a prototype that was working, they couldn't pay their staff anymore. It was literally the case of life or death for that company based on debugging one of these issues. Fortunately, we found the issue and we fixed it. And we're going to be using the methods I'm going to be showing you. But the idea is if you plan ahead of time, if you're at the stage where you're just starting a project and you're able to influence the hardware design, influence the PCB design, just with a few signals, it can make a lot of difference into how you can debug these kinds of system. Now, obviously, most of the people in the room I'd imagine would be software engineers. How many people describe themselves as a software engineer in the room? Yeah, pretty much everyone in the room. So we're probably more familiar with the software debug tools. So, obviously, we have GDB, KGDB, those kind of tools for either directly on the host of our Ethernet to be able to single step through application code, maybe do some kernel debugging as well. And then when you want to get down to the very low level debugging, you'll have a JTAG style debugger, something like a Lauterbuck or an AMD stream or there's a variety of third parties that you can use for debugging. And that will allow you to stop the processor, single step through the code, dump the registers, realise why something's crashed, upload new code and so on. But also on the hardware side with programmable logic, we also have debug engines as well. So Altera, now part of Intel, has a tool called SignalTap. Xilinx has got a tool called Chipscope Pro. And these tools allow you to have a look at an arbitrary signal or set of signals or buses in an FPGA and actually have what looks like a desktop logic analyser. So you can set things up, make a special test implementation of the design and then you can set trigger points to say, okay, I want to have a look at this particular chip enable and I want to trigger when that chip enable occurs and then have a look at what bus transactions were happening or what arbitrary signals were happening around the design. So if we look at a PCB, there's kind of two use cases that I'm giving examples of here. The first use case is if we've got a discrete processor, maybe you've got an arm or an Intel processor or something sat on a PCB and then you've got some kind of programmable logic. And as I say, it doesn't matter which vendor it is or what kind of logic it is. Now what you'll kind of find is that quite often the CPU will have a set of debug and trace signals that could be exposed to pins on the device. And one of those signals could be called many different things. It could be halted or stop or break. You'll tend to have a signal that will indicate that a hardware breakpoint has been hit. The processor stopped and the processor isn't running at the moment. It's kind of in a debug or halted state. And you'll tend to find that on most processors will have that kind of interface. And there's also another pin that you could potentially expose. Again, it could have a number of different names, but the idea behind this signal is that it's an external break. So this signal, if a debugger probe was to toggle that signal high, that would stop the processor. So it's like the equivalent of an external breakpoint that you could stop the processor on. If we then look at that logic analyzer that I was talking about before, as well as having all the different signals that you can hook up on that logic analyzer, you'll also quite often have a trigger input port and a trigger output port. So the trigger input port, if I toggle that high and the logic analyzer is looking for that signal, it will start acquiring bus transactions around that point. The trigger out, if bus transactions that you're looking at have occurred, that trigger output signal will go high. So what happens if we hook these sets of signals together? So what I've done here is very simple and you can imagine on a PCB this is just two traces. I've hooked the halted line from the CPU up to the trigger input line on the logic analyzer and I've hooked up the trigger output signal from the logic analyzer to the external break input to the processor. And these are programmable so both the FPGA and the processor side, I could choose to ignore those signals or enable them, but they're there on the hardware. What I can then do is I can set a software breakpoint after a particular line of code that I'm interested in and then see what's happening at the FPGA at exactly that moment in time. Also if I look at a particular bus transaction in the logic, maybe I've got a bus error or an exception, that could cause a trigger output that stops the processor and I can see exactly what code was running at that moment in time. Two signals, that's all I'm asking for. Hopefully we can fit them on the PCB. Now if you were to have an Altyra SOC FPGA, a Xilinx Zinc device, something similar to that, you'll notice that the processor and the logic analyzer and the FPGA, they're all in the same package, but these signals still exist. So you can hook these up within the design. You don't have to think about it at PCB layout time, but these signals can exist and we can use them for exactly the same process. Stepping into the ARM ecosystem a bit, and we've had a couple of presentations this week on the ARM debug infrastructure and how that's supported under Linux. We have these things called CTIs or Cross Triggering Interfaces. A Cross Triggering Interface is a way of being able to hit a breakpoint for example or a watch point on one CPU and then in a very programmatic way we can stop the other CPU or maybe not. So Cortex A9s, this is a dual core system here that we have in some of our devices. You could be running SMP Linux and running those on both cores, in which case if you hit a breakpoint on one, more than likely you probably want to stop both cores at that point to see exactly what the status of both cores were. If one of them was running your entertainment system and the other one was running the airbag system, you probably don't want to hit a breakpoint on the entertainment system and stop your airbags working. So it is possible to break that link programmatically with these Cross Triggering Interfaces that a breakpoint only affects one of the CPUs. Then at the top we've got something called a CTM, a Cross Triggering Multiplexer. What this allows us to do is communicate Cross Triggering Information between clusters of Cortex A9s for example. So ARM would define a cluster of anything up to four processors and then you could have multiple clusters in an ASIC for example. The CTM allows you to effectively daisy chain that Cross Triggering Interface between clusters of devices. So what we've got as an example, if you have a look at the bottom of this diagram here, we have a Cross Triggering Interface going out to the FPGA. So what this means is that the FPGA gains access to those signals for the debug stopped and the trigger input ports such that we can actually affect the CPUs with that information in a very programmable way. So how does this all fit together? So as I mentioned hardware and software can now start just using these simple interfaces to co-operate together within a debug environment. So I could have a louder back or an AMD stream, some kind of JTAG interface on the processor side. The hardware engineer could have an FPGA logic analyzer on the hardware side, JTAG debug on the software side and those two can co-operate with each other and we can work together hardware and software engineer to code debug a problem. Workout where it is then one of them can go home. So what I'm going to do is I'm going to demonstrate a very simple demonstration just to give you an idea about how this kind of interface can work. So I've called this a code debugging demonstration. I'm just going to go through the setup of this. It's a live demo so what could possibly go wrong. So down the bottom here I've got a demonstration board that's got an SOC device on there. It's one of our boards, an Atlas board. And that board's got an Arduino header on as well as some other GPIOs. So I've gone to Adafruit, grabbed an Arduino header and I've got this NFC RFID shield so I can read contactless cards with that. In addition I've got this blue and red siren on here so I've got some kind of a sounder connected to a relay board that I'm also connecting to the system. Now the idea behind this particular demonstration we've got a biscuit draw in the office so we've got a sales office and behind Betty's desk we've got a draw full of biscuits and those are sales biscuits. Those biscuits belong to sales. My colleague here is in support. These aren't his biscuits. These are sales biscuits. So the idea is if I get my Altera badge and I tap into this system the biscuit draw will open, everything will be fine. If Chris tries to get into the biscuit draw all hell will break loops. We'll get a very clear visual indication and a very clear audible indication that someone's trying to nick Betty's biscuits. So as far as the architecture of this system is concerned I've added a few logic blocks to the system. So down the bottom here I've got a CPU system. This is a standard processor system as we and some of our competitors use. Dual core Cortex A9 in this case. Lots of dedicated peripherals. DDR controller. All of that is hardened logic. It's tested. We know how it works. All the Linux BSPs have been generated for that. That's all good. We then have some bridges. They don't actually quite connect up to the other bus. They should be a bit higher than that. That connects into the programmable logic. In the programmable logic we can have multiple other peripherals. In my case I've got something called an alarm driver which is my custom peripheral that as a hardware engineer I've developed. Now we're going to test this design and we're going to find that this design doesn't actually work. I'm going to tap in, crystal tap in. We'll see what happens. Maybe there's going to be a bug in there somewhere. What we're going to do is we're going to grab the logic analyser and we're going to start having a look at the signals around this custom peripheral. The idea is if I've written a Linux device driver for this particular peripheral as a software engineer my thought is if I followed the agreed specification in our joint documentation, if I'm writing to the correct address, the correct piece of data and if the hardware guys logic analyser can see that bit of data on the bus then all my software is working correctly. That's a fair assumption. Good. I think so too. If I don't see a bus transaction on there it's probably a hardware fault as well but we'll look a bit further into that. There's nothing else that hasn't been done. My get home quickly card is that if I can see an Avalon standard bus transaction setting that alarm signal high at the right moment in time it's not a software issue. That's the setup we're going to use. I'm just going to start off this system. As I mentioned we're going to be using a hardware debugger and a console all on the projector. Fortunately this projector isn't quite as small as some of the other ones so let's see if we can get things running. I'm going to start off by opening a command prompt and we're going to be launching an eclipse debugger. In my case I'm using ARM DS5 but you could be using other debuggers. As I say these mechanisms are in the hardware so we can use pretty much any tool that we want. There's lots of low cost third party tools that you can use as well. I've started the debugger so I'll just show you my configuration for this before we get going. Usually I'd have a big debug probe. It just happens on this board. We can use the programming cable to act as the debugger so it's a lot more compact for this example as a demo. I'm going to be running a kernel aware debug session as an SNP system over both CPUs. I've got some options here for the debug and trace services. The first tab here is the enabling the interface between the CPU and the FPGA. If you've got a discrete CPU you probably won't need to do that stage. You can just gain access to the debug signals. If you've got a combined CPU FPGA those interfaces need to be enabled for that cross-triggering multiplexer in order to work. I've also connected up trace. We're using the embedded trace router. Some of my colleagues call it an embedded trace router but I remind them that ARM is English so it's an embedded trace router. We're going to connect that up to some on-chip memory in the system. I'm not going to be really using trace in this example too much. Then we enable trace and debug for both of the CPUs. It's quite a simple setup, not too much going on. What I'll do is I'll just start my RS2 through 2 and get from Uboot and start getting into the kernel. Ignore the I2C errors that come up. That didn't come up that time, that's good. I've got my example here and I've got my access control bit of software that's running. This is polling the NFC card over an SPI interface. I've ported the NFC tools over to this target. Quite a simple demonstration. I can, for example, take my ID badge and pop it on there. The door's open, I can open the biscuit cupboard and get everything out of there. Maybe now's a good time to check to see how the system's running. Chris, can you come along and try and open the biscuit cupboard? This is Chris, I'll look away. Better you wouldn't have normally stopped him by now. We've got our intruder alert, so that's good. We can see that the NFC subsystem is working. We can read the card, we realize it's Chris. We know that he shouldn't have biscuits, but the alarm didn't go off. There's definitely some kind of problem. I know it's not in the SPI side of things with NFC. Something to do with the alarm device driver, the alarm hardware, the FPGA, something along those lines. Let's start our debug session. I'm connecting over a USB programming cable just for compactness. This is a bit slower than if you use a dedicated bit of debug hardware. I've connected to the board. This has stopped the processors and I can see that it's kernel-aware. I can see all the active threads and what else is running in the system. I'm going to find the part of the device driver for my alarm driver that writes out to the relay that controls the siren. If I look at the functions over here, there should be a function called siren GPIO set. That's the last bit of the device driver that runs before I toggle this relay. If I look at the source code for this, here's my bit of source code for the device driver. Down the bottom here, there's this line, right L, which is the right out to the relay. I'm going to set a break point, not on that line, but the line after it. What I want to do is I want the right transaction to go out and then I want to signal, stop what happened, let's have a look at the buses to see what was going on. We should do the right, or at least send the right out onto the interconnect and then I want to be able to use this logic analyser to see what was happening in the hardware. Let me open another terminal window and we're going to add yet another piece of software onto this example. That's the logic analyser. This is called a quarter signal tap in the case of Altyra or Intel. This is the logic analyser and I'm not going to go into too much of the hardware details of this. What I've done is in this particular example that's configured onto this board, I've got a set of signals that I was doing for a genuine debug when I was bringing up the system. The NFC wasn't working at first and so on, so I was using this tool, eating our endog food as it was to debug the system. I've got a set of signals here which are the local Avalon bus standard signals connected up to my hardware device driver. I can use this to see when the chips let go high. If the chips let go high, we're talking to the address range of this controller. We can see the data values and so on. Let's just finish setting this up. I need to resize this window. I think I've got a different resolution before. We'll connect to the board over JTAG to the FPGA in the JTAG chain. We've got a few triggers that we can use. The trigger that I want to use in this case is the breakpoint out from the CPU. The CPU's got a hardware breakpoint set. That's going to toggle a signal high and that's going to be the trigger input to the logic analyser. That's that trigger input. I've got to resize all these windows to get that sorted. The trigger in over here is connected to the hard processors trigger out. I'm interested about a rising edge on that signal. As soon as that signal transition is high, that's my trigger and that's where I want to look at. I'll just double check down here. I don't seem to be able to resize these windows on this display, but just make sure that nothing else is being looked at. I don't care, so that's fine. What I'll do is get my system up and running. Make sure that's all going. What we'll do is set up the signal tap to have a look at those signals. Let's have a look here. This is looking for a rising edge of that. That's not currently running, so we're going to get the acquisition running on that. Linux is running. As soon as I get a situation where that alarm is about to be set, then Linux will stop, hit the breakpoint, we'll have a look at the logic analyzer and see what's going on. Chris, can you try and grab a biscuit again? We didn't get as far this time, but the right should have happened. If we have a look at the debugger, it's connected and the processors are stopped. We can see that we're in the access control on CPU 0 at this stage. CPU 1 wasn't doing anything useful. It's a pretty lightly loaded system, and we're in this access control executable on the other CPU. Let's have a look at the signal tap and see what was going on here. I apologise, it's a bit small on the screen. We can see that the chip select had just gone high when we received that signal. We can see that a bus transaction coincided with the point when that happened. If I have a look at the data output port, this relay is active low, so writing a zero actually sets off the relay. In this case, that bus transaction was valid. Congratulations, software engineers. You can all go home. This is definitely a hardware issue, not a software issue. But what if it wasn't? Let's have a look at the other way that we can treat this. I'll go on to fixing the issue in a minute. We know it's a hardware issue. I just want to show you that connection in the opposite way round. We can use a trigger in the FPGA in order to stop the processor and to see what was going on. I'm going to get the processor back running. I'll get rid of my breakpoint. Effectively, there's no breakpoints in this system now. I will just get these CPUs back running so it's back happy doing its thing. On the hardware side, we're going to ignore that trigger in now, but we're going to create a different trigger to the system. This trigger in will set back to Don't Care, and the trigger output from the logic analyser back into the CPU is going to be on the active low right pin going low. Right underscore n means it's active low, and when that signal goes low, that means that a right transaction is happening to the GPIO that controls the relays. If I get this running, this is free running now. There's no breakpoints in the CPU side. Let's move back to DS5. This is connected and running. Chris? Keeping Chris busy today. You deserve your biscuit at the end of this. Now, magically, the debug has stopped. We're starting to receive trace data and we can see exactly what was happening down here. Let's have a look at the signal tap window. We can see that this has exactly happened on the right pin going low. If it was that we had an erroneous transaction, say the hardware engineer was saying, someone's writing to my bit of hardware at the wrong time and they're writing dead code to it, and I think it was you. How do they prove that? Well, they prove that by setting a hardware trigger point that then stops the CPU and we can have a look at the debugger to see what was running at the time when that event occurred. In my case, this access control was running and then we can see from that we can unravel the stack and so on Does that make sense? Finally, what we'd like to do is to fix the hardware issue. Let's get this all back to a decent stage again. This should be running again. I can just check with my card just to make sure that everything is good. Then what we do is we go along to our box of tricks and we find the particular issue with the hardware so we realise that our sounder has got a battery back up and only one of them is in. Hopefully, even for a sales guy hopefully quite a simple fix to make. This was £3, very good value. Chris, can you try and get a biscuit, please? There we have it. We managed to save our system. If Chris did have a debit card to hand we could easily pay for one of these biscuits and that would stop it as well or we could make sure that someone else was authorised to do that. What Shell is the demo. Let me just wrap up with some other notes and then we can open to questions. The integration of processors along with programmable logic they're getting tighter and tighter all the time. I showed you a very simple demo but if you think about it it's pretty powerful the visibility that you can get from being able to hook a CPU and an FPGA together and then look at arbitrary signals around that and literally save a customer's business and they have done multiple times. I've worked both at both FPGA vendors and I've used these techniques at both and they genuinely do save projects. But integration is getting tighter and tighter so obviously we've had our first and second generation devices as have other customers in the industry. Stratix 10 one of our devices has just come out that's quad A53 we're starting to get zions and FPGAs together all under a big metal lid and integration is getting tighter and tighter more powerful and more powerful. So as the future goes along we're going to see more and more of that level of integration and it's going to become more and more important to think up front about how these debug connections should work. So really the one message that I want to give to you guys when you're in the project reviews for your next projects I don't mind which CPU or programmable logic device you're going to use but for the purpose of this think about how you're going to debug it. So how are you going to work between hardware and software and what mechanisms are you going to use in order to make sure that you can debug when it comes to a stressful situation you're trying to get a product out the door and you're trying to get things working. My first recommendation to any customer is make sure that you put a trace port at least on your first board and as well have these style interfaces and then hopefully hardware and software can work together have a good debugging experience and be able to ensure that these issues get resolved very quickly. So that's pretty much all I had so I'm happy to answer any questions if anyone has any questions on this. Yeah, great. So certainly you could do things such as watchpoints for example obviously that's still software intrusive in a way less so but you could do things such that of in this example wasn't stopping any of the hardware it was just sampling a window in time you could create that trigger out and then effectively gate a clock enable and then start single stepping but we didn't do that in this example so for software it was intrusive the trace layer as you know if it does back up then it could start to slow things down although realistically that might not be the case but it was pretty non intrusive but if you set a watchpoint or something similar for the CTI to trigger on then the software could continue to run. So I didn't bring this up in the presentation because it's very ulterior specific. We've got a events bus that goes from the FPGA to the debug infrastructure and that will allow up to 28 events to be time stamped by the debug clock domain from external events from the FPGA. So if I connected up the trigger output to the event bus as well my funnel with all of my instruction trace will have these events time stamped in line with the bus transactions. So if I go back so if we look on the so these are the events from the FPGA at the top here going into the STM here and that gets time stamped as it goes through the rest of the system not sure if that answers your question or not but effectively we can have time stamping we've got a cross clock domains the FPGA is not going to be running at the same clock domain as the CPU or the debug infrastructure but there will be a loose correlation of a few clock cycles difference between the two. Anyone else? Any other questions? Ok well that's pretty much all I had I hope that was useful. Thanks a lot for coming and I'll be around for a bit if there's any more questions. Thank you.