 Has anyone in here ever worked with LibUSB or PiUSB? Hands up. Okay, who also thinks USB is a pain? Okay, Sergei and Alexander were here back in at the 2063, that's a long time ago. I think it was back in Berlin. And back then they presented their first homemade or not homemade, SDR software-defined radio. This year, they are back again and they want to show us how they implemented another one. Using an FPGA and to communicate with it, they used PCI Express. So I think if you thought USB was a pain, let's see what they can tell us about PCI Express. A warm round of applause for Alexander and Sergei for building a high throughput low latency PCI-based software-defined radio. Hi everyone. Good morning and welcome to the first day of the Congress. So just a little bit background about what we've done previously and why we are doing what we are doing right now is that we started working with software-defined radios. And by the way, who knows what software-defined radio is? Okay, perfect. And whoever actually used the software-defined radio is RTL-SDR or, okay, less people but still quiet a lot. Okay, good. I wonder whether anyone here use it more expensive radios like usurps, less people, but for you, okay, good. Cool. Before 2008, I had no idea what software-defined radio is. Was working with voice over IP, software person, et cetera, et cetera. So in 2008, I heard about it on BTS, got introduced to software-defined radio and I wanted to make it really work. And that's what led us to today. In 2009, we had to develop a clock tamer, a hardware which allows to use, allowed to use usurp1 to run GSM without problems. If anyone ever tried doing this without a good clock source knows what I'm talking about and we presented this. It wasn't really an SDR, it was just a clock source. We presented this in 2009 in the 2063. Then we realized that using usurp1 is not really a good idea because we wanted to build a robust industrial-grade base stations. So we started developing our own software-defined radio which we call UMTRIX. And it was in 2011. Our first base stations were deployed in 2013, but I always wanted to have something really small and really inexpensive. And back then, it wasn't possible. My original idea in 2011, we were to build a PCI express card. Sorry, not PCI express card, but mini-PCI card. If you remember, there were a lot of Wi-Fi cards in the mini-PCI form factor. And I thought that that would be really cool to have an SDR and mini-PCI so I can plug this into my laptop or in some embedded PC and have a nice SDR equipment. But back then, it just was not really possible because electronics were bigger and more power hungry and just didn't work that way. So we designed UMTRIX to work over Gigabit Ethernet and it was about that size. So now we spend this year designing something which really brings me to what I wanted those years ago. So the X-Tricks is a mini-PCI express. Again, there was no PCI express back then. So now it's mini-PCI express, which is even smaller than mini-PCI. And it's built to be embedded friendly so you can plug this into a single board computer and by the single board computer, if you have a laptop with a mini-PCI express, you can plug this into your laptop and you have a really small software defined radio equipment. And we really want to make it inexpensive. That's why I was asking how many of you have ever worked with RTL-SDR, how many of you have ever worked with usurps because the gap between them is pretty big. And we want to really bring the software defined radio to masses. Definitely won't be as cheap as RTL-SDR, but we try to make it as close as possible. And at the same time, so at the size of RTL-SDR, at the price well higher, but hopefully it will be affordable to pretty much everyone, we really want to bring high performance into your hands. And by high performance, I mean, this is a full transmit receive with two channels, transmit, two channels receive, two by, which is usually called two by two MIMO in the radio world. The goal was to bring it to 160 mega samples per second, which can roughly give you like 120 megahertz of radio spectrum available. So what we were able to achieve is, again, this is mini PCI Express form factor. It has small Arctic 7, that's the smallest and most inexpensive FPGA, which has ability to work with a PCI Express. It has LMS 7000 chip for RFIC, very high performance, very tightly embedded chip with even a DSP blocks inside. It has even a GPS chip here. Do you see my, no, you can't see it, but. So it has, you can actually on the right upper side you can see a GPS chip. So you can actually synchronize your SDR to GPS for perfect clock stability. So you won't have any problems running any telecommunication systems like GSM 3G, 4G due to clock problems. And it also has interface for SIM cards, so you can actually create a software defined radio modem and there are audio controls projects to build one in a 4LT called the SRSUE, if you're interested, et cetera, et cetera. So it's really, really tightly packed one. And if you put this into perspective, that's how it all started in 2006. And that's what you have 10 years later. It's pretty impressive. So. Thanks. But I think it's actually up close to the whole industry who is working on shrinking the sizes because we just put stuff on the PCB. We are not building the silicon itself. Interesting thing is that we did the first approach with said, let's pack everything, let's do a very tight PCB design. We did an eight layer PCB design. And when we send it to a fab to estimate the cost, it turned out it's 15,000 US dollars per piece. Well, in small volumes, obviously, but still a little bit too much. So we had to redesign this. And the first thing which we did is we still kept eight layers because in our experience, number of layers nowadays have only minimal impact on the cost of the device. So like six, eight layers, the price difference is not so big. But we did complete rerouting and only kept two deep microvias and never used the buried wires. So this make it much easier and much faster for the fab to manufacture it. And the price suddenly went five, six times down. And in volume again, it will be significantly cheaper. And that's just for geek porn how PCB looks inside. So now let's go into real stuff. So PCI Express, why did we choose PCI Express? As it was said, USB is pain in the ass. You can't really use USB in industrial systems for the whole variety of reasons, just unstable. So we did use Ethernet for many years successfully but Ethernet has one problem. First of all, inexpensive Ethernet is only one gigabit and one gigabit does not offer you enough bandwidth to carry all the data we want. Plus it's power hungry, et cetera, et cetera. So PCI Express is really a good choice because it's low power, it has low latency, it has very high bandwidth and it's available almost universally. When we started looking into this, we realized that even ARM boards, some of ARM boards have mini PCI Express slots, which was a big surprise for me, for example. So the problems is that unlike USB, you do need to write your own kernel driver for this and there is no way around. And it is really hard to write this driver universally so we are writing it obviously for Linux because we're working with embedded systems but if we want to rewrite it for Windows or for macOS we'll have to do a lot of rewriting. So we focus on what we want on Linux only right now. And now the hardest part, debugging is really non-trivial. One small error and your PC is completely hang because you did something wrong and you have to reboot it and restart it. And that's like debugging kernel, but sometimes even harder. To make it worse, there is no really easy to use plug and play interface. If you want to restart normally when you develop a PCI Express card when you want to restart it, you have to restart your development machine. Again, not a nice way, it's really hard. So the first thing we did is we found that we can use Thunderbolt 3, which is just recently released. And it has ability to work directly with PCI Express bus. So it basically has a mode in which it converts a PCI Express into plug and play interface. So if you have a laptop which supports Thunderbolt 3, then you can use this to plug and play your plug or unplug your device to make your development easier. There are always problems. There's no easy way. There is no documentation. Thunderbolt is not compatible with Thunderbolt. Thunderbolt 3 is not compatible with Thunderbolt 2. So we had to buy a special laptop with Thunderbolt 3 with special cables, like all this hard stuff. And if you really want to get documentation, you have to sign NDA and send a business plan to them so they can prove that your business makes sense. I mean, so we actually opted out. We said not to go through this. What we did is we found that someone is actually making PCI Express with Thunderbolt 3 converters and selling them as dev boards. And that was a big relief because it saved us lots of time, lots of money. Just ordered it from some Asian company. And yeah, this is how it looks like, this converter. So you buy like several pieces, you can plug in your PCI Express card there and you plug this into your laptop. And this is with X-Turics or the plug into it. The only problem we found is that typically, UFI has security control enabled so that any random Thunderbolt device can't hijack your PCI bus and can't get access to your kernel memory and do some bad stuff, which is a good idea. The only problem is that it's not fully implemented in Linux, so under Windows, if you plug in a device which has no security features, which is not certified, that will politely ask you, do you really trust this device? Do you want to use it? You can say yes. Under Linux, it just does not work. So we spent some time trying to figure out how to get around this. There are some patches from Intel which are not main-lined and we were not able to actually get them work. So we just had to disable this security measure in the laptop. So be aware that this is the case and we suspect that happy users of Apple might not be able to do this because Apple don't have BIOS so you probably can't disable this feature. So probably a good incentive for someone to actually finish writing the driver. So now to the goal. So we want to achieve 160 mega-samples per second 2x2 MIMO, which means to transmit, to receive channels at 12 bits, which is roughly 7.5 gigabit per second. So first result, when we got this war from the fab, it didn't work. That's expected. So the first interesting thing we realized is that, first of all, the FPGA has hardware blocks for talking to a PCI Express, which called GTP, which basically implement like a PCI Express serial physical layer. But the thing is the numbering is reversed in the in PCI Express and in FPGA. And we didn't realize this. So we had to do very, very fine soldering to actually swap the lanes. You can see this very fine work there. We also found that one of the components was that bug, which is a well-known term for chips, which, at design stage, are placed mirrored. So we occasionally mirrored the pinout. So we had to solder it upside down. And if you can realize how small it is, you can also appreciate the work done. And what's funny, when I was looking at the dead bugs, I actually found a manual from NASA which describes how to properly solder dead bugs to get it approved. So this is the link. I think you can go there and enjoy. There's lots of fun stuff there. So after fixing all of this, our next attempt kind of works. So next stage is debugging the FPGA code which has to talk to PCI Express, and PCI Express has to talk to Linux kernel. Linux kernel has to talk to the driver. Driver has to talk to the user space. So peripherals are easy. So all the UARTs, SPIs, we've got to work almost immediately. No problems with that. But DMA was a real beast. So we spent lots of time trying to get DMA to work. And the problem is that with DMA, it's on an FPGA, so you can't just place a breakpoint like you do in C or C++ or in other languages. It's a real-time system running, not a system like it's a real-time hardware which is running on the fabric. So we had to... Sergey was mainly developing this. He had to write a lot of small test benches and test everything piece by piece. So all parts of the DMA code we had was routed into a small test bench which was emulating all the tricks. And as Classics predicted, it took about five to ten times more than actually writing the code. So we really blew up our predicted timelines by doing this, but at the end, we've got really stable work. So some suggestions for anyone who will try to repeat this exercise is there is a logic analyzer built into Xilinx, and you can use it. It's nice. Sometimes it's very helpful, but you can't debug transient bugs which are coming out when some weird conditions are coming up. So you have to implement some readback registers which shows important statistic... important data about how your system behaves. In our case, it's various counters on the DMA interface so you can actually see what's happening with your data. Is it received? Is it sent? How much is sent? How much is received? So, for example, we can see when we saturate the bus or when actually there's an underrun so the host is not providing data fast enough so we can at least understand whether it's a Hust problem or whether it's an FPGA problem on which part should we debug next. It's a very multi-layer problem. It starts with FPGA, PCI Express, kernel, driver, user space, and any part can fail. So you can't work blind like this. So, again, the goal was to get 160 mega-samples. With the first naive implementation we got 2 mega-samples per second. Roughly 60 times slower. The problem is that software just wasn't keeping up and wasn't sending data fast enough. So there was like many things done but the most important parts is use real-time priority if you want to get very stable results and, well, fix software bugs and one of the most important bugs we had was that DMA buffers weren't freed in proper time immediately. So they were busy for longer than they should be which introduced extra cycles and basically just reduced the bandwidth. So, at this point, let's talk a little bit about how to implement a high-performance driver for Linux because if you want to get real performance you have to start with the right design. So there are basically three approaches and the whole spectrum in between. Like two approaches and the whole spectrum in between which you can refer to three. So the first approach is full kernel control in which case kernel driver not only is the transfer, it actually has all the logics of controlling your device and only export your QTL to the user space and that's kind of a traditional way of writing drivers. Your user space is completely abstracted from all the details. Well, the problem is that this is probably the slowest way to do it. So the other way is what's called a zero-copy interface. Your only control is held in the kernel and data is provided, raw data is provided to user space as is. So you avoid the memory copy which makes it faster but still not fast enough if you really want to achieve maximum performance because you still have context switches between the kernel and the user space. So the fastest approach possible is to have full user space implementation when kernel just expose everything and says now you do it yourself and you have no context switches like almost no and you can really optimize everything. So what are the problems with this? The pros I already mentioned no switches between kernel and user space it's very low latency because of this as well it's very high bandwidth but if you are not interested in getting the very high performance the most performance and you just want to have some low bandwidth performance then you will have to add hacks because you can't get notifications from the kernel that resources available which is more data available. So it also makes it vulnerable because if user space can access it then it can do whatever it want. So we at the end decided one more important thing so how actually to get the best performance out of the bus and this is where you have to decide whether you want to pull your device or not to pull, get notified. So what is pulling? Everyone is programmer understands it so pulling is when you ask repeatedly are you ready, are you ready, are you ready and when it's ready you get the data immediately. So it's basically a busy loop where you're just constantly asking device what's happening and you need to dedicate a full core and thanks God we have multi-core CPUs nowadays so you can dedicate the full core to this pulling and you can just pull constantly but again if you don't need this highest performance you just need to get something then you will be wasting a lot of CPU resources. So at the end we decided to do a combined architecture where it is possible to pull but there's also a chance to get notification from the kernel for applications which need low bandwidth but also require better CPU performance which I think is the best way if you're trying to target both worlds. So very quickly the architecture of the system so we tried to make it very, very portable and flexible so there is a kernel driver there is a kernel driver which talks to a low level library which implements all this logic which we took out of the driver to control the PCI Express to work with DMA to provide all the details of the actual bus implementation and then there is a high level library which talks to this low level library and also to libraries which implement control of actual peripherals and most importantly to the library which implements control over our RFIC chip so this way it's very modular we can replace PCI Express with something else later we might be able to port it to other operating systems and that's the goal another interesting issue is when you start writing a Linux kernel driver you very quickly realize that while LDD which is a classical book for Linux driver writing is good and it will give you a good insight it's not actually up to date it's more than 10 years old and there's all of new interfaces which are not described there so you have to resort to reading the manuals and all the documentation in the kernel itself well, at least you get the up-to-date information so the decisions we made is to make everything easy so we use TTI and TTY for GPS and so you can really attach a pretty much any application which talks to GPS all of the existing applications can just work out of the box and we also wanted to be able to synchronize system clock to GPS so we get automatic log synchronization across multiple systems it's very important when we are deploying multiple many many devices around the world so we plan to do two interfaces one is key PPS and another is DCD because DCD line on the UART exposed over TTY because again we found that there are two types of applications one API and others that support other API and there is no common thing so we have to support both what's interesting else is that as we described we want to have pull so we can get notifications from the kernel when data is available and we don't need to do real busy looping all the time so after all the software optimizations we've got to like 10 mega samples per second still very very far from what we want to achieve now there should have been a lot of explanations about PCI express but when we actually wrote everything we wanted to say we realized it's just like a full two hours talk just on PCI express so we are not going to give it here I'll just give some highlights which are most interesting if there is real interest we can set up a workshop in some of the later days and talk in more details about PCI express specifically so the thing is there is no open source course for PCI express which are optimized for high performance high performance real time applications there is ZaliBus which is understanding as really open source but they provide your source if you pay them but it's very popular because it's very very easy to do but it's not giving you performance if I remember correctly the best it can do is maybe like 50% bus saturation so there is also Xilinx implementation but if you are using Xilinx implementation with XIA bus that you are really locked in with XIA bus with Xilinx and it's also not very efficient in terms of resources and if you remember we want to make this very very inexpensive so our goal is to be able to fit everything in the smallest RTX 7 FPGA and that's quite challenging with all the stuff in there and we just can't waste resources so decision is to write your own PCI express implementation that's how it looks like we're not going to discuss it right now there were several iterations initially looked much simpler turned out not to work well some interesting stuff about PCI express which we stumbled upon is that it was working really well on ATOM which is our main development platform because we are doing a lot of embedded stuff worked really well when we tried to plug this into Core i7 just started hanging once in a while so after like several days maybe debugging Sergey found that there is very interesting statement in the standard which says that value 0 in byte count actually stands not for 0 bytes but for 4096 bytes I mean that's a really cool optimization so another thing is completion which is a term in PCI express basically for acknowledgement which also can carry some data back to your request and sometimes if you're not sending completion the device just hangs and what happens is that in this case due to some historical heritage of x86 it just starts returning you FFF and if you have a register which says is your device okay and this register shows one to say the device is okay guess what will happen you will be always reading that your device is okay so the suggestion is not to use one as the status for okay and use either 0 or better like a 2-bit sequence so you definitely sure that you are okay and not getting FFFs so when you have a device which may fail at any of the layers you just got this new board and it's really hard to debug and there's a lot of memory corruption so we had a software bug and it was writing DMA addresses incorrectly and we were wondering why we are not getting any data in our buffers and at the same time after several starts operating system just crashes well that's the reason why there is this UFI protection which prevents you from plugging in devices like this into your computer because it was basically writing data like random data into random portions of your memory so a lot of debugging a lot of tests and test benches and we were able to find this and another thing is if you initialize your driver incorrectly and that's what's happening when you have plug-and-play device which you can plug and unplug then you may end up in a situation where you are trying to write into memory which is already freed by operating system and used for something else very well-known problem but also happens here so why DMA is really hard is because it has this completion architecture for reading data writes are easy, you just send the data you forget about it but for reading you really need to get your data back and the thing is it looks like this I really hoped that there would be some pointing device here but basically on the top left you can see requests for read and on the right you can see completion transactions so basically each transaction can be and most likely will be split into multiple transactions so first of all you have to collect all these pieces and write them into proper parts of the memory but that's not all the thing is the latency between request and completion is really high it's like 50 cycles so if you have only single transaction in fly you will get really bad performance you do need to have multiple transactions in flight and worst thing is that transactions can return data in a random order so it's a much more complicated state machine than we expected originally so when I said our architecture was much simpler originally we did not have all of this and we had to realize this while implementing so again here was a whole description of how exactly this works but not this time so now after all these optimizations we've got 20 mega samples per second which is just six times lower than what we are aiming at so now the next thing is PCI express lane scalability so PCI express is a serial bus so it has multiple lanes and they allow you to basically horizontally scale your bandwidth one lane is like x then two lane is 2x four lane is 4x so the more lanes you have the more performance you are getting out of your bus the more bandwidth you are getting out of your bus so the issue is that a typical mini PCI express the mini PCI express standard only standardize one lane and second lane is left as optional so most motherboards don't support this there are some but not all of them and we really wanted to get this done so we designed a special converter board which allows you to plug your mini PCI express into a full-size PCI express and get two lanes of working and we are also planning to have a similar board which will have multiple slots so you will be able to get multiple X-tricks SDRs onto the same carrier board and plug this into let's say PCI express 16x and you will get really a lot of IQ data which then will be your problem how to process so with two A-axis it's about twice performance so we are getting 50 mega-samples per second and that's the time to really cut the fat because the real sample size of LMS 7 is 12 bits and we are transmitting 16 because it's easier because CPU is working on 8, 16 and 32 so we originally designed the driver to support 8-bit, 12-bit and 16-bit to be able to do this scaling and for the test we said okay let's go from 16 to 8-bit we'll lose some dynamic range but who cares these days still stayed the same still 50 mega-samples per second no matter what we did and that was a lot of interesting debugging going on and we realized that we actually made another not really a mistake we didn't really know this when we designed but we should have used a higher voltage for this high-speed bus to get it to the full performance and at 1.8 it was just degrading too fast and the bus itself was not performing well so our next prototype will be using higher voltage specifically for this bus and this is kind of style which makes designing hardware for high-speed really hard because you have to care about the coherence of the parallel buses on your system so at the same time we do want to keep 1.8V for everything else as much as possible because another problem we are facing with this device is that by the standard mini PCI Express allows only 2.5W of power consumption no more and we were very lucky that LMS-7 has so good power consumption performance we actually had some extra space to have FPGA and GPS and all that stuff but we just can't let the power consumption go up our measurements on this device are short about 2.3W of power consumption so we are at the limit at this point so when we fixed the bus with a higher voltage now it's a theoretical exercise because we haven't done this yet that's planned to happen in a couple of months we should be able to get to this number which was just 1.2 times slower and then the next thing will be to fix another issue which we made at the very beginning we procured a wrong chip just one digit difference you can see it's highlighted in red and green and this chip supports only generation 1 PCI Express which is twice slower than generation 2 PCI Express so again hopefully we'll replace the chip and just get very simple doubling of the performance still it will be slower than we wanted it to be and here is what kind of like practical versus theoretical numbers well as every bus it has overhats and one of the things which again we realized when we were implementing even though the standard dies as a payload size of 4 kilobytes actual implementations are different for example desktop computers like Core, Intel Core or Intel Atom they only have 128-byte payload so there is much more overhead going on the bus to transfer data and even theoretically you can only achieve 87% efficiency and on Xeon we tested and we found that they're using 256 payload size and this can give you like 92% efficiency on the bus and this is before the overhead so the real reality is even worse and the interesting thing which we also did not expect is that we originally were developing on Intel Atom and everything was working great when we plugged this into a laptop like Core i7 multi-core really powerful device we didn't expect that it wouldn't work obviously like Core i7 should work better than Atom no, not always the thing is we were plugging into a laptop which had a built-in video card which was sitting on the same PCI bus probably manufacturer hard-coded higher priority for the video card than for everything else in the system because you don't want your screen to flicker and so when you move a window you actually see the late packets coming to your PCI device we had to introduce a jitter buffer and add more FIFO into the device to smooth it out and on the other hand Xeon is performing really well so it's very optimized that said we tested it with discrete card and it performs everything by whooping 5-7% that's what you get for the price so this is actually the end of the presentation so we still have not scheduled any workshop but if there is any interest in actually seeing the device working or if you're interested in learning more about the PCI express in details let us know, we'll schedule something in the next few days that's it and I think we can proceed with questions if there are any okay, thank you very much if you are leaving now please try to leave quietly because we might have some questions and we want to hear them if you have questions please line up right behind the microphones and I think we'll just wait because we don't have anything from the signal angel however if you're watching on stream you can hop into the channels and over social media to ask questions and they will be answered hopefully so that microphone what's the minimum and maximum frequency of the card? you mean RF frequency? now the minimum frequency you can sample at the most RTL-SDR device is quite only sampled at over 50 MHz is there a similar limitation at your card? so if you're talking about RF frequency it can go from almost 0 even though it works worse below 50 MHz it goes all the way to 3.8 GHz if I remember correctly and in terms of the sample rate right now it works from about 2 MHz per second to about 50 MHz right now but again we're planning to get it to this number as we quoted okay, the microphone over there thanks for your talk did you manage to put your Linux kernel driver to the mainline? not yet, I mean it's not even fully published so I did not say in the beginning, sorry for this we only just manufactured the first prototype which we debugged heavily so we are only planning to manufacture the second prototype with all these fixes and then we will release the kernel driver and everything and I'm not sure we will try or maybe one try haven't decided yet thanks that will be the whole Azure experience okay, over there hey, it looks like you went through some incredible amounts of pain to make this work so I was wondering aren't there any simulators at least for parts of the system for the PCIe bus, for the DMA something any simulator so that you can actually first design the system there and debug it more easily yes, there are available simulators but the problems they are non-free so you have to pay for them so, yeah and we chose the hard way okay, thanks okay, we have a question from the signal angel are the FPGA codes, Linux driver and library codes and the design project files public and if so, did they post them yet? did we have posted them yet? can't find them on xtrx.io yeah, so they are not published yet as I said, we haven't released them so the drivers and libraries will definitely available, FPGA code we are considering it will also be available in open source but we will publish them together with the public announcement of the device okay, that microphone yes, did you see any signal integrity issues between the PCIe bus or on those bus to the LMS chip line microchip, I think that's doing the other did you try to measure signal integrity issues or because there were some reliability issues, right? yeah, we actually, so with PCIe we never had issues if I remember correctly, it was just working actually, the board is so small and when there is small traces there is no problem in signal integrity, so it's actually save us yeah, designing a small board is easier with the LMS7, the problem is not the signal integrity in terms of difference in the length of the traces but rather the fact that the signal degrades over speed in terms of voltage and drops below the detection level and like all this stuff, we used some measurements I actually wanted to add some pictures here but decided it's not going to be super interesting okay, microphone over there yeah, thanks for the talk how much work would it be to convert the 2x2SDR into an 8 input logic analysis in terms of heart and software so to have a really fast logic analyzer where you can record unlimited traces with logic analyzer so basically it's just also an analog digital converter and you largely want fast sampling and a large amount of memory to store the traces well, they just think it's not the best use for it probably I don't know maybe Sorgi has any ideas but I think it's just maybe easier to get a high speed ADC and replace the LIME chip with a high speed ADC to get what you want because the LIME chip has like so many things there specifically for RF yeah, the main problem is you cannot just sample original data you should shift it over a frequency so you cannot sample original signal and using it for something else except spectrum analyzing is hard okay, thanks okay, another question from the internet have you compared the sample rate of the ADC of the LIME DR chip to the USRP ADCs and if so, how does the lower sample rate affect the performance? so comparing low sample rate to high sample rate we have not done many testing we haven't done much testing on the RF performance yet because we were so busy with all this stuff so we are yet to see in terms of low bitrate versus sample rate versus high sample rate well, high sample rate always gives you better performance but you also get higher power consumption so I guess it's the question of what's more important for you okay, over there I've got a mixer bypass so you can't directly sample the signal is there a way to use the same antenna for send and receive yet? actually there is input for ADC but it's not a bypass it's a dedicated pin on LMS chip and since we are very space constrained we didn't erode them so you cannot actually bypass it okay, in this in our specific hardware so in LMS chip there is a special to rephrase a special pin which allows you to drive your signal directly to ADC without all the mixers, filters, all this radio stuff just directly to ADC so theoretically that's possible we even thought about this but it doesn't fit with design can I share antennas? because I have an existing laptop with existing antennas but I would use the same antenna for send and receive yeah, so I mean that depends on what exactly do you want to do if you want a TDD system then yes if you want an FDD system then you will have to put a small duplexer in there but yeah, that's the idea so you can plug this into your laptop and use your existing antennas that's one of the ideas of how to use X-tricks yeah, because that's all for connectors yeah, one thing which I actually forgot to mention like I kind of mentioned in the slides is that any other SDRs which are based on Ethernet or on USB can't work with CSMA wireless systems and the most famous CSMA system is Wi-Fi so turns out that because of the latency between your operating system and your radio on USB you just can't react fast enough for Wi-Fi to work because probably you know that in Wi-Fi you carry your sense and if you sense that the spectrum is free you start transmitting doesn't make sense when you have huge latency because you know that the spectrum was free back then so with X-tricks you actually can work with CSMA systems like Wi-Fi so again it makes it possible to have a fully software implementation of Wi-Fi in your laptop it obviously won't work as good as your commercial Wi-Fi because you will have to do a lot of processing on your CPU but for some purposes like experimentation for example for wireless labs and R&D labs that's really valuable thanks over there what PCB design package that you use? I'll film and I'd be interested in the PCI Express Workshop would be really great if you do this one would you say again? would be really great if you do the PCI Express Workshop PCI Express Workshop okay I think we have one more question from the microphones and that's you great talk and again I would appreciate the PCI Workshop if it ever happens what are the synchronization options between multiple cards? can you synchronize the ADC clock and can you synchronize the presumably digitally created IF? yes so unfortunately just IF synchronization is not possible because LIME chip doesn't expose a low frequency but we can synchronize digitally so we have special 1PPS signal synchronization we have lines for clock synchronization and other stuff we can do in software so LIME chip has fast correction registers so when you measure the phase difference so you can compensate it on different boards tune to a station a long way away and then rotate the phase until it aligns thank you a little tricky but possible so that's one of our plans for future because we do want to see like 128x128 MIMO at home okay we have another question from the internet actually we have two questions one is what is the expected price after prototype stage and the second one is can you tell us more about this setup you had for debugging the PCIe issues so you can repeat the second question it's setup debug more about the setup you had for debugging the PCIe issues second question I think it's most about our next workshop because it's a more complicated setup so we mostly remove everything about it in our current presentation yeah but in general in terms of hardware setup that was our hardware setup so we bought this PCIe express to Thunderbolt 3 we bought a laptop which supports Thunderbolt 3 and that's how we were debugging it so we don't need like a full-fledged PCIe we don't have to restart it all the time so in terms of price we don't have the fixed price yet so what Oleg can say right now is that we are targeting no more than your BladerF or HackerF devices and probably even cheaper for some versions okay we are out of time thank you again Sergei and Alexander