 So I'm going to go ahead and start. So welcome to my talk. It's about Grinradian 2019. The world has changed. Let's change Grinradio. What we're going to do in the next 40-something minutes is basically we look back, but I'm going to keep this really really brief on the last five years of 3.7 which come to an end, I think. That era will be over soon. Then we'll talk about what happens next when we'll actually release 3.8 and how that will happen. We'll talk about what happens afterwards and I guess a lot of the things that will be interesting to most of you will be the Q&A session. I'm trying to kind of rush through this so that I can actually take questions. Then we get started really shortly. I was actually going to skip that slide, but I added because just last year at FOSDEM I became the chief architect of Grinradio. Now I wear several hats, the guys that basically paid my hotel here. I supply grumpiness to the editor support. I'm also a research assistant at CEL, at Karlsruhe, which makes me proud that my former bachelor advisor did a quick talk on information theory here. Aside from that, I basically freelance. I said mainly I'm here because I'm the chief architect of Grinradio. Let me talk about Grinradio. So Grinradio, the version that you probably use if you install Grinradio in some binary form is 3.7-Sompton. 3.7 has been around for quite a while. We basically never got around to increasing the minor versioning number. 3.7 has been around since 2013 and somewhere around this time, like somewhere late 2014, the next branch was actually branched of the master branch to develop the next version of Grinradio, which was to become 3.8. Then there was five years of 3.8 in the making. You can imagine that if you have two branches, one being the master branch from which you basically do nothing but release compatibility versions, and the next branch where you do basically everything but can't really agree on which direction to take, this is going to be like a really complicated time for the project. So this really took a long time and we finally got gotten around to merging the next branch back into the master branch, which was a great, great task and I'm really thankful for Andre, Martin and others helping me with that. I wouldn't have been able to do that myself alone, definitely not. But the state we're currently in is that we have kind of the last 3.7 releases have been of a main 3.7 branch and everything that happens on master is like towards the future, towards 3.8. So that's where we are. Our last 3.7 release has been a while in the past and we're getting ready to get 3.8 out the door. So what happened in 3.7 since basically I took over is first of all like I introduced semantic versioning. That was kind of long overdue. I also kind of formalized the change look format that was my thing and I said we retired the idea that we have a next branch that is supposed to be like the next great thing and we all mainly do on back ports. We kind of like do everything on master now and for like releases we have a maintenance branch that has like the release in its name and for example 3.7.14.0 would be tagged of main 3.7 not of master. So with that we're kind of accelerating development of the radio that actually has worked out. We've got a lot of things that were in the pipeline for a long, long time merged in during the last year so I'm very happy about the speed that all this has taken up. So with me talking about releasing 3.8 obviously everyone that's using the radio productively like there's people running you know satellite ground stations of this they might be asking themselves well are we going to support 3.7 for any longer then and the honest truth is like we've been supporting this for five years and everyone knows that if you're doing software long enough then every single part of its behavior kind of becomes API whether it's documented or not. So yeah we will be supporting that for a while longer. I can't actually give you a timeline for that but to be honest like it will probably be as long as like my least favorite distribution Ubuntu 16.04 will be supported because that's basically the thing that holds up like has all the dependencies that still apply to that version. So once people can't compile that reasonably on their modern machines anymore there's really no point in us supporting it any longer. If you want to contact me and say hey I run big software on that I'd like to help you support 3.7 in six years I'd be very happy but I honestly don't see that happening so for now I promise that we're not going to drop 3.7 soon but I promise that we will at some point move on with our lives. So but what that means is that 3.7 will never be Python 3 compatible 3.7 will never be C++11 3.7 will not get any of the cool new more fundamental features that 3.8 has. If we're doing bug fixes on 3.8 there's a high chance that we can back port that we'll do that I've been doing that in the past but you know there's only so much time so if you've been working with 3.7 for the last five years chances are all the bugs that are in there I have already been fixed by you or not noticed by you so yeah. So enough chatter about 3.7 what's just all this 3.8 about? This is kind of like the conclusion of my talk at grcon this year so I'm going to keep this rather short if you want to know more about that I think the ignoradial conference videos are online by now but I really want to I hope so I'm actually not sure but to highlight a few things so the main reason that we need 3.8 is really that you know Python 2 is no longer an option Qt4 is no longer an option so we had to move forward with these dependencies. Cheetah has been like kind of that also it's only Python 2 so our core XML parser for like GSC has gone away so we had to reawand a lot of stuff so all the green dots here are happy faces in my head we have one dependency progression like look for cpp is becoming more and more problematic we don't need to replace that also we need a better logging system and it's not easy to adapt other systems to ours because well we have specific multi-threading needs we have specific problems with runtime discovering of this subsystem so this is going to be interesting we threw out a bit of stuff so to anyone who's using gr committee I'm kind of sorry but I know no any of you so if you need that please do contact me I'm I don't care about it right now if you need it we'll keep it but so far I have not met a single user so might as well not keep it the code quality is questionable because none of us understand what it does so I do want to highlight a bit of like this this is all the opposite of exciting right this is all like maintaining stuff this is all about you know painting the walls they've become slightly gray like the interesting stuff is like one shout out I want to do is like to Swapna who's sitting in the corner over there who's actually come from India to here he's he's been doing the google sum of code for us this year he's been improving gm2 which has become way more easy like nicer to use has better code structure so I'm very thankful for that it has new features like proper detection of blocks that I'm very happy about then we also have Hawke on which I see out there sorry yeah he's he's also here he's been here last year too he's been doing GSOC the year before now we have C++ code generation in GRC so you're used to GRC when you hit the run button it it generates a python module and like python program and executes it we can now generate C++ code so you can you know compile cross compile and run it on your toaster whatever so and the feature that like I like by numbers most users will be interested in is like that GRC has overall become way nicer way cooler what I mean with that is that you can actually have like that the canvas now is a vector based canvas you can actually export PDFs which might be interesting for people who you know put things into printed publications and also like the code base really really really looks a lot nicer structured inside as lot less duplication of functionality cleaner abstraction and I hope that that's really a big hope that this enables every one of you who like probably will find a bug in GRC because we have so many that everyone can get a free one that enables you to just go in and say okay this is easy this is just python I know python let's fix that fix that so that's that's like the exciting things that happen in GRC so these are like the main setting points that I have for 3.8 aside from the whole like being nicer to forward development things so this this question has basically been asked well like years before I even became maintainer when will we release 3.8 and the honest truthful answer is when it's ready and I promise we're not going to do it a minute later like the day it is ready we will release it with all like taking care that it is actually ready but we will release it as soon as possible what we'll not do is like release it a month earlier because that means that someone's going to ship a broken system and we're not going to get that out for years so I do have like a few blockers to highlight here first of all like there's been ongoing efforts to modernize the CMake we use which is best done by Andre which is very important because A it makes detecting certain libraries with like the standard tools possible at all and B it makes using narrator in your own projects way way way easier and way way way safer so this is really something that I want to get out there with the 3.8.00 release because doing it any later only will complicate things so that this is already orange orange that you can't tell this on this be more but this thing is actually work in progress this thing is something that we've not even tackled yet why well it depends on the first one what new template means is that right now we're expecting you as great users to well use can radio and write cool sdr software with that and most people do that by having their own out of three module but having your own out of three modules wouldn't be possible if we you know in the past had not have a template to work from where even like gmot tool is actually just the tool to take that template customize it for you purposes and then add the functionality that uses just so you don't have to write boilerplate code and we don't have that template yet so that's kind of a blocker because if I put out gr 3.8 and no one's able to actually like write software for that then adoptions will probably be about zero then I mentioned look for cpp so we have to reimplement the logging system this is going to be fun there's specific industry needs that you need to be addressed we know the people that are actually willing to help us there to contribute there but we need to like come up with a with a reliable infrastructure idea at least then we also have like a lot of bugs that are simply like oh this crashes in grc for example and like some of them are really just blockers so we need to fix those and like this this like I came up with this like last minute yesterday night obviously this belongs in the same category as an OT template people have not only you know have the tools but also know how to use the tools so we need some documentation we need to update our tutorials this kind of thing so this is this is a bit of a shout out to mark liftman who has been like our new documentation officer he's very very motivated but we've like not been able to actually like nail down how things will work um so that we can tell it to newbies so um yeah that's something that we need to come up with so if you want to help with that because you're really really eager to get 3.8 and I can see in all your faces that you definitely are um do go to like our radio um github repo click on projects and you'll see the release 3.8 project and what you see is that we have open issues issues that are work in progress and a lot of closed issues so do pick one of the open ones and and command on that or fix it I probably prefer fixing it um but to be honest like reproducing a bug is really really helpful for a lot of things that we simply don't know whether they're just broken on that single machine because someone fiddled around with something else or are actually broken so go into that project find an open problem or go into our 248 or something issues that are still open and you know read through them that that would be very helpful to me and yeah that will like help me answer that question uh sooner so assume we have fixed all like the open things what what what will you yeah so after we got the blocker sorted out I'll go in and take release candidate one what will that mean that that will mean that we that I have something that I believe runs on every computer that should be running in radio um realistically there will be bucks so what we need to do with that release candidate is get it out on the street so what we will do and that's that's a new like something that we've not been overly good at so far is like have our own binary packages with that so that people can actually you know add like if you're on your boon to add a ppa if you're on some red add or it's add like the the um source if you're on I don't know mac ports I do the appropriate thing and just get new radio release candidate so um then we'll wait some time for the bucks to trickle in um there will be like that actually depends on on which time that happens if it happens around like someone's doing like someone from the core team doing um vacation that might take a week longer but let's let's say this this time frame was here two to three weeks after that after we think that most of the bucks have been fixed we will do something that is super nice to me we will just apply clangformer to the whole code tree because if you've been looking at at generated core code then you will notice that there's a lot of personality in the way indents are used or a lot of well freedom in the way people put braces and on lines and things so um um that automated um code formatting will be a single commit kill all um uh thing which will hopefully not break anything I'm pretty sure about that because you know the clang people tend to know what they're doing um but yeah um that'll that'll probably like um allow us to release release candidate two and at the same time we can branch off the main 3.8 um branch and start working on master for the 3.9 so after we release our Rc2 we like wait another 10 work days to let you know the latest blockers come in if any then uh we release 3.8.0.0 and then we have a party which probably consists of me having a barbecue or something um uh so um that's like I hope to like raise a bit of a feeling of transparency here that's like we're all sorry that it took forever but we're trying to be like pretty forward about getting it out the door so so what what's what's going to happen once that we have 3.8 release yeah well as mentioned um we'll have a main 3.8 branch from which like 3.8.1 or whatever will be released so um this this is the place where bug fixes for that release will be here on on master we're not going to develop for 3.8 if we decide that the feature is cool enough to be part of 3.8 we'll gladly backport that and in the beginning it will be super easy um but um later on we'll focus like master will only be future development so this is 3.9 we'll not have next branch anymore so um master will be 3.9 um but as as as promised like 3.7 point whatever will still be around we'll still care for a few years um but that at at the moment it's mainly like a single bug fix per month or something so there's not going to be a much of traffic on on main 3.7 okay um so as mentioned 3.8 is kind of a housekeeping release a keeping my car running so that i can still run errands release it is also a transitional release in what way we put a lot of lot of effort in you know maintaining the ability to still um use python 2 which is really unusual for projects most projects that like you know needed needed to let go of python 2 just you know switch over to python 3 um we didn't do that because we anticipated that people would want to port their um out of three modules to a newer ignoradio version but didn't want to like rewrite most of their code if python 3 breaks that so we want still want to enable people running 3.7 to on the same system build 3.8 so um that's why we're doing 3 2.7 and 3 point whatever and parallel on 3 ignoradio 3.8 well we will not do that afterwards because it's really really a lot of effort um but we see it as like the golden bridge making your out of three module future proof is like okay port it to 3.8 that basically takes care of you know all the architectural stuff and if you want to port it to python 3 on the way that would be cool we greatly appreciate that if you don't do it later um but be aware that 3.9 will be python 3 so um 3.9 um yeah um is the thing that I'm like most excited about right now because it's getting so tangible right now um first of all I kind of like mentioned that multiple times because it's exciting if you look at code a lot like um after we um code for method the whole thing we will be able to like we're obviously going to stop having that feature freeze that we have right now are focusing on 3.8 um and I promise and this is really very dear to to my heart it's like we're not going to take another five years to release 3.9 we need like features to be available sooner so um yeah um that's how we'll like directly after release 3.8 we'll start working on 3.9 and what will that entice um so what are the bigger scale uh plans that I have for the future of new radio well first of all I'm I'm going to be more inclusive um and two two aspects like um first of all um I feel that new radio should be more like it is like like the core of your ecosystem and should work out of the box for a lot of purposes so um like the usual user of new radio starts with an outer L dongle or a hacker f or probably not anymore with with a user p so you guys start installing new radio and then compiling gr osmo str from source which is which is fine I like fear osmo str um but it's an additional burden to us users and it's not consistently packaged and things get outdated and um so this this is not something I want to keep that way so I think that new radio should come with batteries included then the other way around means that if I want the ecosystem to work for everyone out of the box that I have to care about the ecosystem right so what I'm going to do is like consider things that I see as center infrastructure and upstream it so well there's a lot of people that have like gr bus out there or gr food that basically has some wrappers some stream operations that they reuse in their own ot multiple times and that's all functionality that should basically be in the radio and we should also notice when it breaks when some some architecture change breaks gr phosphor that would be like kind of bad but we will notice until someone wrote an email to the mailing list that says hey I've tried 3.8 point 40.0 and um you know now all my colors are inverted and my dog started barking um uh so that's something that we need to have like be more of a yeah umbrella organization that actually builds your code too and tries that so that means that we'll have more things in tree then as by basically try to line out like obviously we need to be more future future proof so um we need to and that's that's that's like easy to say for a maintainer we need to track dependencies more aggressively we need to be like oh yeah whatever Qt5 is going away in half like in a year or so it's not happening but Qt5 is going in a year so let's not space our new 3.11 on Qt5 let's be more aggressive at adopting newer dependencies that only works if we're doing a like more regular release cycle so um this is this is an interesting aspect that also requires us to be very clear about how long we're going to support things like 3.8 or 3.9 um so that that's something that we need to come up with a clear strategy for um also what that means is that we remove all burns and replace them with new ones um I mean with modern alternatives for for problems um that we're having right now um I could like name quite a few of these um but I'm going to that in detail in a minute um what we also need to do and that's that's kind of obvious like we had a birth of feather meeting yesterday with with neuradio users and a lot of these were single time users new users would like to be users and it turns out that um installation is a major major obstacle still like in 2019 imagine that um people can't just use like a binary package to get like the recent version of neuradio from from the development tree we have no nightly builds we have no binaries that are outside like every distros maintenance scope so um that's something that we need to change um so personally I have this dream where like I don't only have a nightly version of neuradio somewhere like whatever a ppa that ships nightly versions of neuradio but also like nightly versions of GQRX because that depends on neuradio and nightly versions of GRAAS because tracking ships is cool but you know it depends on neuradio so um that that leads me to like a zoo um of our team binaries that we would need to build like on a regular basis um which would kind of make pi bombs obsolete um which I think is a bad thing um um but um uh yeah that would be pretty cool if I could just go and say app get install you know gr my latest module and and people would get my latest module as binary um and obviously um better docs well obviously you can always document better I used to be feeling bad about neuradio documentation but a special brandon's talk um opened my eyes towards like the badness of other people's documentation and um yeah I I'm getting less and less apologetic about our documentation we just need to be more clear at the basic level so that people are able to research stuff themselves and we don't need to like document the hell out of every single block so that's that's an easier task less taunting that than than I used to think and lastly we need to be better overall like obviously like neuradio has been a great project for like 15 years like no 17 years by now um but that also comes with like some technological that we have um basically we can't exchange our scheduler right now there is this thread per block schedule which is kind of cool if you have about as many threads as we have blocks but most modern machines have like four to 32 cores and like a single um like a single balance flow graph has like 200 blocks so you might see there's there's some mismatch here so we need to be clever about um the way we schedule things we need schedulers to be able to adapt to the problem and to the platform they're running on um then we have this problem where neuradio is really really focused on running on one and the same machine but in in in reality neuradio should be a framework for running connecting blocks and doing signal processing right so it would be super desirable to be able to you know have few blocks run on my laptop and then have this block somewhere in the cloud running like the LPC decoder because that's super super compute intense and that's actually what like mobile network operators are doing they're running more and more of their software wherever that's the cheapest thing to do um and and that's somewhere we need to be but that's really hampered by the fact that we share state between blocks huh um and this is sadly not limited to um samples that cross uh in buffers that's also implemented by things like having an equalizer that both my OFDM receiver need to modify and the OFDM equalizer block actually needs to use so we need to have the ability to pass messages messages with info and be more RPC oriented overall currently our RPC framework is basically broken I gladly admit that um I see a grin over there yes it is um uh we need to like re-implement that and yeah thanks Martin um uh yeah that all boils down to actually writing a new scheduler so that's that's like a very rough idea where I want to take things um but hey um having such plans is awesome um as we all know we're not like kind of 300 people per uh company that dedicates their time to development so uh how much of that is going to happen in 2019 so I can't gladly say that we have a lately foundation for everything in one of these points we're not able to tackle like finally solve any of these in 2019 uh completely but like what we'll do is for 3.9 and I'm very happy to say that we'll be able to upstream GR SOPE so um like thanks to the cooperation with like the developers and especially the Libre Space Foundation um we can um make your favorite SDR work with the radio out of the box that's kind of true um you still need like their the the respective SDR driver but the fact that SOPE is basically a plug-in based architecture so that you can load a shared object at runtime means that at installation time of GRU radio you don't have to have like your the hacker F driver already linked into that you can load it later on and that's that's a very nice thing um it enables us to like more aggressively develop features instead of trying to you know having to have to link to every single hardware driver that that might be there um we'll also upstream GR IO which is kind of special purpose um this is I kind of view this as as a test balloon because GR IO doesn't make too much sense on a PC as far as I can see I hope the ADI guys will correct me um it makes more sense on SOCs it makes sense if you're like building something with an ad chip on it and obviously embedded compute is something that we want to be more in like I think that's that's where a lot of the future of GRU radio will happen it's less on laptops of academics that do simulations this will be more on you know devices actually doing network stuff or satellites um and as said that we we also have like a lot of interesting options that we need to upstream about you know better networking blocks so that's that's what we do about being inclusive for 3.9 as said to future proof um 3.9 we can drop python 2 compatibility like the there's no this or in the future that doesn't have um at least python 3.5 um that I care about at the point that we'll release 3.9 so um we can completely drop python 2 compatibility we can drastically increase the CMake version that we depend on that's that's very nice because CMake is kind of a nightmare to use the older it gets um and this is like the the biggest point on this slide is very clearly we have a problem with PMT PMT is our internal library for exchanging data objects in messages and stream tags and it's I know where it's coming from it it it does a fairly okay job at being what it is it is a polymorphic type to like I can put an integer there in there I can put a dictionary in there I can transport sample data in there but it's all very it is slow it is kind of awkward to use and mostly it's this buggy in the way it's wrapped into Python and C++ and um well and also it's our serializer um with the downside that it's not actually portable and there's no language support aside from Python and C++ which happen to be the languages that you could natively talk to ignore you with so um so we need to get rid of that enable to enabling us to actually come up with a better RPC basis because RPC won't work if you don't have a way to share data so then we have the simplify aspect um that's I already mentioned that we need to have binary releases um I don't know if they're gonna be nightly and we're not that fast maybe weekly would totally do um I guess somewhere in the middle um but we're gonna figure that one out um we're also gonna do I mentioned the uh template for out of three modules that template should have like the minimum infrastructure to make it easy for you to like create a Debian package of a rpm I actually don't care preferably both um so that you can you know distribute this to your potentially customers like I hope that people will use you know I do more in commercial deployments like they should less oh I compiled this on my machine and then I SCP that over um then um I talked about bettering ourselves um I don't think this is gonna be complete in 3.9 but you know we're gonna start with having RPC in every aspect of generator so there's no reason that we should have setters and getters and blocks that can't be called remotely um that's just like our heritage and a bit of our laziness so we need to start working on that and this is making me very happy um lot of you guys probably know walk as our library of simd operations and um it's basically meant to be a spin-off of the radio what it is in reality right now it's it's like its own github repo and that's being a sub module like github module actually in radio and that makes no sense at all because either you develop it out of 3 or you develop it in 3 but you shouldn't do both um but with us picking up speed we can basically ensure that vogue has regular releases so we can actually make vogue an external dependency and maintain it as its own project that it has become a long time ago so with that being said let's draw a quick conclusion will that not bleed let 3.7 bleed out we'll support it for a while I promise we will be releasing 3.8 very soon we could use your help we could use your coffee um but it is coming it's not like it's not abstract anymore we know what to do we know actually have a list of things that need to be done and if that list has become like all striked out we will release that um 3.9 after that is the next big thing uh in that it adopts a new strategy at upstreaming and it starts getting rid of architecture that that hopefully um will allow us to be well cooler for new users so um with all that being said I'm open to question thanks for like your attention before we start questions you mentioned Hawke on and yeah did you mention anyone else Mark is not here Andre and you go ahead you want to use it with full radio everywhere what else would you use your sdrs with latency requirements so so the question was um how to address more like closer latency requirements and that's something that we can't do with the current scheduler because our current scheduler simply is only optimized for throughput on an potentially infinite many core machine so there's no precise plans um last year I I talked with a gentleman that um and we will be addressing like scheduler constraints that will be allowing us to restrict um latency but we're I mean that's not happening in 3.9 likely so um realistically making the scheduler adaptive like pluggable will be necessary to like you know experiment with different scheduling strategies so this first latency later so so the question was what about macOS support yeah what about it I actually don't know what the so it doesn't compile out of the box on macOS um I actually don't know I have never used macOS x so um I have my guys for that um and my impression is that we're actually doing a pretty good job on that so um I can only like like this this hasn't come up the first time um at foster this year I can only ask you to actually write that email to the mailing list or open the buck because we can't fix things that we don't see maybe I mean I can't rule out that you're doing something wrong um I'm not willing to blame this on you on the other hand so please do write that email so we have a yeah I mean yeah you're welcome I mean we can set you up with whatever you need so give me so if there's something one wrong with log for cpp yes um so log for cpp is first of all like um I'll be honest I don't like it that's that's the first thing um it is that it is like a c++ part of log for j um that says a lot and also like um it has recently shifted the the c++ version it it requires um so we're right now running um like with a bit of you work around so um it's simply not reliable enough then also and that's that's the other thing we do need a better logging system I've I've been like at at congress I've been talking to the osmo com guys because they internally have a logging system for their network oriented stuff which is much more like what I want radio logging to become and putting this on top of log our current log for cpp structure that's like not really paying so we might as well replace that with whatever is easier uh so the question was um what what will happen in with respect to uh dynamic reconfiguration of flow graphs um that that question needs a bit of background um currently like if you say the afloa graph I want to disconnect this block from the other block and connect them in a different way what happens if you you you basically stop everything everyone grinds to a halt and then in some more or less specified state the buffers are reconnected and that's a nightmare um we can't be doing that in the future um sadly this is very very integral to like how the current scheduler works so the answer is kind of the same as for the latency restrictions um it probably requires rewriting of the scheduler I mean if this I'm I'm the architect I say probably because you know I have I do feel I have an understanding of how the the current scheduler works but there's just too many corner cases with how it's implemented like it there's literally too many go-to's in the source code that to to make to allow me to predict like corner case behavior like what happens if I reconfigure and someone else says okay I'm done and yeah so this this needs modeling this foremost needs a modeling of what how things should be behaving but modeling existing software is actually harder than like writing software to a specification that you have made up prior based on your experience with your current software so um yeah as said it basically requires you to like write a scheduler all right we need your yeah so thank you