 I was having a discussion with somebody the other day about what was about Adda. They were asking me some questions about what I do and what I like about Adda, stuff like that. I wound up getting on the discussion a little bit about open source licenses and sort of why we're not okay with certain open source models. I got me thinking, I really haven't talked about that at all, and I think I should. And I also know I really should give a bit more direction about what I'm doing with the Adda Tools project, and the more I thought about them, both of these, the more I realized that they're actually a lot connected. So I'm going to be talking about both. First off, open source, what I think about different licensing models. Now, some of you at least, follow me on the GitHub account that I've set up. You know that I'm definitely in favor of open source, generally speaking. I don't think it's right for everything. I don't think it's going to save the world. But I think it's pretty useful, but not for the reasons why a lot of people will say it's useful. For example, I disagree with the Free Software Foundation's attitude that it's important to have access to the code in everybody's hands. I just haven't really seen that, it's particularly beneficial to something called Linus's law that really was never shown to be a law. As far as I've been able to find, there isn't actually any studies done for it. But the general idea is that the more people you have looking at code, the better it's going to be. The more people can fix it. And that should make sense, right? So you get it all into some of the accountability problems and other things that open source is also experiencing. Yeah, it's a mess. I don't really want to get into trying to prove or disprove these kinds of things. I just want to mention that that's not my line of reasoning for why I like open source, but rather what I like about it is that because you have the source code and because compilers are increasingly getting really good at optimizing code, what we can do if the source code is available is to very specially compile code for each individual architecture. Now JIT sort of does this, especially if there's an AOT ahead of time option along with it. Microsoft's been doing this a lot with their .NET, especially .core for the UWP platform. Actually, that might be a bit redundant, kind of like pin number, but UWP does take advantage of AOT. So that is, you download a sort of pre-compiled form of it where some of the work is done, but it's not exactly executable code yet. It's just ready to be run on the bug in the mouth, on the virtual machine. And the AOT steps in, finishes compiling it, so you get much more native code performance. Which is a big reason why if you've compared UWP with, say, especially the old Silver Light stuff, it's pretty apparent which one performs better. That's not to say JIT isn't useful, but the startup times especially, uh, virtual machines suffer pretty badly when it comes to that. And that's sort of something that's driven me nuts about a lot of Linux systems. Not all of them, of course, though. Is that even though you have the source code available, you're often not building this stuff. And granted, some projects, LibreOffice especially, have absolutely horrible build times that you would not want to rebuild them every single update. So part of me gets why packaging is done, but there's still obvious performance benefits out of compiling it for your specific architecture. Now, it used to, at least when I started learning Linux systems, you used to have to know exactly what target architecture you were using so you could tell GCC to optimize for it. A few years back now, actually probably like five years back now, GCC got support for the MRCH flag native target, in which case it would actually figure out what target you were really on and optimize for that. So if you've gone through the Attitools repo at all, you notice that in the compiler interface, because I'm not actually writing a compiler, I'm still using NAT. It does reference both the MRCH native and M-Tune generic flags. The idea behind this being sort of a best of both worlds option, where you can package a pre-existing binary with the generic tuning, but you still, because of the way networks have the source available and can do a specific native optimized build, if that's what you actually want to do. That's really why I like open source. Being able to see what others are doing, that's not pleasant at all. Being able to see how others are doing things, it's pretty cool. You can get that even with closed source stuff, though, through patent applications. The real beauty of open source, in my opinion then, is actually what I described. That allows you to do specific builds instead of like some Linux systems where they still build for the 80486 or 80586 targets, which is just disgustingly old at this point. Come on guys, we're way past that. Way past that. We need to stop building for that kind of shit, what they do. I guess I will actually delve a bit into some of my issues with some open source licenses. I just actually explain why I use the licenses that I use. GPL, regardless of what some of its proponents say, is really anti-business. It's the GMGPL, which NAT uses, the NAT modified GPL, I think is a good compromise between the two, because I get what the GPL guys are trying to achieve. They want to make sure that the source code stays openly available, that even if somebody goes and forks the project and extends it, that their contributions are still available, even if they don't push those or make a pull request to the main repo. It's a fair point, I get it in theory. And as much as I disagree with the thinking in many regards, there are quite a few developers who like to keep a hold of their intellectual property. Now I say I disagree with that idea, because I'm pretty sure you can't really have intellectual property. You can't really steal somebody else's idea. You can just adopt their same idea that somebody else using something that I've come up with doesn't take away the fact that I still have that idea and can do things with it. But that doesn't change the fact that there are a lot of people who do think that way and who probably wouldn't go and develop things without feeling like they're not going to be taken advantage of. I think that really the better option for dealing with all the differences that people have, their thoughts and everything, is something like the BSD license. Now most of the source code I have up on GitHub is 2 Clause BSD, although I've used the others. Some of it's even public domain. This is where we really get into how people don't really steal your shit. Not really. About a week ago now. An extension I developed for VS Code for providing much better, I think, Ida support in VS Code beat out the existing extension for Ida. The code, it's a bunch of JSON formatting, is public domain up on GitHub. Nobody's made a new extension, it's been up there for like five months now. It'd be totally legal for somebody to take that, upload their own extension, claim they were a original author and everything else, it'd be totally legal and I wouldn't even be upset. I'd probably laugh at them and say they're fucking idiot because there's no way they're going to compete with that, but maybe I'd be wrong. Well funded business, maybe they'd be able to. But I know the language pretty well and that's one reason why the extension does a good job of matching things and understanding the same keyword can be different in two different locations and things like that, you know, understanding the language better. Somebody be totally able to copy that work, claim they were originally author and nobody does. Yeah, with that out of the way, I guess some of the direction I'm taking with Ida Tools explain a bit more about why it's doing the different things that it does. I have some issues with IdaCore and it's complicated because they are absolutely the best provider of a Ida compiler. They're pretty much the only commercial entity providing any type of support for the language. I say pretty much because there are others like PTC, but they're really only focused on the defense industry and avionics, aerospace I guess it really should say, because it's not just avionics and you know there are some individuals, myself included, as much as I butt head with the guy's loot guest definitely is doing, definitely deserves credit for his contributions to the ecosystem. There are others as well like the Matreska Library. I forget who the author of that is though. It really pales in comparison to what other even newer languages can offer. That's where my B flies. The things like say dependency managers, package managers, just sort of like package light managers or package manager light, I'm not sure what that would be, but they're dependency managers. Nougat is a great example of that. I really like Nougat, but there are others. What they do for those unaware is you declare your dependencies for your project and if they're not installed on the system, the dependency manager pulls them in. So then you're able to actually build the stuff because you know that's important or run the stuff in the case of it just being a scripting language where there isn't actually a build and they're just useful really. But it's not just a dependency manager that I think I'd really need. With a lot of other tooling that would be useful and just really isn't provided. For example, a unit lister. I've already got that working. That'd be similar to, I'm not actually sure what it's called, damn it. But I know when you open up Visual Studio or other major things, I have seen this with Java stuff as well. You can inside of your package, project, I mean, sorry, you can get a list not just of the files, but of all the actual classes and things like that inside of it. Now, in those types of languages, the class-based languages, a class can be spread across multiple files. This can, with a little trickery, actually be done with Ada. And there is a tool, NatChop, which takes advantage of this. Ada Tools doesn't work with NatChop yet. But I think it'd be really useful to have this. Now, Ada does have a almost one-to-one relationship with the units and the files. So you go through the file list. You can see that, well, if there's an ADS and an ADB file for something and then you see its name, there's probably just that unit and a story. But with the existence of separate functions, separate procedures and things like that, or even just what NatChop does, it'd be useful to have. But there are also more useful features. And in a similar vein, since classes are sort of like Ada packages, but sort of like Ada types, it'd be useful to have a type browser. That isn't entirely working yet. A lot of the simpler types are, but not the more complicated ones. That kind of thing. NatSorta provides with NatFind. But it really just does these sort of regex-based searches that you could pretty much accomplish with said or find anyways. Or sorry, not said, grep. Said something else. How this is different is that it's actually going through and finding where the types were declared, including multiple declarations of the same type. Now, that may sound a little awkward, like some kind of syntax error. Oh, the name's colliding or something. But if you want to have a private implementation of a type, you can actually do that in Ada. Or it's publicly available, but you can't really get any information out of it. You can just use it. That's a thing. So you have the exact same type declared in numerous locations within the same package. NatFind gives some weird output, at least the last time I used it, when you try to find the type using it. IdaTools does correctly handle this for the types it recognizes right now. Furthermore, this kind of thing is also useful for the generation of the export library with PE coughs, which, for those unaware, are what DLLs are under the hood. The actual library format is known as PE cough. It's sort of a pain in the ass to have to deal with those, because ELFs you don't have to do anything with. And those are the shared objects on basically every Unix system. I say basically because OSX is probably the most popular one that doesn't actually use ELFs. They use MacOs. And ELFs you don't really have to do anything for. You just build. Now the export libraries do have some benefits. I'm not really sure if they're justified, though. But being able to automatically recognize types and whether or not they're publicly accessible or not is definitely an important part of automatically generating those export libraries. This same kind of thing needs to be done with recognizing functions and procedures. Operators basically just being a special case of functions. Now, most of that I've described is really just browsing and some package management stuff. For the most part, that's really all I intend to add the tools to be. And a sort of project assistance. But that's not to say that's the only reason why I'm doing all this work. If you've looked at the repo at all, you can tell that IdaTools is really a library. There's a separate project within the solution that is a command line interface for it. But it doesn't do a whole lot of processing. It just calls the stuff from the library and then prints it out. Hopefully the IdaTools library will be useful to other people because the similar attempts by Atacor kind of run into some problems. They have their Lane Kit library and then the Live At a Lane library which is built upon the Lane Kit library. But at least as far as I was able to find can't seem to find a license on their GitHub pages. That's not okay. It's also not particularly well documented. Although maybe they're using a documentation generation tool that you have to download the repo and then run the generator inside. But if they are, those kinds of things like Doxigen and similar usually use these special document comments. I couldn't find a whole lot of them. Maybe they didn't feel certain things needed to be documented because they were largely private to the thing, but I don't like that. And I've showed off the parts of the code quite a few times now and you can really tell that things are documented way up, even to the point of me documenting most enumerations and even what the individual values in the enums are. And I take documentation pretty seriously because if you're providing a library the idea is you want other people to use it. And if it's not well documented it's hard to figure out how to use it. But I know with the seemingly non-existent license maybe I've missed it. If I missed it just please point out where it actually is but it's not obvious. But the seemingly non-existent license makes it hard to know if you even can. Because at least in most countries definitely including the U.S. where I had a course based out of if there's no license you're supposed to assume that it's all right reserved. Which case, even though it would be open source it's not free software. Now I doubt Idecor meant to do this. They do seem to be proponents of free software. The license almost everything under the GPL so maybe they just gave it some strange name because Lord knows Free Software Foundation loves to be obscure as shit. So maybe need a license for the project just in some weird location and that it is GPL licensed. I don't know. But just the fact that it's not even in like a standard license file or included as a comment on top of the sources just makes it hard to figure out if you even can use it. That's not okay in my opinion. The other thing and I think this was just more of a technical matter I will say absolutely Lib Adelang is the more powerful of the two options because Lib Adelang actually understands the semantics. By that I mean it's a full blown parser it generates a parse tree, everything else. Note that you don't necessarily need to develop a parse tree there are other ways of doing this but that's what it does. And that provides a much really much more in-depth analysis of the language whereas I had a tools, really just a scanner just literally a really detailed scanner and that's all it does just scans for certain patterns doesn't really care if they're valid or not as long as it pretty much matches that's good. This is because I'm not trying to use it as a compiler or any type of language analysis in any depth because Nat already does that. It's basically just project assistance so it doesn't need to have a detailed understanding of like what this type is or whether the type is even in the right place just needs to go oh hey this is a type I'll record it. This does mean that I had tools in theory is a bit lighter weight you'd really need to run some profilers on that to be sure but considering a huge part of the Lancet framework is Python I'm willing to bet once I get a real release I had tools would be more lightweight just because it's not trying to do as much note that that should not be confused with being the better product just that they have different goals just trying to get some information out of a unit I had a tools is fine trying to hook into something for syntax highlighting or for code metrics I had a tools isn't going to help you out at all so some of what I've been working on with the library recently since the last time I showed it off a lot of under the hood stuff there when I last showed it off and literally everything before that what the actual parsers would do is gather up literally every single thing about the unit regardless of whether they were needed or not obviously that's not a great solution I'm sure there's plenty of functional programmers out there who are going dude no and right one of the better ideas that functional programmers really stress is on lazy loading now you actually don't want to lazy load everything or lazy evaluate everything but this is definitely one of those situations where parsing slow you don't want to do that unless you absolutely need to so the only thing you're doing is generating a table of the units within the project why parse the configuration file why parse all of the types within all the units and eventually parse all of the functions and procedures within a unit that's a lot of extra work that isn't going to be printed out you don't need it what I got going the lazy loading of those so this isn't actually done with the lazy class and the reason for that is if I were to turn a lazy of the different types that the units can return like a or the project can return like a units collection or types collection or whatever the downstream dev would actually need to call the dot value property of it and cause it to actually load it's sort of awkward so what I do instead is basically recreate that lazy type it's not actually that much work a generic lazy type is but doing the lazy loading isn't actually that much work what happens when you call the property the very first time is it actually parses that and then once it has the full thing that it's supposed to return saves it and then returns it next time you call that property it just straight up returns it because it already knows it has the stuff this way you return and the actual type and not wrapped up in a lazy so it's a little bit cleaner to use but it's still lazy loaded and like I said it really wasn't that much work the property getter should never need to be modified ever there are a few other under the hood changes a few lists got changed to hash sets where I could some things seem to need to stay lists we'll see if I can change them to hash sets or something similar reason why that's an actual improvement is that if you're doing something that's really look up heavy it's quicker to look up in a hash set hash at basically just being a hash table whereas with a list I'm not sure how the net core lists are actually implemented I don't think they're a doubly linked list they're definitely presented as if they are I think they're a tree don't hold me to that though I'd be really curious if anybody has some detail information on that probably look it up in the net core runtime actually but those the types and unit collection which are implemented with lists right now probably should be changed to use a hash set as well I just wasn't able to get it working quickly but I'll think around that because that again those collections are considerably more look up heavy than anything else you don't remove anything from them so it's just add and look up definitely the better thing to use there the other big thing I've been doing is changing the source class what that originally was is just a really long string when you open the file it was read into that literally line by line into one really large string and then all the parse operations on top of it it worked it's obviously not good has no understanding of the different parts of the source and you wind up with the list problem all over again because it's one really long string you have to read from beginning to end every single time it's not good debating between two possible options the first I'm doing is to break it up into separate strings where the different parts of the source code are placed inside this isn't a huge improvement but it does allow scanning certain things to be quicker for example if you're only interested in what the dependencies of a project are it's much quicker to if you have the little bit of include space marked separately so that you know that an include declaration whether it's a with or a use should only be in that you only need to scan that now you don't need to scan the entire source file it's speed up it doesn't help as much when you're doing things that would be inside the unit itself but it's still a speed up another thing I'm considering though is to keep it in one really long string or maybe actually an array of strings see because the way it was before was an array of strings delimited by new lines it wouldn't be that hard to change that to sorry a single string with each line delimited by new line it would not be hard to change that to an array of strings where instead of delimiting each line is one of the strings inside the array then you could just have a reference to the start and end of each section that might be the way I go not certain yet that might be the way I go then you still get the same benefit of being able to scan only the right area but it'd be a little less friendly for people to contribute to that way and I'm not sure if there's any performance differences between the two options so I'd have to profile it either way though both approaches accomplish the same idea of breaking the source into multiple parts where you can scan the smaller area of where it should actually be this would also be important for saying whether or not types are publicly or privately accessible or whether they're say publicly accessible or visible I mean publicly or privately visible or also the case where it's publicly accessible but you can't actually view the type so that is it's the full declarations in the private part but it's still publicly accessible but it doesn't yet aside from that there is a library type now which collects all the units similar to the project class but the library instead is a static class that collects for the library all the installed units get registered with that and the reason for this is because somebody really could go on and develop say like a build server or whatever that can handle multiple projects and whatever but you don't want each project to have to load up the installed units all over again that's just a huge pain in the ass it's not a good way to do things so it's better if they're all collected together and just registered with it any project that registers a library you get a new project that comes down the line later it needs that same dependency it's already registered you already have all the information about it not so much an optimization for the command line project because that's a one-shot works on one project that's the end of it but it is an optimization for more sophisticated things and it needed to be implemented anyways something needed to register the already installed units so I was just thinking down the line instead of registering them with the project or whatever makes sense to register them in one common place that is mostly working already there are a few parse errors that are stopping me from registering the entire at a standard library at least libnat I don't actually have access to any of the other ones to test that with I'd like to I just don't have access I will get that working because it's obviously important but that's definitely a good really good test for Ida tools being able to parse the entire at a standard library because the free software foundations and IdaCore's style is a bit different for mine so if it can properly handle somebody else's style that's a pretty good indicator that the parsers are working correctly so unfortunately with the next video there probably won't be much to show off because these are a lot of under the hood improvements there really isn't a whole lot to show off with them they matter a lot for performance reasons or usability reasons things like that but they're definitely library level under the hood improvements not so much features I nearly forgot one thing to mention about dependency managers and how they're really catching on and why I think IdaCore really really jumped the train on that one even languages like C and C++ are getting package manager or dependency managers I try not to call them package managers because the system at me and me is just knows them as something else but I think it's Conan something like that for C++ but there's also something from Microsoft I'll get the link for it but Microsoft wrote a dependency manager for C++ as well honestly I don't know why they didn't re-use Nougat it's probably some technical issue but considering Nougat has been modified with Choclity to actually do install packages I figure they probably could have just re-used Nougat but there is actually two dependency managers for C++ probably even more just those two that I'm aware of I'm not even really a C++ dev so there's clearly an interest in these things and the closest thing I had of course got going on to that is the GPR tool set now not install or GPR install sorry only does from local projects that you've already built and installs them in a weird way I don't really agree with it and I think the pretty good proof the installer is broken is that when you need to update XMLIDA you usually have to uninstall the entire thing everything that XMLIDA depends on reinstall XMLIDA and reinstall everything else it's not so much of an issue for you guys who use the IDACore toolchain but if you use the free software foundations version because of licensing nonsense with IDACore you probably know what I'm talking about it's not okay in my opinion it's a broken installer in my opinion but it doesn't really do anything to help you out when those dependencies aren't already on the system it just complains that hey you know I couldn't find this project it doesn't provide any kind of assistance for downloading them it doesn't do anything like .NET Restore where it pulls in all the dependencies and says okay you're good to go now it just yells at you it's very 90s come on guys and I really don't think that's okay especially for I think a lot of you guys need to to remember that we're all newbies once and I think it's because of the way I wound up learning I didn't have a really nice formal education with everything handed to me learning at it was fucking terrible and it's a good language I'm glad I stuck with it but it's terrible the community around IDACore is basically non-existent the tooling is terrible and everything else and it was without any doubt the hardest language to learn that's not okay I care about the language I hope that's obvious I know it's at least obvious to a few people and uh I'm trying to actually do something about it by looking back at how bad my own experience was and provide those things that really should have been there when you're not just learning the language to learn how to install dependencies and where to even find them and all that crap that considerably worsens the learning curve that's how you get people giving up going to learn other things instead not coming back to it that's not okay so yeah trying to that's a huge part of the the Attitools project trying to provide all those things to make it as easy as possible to develop in IDA so things like the automatic project configuration where I can figure out what's a unit, what's a program build them appropriately figure out the right build order stuff like that I mean you can kinda do that with a glob for all the Addis sources in a project but the difference is that I'm actually working on adding stuff to pull in the dependencies kinda like with .NET Core 2.0 and 2.1 where it automatically calls .NET Restore if it needs to doing that kinda thing my intention is to look through the system first see if those dependencies are there and then link them over this is because you can't you kinda can't reuse the .NET search path there's this weird issue and I've talked about it with other videos but it it the whole thing sort of breaks down if you already have the thing you're building installed which is soft in the case when you're developing so we run back again to those issues with just developer unfriendliness that's not ok yeah so sort of hijacking the build system there hide the search path so .NET doesn't see the system and then knowing the search path otherwise link in the appropriate things but then if they're not in the system actually try to fetch them I'm not entirely sure how I want to do that yet I've been thinking torrents would actually be a really neat idea you can use a tracker to have an actual registered list so that you have you know actual package repo page you can register site that you can register with and browse the packages and everything else but then when it comes down to the actual download you use torrents because that automatically provides the check summing and validation automatically provides the distribution method even if nobody else seeds I still have wherever I have the packages up seeding the torrents well nobody else contributes I still have a viable server just from that alone that kind of thing pretty useful in business settings though where you can have you need to update say 500 machines it's much quicker if they can start seeding between each other that's one of the reasons why torrents were developed there not just popular among pirates hopefully I remember to edit it in here but starting with Windows 10 I think I didn't see it before Microsoft actually has a setting you can turn on to help seed updates to other people I'm not sure if they're using torrents exactly or some custom protocol or something but they're definitely utilizing that idea recognizing that it's a good way to help not just lighten the load on them but help people actually get the stuff faster but I'm probably going to do torrents just because they provide so much pretty much out of the box and there's wouldn't be that hard to get it working another issue I have with iDecore I was browsing their blog the other day and I think it was their blog anyways and I think this is pretty much the perfect example of how sort of unaware their management or something is they actually spent time and resources and everything else developing a font with special ligatures for edit programming and at first some of you are like hey hey hey but there are things like furacode and there are some others that do that they're actually really nice I know because I use furacode I'm not talking quite like that though I'm not talking about taking the operators with two or three symbols right next to each other and joining them into one single symbol that look better I didn't do that their ligatures taking things like begin and end and replacing them with curly brackets I wish I was kidding because at that point you might as well just use rust rust has cargo and other things that make it more pleasant to develop in but seriously that was something somebody honestly thought was a good idea to spend money on to spend time on to help people move to fucking serious people need something that actually make development easier not make it look like a different language entirely but come on guys