 As you know, I've been cranking out videos in the Aida vs C-Sharp series and this is not one of them. I've largely been using that video series as a good opportunity to take little breaks in something that I'm working on because there's a lot of thinking that goes into it. A lot of times where I get stumped, I need to think things through and it's good to do something else for a while. Clear the brain, let it start all over and it's good to take those breaks. Incidentally, that is one major reason why I don't think the office style 9-5 kind of workflow is good for programming. It's primarily a thinking kind of job, not a busy work kind of job. And it's not a storefront, you definitely don't need to stay open during very specific hours. So it's sort of better to do it as your mind's actually being cooperative, do something else when it's not. So what I've been working on. I've said quite a few times that stringier is largely meant for my own purposes. I recognize that it's stuff other people can definitely make use of, but everything I've written in there has been things that I've needed and it should make sense. I focus on those things. Other people are totally free to contribute, of course, but what's in there is in there because I've needed it, not just because it's something that I've thought of. Sometimes the reasons for those are little funky. Chopping a string into its constituent parts, all of a specific size is definitely one of the weirder ones, but there's valid reasons for that. Largely has to do with sending massive chunks of text across network streams where the packets all have to be of a specific size. It should start to make sense why you'd want to chop a string into pieces, but that applies to everything in there. Everything in there has some kind of purpose. One of the big reasons for huge parts of it. In fact, aside from some of the libraries, like the patterns and streams engines being used for implementing something, the entirety of string here is going to be used as a runtime for something. Now, what has runtimes? Programming languages. In this instance, we're talking about a DSL, a domain-specific language. I've mentioned this in at least one video. I don't think I've gone into particular detail, but langly. The intention of the langly is to be a DSL to enable rapid creation of other language analyzers so that I can then use it for development of a more sophisticated, full general-purpose programming language. Langly has a few things interesting about it. A few things relevant. It should. You don't want to create just some yet another thing that is already out there without providing anything useful on top of it. You can largely think of langly as being similar in nature to Weissen, Yak, Antler, those kinds of things. There's a language in which you define the grammatical constructs of another language. Compile a thing, and you get back out a library containing the analyzer for that language. You can see why this would make development of new languages much more rapid. But what does a langly have that Yak, Weissen, Antler, the others don't have? That's one of the big ones. With those existing tools, typically what happens is you compile a thing and you get out a C library. You can bind to that library to get the interface and the language that you want. But thanks to the .NET and the CLS compliance that it has, you could instead build a .NET assembly that just works with any .NET language that is also CLS compliant. That sounds a lot easier to integrate than it is. Furthermore, with building up the tooling surrounding it, you can get a fully integrated experience with Visual Studio or other IDEs. Fully integrated into the MS build system and treat it like just any other project. With that in mind, writing a compiler for a language would be as simple as creating a solution in Visual Studio with two projects in it. One, the analyzer written in langly. And two, the synthesizer written in whatever other language. That sounds like a pleasant experience. Anybody who's done these kinds of things before, written a compiler or interpreter or whatever, knows how ridiculous that kind of experience can be. How much jumping through hoops there is to get everything to play nicely together. But you just wind up getting that. And that sounds a hell of a lot better. There are a number of details I repeated to work out before I can really finish or sometimes even start the project. There's been a lot of prototyping around Langly. A lot of verifying that the different things that I want to do can actually work, but the actual project itself, the intended design, has not started. One such barrier is not really a technical issue, but more of a, hey, it just makes more sense to wait. C-sharp 9 is adding in a lot of new features. Many of those features are actually things that Ida did really well and C-sharp is starting to get. And this happened to some extent with C-sharp 8.0 features that were definitely, I don't want to say out of inspired because I don't believe that was the source of where those features came from, but things that Ida definitely had and worked well that C-sharp started to add. C-sharp 9 adds in particular record types. Yes, they behave immensely like Ida records. By this, specifically the tagged records. I do need to be clear about that. The non-tagged Ida records without discriminants behave exactly like the .NET structs, so there's no need to replicate that. It already exists just with a different name. So the .NET record is specifically a tagged Ida record. But it behaves exactly like it. You can inherit from it, gets passed around by reference by default. In fact, in almost every single way, it behaves like a class. Initialization of it is done, however, much more like the Ida record initializers. You're not necessarily calling a constructor, there's a very, very similar syntax you can use. Very similar to how records are done in F-sharp, actually. In fact, that's basically where this feature comes from. It just so happens that the way F-sharp implemented records, the way I implemented records, were very similar. But where it gets different from a class, despite being reference semantics in so many regards, is that equality and comparisons are done using value semantics, just like they are in Ida. Deciding whether two types are equal or not is done based on equality of each of the individual components, and each of the fields auto-implemented properties would still be a field, as long as those are equal, then you have equality. Now, they generalize that a little bit to where there's an equality contract that has to hold, and that is what actually decides whether they are equal or not. You can override the equality contract, but by default, it ensures that all of the fields are equal to each other. Now, why would I want to wait for that? Because that is actually the exact type of semantics that you want in a language front-end. In the analyzer, think about it. You have two variable declarations. They both share the same name. Even if they're in different positions in the source file, even if they're in two entirely different sources entirely, as long as the fully qualified name is the same, they are the same. As far as most languages are concerned, anyways. It depends on the language, of course, JavaScript. They would not be the same. JavaScript, you would be referentially equal. If they're not the same reference, they're not the same thing, and it resolves it based on the most recent definition, but most languages don't do that. Most languages, if it's already defined and you try to define it again, you end up with an issue. You can't define it again. It's already been defined. You could assign it a new value, depending on whether it's mutable or not, or whether it's read-only or not, but it's already been defined. You can't define it again. That is most easily achieved through value semantics for equality. That being said, you would want it to be referential otherwise, especially when putting it in a tree. You don't want a tree entirely made up of values that gets immensely difficult to actually program correctly. You want them to be references. You want them to be pointers. You want it to point to the next object in the tree. You want it to point to the first object it contains, but the equality, that you want as value-based. Now there is a system to do this before records were implemented. It worked out a pattern to achieve this type of semantics. I've utilized it in many situations. It works. It's robust. There are some situations in which it's problematic. I've put more effort into resolving those and it turns out there's two different patterns you can do, depending on specifically exactly how you want equality to work, and then you choose which one of those and implement exactly that. But instead of dealing with all that craft, using records just deals with it for you, just implements it for you. That's a lot better. The way it's implemented turns out to be different than the pattern I had worked out, but the goal, what it accomplishes is essentially the same. Because that is the default for records, it's far quicker to write. You can just go and create all of these as records and get the semantics that you wanted, rather than creating all of them as classes, overriding equality as well as the specific equality interface I equatable for that type. You can just get what you're meant to. That saves a lot of typing. That saves a lot of development time. And that eliminates the potential for bugs involving this, assuming that you didn't write it in exactly as you were supposed to. That's better all around. So I could start Langley a little early, implement it using classes and the pattern that I devised, take longer, but get it done sooner overall. But it would take longer. Or I could wait until C sharp 9 comes out and utilize those records. Take longer for Langley to get out. But it would take less time overall to develop. Now, we're all only here for a fixed amount of time. I'd rather optimize the total amount I can do over the course of my life than get this out as quickly as absolutely possible. Especially since there's nothing in particular that I am trying to do that needs to be rushed. So, the delay. C sharp 9 is due to come out together with .NET 5. .NET 5 is due to release around November, with release candidates coming out very soon. Once release candidates are out, the syntax is stabilized enough that I can safely use preview components and not have to worry about rewriting my entire code just because something changed, because a beta idea needed to be tweaked because of issues that they found out during the beta testing. So, you would wait just a little bit until the release candidates started coming out. But that's not the only thing that I was waiting on. See, if it was only that, then sure, I could implement what I could with classes in the pattern I'd developed and then just rewrite them as records, remove the equality overrides, and otherwise the same just works. But those are the things that I was waiting on anyways, and were those. One of them, I intend, Langley, to be a goal-directed language. Goal-directed execution is not something you see particularly often, but it did originate, seems to have originated anyways, in languages specialized for text processing. Now, I feel like it's useful in situations outside of that, but it's definitely well-suited for text processing. Ideally, you would want a high degree of compatibility between these languages as possible. So, if Langley is goal-directed, and the other languages aren't, how do you communicate parts of the goal-directed execution to these other languages? That's a problem. See, there are different ways to achieve goal-directed execution, but you can essentially view it as every procedure having a single return value, the error state, and every function having two return values, the actual declared return value, and the error state. That's essentially how goal-directed execution works. There's an error state that is implicitly passed around continuously with everything that gets executed. There are certain things, like Boolean logic, rather interesting. One is less than x does not return true or false. It returns successful or failure, and then x, the value of x, which then gets passed into the next part of the Boolean expression. If you have x is less than or 10, that gets evaluated, returns, success or failure. Now, if it already got passed a failure, then it just passes the failure on immediately and doesn't even try to evaluate anything. That allows you to write syntax much closer to what you would naturally, because essentially goal-directed execution is much more like what we do naturally. And return values are very obviously different in an incompatible kind of way. So, that brings us back to how do we get non-goal-directed languages to communicate with the goal-directed languages? Now, remember what I said. I want stringier to be able to act as the run time for Langley. That means that not only do I want to parse Langley using stringier.patterns, but I want the code written in Langley to use stringier.patterns to execute. I want it to use all of stringier to execute. Stringier is in many instances not goal-directed, and there are instances where it is sort of goal-directed, but the implementation of it is different for each library. Stringier.patterns has a result object that contains what looks like the value, the error, and whether or not it was successful overall. Result implicitly converts to these, but how it holds them is unique. Stringier.streams has a result object as well, which does a similar thing. It implicitly converts to the value, the error code, and whether it was successful or not, the Boolean state. But how it holds things is radically different from how Stringier.patterns' result holds things. You could, in theory, code all these to deal with it in very particular special ways, but that requires a lot of special case code. Coding for edge cases is obviously not the ideal thing to do. It'd be better if there was some kind of goal-directed framework that these things could start to use. Even if it's only tracking the error, that's something that brings us much closer to the intended behavior. That gives us a system which non-goal-directed languages could communicate through, and goal-directed languages could just use implicitly under the hood, adding to the framework, but still giving us that component. So then a few days ago, I'd found out about a particular attribute that's very useful in implementing this. See, you could rephrase what I said a little bit. Each of these functions, instead of returning two values in each procedure instead of returning one value, could return exactly what they would under normal execution models, but also have a static global that is set with the error. This gives us a shared communicated value, operating very similarly to if there was a register in the processor that was set with the error, which is actually a major way in which goal-directed execution is done. A specific register is reserved for holding that error, that works, but it only works as long as multi-threading is never done under any circumstances. Good luck ensuring that's never going to happen. All your code could be written synchronously, but what's stopping somebody from spawning a thread that executes your code? Now, even though your code is running synchronously, it's running in another thread and you have thread contention. You could lock, but there's a problem still. See, each thread can be viewed as an individual process, each one operating entirely synchronously inside of itself. Another thread comes along, sets an error. Your thread goes to read that because the next goal-direction function is being called. Your thread, which did not set the error, goes to read the error, sees that there is an error set and proceeds to go through the entire execution pipeline as if there was an error in it, but that's not what happened. You need this error field, this global, to be unique to each thread, and that's where this attribute comes in handy. The thread static attribute takes a static field and instead of having it as a global, has it thread local. So it is. As far as each thread is concerned, a global field, but it's not actually global. It cannot be used to communicate across thread boundaries, but as far as each thread is concerned, it will track static state. That is exactly the behavior we want. I just need to create a system which does mapping between error codes and exceptions and allows these error codes to be rapidly set, just like I had been doing through stringer.patterns and stringer.streams. Every function would be quite literally CLS-compliant because it's not doing anything that other CLS-compliant languages can't understand. A function that reads from a stream will return the value that was read. Simple as that. They just so happen to set a thread local field as to whether there was an error or not. Mapping these kinds of functions, binding them, rather, into a purely CLS-compliant kind of way that functional or procedural or object-oriented languages could understand. Languages that are not goal-directed could understand. Could quite literally be as simple as instead of if there's an error, we're going to pass that, don't execute the function at all, creating the binding that if there is an error, throw it because that's the execution model that they expect. This would then allow full compatibility. Similarly, these goal-directed functions could be written in non-goal-directed languages just as I am already trying to do. But this provides the framework that allows full compatibility between them. Once, if languages I develop with it are actually implemented, then I can have them goal-directed using this framework as the runtime for how the goal-direction is actually implemented. It can simply be done by using that thread-static field. They'd have their own special syntax for working with that field, but it'd still be there. Every CLS-compliant language could communicate with any goal-directed language using this framework. That's fucking awesome. One of the issues I was running into, one of the things making it tricky how to develop this, is you want that level of communication. You don't want just an error table where you have this massive enumeration of each one of those maps to an integer that's the new. That's how the WinAPI and COM do it. And it's all right. As long as you control the entire ecosystem, you can do it, but you can't control the entire ecosystem, especially considering what we're talking about, that it has to communicate with other CLS-compliant languages. We need a way to take an exception type, not instance, exception type, and get an error code from that. So how do we do that? Part of .NET's design is actually hugely useful for this. The .NET framework and .NET Core by extension was largely created with the intention of having COM interrupt being a core feature that you did not have to mess around with to accomplish. There are special attributes for accomplishing this. And when achieved, you can call .NET COM-compliant code from any COM-compliant native library and vice versa. So if COM is doing the WinAPI thing and having its exception error thing going on as integer codes with each code type, each code rather being a specific error type that .NET should have a way of communicating that, right? Hopefully it's publicly accessible, but it would be there. As it turns out, there is. Every exception instance has a property hResult. hResult being the struct macro. Regardless, it's at its core just a single integer code for the exception type. Every .NET exception gets a unique code for its type. All we need to do is when every single exception is registered, an association between that code and that exception type or that exception instance that was passed gets stored. From there, if we want to throw when any exception code is set, we can use that mapping table to give back the exact exception to throw. We have the efficiency of goal directed error setting and still am able to turn it into a full exception like any CLS-compliant language would expect. We can do the whole thing that was done with stringer.patterns result type, where the error code is just repeatedly set and unset throughout the entirety of goal directed execution, but still get that exception if necessary. I figured it out because that sounds like an absolutely fantastic solution to this. And as it turns out, the implementation, it's not even that hard. You could use one of the standard generic types. I'd rather not. I'm intending to do it through a skip list. Initially developed thing is just a singly linked list, but it will become a skip list over time because that's a huge optimization. This ensures that the entire thing is sorted. That sorting is significant because that helps you look them up faster and we want to look them up fast. So it would then be recommended that the exceptions that would be used potentially get registered at the time a module is initialized. Hopefully the C-Sharp9 modular module initializer's proposal actually goes through. But if it doesn't, well, I can still accomplish that with FODI. It's not the end of the world by any means. So there's also some clever tricks you can do to get around that. That FODI module initializer's weaver would accomplish what I need. So I've got it. Langley will happen. It will be immensely more compatible with existing languages than I'd originally thought, and that's absolutely fantastic. And we'll be able to have stringier in its entirety written in a non-gold-directed language but fully consumable from a gold-directed language as if everything was gold-directed or require a very minor change for those consuming from C-Sharp Visual Basic or F-Sharp. You'll have to call the error.throw method at times. Or you could do your own area handling in a little bit more global-directed way, which would be, why is it anyways? Because holy hell, is it a lot more efficient? That's where I'm at with this. Have a good one, guys.