 Hi guys, we're going to talk about where I'm at a stringier in general and Resurrecting one of my site projects how it's different and why This will be a general overview of where I'm at a stringier not any specific component so as part of the v4 rollout obviously auditing every single individual project Sub-project library, whatever you want to call it the types such as rune and glyph are done I'm glad to say that life is done. The entire thing is actually open source now There used to be a separation between Open source and close source internals The entire thing is open source now I Had done that originally because I had some concerns about some very similar stuff Appearing in some other libraries without credit so Instead of call out and maybe be wrong and then to have to deal with that whole social full-pot and everything Just close source part of it for a while if they are copying force them to come up with their own implementation if they're not Well, they just wind up with their own implementation anyways either way it accomplishes the You know it addresses the concern my code is not in other place those places on You know without proper citations No conflict which is nice Glyph has been Reworked a bit quite a bit as far as internals go as part of cleaning things up and optimizing various things It uses a try now for its lookup. I know I'd a use the Diction area an associative array specifically hash table backed one before but it's now using a try for the equivalency code lookups which oh There's some bizarre stuff going on there But overall it winds up being faster for the general purpose API and With the stamp typical use case so if both of those are faster, that's a good thing Now Those are done we've got in Coatings got Did I have that as its own thing before? Oh? I don't remember. I think that was in core before but it's been extracted out It's own little library which helps with discoverability Allows me to put specific encodings that would like to see there It's convenient in that regards In a similar vein. There's another unicode related to thing a lot technically encodings isn't strictly a unicode thing But obviously you're implementing mostly unicode encodings But a closely related thing called categories. I have a write-up on DevToe about that I'm very happy very Very happy with how that system turned out, but it's a structured approach to rich categorization and It is fantastic To go along with that Literary had its linguistics component yanked out of it into its own library called Linguistics the Orthography class that was you know originally part of literary has actually become derived from categories now So each category can be each Orthography can be used as a category Further richening the categories API, which is also fantastic There's almost no cost in doing that because it was largely setting that up anyways It's just now we're derived from the right type to actually utilize that So if you say wanted to do something like I don't know stripping out all Russian letters from a English text You could do that. I don't know why you would ever want to do that, but the convenience thing is That it's written the exact same way no matter what it would be that you're stripping Because these are declarative Objects so I can program them in a highly general way Downstream programmer doesn't have to deal with the specific things It just works Program any category stuff and it just works regardless of the category no matter how specialized it is even orthographies Oh categories also supports an expression syntax thing going on similar to the how the patterns engine Expression syntax works So you can build up at runtime or at construction time depending on exactly how it can resolve things These expression trees that represent Well set theory for the categories because categories are essentially a set So unions differences junctions stuff like that Art supported and that would obviously then extend to the orthographies considering orthographies are categories So you could get Really specific But it would be it would be incredibly easy to actually do something like search through a piece of text and see if it has Any Latin characters that are not English That would be an incredibly simple line of code to write But that's a remarkably complex operation. So that's that's you see why I'm very excited and very happy with how the categories API turned out To go along with that Core has been getting a lot of work a lot of lot of work You can see some of the issues up on github right now. They're rather Major things such as documenting in the function signatures whether a given Function allocates or not depending on what its return type is String for allocate allocating functions and read only span of car for non allocating functions So that requires a lot of going through and auditing everything, but there's other things going on too for example adapting certain APIs to use the Orthographies when appropriate instead of just exclusively culture info and there are actually some instances where it has to use Orthographies because it can't use culture info at all. So if it can accept both it has to use both it can only accept orthographies It has to use orthographies and Yeah there's a Lot core is taking a lot of work Since I mentioned yanking out the linguistics part of literary and putting it into its own library What's going on if the remaining functions that were in literary? Well, they're actually being merged into core That work is already done actually so any literary specific functions are still going to exist in string your core There really isn't that many I had a kept it separate because I was concerned about code bloat But there's just so little that it's fine to go there It's not really going to contribute much to bloat even with everything that's in string your core. It's a small library so Again, I'm not concerned about the bloat at all Obviously given its status as the core of string here That means any work on the patterns or streams API is Cannot take place until core is finished so Yeah That's that's fine though. I want their chain important changes. I want to make to core That effect patterns and streams So that's that's totally fine One thing that did exist in core It's definitely possible not too many people recognized it and I'm thinking about actually extracting this into its own API because there's increasing situations where That's what we're getting into Having the right data structure helps you programming a lot To some extent this includes The stringier internals such as that try that is used for Glyph. That's a specialized try It's not used. It's not like you'd find it in a Collections or containers library because the way it works is so specific to that exact problem Which does conveniently mean it was relatively good to write but but There's plenty of much more General collections that have been using extensively Some of these are actually meant for downstream programmers examples of that are Sort of formatter analogs to string builder I'd originally called it a format table and format list, but I'm gonna be In the before release. I'm gonna be calling it list builder and table builder And there there may be additional formatters to go along with that, but They are collections. They are specialized collections Just with the unique behavior of a two string that does different stuff But otherwise they are collections there's Actually additional situations where I have collections that I would like to Other have an easier time maintaining through better sharing between the internals Things that I want to implement but know that if there isn't that sharing then it would be just too much Stuff like that and this should be sounding very familiar to a side project. I had Collectathon Which was a more of an experiment than anything else. I didn't really intend for that to ever be a library Which is why I never went up on Nougat at all Just got written and then archived and unlike some of the other ones that I had done assigned projects it never got deleted from github because It was a successful experiment, but it was only an experiment Now I don't think it should be an experiment anymore But I do think it should be tweaked a little bit And I've been spending that most of my time today actually doing that So the original experiment behind collectathon was to implement a Container's API or collections API using as much Sharing of implementation details as absolutely possible by combining Object orientation inheritance and polymorphism Together with CRTP or the curiously recurring template pattern Which is a generics thing which I've got an article up on devto if you want to read about that I'd I'd go for that instead of me trying to explain it Real quick because it is a complicated topic but CRTP is useful for Sort of de-generalizing what you get with polymorphism They go hand-in-hand, but it is huge useful once you finally wrap your head around it As well as Implementing a collections API that Actually had a distinction between abstract data type and data structures, which is important However, it was that the academic notion of the separated abstract data types that Actually got me into problems. So collectathon continued to six extremely successfully Building up from this base class that all collections were derived from no matter how they were implemented to then increasingly and increasingly and Increasingly specialized collections which implemented more and more shared stuff. You should be noticing a problem though Collections are not taxonomies But we are implementing a taxonomy So that's one of the complaints I have about object orientation specifically the inheritance side of it now object orientation is fantastic for so many reasons That's actually a huge reason why my Patterns engine is as efficient as it is as Easy to maintain it as is and is as slim as it is It's like a third of the size of Microsoft's regex engine Because of object orientation, so I'm not bashing object orientation, but again every tool has its It's what it's good for and what it's been for object orientation assumes hierarchies assumes taxonomies But you don't always have taxonomies. What we do have Is a system of similar things with common features so We could do this through composition I mean it'll work potentially What's the composable units though and you sit there and think about it and? Actually, there aren't really composable units, but there are traits trait wouldn't be Lawless you basically but implemented as an interface that Has the bare minimum to support that Every type that implements that interface that has that trait Still has to implement that functionality, but because that is the bare minimum functionality we are talking about functionality that Can't be generalized and shared between the implementations or if they can they are actually so similar as to be taxonomic in that specific area for example List and a singly linked list and a dolly linked list obviously share a lot of implementation details those different variations of lists Could derive from the same taxonomy, but could a Dynamically resizing an array and a doubly linked list share many actual implementation details not really But what you can do is once you have those traits defined Start to write up generic extension methods which conveniently Narix can have a type parameter be required to Derive or implement multiple interfaces multiple traits this works consider something like index of all you actually need is For the collection to be innumerable and to be indexable That's it and even then only read only indexable doesn't need to be The the indexer doesn't need to support mutations. You can implement that Every single collection that supports those two traits and there are plenty of other examples Similarly, there are some convenience Functions that can be provided through similar vein Push and pop for example the stack operations, but that also apply to lists and plenty of other things Similarly the NQ and DQ, but we do this example of push and pop as a matter of convenience It is nice to be able to specify push and then multiple things to push in sequence well All the trait for pushable needs to actually be is the single push Implementation you can then implement a universal Push for multiple by just simply iterating over the items you're passing into it pushing each time That works Do the same thing with pop Only in this instance You're specifying an amount to pop off and then returning in innumerable of the elements But now that works for anything that has the pop trait. I don't know how far I can take that but It's worth playing around with and so far with what I've got done today. I'm actually quite happy with it Now now how does this play into stringer? well remember, we're talking about a Containers implementation whose goal is to provide as much sharing of code as Absolutely possible Then making it extremely easy to implement new collections and containers or whatever use that as the foundation for implementing the various Specialized collections that I need in stringer Hmm that then makes it easier to also provide Specialized collections useful for downstream such as the formatter collections. What you see now Luckily, this isn't too bad if I can get anywhere near close to the amount of sharing that I had before and I suspect I can actually get even better sharing now then Writing up entirely new containers was actually doable in 20 30 lines of code I'm not kidding I'm not even close to kidding and we are talking near complete parody to the standard dot net collections With like 20 to 30 lines of code because of the amount of sharing that I was able to implement before So I Think this will be worth it. I do As far as optimizations go because obviously you want things to be Improvable we're talking about a high level of generosity, which I've also got an article about that on devto And why you should avoid that so I'm taking a similar approach to how link addresses this by the Most general methods being extension methods, which are incredibly general they work for everything imaginable, but You can't make assumptions and if you can't make assumptions you can't optimize But we don't have to worry about increasing levels of specific extension methods Instance methods are always preferred over extension methods Want to provide an optimized implementation? Just stick it right in the collection. That's providing it boom done Simple as that. That's where I'm at with this whole thing Is it gonna take a while to get to work over collectathon? Yeah, a few days my days give me a nice break for doing something very different than Auditing because I have been auditing for several weeks now. I know it's for the best, but oh my god, is it not very enjoyable? so Give me some much needed brain candy, but it is Directly and obviously useful throughout string here as well so Be good for that Have a good one guys