 You know what software packaging sucks. I mean it really sucks and My question I've been thinking about it a lot this week Why does software packaging sucks so bad and that's what I want to talk about a little bit along with progress support so What I've been up to is The simple task of figuring out, you know Ulan 5 the latest version of the programming language that underlies the my particular approach the movable feast machine approach to robust first and indefinite scalability hasn't had gotten an update since 2018 and it's way overdue. So this past since the last update that was what I took on and boy So I don't usually put up cartoons, but you know, this is this is where we're going with this There's you know, how many different ways are there to package software? There's a ton and every time you turn around. There's another one and I can even understand the in the Reason that one would want to do that. I mean, I've been in the same situation myself We had our own packaging for a long long time ago so Had a release one five So I'm going to go through several possibilities and just Where they where I stand on them whether I think we're going to go with it or not So PPA that stands for personal package archive. That's a way so one to run by the company canonical funded by a really rich guy Has their repositories and you know, the Linux miracle to some degree has been this idea of once you have software That's open source that people don't have to go Begging to some particular person who claims Exclusive intellectual property rights to some software that means you can collect it all together into a giant Repository and you can have a systematic access to it rather than which we always use to mock the Windows approach where you have to go here to get this piece of software and pay him some money You have to go there and and to get another piece of software and pay him some money and so forth And it's a big mess and you're running installer.exe's all the time in Linux We have repos and it works so much better Well, that's great if your software is done enough to be, you know, mature enough to be folded into The main open source tool chain And I was never comfortable Was saying MFM had reached that point. Now, maybe that's my problem There's plenty of crappy software out there in Open-source land too. So, you know, I probably should have just released it and tried to get it into the official channels But instead I used a personal package archive and in some ways I now think this has sort of all the downsides of the repository and few of the upsides because number one You have to as a user if you want to do this, you have to be running Ubuntu or something that can handle this particular style of software packaging And you have to accept my Crypto key That signs this that says yeah, it's from me, but who am I? I'm just some schmo and you're essentially giving root access on your machine You're essentially giving total access on your machine to everybody whose PPA archives you download from and you know I've done several of them over the years for particular pieces of software that I needed newer versions of and so forth It's been fine as far as I know, but not the greatest way to do it. Well, and why has it been? So there it is uploaded by you know me 2018 since December. That's the most recent upload. Why? because There it is Oolong 149 weeks ago failed to build went through a whole bunch of stuff. I've mentioned that on and off so PPA seems like it's kind of the old way There's all sorts of new competitors like the xkcd cartoon suggests so How about snap snap is a new thing put out by this group called snap craft Which is actually it's part of Ubuntu part of canonical That in effect is a lot like a PPA But it's got a different kind of repository and and so forth and so all right You know, I looked into it some I tried to get going on it and I got frustrated You know, this was a few weeks ago, so I don't even have a lot of notes on it But you know, okay, so snaps over devs devs are The packages that the Ubuntu main repository uses for distributing software snaps are confined meaning You know, there's more reason to trust them because they're sort of isolated inside their little snap And that sounds like a good thing, but you know, of course There's incredible number of fine print asterisks on it like other snaps use interfaces Which are inherently insecure like x11, which means if your snap involves oh Drawing anything on the screen Then the confinement is a joke So again seems like really a lot of the downsides with relatively few of the upsides Flatpak is a competitor to snap that is done by The red hat folks. I didn't even actually look into it because I moved on to app image And app image is something that I know because I've used them in fact the Video editor that I put these updates together with it. Well, not when they're doing the live stream but when I was You know recording them ahead of time and then just uploading them once It was Caden live that I used and I used app images to get to it and it's very nice You download one file you make it executable and you run And that has you know And it runs on lots of different systems. That's really very nice And so I say hey, you know, come on. Let's do let's do an app image. So I started down that road This is a comparison Between app image and snap and flat pack the other way flat pack Now this this comparison is done by the app image folks I mean both snap and flat pack have significant corporate backing app images more the the rebel band that are In the more community open-source spirit and so forth. So, you know, this is a little bit biased But and as always with these things the key is not whether you have the checks in the columns or not So much as what you decide to put on the rose But you know, it's true So the great thing about app images is very you don't have to install it You just copy one file and do one little thing with it and you're going which also means if you want to get rid of it You just delete that file and it's gone snaps and flat packs that do real installation scatter stuff all over the place and so on and on bunches of Comparison issues so a week ago today last Tuesday October 5th. I started down the road of saying, okay Let's make an app image. Now, you know, should it be for MFM the simulator MFMS should it be for oolong? I said, let's make a single app image for the Living Computation Foundation tool set all of them put together So you just download one thing and then we'll have some scheme to dish off from one to the other within Inside it. So that's what I started looking for And you know, okay, now it's Wednesday a day later. I've been working on this and working on this and man It's pretty frustrating Part of it, of course is because of these open source rebel band number one It's a relatively small total number And number two, there's many many more people using app images than there are making them That's a sign of its success But it also means the Google search as you do keep finding, you know Yeah, all you have to do is set the executable bit, you know And when you're actually trying to find details about why my particular attempt to build an app image is Screwing up one way or another. It's a lot thinner on the ground and a lot of it is outdated All of the stuff that every programmer these days is familiar with I Got to the point of I was trying to use this Linux deploy thing. It seemed reasonable It got very frustrating. I didn't really understand so I was googling around for alternatives And there's this thing app image builder and it's like, you know, is just one of those related to the mother Do you use app image builder with Linux deploy? It was really unclear Eventually a little bit more googling. I found this quote In one the app image builder documentation that says and and unlike other app image creation tools It can be used in modern systems, which seems pretty snide But it turns out the other app image creation tools is a link to Linux deploy. Ah, I see their competitors And you know, so the app image builder approach It says, you know, let's just package everything I mean, there's this fundamental decision when you're packaging software to run on a system that already has Software, which is how much of the software that's already there. Do you use versus how much do you carry with you? And you know, that is an inherent fundamental problem The more that you can forage off the land the lighter you can travel But the more you have to be adaptable the more you have to say whoops, there is no regular expression engine There is no purple and what are you gonna do? So it's a lot harder to live off the land But you can move much faster if you can do it and that battle back and forth between how much do you include versus? How much do you trust will be there is at the essence of the software packaging problem and it's fundamentally why they all suck This has been a frustrating week it has been a frustrating week So I decided to try app image builder and by Thursday. Hey, I was just about at wit's end Um, I couldn't get it to work. I could get this to work and not to that to work all these little bits I could all these ambiguous clues from Google searchers But you know So part of it is is with the idea of packaging almost everything that you might need Part of it is this whole idea of a dynamic library now You know a long time ago when I was a kid when you would compile a program You would end up with an executable file that essentially had everything you need Kind of like an app image. You know these things keep recursing and reappearing in themselves But then a big improvement Over the time was this idea was a dynamic library that instead of having all the libraries compiled in with you You just request them when the program starts up. You say oh, I need the C library. I need the DNS library I need whatever it is and the The system would have more responsibility for linking you together with the dynamic libraries that you ask for now You know that not only saves space I mean it saves a lot of space because now if you have a hundred programs that are all using the C library They can all use the same copy. It's very nice very tricky But now when you're trying to do an app image it comes back and choose on your butt So you have to figure out all the dynamic libraries that that your program is expecting to use And there's tools for it LED and so forth. I'm doing all of this, but you know, it's not easy and Then you know as I get deeper into it deep, you know, because the simulator Uses other programs, you know, it uses Pearl programs to do crypto identification and so on So just finding out the dynamic libraries that the simulator itself needs is not the end of the line And you end up doing this, you know There's there's a program called s trace that allows you to sort of get a little Observing point all the way down at the very bottom where this program talks to the Linux operating system directly And you can get a hook in there and see what it's asking and this is it It's asking for you know Open a file for user share pearl user share pearl like this Which meant there was another pile of things that had to be included in the app image for app image builder to make it as standalone as possible, you know So by Thursday now, I am googling around looking for what other people have done trying to find other folks that have you know Used app image builder to do stuff and seeing that. Oh, yeah, you know, they are explicitly messing around with the list of places to look Because the way app image builder works is it tries to say I'm going to give you a fake entry point You know, if you thought you were going to look at a place called slash user slash bin in order to find a program like LS Well, no, you shouldn't you're actually supposed to look at slash temp slash barf Slash user slash bin and that's the way that the app image sort of isolates itself So that when you try to run a program instead of running the real one in the operating system that is hosting you Because the operating system is still there and the app image file is kind of laid over it without being separated from it And from the app image point of view from the app image user point of view. That's mostly a feature except for security Jump cut to Sunday two days ago now Cup of frustrating frustrating frustrating, but it looks like we had MFMS running on Fedora 30 That's a distribution of Linux made by Red Hat Debbie and staples Sentos Ubuntu Xenial and so forth could not get it working on Arch Linux at all and you know lost a lot of time on it and Have I mentioned this has been a frustrating week but Okay, later on Sunday I had managed to repeat the whole thing so that I could actually do it get an app image that I could run the simulator on a bunch of different Linux this make this same app image file Put it in a bunch of simulators That ran these different operating systems and try it out and I was able to get the simulator run So that was pretty good. I said, you know the heck with Fedora 33 and Arch Linux if somebody wants to help me out with that That would be super and then I said well, okay We the we had MFMS the same route was packaging up Luam in the same way let's let's go for splat and so I did that and It turned out that I built you know my original test just ran MFMS And it said, you know making that make an atom halt after apps, you know So the dash dash run so that you'd start the thing up And it would just run for a brief period of time like a few seconds of wall clock time and exit And that's how I knew things were working Well when I put splat in I made the test more elaborate so that it actually Initialized a little directory with a sample my element dot splat in it and then tried to build it and run it Oh, oh, no, I was not nearly as far down the road as I thought I was Because I hadn't tried the compiler I had packed Luam, but I hadn't tried running it and it was a mess and again not Luam's fault Just more packages more stuff And you know the compilations were actually just just crashing And I spent, you know a lot more time on that but now this is yesterday. Oh boy, so I'm letting app image go. I'm letting app image go at least for now Somebody's gonna have to be a lot more skilled than I am With using it, but there are other alternatives. So there's docker And in fact as I was sort of rooting around in the Luam directory, I realized. Oh, yeah two years three years ago Joe collard and AJ's F had worked on a docker script to wrap up Luam and MFMS Together in a single thing so that if you could run this docker image You could do it. So I said, hey, whoa, I could I could try to refresh that so I did that and it kind of actually got it working and I Actually got to the point where I could run a Compile a splat program to do a fork bomb and run it and see it going in a Docker container and a docker image with a container on top So I created an account on docker hub and got it to work and in fact You know, there were still issues, but it was a lot less frustrating The attempt to do the app image but again Docker is not nearly as inviting. You have to have the docker engine Running on your machine. Now, there's versions for macOS There's versions for Windows and Linux, but you actually have to have it there unlike the app image beautiful download execute go like that. So There's still trade-offs But you know, I was at the point where you know Let's say anything is good so that we can declare victory and come talk to y'all So that's where we're at. We're at docker and here it is I am live comp Docker hub and we have LCFT the living computation foundation tool set Mfm lulam splat all the accessory Mfm Z run and so on and so forth and it's there and in addition We've got a new repo on GitHub that is the living computation foundation is owning it that is elif's LCFT living computation foundation tools A docker the thing that that has got all the stuff in it that I used to build that image and in addition it's got So blah blah blah and a lot of this stuff is just copied from what Joe and AJ had done in 2018 I tried to update some of it, but you know, this So Do I dare do I dare try this I will I will So just press the correct button. All right, so I'm gonna make a directory Demo, I'm gonna go into it And then I'm going to get the Got the Earl here for LFC LCFT docker dot get And it's a small little thing, which is good LSE docker and now I'm just gonna try to run The in it thing. Let's see what happens All right and It died. I Think I know why that might be Can't believe I'm doing a demo. Yeah, all right. Yeah, let's get rid of this thing because we want to actually All right, let's try that again All right, all right Okay, there's docker pulling down all the little bits and pieces and making a an LS LCFT Local that has been customized to connect to my home directory so that I can use my existing files and so on and So I think we were so MFMS Look at that. Whoops We're running, okay, so let's see if I can do this in here This maker Test CD test Splatter dash initialize. Oh man, I'm really pushing my luck So now The splat translator took my little element The out-of-the-box element and translated into oolong and now oolong is running under the hood in the in the docker container and It takes a little bit of a while But hopefully it won't take too long and then we ought to be able to see my element Actually running in MFMS Okay, here we are Bring up the tool palette look at that M. E. Ah, it's my elements. All right and Let it go and sure It's a fork It really seems to me, you know, I used to feel bad about the fork bomb, but now I really feel like it's absolutely The logical starting place for distributed stuff. It is the hello world of indefinite scalability the thing that just says hey You've got events there You're a cellular automata that are willing to do things all over the place. Well, hey, I'm willing to go over the place Okay, that's the demo. We actually got it working. Yay And So we did it now Well a few more little points and then I'll wrap up now. Why does software packaging systems all suck so bad? Well, you know, it's a little story that sort of what goes around comes around This is me in 1994 I had just moved to New Mexico To become an assistant professor of computer science at the University of New Mexico and I was hearing about this stuff You know, okay, so there's this thing Linux. It's this new Unix like operating system cool Great, you know, I've been using Unix for years Linux and new thing and neat But you know, what the heck is it the buy-in? What the Debian Debian Like that people were talking about it. I didn't know what the hell it was And as far as I could tell from you know looking around it seemed like basically it was a bunch of marketing people that were just trying To steal the thunder from Linux, you know The hard part was making the actual operating system itself and then you know What the way it would work is you'd get the operating system with just the bare minimum of stuff And then you'd put your own software on it or you get your what the one package you needed Use it. So these guys they're just sort of putting themselves between the user and everyone else and saying what are we doing? We're collecting a bunch of software together. That's all we're adding. We're not writing it. We didn't write the operating system We didn't even write any software. We just write the collection Debian Ian Murdoch and Deborah Murdoch the folks that started the thing and The whole concept of a distribution a distro was absolutely foreign to me and it seemed just like you know It seemed like marketing it seemed like people coming Johnny come lately trying to come and take the credit and I now see you know, I was totally wrong and you know The more I have looked at spatial computing the more I have looked at indefinite scalability The key point is structure at all scales. So yes, the Linux operating system incredibly important incredibly challenging But and you put one or two programs with it. No problem You put hundreds and thousands of programs with it problem And so you need organization at the next level up and the distros are doing that And now the software packaging systems the app image snap snap chat snap chat snap pack flat pack They're trying to do another level of it as well and That makes sense the thing as everything gets bigger and bigger new levels of structure will emerge and will compete with each other and you know May perhaps one will win You know USB see maybe it will win But in the case of software, it's all built on the sort of completely Structuralist CPU and RAM model and so that means it all collapses and the Isolation that you think you have between different apps between snap confinement, you know that says it's confined So it's better, but no, it's not really confined and you know app image goes the other direction It really doesn't even try, you know, it's Infiltrated with the actual system underneath and you just got a trust you just got a trust Why it all sucks. Well, it sucks because it's hard Because you know software incredibly complicated Computers incredibly powerful, but ditching space giving up on having spatial computing and going with CPU and RAM Makes it that much harder and fundamentally the way to get along is with good fences making good neighbors This is all my acreage here, you know, I got Bob Smith on the South of me and Sally Jones to the West But you know, this is my spouse and we know where one ends and the next begins Can't do that and CPU ran. It's all one big sack of soup inside. All right It's a very frustrating week So the insiders report and then I'll wrap up real quick So I've been you know, that's my own thoughts, you know Maybe this is gonna succeed. Maybe it's not gonna succeed in my lifetime Maybe a little bit who knows but how are we gonna have some fun doing it? and I came up with one idea and it's Let's build the damn grid, you know I built a hundred and fifty plus tiles over a year ago now and have put together, you know A handful of them, you know, twenty twenty five or thirty or so of these But the other hundred and twenty plus are all sitting in pieces in the boxes because I wanted to get the software into better condition Screw it. Let's just build the damn grid And look at it see what it does try to run things on it and see where it goes wrong. So We I'm gonna get a better camera because as you've seen as I've been struggling to try to figure out a way to record these grids You know, it's very low light and you know a fixed focus camera is a pain in the butt So I need I'm gonna get a sort of half decent camera that at least claims to have good low light Autofocus performance a lot more pixels so that we've got a chance of getting a fairly big grid and to make sure that it can be Controlled from Linux so that I could do a time-lapse stuff who scripted and so forth gonna get a new t2 disk image It's that's been several years on that as well Get a bunch of copies ready so that we can start to build the thing In the hardware side, there's a few little bits that are necessary number one the You know, there's heat is a significant problem It's one the reason that we want the grid to be vertical because it helps heat rise and clear out rather than putting it all out on a table I started using little aluminum heat sinks on the processor when I put these things together And that was a last-minute change once I learned about the heat issue being more than I had thought Gotta get more of those I only got a smattering of them gonna have to figure out how to support Source them in the modern supply chain disrupted era also some more power supplies and then just sort of go through all the Inventory and make sure we've got all we need You know, I mean some of the the hanging stuff is busted and gotten used and so forth Concerned particularly about the intertile connectors that are data only the ones that are supposed to be go in a ring Around a group of tiles that has a single power supply So the timeline I my plan The aspirational timeline is we'll have one more update on October 28th to catch up on where the preparations are at I am gonna be gone for the month of November And you know, am I gonna be writing fiction putting stuff together? I don't know But also the hope is to come back on December 6th with t Tuesday update 31 11 and have you know Half-decent by the t2 tile project standards build video. That's the plan I Hope everybody is doing okay. I will be hanging around as always in in the livestream until one o'clock and Hope to see you next time