 Hello, everyone, and welcome to today's Bitesize Talk. Here with me today is Matthias from the CELAF Lab Data Center. And he is going to tell us anything new about the end of core website. Yes, welcome, everybody. Me and Matt might have already seen that there are some smaller changes on the website. Actually, in the background, there were quite a lot of changes. And I structured this Bitesize so that the start is just a quick overview of what's happened and how you can interact with the new website. And then the later part is more the technical part. So if you're more interested in that, stick around. We'll talk a bit more about why we choose what we choose and how you can use it if you want to make some changes. Yes, so we made a new website. But what was actually wrong with the old one? You might have seen more and more desperate messages from Phil over the years that we made some decisions, which came back to us and also, which includes also, for example, the hosting where we kept crushing or like kept overflowing the memory by opening the stats page on two different places. Stuff like that, hopefully, is of the past with a new website. And then I think at the beginning, jokingly, just kept sending this message, especially to Varsal. But yes, I accepted this challenge. And then one wintery day last year in December, I went out to Newcron, which is really far away from Stockholm, not so far, but it is outside of Stockholm. And we had a nice sit-down in this and felt living room and also then in this garage to just start coding on this new website. And yes, started on like December 8th. And just eight short months later, and a few thousand lines of code changes, we were at the state where we needed to or we could launch the website. And we pushed ourselves with some encouraging gifts. And now we have the new website. And what do you do now with it? Well, actually it didn't change that much. We tried to basically rebuild a lot of things we have because we liked, for example, that you, when you want to edit something in the markdown, you can just hit the edit page and are directed to the repository on GitHub. Or if you scroll down the whole page, there's also always a link to the markdown if it's a markdown file, which is a basis for a lot of pages on the website. If you are actually in a website repository, most of it lives inside the source slash content folder where we have this and then for the about files, docs and events are the main files for the, which are rendered on the website. Of course, the pipelines are also rendered there, but they live in the pipeline repositories. More about that later. One thing you might encounter is that we have now stricter build checks for the front method. So when you write the markdown, you might have seen that we have this, not just markdown in our files, but we have the front method distinguished by the three dashes, which we use to collect some metadata about the file itself, usually title and subtitle. But here, for example, in the events, we also have a start date and a start time. And now because we have so many problems with dates and like countdowns for dates and that they show up correctly, we now are stricter and how they have to look like. And but with the start time and the end time, we now enforce that there's a UTC offset. And if you don't do it, you get a build error shown, which looks like this, which hopefully has a helpful error message for you to know what went wrong. But that's not all what we can do now. We have a lot of cool other things. For example, you can have now more than one usage pipeline documentation or the same for output. So if you want, for example, to have like troubleshooting as its own site on the pipeline documentation, you just need to create a folder called usage for output. You still need to have a usage.md and an output.md, but inside that sub folder, you can put whatever markdown file you want. And then it is represented on the website with this nice sidebar on the left. And those are the icon itself changes a bit. Other things are the, you can, we have a bit more markdown flavors now or like markdown magic. You can have these things called cobalts or admonitions. You might have seen now with the new template. We include it then in every readme. And the basic syntax is always three columns around basically a block quote and then some signal words like notes, warning, danger, or you can also customize it however you want. Like you see below that you have your own icon and their own specific title. Another small useful things are like that. We render our mermaid crafts. So if you don't, yeah, if you want to quickly show flow of some information in your pipeline or whatever, want to describe with a quick mermaid crafts, just write it like a normal code block with mermaid language. And you're good to go. It will be rendered then on the website. Actually, you can not only write markdown, you can write also MDX, which is basically markdown with a little extra and the little extra are components. For example, here, a swells component, you just almost write it like you would write it for Astro. You import it, then write your normal markdown, but you can write this kind of HTML style in your markdown and this is then rendered as the actual component. But actually all the rest of the file is normal markdown syntax. So with that, we can get a bit more styling and custom features into our markdown files and don't need to rewrite or don't need to write so many pages in like Astro or swells, but actually can't have them. So still that's basically markdown files around. Another cool thing is because we changed hosts, we have now Netlify and that brings us cool feature that every pull request you open on GitHub with the website has its own deployed preview, which means the whole site with all the data of the normal site is built with your changes in it. And you can then quickly see if actually all the changes you wanted out there and everything actually builds correctly. Very handy, very cool feature that build time currently is around eight minutes, but we are getting closer to five minutes now. So it takes a bit, but then it's there and it's very useful to see if actually the markdown you wrote was correct or if you maybe find more typos when it's correctly rendered. And also quite recently, we just added the site-wide search or I also use it as a quick navigation some top of the in the north bar or if you had hit command K on Mac, you get this command pallet almost where you can search for all the things, all the through all the pages and quickly go also to recent ones to yeah, very nicely jump between the pages and hopefully discover all the chance we have on the pages because there's often a lot more on the page than people discover by themselves. Also something nice now is that we have all the information we use to, for example, all the pipelines and the modules we have. This is not done via get-up action which runs once every morning or you can also take it off manually and then you will see, for example, your new release of the pipeline or whatever changes you made on the pipeline or modules side of things. With that, we will go over now to the more technical details. If you have some questions to the previous part, stick around to the end, you can ask them there or just write them in the chat and I'll come back to them at the end. So now a bit about the technical details and what were the challenges we had. One big thing which keeps getting to us is that we have our data, not just like a classical and more repository. We have the website repository which by the way has also been renamed. It's not NF-CO-RE anymore. It's now website. Then we have the events, the docs and all the website code. But of course, we also pull in all the data from the website repository, especially the documentation markdown files from there. And with that, it makes it difficult and no real framework is actually built to handle this very nicely. But we have it like that and we figured out our error on it. And other requirements we had is like I mentioned before that we want the markdown from different sources but also we want it to be mainly static because most of the files don't need to be intact or like need to be built every time somebody loads it which makes them also fast. We also wanted to make it easy to add things to the website if we come up with something new or if next book comes up with something new suddenly we need sub-sub workflows or sub-modules, whatever we should be able to handle this now quite nicely. And also we would like it to make it easy for people to contribute which also was one of the reasons why we wanted to get away from the old website because the old website was written in PHP and it's a very nice language. It is just, it has its age and you feel and see its age. So our thoughts was okay, let's try maybe a new website and then maybe more people are willing to touch code on the website. So first we thought, oh, let's look at the cool more kid on the town. But I had my problems with it and thankfully there was a newer kid on the block which was Astro. And for Astro it does like nice aesthetic websites but it needs other frameworks for interactive parts. So we use Swelt for all the interactive things there. So why Astro and not for example, Gatsby is because Astro really tries to just send HTML over. If you don't have any interactive parts or if you don't explicitly tell it to send over some JavaScript, what you get from the server is just HTML and CSS. Very nice, very fast. I like it. But you can still use some JavaScript. So it's not that you have to give up completely on a nice features of JavaScript. It is just that it tries to not ship more than actually need it because like for a lot of things we use JavaScript to like filter out things or get some information, but that is just happening on the server and the user or the browser actually doesn't need to do a lot of these things. And another cool thing is because we weren't sure which interactive framework we wanted to use additionally to Astro. It is basically you can use more or less every framework and we still could now add react, react, do, swell, whatever you want, you can have on one page basically. So that's very nice. Astro of course comes with its own philosophy and specifically the password for Astro is islands. Everything is an island or an Astro and therefore it can decide how to handle it. So here you see different islands we have on the main website and actually some of them are interactive. So some of them need JavaScript loaded and some of them are built with JavaScript but just on this on during build time and then the JavaScript is not run anymore. And you also see here already that we can even tell it when to load the JavaScript. So it can load the JavaScript on load or also just if the element becomes visible or also if it's just when it has time. So very nice way to kind of decide when to actually put some load on the browser. How does an Astro file look like? Quite nice I think. You have front meta. So like we have for Macdon here you put all the JavaScript code there then you put the HTML and GSX and then the styling at the end which is the component wide styling which means we can also reduce our CSS amounts quite a bit. We don't have, we still have one big ish file but it's not like dealing with everything anymore. We can now more nicely target our stylings just for the components where we want to have it. Another cool thing with Astro is something I mentioned before that we can now check for things on the front meta of the Macdon files which Astro calls content collections. And it's basically just the JSON schema here it's a JavaScript schema for your Macdon and Macdon X files. So we can here for example, write some checks for formatting but we can also transform it and mix it a lot easier to keep everything in shape and all the Macdon files in similar way. And with that, keep the code quality higher in our repository. Yes, so that's why we chose Astro and why it's well done. Well, of course the big leader is React everybody kind of knows React not so many people actually want to write React and I'm one of these people. So here is what you would write to change a checkbox from checked to uncheck. And in Swelt it is like this you just give it, you bind the status checked on it and have a state on top in the script part and that's it. It's a lot less code, a lot less things to initiate. It's just, it's also I find when we look at the file structure a lot closer to how you write Astro. So that you don't need to jump around that much. Still sometimes confusing because that makes them a bit too similar sometimes. And then you write Astro as well, which doesn't work but in general the structures you have on top a script section, then HTML with some JavaScript flavors in it and then CSS. So the big change is that you actually need to declare it as a script tags and not with the front meta like in Astro and you then add it like if you want to then use a Swelt component in Astro. It's very easy, you just import it like whatever you are else you import into your front meta and Astro file and then you add the component there. Something I often forget so I wanted to mention it here explicitly. If you don't add, if you add for example, a button and don't add the client idle or any client directive it doesn't do anything, it's just rendered and no JavaScript is loaded. So if you think something is wrong in the Swelt component check first if the import and the Astro component actually is then run on the Astro side as well. Yes, and if you want to give it a try locally and just clone the website repository then you install the dependencies with npm install. You build the cache because we have all the markdown files we cache we cache all the pipeline information and the module information we build a big caching file to save on build time and then you just do npm run dev and then on localhost 4321 you kind of see the websites and if you just want to test it if it's still built you can either make a pull request and let if I build it you can build it locally with npm run build. Yes, quickly some stats before I come to the end. So in total we have now almost 4,000 pages built which is all the websites all the different, all the pipelines all the different versions and one of these at 3,900 is actually server-side rendered well actually it's more than one but one subgroup of these all the AWS result pages. So the first layer is still statically built because it's faster and I want to speed with this website but all the subsequent steps are then rendered on the server so they're a bit slower but you shouldn't recognize it too much the delay is actually coming from the call to AWS. Yes, and I mentioned before the average build time on Netlify is eight minutes I just yesterday night got it below six minutes so there are also some improvements in respect to that for the hackathon coming on the way but still it's a bit of a hefty website so be a bit patient when it builds but it has done everything there. Yes, with that, thank you very much especially to Phil for up in code and also like pushing that we re-write it and accepting all my weird quirks of I want to have it like this and let's not use Gatsby please no and also thanks to Edmund and also Matthias Zepa for pushing towards different frameworks pointing different things out and also yeah, keep pushing Phil that we actually rewrite this website and the Zylaflap data center for financing me. Good, with that, do we have any questions? So anyone can now unmute himself if they want and also start the video, so. Yes, one comment from Phil is correct if you, so only the build takes this eight minutes if you run unpair and run death it actually doesn't build the whole site it is basically doing server side rendering so you get it, it takes like 30 seconds to start up everything. Are there any more questions or comments? Is there anything you do differently Matthias now if we're starting this project again? Would you still use Astro? Yeah, I think I would still use Astro actually I'm still very happy with it and let's see how it develops but like it just got, yeah I'm still very happy of how little JavaScript do you actually ship and what I would, well, no, no, actually for now I'm still very much in the honeymoon phase with the website I think it's like, yeah of course like all the stops pages and like maybe not rather than chasing the file but that is something we will of course fix in the future. Yes. Is there anything else, comments, questions, requests? If not, yeah. I was just gonna say to note on the cat's pee thing just in case anyone's not aware of the gossip of different JavaScript build frameworks on the internet which is quite a niche topic. Gatsby, since we started this project Gatsby was acquired by Netlify and then just recently there's been rumors going around that Netlify is basically fired the whole team that works on Gatsby. So just the week or two ago we were sort of like, okay it seems like hopefully we picked the right horse I think it's a question. Another fan of JavaScript. Cool, then thank you very much you did amazing work over a long time and I hope this website will stay with us for a long time to come. Yes. So thank you everyone also for listening and of course as usual the Jean Zuckerberg Initiative for funding our Bites as Talks. Hope to see you next time. Bye.