 All right, where do we put the stickers? Yes, all right, cool. Hi everyone. I'm Likai from NUS mods, right? So today we're gonna talk about NUS mods.com, but What is NUS mods? Anyone here who doesn't know what NUS mods is? All right, since like everyone knows but we'll still just go through it, right? So NUS mods is a timetable builder for all your random school module stuff, your timetable planner and There is a question like what NUS mods mean. So does anyone here know? Anyone wants to give it a try? Yes, that's right. It's NUS modifications and for that you get a sticker, but we have too many so Everyone's gonna get a sticker. Yes. Yes Sorry All right, I'm just gonna pass you a bunch of stickers right Yes Yeah, so anyone who wants NUS mod stickers have to make friends with him, right? Right, so what are we gonna talk about today? So it's very simple It's just like going a rough history of what NUS mods is and how it came to be then it is we're gonna talk about the rough architecture of how we build it and How you can start contributing to NUS mods as well as like some random Ideas and thoughts we have about what NUS mods in the future would look like So how do we get here so entire story of how NUS mods got here is Must start from our founder in 2011 so his name is bang as in up bangs bang So bang had this itching problem because the timetable builder then was like terrible like Really hard to use and he got very frustrated. So basically he decided to build NUS mods as the exercise right and Coincidentally it solved his problem because he wanted to learn some JavaScript. So it was like a good fit and in general, this is how we want to promote building things which is Try to fix what is going on with your life like certain small issues that you don't feel like there is a good solution for Or even if there is a solution, it's not like really nice to use like you can invent your own way to build things right because we are after all software engineers and Building things is a really good way to practice your things So if you want to learn something new like JavaScript, you can build a web app and if you want to Learn some Python. Maybe you can build some with machine learning output like Deep fake your face over Donald Trump. Yeah, things like that So we had many many iterations So the slides have all the links to all of this, but I will just show you So the very first version is called core set For those who don't know We used to have this Old website called course that had all the module information stuff, but now it's gone. Yeah, we we killed him Which I mean we we sort of replace them we sort of replace them in some functionality Yeah, so we can see like very early Idea of what any of us mods was gonna look like the UI wasn't like very pretty And so you when you type in you know got with stuff that when you hover over like this won't ever work on a mobile phone Right, so, you know and in order to move models around you had to drag and draw Yeah, so it turned out into discount layout So I mean it worked It's functional and It's better than what they had so it was a good idea, right? then We started iterating more and more so this was built by Bang and partially Parts of it by young Sun who is the second person who worked on any of us mods. So You can see like it has gotten a lot more powerful. Now you have Filters now you have various faculty selections You know you could add modules now and you had a timetable builder that looks Way better, right? So This same idea that you could track stuff It's there then Johnson came into the picture and he decided to build a new version like Not sure if it was a complete rewrite, but it was built with Very modern tech then like JavaScript moves like a bullet, right? But this was considered modern. So it was like marionette and handlebars. Yeah, and it looks Quite modern actually even now then it still looks quite Modern yeah, so you have All your various track and drop you had module search with all the very powerful selections and You had Pre-requisite trees that were rendered with D3.js and you have discuss For auto reviews and you had venues to look up All right, you had an ass whispers link or embedded right in there and you have news and Other love more powerful things So this is The user interface that most of you should be familiar with so this is the third version of NUS mods So this was built in 2016 all right, and it was triggered by this random issue saying oh Our stack is starting to get a little old and the modern version of web apps are built with react and Redux and we want to move towards that direction And just so happened I was building my own version of NUS mods I was like hey, you guys have the same requirements that I build like mobile first Having new color schemes adaptable color schemes night mode, you know so I joined and I Brought over a few a few other people and we built this together So this is a current interface. I won't go too much into it Yeah, so now hand over to Yiliang would go over how and what our architecture is Yeah, hello, so oh Right so so Okay So like how So Like many of you will be wondering how NUS mods actually works Because like as a user is quite cool right you open NUS mods and it just works But you don't really think about how how things are laid out behind behind the scenes. So This will be a high-level like discussion of NUS mods as a whole as a system. So So So if we won't go into the code, it will just be a bunch of diagrams. So It's actually really simple. Okay, it's just this so NUS mods can be divided into two main parts. So There's the front end which is the web though and your smalls.com website the one that you're familiar with and that's the back end Which currently consists of two services. So the first is the elastic search module Modified surface powered by elastic search. So what what this is is the It's actually our module if you go to the modules search page the module tab of NUS mods It's powered by elastic search. So we have very fancy features that I will show you later And then the second one is just a data server Which provides the module data and all that to the front end when you load NUS mods. So The architecture is really very simple. It's just this is at the highest level is just like that So the reason why it is so simple is because of historical reasons So as leek I mentioned just now NUS mods In historically NUS mods has very very few people working on it So back then first one was bang because first guy was working on it. Then after that there was young shun So then the NUS mods was built and maintained by one person at that time And even though like basic whole of any of us using it as we are as we are using it now So right now the NUS mods team has like four people three developers and one designer. So it's still very tiny Which means that we all need to like conserve our engineering resources because it's very limited, right? so we try to make sure that things are very very simple and So that it can be maintained or like you can just be left left to run even if nobody has time to work on it That that is a that is a goal behind why NUS mods is structured in such a simple way so but It's actually not so simple. So this is at the highest level, but we can drill down a little bit in And it will look like that So this is basically how data flows throughout NUS mods at a slightly Deeper level. Okay, I realize this but maybe a bit low for the people at the back, but I'm sorry. So Yeah, so the first thing you could you would ask is how does NUS mods get its data and it actually all starts here so So starting Yeah, starting this year. We've actually had access to NUS APIs directly So what happens that we will take so our we call it a scraper, but it's basically a tool So it's written in Node.js. It's a It's It's just a system that will run every hour and it will pull data in from NUS So it will it will take it will like look up all the module information and time table information for NUS Then it will collate collate and process some data So it will generate like the prerequisite trees and all that and then you'll dump into some files on the disk So and NUS mods doesn't have a database So we store all our data on the hard drive and it's actually pretty cool So you can yeah, you you can see it You can see it So if you go to API.NUS mods.com you will notice that you have this list, which is basically generated by NGINX So you can browse you can browse the data. So this is this is what the old data look like We are like semester two for example, and then you have a whole bunch of JSON files It's just sitting on a hard on us on the hard drive on a server somewhere that That's really it so you could open anything and then you can you can just view. Okay, I guess there's nothing this far But yeah, you get the idea Yeah, so the the files as mentioned is served by NGINX which is a reverse proxy But you can think of this file survey if I'm not familiar with reverse proxies So you have a bunch of files on disk We serve it up to through NUS API.NUS mods.com which this reverse proxy is listening to Yeah, so at the same time because now we have elastic search. So this is also a new thing we recently implemented that so now Since like July also we have had this so previously we did this so it's even easier as you can see But but but now we have elastic search. So what? Yeah, so we need to update elastic search Whenever we scrape so that the data on the server on the search server is synced up with the actual modulator that we have So elastic search is hosted on the cloud. We use this thing called elastic cloud, which is basically a serverless hosted elastic search and Kibana if I'm familiar with that So the raw data on elastic search actually looks something like that. So you have a whole bunch of like modules and each module has some stored data and So because elastic search is optimized for search like it's a search engine. So you could we enable things very fancy search features If you refresh this you notice this search like if you ever open the network tab on the off your web debug I don't know whether any of you actually do this like for fun sometimes But there is this underscore search This is a it hits a elastic search endpoint. I just over here And so we get very fancy elastic search features like if you search for like lawyer it will it will it will suggest it will actually suggest like like That you search for laws instead. So you get this very fancy like very advanced spell checking and Sorry Yeah, there is fuzzy matches. Oh, yeah, this one is like a side suggestion. So the spell spell. Yeah for the match Yeah, so you could get things like that Yeah, so that that is our back end. There's the whole of our back end that there is not much more to this Then on the other side will be our front end which is Yeah We all our code is stored within this website So we have used a mono-repo structure which I will talk about later But we have a website code so the website written in react and redox. So it's a very very modern web web Text tag so react without web pack you SCSS Okay, I guess that's not very modern anymore. But so we have that we will compile that Because it's JavaScript so we can compile it into a static bundle Which also is on this It's similar that and we use a reverse proxy which we use a different reverse proxy in this case But we use a reverse proxy to thought to serve it up to users So it's very it's also a very simple structure. So this compiles this compiles We run this compilation whenever we want to update and yes, but then after that it is just on this that we can just serve it And so it's very cheap to serve The one fancy complicated thing is this this exports of it. So if you ever notice when you Like when you export your timetable it lacks a bit Okay, it takes a little bit of time like this download button on the right Yeah, these buttons actually hit that spot server and the so it's actually quite a funny hack What what it does is that it opens and yours mods on our on a headless chrome instance on the server. So Yeah, which is why this arrow, right? So we actually load the website on the server then we take a screenshot and we send it to you It's genius. It's genius. Yes That's how it works. Yeah, it's so smart, right? Yeah, it's kind of like a glove Yeah, and everything goes through the all traffic goes through the reverse proxy to Oh, yeah Just you guys may think this is a heck right, but actually this is like the proper way to do things previously we Had a hackier solution. We actually rendered HTML directly into a canvas then through the canvas it Generates a PNG file. So we had no back-end like everything was client side Yeah, that was way hackier and with our proper solution you actually get consistent screenshots now Yeah, I think the problem with that was that I think Safari couldn't export PDF or something, right? Something was broken in Safari. So Which is why we had to do this Yeah, so and this is basically how NUS mods This is NUS mods that there is no much more to this the reverse proxies do more a few more things And also, oh, yeah, I guess another thing is So some of these things are actually served with Docker So if you look at this blocks this box over here, you notice that we say this is on 3.nus Actually, so we have actually have two digital ocean droplets. We are too busy to serve One is at 3.nus mods.com. This one runs some Docker containers. So this and this and this I'm Docker But then this is not this is our old one.nus mods.com Server so we actually wanted to put this under Docker as well. Then we can combine everything into one server but we are getting there and It's taking a lot of work and we don't really know how to do it. So That's why we have two servers like that Yeah, so that that that's actually all for the architecture. So I'll pass on the time to Vika again to talk about how you can contribute to NUS mods Yes, it's highlighted for some reason. Yeah. Yes. Oh, no All right, I think I think it sort of does make sense. We want to highlight this section. Yes Yeah, so why do you want to contribute to NUS mods, right? So I Would think for myself my primary reason for spending so much time on it is that I actually Contribute some real-world impact to students to all of you guys, right and myself include it I can't imagine my life without NUS mods. It would be so painful. Yeah So your work when you contribute to NUS mods is used by Yourself your friends your friends friends your friends friends friends and so on like everyone I talk to you like hey I contribute to NUS mods and then it is like you generate more topics and stuff. Yeah, and Really a lot of people use NUS mods like way more than I would ever expect like we have analytics that show this so This are our visits over time, but the important part is like this section down here for those behind who can't see we get like We got like 2.2 million page views Is it we didn't last month? Yeah, just this last month. So that's how many people are hitting and NUS mods.com right and You know, it's like what 63,000 searches Yeah, you know and we I think for Unique number of visitors roughly 60,000 people. Yeah, so that's like the entire school body who's using NUS mods. Yeah, so that's how much impact you guys will be Doing but just contributing to NUS mods Yeah, and the next thing for me as well, right when I started contributing to NUS mods is I didn't really know that much I only knew front-end I only knew like View JS and I didn't know react. I didn't knew no redux. I didn't know what webpack was right so the previous Contributor young son actually guided me along and he taught me so much. I wouldn't be here And I wouldn't know as much if he wasn't there like guiding me along the way being a mentor and like giving advice And similarly, I've learned things from my peers as well like iliang like Other contributor who is Yijiang Yeah, sorry. I'm so bad at names Yes, yeah, but you know, I just learned so much from everyone and that's how you grow as an engineer Which is also another reason why you should contribute So the next reason is, you know, you gain experience NUS mods. We use a lot of modern technology and Throughout this entire experience you get to like play around things you can make mistakes because well if we screw up Well, you're getting impacted as well. So you probably should fix it Yeah, and the technology that we use are Also used by companies that you would want to apply to right. So it's not some legacy Java that we have Yeah, so Yeah, but at the same time is also quite weird Yes, so, you know, you can gain experience doing all sorts of things Yeah, so this is a list of what we use don't be scared if you don't understand any of them. I only knew I Only knew this this when it started. Yeah, that's all I knew when I started, right? Yeah, the rest was like a slow figure things off. Yeah Yes, and I think this wasn't really obvious and but now it's quite obvious So when you contribute to NUS mods as a core contributor with like a lot of comments and pull requests You actually get some sort of attention to your skills so because it's public because it's Out there for everyone to see people Recognize the work that you put in and they know the quality of your work Right. So it's obvious to employers like why they should choose you over the other intern or the other full-time worker so personally when I Summitted my first resume to I think it was a computer science computer science fair That means yes, but I submitted my resume at Carousel and Turns out the recruiter actually circle my resume and put like mass Like must find and get them to you know try out for Carousel Yeah, so that got my foot into Carousel, which then got me into other places and you know has a similar story Yeah, so I So I asked I asked one of the okay, so my internet manager at Facebook actually told me that one of the reasons why like they They hired me as an intern is to it's because He could look at he could look at my my PRs on and for any of us And he can he can get a holistic view of like what it would be like to work with me as an engineer Because I like a bunch of big PRs, especially so where you know You basically list out like pros and cons of various engineering decisions and all that and also like like the code quality of like your work Yeah, so I I think in general is very valuable to have such public Work that people can use and also at the same time you have some impact So it's not like a very like contrived project So you have impact and you can People can see what we like to work with you as an engineer Yeah, so this is one for the people who support open source. Yeah You got interested I have Facebook open source tickets. Yeah Yeah, so The other thing with any of us mods is we are nonprofit like we don't get paid and we do this for ourselves and To help other people around us. So it's not saying that you know We're only doing we're in it for the money and we also do it for some philosophical stuff Like always open source and open source is good. Like we should contribute to open source. Yeah All right, so How to get your fit wet is pretty simple I would think it's just picking issues that you're interested in and issues that you think might be a Good entry point to understanding how any of us mods work and just like asking questions If you're not familiar with it reading up documentation and you know, just keeping it up with it for me my first PR took Took a while because it was a webpack stuff and a new zero thing about a pack. Yeah, so, you know, I Saw like fake it till I make it. Yes, and that's how I got in Yeah, so for our demo, we're going to assume that you know get get up How to CD into a directory how to view the code how to edit a code then how to click save or control S Yeah, I'll hand it over to you know, who's gonna step through Yeah, so for our demo, so we're going to basically it We are going to demo like what what do we like to actually Contribute something to NS Mods like the entire flow from getting the code onto your computer setting up and then making a PR to NS Mods, so you can actually we actually make a PR to ourselves right now No Yeah, so what what we are going to do today is that? Okay, so some contact so basically and your smogs has a Yeah, so um, and your smog has a beta mode in case you're not aware But we have a beta mode where if you enable it you will have a today and a and a planner tabs in the in the cyber and so these are features that are That are not completely ready. They are somewhat buggy, but they kind of work so if you ever never tried that out you should but For the purposes of today, we are going to assume that like we Like these features are great and that we're going to release it to the public So we're basically going to disable beta mode on these like right now, so Yeah, so our GitHub repo is on our GitHub repo is here It's a github.com slash ns modification slash slash and yes, but if you can't remember it You can actually go through and you can go to any smogs.com at the bottom There is a github link so you can just click that you will end up at you end up on github and So the first thing you need to do is that you need to fork the fork the repo to your own to your own account So we are assuming that you are you'll have a github accounts But if you don't just need to sign up for one, it's quite simple Then you just click the fork button up there and it will copy it to your account. So we will do it now It usually takes a while Yeah, so now if you basically clone it to your account so your account his account has it will be at github slash leekai slash and your smogs and This is not on your account. You have you have our code on your account So you can modify it however you wish so to do that the first thing you do is you need to clone it to your computer So this is basically at the gate the gate clone process If you are familiar with get that should be quite obvious Anyone not familiar with it By the way, is everyone familiar with it Okay, great Yeah, because otherwise you may get a bit lost because the contribution process requires get Yeah, so this is just we are just cloning the repel now And you must actually clone the repel because otherwise you can't push it because you don't have right access to our repel Our repel on on our account, right? So then you won't be a push your code up So if you'll fork it then you can push your code up. Yeah, so now we just open the code and He's using VS code. I would recommend Vim if anyone's asked Yeah So um and just most as we mentioned just now and your smalls is a is a mono repel So did what this means is that all our code for any small is stored in one giant repel. It's not that I mean It's not a giant but basically Everything is split into a bunch of files a bunch of folders So you can see like the export server is up here then the the website is stored here So it's it should be quite self-explanatory So today what because we are working on a website actually most of the most of the world and yes Most is done in a website. There's a barely anything being done for the other things But so you basically go into the website folder and there will be a remi Which should be rendered by I guess I guess it's not rendering There you go. So the the remi is quite comprehensive When you open it you will see like we go through details like how the folders are organized within the website project and other things like that So what we're going to follow today is the it's a setup process. So there is the somewhere That's what I was talking about the photos. So we are going to do the we're going to set it up on our on this fresh clone So the first thing we're going to do is we're going to run yarn to install all the dependencies for website onto the machine So basically see team to website You didn't see the website Yeah, you need to see the website first then you then you run yarn and it will install all the dependencies to your machine And then once that's done According to me you'll run young start which will start the server and you will be to open and your smalls on your machine But we need wait for this Or in the meantime you can get a sticker Yeah, I think we'll pass you around Yeah, okay. So yeah, yeah, it's done. So it has installed all the dependencies for Yeah, yeah, and then you just run young start so that you start the server I See the young start come out actually open your browser to locals 8080 Which is what we're doing. So now we're waiting for right back to compile the come out the code So if you are if for some context, I guess young young is our dependency manager, like it's like MPM, but better It's I mean it's supposed to be it's supposed to be better. That's a result is created Yeah, I guess even on the I-9 Okay. Oh, yeah, it's done. Okay. So this is if you if you sort of you are just on its locals 8080 Which is the the code that's running live from your machine. So this is a code that you have here and Basically whatever you change here will show up here So the so what we're gonna do now is that we are going to enable the we are going to make the button show Right, like what we said just now even if you're not in beta mode So right now as you can see you are not in beta mode because the buttons are not there So what we need to do is because because we are using react So we basically need to find the component that Represent this sidebar So what we can do is because we like let's assume we don't know where this file is right You can just search like one of the button text so we can search like Like and try like I guess planner. Yeah planer works So if you if you tap through a few a few times you will find an F tabs An F tabs dot txx file. So this is Yeah, so if you open it, this is where all the all the buttons are so there So all we need to do is that we need to get rid of the Checks the props to beta check and then the the buttons should appear if you're familiar react like this should be Basically, this is a this is a Boolean and if it's if it's true this will be rendered Um So, um, what while it will remove the things what he will not save this stuff What will happen is that when you save actually two things will happen. So So our tooling is actually quite fancy It's also very modern. So what will happen is that there's One so there are two things right the first one is that if you notice now because he removed the code The check now this part of the code is Invented wrongly like you see is there's this extra two spaces over here And also there's an empty line, which is a bit ugly So what we have is that we have prettier if you have if I'm if I'm not familiar that prettier is a tool that will format a code According to some rules that you specify. So we have this setup in our grapple So when you save that will that will happen. The second thing is that we also have a whole module replacement which which means that When a react component is updated, it will be it will be swapped out life without reloading a page So when he saves this well, what you will expect is that the two buttons will appear here and the code will reformat. So that If you save it Okay, that that happened. Yeah, so it is reformatted now and the buttons have appeared So even though we are out of we are the better most of So that is our that that is basically all the changes we need to make to the files So now we're going to show you how you can commit and you can make a PR 2 and your smalls So you first need to commit because get right. So you need to like get at get commit Yeah, we're just going to push the master on this on this new In on this new grapple even though it's not best practice but for demo purposes Yeah, so now he has pushed it up to get up if you saw he he did a gate push. So now Yeah, the the latest comments here, so this is the comment that we that we made that then we move it we moved it So he's going to make a pull request So it's yeah, it's just clicking a button called pull request If you have never put a main a poor egg pull request before this might be quite scary So I remember my first pull request like I Was quite scared right because you feel like your code is going to be judged but don't you don't need to be scared Yeah, we won't judge you It's a promise Yeah, so we can just create so that did that was reviewing the changes. He clicked the button to To create a PR. So this is where we type a description So you basically like we have a template here. You just need to follow it fill out the sections and then just make the PR Yes, we show off your coolness, right show off your coolness. Okay, so this is our PR text So you just gonna click create full request and it will appear on any of us. So now There that there's that is our PR So this is the this demo shows that is actually very simple to contribute to any us mods So we hope that more people contribute to any us mods because right now we our team is three people We have so many things that you want to do which Leica will continue to open about Yeah, we are not releasing it. Yes Yeah, so I handed it back to Leica to talk about our future plans Okay, really fast because I realized that we're slightly ahead of Sorry behind time. Yeah, so the future of any us mods is sort of influx Like we've been wanting to do this for like one year's two years Yeah, but it's taken a long time because well, three of us currently do much especially to our full-time employees and one is Overloading and doing a lot random other modules. So, you know Very hard modules very random, but very hard for this. Yes. And so, you know, we want to add accounts We want to add syncing we want to release our GraphQL API and we want to like slightly change how things work with accounts and sort of integrate planners with timetable and You know at like a today page All sorts of rental ideas that we want to do but we just don't have the time. So yeah, these are Ideas that we want to encourage new contributors to maybe open up issues and see how you guys would want to see any of us mods develop Yeah, and yeah, any questions we can only take like two. Yes So there's Unfortunate reality is no we are not using GraphQL API yet, but we already have the PR It's just like waiting for reviews Waiting for Approval to merge it for a year Okay, next question Yeah, so previously we had Normal repostructure. It's just that with a mono structure with a mono repostructure It's easier for contributors to just like clone and make changes across the board especially since There's really not as much code as compared to a real company. So The folders are Quite small although we have many but they are quite small. So this is sort of to like make Changes more rapidly and to encourage contributions. Yeah So we just have a cron job that basically runs on a computer every day or so No, it's like every hour. Oh, yeah, I mean it changes depending on your needs We just put it on a computer Yeah, it's just on our one dot and just mouse.com server. So that the server holds the yeah It's just a script lying there and then we just run it Yeah Yeah, we do we do it access your data, but we need to do some pre-pre-processing on and like we want to generate We basically do some like value add on top of the NUS raw data Okay, actually, there's more than that. So first is that NUS data is taught It comes in separate API endpoints, which are quite difficult to Come to consume so we basically download everything then we process things like we combine some very a lot of duplicate records We've generate prerequisites generate What else we generate? I mean we do a bunch of stuff Yeah Yes What's ETL Run time is about five minutes on a computer, but if you are talking about time Yeah, but if you are talking about time spent writing it maybe like two weeks, I think Yeah, a lot of time All right, I think there's other questions that we have if you have more questions and ask them at the end of this session Yeah, thank you