 Hi everyone, my name is James. Some of you may know me over here. I spoke at Hackware yesterday. Every year I try to give like at least one talk in Singapore. I currently work in the US states for the last like six years. Currently I'm at Roblox which is well it's like a game platform but you know you don't really compete with JavaScript but it's a game platform where you build games and you play games and there's like a micro payment system and many kids actually build games using lower script. Yeah, this is Jerry. He works in Singapore and he's sorry? No, we go carry on. I'm trying to connect to the Wi-Fi. Yeah, because we have a demo that actually uses the online stuff. Yeah, so this is actually a project that started off a few years ago. I decided that I would rather like you know like release it as soon as I can. I have to make a commitment to release otherwise it will sit at the bottom of your project's bin for eternity. Yeah, so what is this? Snapcache is basically an open source alternative to Firebase. So how many of you actually use Firebase over here? Oh nice, quite a few of you. I was worried that no one would understand what I'm doing, right? So I used Firebase many years ago ever since they were a startup. There were some hackathons and they were probably the first real-time database before parts came out. I'm not so sure which came first. After that, Firebase was bought by Google and they are now part of Google Cloud. In my previous company, we actually use Firebase in production. So we have quite a bit of experience with it. Yeah, so yeah, there are two other people on my team, Frank and Theron. Theron couldn't be here today. He had national service commitments. Yeah, so history, right? The Firebase is on Google Cloud and the story is like this, right? Usually I wouldn't rewrite something that already exists but when we were running, using Firebase in production, what happens is that Friday evening, I'm not sure who in Google decided to push and update their service and it broke our service with like 10,000 users using it. So naturally being one of the few backend people on the team, we had to hack out some fix and we fixed it over the weekend and on Monday morning, they reverted whatever they changed on Friday. So it broke again. So naturally like our higher ups were very happy and it's not really our fault but we just decided, okay, it's time to DIY and we want to in-house this core component. Why? Because Firebase is like a real-time database, a real-time communication system that we use it to support our applications. So the key components are essentially Redis and Snapcache. How we started off as, how we started off doing this is primarily when we were looking around for Firebase made backend source code which doesn't exist but however, there was a repository created by an ex Firebase engineer which created like a stub, like a stub, how would I say, a stub library which allows you to test Firebase offline. However, it doesn't have persistence and it doesn't scale. It's really just for you to do like a test, part of a test driver or something. And for our architecture, the current architecture, you usually use Firebase as part of Google Cloud. Typically, you have your app, you have a Firebase SDK, you could use the JavaScript SDK or the Java SDK. The SDK is really efficient at communicating with the backend because it uses web sockets, not your REST HTTP stuff. And when we wrote our own Firebase backend, we essentially want to replace it. So what do we do? We actually have a Node.js server which runs Snapcache. We are able to reuse the web socket protocol. In the past, when it was closed source, the web socket protocol was kind of like obfuscated but after the open source, the SDK, understanding the protocol was much easier. We use Redis as the backend persistence storage primarily because we are able to scale. I'll explain more about scaling later. Setting this up is really easy. I just like released it publicly earlier today. And you just need to start with Redis, start up the Snapcache instance. You can run multiple but over here, let's assume you run one. Jerry will give a demo later with the demo application. So it's really just these two components. One of the key differences, if you have used Firebase before, you would have known this thing called the permissions setting. The rules. Rules basically define which node in your Firebase database is a particular connection able to read or write or both. On Snapcache, we don't have any nice user interface. So you actually modify part of the JavaScript on the server side. So how do we scale? We actually use this in production in my previous company. We have like 10,000 concurrent users. We simply add more nodes to the Redis cluster and we have just launched more instances. Snapcache is designed to scale horizontally, which means that you just add more instances to it as you scale. We are primarily bound by the number of nodes. The number of subscriptions because Firebase, you are able to like subscribe to a particular data node. And when there are changes on it, your client will receive those changes. The number of subscribe nodes and the Redis write throughput because Redis is like single trailer and you can only like, how would I say have like only one instance in the Redis cluster would be the master of all the writes going to it? So our production architecture is not too complicated. We simply horizontally scale the Snapcache instances, which is just more node.js instances of it. And for Redis, we just add more nodes to the cluster. This primarily helps with the read throughput because usually for most real-time databases, you have one write, but then the read is distributed to multiple clients. For example, if you have a chat application, when there are 10 people in the chat, you send one message to it, nine other people need to receive it. So for future work, we know that this is kind of like a really like simple, how would I say? Not really simple, it's more like we did it primarily for our own use. There is no dashboard. So unlike Firebase, Google Cloud, there's no like nice dashboard for you to see metrics. Some dashboard will be nice. We definitely need test cases and documentation, which usually the last few things most people have. Yeah. And one thing I do know that is that we do not currently do not support sharding. So if let's say you have multiple Redis clusters, you would want to enable sharding to basically increase your write throughput if let's say you have a really large pool of users, maybe 100,000 or 1 million depends. Yeah. So now it's time for the demo. Yeah. I'll pass over to Jerry. All right. Thanks, James. So I'm Jerry. I'm a developer, full stack developer. I actually contribute actively to the local startup scene. I take quite a lot of hackathons in Singapore. Some of you may have seen me before. Yeah. So basically today, I'm going to talk about like, okay, if you're sitting here, maybe you're a front-end developer, maybe wondering how am I going to integrate or use this on a front-end, right? How do I call the Firebase or how do I use the Firebase SDK and like call or use Snapcache? So this is any like React developers here, just like show hands. Okay. Cool. Good. So I know that basically, yeah, I just did the tutorial like last night or something. So this is the demo, demo app that I did in React. Yeah. So to just show you guys how, basically how you do it in your, if you're a React developer. I'm sorry, I'm not a ViewJS developer, so I didn't know how to view. Okay. So basically, AppJS. All right. You have your props. You put your state, right? Simple boy plate code. Okay. Actually, it's very simple, right? It's like all your five days as SDK all works out of the box. The only thing you need to change is like the configuration and point or where you point your data of these two. You have your subscriber. You subscribe to, you have a value subscriber. There's also the, what's the other one where you have like more, but these are the value subscriber on, you mount it, you edit when, you know, on component amount, just remember to amount it. And then you have a value callback basically when something changes, let's say, so this is a do-do app, right? Yeah. So we have a list of tasks and then somebody, and you have many people that's trying to add things to this task, do this list of tasks. Okay. So what happens is somebody else adds a task. It's actually subscribing to the entire collection. So the entire collection is actually returned in the snapshot. Just iterate through your snapshots and then you set state. This is where you set the state, right? And then your react component will be rendered. So pretty basic and very simple. Yeah. Okay. So the only thing you need to do is basically, basically we are replacing five days with like our own, it's like five days server, right? So basically you just change the database URL to the whatever IP address or whatever pod that you mounted your snap cache on. Okay. The rest of the stuff is, yeah, you can implement it if you want. Yeah. Okay. So this is like, okay, the last piece, just make sure, so you just make sure you have the Firebase SDK as your dependency, you download it, install it. It's the usual stuff, right? It's on like the Firebase tutorials. Initialize the app with the config, with the new config. And the last part is very important. Like, okay, so this is like where you put it on your database API. And they just remember that you have to make sure it's a single turn. Like you don't initialize it multiple times. That's what a try-catch that is there for. Basically Firebase doesn't like it when you try and initialize multiple times. Yeah. And you got it. So let's see how this works. Okay. Enough code. Hold on. Right. Okay. So, right. So you're going to use like two separate browsers to like simulate two different clients. Okay. And all right, let's try and add some tasks. Anyone volunteer some tasks? Like, so one of the tasks that I was doing just before I left for this talk was writing a readme for this repo. If you go and look at the comments for our repo, the readme was just updated like at 5.30 today. Okay. Yeah. Okay. It pops out on the other screen. Yeah, it works. This real-time database. Okay. So, yeah. Essentially, yeah, this is like what we're doing. Like you have sockets, like sockets subscribing to changes on a database and then you have changes on one, it shows up on the other, all the other clients. Yeah. And then maybe another one like arranging something for Valentine's Day. Okay. Yeah. So, I think that's it. Right. Okay. Oh, I don't know what to spell this. Yeah. So, that's the demo. Yeah. Okay. So, any questions? So, just to confirm, do you use the actual security rules in the same format that Firebase uses? Yeah, that's correct. And two, how do you handle the off, the fact that you can do authentication with Firebase that's tied to the security rules? Yeah. So, this was built in a time where, sorry, this was built in a time where we weren't using OAuth. I'm not sure what new authentication methods they used. In the past, we would actually use like its own internal keys. Yeah. So, there is no OAuth support and moments. Yeah. Any other questions? Yes. This one is for the Firebase web plan SDK, right? The real-time database? Yes. Just as one is also for the back end part, the Firebase admin. So, will it also work with Firebase admin? I haven't really used it for quite a while, but it doesn't really have like the new stuff that Firebase introduced. Yeah. This is purely for the real-time database. I think the SDK we're using, what version was it? Five more, right? There's a version basically you and me. Okay. Right. Yeah, there's a version that we tested until. I'm not sure if the newer stuff actually works. Yeah. But usually, I think what we like to use this for, for example, like one of the use cases that I actually want to try to use this for home automation, because previously, my home automation would run off Firebase and would like go over the internet and I find it's quite kind of silly. So, if I wanted to like bring it down into my own like, you know, run it locally, right? There isn't any other way except using this. Yeah. And if let's say you want to give a demo that purely runs on your laptop without the internet, you know, this would be another way as well. It's great for hackathons. Yeah. Any other questions? That's the repo. Oh, yeah. The repo is... Yeah, this is the repo here. Yeah. You can just go to snapcast.js.org. Yeah. Welcome for contributors. Yeah. Yeah. Anyone like, you know, you still use Firebase and you want to go check this out. There's many things that are not really implemented, you know, completely. So, you know, feel free to contribute. Yeah. Yeah. We are not packaging this up. There's an MPM package. Yeah. It's not an MPM package yet. So, yeah, you have to fork this, but we'll get there someday. Okay. Thank you. Thank you.