 thing now. Cool. Yeah, that way this will be recording, explaining the walking through the whole R covers app. Gosh, the record, this thing makes me so nervous. But yeah, so how are you feeling? Yeah, I just discovered the in Chrome debugger for the server. Nice. Yeah. That's fun. Yeah, I mean, it's, you know, the sort of the front end, you know, just kind of use some JavaScript to pull some markup together. You know, that stuff makes sense. You know, your sort of API calls for, you know, for adding and removing stuff more or less makes sense. I haven't worked with Mongo before, but you know, I understand those sort of, you know, JSON based database, you know, so for kind of like data cleanup tasks, you know, sort of, you know, running a query to kind of find everything, you know, so that we can make a list and that kind of stuff, you know, they get into that. Totally. And I know that there's some cleanup stuff, you know, kind of in the queue, you know, related to, you know, issues that URLs that may not have gotten checked and whatnot. So that's... Oh, yeah. Yeah, that's getting big. But first, let's say hi, John. I'm in a place, unfortunately, that I can't talk in and around later too much, so I... Totally fine. Yeah, unfortunately. But hey. Before we get started, do you have access to the repo? I don't know. I can't... No, I don't. Okay. So I'm going to... And you're... Where are you at in terms of devs? You're a solid dev, I remember correctly. Is that it again? You are... Are you familiar with working on front-end dev projects? I have some experience. I do know... I know JavaScript fairly well. I know fairly well. I don't have a huge range of projects, but you know, I have a decent techno background, and I think I can onboard pretty quickly. I'm perfect. Okay, cool. So we'll... I know the basics of Angular or whatever. What is... You're using a Meteor... It's Meteor and React, yeah. Okay, I think I can pick it up quickly though. Cool. So let's get you invited to the repo, and then if you end up being like, I don't want to contribute to this, then you don't have to. Well, I'll get you an invite right now. As we had mentioned before, the hope is to open this up relatively soon, but for now we're closed still, which is something we really got to rectify. Yeah. But yeah. I'm sure... The collaborators. Okay, and what's your GitHub panel? It's exactly the same, I think, as my user in here. Yeah, it is. It's definitely the same. John G A N C. Cool. Make sure I spelled that right so I don't invite some stranger. Okay. So I'm going to copy the URL. Invite you into this chat and then we'll get moving. So yeah, take a second to sort of find that, but I'm going to share my screen and just walk through a bunch of the code base today. So feel free to... Either of you feel free to stop me at any point. We can do this. Hopefully we'll just do this in like 25 minutes and be able to sort of get it over. That sound good, John? Yeah. Sounds great. Can I clone the repo real fast? Yeah, go for it if you want. Yeah, that way you can follow along. That's probably a good idea. Yeah, and while we wait for that, Kevin, the ESLint thing, awesome. Such a smart move. Good. Code convention is silly, silly smart. Yeah, particularly coming out, I was telling you coming from GoFont Land where like when we go, like everything just has to be, it'd be exactly like Go. Right. And so it's really nice to have that applied even in JS Land. Yeah. I won't put a stickler kind of coder, but having something that just sort of, you know, does that checking for you as you go? And we'll also, you know, just let you remember different syntax conventions for different projects. Cause like, you know, like, I mean, Drupal has a set of JavaScript rules and they want a space between the function name and the parentheses, you know, just having the editor just tell me, you know, it's a, yeah. Well, and it'll already clean up a good issue where like Dan's editor was doing some like crazy line break thing that made GitHub really angry. That's just producing all that red. Right. And it'll cure my tab addiction. I'm a big tabs guy, but you know, yeah. I got it. Cool. Cool. Cool. Cool. Cool. Cool. Cool. Cool. Cool. Cool. Project. Can you guys see this? Okay. Yeah. Let's get some code up and see if that's big enough. Is that visible? Or should I make it bigger? No, that's fine. I can see it. Cool. So let's start. So top level folder just so that we can get a feel here. God. Would you mind meeting your mic while you're not talking that way? There's just a little rustling noise. There it is. Perfect. Thanks, man. Yeah. So we've got a couple of things, client imports, node modules, obviously private and server. Just to give you a really quick review overview of all this stuff. Private is stuff that does not get exposed anywhere for the actual server use. So this is where we have our import and export scripts for getting stuff into and out of the DB. Kevin's added these wonderful contributing guidelines. So read me as well here or whatever. Our server stuff is that's, so all the code in this code base is because we're using Meteor, everything gets, a lot of files get run on both the client side and the server side. And so that's a huge thing to sort of get used to. It used to be called isomorphic JavaScript. And now it's referred to as universal JavaScript, you know, whatever. Terminology. Yeah. I remember trying to spend a little while trying to figure out what the hell that is. It's just JS that runs in two places. And so, yeah, server code is obviously just only ever going to run on the server. And so this is sort of the main, if you're, if you're used to thinking in the traditional server client model, and you want to start by understanding the server, this is the best place to start. This is sort of where we do all of our imports of like our API's folder, which will show you in a sec, but like this is like where everything boots from. This is one of this client of this code base. When you start like with a no, when you don't have any Mongo instances, this thing automatically creates the root user, which is this function and a whole bunch of stuff. And so like the first time you spin this thing up, it creates a root user and just logs out the actual username and password and expects that you'll change that for the root user on your local machine. So if you can get that up and running, awesome. We should let this too. So as I was saying to Kevin, we're in the process of linting everything. So like I think currently this is done with, oh, this is the space is cool. But we're going to try and sort of observe the code conventions of linting that are in the package.json. We should probably get a read me written up on how to set up your linter, but that can be a long term. Okay, cool. So moving on, we've got client imports. So the client is there by the way, is there a particular linter? Is it ESLint? Yeah. ESLint with the Babel. So all of this, all the code that you'll see is being written with ES2015 syntax or is that what it is? Yeah, that's the 2015 spec. And so, yeah, this thing that I'm showing you right now actually has the ESLint config and this is included in the project. And so you can. Yeah, it'll all come down with no, the only trick is just whatever editor you're using. Yeah, I had a bit of a pain setting it up. I actually had to like edit my, where is it my user settings for this ESLint thing. And I ended up just cheating and like pointing it directly at my. Yeah. I mean, whatever works. Yeah. And it went off without a hitch. I did make one addition, Kevin, to the, to your configuration. Which one was it? It was no, the no bit wise flag. I use operators because sometimes we do stuff on the really efficient way. That's the only thing I changed. Cool. So getting back to the overview. The client. Question. So yeah. I'm very happy to listen to you guys talk to us, but is there, like you gave people very short notice. I wonder if there might have been some other people who would might have wanted to. Totally. I should have mentioned this, John. We're recording this. So we will be walk, we will all post this up once it's done. And we'll definitely make it so that everyone can see it. Yeah. And yes, this was very short notice because this is something that Kevin and I have been meaning to do for a while. And we will obviously want to just have our meetings in the open, but we didn't really sort of take a lot of time to plan this. I just, it just occurred to me. Yeah. No, it's a great call. And so yeah, that's why we're recording it. I apologize everybody for who's watching this recording and watching us to make all this, but yeah. So yeah, moving on. If you don't mind, we're going to go to the client folder. Client is where just stuff that only loads on the client sides. Client is where all of our style sheets get dragged in. So we have this SDSS folder, which all gets automatically compiled down. I'm actually not completely sure how that works, but I don't care at this point. So, and this also contains our main HTML, which is quite empty as you can see, because we populate everything from the app. And then main is the actual sort of init function that gets called on the client side. So really all we do in this folder, and this is a great place to kick off is we pull in our routes. We use react-router to do our routing. And then we just call render, which looks at this root element in the main, which is called app. And we find that element and then we just populate that with our react app. So it's pretty straightforward. I think that should be fixed. I'm putting that up. Yeah. So that's the client folder. Lots, if you have to edit style sheets, it's always in there. And then a lot of the real action happens in this imports folder. It's labeled imports because it's imported on both the client and the server. And this sort of subset or sub directory is actually fairly important. So we'll go through all of these one at a time. All of the model layer stuff that you would traditionally think of in an NBC app is in the API folder. This is where we define all of our models, where we define all of our methods on models. And this is sort of how everything gets set up. I'll dive into one of those files in a second. But Libs is where I've had to sort of jam a couple of higher level libraries that weren't written using the import syntax into this Libs folder. Eventually, I'd like to see this folder actually go away. But for now, we're using it in place of proper imports for a few things that I've just needed to get into the code case, namely this normalized URL and S3 upload functions, both of which I'd written in other places and just needed to get in. So one day, we'll delete the Libs folder and we'll have one less folder to worry about. Startup is a folder to just let just gets loaded on startup, sort of configures a couple of packages and gets sets of default stuff. Most importantly, we have this client routes folder file, which is where our actual, you can see the actual routing layer here. This is where we do all of our URL stuff. So this defines sort of all of the routes for the app, and that's actually how the app interprets everything because it loads on the client side. And then the other, this folder is obviously one of the biggest ones. This is all of our React UI. So we split up between components and containers, which I'll explain. And we also have a selectors folder, which actually should probably be elevated to the imports directory. But I'll end up walking through this components and containers in detail once we talk about the UI. But first, I want to go back to the API stuff. Is this feeling too fast, too slow? How are you guys doing this so far? Good to think. I have to. I can look at it again anyway, right? Totally. Awesome. Okay. So I'm going to use an example. Let's use agencies now. Let's use URLs as an example. Make sure that my syntax is set to Babel. And so this is a classic, like URLs is obviously in our inside our app. Our URL is a representation of a URL for harvesting. And so this is all of the stuff that defines how to work on and with a URL. We'll see this media publish stuff, which we'll get to in a second. This is all running inside of a code block. It says media is server. So all of this only this code inside of here only runs on the server. This is the first export function, which is like we zs 2016 syntax. So 15 syntax. So exporting is how you sort of elevate. If you're used to note, it's sort of the module dot exports function. That's sort of the same thing. This is all just definition for making a new URL that has a blank set of values that we need. And then this media methods is definition of stuff that you can do to URLs. So this URLs dot bulk insert URLs dot insert. That's that priority. These are all actual methods. Meteor asks that you sort of namespace them with this dot thing. I'm not a massive fan of that, but it is what it is. So we roll along with that. And so this is like, this is where you can see all of the actual sort of actions that the app can take. So this is the URL locking mechanism when somebody checks out a URL inside of the app. It's run by this URLs dot lock. And then dot unlock is obviously the converse of that. And then we've got merge and unmerge, which actually should be deleted because that feature is deprecated. So private is making a URL private. And then yeah, so like this is sort of the general gist and you'll see this pattern time and time again, where we do a whole bunch of these publish functions, which again, I'll talk about in a second. It's not edit our code. And then we define methods. And then the biggest thing that each of these files do is they export a Mongo collection. And so this is sort of like the every single one of these API files, the sort of final like output of each file is this Mongo collection of the actual URLs. And so yeah, it's important to realize that in media or Mongo collections operate on both the client and the server. So we have that in both ends. And that's just worth thinking about. On the client side, it's actually simulated. There's like a Mongo simulation thing that emulates the effects of doing Mongo queries. So you'll see some of that if you ever end up running into exceptions on the client side. It'll be like, you know, simulation exception, which is basically saying, hey, we tried to pretend this query was being run on the client side, but it didn't work. One of the huge advantages of the media framework is it does a ton of client side caching. And so it actually sort of pretends that there is a Mongo database inside of the user's browser. I think it leverages local storage. I haven't actually bothered to check how it does that, but I'm pretty sure that's how it accomplishes tasks. It also makes the synchronization sort of between for updates and changes really, really nice. Okay. A question. So this new Mongo collections URLs, that's a collection step, it's like a table of URLs, it's not a URL, right? Exactly. Exactly. So Mongo uses the word collection in place of table. It's also worth noting that in Mongo collections can contain any number of nesting of objects so you can put objects inside of objects instead of your usual sort of Postgres style. And that's one of the big sort of Mongo things. Yeah. So, and while we're at it, let's talk about this completely separate subject, which is the Meteor Publish function. This is actually how we define stuff that feeds that the client can subscribe to. So all of Meteor is based on a PubSub model. So users, when they navigate to certain parts of the app, they actually end up subscribing to changes on specific feeds. And this is where we actually publish those feeds that they can sort of see. And so this is important because this is where the server actually determines what is permitted to be subscribed to. So if we wanted to sort of keep something confidential, keep something server-side, this is actually where we would define it. If you filter things out of the results, the return results here, so like we said return, I can actually show that in the users tab because we don't expose, like here we're publishing, just give me a sec, here we're publishing a list of users, but obviously we don't want to publish the passwords or anything. So we publish with a fields record inside of the users, which actually scopes down the number of things that can be returned when someone subscribes to the user's list feed. And I'll show you where subscriptions happen. That happens at the UI layer inside of containers, but just so we to get a sort of initial lay of the land metered out publish is the big way that the server sort of lets people subscribe to these change feeds. Where was our URLs? So yeah, you'll notice we publish a lot of different URL feeds. These are all the different phases inside of the app. And in here you'll notice that we're calling to this phase selector thing. It's a very common pattern that I've sort of inherited from the Redux framework to try and do a computation on state inside of their own, like to isolate that stuff. So this phase selector function gets defined in this selector's thing that I told you should probably get elevated, which is UI selectors URL. And so here we actually define this is the phase selector function and we talk about like how to actually select from the total canonical list of URLs what a crawlable URL looks like. So this is saying if it's a URL that has a crawlable field and that crawlable field is a Boolean value set to true, that's a filter that would return yes for that field. So the research phase, same thing. So it started saying, hey, we got to have the research done sort of checked out as false because we want to see something inside of the research phase. If the URL is marked as crawlable as false and merged as undefined, because this is carryover from old deprecated code. Yeah, this is sort of how we do it. But you'll see this all over the place. Like we basically try to centralize the phase selector function so that anytime you're looking for the merged URLs list, you just call this phase selector thing and that gives you back the actual thing to feed to Mongo. So over here, you can see we're looking for a phase selector of harvest type to represent the harvest feed. There's lots of reading on the Mongo stuff. A lot of this is probably starting to come a little quick. Like how to do this like URLs.find method is a Mongo method. So you can read about that on the MongoDB documentation. You'll see stuff like find, find one. You'll see stuff like update. That's inside of our methods. We've got URLs, we've got insert, update. These are all database calls and they're sort of wrapped inside of these checks. So if we want to insert a URL, this check is a validation function. Obviously we need to do a little more validation work. And then we sort of normalize the URL. We try to make sure it doesn't exist already in the database and then make sure that there's a user logged in and then if all that passes, we sort of do the insert. So it's sort of the general feel. You'll see this like URLs. This is working with Mongo stuff. And all of these methods are sort of the only way that you should ever be interacting with the database from the UI layer. Okay, quick pause. How are we feeling so far? Yeah, pretty good. One thing that I have found, John, there is a Mongo command line interface. This shell you can get into from wherever machine you're running the app. And it's kind of, if you're used to relational databases, it just has its own sort of query language which feels more like JavaScript than SQL, essentially. But you can at the very least, it's been useful because I have a local copy of the app running just a few URLs and a few users. So I can just kind of concat them all out to the screen and just sort of see what's in the back end. Yeah, it's probably going to do this. The stuff is looking right. Yes. And then I actually did end up using that shell a lot because currently we're sort of lacking certain functionalities in the app that I end up using the shell to just manually manipulate the database. So yeah, not the greatest, but it's sort of how we're getting by right now. And so, yeah, very good points. Okay, so that's basically the API folder. It's worth sort of like sitting down and reading through all of the API stuff if you're going to sort of manipulate anything, reading hard of the app. The API folder is where it's all made and broken. But obviously if you want to manipulate stuff, you've got to be able to see it. So that's where our UI folder comes in. And this is where everything gets super react.js oriented. So anything inside the UI folder, if it's sort of like got some syntax that you're confused by or it looks strange and mutant, because let's give you a feel for some of this container is a good place to start. Let's start with the app one. We have like this is react code like Dijour where you see these HTML tags actually mixed in with JavaScript. So, yeah, if you want to read more about how the components or the UI folder works that I'd recommend reading a ton about react. It's sort of a great way to get acquainted with this code base. If any of this feels foreign or scary or weird, feel free to just leave it until it doesn't. There's a lot to learn about sort of the react way of doing things. I think it's like hands down the most functional way to work on front-end stuff. You can see how some of these components end up tying together. I think it's a really fun framework to work in. So it's run through some kind of preprocessor? Does that would react this? Oh, yeah, yeah. So well, okay. So actually it's worth doing a 10,000 foot view of react right now. So react emerged. It's a Facebook technology. It came out as a effort to improve on the DOM rendering speed. And it also was trying to solve a lot of these issues of large scale front-end apps. And so react introduces two really big concepts. They introduce the concept of a component, which is sort of their fundamental unit. Components are intended to be composed together to form stuff. So a good example of components are, you know, we have like a modal component, which is like literally a modal dialogue. And then we've got like a spinner, which is our loading. So like literally right now, our spinner is just literally the text loading in a div, which probably could be improved. But that's where else that you showed earlier, they essentially map to components, right? And then, you know, it's kind of top level components, and called sort of more atomic components to do their thing. Yeah. And the most important thing to have realized about the way react works is it is expected that the programmer can call render a lot. This is most front-end frameworks you tend to think about rendering as a fairly expensive function and like very time consuming. And so you try to sort of minimize the number of times you call it. The way that react does this is they've actually sort of modified this framework. I apologize if your hearing is siren. What did you do? I can't tell you. We're recording the call. But what's it saying? Okay, so calling render is there's actually a diffing algorithm underneath the hood that avoids touching the DOM. And so when you call render in react, it produces this giant in JavaScript object tree that represents the way the DOM should look. And then it looks at the way the DOM actually looks. It looks at the way the DOM is actually sort of structured at that point in time. And instead of changing the entire DOM, which is really slow, it only makes modifications to the... It actually produces a diff of what the DOM is and what the new render function wants. And it only applies the diff. And it is just brilliant in the way that it gets around the slowness of the DOM. That's the way that it used to work. There is actually a new algorithm in react 15, I think, called fibers. And so it actually works a little different than that. But it's worth just thinking about it that way. So you'll see render get called a ton because render can actually end up being a no-up. You can call it and the diff that gets produced could actually end up being nothing and it'll just do nothing. Which is a very desirable property for people doing UI work because now you can call render as many times as you need and you no longer have to worry about sort of synchronizing the state at render time. It just says, no, let's always just determine what state is and call render. And so you see that sort of loop happening time and time again. And state is very, very, very explicitly handled inside of react. We only really have two ways of plumbing changes to a component into a component and that's via something called props, which are properties or state, which contains state within the component that lets you modify stuff like, hey, is this checkbox checked? That's a state. But you could also just as easily send a checked property to that component and have that sort of plumbed in and represented. I'll show you examples of that. Let's start with, okay. And the final thing, on top of react, a lot of frameworks. React is only a view layer framework, so it only deals with the notion of components and the notion of rendering stuff to the DOM. React doesn't have anything to say about the way you store your information or how you even handle sort of updating or changing actions throughout the app. So that has, Meteor does have some opinions about that, however. And so those opinions end up getting pulled in as in this distinction between containers and components. And so a component is, it can be thought about as a DOM. Like, it doesn't have any knowledge about sort of like how to talk to the underlying Mongo sort of data. So if we look at all these components, they're just like really, they should be incredibly dry, right? This like every, all components always have to define only one function called render. And it takes no arguments and has to return this sort of like JSX syntax stuff. You cannot return undefined from a render function. Well, you can, there are some exceptions, but anyways. The point here is everything in the components folder is, these are really dumb components. They shouldn't sort of like do anything too complicated. They should basically just be these sort of like representation of, you know, how to work with some, of how to represent something on the screen. So an agency item is an item inside of a list. So it's a table row that just shows this is literally how do we represent one? How do we represent a single agency in a list? Forms are what you'd expect. Forms are a little more interesting because you actually have to handle interaction. And so we handle interaction by defining these custom functions like handle submit, which is the actual submit call. And we hook that up via this sort of on submit right here. So we'd say we have to do some funky binding nonsense because of the way that JavaScript works, but suffices to say inside of our returned final component, we have this on submit, which connects to the handle submit function, which is here. The first thing we do is prevent default because we don't want to actually submit the HTML form. We're going to do it all react way. And then here you're seeing that this component calls a passed in property, which is a function, and the property that's passed in is called on submit. And then we hand up the data from this component. And basically what this is saying is I'm a really dumb component. All I do is when somebody clicks the submit button, I wrap up all my data, which is contained in state. And I pass it along to whatever that function that I was passed in the first place. So if you look down in the prop types definition, we see that this is sort of a, prop types is a way of listing everything that this component expects to be passed. So you'll notice here we have the on submit. And it's saying this is a prop type of function and it's required. So if you don't give me a prop called on submit and it's not a function or if you give me something in on submit that isn't a function, I am going to crash. So these checks only happen at Dev time at runtime. These are all taken out. And so the program will just actually, it'll try to just roll with whatever you give it. But while we're working at Dev time, this thing, if you were to like try to do like agency form and not pass it and you do on submit equals, you know, and we just actually explicitly passed undefined in here, this would crash the program. But if we passed in a function and we actually did something with it, that would be valid. Or at least it would pass validation because it doesn't do anything. It wouldn't work very well. It's parsed by the other react preprocessor. Exactly. Yeah. And so just so you know, like all of these calls to react preprocessor will render this out as these are all function calls in react land. So this when it gets actually gets preprocessed to what is it react dot dom dot div a function call. And then it'll have a product you'll get passed. Yeah, basically so that it actually creates this giant JavaScript object. And that ends up being the return value. So it'll be a call to form and then the form would get passed all of this stuff. And that's all handled by the preprocessor. We don't ever touch that stuff. Because it ends up getting way too messy. But yeah. So following so far, how are we feeling? Yeah, I'm okay. I think I'm doing as well as I can not have seen react before. Yeah. It's a lot to take into once. And it's going to get just a touch more complicated. But so the biggest thing that those last sort of thing to explain is how do we actually connect Mongo data or sort of like state into how do we plumb that into the app? And that's where containers jump in. And so a container is just a component, but it's wrapped in a special function that connects data from the database to the react app itself. And so let's start with the app. So we and we this is pretty important distinction, right? The containers are much smarter than the components. So it's expected that if you look at the top level of like if we look at our router on the client. Oh yeah, it is in startup. If we look at our routes, almost all of these routes are importing containers. So this is all coming from the UI containers folder, right? Like app. This is how app gets dragged into the actual routing. So the top level route. So which is just like literally the slash, which is like the empty bit that gets that gets past the app container. And so we can look at the app container here. And this is where it actually sort of it's got this special function called create container. And this is this is where we take a component called app, which we've defined here. This is defined in a special way. This is called a stateless function component. That basically, this is the exact same thing as class app extends react dot component with a render func. But instead, if you don't have any state and you don't have any handlers, you can actually just make a function and return it. And react looks at this as the same thing as just like one call to render. So we have this one, we have this function or this component app. And we're now going to pass it into this create container function here. And what we're going to do is this create container function is the place where we actually get to talk to the database and bring stuff into our react components. So whatever we return from here, this from this create container gets passed to the app as the props, the fact as the props stuff. So let's give you a better example than that. So dashboard is actually a great start. So like in dashboards, oh, it's also functionless. Sorry, I've been refactoring a lot of stuff. I need to find an easier. Here we go. Login. Let's use login because it looks a little more like our classic. We've got the handlers. These are all the places where we actually handle the functions. And you'll notice that the containers like handle submit function actually does stuff. Like this is actually a call to Meteor's login with password function. The containers do a lot more sort of like talking to the actual backend and they're expected to be the sort of intelligent places where the more complicated stuff happens. So if we look at this create container function. So what we're doing here is we're passing in the login component, which we declared here, which is a much more classic react component. And the first thing, so you can see that logins, prop types expects a prop of current user and it expects it to be an object. And so the question is how do we get the current user into the login screen? And so what the current user is for is it's a check to see if someone visits login and they're already logged in, right? If the current user already exists, we want to redirect them. And so that's where this says if user is logged in. So inside of our render function, if someone, if we ever actually get a problem called current user, we push them to their own profile so that they don't actually see the login screen if they're already logged in. And that's how that works. And then we return the spinner from that because we don't actually want to render out the full login screen ever in that case. So the way we actually do the connection is here in this create container function, we have this meteor dot user. And this is where we pull the user out of Meteor and then we pass it to the component as a prop. So you notice this props, this is some special fancy syntax in ES 2015. All these things is this dot props is currently an object. Is this basically, this is what is here. So we're returning props. So props is an object. And then this thing that you're seeing here is called destructuring. So we're just saying we expect props is an object that looks like this and has a value that's an object. This would be like user name and all this stuff would be in here, right? Is the actual data. So whenever you see this constant current user, this is called destructuring. So we're just pulling the current user value out of the props objects. And so we're doing the same thing with state here. So we have a state object and we're pulling user name password and error. These are the actual values for the fields. And so you'll see the form fields here. And that syntax, let's do it based on the property name, I think, right? Yes, exactly. Yeah. So basically, you don't have to enumerate every single property. They don't have to be in the exact same order as they're coming out of the return function. Exactly. Yeah. And not only that, you can assign defaults to these things. So you can say username equals default username. And this is valid as well. Yeah. So which is kind of nice. Just to be able to deal with that. You used to run into this constant problem right in JavaScript where you'd go, okay, you know, fair username equals this.state.username, right? This code ended up being very terse. And so that's where we came up with it. They came up with the destructuring. Yeah. Does that make sense? Yeah. And to your point, Kevin, it's also worth noting that you don't have to pull everything out. You can just pull the things you want out. Yeah. I figured that out through positive thinking, you know, that I made a change and it still worked. I'm like, all right, this is, you know, there's a kind of a similar sort of list syntax that you come across in PHP, but it's not as flexible as this. Totally. Totally. I noticed that before too. Because I think it's been adopted pretty quickly. Because like you said, I think it's very useful. Yeah. It cuts down on a lot of the, like, irritating boilerplate of JS. If you want to learn more about this stuff, it's Babel.js.io. That's the place to check. Definitely, definitely worth a read. That'll explain everything about ES 2015 syntax. And there's some weird stuff, right? Like this, like destructuring is a bit strange. The defaults, because now you can hand defaults to function. So I can actually do name equals stuff. That's permitted. The default thing I did not know. Yeah. And we can do this kind of stuff where we can do values as keys by using what looks like the array syntax. But what this is saying is take whatever the variable name is and make its value the key for this object. So there's a whole bunch of little, and then we also have arrow functions. I don't know if you guys have seen those. Yeah, that was the first thing actually that I saw that was unfamiliar. Yeah. This is, yeah. And are you familiar with how are you guys feeling about this and handling context in JavaScript? I think I personally, it's kind of in my, I'm kind of an abstract person. So I think I'm kind of okay with it. Yeah. It's good to hear because it can be very frustrating, right? Yeah. Like all of these. When I read about the arrow thing, you know, I mean essentially kind of creating an anonymous function, but allowing you to be scoped, you know, to maintain the variables that are scoped outside of it. Exactly. Yeah. I was like, yeah, you know, I don't, I haven't done a ton of JavaScript compared to folks who work, you know, all the time in these kinds of frameworks, but you know, but I've come across that problem before, you know, immediately I was like, okay, yeah, I see what they're trying to do here. And yeah, it's good. Yeah. It's a nice little trick. You can run into problems with it because it can get a little confusing if you're not used to juggling this value. It's also worth noting that it doesn't carry much beyond this value. Like if you declare a variable, like here, currently we have in this scope, we have username password and error pulled out as constants in this state. If we did this and talked about username in here, it would obviously absorb. It would, if you declared a new one, a new username value, this would shadow the variable from above, right? So this, you know, username here would be fave. And if we obviously didn't declare this, this would be username here would be that outside scope, but that would also be true of a function. So a plain old function, blah, it'd be the same thing. The only thing that the fat arrow syntax affects is this, right, the phrase this. So if you call a raw function inside of this context, it may or may not be called in the context, depending on where you call this, a raw function call, like if you were to just immediately call this, this could end up being bound to window. Instead of being bound, which is really irritating. That's where the fat arrow syntax is really helpful. Because right now, if I were to say like this.state here, that's bound to the component, right? And that's what we want. So react under the hood comes in and sees the render function and goes, ah, you probably want to call that in the context of this component. So when you talk about this.state inside of the render function, the react has sort of handled that for you. Now react doesn't know anything about our, sorry, let me clean this up. Doesn't know anything about our handle submit calls, right? Because these are custom functions that we're defining or methods that we're defining. So we have to actually manually bind those in our event handlers. And so that's why you see this bind.this. And then this is one of my favorite tricks because if we're calling bind, we can actually do something called function prebinding. And so what we're doing here is, so this is the return value of all of this is the function handle change bound to the context of this. But since we're returning a function and not actually calling it, we can pre bind an argument to that function. So the handle change, if we look up at handle change, it's function signature expects a name and then the event, the actual event object, which is like would be a DOM click, DOM click event in most case, in this case it would actually be a change event from the DOM. So this name, you know, without this, we leverage this function prebinding. So we're actually passing in one of the arguments before we've even called the function. So when somebody actually invokes this bound function with the event, what gets passed to handle change is actually username and then the event. And so where this really cleans stuff up, is this avoids us having to do stuff like handle username change in the event. So we don't have to have separate ones for each. We can just pre bind the field that we're talking about. And so you'll see this constantly in the code where it's like the value should be the username that we've pulled out of state here. And when we change it, we want it to affect the username field of state. So it's kind of janky and it's a little tough to get your head around, but once you understand function prebinding, it's a super powerful concept that can save a lot of boilerplate. So yeah, how are we feeling about that? That's okay. I've done a little bit with the functional programming. So it comes up to, I think they call it partial application or something. Absolutely. Yeah. Partial application. It's related to the notion of couring, but it's, yeah. It's not, this isn't technically couring, but this is, yeah, partial application or partial applying a function. The only reason we call it prebinding is because we're actually using it in the context of calling bind, which connects, which actually connects us to state or to our current context. I think it's really, it's really nice that you're reminding me of this stuff. Cause like I, you know, I, I've worked with, you know, this and stuff, but it's always a little confusing. It's nice, certainly nice to, you know, it's nice to, it's nice to be useful then. Yeah. Yeah. So, I mean, moving right along, I think that's kind of the general gist of things. Like it's worth noting that every one of these containers expects to have the create, like you'll always have the create container function at the bottom. You can always expect to see, oh yes. And then finally to connect that final dot, you see this is where we actually do the subscribe. And so this is where Meteor does its magic. And we say we want to subscribe to changes to the events feed. So in our API folder, we declared a publish, we published a few total events, which is just a list of all the events and without any, you know, changes or anything. Clearly we're making some more private events happen. And then inside of our container, wow, node modules, you're a big folder. Inside of our event container, we're actually subscribing to changes on that. So what this does, what this effectively does behind the scenes is now that we've called subscribe on events, this Meteor will use a WebSocket connection to the database and anytime Mongo pushes out, publishes a change to the events feed, this will actually connect to the create, Meteor will get a notification of that. We'll recall render with all of the new values and we'll update all of the state automatically. And so this is how we get the magic. You know, when one user changes one thing, it changes across the board for every user. And so it's a really, I think it's a really beautiful sort of set of tricks. You'll notice that the first thing in our create container functions we do is we always do all of our event subscriptions first because all of these sort of like, like find one event. So this is where we're actually like pulling this database and like setting up the actual props that we're going to return. Are we in Babel syntax? Yes we are. Yeah, so it's, I don't know, I think it's super fun. It's a slick system where render is going to get called on its own. Like you'll notice that our code doesn't have any direct calls to render, right? Like rendering is being handled by Meteor and it's being handled in reaction to changes to state on the server side or changes or changes that the user is initiating inside of our handle. In high level, you just basically you're telling Meteor like what objects you want to wear and then it handles updating them when they actually get updated. Exactly, exactly. So inside of our events right now is the model container and so we're only subscribed to the events feed so this way changes to URLs we don't get notified about, right? And so, and Meteor handles the unsubscribing for us which I think is like insanely magical and maybe too magical, but... It's based on what I know, thanks. Yeah, it's, you know... It's not Meteor so that's reactive. Yeah, yeah, yeah. Now React, this isn't, this is more Meteor than the auto magic nonsense but whatever. And so like you'll notice on the invite we only subscribe to the invites sort of feed. So it's, yeah, I think it's a slick sort of like way of delineating what is current... So if a user is currently looking at the invite prop or invite component, that's when we actually create all our subscriptions and that's... And so they all get torn down every time somebody navigates away from a page. Yeah. So how is this feeling? Kevin, you're feeling acquainted? Yeah, yeah, pretty well. I mean, you know, I've kind of... You know, yesterday was the first time that I really opened up the code to try to start changing things. And so, you know, I kind of figured out sort of half of this and, you know... And then, you know, kind of the other things, you know, the sort of the exports and the, you know... Oh, can't hear you. Hold on. You're breaking up, Kevin. No, just can't hear you still. Not sure. But yeah, I'll let you sort that out. Sorry, I feel like... I mean, obviously, I don't... There's a lot I don't know, but I mean, it's definitely been very helpful both in terms of through the overview of the thing and the organization. Oh, that's yeah. I think working through the folder structure is like a big, you know, there's a lot to understand there and just, like, where stuff goes. The organization of, like, what's going forward and what... What is it, like a 10-band board or something? How does that work? It's a figure of how to proceed or... I mean, I guess there's a Tuesday meeting usually, right? Yeah, absolutely. That's the big sort of, like... I mean, Kevin's been working on this amazing issue queue, which is... I'll paste that into chat, but that's actually hosted on the edgy GitHub. And that's... Yeah, I'll as energy after, but basically that's where all of the current sort of work to be done on the app is listed. Yeah. So I think that's it. I guess it's worth noting that this is our CSS framework, so we leveraged that a little bit. I've actually gone to the trouble of, like, pulling Bootstrap out, like we manually sort of generate Bootstrap every time. This way we can sort of, like, manipulate stuff, like the variables folder inside of Bootstrap is actually, like, custom handled by us. We're able to manipulate some of these values so that we don't sort of have to talk on top of Bootstrap. Okay. Oh, I think that's a little better, Kevin. Yeah. I don't know what... nothing visible changed here. Well, it sounds great now. Yeah. And so, like, inside of this Bootstrap folder, or inside of this CSS folder, we have the actual definition of some components here. So, like, we have, like, and I also define a couple of, like, conventions, like this list.page. You'll see this a lot in... if we go to components, we'll see containers, list containers. So each container declares the way it inherits a lot of its style is from this list... list page class name. And so here we're sort of saying this is generally how a page list should look, and so if we... and, like, page has a whole bunch of stuff. So just a raw... we take the raw page handler, and then... or not handler, page class, and that's how this is sort of, like, general description of how a page should look. And then a list page should have, you know, these extra additions. And so it's just a more flexible way of dealing with the CSS. So everything to make a page look proper, usually all you have to do is add page, and then if it's a list, we just add list. And then if we want to do further customization, we would declare a new file called events somewhere in this folder, and we'd actually do all of our stuff off the ID. So if you'd like to give you a feel for that, that would look like this. Events. And then we'd do all of our event stuff in here and we'd save that in the CSS folder. Yeah, I think that's kind of it. Feeling good? Feeling great? Yeah, I think so. Yeah. So, yeah, so a couple questions. It would be great to get a QA environment set up. So that we can kind of stage changes. And then the other question is just about getting sort of test data into that environment or down into local environments. Just so that we can deal with things. I notice that there's the URL's page just kind of caps out at 200. And there's a little bit of code in there sort of sketching out possibly doing a load more or just being able to work on that without having to make 200 URLs. You know that kind of thing. Yeah, there is a script to pull from production in the private import, but I don't I obviously for security reasons haven't published the database access URL. But what we can do at bare minimum is just get a dump of old staging data, which I think is probably the best to work with because that means that you're actually seeing like real information full of errors and craziness. Sure. And so yeah, we'll definitely, so yeah, that's one thing we should do. Obviously that will contain passwords and user information. So we'll have to figure out a way to sort of anonymize that. I'll sort of see if I can't publish a proper data set that we can use as like the Mongo import. There is also a there should be a sync. Oh no, I don't think I actually put that in the folder. Yeah, because I was worried about that stuff. There's some sort of, you know, stuff that is worth protecting security wise, but from the CI perspective, I couldn't agree more having some sort of, I think I'm going to get you guys access to the staging environment actually on Heroku. So if you want you can push to it and just zap the database whenever needed. I think that'll probably be one of the easiest ways for all of us to sort of grind on changes. And then continuous integration, we really should be you know, once we have a proper test, you notice I didn't cover tests at all because there aren't any. We talked with Dan a little bit this morning. There's some thinking that we'll actually try and tackle that in the near future. I do want to sort of say we do have a lot of brand new tech coming out next week and one of the big things on the roadmap will be taking a lot of this code and I'm so this weekend we had six events and I had to spin up seven servers just to handle the load and we were maxing the RAM on all of those servers for a lot of time and so like seven boxes for I think what is basically maximum 600 people using the app at any given point, that's just too much. I think the issue is that me or just eats RAM like it's potato chips. So I think what I would like to think about for next week is actually taking our archivers front end, it's written in React that I think the React code is great and I may actually write a custom back end for it that just talks to the WebSockets work with the next iteration. If there's not a huge investment in the media part, I definitely see that having people's clients update in real time is not necessarily the most important thing, especially if it's getting servers getting bogged down, for example. Yeah, I mean actually at these events it is very important because that's what keeps people from stepping on each other's toes. The actual like mutex locks that somebody checking out is really important. That's like something that we've we learned the hard way when we were working off of Google Sheets. So we do need something but we really only need it for that lock unlock feature, right? Everything else you know and one of the huge issues that you'll notice with Meteor when you start to get working through this live data stuff is like pagination is just a nightmare, right? Because if you think about it, the reason that we only wrote low 200 URLs and the like load more stuff that I started grinding on you can understand how like Meteor publishes a change has been made but it has no idea what subset of the URL list you're looking at and so it just gets really confused when you start doing paginated work. So right now the current work around is just to show you hey there's 200 and some URLs and then we give you search so that you can grind down into actual specific URLs. Moving forward I think it would be much more ideal to have stuff like URLs not delivered over the Meteor subscription sort of style and instead publish that in the classic sense of like hey you're talking directly to a database and we'll paginated the normal way and we won't sort of eat up all this memory trying to serve you change feeds of thousands of URLs and instead we'll only save that memory for a WebSocket connection that explains hey this person is currently working on this URL and so we can do we will always have that WebSocket oriented we can push updates to you and say hey somebody changed this file and so we'll still have that on the one to one when you're looking at an individual model this stuff I think Meteor is just choking partially because we're not using it to its fullest capacity that's for sure but just thinking about the like long term need of the project where we're going to start to load many more URLs into the app once we get to a one to one relationship and so I think Meteor starts to become a tougher fit when it's like no we're actually going to need to jam like two to five thousand entries down to a client and have that just sort of stay there so there's lots of things that's yeah so just before anybody goes crazy being like hey I'm going to like you know pour 10 weeks of my life in this thing we have some changes coming down the pipe that would be worth thinking about before we do that already though I mean I think today it was so worth going through and like linting all of that code and like having a good look at it I think it really ups the chances that we'll sort of carry a lot of those components forward into our new stuff I'm super pumped about it I think I think the project's headed in a great direction particularly all of the issues that we're seeing show up on our public repo I think is really nice just to get good UX insights into what is and isn't confusing people like that knowledge will we will absolutely use moving forward and so if we can chip fix this that stuff sooner awesome but if we can't I'm okay with catching a lot of that on this like version two stuff once we sort of put it to the community get some feedback and understand where where the direction launcher direction of the app is so it's not good you guys yeah I question I was a slightly a field but as one of the people who are kind of thinking that some of the stuff is not not so much this this the UI URL sorry stuff because that has to be an app telling people what to look at but like the actual like coding scraping stuff could be done in a wiki I didn't want to say would they have people talk to that yet at all or are you having so much stuff to think about so that it's kind of you don't have to tell me now that maybe it's been done brought up on Tuesday I think I will get brought up on Tuesday a lot of the people from MIT are taking on a lot of that project of actually wicking out how to work with specific styles of scraping yeah I'm actually I'm affiliated with them oh right are you at MIT right now I know and so so Jeff Lou at MIT is and him and a guy named Max who I haven't met properly but is wonderful are sort of really taking the initiative on that we've been having some chats with Dan Allen who has floated the wonderful idea of potentially running all of this in a sort of cron based continuous integration style environment where we would ship the code for scraping like tough to scrape stuff in a specific format so that we can actually just automate its execution I think that would work really well yeah yeah and I think it'd be really fun because that'll mean that we can actually that'll save a really great space for people to contribute awesome usable code at these events where developers can show up and write these you know difficult to scrape scripts and know that it'll get used and rerun and like I think that's super helpful for us and it would be a fun challenge for people looking to work on a tough problem so I think that's the hope we don't we don't have a full plan in place for that yet but that's sort of what we're thinking about as a long-term solution for tough to scrape content yeah I'm definitely going to go to the try to go to Tuesday again it's well worth it we'll talk about that because I do a thought but yeah this has been great I mean obviously the pipeline app is essential to figuring out how people decide where to put stuff totally cool well it looks like Kevin's dropped off the call so I think we'll probably wrap it up here if you don't mind yeah thanks so much for joining John and and we'll yeah let me know if you have any questions moving forward we'll do