 demo. The T2 Tile project is building an indefinitely scalable computational stack. Follow our progress here on T Tuesday updates. Hey, welcome back. It has been a month since the last T Tuesday update and pretty much all I have to show for it is the We Are Coders, the second lecture in the Hyperspace Academy HSA 101 lecture series is now finally out and you know last December I promised it in January of 2021 and I barely made it. It came out this past Sunday but as a result I have really no worthwhile news to report about progress on the T2 tiles which is very sad but I will circle back to the at the end of this video about my plans for two weeks from now when it's time to really get serious and get back to T2 stuff. But since my head was full of it today all of this video production stuff for the HSA video and since you know since seven years ago how do you do your recordings up till you know a couple of months ago when Abhinav was asking the same thing this is like the number one question I get you know how is it that I make these videos that mix together stuff and so forth. So today I am going to break it down into I hope completely clear pictures. I mean I give people the high concept a lot but nobody ever actually makes one and so why not. So I thought I would actually try to go through it and find enough grain detail and in particular talk about some of the non-obvious little problems and tricks that make it look better and that's what this one is going to be about. How to make Max Green video. I call it Max Green for reasons that will become clear. It's like green screen or blue screen it's something like that but it has different advantages and different limitations. Alright so the basic idea is that we're going to be grabbing the workstation computer laptop screen whatever it happens to be. Simultaneously we're going to be recording live video of the speaker and then we're going to put them together somehow so that it's still possible to see what's going on in the screen content and the the speaker does not look horrible. There's a lot of limitations on this particular idea that I want people to know up front so they won't be disappointed when they think get excited about using it realizing it won't work for them. So there's limitations of two kinds there's the fundamental limitations of the ideas and of the idea and then there's just more specific limitations of my workflow. So in particular the idea limitations of Max Green is that it affects content design. You can't just take any screen content and expect this to work. You have to design the content knowing that you're heading for Max Green and similarly you need to control the set where you're recording the stuff. You can't just record it anywhere and have zoom remove the background for you. I mean you know software has moved on since then so if you don't have control of the content if you don't have control of the screen appearance and the fact that you don't have control of the set this is not going to work. Beyond that the workflow limitations the way I do it involves post-production. You record stream separately you put them together afterwards that means it doesn't what I've got doesn't apply to live streaming. People could probably figure out a way to do a live streaming version of it but this is not that and I do all my stuff on strictly Linux only I'm sure there's plenty of people that could explain how to do it with you know you have to pay for it tools if you want. So that's I think the limitations I think the easiest way to start to get into it is to show how I came to develop it so we'll start with some a little history. So back in round the end 19 and 2010 thereabouts there was a lot of discussion about teaching computer science to in grade school and so forth and you know there was a big push for it and eventually by 2013 you know once it reaches the president then you know it's been circulating under the hood for quite some time and in fact it had and in particular there was a program called New Mexico CS for all which was a grant funded by the National Science Foundation to build out a teaching structure so that the people that were working on the grant would teach high school teachers how to teach programming and then the high school teachers would turn around and teach programming in high school and I got involved as part one of several faculty members that were producing content for this remote so it would be hands-on because the high school teachers would be at the high schools but it would have remote content video content and so forth and just to make sure we remember the material based this material is based on work so including the entire concept of the max screen thing was kind of you know supported by your tax dollars a National Science Foundation that led to all of this getting happened led to me being recruited to make some content so my attempts began in December of 2012 and you know I was gonna try to use green screen I had never done it I thought no you know I looked at only one or ten of the million YouTube videos even back then on how to do this sort of stuff and I got a green screen and whatnot and I tried to do it and so this particularly NMCS for all programming is modeling video is green screen and it was died did it with a tablet and it was handwriting and so forth what kind of like Khan Academy was at least at the time and you know it looked awful I've got hair going all over the place and green screen has this terrible problem that it can't render you know fine details very well because basically the way it's working green screen is trying to make a very nonlinear decision on each pixel are you part of the background or are you part of the foreground based on color in this case and color green blue screen various other kinds of keys keys all do the same thing and you know if it gets it wrong or get you know it just chops through a pixel it makes this very nonlinear decision and it looks terrible and so I said you know I'm not gonna do that again and I started to think about it and it was like well you want a more gradual transition instead of a key that says yes or no foreground background there ought to be something where if the foreground and background are similar they would kind of be all nice and gentle a gentle arrival and then transiting over and that's when I got the idea of saying well suppose we just took the maximum value of each pixel if the background was brighter take the background if the foreground is brighter take the foreground and then if we can arrange to make the background dark it'll fall away and so by the next video which was the demon horde sort video that was my first attempt at doing this max green idea where instead of having a key green screen blue screen that's a black behind me and it's just picking the max so wherever my screen grab happened to have light colored stuff green red and so forth that got picked up and the darker stuff got picked up by me and you know look at all that fluffy hair you know green screen will not do this on its best I mean maybe you know Hollywood green screen where it's they're shooting like 8k and then they can mix it down but you know this comes for free again as long as you can control the set and you can control the content so at that in that first video I hadn't really caught on I was going to all lay out so my slides were like normal slides for a talk and they therefore they stomped on me but then by the next video which was the New Mexico CS for all one on hill climbing I had gotten the idea of you know just push the content to the left side of the screen and leave room for the speaker to go in there that's how max screen came to be so the basic rule is everything should have a dark background so that it will not win in the pixel by pixel max so you want very dark or black backgrounds for your slides or whatever the screen content is and you want a very dark or black background for behind the speaker like that and that's why you need to be able to control the screen content and the set and let's see here so here's what the world looks like to me as I look at you there is the the camera over there there's a ring light around it has turned us turned on now but it was turned off then here's the microphone that's sticking just out of camera range underneath and then up here is the monitor that shows the content that's being screen grabbed and this thing down here is part of the weircoder stuff and it's not part of the general purpose max screen idea if you want to know more about that go see the weircoder's video and so that's what it looks like ahead of me here's a closer look yeah so in the in the weircoder's video I was using a teleprompter this home homemade little cheesy thing that I did with a little cheap HDMI you know watch cartoons in the backseat monitor you can see the camera through the glass back there but you know I'm not using it for this I'm not using it for this one because you can tell it's a teachers day update and so that's what it looks like in front of me behind me it looks like this it's black curtains it's just blackout curtain panels that you buy these ones are fairly velvety they do a pretty good job soaking up the light but as you can see there's they're not magic black they're just dark and so as part of the set design we try to get as much light on the speaker as possible so that we can stop the camera down we can reduce the exposure on the lens so that the black curtains become very very black and again that's being able to control the environment for that so okay and this is what I've got a monitor to my right that's now this is a relatively new feature got a monitor so I can kind of see approximately what the camera is seeing and there's I guess you know if you can see it up there that's he clips in the background which is running Java code which is doing this the visualization stuff the slide stuff and so when it's time to make a video I start recording this this is the screen grab it's just a script using FM peg people can use anything they want obviously start that up start the camera recording so now it's happily recording away the entire screen and I record the take and the camera at this point puts you know the video on SD cards I actually have to move it over and plug it in and get the files off the camera that way but then I take the screen grab which has been recoded to take up less space and I put it in the directory along with the camera stuff and I fire up Kaden live KDEN live I'm using an app image version so that I can get a newer version version 2008 is reasonably new and Kaden live and you know like all these nonlinear errors it can be pretty confusing when you get start using it and I can't be a tutorial on Kaden live here but I'll just sort of show you how it goes this is what it looks like you got the timeline where you put your clips and you move them around and so forth you go up to the project bin and you say give me select your two files the MP4 file is the one from the camera the MKV file is the one from my screen grab bring them both in Kaden live grinds over them and makes you know little proxy clips and what not so that takes a moment and then we stick them on the timeline the output smaller slow that's the screen the 2020 with not the correct date because my camera doesn't have a battery and this is the trick right here I'm clicking on the upper track which is the screen and I'm inserting a composition specifically I'm inserting a composite and transform composition between the tracks that's the way Kaden live views things other nonlinear editors take other approaches have graph models and so forth but this one if you want stuff to interact from one track to the next you put compositions between them so we do that and now we have a composite and transform effect taking place between the two tracks there and then here it is we go find the compositing entry in the thing which is currently set to alpha blend you can see it there and we change it to lighten which does a max light means a pixel by pixel max in Photoshop the equivalent is I believe lighten only a blend mode lighten only which again does a max so we change it and there I am we are now composited together and that's really it that's the fundamental you know raw ingredients of this idea and then it's a matter well and then you have to slide them around and get the audio in sync and a whole bunch of other stuff and trim the ends and you know put title cards in and so forth group it together group clips so that you can handle it as a unit once you get the audio aligned and so on but that's the real basic idea I made a title this is just the demo of the little clip that we saw at the beginning of this update this is the process of making that and create it align it add some things and so far I'm not gonna spend time on it this is more arcade and live stuff and and there it is so that's it for that basic part of it now there is a second level in the advanced story to this and that is you know so what what we just saw will do the basic idea but there's this other stuff right you see me pointing at the screen you see me pointing at things on the screen not just with the mouse I mean I sometimes I pointed stuff with the mouse when it has to get details or when I'm really interacting but if I'm just talking about points I'll you know pick the high one middle one pick the low one you know come in and look at it and so forth how do we actually manage to get that to work that is not immediately obvious that that's the way it would look and if one tries it if you just sort of say oh I got some black curtains and or you tend to run into this problem it looks weird so I have a demo and we'll do that and then we'll wrap up so okay now if I have any skill with blender which I do not I'm going to use open scad to show this model but alright so there's okay here's me right I'm sitting in an empty space where let's get the camera alright so the first point is that so I we've got you know the camera is basically off my left shoulder so I'm in a three-quarter stance so that I'm splitting my attention between the screen ahead of me essentially and looking to the camera to my left so that wants the camera to essentially be somewhere off my left shoulder about like that and we then need a curtain behind me the black screen so there it is and now what I can't do if I could use blender I could actually show you the view through the camera but I don't really know how to do it in in open scad but you know it's something like this so you know we would the camera view would be basically like this and you know it puts the speaker on the right hand side of the screen which is what we want and leaves a big black area on the rest of the screen which is what we want now let's get the screen in here oops dear it's fun just too tiny another nice thing is by putting the microphone under the camera like this when I lean towards the camera you get a more intimate sound because I'm closer to the microphone and that ends up being nice okay so there is the screen you see and I am looking at the screen and that's what we're going to see on the screen so this is the big trick that essentially you want to have the camera essentially at eye height maybe a little bit lower and so if you think about it now what the camera is going to see looking through here anytime I am looking at something that's actually above the camera in the set it'll look like I'm looking up and anytime I'm looking at something below the camera it'll look like I'm looking down which I am so we've set it up so that the camera is about halfway up the monitor that I'm actually looking at and this you know it wants to be a fairly hefty monitor this is a 24 inch monitor or whatever it is so that you can actually see the the eyes moving and so forth but really that's the whole trick it doesn't have to be that accurate it kind of surprised me how easy it was when it all came together and what oh and then we already have we have a microphone but we've already seen the real microphone so that's not what we need you know we've got a microphone in the same direction as the camera so that volume makes sense relative to the camera and that's it so the idea is you know make the camera about eye high have the monitor be partly above camera height and partly below it and so the fact that I am looking in this direction which is really a very comfortable turn to me but when it's on the screen it looks like I'm gesturing it's something that's like all the way 90 degrees away from me and I'm not but it reads so that's the trick that's the demo and then it's just a matter of you know trying to get as much light on the egghead and his little light on the black background so that you get a nice crush to black and it doesn't show up and I've got problems with that now you'll be able to see a little bit of curtain down here because I have to mess around with lights some more that's it that's how to make max green video it served a particular purpose for me for the needs that I had and it looks half decent like that for a total investment of curtains and of course now I've got lights and whatnot but I didn't originally so that's it any questions as they say you know put them in the comments below if they if they make sense I'll see what I can say about it you know and again this is just my homemade DIY made nice soft specific problems back in 2012 and 2013 that I've been using ever since all right so two weeks from today it's about going back to the 2d printer and the copier and I've decided I'm gonna do what I kind of wanted to do but was afraid because I was trying to show results quickly which is to take this whole idea of build plate that I developed for helping the 2d printer idea out and generalize it out and say you know actually this is a useful thing for membranes creatures organisms in the grid to have some of this plate around them even when they're not in the process of making copies of themselves so that's what I'm gonna do I'm gonna try to develop general plate that we can put that things will put around themselves to kind of reserve space and give themselves a little buffer and give them a cleaner calmer environment to do their work within and therefore by their holding space that means someone else won't be able to hold space and if we can get there one two three four we'll have a space war and the place will be filling up with these you know claims on various bits of space and we'll see how it goes thanks for checking in if anybody does you know make a max screen video that that they put up you know let me know I'd love to see it thanks for stopping by hope everybody is okay see you in two weeks