 Hey everybody, this is Brian, and this is our 137th cute tutorial with C++ and GUI programming. We're going to be hopefully wrapping up the high performance TCP server design This will be part 6. I'm showing you this so that you know that there's Five other parts to this series. So if you're just now tuning in, please watch the what is it three and a half four hours of coding that? I've done Yeah, and I've done 130 now there's 37 tutorials on cute Be sure to check those out Now I kind of noticed there's a thumbs down here. Somebody disliked this video and I generally don't have a problem with that But I don't see a comment From the person who disliked it. I'm like that bums me out because if you don't like it I really want to know why you don't like it Maybe you didn't like it because I spent you know the first two minutes of the video rambling much like I've done with This one so Anyway, so let's just jump right in All right quick primer on where we at we've made our TCP server Which you know has the connections and the individual connection, which is a wrapper around the socket and then we've inherited that and we've made our HTTP server and HTTP connection now this doesn't actually send the file at this point. That's what we're going to accomplish in this video The last video we did this rate transfer class and that might actually be why they gave it a thumbs down But whatever so the whole point of this class was to transfer from one qi o device to another qi qi o device at a set rate Hence that's how a web server works when you go out to Whatever you're downloading and you go to download a file You're not going to do it at your full bandwidth. You're going to do it at a throttled limit I think for example Apache's default limit is 15k a second or something like that. I might be wrong Anyways So that's what we're going to do with this video is we're going to connect the rate transfer class into the HTTP connection So that we can actually to the serve files like a web server now I have in our little documents directory. I have a website folder, which just has an index page a 81 meg test file and then they Images folder just has an image in it so we can actually see this thing working So without further ado, let's just dive right into the code here and start coding So first thing we're going to do here is we're going to give this thing a good build just to make sure I'm going to give it a rebuild Just to make sure there's no little bugs that are going to catch us later on there shouldn't be because we've rebuilt this like 400 million trillion times And while we're waiting up there goes just couple unused parameters not a big deal In case you're wondering why I actually want the bad feedback because I want to do better There was actually it was kind of funny There was a fight that broke out on one of my videos and I had to go in there and kind of break it up because somebody Gave it a thumbs down and then somebody else came in and was like who the f gave this a thumbs down I was like dude chill Everybody's entitled to their opinion here But he was very Very unhappy that somebody gave my video a thumbs down, which I was I was actually kind of impressed. I was like wow all right, so Transfer maybe that's why I gave it a thumbs down. I can't spell so we're gonna make a point or two are right transfer class and Basically, we're just going to just do a little double checking here Remember we separated these you know just with this this empty line here so that we know these are going to connect To that rate transfer class that we just made so we're gonna get started transferred a finished and transfer error And I don't remember if we actually implemented transferred We can actually jump in there and look real quick We did not So let's actually implement that real quick. Oh What am I looking for here? Transfer there we are Gosh my eyes got like all weird so we're gonna write Emmett transferred And I think that's a Qn64. Yeah, so we're gonna have to change that a little bit here In case you're wondering why I do a Qn64. Whoa is because that is the an integer big enough to actually Hold The size of a file. So if you have like an 80 gig file, it's not gonna fit in an end It's gonna have to go into a Qn64 All right, so let's double check that transferred 64 transferred 64 Let's go into our rate controller here transferred. Where's our signal? Yeah, here it is. Make sure let's give this another build just in case and drumroll Maybe possibly probably. Ta-da All right, so we got a good building start fleshing this thing out So we have got our signals and slots. We've got our class. We got our pointer to it We can go in here and start fleshing this out here And if you remember I commented that out cuz I was looking at my notes So where is send file here? There we go To do make a rate controller. Well, we've done that so let's start banging this bad boy out here All right, so We want to check to make sure we got an actual socket If not, we're gonna just you know say, but forget you I'm done. We're gonna make a new Q file Now you've noticed that I'm making these as pointers. I'm doing that just strictly out of habit They really don't have to I Tend to do things in pointers in case I want to pass them around so instead of copying an object I'm copying just the pointer. You could always do a reference to or whatever, but And you also notice that we're setting the parent we can do that because This guy the HTTP connection lives on the individual thread I shouldn't say the individual thread the worker thread that we've created. Now. Let's just say created My brain is just mush. I had a really long day at work. So I'm like And through the magic of copy and paste I'm going to save a little bit of time by just pasting the connection strings in here We're just connecting the signals and slots Rather than you watching me type for 10 minutes straight You're just going to copy and paste those in there. Notice how we are just we're not doing the cute direct connection We're just doing an auto connect That's because this all lives up on the same thread So we don't have to do the the cute connection or the auto connect or whatever. It's just we're doing the default auto connect All right, so So we're going to take the ray transfer. We're going to set the source to the file and actually Before we do that, we should actually try to open the file. What did I do there? There we go So if we cannot open the file, we're just going to close the socket We're going to see now at this point We've already sent the 200 okay header and then we're immediately closing the socket So the client's going to freak out and go whoa our connection dropped So we're going to set the source. We're going to set the destination Which is going to be our socket. We're going to set the rate And we're going to set the transfer size Did I not do that? Hmm. No, I didn't let me double check. I Don't think we did a set size now. We didn't we did a set rate? All right, so we're just going to set this to an arbitrary number here because this is our web server We're just going to do this. We're going to say 1k is the maximum size and then we're going to say Q to bug We want a lot of Q to bugs in here because we want to see what's going on Damn you cursor. There we go So we know that the rate transfer class will transfer from one Q over one QIO device to another So all we're doing is we're setting the source setting the set destination the rate the size And then we're just going to kick this bad boy off and let the rate transfer class do what it needs to do all right, so Before we do that we're going to do M. Oops M response remove code Now we're doing that because In our I think it's bytes written. Yeah right here. We're checking for a code So if we have a code and it's 200 we're going to send the file Otherwise we're going to you know check for 4.4 So we don't want to check the code while we're sending the file because it'll start trying to send the file again and again And again and again every time we write a packet and we don't want that So let's do this Last but surely not least Start we're gonna actually start the transfer now. There's a couple little gotchas here When we look at finished This is the rate transfer when it's finished. Notice how we're closing the file and closing the socket We have to do that to flush the buffer You could call flush but some operating systems don't actually do that automatically in if there's an error We can might as well just do it there too. All right. Let's give this a good build and There's a couple little others snippets of magic we need to do here We want to set the rate. I need a good rate here Let's set it to 4k a second and We're going to set the root I've already got the roots set up. This is just going to be a little website here Now notice how there's no no slash doesn't really matter because we have the code in there to check but So let me go out there and this is our Slash home root shell, which is just the username on this box documents website and it's going to look for the Existence of this index.html. So it's gonna actually serve that file So You know what? Let's just run this see what happens. Let's be brave All right, so Oh Still got our code in there. We should take that out because we don't want that. I Was like I saw this debug and went what is all this transferring crap? This is from our previous tutorial my bad Let me actually go back out here All right Make sure we didn't Blow anything up here All right, so we're going to start this Now I'm going to jump over to a browser here I'm going to get a little creative with this so we can see the browser and the debug window at the same time And we're just going to connect to localhost port 2000 and it didn't work processing yet Let's see what happened here Tick tock tick tock. What happened processing get Processing get handle requests state change to closing unconnected Why is it doing that? Obviously ladies and gentlemen, I forgot something so let's start digging through the code and see what I forgot And this is why these tutorials are an hour long because I cannot remember to do everything so All right, so we're going to write the response This is why we put Qtabug statements in here Let's actually put this in here and down in the bytes written because we're checking to see here We're going to actually send the file We're going to just do this code equals And ironically this is what I get a lot of feedback about people are like well I love watching you try to figure out your mistakes. I'm like grr. It's so frustrating But I guess that's you know part of life here so this Say attempting to Send file Blast you There we go I'm willing to bet the problem is in there somewhere. Let's double-check our output here Hey watch this Okay Processing get processing request Let's actually put another Qtabug in here Qtabug Writing header to socket and that's handle requests. So we know we're handling the request at some point We're saying state change disconnected See it this is the this is what's going on right here abstract socket closing state meaning we are closing the socket not the client and Then we're disconnecting and then we're removing that we're deleting So it's something that we're doing or more to the point Client is requesting a directory. Here's the problem right here. Okay? Client is requesting a directory. Oh, maybe if we just you know like put that little slash in there That might actually be helpful All right, because I'm willing to bet it was shooting out of 404 not found But we didn't see that because we didn't have a Qtabug statements in there Silly me. Let's try this again start and Nothing code 404 Why are we getting code 404? writing header to socket Code 404 disconnecting so it's doing what it's supposed to but it still thinks it doesn't have the file here I'm willing to bet we have a very simple problem, and I'm just overlooking it. That's probably the issue here So if I set file index file index files X1 plus that let's go out there and make sure the file actually exists Index HTML index HTML the one thing we're not seeing here is Clients requesting a directory then go ching Then it's instantly writing header to socket. So it's not processing this And it's not processing that It's just an immediately writing headers. So actually it is processing this because codes for 404 not found So Q file info Q file info exists. Let's just say Just for giggles. All right, so Q warning I'm gonna say index files missing and index file And let's see what we're actually doing here This suddenly became a lot more difficult than I thought it was going to be I do apologize, but we will get this going Huh index files missing slash slash index. It's trying to send root slash index. So there's a problem there right there As my plumber always says well, there's the problem. It's leaking. All right, so we obviously need to set the root directory So where is set root? We're not setting root there. That's part of the problem or we are there, but I'm not seeing root set to See, there's the HTTP server root set to But we're not handling it in the actual connection object. So Yeah, ha. Yeah, see there's our to-do Derp it helps if you look for the to-do right set rate Now I feel really stupid So if I just wasted however many minutes of your life, well, I'm sorry, you cannot have those minutes back You're just gonna have to suffer through it point in case that's why you have a lot of Q to book statements in there So you can very quickly kind of go through without setting breakpoints and all that other stuff You just figure out what's going on Now this absolutely should unequivocally run this time And if not, I'm probably gonna throw something across the room. There we go Now you see how that smiley face was loading slowly. That's our rate controller at work Let's do that again. Hopefully it's not cashed. Yeah, you see it slowly slowly chunking like 1990 or something on the internet with dial up an AOL. I Never actually used AOL, but that's imagine if you will what it looked like That's how you throttle the connection now point in case if I click this download It's gonna download this test.zip which is an 81 meg file So we'll just do that Show our download here And you can see how it's transferring in 4096 chunks and you can see how It said at 3.6 kilobytes per second. So let's go back. Well, no, there was seven there for a second there So what did we say our rate was? Anybody 4096 so we're saying about 4k a second is our max rate on this thing and you can see it staying under 4k a second That is why we did the prediction in there Once in a while, you will see that shoot up in size Couple little caveats, let me actually cancel that then you see how the file transfer error destination device not open not writable blah blah blah We're gonna clear that Let's actually fiddle with this thing a little bit. Let's see if we can really you know Let's set it to 15k as the rate. So we're gonna set it 15k a second and let's try this again Start All right, let's See how the smiley face loads much faster still saw a little bit of chunking because we're only 15k a second if we try to download We'll see now that we're at around around the 15k mark It's not perfect. It does jump over 15k But you can see how it's definitely throttling it and it would take us two hours to download this file now We do this because that timer Will you know sit there and wait for its single shot to fire off and it allows other clients to get a slice of the time Slice of the network bandwidth and do their thing So let's clear that whoops. Let's go. Let's cancel that clear this and Let's just give it some gigantic number 150k Actually, let's just more more is better right start Refresh this smiley face appears almost instantly and you can see how whoops It's going much faster. We only have 48 seconds instead of two hours because we're going a little over a mega second, right? Then you can see the window down here is going much faster So that ladies and gentlemen how's how you throttling network connection? I'm going to actually cancel this So that was a long series man, but we covered a lot of ground I kind of want to do a quick recap of what we've done That way if this is the only video in the series you've watched you can kind of pick up where we've left off and not spend Three some odd hours watching these videos All right, so We started off. We made our TCP classes They just inherit QTCP server and then the connections basically keeps count of our connections and does some memory management in the background Does some cleanup and then the connection object itself the TCP connection This is just pretty much a wrapper for the Q socket From that We can inherit and you can make all sorts of things with this right So we could make an HTTP server and echo server and FTP server Excuse me. That was rude We could do all sorts of things The rate transfer class is reusable because it uses QIO devices You could you know say you want to make a web server an FTP server an SSH server, whatever doesn't matter Some some pitfalls and caveats here You will need to run this as route if you're running on a port lower than 1024 Or if you're in Windows you'll need to run it as administrator Yeah, you may also need to add an exemption to the Windows firewall I have not tested this on Mac, but I'm gonna assume it'll work fine. And you know what just for giggles Let's test this with Siege All right, let's start this let's load up Siege here We're gonna pop 800 connections at a time and you see how now it's returning 200. Okay, and It's you can see it's just like going freaky crazy down here Now it's actually transferring that file like if we control see this Maybe if we can find something here TCP connections Just connecting client here Yeah rate transfers so you can see because there's rate transfers in our debug We're actually transferring the file over to Siege see Rate set to size set to starting file transfers start called file transfer da da da da Riding the destination blah blah blah blah So our Siege statistics for that little amount of time was 12,000 hits So we actually transferred that indexed HTML 12,000 times we transferred 6.14 megabytes a second over 11 seconds Transaction rate yeah self-explanatory concurrency success all of them zero failed longest transfer was 1.82 seconds Siege is an amazing utility for stress testing web servers by the way This is not How do I want to put this? This code is not the next Apache server. It's not the next is but it's a good example of how you can handle high volume large amounts of connections and traffic and It's it's scalable in the sense that you can handle bursts of traffic from your at one or two connections to sudden You're at 5,000 connections But you know you should do things the correct way and put them behind load balancers and things like that so In closing I've done these tutorials about the TCP server because it floods my inbox It's by far the number one question I get is hey I don't understand TCP servers your tutorials relating cover it well or how come my you know server won't handle more than a hundred connections You should be aware because I'm gonna make the assumption that you skipped the other videos And you've gone straight to the finale here You should be aware that this is gonna be limited based on your operating system for example Let me just try to bring it up here if you're on a Linux Unix derivative. It's you limit whoops I Think it's dash n there it is You limit the dash n displays that I have 1024 File limit this the maximum number of files that I can open. I'm not sure if that's per user or per application It might be per user You can increase that but you do so at your own peril And then there's other things like you may actually find the command to increase it But then find that there's some other file that limits the number of sockets or limits the number of connections And it's it's very very much off your OS for Stress testing purposes. I was capping this out on my Linux box at around 900 connections And on my Windows box at about I think it was 8,000 connections The reason is Windows 7 has a higher file limit basically and it uses I OCP instead of hard files So it doesn't mean Windows better. You can definitely increase the limit in Linux I just I haven't done it just because I have no interest in screwing around with my Linux box in that manner All right questions comments concerns feel free to ask I try to visit the video, but you'd be better off going to the void realms Facebook group There's like almost 300 of us out there and The source code for this and all other tutorials will be on my website void realms comm go under tutorials cute And it's going to be way way way at the end here, and it will be the source code And you notice I've got you know part one through and I'll put six out here tonight And so you can see the progression of the code I should note this site is 100% funded off your donations So if you found this useful, especially if you're a business and you're going to use this code to build your Next latest and greatest thing. Please feel free to donate all the funds that get donated go into either a pizza or B paying for the website Honestly 99% of it goes to the website except for last year where I actually donated a sizable chunk of it to charity Just because I got more donations than what I could use All right, that's it. Thanks for watching