 Note the the characteristics of that video. It was 52 seconds long and The video gave and it was an average bit rate of 535 kilobits per second and Audio 93 kilobits per second Which is a total of 620 630 kilobits per second So our file size 4.3 megabytes 52 seconds is So 4.3 megabytes divided by 52 seconds gives us a rate of about 630 kilobits per second where 90 is the audio and 530 or 540 is the video that the average rate across that 52 seconds Of course it varies over time, but that's the average rate That's the actual rate after using the The h264 the mpeg 4 codec for the video and the mpeg 4 a a c codec for the audio the raw rate 472 megabits per second although I Saw before as 48 frames per second. Maybe I was wrong Maybe it was just 24 frames per second if it was in which case we'd have to half this value But this is still the raw data rate so The data rate of our network the errors or Packet loss in our network the delay in our network and the jitter or More formally called the delay variation impact upon our multimedia applications That was demo was just for streaming with interactive applications the performance requirements more strict especially with related regard to delay with Interactive application someone talking to someone across the internet the delay needs to be small so that it's as if they are in the Same room talking to each other. So typical values expected with a voice call if the delay So not that the jitter or delay variation But the actual delay from one endpoint to another is 150 less than 150 milliseconds It's not noticeable by the two users 150 to 400 milliseconds is considered noticeable, but people can tolerate it much greater than 400 or even half a second and The application may become unusable. So delay is another Criteria that impacts upon our interactive applications Let's look more detail with an example about jitter and I'll try and draw this diagram You'll notice there's an error in there and on each of the slides you'll detect that To see the general concept of why jitter impacts upon our streaming and also why YouTube for examples pauses for some time and then starts again or Why when I started the video playing there was some delay before it started playing back This not this diagram tries to illustrate When the source sends the packets So the rectangle on the left is the source computer They send the packets across the network and when the receiver receives the packets and Plays back the content of those packets So for this simple example, let's assume what happens. Let's say it's video or audio each packet Contains some portion of the video that we need to send across the network and playback a Frame or a set of frames of the video So what the source does our server in in our example the source I'll call it the server the source creates a packet puts the video inside that packet a Small portion of the video that is several frames or one frame inside that packet and sends it across the network when the Other computer receives that packet it takes the content and plays it on the screen So in this example The source is going to send a packet every 40 milliseconds just made up that number just for this example and Our network has a delay of 50 milliseconds What that means if I send a packet at time zero The other device will receive that same packet 50 milliseconds later If we send at time zero will receive at time 50. That's the network delay in the first example The jitter is zero the jitter is the delay variation the difference between the delays of subsequent packet So in this case at time zero we create the first packet Send it across the network It arrives at time 50. That's this case. You don't have to draw it again and In this case we creating a packet every 40 milliseconds to store the video and the next packet Therefore is created at time 40 and sent at time 40 With a 50 millisecond delay it will arrive at time 90 Next packet packet three When is it sent? When is packet three sent? Easy question don't say 120 120 is wrong We send every 40 milliseconds my server Sends the first one at time zero the second one at time 40 the third one at time 80 okay good the slides wrong Okay, I missed a packet in all of these diagrams. So that's the point here that should be 040 80 120 160 and so on so be careful You would have detected that anyway and received 50 milliseconds after it was sent Okay, look at and we can keep drawing that for the others look at the Rate or the the time at which the client receives the packets at time 40 Sorry at time 50 another 40 Milliseconds later it receives the second packet Another 40 milliseconds later receives the third packet and another 40 milliseconds later It receives the fourth packet and so on The source is generating and sending packets every 40 milliseconds in this example the client receives and Plays back the content every 40 milliseconds Would say that this produces smooth playback Because the rate at which it's being generated and the rate at which it's being played back is the same One packet every 40 milliseconds Okay, and hence if this was video it would look like smooth motion video There will be no pausing and and starting again The only problem in this case is that the client If this is YouTube for example you you have the client your browser open you press play at time zero That triggers the server to send packets from when you press play Until you start receiving the the packets and start playing your client plays back those packets It's a delay of at least 50 milliseconds So in this case there's a delay upon startup So I press I press play Sends a message to the server the server starts sending packets and then at time 50 I start receiving the packets and playing back the video So the network delay in this case of 50 milliseconds impacts upon the time for the the playback to start But once the playback starts we start receiving the packets and playing back the content at a smooth rate Every 40 milliseconds That was with no jitter No jitter means that the delay between each packet is the same Let's try a different example playback means Playback on your Multimedia client video for example When I was streaming my server sends the video in encoded in packets It's more complex than what we show because it's in compressed, but the server Takes the video sends it in a a portion of that video in a packet And when my client receives that packet VLC on the client Plays back that video It shows the frame on the screen so In a simple example, let's say that this packet Contains one frame of video Okay So the server takes a frame of video puts it in a packet sends the packet when my client receives that packet It shows the frame on my screen. It shows that 854 by 480 pixels and Then when I receive the second packet my client plays back the next frame on the screen and so on and Of course video we need to get this illusion of motion. We need the frames changing at this rapid rate 25 frames per second or so on so if We receive the packets at a regular rate We can display the frames on the screen at a regular rate and Hence we get smooth Motion on the playback so the playback is the playback of the video in this case We can apply equally to audio of course in in real life. It's much more complex because there's compression, but the concept Should be clear This was in a network with some fixed delay. No jitter. Here's a network with the same amount of delay but there's a jitter of We'll see we'll calculate in a moment We say delay and jitter Delay the jitter is plus or minus 10 milliseconds in the first example We had a delay The average delay was 50 the jitter was 0 and therefore packet 1 So this is the average delay of the packets and the average jitter That it delay variation packet 1 had a delay of 50 packet 2 50 3 50 and so on Every packet had a delay of 50 in the the first example this one In the second example we're saying that the average delay is the same The jitter is plus or minus 10 Means the delay of individual packets ranges between 40 and 60 50 plus or minus 10 and It would be random So I've randomly chosen some numbers here and we're missing one again My picture in all cases missed the third packet, but for example here The first packet has a delay of 50 milliseconds The second packet has a delay of 40 milliseconds Let's say this is not 120 but 80 here has a delay of 60 actually, what's Let's fix this What's the delay of the missing packet in this case? Fix my error in the slides. What's the delay of the missing packet choose a value? Give me it Let's Well, it should be between the delay of the packet should be between 40 and 60 okay, so we need a value between 40 and 60 Let's see if we can get an average nice of Let's make it 50 and let's draw it so it's clear a bit smaller Of course, they're more than five packets sent, but we're just simplicity cover the five zero forty eighty 120 160 so the source sends the packets at the same interval But let's say the delay of those individual packets are these five numbers 50 for the first packet 40 50 60 40 When's packet one arrive? Santa time zero delay of packet one is 50 so arrives at time 50 packet two packet two arrives At time 80 okay, my slide is correct packet three hundred and Thirty Because packet three we send at time 80 and I've just made up the delay here to be 50 and packet four Send at time 120 180 and packet five 210 the delay of the first packet was 50 the second packet was 40 third 150 and then 60 and 50 so the delay is varying I've just chose these numbers for the example. It shouldn't it doesn't have to be 50 40 50 60 50 There could be different values. It could be 40 355 but the idea is it's between 40 and 60 and on average If we look at many packets would say on average the value is 50 And it turns out in these five packets the average of these five delays is 50. That's our average delay Now let's look at the playback What happens our receiver when it receives a packet it Plays back the content. Let's say it shows a frame of the video Is the playback smooth in this case? Why is it not smooth? The delay variation look the first frame is shown on your screen at time 50 the next one 30 milliseconds later the next one 50 milliseconds later this one 50 this one 30 Think that the first front's displayed and then a short time the next one's displayed and then a longer time before the next third frame is displayed so the difference is Causing some unsmoothness in the playback and The user if this become if these values are large and the user starts to notice that the video starts to Not look like smooth motion What we saw in our demo was when we had a large jitter We saw this pixelation occurring and the quality going down because it's it's more complex than just sending the frames Because of the way the compression works if you have some delay it cannot play back all parts of the frame But some parts it can So it depends upon the compression algorithm But the concept the main point if you receive the packets with varying delay and play them back immediately The quality at the receiver will not be as good as if we have no jitter So how do we fix it? How do we overcome this problem? Because in a network sometimes we cannot control the jitter So we can use buffering to try and fix it any problems there are questions before we look at buffering Okay, so Expert on video Why not? Watching video so you can discern the quality you can detect the quality if there's jitter if you're expert of watching video You should now understand what causes the change in quality Our problem here is that our variation in packet delay Causes a poor or a reduction in quality at the receiver In our network we cannot control the jitter It's something we have to deal with and cope with one way to cope with it is to introduce some buffering at the receiver In the previous case whenever the receiver received a packet It immediately played back the frame or the content in that packet with buffering You wait some time before you start the playback with the idea that the playback will be smooth and For the smooth playback in our example we mean a Constant delay between the playback of the content. Let's expand our current Diagram again this picture all based on the same wrong picture the idea here is that Instead of immediately playing back The content delay it Such that we can play them back play back the individual frames at a smooth rate in this example We're going to buffer the first packet for 20 milliseconds We'll come back to why 20 In a moment, let's see what happens The client receives the first packet at time 50 Let's not play back yet Let's buffer for 20 milliseconds and at time 70 playback So we receive at 50 store in a buffer in memory and then at time 70 play back the content the second Frame is received the second packet is received at time 80 What we do is Okay, watching the video on your phone you can test the stream when you get home tonight There's instructions on the website. You don't need to do it now and Maybe you can show me all the problems or explain it tomorrow in the lecture. Okay. Give a demo When do we play back the second frame? Using buffering the idea is that the first one was delayed by 20 no in The previous case we played back as well Not immediately after we receive it in the previous case as soon as we received it at time 80 we played back The problem was that we got this unsmooth playback Because the delay between each of them varies Given the same scenario We receive the first one at time 50. Let's wait for 20 milliseconds and then play back at time 70 The second frame is received at time 80 When do we want to play it back? When should we play it back? What we'd like we'd like to play it play the content back at a rate at which it was generated every Frame is generated or a frame is generated every 40 milliseconds To get smooth playback We play back every 40 milliseconds So if the first one is at time 70 The second one should be 110. That's when we'd like to play it back and the third one should be 150 The fourth 190 and then 230 if we do play back at those times then it means we'll have smooth playback because Difference of 40 40 40 40 and When we created it it was a difference of 40 40 40 40 so we want to play back the same To achieve that We buffer some of those packets when we receive them receive number one at 50 Buffer it for 20 milliseconds and play it back a little bit later at time 70 Receive number two at time 80 Buffer it for 30 milliseconds play it back at time 110 This one's received at 130 buffer for 20 playback at time 150 This one received at 180 buffer for just 10 milliseconds play back at 190 This one at 210 buffer for 20 milliseconds play back at time 230 if we buffered the first packet for 20 milliseconds in this Set of five packets we can play the packets back at the right time By the right time. I mean the time such that they get a smooth playback if that works By this buffering we get a smooth playback What's the problem? What's the problem smooth playback in this case because of the Content is played back at a regular rate every 40 milliseconds, but what's the problem memory? complexity and Is something that the user may notice you who's watching the video may notice? The delay at the startup note that from when you press play until when the first Part of the video is played back a delay of at least 70 milliseconds Whereas in the previous case it was a delay of just 50 milliseconds So by buffering we introduce this extra delay at the startup So you press play It takes a bit more time before it starts playing the video because we buffer some of the content But once we it starts playing It will be a smooth playback because Even if we have some jitter that buffer tolerates or deals with that jitter by Playing back at a slightly later time If a frame arrives too early Then we can delay it if it arrives too late then we can buffer it for a much shorter time So we play it back at the right time So this is a trade-off in that if we have jitter have some playback buffer Which can result in smooth playback But this extra delay at the startup and that's what you see in your your online video sometimes you press play There's a while before it starts once it starts it's smooth But if your network conditions are not good, then maybe it pauses for a while while it's building up the buffer again and Then plays back again and if if there's some problem in the network then again It may pause build up the buffer so it can play back smoothly The challenge in this case is working out how big the buffer should be Here why did I choose 20? Well in this case if I know what the jitter will be we can calculate what our delay should be at the start If you choose a large value here Then you can cope with larger amounts of jitter. So if I chose 100 instead of 20 here Then I can cope with a larger amount of jitter plus or minus 50 for example But if I choose 100 here, then again, there's a large delay before my video starts It wouldn't start at time 50 or 70 it would start at time 150 So we need to choose how much to buffer and that depends upon the network conditions So if you've got a large jitter, then you buffer more if the jitter is small you can buffer less So buffering is common commonly used in streaming applications Not so common in interactive applications. So buffering introduces extra delay But the advantage is it smooths smooths out the playback it Overcomes the problem jitter So playback buffers are a way to deal with jitter For for it to work. We need to keep track of which packet comes first. So we have sequence numbers and often timestamps We have some extra memory needed at the receivers and complexity of the buffer and What's a good value well, it depends upon the application and the network Characteristics so in stored audio or video streaming usually several seconds is tolerated. That's with YouTube You press play you'll usually wait one or two seconds For the video to start and you can tolerate that if I press play and it takes one minute for that video to start I probably have disappeared and gone to another video. Okay, so if the buffer is too large the user will not be happy But for live we need smaller delay So the the buffer needs to be smaller. Well several seconds even less I was like live video streaming in this case for interactive Even less is needed because for interactive applications of voice call. We need very strict delay Very small so buffering is not so useful any questions on buffering