 Awesome. Thank you. Thank you. Hello everyone. This is the art of video game sound effects. So let's listen to this clip with sound. And now let's listen to it without sound. What did you notice? Well, the experience itself is a lot emptier. It doesn't really convey the same feeling of dread. It doesn't really give you the characteristics of the large spider. And it lacks a lot of threat and it also lacks a lot of story elements that actually tie into the sound effects. So what I'd like to get across in this talk is, well, what are sound effects? My philosophy for creating them. What to consider when making sound effects specifically for video games. And what we use to create them. So a little bit about me. I'm Kai Jones. I am a sound designer. I've shipped six games. I've done a bunch of talks and panels. I've done sound design and audio programming for VR games. I'm currently working on some PS5 projects and in the past I've worked in VR, PS4, PC Switch and mobile. So what does a sound designer do? Well, we create custom sound effects for video games. We create these sound effects to match the visuals, to help support them, to immerse the player and also to make a believable world through the audio so that the player actually feels like they're actually in this world that we are creating. So what makes up a sound effect? Well, here's four things. The pitch, dynamics, frequencies and the emotional response that that gives when you combine all three. So first off we've got pitch. Pitch is the highest or the lowest of a sound. You kind of know this if you've ever played a piano, if you play a low note on the left-hand side in comparison to a high note on the right-hand side. We use this to kind of convey the size of a person and enemy character, weapons, power-ups, whatever. So the pitch is basically just the overall tone of the high and the low to give that kind of feeling and emphasis of weight to the actual things inside the game. So low pitch sound effects, they can often be perceived as large, powerful and threatening and these can be applied, like I said, to weapons, power-ups, enemies and the lowness needs to be relative to other sounds. If you really want a sound to make an impact, like if you have a boss appearing in the game, you need to reserve the low pitch for those kind of things so then people realize that it's slightly different and a lot bigger than what they're used to. So for an example, let's use the Sturmgeig Guardian from the game Elden Ring and listen to its footsteps and see what you hear. So what does this sound tell us? Well, they're a pretty big lad. And the pitch they really emphasise is the kind of threat that it gives off because it's huge and it's charging at the character, it has a health bar, which is immediately scary. And it's really low in comparison to the protagonist, which is the player, because you can't really hear the player much in there because it's really giving emphasis to how big this one particular character is. So you can't really hear it as much as you can, as you can, like the big enemy in the game. And then high pitch sound effects, often quite the opposite. So it's often a lot smaller and more subtle and you can have more high pitch sounds for less significant things inside of the game. So it just sounds that you don't need to bring attention to in the story or like UI sounds, that kind of thing. So let's use full guys as an example. So what does that tell us? Well, they're a pretty small boy. The design itself is really, really cute. It's waddling, which honestly, waddling doesn't really give off much of a threat vibe to me personally. It's also a lot smaller scale and the camera is quite panned out, so it really gives a bit more distance to it. So footsteps aren't really important to the game full guys. It's mainly about winning multiple courses so like the footsteps don't really play an integral role in the game itself. Let's move on to frequency, which is kind of one of the hardest things to explain. Frequency is like the characteristics of a sound which are kind of split, broadly speaking, into treble, mids and lows. So like if a sound lacks bass, it's often a lot less powerful, a lot smaller and a lot less threatening. But then when you base move something, it's more powerful, bigger, and there's a lot more threat to it. You'll know this if you've been at EMF so far when you just hear the thumping bass regardless of where you are in the camp. It kind of gives you a lot of power and it makes things feel a lot more powerful and engaging. So this is kind of like a graph of human hearing from 20 hertz to 20 kilohertz and from 20 hertz it's like the bassier side 20 kilohertz or 20,000 hertz is the kind of high sound. You can see there there's like the bass, the treble, and let's use the Stormgate Guardian again as an example of the bass end and how that works to give off a threatening vibe. So I've just isolated the bass sounds in this recording. So visually, using that last graph, there's a lot of bass, if you look on the left-hand side, it's like really, really boosted so it really gives that threat and presence. In comparison, let's again use Fall Guys as an example. I've isolated the bass in this. Yeah, there's not really much going on there. You can see it's a lot lower than the last sound. So there's really not much bass because it doesn't really need to give off that vibe. And then next up, we've got the Dynamics which is a kind of like tension release within individual sound effects and it creates a little story arc within the actual sound itself and the Dynamics is basically like the difference between the loudest bit of the sound and the quietest bit of the sound which gives a lot of emphasis to what the player actually needs to hear. Monster Hunter World as an example. It powers up and then it swings and hits and you can kind of hear the difference of volume and dynamics in this actual sound. So visually, you can kind of see the importance. If you look to the left-hand side, you see like a bit of a peak or like a lift in sound and then it starts to ramp up in volume in those little bumps which signifies the importance of what it's actually giving to the character. First off, we have the character when you're listening to it and you're playing the game, there's a little emphasis to let you know that the actual attack has begun so it's a little bit louder and then it ramps up in loudness to then let the player know that the blade itself is charging and is ready to swing and then you have the climactic release of the sound which is the loudest point and then it dips in volume after that. Another great example is the Spartan laser from Halo and you can see just visually, it has a lot of tension right after the actual sound plays and if you notice, just before that really loud point where it's a lot more stretched and big there's a little dip in volume there that's something that sound designers do as a little trick to make you feel like something's a bit more powerful is get rid of the sound and then add the sound again with a tiny little gap because then that little gap in volume where you don't actually hear any sound actually then emphasizes it even further and sounds a lot bigger and as you can see via this little graph there's a lot of bass if you look on the left-hand side so that also signifies a lot of power another thing to consider as a sound designer is something called micro and macro sound design so micro is like the details of each individual sound effect and macro is the big picture of the game we'll get a little bit more into that now micro sound design, so things to consider with micro sound design is when you're basically designing one individual effect and lots of them is you need to consider the animations that are in the game like the Spartan laser, it's quite a sci-fi weapon it's really really big, it's almost as big as the player it's a lot more powerful and the same with the sword prior to that, it's absolutely huge so you need to consider what's going on in the individual animations what the little elements are, is it like fire, does it need bass is it one of the more important sounds and then you need to consider what information it conveys to the player which is very much like the charge up sound where it ramps up in volume and pitch and then also the same with the Spartan laser the big laser weapon there that also has to convey a lot of bass and then micro sound design that's a big picture of how all the sound effects actually fit into the game itself and as a result of that not every sound needs to be impressive because I've found that if you make every sound impressive inside a video game then nothing sounds impressive which is basically like listening to the sound of like a Michael Bay film where it's all just loud consistently for about two hours and things to consider micro sound on micro sound design is yet again pitch, frequency and volume and how they relate to every individual sound effect because then that comes back to the story emphasis of like you can apply these things to what's important to the actual game itself so yeah considering the game's design like what is the function of the actual game if you're looking at something like Resident Evil that game is very linear in storytelling you start here, you progress through the game and it ends here whereas games like Rocket League there's a bit more emphasis on it's basically if you haven't played Rocket League it is basically you're playing football with cars and they have to score goals so that's very different to how Resident Evil does it which is how over time there's like a story that's linear with Rocket League it kind of depends on like can you score the goal and the more significant sounds are are you winning or not so let's use an example of Resident Evil and let's listen to this door sound what's the role of this door it's basically just to sound like a normal door it's not really designed to be attention grabbing because in the game itself like you hear this door all the time and it's everywhere so it doesn't need to be important it just needs to play the role of a door which is fantastic and then next up let's listen to this door which often appears in the game Resident Evil as well this is in Resident Evil 8 Village now that's actually not the original sound that's actually one that I recreated using items in my house which I'll go into a little bit more detail into later so what is the role of this door it's actually really defining point in the game that door specifically appears after you defeat a boss or have an encounter with a boss in the game so it's kind of a defining point it lets you step from one chapter of the game to the next chapter so it's actually kind of telling the player that you're about to move into the next location so it appears a lot less often than the other door so if you really designed like the absolute hella of a normal sounding door it would just kind of get annoying over time but if you do it to like a game if you do it to a sound in a game that's like significantly more important then you can really just fill out the spectrum and make the sound really really good and have a lot more defining characteristics and this door in itself in the game is a release from tension because it signifies the end of a chapter so it needs to be a big dramatic moment and that moves on to realism versus hyper realism so what is realism realism is basically like the first door I showed you before so it's kind of just taking source recordings and that's basically like if you hear a door in a game someone's probably gone out and just recorded a door it's basically as simple as that and it grounds the game in reality because we all kind of know what a door sounds like so you don't really need to do anything particularly special with it basically it goes back to my earlier point which is not everything really needs to sound cool what is hyper realism this is where we exaggerate reality to kind of signify what's important to the story and the player so we make it cinematically bigger and it provides a lot more audio feedback so you can hear what's actually important what stands out because then that lets the player know what's actually important to them let's use an example sword sounds so just grab a little clip from Skyrim with the original sound of a guard pulling out his sword and you can hear the crimes against Skyrim and her people swords don't really sound like that when they're in sheath they sound really really boring which I'll show you now that sound there is kind of to kind of like exaggerate the sword sound because it's really important to the player to know that they've literally just gotten in trouble for stealing something from inside this guy's pocket and this is what a sword actually sounds like when it's in sheath I suddenly feel a lot less threatened by this guy so this is why it's kind of important to emphasize sound effects so going to the source this is basically like I said with recording actual doors for doors or whatever that kind of objects so we need to make sure that sounds in the game relate to actual objects in the game because that grounds the player back into reality and sounds need to be believable but also fun and engaging so then we need to pick where do we emphasize things to make them sound more impressive so we look at the visuals to kind of determine what's going on but we look at the visuals to determine what items to use to record sound effects how we record them and use them and also why here is an example this is a clip from a game called Black Myth Wukong and I'm going to play it without sound kind of let you kind of think about what we could use here and we'll listen to it again alright so first up you might notice that the character's carrying a staff so the first thing we need is handling sounds basically the character is moving that in the air and he's also grabbing it so we need to show that it is spinning and that there's a spin attack on the ground we need to exaggerate it and also do the sounds of it dragging across the ground so for this I used a baseball bat basically slapping the baseball bat in my hand a skipping rope and spinning it really really fast in front of my microphone and then grabbed like a snowboard and dragged it across ice and dragged other things across ice together now they don't need to sound particularly impressive together at this rate because we haven't finished yet so the next thing is the whoosh sound for when they spin it in the air and like they drag it across the ground there's a lot going on here because you might have noticed that the staff itself has flames on the end so a friend of mine actually uses a contact staff which is literally a staff with flames on the end so I went into his garden and I recorded him swinging in front of my microphone and trying not to catch it on fire and then also holding it static so I could just hear the flames and record those for when there's actually flames on the ground and then for the whoosh this is talking about like the hyperrealisms of exaggerating a sound I used the sound of a jet fighter flying by and then I used like a gust of wind from a blizzard and I laid it together and then created this next up we have the impact of the sound so this is when the enemy there is being hit with what is coming off that staff so we need to give the player feedback it's really really important to let them know that they've hit this thing because if they haven't and they've missed then they need to try again so we need to make sure that they know what's going on and we need to show the strength of the attack as well because this isn't like a regular attack they're not just hitting them with a staff they've done something magical so I laid together the sounds of meat basically just like slapping pieces of chicken together and slapping them with my hand because honestly I'm really surprised that people pay me to do this and then I used like leather jacket sounds so basically flicking it in my hand and then I got a bunch of snowballs when it was snowing and kind of slapped them against stuff to kind of exaggerate this and this is what this sounds like and you can see much like the the charge blade the weapon the big sword earlier it ramps up and then we have basically ripples of water that come off of the sound I don't really know why they're there but they are so I've got to work with the source recording and they have these ripples so they're like watery impacts with magic and stuff and I kind of just used snowballs there dropping stones in the river and also a bottle pop sound and I then pitched that up as it gets more aggressive toward the enemy you can hear that pop at the end there now let's talk about going away from the source basically not everything you can record realistically and not everything in a game is really a real life object so we need to find ways of tricking you audibly but they also need to be believable to the visuals and that's when my job kind of gets hard so we need to relate it to objects in a game so let's go back to the Resident Evil door clip that I redesigned basically I recreated the sound effects let's listen to the dust and debris falling from the door what is that sound I'll show you alright and let's move on to the little gears on this door as well see if you can guess what I used yeah so if you guessed like a little rack that you can cool cakes on then well done so sound design creation I've shown you a lot of just like using actual objects and stuff and they kind of just sound similar but there's a lot of editing involved in my job because a lot of what I've just shown you is field recording and Foley art which is basically what it's called but my job is a sound designer so I have to design sound effects one of the things we use is layering so we grab multiple recordings stack them together blend sounds together and then create one sound effect using multiple recordings so it's just a little quick example from my Instagram I do these little videos where I just do really quick cuts of how I create sound effects so let's use an example and show you a bit more detail so let's look at this dragon animation designed by animator Diana Schneider and I created that sound and let's see the little characteristics of this dragon so unfortunately you can't go to source recording for this and as a Welsh person I'm rather upset because dragons aren't real and if you look at the antlers they're facing away which kind of gives off a non-threatening vibe they have like cute eyes and not many teeth and they're really like vibrant and happy so we can't go for like completely aggressive recordings so what was that sound first of all there's a little bit of aggression because it's a big creature so there's a leopard in there there's a lemur there is the most rarest animal of all a rubber dog toy dragged across a window pane and a balloon so that's what it looks like layered together in my project that's all the sounds and now processing which is the next bit where we edit it so pitch shifting is a big thing in editing sound effects like I said earlier the highness or the lowness of the sound kind of impacts a lot of what's going on speeding up and slowing down sound effects like speeding up and slowing down a record distorting them which makes them fuzzier more aggressive and more warm and then EQ is basically frequency so like bass boosting or lowering or that kind of thing so let's have a look at the leopard first of all I pitch shifted it and I got rid of the low sounds on it and I know it doesn't sound very impressive now but layered together it's fine don't worry about it then I got a lemur recording and then I sped that up like speeding up a record or speeding up a YouTube video and then I pitched it down I grabbed the rubber dog toy and I did something called a formant filter which basically changes the vowel of a sound but we don't need guns for that right now and then I grabbed the balloon and there's a tremolo on it which is basically like tuning the sound on and off really quickly and then pitched it up it gives it quite a good vocal element processing extreme addition so sometimes you can use entirely the wrong sound to create something that clip of the spider at the beginning isn't the original sound I actually recreated that so let's have a listen and you can kind of try and guess what I used you won't get it right because I processed the hell out of it but hey give it a go what is that sound I'm sorry to say it was a it was a rubber chicken so why even have unique sounds in the game in the first place well the thing is that we need to create the illusion of a world of what the player kind of sees but also not what they see we need to create an illusion that we're actually in a real space and that's really hard to do with just sort of assets and sounds that are already pre-made that you can buy online so creating custom sound effects will then give that world a unique identity and actually give it some realism and people will believe what's actually going on instead of it being generic and having someone create custom sound effects like myself means telling a really authentic story because like all the details I've shown you earlier those are the things you need to pay attention to if you want to create a really immersive realistic to the game experience and each sound in a game tells its own individual story and you need someone that kind of emphasizes one of the most important bits and what aren't to tell the most effective story to people so that concludes my talk thank you so much for coming out to this feel free to reach out to me on any of these and honestly yeah thank you thank you so much cool thank you very much we're a few minutes ahead of time so if anyone's got any questions ah we'll go over here actually hold on I'll be lazy as well I'm still reeling from your statement that dragons are not real and actually as someone who once had a three meter long Komodo dragon walk through their kitchen unexpectedly I can also disagree with it who had a question I'll catch fortunately I can't actually hear you because the speakers are facing away from me so if there's any little fix maybe we'll come up with questions oh yeah the inverse does sometimes happen and the only time that's ever actually happened is because games I've worked on are forgotten to hire an animator so basically it's when they haven't hired them and they kind of just give me a brief so they give me concept art and stuff to kind of create stuff too later so yeah it does happen but it's often because the pipeline of an actual game is in the wrong order what's your what's your favorite tool for editing the sound and doing all the mixing what's my favorite tool for for layering the sound and mixing and all that oh so I used a workstation called Reaper and it's really really good and really easy to make sounds for games in that so yeah it's really just a case of dragging them in and layering them together so it's not too complicated and then when it comes to actual effects and stuff I use killer hearts quite often they're really really good plugins for like editing sounds so when recording sounds how much of the environment that you're recording them in matter you know because you have sound boos and things for voiceovers usually does reverb and that kind of thing matter and treatment matter much when you're actually recording the sounds or does it because you're processing them it does matter oh so you're saying about does like reverb often impact the sounds yeah so would you like would you specifically record in a kitchen or in a bathroom or outside for example what does it matter it can it can't it's really dependent on the game and the sound effects themselves often with reverb we often don't really have it in the game and then create it inside the game's world itself by programming it in to create a realistic environment and say if there's footsteps and you're walking through a hallway then you need the sound of reverb in a hallway as opposed to if you're in a like a church and the sound is different we usually just add it later if we can get away with it when you were growing up were there any particular games that really inspired you to get into this yeah definitely Halo early on and also the game Streets of Rage on Sega Mega Drive was fantastic because I played that at the age of 5 and absolutely loved the sound effects in that question how do the sound how does the design of sound effects and the design of a games music interact I imagine they work together for an overall sound how does the pipeline work there's a lot of that and you have to consider like the pitch of every sound in the game in comparison to the music like as an example I know that the sound effects in the game doom are in the key of D and so is all of the music and from what I can gather the reason that they did that is because the letter D is for doom and that's the only explanation that they've ever given so yeah they're all in D so yeah you have to consider pitch and how it interacts and if you're doing like a power up sound it needs to match the music that fits the actual game itself I work really really closely with composers in games when I'm actually creating stuff to make sure that we're not causing any clashing in that kind of retrospect I'll just go over here first so I notice you use a lot of common household items to create sounds how do you search for new items that make interesting sounds so basically I go to lots of junkyard sales or I happen like often like I'll go for a run in the morning and someone's throwing something out and then I'll just think oh is that kind of cool and then I'll go grab it for instance when I was going for a run the other day there were two bread bins sitting on someone's like lawn and because I'm a sound designer I went up to them and I kind of tested them and I was just like cool I'll just get the squeakiest one and then I took that home and then just recorded it in every way possible and then I had just like a library of my own sounds and basically I just collect random stuff and record it and see what happens and eventually you end up with so many recordings that you don't know what to do with them and then that's when you kind of figure out what you can do there and you just basically just search for stuff all the time and just see what happens with it I have a question on a filter that you used in these slides you showed like a parametric one I didn't catch that or understand which one was it like for modifying the balloon sound or the duck over the window the pitch or something I didn't get what that effect did not only because of feedback I couldn't actually hear that could someone come up and repeat it for a chance was it about if I understand correctly about which parametric filters you were using is that correct they were together so like what mixing them which filters do you use so filter I just use reapers like stock one to get rid of the frequencies but like some sounds yeah I like cut the bass on them others I'd like boost the highs it just depends on what layer I think actually matches the characteristics of the actual dragon considering that it's like animated and stuff then I need to kind of base boost the more fake layers like the dog towing the balloon then the actual real life animal itself to basically match the animated feel but then like with what filters and stuff do I use yeah I just use like the stuff that comes with my my workstation which is Reaper it's just a free one we've probably got time for one last question here and then hi possibly a bit of a personal one how do you categorize your field recordings how do you saw all the amazing things that you've recorded all the files you've got into some sort of organizational structure that you can then go back to so thankfully like the handy people over at Skywalker sound who do all the Star Wars stuff literally have like a guide on how to do it so basically just copying them and using like words to describe it like does it sound bubbly does it sound electric does it sound you know just all these little details could it be used for a flame you know that kind of thing you just add little buzzwords all together so then when you search for them the right stuff comes up and then other times you totally forget to do that and you have about 30 gigabytes of recordings that just don't have names and just have numbers and then you have to just sit there and decipher like oh what was it that I recorded where is this what is this thing and then you just sit there and then just question your existence for a good half an hour cool okay thank you very much indeed thank you very much for the questions