 What's up everybody, my name is Jon Hammond and we're moving into the grab bag for a hundred points challenge in the MITRE STEM CTF. So let's check out this challenge. It's called Journey to the Center of the File. The description is we have to go deeper. It looks like there are nested parentheses there. So this challenge is a doozy if you have not done something similar before or don't want to script it. Because you have to script it. You need to automate this process. We're given a download link so I will go ahead and copy this link location so we can W get it when we need to. Let's make a directory 100, Journey to the Center of the File. Let's jump into there and let's W get this stuff. I've been trying to be cognizant of the fact that in the last video I completely removed the command visibility because I had my face in the way. So I tried to lose a little terminator stuff on the side and hopefully I am not obstructing your view anymore. So we got a zip file. Let's go ahead and unzip this and we've got a flag file and some Mac OS X garbage that I would normally take a look at but I don't think is important. If I didn't already have a lead with something else I would go dive into that. But right now I think it's just necessary for what the zip archive is. Okay. So we've got this flag. So let's check it out at what it is and it is a B zip to compress data. Okay. So let's bunzip that. That's fine. Bunzip flag. I couldn't guess the original name for the flags using flag.out. Okay. So now we have flag.out and we'll check out what that is. That is a zip archive. Okay. So let's unzip flag.out. It needs to have a zip archive at the end. So we can unzip flag.out to flag.out.zip. And then we can unzip unzip not funzip that good. So now we have flag. That's BZ2 et cetera. So let's do that again. File flag.out again. Now it's ASCII text. Okay. Maybe that's our flag with very long lines. That's worrisome flag.out. This is base 64. Okay. So let's base 64 redirect that or and now we have something else here. Let's check out what that is. Let's go like something, file something. That's BZ2 again. Okay. So hopefully this has driven the point home, right? That we have a matrioshka doll or nested an inner and repeatedly compressed an archive or somehow manipulated data. So we have to be able to script this and automate it. So what I'm going to do is actually remove something and flag.out and flag.out.zip. Let's just start fresh from GB, blah, blah, blah. Yes. That's fine. And let's start now to script this. So I'm going to open up a new script. Let's call this matrioshka.sh and you can decide really what you want to write this in. I've seen some people try and do this in Python. I think that's what a lot of the guys on the Discord server, I know KVenom wanted to go for it and Sol and some others took a look. But I wanted to take this in bash and I got it done in bash before they did. So I think that's a smarter move in this case, but whatever. We have, let's say, an argument or a file that we want to work with. So let's say file name can equal $1 and that's the file that we're going to work with. So we had an issue where we wouldn't know what file name comes out of the file that we're working with when we extract it or when we work with it. So I figured, well, we can get around that by working in a kind of a dedicated workspace or creating a directory that only has the files that we're going to be working with. Let's create a directory. Let's just call it temp. Let's copy the file name into temp. Let's cd into temp and then let's run file on file name. Right? Okay. So let me now work with this. Let's run matryoshka with flag and it cannot open, oh, what are you talking about? What the heck are you talking about? Oh, temp is not a directory anymore. Let's make this script a little bit cleaner for us and actually usable to work with. So in case we have already created and make their temp, let's just remove it, which could be good or bad depending on what we need to do, but whatever. This is very hard to read and I'm sorry for it. Let's run matryoshka with the flag. So now we have the bz2 compression type. So we don't know exactly what it's going to be though. We don't know every single time what that file might be. So a technique that I've liked to do is to actually be able to set a variable depending on what the file actually is returned as. So I would also run file name or run file on the file name that we're working with and I would also use grep to determine is bz2 actually in this and then I would actually check out the value of the dollar sign question mark variable because that will tell us whether or not it successfully found something in that output or not. So if I were to grep for anything, that would return a one instead of a zero. So that way we can determine what is the actual result of what our file command is returning for us. So that we can test this, we can test if dollar sign, let's use quotations around this because I can never figure out what strings are supposed to be in bash when they're being tested or what these less than equals should be used, blah, blah, blah. Let's echo this is a bzip, great. So we know it's a bzip and we'll do things appropriately with that. Let's go ahead and actually do just some good practice. Let's move the file name to filename.bz2 and some of these like bzip2 doesn't require the file name extension, but some things like zip might when we get to that later. So now we can go ahead and bunzip to filename.bz2 and let's try this. I'm going to try to get out of flag, bzip2 looks like it did it. What is in temp? We have flag, let's file temp flag again. So I'm trying to, what I'm trying to showcase is that I would build this incrementally with the data that I'm actually trying to work with so that I know that we'll start to work through it and roll through it like it'll slowly process through and handle the file. So okay, it removed the bunzip to file, the bzip to file for me. So if I were to ever loop this or keep making this actually run through multiple Matroysk levels, I don't need to worry about removing the previous file because it already cleaned itself up for me. So that means now I can determine what the filename is again, no matter what it changed it to, just by simply because I'm in my clean working directory, ls everything or see what is in the current directory because that's going to be the next file that we're going to work with. And I'm using that same variable filename, so when I loop, I won't have to worry about that again. You know? Okay, let's tighten this up now and let's actually, I still want to be able to see what file I'm working with so I'm going to use while 1 now to start to loop this because if we ever get a bzip to again, we know how to handle it. Now we need to figure out how we can handle a zip file. So we've got this code block that we've just written to determine whether or not there's a bzip to in there. And if there's a bzip to in there, we'll do this, but now let's test, okay, if we run file on filename and there is zip in there, capital Z, capital Z is good for us because we don't want that case sensitive work with the previous grep. We can say this is a zip archive, great. So let's move that filename to filename.zip, get the appropriate file extension there and run the correct file unzip on it, good. And this does not remove from what I understand, like if you actually were to go into temp right now, and you try to do unzip, okay, let's get the correct filename in there, move flag to flag dot zip, let's unzip flag. Now we have flag. It did not remove flag dot zip. So if we were to LS everything and try and get the new filename, we wouldn't have it anymore because we've got some extra data in there. So let's make sure we remove filename dot zip, and now we can properly get the filename, good. Okay, so let's try this, right, let's see what we can do. I'll go back and run matryoshka on flag, be zip to zip, and now we get ASCII text, okay. So let's check what we've got in there if I move into temp. This is again base 64 that we've seen previously, right? I guess I didn't need that other terminal, sorry. So let's determine, okay, if we see ASCII in there, then what we'll do is we'll say this is ASCII. So I'm going to name this just dot text or something, or dot new maybe. It doesn't matter, as long as it has some extra extension so you can figure out what it may be. Then let's base 64, actually we don't even need to move this until we've decoded it because base 64 tack D on that filename, we'll need to redirect that to a new file because if we redirected the same filename it gets overwritten and it doesn't do anything good for us. So we've redirected to a new filename and we can remove the old one because base 64 won't get rid of that, we're just redirecting the output. So let's try that now, if I matroyshka the flag, bzip, ASCII, that's being processed, now we have gzip to work with, okay. So move into temp, check out what we got here, file flag dot new, it's gzip. We can gunzip that, right, hello, is anyone home? It needs to have the file extension. So this is good, this is good that we're testing it kind of as we go along, building up the script to work with it as we need, because this is kind of a simple process right now, right? We know that we have gzip, and if we see this, this is a gzip. So let's move that filename to filename.gzip and then we can gunzip that, let's actually do that, let's move this to this.gz, just doing it in the terminal so we can see how it works now. Let's gunzip that, and it removes itself for us, so we don't have to worry about that, let's just gunzip filename.gzip, perfect, okay. So what do we got now? Move back, run matroyshka on the flag, go, go, go, go, go. Looks like we're getting far along, far along. Looks like it's going through these iterations really well, this is good, carving, heck yeah, okay. This is new, some people, I don't know if individuals had to deal with this, if they used this route, but this is still the output from the file command. So I actually just tested this again, once I got into temp, move into temp, right? So we have this, it's telling me this is init, stat, etc. But if I check this out, it's still just base 64, so I thought, okay, let's just duplicate this ASCII one then, and then grep for init equals, because that's also just going to be doing the same base 64 operation it did before. That was a weird exception to the rule that I found, but if you weren't into it, that is how you can handle it as well. Let's run matroyshka on flag and let it go. Take some time here, but it is drilling pretty well, blah, blah, blah, blah, blah. I love how fast it gets, I don't understand how or why, like it was slow to begin with, but now it's just getting bigger. Oh boy, okay, so now we have got an empty file, like completely, completely rolled over. Base 64 kept trying to decode it, and now it's just straight up ruined, okay. So potentially we have data in there, actually if you actually are kind of observant, you can see as we look through our output here, we have ASCII text with very long lines, and then as we go closer and closer down, we have ASCII text with very long lines, ASCII text with very long lines, and then suddenly we just get to ASCII text, and it does not tell us it has long lines. So we know we're close. So now we probably have the flag in one of those ASCII text files, and that's something we've got to assume is because there's no other way we could see it in this case, because we aren't seeing any other different file types. So let's be cognizant of this, because now we know potentially in this loop, in this case specifically, we've got to be able to see the file, because otherwise our Matroyska, the while loop is just going to keep trying to remove this thing, and that's what it's doing right now, that's why we get to an empty file. So what we can do is actually test the value or try and display the value that we're seeing from the file name. Normally I would just tell you, oh, just cat the file, because that's all you need, right? But you've got to keep in mind, when we see a lot of Base64, that's going to slow us down, and it's going to look like garbage. So what I did actually was I tried to use the tail command to get the last couple lines from it. And again, this is going to be bad, because it's one very, very long line. So you've got to use tail to get the certain number of bytes out of it. So you can use taxi to specify the bytes, and then I get like the first last 100 bytes, 100 characters. Because maybe, just maybe, that we have actually, excuse me, sorry, I'm trying to get my thoughts here, the flag in there. And that's probably not going to be more than 100 characters. You can make this also very smart, if you wanted to test if, when you run file, if you run file on it, it tells you the file is empty, because if it's empty, then we have broken the thing. So what we can do is actually remove that. But you can see our flag is hanging out right there. If I just said those words and none of those made sense to you, here's what we can do. I hope that was visible, by the way, my tail taxi, 100 file name. I've got to be cognizant of the face cam. I did not duplicate this yet, but let's determine if it says empty. If grept empty, then what we can do is we can just straight up break. Like stop, stop looping. We don't want to do that anymore. So this is a kind of dirty but good utility, because you've got now a Matroysk solver or something to keep, keep unpiling those. And if you see something that's not ASCII text and you aren't getting the full data out of it, you're going to be warned with that file command. And you'll know what your loop, the script will keep telling you this is where you need to move on to next. But if it straight up gets to an empty file and completely like kills itself, that's OK, it'll stop and you can kind of review your output. If you do get something valuable like the flag out of it, that's displayed now because we're tailing that out when we have potential plain text. So that's that. That is the script that we can put together to solve a Matroysk doll and I just did that in bash. Yeah, it's dirty, but it works. And maybe it's clever or maybe it's smart. Maybe it's doing some interesting thing. That's how we can unpack that Matroysk doll. So all righty, let's save this. What I would recommend is actually saving this script in your home directory or in your tools directory or something like you should probably keep track of this because I think this is a good thing for us to write because we might see that again in later CTFs. You got to keep a repository. You got to keep your archive and your tool set of stuff that you'd like, but let's save this flag, paste that in there. We can mark that as a get flag script if we wanted to with Matroysk, but we have completed that challenge and we are ready to move on. So thank you guys for watching. I hope you enjoyed this. I hope it was fun. I hope it was cool. I'd like that script. I was kind of pleased with it. I had to help a lot of other people that were struggling with it, but you got to do it. It's got to bang your head against the wall and keep working at it. So if you did like this video, please do like, comment and subscribe. I would love to see you on the Discord server. I'd love to see you on Patreon or PayPal. I am super duper grateful for how engaged you guys are in the community. It's incredible to play these competitions with you. I love CTF in a voice chat on Discord. We just goof off and have a great time, learn a lot, and that's what's really important. And I'm so grateful for each of you that helps support and help the channel grow. Thank you. See you in the next video, everybody, later.