 Well, it's that time of the week again. It's time for Chit Chat across the pond. This is episode number 778 for November 18th, 2023. And I'm your host, Allison Sheridan. Back after a normal hiatus that we cannot explain, our guest is Bart Boo Shots with Programming by Stealth, installment 155. How you doing today, Bart? Yeah, all right. If it wasn't for the fact that I'm sure we have noise cancelling, I was gonna make a blowing noise on my microphone to get the dust off, but I think that would probably just get noise canceled out so that you could fall very flat. We were talking about it before we started up is we take a break every year. It's unannounced and of indeterminate length at indeterminate time. It appears to have been our fall break this year, but it's been off and it's been our summer break, but we don't know what causes it. It just doesn't. We don't do programming by Stealth 12 months of the year. It appears to be about a nine month show. Yeah, we don't plan it. You might think, because I know podcasts that have a plan to summer hiatus and they lead up to it and they say it has three shows left to turn to summer hiatus and we just, yeah, we just vanish. We just go poof. How rude. Hey, it's our show and we're having fun. So we are back with 155. What are we gonna do today, Bart? So we're doing a little intermission. So I am waiting on the very final approval for what is going to be my first ever in my entire life as a working person, a pseudo career break. I will be taking seven weeks of annual leave consecutively, which is not something I have, I don't think I've done that since I was a child in school. Wow, I don't remember. So when that happens, I am going to basically deep, deep, deep dive into getting XK Passive D rewritten, but that's not now. That starts on the 15th of December if I get my final approval. So what do we do between now and then? Well, by pure coincidence, both my personal me and my work me have found themselves in great need of processing JSON files on the terminal. Work me because there's an awful lot of security related things, APIs and things that basically throw JSON at you. You query them and they give you JSON. And then if you wanna act on them, you need to understand that JSON. And for personal me, I am, actually no, I'm gonna keep my powder dry and what it is I'm trying to do. But I've ended up with a giant big database in JSON format that I need to process automatically because otherwise it will take me from now until the end of the universe. And I have discovered I know just enough JSON to swear very loudly at it and get about 80% of what I need. And you taught me and everyone else who was at Mac stock, or everyone who was at Mac stock because I wasn't, that if you wanna learn something, the best thing to do is to teach it. So we're going to learn how to process JSON from the command line because you can't use grep or any of those tools because JSON is far too structured. And that is that way madness lies. If you try to grep your way to getting data out of a JSON file, you will go insane because you'll always get the wrong answer. In other words, you tried that first. Of course I did because I'm regular expression person and that is the answer to life, the universe and everything except for when it isn't. And when it isn't, it's usually quite infuriating. So the JQ was written specifically for this job. That is it's one function in life. I would love to think it stands for JSON query, but officially it doesn't stand for anything. The website, the official website has no mention of what the name means. The official website has, in fact, nothing like that, nothing so frivolous. Neither does the Wikipedia page. So JQ is JQ. So I'm kind of surprised. We've looked at JSON several times and I happen to have just been working with HomeBridge and all of the configuration files are in JSON. And it's perfectly reasonable human readable file. I mean, it's plain text. It's got little bracket structures. It's got things in quotes and colons and it's very easy to read. I'm just shocked that it's hard to work with. I would have expected it to be trivial. Well, okay, so how do you find a value that's inside a specific name property inside the fourth element of an array using grep? Okay. I was gonna bring it up in a text editor and type command F is what I was gonna do. You have to do it a few more times. Shell script, right? Yeah, your shell script isn't gonna work like that. So to get at this, because the data's very structured. You describe it yourself, right? You have lists of things. You can have dictionaries, key value pairs. And it's all very structured. So you can represent very complicated data in JSON and retain all of its structure, all of its meaning effectively. But in order to pull that meaning out, you need a tool that speaks JSON that actually understands that this is the fifth element of an array which is inside, which is a key value pair and so forth. You need something that actually understands the JSON structure, which means you need a tool for the job. Now, JQ, despite having an extremely short name, is stupendously powerful. And its syntax reminds me a lot of my good friend, the regular expression, because you can say a lot by typing very, very little, which is easy on the fingers, but hard on the brain. And it takes a lot of understanding to write JQ and it takes even more to read other people's JQ, but when you get good at it, it is magic. You know how I love a dense language part. Don't expect any whining at all on me for me on that part, right? Nope, not at all. Especially not when Square Bracket has about five meanings. Oh, yay, I love that. So we're not going to do everything JQ can do because it is a full-featured language. It is a spectacularly powerful. But we are going to do three things with it, which are the three most useful things to be able to do. The first thing we're going to do is its simplest superpower. It can take unformatted JSON and make it pretty. So a lot of web service APIs and things will give you back your JSON as one giant big line. No space, there's nothing. I've seen that. And it's just this wall of text. Could you give me a character turn here or there? Help me out. Right, exactly. Yeah, so how in earth can you tell what's inside what, what the key value pairs out when it's just one big line of text? So you pipe it to JQ and you get back syntax highlighted pretty JSON on your terminal. So that is one of the most, one of the things I do with it more than anything else. It's like, what is this API telling me? Oh, okay, it's decided to wrap everything in an array named answer. Okay, then I'll just answer dot whatever it is I really wanted. Fine, thank you. The other thing you can do with JQ, the next most powerful thing is that you can dive into the structure. So, you know, go into the array, find the fourth element in there, find the key value pair called surname and then give me just that value. You know, so you can pick little pieces out. So imagine you have a config file and you're trying to draw a script that goes to the same folder. Well, how do you pull the folder name out of the config file? Well, if you know that it's under the key value pair called the home dear or whatever, then you can just say, JQ, go fetch me the value for home dear from this file over here and then it will read into it and pull it out for you. So just pulling out pieces of data is very powerful. And then magic and or insanity happen. The last thing you can do with JQ is you can actually take JSON as input and process it like you would at database with SQL and produce entirely new JSON that is pieces of the old one completely reassembled. Maybe you do some math so you can do things like take this giant big list of sales, collapse it down into my top five products with the average retail price. And it can do all of that math probably within like 10 characters of JQ. And so who sends that kind of data in JSON? That's kind of surprising. A lot of web services. No. There is, sorry, we had a freeze there. That was weird. That hardly ever happens to us. So what I asked was who, I've never seen that kind of data like sales data come in a JSON format. A lot of enterprise systems when you asked them to talk to each other, they used to talk in XML, but XML is evil. So now an awful lot of them talk in JSON. Oh, I didn't know we hate XML now. XML was a good first step, but JSON is better. Okay. So yeah, so a lot of things that need to talk to each other that are written in like five different languages, they're common. The lingua franca of the enterprise is JSON these days. So you actually do get a lot of JSON data floating around. And also a few of them are my favorite phrases I ever learned from a linguist is lingua franca. For anybody who doesn't know what it means, it means common language. It's the common term. A great phrase. Yeah, so in science at the moment, the lingua franca is English. It used to be Latin. Thank goodness it's English now, because my Latin sucks. It's a little bit easier. By the way, since the freeze, we are starting to step on each other. I think we got out of sync a little bit. So I may raise my hand to Bart to get, so I don't keep stamping on him. And Zoom may correct itself. It may figure it out. Okay, so, yeah, so basically we can make it pretty. We can pull out pieces and we can do a full-on transformer jobby. And we're gonna look at those with three common use cases. The most common places we as regular folk are likely to bump into JSON. So one of the places is in those wonderful free web services APIs that we played with a lot in our JavaScript days, where we did our currency converter. That was a bunch of JSON data that came back. When we were querying about our IP address, that was JSON data. When we were getting that cool weather API, that was JSON data. So a lot of web services took JSON. And on the terminal, that means we take curl to get us the stuff from the web, and then we shove it through JQ, and then we can get answers out. So we can take the weather. Curl is the command line tool for doing an HTTP request. Okay. So you say curl URL, and then you get back to data. Yeah, it has, it has been a while, cause that may even be taming the terminal territory, maybe crossing our streams a bit here. So that's one thing where it's likely to do. The other thing we're lucky to come across is reading config files. Because if you're trying to automate something that's part of a bigger system, if you can read the config from your shell script, then you don't have to remember where it is you put things. You can go read it from the config file, where you put things. What password is on that database? I know the config file knows, I'll get it from the config file. And then the last thing is there is actually, if someone wants to publish a free database, and they don't want to run a database server and have to pay a big server from, they just want the database as a single file. You could make it an Excel file, but the problem is you're now in a proprietary thing, only a few programming languages can handle Excel. A lot of these things are published as JSON, because it's completely open. So if you have a giant big JSON file, you're going to need a way to query that giant big JSON file and actually pull out the information you want, because that's kind of what you want from a database is to pull things out. So there are the three use cases. So three features, three use cases. And that's what we're gonna focus on. So a JSON file can be considered a database? It's a bunch of information in the format. Yeah, so if you, you're gonna, okay, so this installment has some examples in its resources folder. And one of those examples is a config file. The other example is a JSON file that it contains a database of every Nobel Prize ever given. Ooh, so I can go learn how to query for Andrea Ghez. She must be in there. She absolutely must be in there. Yes, you absolutely can query for her and figure out, because it has the citation, so you can figure out what wording it is that they gave her the Nobel Prize for. And it also tells you what share of the prize. Did she get a third, a half, a quarter? Because they're not always even e-divided. Oh, yeah, I got it. I definitely am gonna want to do that. Yeah, there you go. So yeah, it is, I mean, it is information in a structure that can be queried. So that is a database. It's not a relational database, but it is a database. Ah, okay. Okay. Okay. All right. Okay, so a little prologue. Remind ourselves of some JSON. So many, many moons ago, I did a blog post on it on BartB.ie when I still use to write blog posts. So that's linked in the show notes. There is a nice tutorial from Fried Code Camp that I've also linked in the show notes, but honestly, it's quite straightforward. So everything in JSON is one of six things. It can be a number. 42 is a number, 3.1415 is a number. They're just written as is. You can have a string in JSON. They are written in double quotes. JSON doesn't give you this luxury of choosing your quote. No, no, no, in JSON, it is a double quote. You can have a Boolean, which in JSON is T or U E, or F-A-L-S-E. No quotation marks or anything just true or false. You can also expressly say nothing with the keyword nul. So you can have a JSON file that says that the, I don't know, the number of, I don't know. You can have something mean nul. That can be meaningful. It can be meaningful to say there is nothing. And then you can have an array or a list, if you prefer, which is square bracket, value comma, value comma, value comma, close your square bracket. So very JavaScript-like syntax. It's basically a subset of the JavaScript syntax. The other thing you can have is a dictionary, sometimes called an object. Open curly bracket. The key must be in quotation marks, which is different to JavaScript, followed by the colon and then the value, comma, key, colon, value, comma, key, colon, value, close your curly bracket. And you can nest anything inside an array, including another array or a dictionary. And you can nest anything inside a dictionary, including another dictionary or an array. So with those six pieces, you can assemble basically any piece of information. And that's JSON. That kind of is all there is to it. Okay, good. So I have here an example of the Nobel Prize for Chemistry for 2023 from that data file. And so you can see it says key value pair year 2023, category, chemistry, laureates is a list of dictionaries. They're then indexed by who the laureates are. Each laureate has an ID, which I guess means if the same laureate wins multiple years, oh, Einstein won a few times. He probably has the same ID each time he won. Oh, we could find out when you're done teaching us. We could find out it's in the data file, exactly. So this is an array. And then the first name. You called that a dictionary? That's an array. Okay, so laureates is an array of dictionaries. Yeah, you said laureates was an array, or it was a dictionary. And I want to make sure it was an array of dictionaries. It's an array of dictionaries. So each laureate gets to be a little dictionary where they have an ID, a first name, a third name. Then there is the motivation for giving them the prize. So for Mugi Bawendi, who won the Chemistry Prize this year for the discovery and synthesis of quantum dots, they make great monitors, by the way. And yeah, so all three of them are the same. They are, and they also have the same share. They're each getting a third, a third, a third. So it's interesting, share is three. There are three shares, and they got three. That's interesting. I think it's a fraction without the one over. Looking at the data set, it seems to be a fraction, but without the one. Yeah, so whatever you call that bottom side of a fraction, it has a name. Denominator. Sounds plausible, definitely a word. It is. In my math class. It's definitely denominator. Okay, good, good, good, good. Oh, the blindee in the blind on mathematics. Well, certainly take your word for that. Wait a minute, I'm not blind on that. I'm blind on, I'm bad at arithmetic. Let's make sure we keep that straight. Yeah, I'm bad at both. So how did I ever make it through physics? Anyway, I did, just about. Right, so that's a little primer on Jason. So now let's look at JQ. So the first thing is I would like to take the creator of JQ, whose name is in the show notes somewhere, and tell him that he's a bloody idiot for naming the terminal command and the language the same thing. He didn't like have the JQ command speaking JQL or something so that you could actually look up JQ and JQ and not get the wrong, like sometimes you wanna know about the command. Sometimes you need to know about the language. They have the same names. So it's like Googling for Apple photos. You look for help with photo. It's like, well, everything's a photo. I mean, it's the same problem. So to try save everyone's sanity, I am going to use JQ in a fix with font when I mean the terminal command and just the letters JQ when I mean the language. And also because this, I mean this extremely intelligent and wonderful human being who wrote this amazing language makes me mildly cranky because he didn't capitalize it. So it's lowercase JQ. And I checked the documentation. It's always lowercase JQ in both cases. So anyway, I've kept, I've kept, it's his language, he has the right. So I have kept it. But as I say, I'm using fix with font when I mean the terminal command. The other thing we should say is that JSON is much newer than the shells like bash or SH or even ZSH, frankly. So there is no built-in support for JSON in the shell. The JQ command is a third-party add-on. You know, it's a terminal command, not part of bash or SH or ZSH. It's a completely separate thing. You have to install separately. So let us, you know, we're going to have to do that. So we'll jump to that in a sec. The other thing is I have put a list of useful resources here in the introductory episode because I figured that's easiest place to refer back to them. So we have a link to the official JQ homepage. We have the official docs. We have a very nicely formatted HTML version of the JQ man page, which is very useful. The languages Wikipedia page is actually quite nice. It gives a nice summary of the JQ syntax. For now it's going to look like complete gobbledygook, but when you start to speak a bit of JQ, that Wikipedia page is actually quite useful. And the other cool thing is there's a thing called JQ Play at jqplay.org, which is a tester for your JQ syntax. So you have a text box at the top where you put your JQ syntax, a text box underneath that where you put your JSON, and then it gives you the answer. Oh, I love it. So yeah, that is very convenient when you're trying to figure stuff out. So it really helps you hone your syntax. So in order to play along here, you definitely need to install JQ. And if your operating system doesn't ship with curl out of the box, then you also need to install curl. The Mac does come with curl out of the box. Most Linux distributions come with Wget instead. So if you're on Linux, you should be able to get both JQ and curl without any problem using your standard package manager. So if you're one of the Red Hat-like Linuxes, that'll be yum install JQ. And if you're one of the Debian-style Linuxes apt-get install JQ. If you're in Windows land, what you need is the Windows equivalent of Homebrew, which is something called the Chocolatey Package Manager. Oh man, I want chocolate. Yeah, they're getting behind chocolate. I mean, I like beer. Yeah, I like beer, but I prefer chocolate. Right, right, me too. So once you have installed Chocolatey, you can install JQ by saying, Choco install JQ minus Y. I guess if you leave off the minus Y, you'll have the type yes. Which is no, not the end of the world. And then for those of us in Mac land, Homebrew is what you really want to get JQ easily. So once you've installed the Homebrew Package Manager, you can say brew install JQ. And in Windows land, you definitely need to install curl because Windows definitely doesn't ship with curl. So that's Choco install curl minus Y. And then you're good to go. Awesome. So the JQ command, its input is going to be some JSON and some JQ syntax. And then it's going to give us whatever it is we asked for with the JQ syntax. So the input will come from standard in if you like. So you can pipe things to JQ or you can give it a list of files at the end of the command and it will read the JSON from the files. Okay. The, what they call the filter. So the bit of JQ syntax is called the JQ filter. That is the first argument. So the first argument is your search query, which is just the way grep works, right? You say grep what you're looking for, where to go look. JQ what you're looking for, where to go look. Okay. And there are some command line options, but we will leave those for now and we'll bump into those when they become useful. I have one question. I actually read this far in the show notes and no further. So I don't know where we're going to go, but why do I care about there being a language and a terminal command? Do we care about the language? Yes. Cause the language is how we say what we want. I thought we say that in the terminal command. Okay. So the terminal command takes as its first argument, the filter written in the language. Okay. That sounds kind of like the same thing. Like the, the really a distinction without a difference for us. Okay. So the grep command takes as its first argument, a regular expression written in Perl style regular expression, PCRE. Oh, okay. The JQ command takes the JQ filter in JQ. But had you never told me, I never probably would have noticed, right? Until you start Googling for things. And then it starts to tell you to use minus, minus, whatever, only you're, you want the other JQ where there are no minuses. Because the language doesn't have command line flags. That's the terminal command has flags. Oh man. I think I'm definitely confused on what the two of these things are. Maybe this will come out in the proverbial wash as we go through this. Well, yeah. So the filter is just a, what you want. So the, what you want is one string that is in a completely different language, right? That is in the JQ language. The JQ terminal command takes command line arguments and flags and options and all that kind of stuff. Oh, I see what you're saying. Okay. Yeah. Okay. So you just Google for JQ. How do I do whatever in JQ? Are they telling you how to make the terminal command do something? Or are they telling you how to express it in the language that goes into the first argument? Do they even know? Sometimes, yeah, anyway, it gets very confusing. The second thing that, okay, Stephen Dolan, that's the chap's name. Look, he's obviously a genius. This is a really powerful language, but he made a decision that I think he probably regrets himself. A very important piece of syntax in the JQ language involves the pipe symbol. The pipe symbol is rather meaningful on the terminal, meaning how we separate different terminal commands. Right. So the single most common way that people get completely tied in a mess with JQ is that they do not put single quotes around their JQ filter, because the filter will contain pipes a lot of the time. If you haven't posted your pipe and stuff all over and, oh no. Yeah. Okay. You think the pipe is JQ, but because you haven't quoted it, the terminal says, oh, I know that, that's for me. So always, always, always, always, always quote your filters. Okay. Because while there isn't a pipe in every JQ filter, if you don't get into the habit of quoting it, it is not a matter of if you will get a complete mess thanks to the pipe. It is a matter of when you will get a complete mess thanks to the pipe. So rather than have us just get lucky and screw up early using a pipe, you're telling us now, got it? Yeah. Just always, always quote it. And that way sanity lies. Okay. So for today, we don't want to get too heavy a lift, but I do want to show you the JQ command in action before we, before we call it a day here. So the first thing I want to do is show you how to make it pretty print. So there is a lovely free online API called free IP API that will tell you your IP address and some information about your IP address. So the URL is htps colon slash slash free IP, free IP API.com slash API slash JSON. So if you just curl that, you will see the raw output. So if you say curl space that URL and it's all on one line, it's just as one big line of glob. Yeah. So what if we want it a bit prettier? Well, if we take that curl command, now the minus S flag in the curl command is for silent. If you don't use the minus S curl, we'll show you an ASCII progress bar while it downloads the JSON, which is very annoying. So I hate that ASCII progress bar. So I just use curl minus S all the time. And then you pipe it to JQ with no arguments because we don't want JQ to filter parts of this JSON. We just wanted to show it to us. And if you give JQ no arguments, you just pipe something to it, it will pretty print it in color. Oh my gosh. You can use syntax highlighting. It's beautiful. IP version four. It's beautiful. IP address, blah, latitude and longitude. Hmm, wonder where that is. Yeah. Yeah. Now I did this on my server. I was about to do this on my home Mac and I was like, I'm not sharing that with the world. So that is the server that hosts part B.ie. I was in the Netherlands, which is the potato center I chose. I was thinking of just turning on a VPN, make it more interesting. That would have done it too. Yeah, they could be anywhere. Where is Alison now? Oh, Tasmania. Yay. So immediately. Yeah. Immediately you can see the power of this. When an API gives you back garbage, just pipe it to JQ and don't even bother with any arguments, just pipe the JQ and hey presto, pretty JSON. So that immediately is a win. Yeah. Straight away we have a win. We can also pull some information out of a config file. So the, I took the package.json for the website this-time.me, which is in GitHub. So it was easy to get to. I open sourced it. So the npm package.json file is sitting in the folder here. I've renamed it so that things don't get weird inside our Git repository for the show notes. So I've just called it this-time.me-package.json, but it's just a package.json file. And so that contains information about the this-time.me npm project. And so what if I wanted to know how many dependencies does that project have? Well, I can use JQ to pull that information out by giving it the filter inside single quotes. Dot dependencies pipe length. So that will find the dependencies key inside that.json file. And that the dependencies is an array. So I can then pipe that array to JQ's length function, which will tell me the length of the array named dependencies. So wait a minute. So pipe does mean pipe inside JQ. It's just not the pipe that the terminal command is allowed to use. Right. I thought it was gonna have a totally different meaning. No, it is how you chain parts of the JQ pipeline together. So it is still for chaining parts together. So yes, you're right, it doesn't have the same meaning, but if you don't quote it, bash will be off with it instead of JQ. And bash also has a length command, but that's the length terminal command that is completely different to JQ's length function. Didn't you tell us that we had to use double quotes? No. So double quotes are if you have a variable that you want to expand the variable. So a double quote is an interpolated quote. Okay, well, we'll come back to that then. Okay. Well, if you go back one episode, we recorded months ago, the last thing we did was a table summarizing all the different kinds of brackets. So if you're ever wondering what the difference is in bash between a single and a double, because they both have their uses, then that table will tell you straight away. Okay. And that's the same, because we are sort of in bash. JQ is a terminal command. Well, no, JQ is a terminal command. So this is a bash command. The first argument is the string dot dependencies pipe length. Okay. Okay, so what does dot mean? Okay, so inside the string, we are now in JQ LAN. So I'm going to say park that until next week, because this is a JQ filter. It does something useful. It contains the pipe. It's a nice example. Okay. The meaning of that JQ is, well, it's going to be the future. It's going to be the focus of our next quite a few installments, actually. Okay. So you've asked it to tell us the dependencies. Oh, do you have to spell dependencies? How many dependencies are there? Do you have to spell it correctly? Yes, otherwise the answer is probably zero. It is. So I was going to ask you, do you penance? Okay. Geez. I'm just going to copy it. Because I got zero and I was thinking, what am I doing wrong? Yeah, I think it's 12 if memory serves. Yes. Now it gets 12. Yeah, I had an E. I where an E should go. Good, good. Okay, so we gave it JQ, which is a bash command. And then we gave it a terminal command, so works a bash. And then we quoted our filter and we asked it for the length of the number of dependencies. And then we fed it the file that we wanted it to look at. Okay. Cool. I'll say it exactly. All right. So it's a terminal command with two arguments. And then the last thing is, how do we pull information out of a database? So the file, noble prices.json contains all of our information about Nobel Prizes. So what if we just want to see the 2023 prizes? Or actually, sorry, what is it I have here? I want to see the 2020 prizes because that's when Andrea won. Okay. Park that one for a while, we will get there. So this example command shows us the first prize in the list. It's basically very simple syntax, give me the first prize. So it's JQ with the filter dot prizes, open square bracket, zero, close square bracket, which is the first prize in the array of prizes. And that prints it out nice and pretty, the first prize, which is the chemistry prize, I guess, the most we do in them alphabetically. Okay. Okay, because the winners are in a dictionary, we've said, I'm not a dictionary, in an array, we said we want the first one, which is zero. So it's JQ, quote, dot prizes, square bracket, zero, and then fed at five. Okay. And the colon following it in the show notes is not part of the command. I figured that out. Oh, yes. Yeah. I guess that's poor typing on my part. It's not in the fixed with highlighted style, but still. But it's closed. Yeah. So now I have a pretty list of year 2023, category chemistry, and the laureates are, and it gives the three laureates. Very cool. Now, I'm now going to blow your head slightly. So this is pulling information out. But remember, where we are going in this series is transforming the information. So we don't have to get back, JSON, and the same structure of what we put into it. So the very large and very cumbersome filter you're going to copy and paste without worrying about too much for now is going to take all the prizes for 2023, throw away almost all of the information, and just remember what the prize is and how many recipients there were. Okay. And so you'll notice it contains a lot of pipes. Yeah, it's a lot of pipes in there, right? That is a lot of syntax, a lot of pipes, select statement, double equal signs. There's a lot going on in that little bit of JQ there. It will all make sense to us in two or three installments time, but the end result is that we now get an array of dictionaries that look nothing like the original. It just has two key value pairs, prize, no recipients. Neither of those two key value pairs were in the original data. We have built new data based on the old data and we've applied math. Oh, cool. Yeah, that's interesting. Quite powerful. Yeah. So we have processed a database. Yeah. So here we are. Right, so the last thing we're going to do today, which is still an easy lift is we're going to learn all the little, gritty details we need to know about making our pretty JQ a little bit prettier. So there's nothing difficult here, but sometimes you want slightly different to the defaults. So this is another excuse for me to remind us of how cool the free wttor.in web services API is, wetter.in, I guess. I think of it as wetter as in more wet, but I think it's supposed to be weather. But you know, I'm an Ireland. And so that will return to you by default, actually it returns to you like ASCII art of the weather, which is just cool because it's ASCII art of the weather. But if you say question mark format equals J1, you get JSON for your current weather. And so if you do a curl minus S and you pipe that into JQ, you can give it the JQ filter dot current underscore conditions zero, which gives you the current conditions this hour, like not next hour or the hour after. So basically the current conditions right now is current underscore conditions zero. And it will print you them out in syntax highlighted pretty form. And it's does that clever thing where if you pipe it to another terminal command, it will do it in plain text. But if you pipe it to a terminal, it will do it in pretty colors. But what if you don't want the pretty colors? What if you wanted to output to the terminal completely vanilla? Well, then you can use either minus minus monochrome minus output or minus capital M. One of those is shorter to type. Where is this the weather for? Is that here at my house or at your house? Yes. If you don't put anything between the forward slash and the question mark, it uses your IP address to determine your location. If you do put something between the forward slash and the question mark, you can get the weather there. So if you pop in Paris, you can see what it's like in Paris or Amsterdam or whatever, or Dublin if I wanna see what it's like here. Dark, wet, cold. I am. All right, let's see. I try that a silly way. Okay. All right, I've got the same thing, but it's not as pretty. So that's the boring monochrome version. Got it? Okay. Yeah. Now, if you're doing something with that, Jason, so imagine you spent, you know, 500 characters of JQ transforming your data a million ways from Sunday and you wanna send that data to another program, maybe you do not want it pretty printed. Maybe you actually want it all squished together. The minus C flag or minus, minus compact, minus output will put it all on one line for you. It'll squish it all back together for you. Okay. Can you combine? Can you make it minus CM? You can indeed. Then you get it monochrome and all squished together. Yeah. Now I can't read it at all. Perfect. Yeah. Which may be what you want, right? If you're using a terminal command to feed another terminal command, that may be what you need. The other thing you can do by default, it uses two spaces as the indent. What if you want a genuine actual tab character? Well, minus, minus tab will give you an actual tab character, which may be important for, you know, something you're processing with a regular expression or something. Or what if you just want like a little bit more space because two isn't more, isn't obvious enough? You can also say minus, minus indent and give it a number. And that would be the number of spaces. So minus, minus indent four will give it to you with four spaces instead of two. Oh, but by the way, you can only indent between minus one and seven. I happen to try eight. Minus one. What does minus one mean? I don't know. Let's see. Probably no indent. That'd be my guess. Zero, surely. No. No, that's odd. It actually looks like eight. It's one more than when I asked it for seven. That's some interesting binary math going on there. I just happened to get lucky to pick outside of the range. Yeah, and I guess a tab is eight characters by default, isn't it? Maybe. Yeah, I think it is. Interesting. Anyway, so that's all of the different ways you can have it pretty. But to be honest, I just use it with it arguments, which gives it to me two spaces and pretty. And that's usually exactly what I wanted to try to figure out what this weird API is trying to tell me. Because that's actually what you'd often want, right? You know this API is going to give you some data and then you want to program that data into something. Well, what are the key value pairs? The easiest way is just show it to me. But show it to me pretty so that silly human me can read it. So do you use this when you're trying to diagnose is the word that comes to mind, but you're trying to figure out what you're looking for and where it is and how it's going to look and then you're like, OK, now I know how to attack it to actually do what I need to do. Yes. Yeah, so with my work hat on, a lot of security tools when you query them will give you back a bunch of JSON and they'll tell you, call this URL to get the answer in JSON. What it won't necessarily tell you is what the key value, what the structure of that JSON is. So it's OK, great. I now have all of this JSON. How do I actually use this? Just show it to me. Just let me look and then I should be able to see what on Earth it's doing. And that's why I use JQ for a lot. Cool. I really like this because you gave us enough little fun at the end to give us the motivation. Why do we care? And not, but even though we didn't actually learn how to use it, we got a feel for, oh, look, that was cool. That's really nifty. And by the way, it's just occurred to me there's a reasonable chance that a percentage of the audience greater than zero listens to programming by stealth, but may never have heard my interview with Dr. Andrea Gez, the fourth woman in history to ever win the Nobel Prize in physics. So that's who I keep referring to if you've heard me say, oh, let me look up Andrea. That's who I've been talking to, talking about. So not that I'm excited about it, but you should really go watch that interview. Anybody who hasn't, you've got to go watch it. It's amazing. Look forward on podfeed.com. Absolutely got it there. She is an extremely good science communicator and clearly an extremely good scientist because you don't get a Nobel Prize for communicating. You've got to actually do the science as well. So there's that fascinating conversations. Yeah, absolutely fascinating. Well, this is good. Well, we're going next. So as you could probably tell by that final example where we constructed a whole different piece of data from the original data, it can get complicated. And I don't want to jump straight to complicated because your head will explode. My head will explode. It will be a mess. So we're going to build up to it by that middle use case, by the extracting existing information sort of with surgical precision. So that's what we're going to do first. And then after we've done the extraction, then we're ready for the transforming and the more advanced stuff. Because there's a difference between extracting and querying, right? Give me the fifth element in the array is different to find every element in the array which has a length greater than four. They are quite different things. And so that's going to be our progress, basically. Precision strike, query, complete and utter madness. That is the order we're going to tackle this in. And by building it up that way, I am hoping that the density, the denseness of the syntax is never overwhelming because we're going to build it like a little Lego fort layer by layer by layer. That's the theory. We shall see how it goes. Also, you need a little more time to learn the rest of it, right? I do. You have no idea. You know what took me longest in these show notes? Those few examples. Because I know just enough to nearly get what I want. And the amount of time I spent on that bloody Nobel Prize one, trying to just get it to count me the number of recipients was that was 40 minutes of my life. And a few choice words were uttered. Well, what do you always tell us? It's the practice that does it, right? So now you know it. 10,000 hours. Yeah. The other thing I want to tell people to appreciate is, Bart and I were talking about this in the back channel for the last couple of days, was he was struggling with how to tell the story. And I think that's what Bart really brings to this is obviously the technical knowledge. It's sort of like you had to be a good scientist to win the Nobel Prize, but you also have to be a good communicator. But it's the way Bart puts the story together that brings us along and helps us know why do we care? And then brings us into it. So I think that was time well spent getting the story down. This was great. I loved it. Yay! Good first episode back. Always nice to get back on the saddle. Without falling off the horse. Yeah, we're jumping over and landing on the other side. All right, there can't be a better place to end the episode than right there, Bart. Indeed. Until next time, happy computing. If you learn as much from Bart each week as I do, I'd like you to go over to lets-talk.ie and press one of the buttons over there to help support him. He does 98% of the work here. I'm just the stooge that listens to him and asks the dumb questions. If you go over to lets-talk.ie, you can support him on Patreon, you can donate via PayPal, or you can use one of his referral links. I really hope you'll go over and help him out. In the meantime, you can contact me at Podfeet or check out all of the shows we do over there over at podfeet.com. Thanks for listening and stay subscribed.