 Welcome to SSUnitex, so see this side and this is continuation of PySpark tutorial. So in the last video of this video series, we have seen about the array type array and array contains function. So before going forward in this video, I would strongly recommend to watch that video if you haven't watched. So in this video, we are going to see about the map type, map key, map value and explode function. So these functions are very important. So what is the map type function? So basically map type function you can assume if your data is having the key and value pair that is in terms of python dictionary. If data is in dictionary format then map type will be there. So in PySpark it is calling as map type data type for key and value pair. And what is the map key? So map key is nothing but the key part of that data and what is the actual value that is the map value and then explode we will see. So as per the today's agenda, first we will see about map type. Then we will see map key, map values and then explode function. So explode function what is that? So it is used in PySpark data model to explode an array or map related column into rows. So let's assume if we are having any column which is the array type or that column is the map type. Then how we can convert those key value pairs into the rows we can use the explode function. So don't worry we will see in this video. So let me quickly go inside the browser and we will try to see in practical. So here as you can see first I have imported all the data types and after that in the data you can see we are having the country is the first column. Second is the state and what is the capital of that state we can see over here. So for Uttar Pradesh capital is Lucknow, similarly for Bihar is Patna, for MP is Bhopal, for Delhi is New Delhi. So like that we are having the data and this data we can see is having key and value pair. So this data is in Dixen reformat. Similarly for the second row the country is US and then we can see state and the capital of that state. Now how we can design the schema for this? So for designing the schema first we are required to use the stuck type. Under the stuck field here we can see we have total two columns. First is the country column and second is the state and capital combination column in the map type data type. So here first you can see I have designed this as country and the data type of that will be string. Second we can see state capital and similarly the data type of this will be map type and what will be the key and the value. So all the key is in string as well as the value is in string. So that is why the data type we have defined as string. Here we are simply creating the data frame. Let me try to execute this and we will see the output of this. So now here as we could see we have already printed the schema. If you can see here we are printing the schema as well. So the data that we can see is having country, India, US then state, capital over here. So this data is in map format. So we can see the schema so country is string format and then state, capital here the value is in map and this map will be having again two columns under that key and value there and both is the string format. So here we have used about the map type. Here we can see the state capital. So this state capital is in a key value pairs. So next we just want to create two additional columns here. First for the key and second for the value. So how we can do that? So for that we have to use the explode function. So first for it using the explode function we have to import it from the SQL library. So SQL dot functions then I am going to import all the functions. Now here I am going to use the DF dot I am going to use the select and inside the select I am going to specify the columns. So first I just want to this country column. Second I want to this state and capital column. Now here we just want to add two more additional columns for the key and value pair. So how we can get that? For that we can use the explode function. So inside the explode simply we have to pass this state and capital. So this state capital is having key and value pair. So it will automatically generate two columns and those two columns will be key and value. So let me try to see the output of this. So this is DF1. Let me try to execute and we will see the output of this. So here as this is the case sensitive so C should be in capital letter. Let me try to execute it again. Here you can see we have not explicitly defined this key and value. But while using the explode function it is automatically adding these two columns one for the key and second for the value. And how we are getting this we are getting this from here. So for India we can see total four rows we are having because in the key value pair we have total four rows Delhi, Bihar, MP and UP. For US we have total three rows. So that is why we can see three here. Now so this is the main purpose of using explode function. So the explode function can be used inside the array as well. So let me go and try to use this map key and map value function. So what map key and map value function will do? If you are going to use the map key then it will be going to fetch only key part from this state capital column. If you will be using map value it will fetch only value part from here. So let me quickly show you that. So let me try to remove this explode function from here. Let me execute so it will have only two columns as we can see. Now let me try to use map keys. So this function we can use and under that we can simply pass this state capital column. Now let me try to execute and we will see the output of this. So it is having this map key state capital. We can also specify the alias of this. So let me use the alias and under that we can call this as key. Let me execute and we will see the output. As we can see here we are having this is an array of only key like Delhi. If you can scroll down, Bihar, MP and UP. So only key part is coming here. You can also use the explode out of this. So what explode will do? It will be going to separate these array into different rows. So let me try to use the explode here and you will see we will be having this into different rows as we can see Delhi, Bihar, MP and UP. So like that we can split it. Now we can also use the value. So let me use the values instead of this keys. So what it will do? It will be going to return only the value part that you can see only the value part are coming. So now we are having values. So if you are going to use the explode function it will automatically split your dictionary into two different columns one for the key and second for the value. Finally you can also use the only key or only value part. So first you can use the map value or map key for the value and key respectively and after that we can use the explode to split that into different rows. So I hope guys you have understood how we can use the map type, map key, map value and explode function. So thank you so much for watching this video. If you like this video please subscribe our channel to get many more videos. See you in the next video.