 Hello, welcome to SSUNI Tech Sushil this side and this is continuation of PySpark tutorial. So today we are going to start with the date time functions. Those are available in PySpark. So this is part one of date time function. So here the agenda is first we will see what is the current date function. Then we will see about the date format. Then to date current time stamp to time stamp and how we can get the hour minute and second from any date. So let me quickly go inside the browser and we will try to see in practical. So here the cluster is up and running. Let me try to execute this cell for creating this DF data frame. So this DF data frame should be having total five columns name, math, physics, chemistry and the passing year. Next let's see what is the data type of all these columns. So let me try to execute for this DF.schema. So here we can verify like the name is having the string data type and then we can see math is long type, then we can see physics as long type, chemistry as long type and the pass year that is again the string type. So the first thing we are required to convert this string type to date type. Then we can apply the date functions. So how we can convert? So for that let me try to add a new column. So for adding a new column we can use the with column option and in with column option first it is asking the string name that we want to specify. So the name could be new pass year and what expression we want to use. So we want to convert this into date format. So we can use the to date and inside that we have to specify the column. So for which column we want to convert. So we want to convert on pass year. So simply we can do and let me replace the existing data frame that is DF. Let me use the display of this DF. So we will see the output of this. So here the new column has been added and if we can expand and check this data frame data types. So for the new pass year that is date. Next how we can get the current date and time. So for getting the current date and time we can use the DF dot let me use the select first let me select all these columns and then let me use the current time. So that is the current date. So we can specify this column that is the current date and use the display of this. So it will be going to add a new column and that is the current date. Let me try to scroll down and this is the current date we are getting. So today 2nd of March 2024 so the date is coming like this. Next we have to use the date format. Let me use DF.select. I am going to select all these columns first and in the second column I am going to use the date format the second function. So inside the date format it is asking the column name first and then what is the format that we want DF.new pass year. So this is the column name by which we want to do the conversion and here we have to specify the format. So the format that we want to like yyy then maybe pipeline as a separator. So we can specify like this and then like that. So here yy should not be in caps it should be in a small and only mm should be in caps. Now let me try to execute this cell before that let me use the display so we will see the output of this. Let me execute and we will see it should be added a new column and the format that we are having as pipeline separator. What was the separator you are going to specify here maybe with the question mark it will be going to specify in the output as well. So here the separator between year, month and day as question mark. Next we can use the current time stamp. So for that let me comment out this cell and here let me use the DF.select and here let me use the as tick for all these columns and then we want to use the current time stamp. So current underscore time stamp. So this function we can use for getting the current time and date both it will be written. So let me execute and we will see the output of this. So here we can see we are also getting the time part earlier we were seen how we can get the date by using the current underscore date for getting the current time stamp we can go current underscore time stamp so it will be also adding the time there. Next let's try to convert this new pass year column as a date time as of now this column is only having the date part but we just want to convert this into the time part as well. So how we can do that so for that let me use DF.select let me select all these columns first then let me use to time stamp. So this function will help us to convert your date type column to the date and time type of column. So DF.new pass year let me use the display of this. So we will see the output of this so it will be going to add a new column and that column could be having date and time that we can see it is added the time part as well the time part as of now 0000 because here we don't have any time so while we are going to convert this into the time it is going to add as 0000. Next let me try to get the R minute and second from this newly added column. So let me try to put a alias name of this so could be the alias of this as new time let me execute this again so the alias name of this as new time and instead of display let me replace this existing data frame that is DF so we can execute and will be going to create a new data frame and if we can check the data types so for the new time the data type is time stamp for the new pass year the data type is date. Now let me try to extract the R minute and second from this so how we can do that we can go with DF let me go with select and here let me select all this column first then let me try to use the R and here we have to specify from which column we want to get it so we can specify like new time so as of now the new time is having 0000 so it will be going to get zero but if we can use the current time stamp so here we should be having some time so we'll be going to get the date from there let me use the same let me paste it here and instead of R we can specify as minute for getting the minute and the third we can specify as second and let me use the display of this so we'll be going to see the output it will be going to add three additional columns if we can scroll to write we should be seeing R minute and second we can also verify here if we are going to use the current time stamp in the select list as well we can verify let me execute and we'll see the output of this so we can scroll to write in the current time stamp we can see R is two so that's why R is coming as two minute is 15 we can see minute as 15 second we can see 5 so second we are getting as 5 so let's recap by using the current date we will be going to get the today's date by using the date format it will be going to convert any date data type into whatever the format that we wanna specify it will be returning on that format in today it will be going to convert your string data type into date type by using the current time stamp it will be going to get the current date and time next to time stamp it will be going to convert your date type column into date and time type column next we can see R so R will be going to get the R part from the date and time type column similarly for the minute and second as well so I hope guys you have understood how we can use all these functions thank you so much for watching this video see you in the next video