Get difference between two dates in days, years months and quarters in pyspark

In order to get difference between two dates in days, years, months and quarters in pyspark can be accomplished by using datediff() and months_between()  function. datediff() Function calculates the difference between two dates in days in pyspark. Dividing the result by 365.25 we will get the difference between two dates in years in pyspark. Months_between() Function calculates the difference between two dates in months in pyspark. Dividing the result by 4 we will get the difference between two dates in quarter in pyspark. Let’s see an Example for each.

  • Calculate difference between two dates in days in pyspark
  • Calculate difference between two dates in months in pyspark
  • Calculate difference between two dates in years in pyspark
  • Calculate difference between two dates in quarters in pyspark

We will be using the dataframe named df1Get difference between two dates in days, years months and quarters in pyspark 1

 

 

 

Calculate difference between two dates in days in pyspark

In order to calculate the difference between two dates in days we use datedfiff() function. datediff() function takes two argument, both are date on which we need to find the difference between two dates.

### Calculate difference between two dates in days in pyspark

from pyspark.sql.functions import datediff,col

df1.withColumn("diff_in_days", datediff(col("current_time"),col("birthdaytime"))).show(truncate=False)

So the resultant dataframe will beGet difference between two dates in days, years months and quarters in pyspark 2

 

 

 

Calculate difference between two dates in months in pyspark

In order to calculate the difference between two dates in months we use months_between() function. months_between() function takes two argument, both are date on which we need to find the difference between two dates in months.

### Calculate difference between two dates in months in pyspark

from pyspark.sql.functions import months_between,col

df1.withColumn("diff_in_months", months_between(col("current_time"),col("birthdaytime"))).show(truncate=False)

So the resultant dataframe will beGet difference between two dates in days, years months and quarters in pyspark 3

 

 

 

Calculate difference between two dates in quarters in pyspark

In order to calculate the difference between two dates in months we use months_between() function. months_between() function takes two argument, both are date and returns the difference between two dates in months. We divide the result by 4 to calculate the difference between two dates in quarter as shown below

### Calculate difference between two dates in quarters in pyspark

from pyspark.sql.functions import months_between,col

df1.withColumn("diff_in_quaters", months_between(col("current_time"),col("birthdaytime"))/4).show(truncate=False)

So the resultant dataframe will beGet difference between two dates in days, years months and quarters in pyspark 4

 

 

 

Calculate difference between two dates in years in pyspark

In order to calculate the difference between two dates in months we use months_between() function. months_between() function takes two argument, both are date and returns the difference between two dates in months. We divide the result by 365.25 to calculate the difference between two dates in years as shown below

### Calculate difference between two dates in years in pyspark

from pyspark.sql.functions import datediff,col

df1.withColumn("diff_in_years", datediff(col("current_time"),col("birthdaytime"))/365.25).show()

So the resultant dataframe will beGet difference between two dates in days, years months and quarters in pyspark 5

 

Get difference between two dates in days, years months and quarters in pyspark                                                                                              Get difference between two dates in days, years months and quarters in pyspark