In order to populate current date and current timestamp in pyspark we will be using current_date() and current_timestamp() function respectively. current_date() function populates current date in a column in pyspark. current_timestamp() function populates current timestamp in a column in pyspark Let’s see an Example for each.
- Get current date in pyspark – populate current date in pyspark column
- Get current timestamp in pyspark – populate timestamp in pyspark column
We will be using the dataframe named df
Get current date in pyspark – populate current date in pyspark column
Syntax:
current_date() function is used to gets the current date in pyspark. The current date is stored in the new column named “current_date” and thereby we will have the dataframe with current date populated in a new column.
### Get current date in pyspark – populate current date in pyspark column
from pyspark.sql.functions import current_date
df1 = df.withColumn("current_date",current_date())
df1.show()
Current date is populated and appended to the dataframe, so the resultant dataframe will be
Get current timestamp in pyspark – populate current timestamp in pyspark column
Syntax:
current_timestamp() gets the current time in pyspark. The current timestamp is stored in the new column named “current_time” and thereby we will have the dataframe with current datetime (timestamp) populated in a new column.
### Get current timestamp in pyspark- populate current timestamp in pyspark column
from pyspark.sql.functions import current_timestamp
df1 = df.withColumn("current_time",current_timestamp())
df1.show(truncate=False)
Current date time is populated and appended to the dataframe, so the resultant dataframe will be
Other Related Topics:
- Get week number from date in Pyspark
- Get difference between two timestamps in hours, minutes & seconds in Pyspark
- Get difference between two dates in days, years months and quarters in pyspark
- Populate current date and current timestamp in pyspark
- Get day of month, day of year, day of week from date in pyspark
- Add Hours, minutes and seconds to timestamp in Pyspark
- subtract or Add days, months and years to timestamp in Pyspark
- Get Hours, minutes, seconds and milliseconds from timestamp in Pyspark
- Get Month, Year and Quarter from date in Pyspark
- Get String length of column in Pyspark
- Typecast string to date and date to string in Pyspark
- Typecast Integer to string and String to integer in Pyspark
- Extract First N and Last N character in pyspark
