Convert to upper case, lower case and title case in pyspark

Converting a column to Upper case in pyspark is accomplished using upper() function, Converting a column to Lower case in pyspark is done using lower() function, and title case in pyspark uses initcap() function. Let’s see an example of each.

  • Convert column to upper case in pyspark – upper() function
  • Convert column to lower case in pyspark – lower() function
  • Convert column to title case in pyspark – initcap() function

We will be using dataframe df_states

Convert to upper case, lower case and title case in pyspark 1

 

 

Convert column to upper case in pyspark  – upper() Function :
Syntax:

 upper(‘colname1’)

colname1 – Column name

upper() Function takes up the column name as argument and converts the column to upper case

########## convert column to upper case in pyspark

from pyspark.sql.functions import upper, col
df_states.select("*", upper(col('state_name'))).show()

column “state_name” is converted to upper case as shown below

Convert to upper case, lower case and title case in pyspark 2

 

 

Convert column to Lower case in pyspark – lower() function :
Syntax:

 lower(‘colname1’)

colname1 – Column name

lower() Function takes up the column name as argument and converts the column to lower case

########## convert column to lower case in pyspark

from pyspark.sql.functions import lower, col
df_states.select("*", lower(col('state_name'))).show()

column “state_name” is converted to lower case as shown below

Convert to upper case, lower case and title case in pyspark 3

 

 

Convert column to Title or proper case in pyspark  – initcap() function:
Syntax:

 initcap(‘colname1’)

colname1 – Column name

initcap() Function takes up the column name as argument and converts the column to title case or proper case

########## convert column to title case

from pyspark.sql.functions import initcap, col
df_states.select("*", initcap(col('state_name'))).show()

column “state_name” is converted to title case or proper case as shown below

Convert to upper case, lower case and title case in pyspark 4

 

Convert to upper case, lower case and title case in pyspark                                                                              Convert to upper case, lower case and title case in pyspark