Convert to upper case, lower case and title case in pyspark

In order to convert a column to Upper case in pyspark we will be using upper() function, to convert a column to Lower case in pyspark is done using lower() function, and in order to convert to  title case or proper case  in pyspark uses initcap() function. Let’s see an example of each.

  • Convert column to upper case in pyspark – upper() function
  • Convert column to lower case in pyspark – lower() function
  • Convert column to title case or proper case in pyspark – initcap() function

We will be using dataframe df_states

Convert to upper case, lower case and title case in pyspark 1

 

 

Convert column to upper case in pyspark  – upper() Function :

Convert to upper case, lower case and title case in pyspark c2

Syntax:

 upper(‘colname1’)

colname1 – Column name

upper() Function takes up the column name as argument and converts the column to upper case.

########## convert column to upper case in pyspark

from pyspark.sql.functions import upper, col
df_states.select("*", upper(col('state_name'))).show()

column “state_name” is converted to upper case as shown below

Convert to upper case, lower case and title case in pyspark 2

 

 

Convert column to Lower case in pyspark – lower() function :

Convert to upper case, lower case and title case in pyspark c1

Syntax:

 lower(‘colname1’)

colname1 – Column name

lower() Function takes up the column name as argument and converts the column to lower case

########## convert column to lower case in pyspark

from pyspark.sql.functions import lower, col
df_states.select("*", lower(col('state_name'))).show()

column “state_name” is converted to lower case as shown below

Convert to upper case, lower case and title case in pyspark 3

 

 

Convert column to Title or proper case in pyspark  – initcap() function:

Convert to upper case, lower case and title case in pyspark c3
Syntax:

 initcap(‘colname1’)

colname1 – Column name

initcap() Function takes up the column name as argument and converts the column to title case or proper case

########## convert column to title case

from pyspark.sql.functions import initcap, col
df_states.select("*", initcap(col('state_name'))).show()

column “state_name” is converted to title case or proper case as shown below,

Convert to upper case, lower case and title case in pyspark 4

 


Other Related Topics:

Convert to upper case, lower case and title case in pyspark                                                                              Convert to upper case, lower case and title case in pyspark

Author

  • Sridhar Venkatachalam

    With close to 10 years on Experience in data science and machine learning Have extensively worked on programming languages like R, Python (Pandas), SAS, Pyspark.

    View all posts