Drop Rows with NAN / NA Drop Missing value in Pandas Python

Drop missing value in Pandas python or Drop rows with NAN/NA in Pandas python can be achieved under multiple scenarios. Which is listed below.

  • drop all rows that have any NaN (missing) values
  • drop only if entire row has NaN (missing) values
  • drop only if a row has more than 2 NaN (missing) values
  • drop NaN (missing) in a specific column

First let’s create a dataframe.

import pandas as pd
import numpy as np

#Create a DataFrame
df1 = {
    'Name':['George','Andrea','micheal','maggie','Ravi','Xien','Jalpa',np.nan],
    'State':['Arizona','Georgia','Newyork','Indiana','Florida','California',np.nan,np.nan],
    'Gender':["M","F","M","F","M","M",np.nan,np.nan],      
    'Score':[63,48,56,75,np.nan,77,np.nan,np.nan]
    
   }

df1 = pd.DataFrame(df1,columns=['Name','State','Gender','Score'])
print(df1)

so the resultant dataframe will be

Drop Rows with NAN or Drop Missing rows in Pandas Python 1

 

Drop all rows that have any NaN (missing) values:

Drop the rows even with single NaN or single missing values.

df1.dropna()

Outputs:

Drop Rows with NAN or Drop Missing rows in Pandas Python 2

 

Drop only if entire row has NaN values:

Drop the rows if entire row has NaN (missing) values


df1.dropna(how='all')

Outputs:

Drop Rows with NAN or Drop Missing rows in Pandas Python 3

 

Drop only if a row has more than 2 NaN values:

Drop the rows if that row has more than 2  NaN (missing) values


df1.dropna(thresh=2)

Outputs:

Drop Rows with NAN or Drop Missing rows in Pandas Python 4

 

Drop NaN in a specific column:

Drop rows with NaN in a specific column . here we are removing Missing values in Gender column


df1.dropna(subset=['Gender'])

Outputs:

Drop Rows with NAN or Drop Missing rows in Pandas Python 5

 

Drop Rows with NAN / NA Drop Missing value in Pandas Python p                                                                                                          n Drop Rows with NAN / NA Drop Missing value in Pandas Python

Author

  • Sridhar Venkatachalam

    With close to 10 years on Experience in data science and machine learning Have extensively worked on programming languages like R, Python (Pandas), SAS, Pyspark.