IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

by

Sometimes while dealing with a pandas data frame, we might forget to handle null or infinite values and when we tried to perform some operation, we got IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer error. This error shows that our data frame contains nan or infinite values. The error is simple to handle. In this short article, we will discuss how we can solve the IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer errors using various methods. Moreover, we will also learn how to understand and interpret errors in Python.

IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer-error

IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

The error usually occurs when we try to perform some operation on the data frame that has Null values in it. The simplest way to handle and get rid of the error is to remove or handle Null values in the data frame.

Let us first see, how and why we are getting IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer error:

# importing the modules
import pandas as pd
import numpy as np
   
# creating the data frame
data = {
    'Name': ['Jack', 'Rack', 'Max', 'David'],
    'dic':[np.nan,85, 99,56],
    'Subject': ['Math', 'Math', 'Math', np.inf]
}
  
# creating the data frame
dfobj = pd.DataFrame(data)
  
# applying operation on the data frame
dfobj['dic'] =  dfobj['dic'].astype(int)
print(dfobj )

Output:

---------------------------------------------------------------------------
IntCastingNaNError                        Traceback (most recent call last)
/tmp/ipykernel_418062/386540908.py in <module>
     14 
     15 # applying operation on the data frame
---> 16 dfobj['dic'] =  dfobj['dic'].astype(int)
     17 print(dfobj )

~/.local/lib/python3.10/site-packages/pandas/core/generic.py in astype(self, dtype, copy, errors)
   6243         else:
   6244             # else, only a single dtype is given
-> 6245             new_data = self._mgr.astype(dtype=dtype, copy=copy, errors=errors)
   6246             return self._constructor(new_data).__finalize__(self, method="astype")
   6247 

~/.local/lib/python3.10/site-packages/pandas/core/internals/managers.py in astype(self, dtype, copy, errors)
    444 
    445     def astype(self: T, dtype, copy: bool = False, errors: str = "raise") -> T:
--> 446         return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
    447 
    448     def convert(

~/.local/lib/python3.10/site-packages/pandas/core/internals/managers.py in apply(self, f, align_keys, ignore_failures, **kwargs)
    346                     applied = b.apply(f, **kwargs)
    347                 else:
--> 348                     applied = getattr(b, f)(**kwargs)
    349             except (TypeError, NotImplementedError):
    350                 if not ignore_failures:

~/.local/lib/python3.10/site-packages/pandas/core/internals/blocks.py in astype(self, dtype, copy, errors)
    525         values = self.values
    526 
--> 527         new_values = astype_array_safe(values, dtype, copy=copy, errors=errors)
    528 
    529         new_values = maybe_coerce_values(new_values)

~/.local/lib/python3.10/site-packages/pandas/core/dtypes/astype.py in astype_array_safe(values, dtype, copy, errors)
    297 
    298     try:
--> 299         new_values = astype_array(values, dtype, copy=copy)
    300     except (ValueError, TypeError):
    301         # e.g. astype_nansafe can fail on object-dtype of strings

~/.local/lib/python3.10/site-packages/pandas/core/dtypes/astype.py in astype_array(values, dtype, copy)
    228 
    229     else:
--> 230         values = astype_nansafe(values, dtype, copy=copy)
    231 
    232     # in pandas we don't store numpy str dtypes, so convert to object

~/.local/lib/python3.10/site-packages/pandas/core/dtypes/astype.py in astype_nansafe(arr, dtype, copy, skipna)
    138 
    139     elif np.issubdtype(arr.dtype, np.floating) and is_integer_dtype(dtype):
--> 140         return _astype_float_to_int_nansafe(arr, dtype, copy)
    141 
    142     elif is_object_dtype(arr.dtype):

~/.local/lib/python3.10/site-packages/pandas/core/dtypes/astype.py in _astype_float_to_int_nansafe(values, dtype, copy)
    180     """
    181     if not np.isfinite(values).all():
--> 182         raise IntCastingNaNError(
    183             "Cannot convert non-finite values (NA or inf) to integer"
    184         )

IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer

As you can see we got the error because of the null values in our dataset.

In this short article, we will discuss the following three methods to get rid of the error:

  • Using the fillna() function
  • Using dropna() method
  • Using replace() method

Let us now go through each of the given methods and explained each of them.

Solution-1: Using fillna() method

The pandas have many useful functions that can be used to handle null values. One of the method that can be used to handle null values is fillna(). The fillna() method takes a parameter value and fills it with the given value.

For example, fillna(0), will fill all the null values with zeroes. Let us now use the fillna() method and use zero to fill the null values to get rid of IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer.

# importing the modules
import pandas as pd
import numpy as np
   
# creating the data frame
data = {
    'Name': ['Jack', 'Rack', 'Max', 'David'],
    'dic':[np.nan,85, 99,56],
    'Subject': ['Math', 'Math', 'Math', np.inf]
}

# creating a data frame
dfobj = pd.DataFrame(data)
 
# handling the null values and filling them
dfobj = dfobj.fillna(0)
  
# applying operation on the data frame
dfobj['dic'] =  dfobj['dic'].astype(int)
print(dfobj )

Output:

    Name  dic Subject
0   Jack    0    Math
1   Rack   85    Math
2    Max   99    Math
3  David   56     inf

As you can see, we filled the null values with zeroes which help us to get rid of the error.

Solution-2: Using dropna() method

Another important method in pandas that help us to handle the null values is dropna() method. This method drops all the null values which are available in the data frame.

Let us now use the dropna() method to drop the null values from the data frame.

# importing the modules
import pandas as pd
import numpy as np
   
# creating the data frame
data = {
    'Name': ['Jack', 'Rack', 'Max', 'David'],
    'dic':[np.nan,85, 99,56],
    'Subject': ['Math', 'Math', 'Math', np.inf]
}

# creating a data frame
dfobj = pd.DataFrame(data)
 
# dropping the null values
dfobj = dfobj.dropna()
  
# applying operation on the data frame
dfobj['dic'] =  dfobj['dic'].astype(int)
print(dfobj )

Output:

    Name  dic Subject
1   Rack   85    Math
2    Max   99    Math
3  David   56     inf

Notice that the whole row with the null value has been removed.

Solution-2: Using replace() method

The replace() method in pandas is a very powerful method that helps us to replace any value in the data frame with the specified one. We can use the replace() method in order to replace the null values with any other values.

In this example, we will replace the null values with zeros.

# importing the modules
import pandas as pd
import numpy as np
   
# creating the data frame
data = {
    'Name': ['Jack', 'Rack', 'Max', 'David'],
    'dic':[np.nan,85, 99,56],
    'Subject': ['Math', 'Math', 'Math', np.inf]
}

# creating a data frame
dfobj = pd.DataFrame(data)
 
# replacing the null values
dfobj['dic'] = dfobj['dic'].replace(np.nan, 0)
  
# applying operation on the data frame
dfobj['dic'] =  dfobj['dic'].astype(int)
print(dfobj )

Output:

    Name  dic Subject
0   Jack    0    Math
1   Rack   85    Math
2    Max   99    Math
3  David   56     inf

As you can see, we solved the problem and we are no more getting the IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integers.

Summary

In this short article, we discussed how we can solve IntCastingNaNError: Cannot convert non-finite values (NA or inf) to integer errors using various methods. We solved the problem using three different methods and explained each one.

Related Errors:

Leave a Comment