This error occurs when you try to serialize a DataFrame object to a JSON string using the json.dumps()
method. You can solve this error by converting the DataFrame to a JSON string using the DataFrame to_json()
method. For example,
json_str = my_dataframe.to_json()
You can also convert the DataFrame to a dictionary using the to_dict()
method and pass the converted dictionary to the json.dumps()
method.
json_str = json.dumps(df.to_dict())
This tutorial will go through how to solve the error with code examples.
Table of contents
TypeError: Object of type DataFrame is not JSON serializable
Let’s break up the error message to understand what the error means. TypeError occurs whenever you attempt to use an illegal operation for a specific data type. The part “Object of type Decimal” tells us the error is due to an illegal operation with a DataFrame object.
Serialization in Python refers to converting a Python object into a transmittable format that we can recreate when needed using deserialization. JSON serialization returns a human-readable string form called a JSON string. The JSON encoder json.dump() and json.dumps() can only serialize certain object types like dictionaries, lists, or strings.
“is not JSON serializable” informs us that the JSON serialization is an illegal operation for the DataFrame type.
Example
Let’s look at an example of JSON serializing a DataFrame object. The DataFrame contains the names of five pizzas and their respective prices.
We will attempt to serialize the DataFrame using the json.dumps()
method.
import pandas as pd import json df = pd.DataFrame( { "pizza": ['margherita', 'pepperoni', 'hawaiian', 'marinara', 'four cheese'], "price":[8.99, 9.99, 10.99, 7.99, 11.99] } ) json_str = json.dumps(df) print(json_str)
Let’s run the code to see what happens:
TypeError: Object of type DataFrame is not JSON serializable
The error occurs because DataFrame is not a serializable data type.
Solution #1: Use to_json()
Fortunately, the DataFrame class has the to_json()
method to convert the DataFrame to a JSON string. We can call the to_json()
method directly on the DataFrame. Let’s look at the revised code:
import pandas as pd import json df = pd.DataFrame( { "pizza": ['margherita', 'pepperoni', 'hawaiian', 'marinara', 'four cheese'], "price":[8.99, 9.99, 10.99, 7.99, 11.99] } ) json_str = df.to_json() print(json_str)
Let’s run the code to see the JSON string:
{"pizza":{"0":"margherita","1":"pepperoni","2":"hawaiian","3":"marinara","4":"four cheese"},"price":{"0":8.99,"1":9.99,"2":10.99,"3":7.99,"4":11.99}}
Solution #2: Use to_dict()
We can also solve this error by converting the DataFrame to a dictionary using the to_dict()
method and passing the dictionary to the json.dumps()
method. Dictionary is a JSON serializable type.
Let’s look at the revised code:
import pandas as pd import json df = pd.DataFrame( { "pizza": ['margherita', 'pepperoni', 'hawaiian', 'marinara', 'four cheese'], "price":[8.99, 9.99, 10.99, 7.99, 11.99] } ) df_dict = df.to_dict() json_str = json.dumps(df_dict) print(json_str)
Let’s run the code to see the JSON string:
{"pizza": {"0": "margherita", "1": "pepperoni", "2": "hawaiian", "3": "marinara", "4": "four cheese"}, "price": {"0": 8.99, "1": 9.99, "2": 10.99, "3": 7.99, "4": 11.99}}
Solution #3: Define a custom function for default kwarg
We can also solve this error by defining a custom function that converts a DataFrame object to a dictionary. Let’s look at the custom function:
import pandas as pd import json def serialize_dataframe(obj): if isinstance(obj, pd.DataFrame): return obj.to_dict() raise TypeError ("Type %s is not serializable" % type(obj))
The function checks if the object is a DataFrame and converts it to a dictionary. Otherwise, the function raises a TypeError.
Next, we will set the default
keyword argument of the json.dumps()
method to this function to convert the DataFrame to a dictionary.
df = pd.DataFrame( { "pizza": ['margherita', 'pepperoni', 'hawaiian', 'marinara', 'four cheese'], "price":[8.99, 9.99, 10.99, 7.99, 11.99] } ) json_str = json.dumps(df, default=serialize_dataframe) print(json_str)
Let’s run the code to see the result:
{"pizza": {"0": "margherita", "1": "pepperoni", "2": "hawaiian", "3": "marinara", "4": "four cheese"}, "price": {"0": 8.99, "1": 9.99, "2": 10.99, "3": 7.99, "4": 11.99}}
Solution #4: Define a JSONEncoder subclass for the cls kwarg
We can also solve this error by building a custom JSONEncoder
subclass. This subclass will override the default method to serialize additional types.
import pandas as pd import json class dataframe_encoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, pd.DataFrame): return obj.to_dict() return json.JSONEncoder.default(self, obj)
Similar to the custom function, the default
method of the dataframe_encoder
class checks if the object is of type DataFrame, converts it to a dictionary and returns it.
We have to specify the custom JSONEncoder subclass with the cls
keyword argument. Otherwise, JSONEncoder is used. Let’s look at the updated code:
df = pd.DataFrame( { "pizza": ['margherita', 'pepperoni', 'hawaiian', 'marinara', 'four cheese'], "price":[8.99, 9.99, 10.99, 7.99, 11.99] } ) json_str = json.dumps(df, cls=dataframe_encoder) print(json_str)
Let’s run the code to see the result:
{"pizza": {"0": "margherita", "1": "pepperoni", "2": "hawaiian", "3": "marinara", "4": "four cheese"}, "price": {"0": 8.99, "1": 9.99, "2": 10.99, "3": 7.99, "4": 11.99}}
Below is the collection of objects that the JSONEncoder
class supports and their JSON
equivalent.
Python | JSON |
---|---|
dict | object |
list, tuple | array |
str | string |
int, float, int- & float- derived Enums | number |
True | true |
False | false |
None | null |
Summary
Congratulations on reading to the end of this tutorial.
For further reading on errors involving JSON serialization, go to the articles:
- How to Solve Python TypeError: Object of type int32 is not JSON serializable
- How to Solve Python TypeError: Object of type Decimal is not JSON serializable
Go to the online courses page on Python to learn more about Python for data science and machine learning.
Have fun and happy researching!