How to Solve Python TypeError: Object of type bytes is not JSON serializable

This error occurs when you try to serialize a bytes object to a JSON string using the json.dumps() method. You can solve this error by decoding the bytes object to a string using the str.decode() method and passing the string to the json.dumps() method. For example,

my_string = my_bytes.decode()

This tutorial will go through how to solve the error with code examples.

---

TypeError: Object of type bytes is not JSON serializable

Let’s break up the error message to understand what the error means. TypeError occurs whenever you attempt to use an illegal operation for a specific data type. The part “Object of type bytes” tells us the error is due to an illegal operation with a bytes object.

Serialization in Python refers to converting a Python object into a transmittable format that we can recreate when needed using deserialization. JSON serialization returns a human-readable string form called a JSON string. The JSON encoder json.dump() and json.dumps() can only serialize certain object types like dictionaries, lists, or strings.

“is not JSON serializable” informs us that the JSON serialization is an illegal operation for the bytes type.

Example

Let’s look at an example of serializing an image using JSON. First, we will look at the image we want to use, which is a picture of the ATLAS detector.

We will download this image and store it in the Downloads folder. Next, we will import the base64 module to encode the image and store it in a dictionary. Base64 is a binary to a text encoding scheme that represents binary data in an ASCII string format. Base64 encoding helps ensure that the data remains intact without modification during transport.

Once we have the dictionary, we will attempt to serialize the data using json.dumps().

import base64
import json

atlas_img_dict = {}

with open('Downloads/Installing_the_ATLAS_Calorimeter.jpg', 'rb') as img:

    image = base64.b64encode(img.read())

    atlas_img_dict['ATLAS_image']= image

    json_str = json.dumps(atlas_img_dict)

Let’s run the code to see what happens:

TypeError: Object of type bytes is not JSON serializable

The error occurs because the base64.b64encode method returns a bytes object, not a string. The b64encode method encodes the bytes-like object from img.read() using Base64 and returns the encoded bytes object.

Solution #1: Use str.decode()

The simplest way to solve this error is to call the decode() method on the bytes object returned by base64.b64encode to get a base64 string. We can then store the base64 string in the dictionary and serialize the data. Let’s look at the updated code:

import base64
import json

atlas_img_dict = {}

with open('Downloads/Installing_the_ATLAS_Calorimeter.jpg', 'rb') as img:

    image = base64.b64encode(img.read())

    print(type(image))

    image_str = image.decode()

    print(type(image.decode()))

    atlas_img_dict['ATLAS_image']= image_str

    json_str = json.dumps(atlas_img_dict)

We can check the type of the image object and the image_str object, which should be bytes and string respectively.

<class 'bytes'>
<class 'str'>

We can deserialize the JSON string to get the base64 string using json.loads().

dict_from_json = json.loads(json_str)

image_base64_string = dict_from_json['ATLAS_image']

We can convert the Base64 string to an image using Code Beautify, ensure that you remove quotation marks surrounding the Base64 string.

Solution #2: Define a custom function for default kwarg

We can also solve this error by defining a custom function that converts the bytes object to a string and passing this function to json.dumps() as the default keyword argument. The default value for the keyword argument default is None. We can set default to a function for objects that are not serializable to convert them to a serializable format.

import json
import base64

def serialize_bytes(obj):

    if isinstance(obj, bytes):

        return obj.decode()

    raise TypeError ("Type %s is not serializable" % type(obj))

The custom function checks if the object is type bytes and then converts it to a string using decode(). Otherwise, it raises a TypeError. Let’s set the default keyword to our custom function and run the code:

atlas_img_dict = {}

with open('Downloads/Installing_the_ATLAS_Calorimeter.jpg', 'rb') as img:

    image = base64.b64encode(img.read())

    atlas_img_dict['ATLAS_image']= image

    json_str = json.dumps(atlas_img_dict, default=serialize_bytes)

Solution #3: Define a JSONEncoder subclass for the cls kwarg

The third way we can solve this error is by building a custom JSONEncoder subclass. This subclass will override the default method to serialize additional types. Similar to the custom function, the default method checks if the object is type bytes, converts it to a string and returns it.

import json

class BytesEncoder(json.JSONEncoder):

    def default(self, obj):

        if isinstance(obj, bytes):

            return obj.decode()

        return json.JSONEncoder.default(self, obj)

We have to specify the custom JSONEncoder subclass with the cls keyword argument. Otherwise, JSONEncoder is used. Let’s look at the updated code:

atlas_img_dict = {}

with open('Downloads/Installing_the_ATLAS_Calorimeter.jpg', 'rb') as img:

    image = base64.b64encode(img.read())

    atlas_img_dict['ATLAS_image']= image

    json_str = json.dumps(atlas_img_dict, cls=BytesEncoder)

Below is the collection of objects that the JSONEncoder class supports and their JSON equivalent.

Python	JSON
dict	object
list, tuple	array
str	string
int, float, int- & float- derived Enums	number
True	true
False	false
None	null

Summary

Congratulations on reading to the end of this tutorial.

For further reading on errors involving JSON serialization, go to the articles:

• How to Solve Python TypeError: Object of type datetime is not JSON serializable
• How to Solve Python TypeError: Object of type set is not JSON serializable
• How to Solve Python TypeError: Object of type ndarray is not JSON serializable
• How to Solve Python TypeError: Object of type TextIOWrapper is not JSON serializable

Go to the online courses page on Python to learn more about Python for data science and machine learning.

Have fun and happy researching!