Select Page

How to Solve Python TypeError: object of type ‘Response’ has no len()

by | Programming, Python, Tips

This error occurs when you try to parse HTML code using the BeautifulSoup constructor but pass a response object instead of the response’s content.

You can solve this error by accessing the Response object’s content using dot notation.

For example,

import requests 
from bs4 import BeautifulSoup

URL = "https://datahub.io/awesome/football"
page = requests.get(URL)

soup = BeautifulSoup(page.content, "html.parser")
print(soup)

This tutorial will go through the error and how to solve it with code examples.

Table of contents

TypeError: object of type ‘Response’ has no len()

We raise a Python TypeError when attempting to perform an illegal operation for a specific type. In this case, the type is Response.

The part ‘has no len()‘ tells us the map object does not have a length, and therefore len() is an illegal operation for the Response object.

Retrieving the length of an object is only suitable for iterable objects, like a list or a string.

The len() method implicitly calls the dunder method __len__() , which returns a positive integer representing the length of the object on which it is called.

All iterable objects have __len__ as an attribute.

Let’s check if __len__ is in the list of attributes for the Response object.

import requests 
from bs4 import BeautifulSoup

URL = "https://datahub.io/awesome/football"
page = requests.get(URL)

print(type(page))
print('__len__' in dir(page))
<class 'requests.models.Response'>
False

We can see that __len__ is not present in the attributes of the Response object.

We can retrieve the content from the response object using dot notation. Dot notation requires putting a dot after the object followed by the attribute we want to access. response.content returns the content of the response in bytes. In Python, bytes is an iterable sequence with a length.

Let’s verify that response.content has __len__ in its list of attributes.

import requests
from bs4 import BeautifulSoup

URL = "https://datahub.io/awesome/football"

page = requests.get(URL)

content = page.content

print(type(content))

print('__len__' in dir(content))
<class 'bytes'>
True

We can see that __len__ is present in the attributes of the bytes object.

Example

Let’s look at an example of trying to parse HTML code using BeautifulSoup and Requests. First, we will import the requests module and BeautifulSoup.

import requests 
from bs4 import BeautifulSoup

Next, we will make a GET request to a web page and save the response as a response object.

URL = "https://datahub.io/awesome/air-pollution"
page = requests.get(URL)

Then we can parse the HTML code using the BeautifulSoup constructor. The first argument of the BeautifulSoup constructor is the response object from the GET request, and the second is the appropriate parser for the HTML content. 

soup = BeautifulSoup(page, "html.parser")

Let’s run the code to see the result:

TypeError: object of type 'Response' has no len()

The error occurs because the BeautifulSoup constructor requires the response content, not the entire response.

Solution

We can solve the error by retrieving the response content using .content after the response object name.

It is preferable to use .content instead of .text as Requests guesses the text encoding for the response based on the HTTP headers.

Let’s look at the revised code:

import requests 
from bs4 import BeautifulSoup

URL = "https://datahub.io/awesome/air-pollution"
page = requests.get(URL)

soup = BeautifulSoup(page.content, "html.parser")
print(soup)

Let’s run the code to get the result:

<!DOCTYPE html>

<html lang="en">
<head>
<meta charset="utf-8"/>
<title>
    Air Pollution Data - Awesome Datasets
    - DataHub - Frictionless Data
  </title>
....

We successfully parsed the HTML content using the BeautifulSoup constructor.

We can also use .text, for example:

import requests 
from bs4 import BeautifulSoup

URL = "https://datahub.io/awesome/air-pollution"
page = requests.get(URL)

soup = BeautifulSoup(page.text, "html.parser")
print(soup)

Summary

Congratulations on reading to the end of this tutorial!

For further reading on the has no len() TypeErrors, go to the article:

To learn more about Python for data science and machine learning, go to the online courses page on Python, which provides the best, easy-to-use online courses.