This error occurs when you try to parse HTML code using the BeautifulSoup constructor but pass a response object instead of the response’s content.
You can solve this error by accessing the Response object’s content using dot notation.
For example,
import requests from bs4 import BeautifulSoup URL = "https://datahub.io/awesome/football" page = requests.get(URL) soup = BeautifulSoup(page.content, "html.parser") print(soup)
This tutorial will go through the error and how to solve it with code examples.
TypeError: object of type ‘Response’ has no len()
We raise a Python TypeError when attempting to perform an illegal operation for a specific type. In this case, the type is Response.
The part ‘has no len()‘ tells us the map object does not have a length, and therefore len() is an illegal operation for the Response object.
Retrieving the length of an object is only suitable for iterable objects, like a list or a string.
The len() method implicitly calls the dunder method __len__() , which returns a positive integer representing the length of the object on which it is called.
All iterable objects have __len__ as an attribute.
Let’s check if __len__ is in the list of attributes for the Response object.
import requests
from bs4 import BeautifulSoup
URL = "https://datahub.io/awesome/football"
page = requests.get(URL)
print(type(page))
print('__len__' in dir(page))
<class 'requests.models.Response'> False
We can see that __len__ is not present in the attributes of the Response object.
We can retrieve the content from the response object using dot notation. Dot notation requires putting a dot after the object followed by the attribute we want to access. response.content returns the content of the response in bytes. In Python, bytes is an iterable sequence with a length.
Let’s verify that response.content has __len__ in its list of attributes.
import requests
from bs4 import BeautifulSoup
URL = "https://datahub.io/awesome/football"
page = requests.get(URL)
content = page.content
print(type(content))
print('__len__' in dir(content))
<class 'bytes'> True
We can see that __len__ is present in the attributes of the bytes object.
Example
Let’s look at an example of trying to parse HTML code using BeautifulSoup and Requests. First, we will import the requests module and BeautifulSoup.
import requests from bs4 import BeautifulSoup
Next, we will make a GET request to a web page and save the response as a response object.
URL = "https://datahub.io/awesome/air-pollution" page = requests.get(URL)
Then we can parse the HTML code using the BeautifulSoup constructor. The first argument of the BeautifulSoup constructor is the response object from the GET request, and the second is the appropriate parser for the HTML content.
soup = BeautifulSoup(page, "html.parser")
Let’s run the code to see the result:
TypeError: object of type 'Response' has no len()
The error occurs because the BeautifulSoup constructor requires the response content, not the entire response.
Solution
We can solve the error by retrieving the response content using .content after the response object name.
It is preferable to use .content instead of .text as Requests guesses the text encoding for the response based on the HTTP headers.
Let’s look at the revised code:
import requests from bs4 import BeautifulSoup URL = "https://datahub.io/awesome/air-pollution" page = requests.get(URL) soup = BeautifulSoup(page.content, "html.parser") print(soup)
Let’s run the code to get the result:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title>
Air Pollution Data - Awesome Datasets
- DataHub - Frictionless Data
</title>
....
We successfully parsed the HTML content using the BeautifulSoup constructor.
We can also use .text, for example:
import requests from bs4 import BeautifulSoup URL = "https://datahub.io/awesome/air-pollution" page = requests.get(URL) soup = BeautifulSoup(page.text, "html.parser") print(soup)
Summary
Congratulations on reading to the end of this tutorial!
For further reading on the has no len() TypeErrors, go to the article:
- How to Solve Python TypeError: object of type ‘zip’ has no len()
- How to Solve Python TypeError: object of type ‘builtin_function_or_method’ has no len()
- How to Solve Python TypeError: object of type ‘generator’ has no len()
To learn more about Python for data science and machine learning, go to the online courses page on Python, which provides the best, easy-to-use online courses.
Suf is a senior advisor in data science with deep expertise in Natural Language Processing, Complex Networks, and Anomaly Detection. Formerly a postdoctoral research fellow, he applied advanced physics techniques to tackle real-world, data-heavy industry challenges. Before that, he was a particle physicist at the ATLAS Experiment of the Large Hadron Collider. Now, he’s focused on bringing more fun and curiosity to the world of science and research online.