If you try to compare DataFrames with different indexes using the equality comparison operator ==, you will raise the ValueError: Can only compare identically-labeled DataFrame objects. You can solve this error by using equals instead of ==.
For example, df1.equals(df2), which ignores the indexes.
Alternatively, you can use reset_index to reset the indexes back to the default 0, 1, 2, ... For example, df1.reset_index(drop=True).equals(df2.reset_index(drop=True)).
This tutorial will go through the error find detail and how to solve it with code examples.
ValueError: Can only compare identically-labeled DataFrame objects
In Python, a value is a piece of information stored within a particular object. We will encounter a ValueError in Python when using a built-in operation or function that receives an argument that is the right type but an inappropriate value. The data we want to compare is the correct type, DataFrame, but the DataFrames have the inappropriate indexes for comparison.
Example
Let’s look at an example of two DataFrames that we want to compare. Each DataFrame contains the bodyweight and maximum bench presses in kilograms for six lifters. The indexes for the two DataFrames are different.
import pandas as pd
df1 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[135, 150, 170, 140, 180, 155]},
index = ['lifter_1', 'lifter_2', 'lifter_3', 'lifter_4', 'lifter_5', 'lifter_6'])
df2 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[145, 120, 180, 220, 175, 110]},
index = ['lifter_A', 'lifter_B', 'lifter_C', 'lifter_D', 'lifter_E', 'lifter_F'])
print(df1)
print(df2)
Let’s run this part of the program to see the DataFrames:
Bodyweight (kg) Bench press (kg)
lifter_1 76 135
lifter_2 84 150
lifter_3 93 170
lifter_4 106 140
lifter_5 120 180
lifter_6 56 155
Bodyweight (kg) Bench press (kg)
lifter_A 76 145
lifter_B 84 120
lifter_C 93 180
lifter_D 106 220
lifter_E 120 175
lifter_F 56 110e
Let’s compare the DataFrames using the equality operator:
print(df1 == df2)
Let’s run the code to see the result:
ValueError: Can only compare identically-labeled DataFrame objects
The ValueError occurs because the first DataFrame has indexes: ['lifter_1', 'lifter_2', 'lifter_3', 'lifter_4', 'lifter_5', 'lifter_6'] and the second DataFrame has indexes: ['lifter_A', 'lifter_B', 'lifter_C', 'lifter_D', 'lifter_E', 'lifter_F'].
Solution #1: Use DataFrame.equals
To solve this error, we can use the DataFrame.equals function. The equals function allows us compare two Series or DataFrames to see if they have the same shape or elements. Let’s look at the revised code:
print(df1.equals(df2))
Let’s run the code to see the result:
False
Solution #2: Use DataFrame.equals with DataFrame.reset_index()
We can drop the indexes of the DataFrames using the reset_index() method, then we can compare the DataFrames. To drop the indexes, we need to set the parameter drop = True. Let’s look at the revised code:
df1 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93, 106, 120, 56],
'Bench press (kg)':[145, 120, 180, 220, 175, 110]},
index = ['lifter_1', 'lifter_2', 'lifter_3', 'lifter_4', 'lifter_5', 'lifter_6'])
df2 = pd.DataFrame({'Bodyweight (kg)':[76, 84, 93, 106, 120, 56],
'Bench press (kg)':[145, 120, 180, 220, 175, 110]},
index = ['lifter_A', 'lifter_B', 'lifter_C', 'lifter_D', 'lifter_E', 'lifter_F'])
df1 = df1.reset_index(drop=True)
df2 = df2.reset_index(drop=True)
print(df1)
print(df2)
Let’s look at the DataFrames with their indexes dropped:
Bodyweight (kg) Bench press (kg) 0 76 145 1 84 120 2 93 180 3 106 220 4 120 175 5 56 110 Bodyweight (kg) Bench press (kg) 0 76 145 1 84 120 2 93 180 3 106 220 4 120 175 5 56 110
There are two ways we can compare the DataFrames:
- The whole DataFrame
- Row-by-row comparison
Entire DataFrame Comparison
We can use the equals() method to see if all elements are the same in both DataFrame objects. Let’s look at the code:
print(df1.equals(df2))
Let’s run the code to see the result:
True
Row-by-Row DataFrame Comparison
We can check that individual rows are equal using the equality operator once the DataFrames indexes are reset. Let’s look at the code:
print(df1 == df2)
Let’s run the code to see the result:
Bodyweight (kg) Bench press (kg) 0 True True 1 True True 2 True True 3 True True 4 True True 5 True True
Note that the comparison is done row-wise for each column independently.
Solution #3: Use numpy.array_equal
We can also use numpy.array_equal to check if two arrays have the same shape and elements. We can extract arrays from the DataFrame using .values. Let’s look at the revised code:
import pandas as pd
import numpy as np
df1 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[135, 150, 170, 140, 180, 155]},
index = ['lifter_1', 'lifter_2', 'lifter_3', 'lifter_4', 'lifter_5', 'lifter_6'])
df2 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[145, 120, 180, 220, 175, 110]},
index = ['lifter_A', 'lifter_B', 'lifter_C', 'lifter_D', 'lifter_E', 'lifter_F'])
print(np.array_equal(df1.values, df2.values))
Let’s run the code to see the result:
False
We can use array_equal to compare individual columns. Let’s look at the revised code:
import pandas as pd
import numpy as np
df1 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[135, 150, 170, 140, 180, 155]},
index = ['lifter_1', 'lifter_2', 'lifter_3', 'lifter_4', 'lifter_5', 'lifter_6'])
df2 = pd.DataFrame({'Bodyweight (kg)':[76,84, 93,106, 120, 56],
'Bench press (kg)':[145, 120, 180, 220, 175, 110]},
index = ['lifter_A', 'lifter_B', 'lifter_C', 'lifter_D', 'lifter_E', 'lifter_F'])
# Get individual columns of DataFrames using iloc
df1_bodyweight = df1.iloc[:,0]
df1_bench = df1.iloc[:,1]
df2_bodyweight = df2.iloc[:,0]
df2_bench = df2.iloc[:,1]
# Compare bodyweight and bench columns separately
print(np.array_equal(df1_bodyweight.values, df2_bodyweight.values))
print(np.array_equal(df1_bench.values, df2_bench.values))
Let’s run the code to see the result:
True False
The above result informs us that the first column contains the same elements between the two DataFrames, the second column contains different elements between the two DataFrames.
Summary
Congratulations on reading to the end of this tutorial! The ValueError: Can only compare identically-labeled DataFrame objects occurs when trying to compare two DataFrames with different indexes. You can either reset the indexes using reset_index() or use the equals() function which ignores the indexes. You can also use the NumPy method array_equal to compare the two DataFrames’ columns.
For further reading on errors involving Pandas, go to the articles:
- How to Solve Pandas TypeError: empty ‘dataframe’ no numeric data to plot
- How to Solve Python ValueError: You are trying to merge on object and int64 columns
For further reading on Pandas, go to the article: Introduction to Pandas: A Complete Tutorial for Beginners.
Have fun and happy researching