Concatenating joins a sequence of tensors along an existing axis. The PyTorch function for concatenation is cat(). Stacking joins a sequence of tensors along a new axis. The PyTorch function for stacking is stack().

This tutorial will go through the two PyTorch functions with code examples.


PyTorch Cat

We can use the PyTorch cat() function to concatenate a sequence of tensors along the same dimension. The tensors must have the same shape (except in the concatenating dimension) or be empty.

Syntax

torch.cat(tensors, dim=0, *, out=None)

Parameters

  • tensors (sequence of Tensors): Required. Any Python sequence of tensors of the same type. Non-empty tensors must have the same shape except in the concatenating dimension.
  • dim (int): Optional. The dimension to concatenate the tensors over.

Keyword Arguments

  • out (Tensor): Optional. Output tensor

Example

Let’s look at an example where we concatenate three tensors into one tensor using cat(). First, we have to import the PyTorch library and then use the tensor() function to create the tensors:

import torch

x = torch.tensor([2, 3, 4, 5])

y = torch.tensor([4, 10, 30])

z = torch.tensor([7, 22, 4, 8, 3, 6])

Next, we can concatenate the tensors along the 0th dimension, the only available axis.

xyz = torch.cat(

(x, y, z), dim=0

)

print(xyz)

print(xyz.shape)

Let’s run the code to see the result:

tensor([ 2,  3,  4,  5,  4, 10, 30,  7, 22,  4,  8,  3,  6])

PyTorch Stack

We can use the PyTorch stack() function to concatenate a sequence of tensors along a new dimension. The tensors must have the same shape.

Syntax

torch.stack(tensors, dim=0, *, out=None)

Parameters

  • tensors (sequence of Tensors): Required. Python sequence of tensors of the same size.
  • dim (int): Optional. The new dimension to insert. The dimension must be between 0 and the number of dimensions of concatenated tensors.

Keyword Arguments

  • out (Tensor): Optional. Output tensor

Example

Let’s look at an example where we stack three tensors into one tensor using stack(). First, we have to import the PyTorch library and then use the tensor() function to create the tensors:

import torch

x = torch.tensor([2, 3, 4, 5])

y = torch.tensor([4, 10, 30, 40])

z = torch.tensor([8, 7, 16, 14])

In the above code, the tensors x, y, and z are one-dimensional, each having four elements. Next, we will stack the tensors along dim=0 and dim=1.

# Stacking Tensors using dimension 0

stacked_0 = torch.stack(

(x, y, z), dim=0

)

# Stacking Tensors using dimension 1

stacked_1 = torch.stack(

(x,y, z), dim=1

)
# Resultant combined tensor with new axes along dimension 0

print(stacked_0)

# Shape of combined tensor

print(stacked_0.shape)

# Resultant combined tensor with new axes along dimension 1

print(stacked_1)

# Shape of combined tensor

print(stacked_1.shape)

Let’s run the code to get the result:

tensor([[ 2,  3,  4,  5],
        [ 4, 10, 30, 40],
        [ 8,  7, 16, 14]])
torch.Size([3, 4])
tensor([[ 2,  4,  8],
        [ 3, 10,  7],
        [ 4, 30, 16],
        [ 5, 40, 14]])
torch.Size([4, 3])

The resultant concatenated tensor is two-dimensional. As the individual tensors are one-dimensional, we can stack them with dimensions 0 and 1.

With dim=0 the tensors are stacked row-wise, giving us a 3×4 matrix. With dim=1 we transpose the tensors and stack them column-wise, giving us a 4×3 matrix.

PyTorch Cat Vs Stack

The two PyTorch functions offer similar functionality but differ in how they concatenate tensors. The cat() function concatenates tensors along the existing dimension. The stack() function concatenates tensors along a new dimension not present in the individual tensors.

We can derive the same results of the stack() function using the cat() function. We can apply the unsqueeze operation to each tensor before passing them to the cat() function to get the same result. Let’s look at the result with the tensors from the previous example

import torch

x = torch.tensor([2, 3, 4, 5])

y = torch.tensor([4, 10, 30, 40])

z = torch.tensor([8, 7, 16, 14])

xyz = torch.cat((x.unsqueeze(0), y.unsqueeze(0), z.unsqueeze(0)), dim=0)

print(xyz)

print(xyz.shape)

The unsqueeze operation adds a new dimension of length one to the tensors, and then we concatenate along the first axis. Let’s run the code to get the result:

tensor([[ 2,  3,  4,  5],
        [ 4, 10, 30, 40],
        [ 8,  7, 16, 14]])
torch.Size([3, 4])

Therefore torch.stack((A, B), dim=0) is equivalent to torch.cat((A.unsqueeze(0), B.unsqueeze(0)), dim=0

Summary

Congratulations on reading to the end of this tutorial. We have gone through how to concatenate tensors using both cat() and stack() and explained the differences between the two functions.

For further reading on PyTorch, go to the article: How to Convert NumPy Array to PyTorch Tensor.

To learn more about Python for data science and machine learning, go to the online courses page on Python for the most comprehensive courses available.

Have fun and happy researching!