How to Check if a File or Directory Exists in Python

Checking if a specific file exists with Python is very useful. You may want to read data in to analyse or check if a file exists before writing data to file. There are three ways to check if a file exists using the os library: isfile(), isdir(), and exists. We will discuss using these methods in Python and provide simple examples to illustrate how each technique works.

What is the OS Library in Python?

Python has a built-in os module containing operating system functionalities that can integrate into your code. This tutorial will focus on the path functions, which check whether specific files or directories exist. Therefore, we only need to import the os.path module. Python has libraries for various tasks relevant to data scientists and machine learning researchers. To learn about these libraries, visit my article titled: “Top 12 Python Libraries for Data Science and Machine Learning”. To import the os library, we use the Python import statement:

# import os library

import os.path

What is the pathlib Library in Python

Like os, pathlib also provides file system operations, and is arguably more intuitive in terms of syntax and has more features out of the box. pathlib is a more object-oriented way of checking files and offers more options for file interactions.

Check if File Exists in Python

Using exists()

We can write a program that checks whether a file data.txt exists. If the file does not exist, we create it and write content to it. We can write this program as follows:

import os

#check if file exists

if not os.path.exists('data.txt'):
    with open('data.txt', 'w') as f:
        f.write('This is data\n')

If the data.txt file does not exist, this code block will create a new file. You can use this feature to have crash-free logging in your program.

Using isfile()

We can do a similar operation as above with isfile(), which will return a boolean value True if the file exists and false if the file does not exist.

If you wrote the data.txt file from the previous step and are in the directory where that file is, this will return True.

import os

print(os.path.isfile('data.txt'))

The isfile() method only works for files. If you apply isfile() to directories, it will always return False, even if it exists. To test this, make a directory ‘test’ using os.mkdir.

os.mkdir('test')

From the root directory where you made the directory, you can use isfile() on the directory.

print(os.path.isfile('./dir'))

False

The code will always return false.

Using pathlib

To check if a file exists using pathlib, we can use the Path class and call the exists() method as follows:

#import Path class from pathlib

from pathlib import Path

data_file = Path('data_2.txt')
if not data_file.exists():
    with open(data_file, 'w') as f:
       f.write('This is also data\n')

Similar to the os example, the exists() will return True if the file exists and false if the file does not exist. The advantage of using the Path() class is it does not matter if the file is deleted somewhere else in the code by a concurrent process because Path() stores files as an object. Therefore, if a file is deleted somewhere between the running of the if statement and the file writing, the file writing operation will still go forward because the program will write to a local copy of the file.

pathlib vs os

Compared to os, pathlib executes slightly slower. os remains a robust library, capable of advanced parsing abilities, such as os.path.commonpath(), which will take a list of file paths and find the lowest directory that all the files have in common. However, pathlib is the optimal choice as it is object-oriented, and you can join methods and functions together quickly.

Check if Directory Exists in Python

Using os.path.isdir()

We can use os to check if a directory exists using the isdir() method. Let’s look at an example where we create a folder for our data files and check if it exists:

import os

os.mkdir('./data_folder')

print(os.path.isdir('./data_folder'))

The directory exists, therefore our program returns True.

True

Using os.path.exists()

We can use os.path.exists() to check if a directory exists

import 

os.mkdir('./data_folder_2')

print(os.path.exists('./data_folder_2')

The directory exists in the example, therefore our program returns True.

Using pathlib

Let’s move the “data.txt” file to “data_folder” from the shell.

mv data.txt data_folder

We can specify a Path object as follows:

from pathlib import Path
my_file = Path('./data_folder/data.txt')

To check if a file exists we can use the is_file() method:

if my_file.is_file():
    print('file exists')

'file exists'

to check if a directory exists we can use the is_dir() method:

if my_file.is_dir():
    print('directory exists')

'directory exists'

To check whether a Path object exists independently of whether it is a file or a directory, we can use exists():

if my_file.exists():
    print('path exists')

'path exists'

Summary

Congratulations on reading to the end of this tutorial. You know how to check for the existence of files and directories using both the os and pathlib libraries. The os library is ideal for beginners as the operations are simpler and can be faster. For developers who have more experience with object-oriented programming, pathlib is more suitable.

The table below summarizes what the available functions for file and directory checking for both os and pathlib. Knowing these functions will ensure your program does not crash by checking for the existence of particular files or directories before performing file operations.

os function	pathlib function	What the function does
os.path.isfile(‘file’)	Path(‘path/to/file’).is_file()	Checks if the ‘file’ exists
os.path.isdir(‘directory’)	Path(‘path/to/file’).is_dir()	Checks if the ‘directory’ exists
os.path.exists(‘file/directory’)	Path(‘path/to/file’).exists()	Checks if the ‘file/directory’ exists

To learn more about Python libraries and functionalities, go to the online courses page for Python. There are other solutions for common Python tasks and TypeErrors including TypeError: can only concatenate str (not “int”) to str.

Have fun and happy researching!