[python] How to check whether a file exists?


Answers

You have the os.path.exists function:

import os.path
os.path.exists(file_path)

This returns True for both files and directories but you can instead use os.path.isfile to test if it's a file specifically. It follows symlinks.

Question

How to see if a file exists or not, without using the try statement?




import os
#Your path here e.g. "C:\Program Files\text.txt"
#For access purposes: "C:\\Program Files\\text.txt"
if os.path.exists("C:\..."):   
    print "File found!"
else:
    print "File not found!"

Importing os makes it easier to navigate and perform standard actions with your operating system.

For reference also see How to check whether a file exists using Python?

If you need high-level operations, use shutil.




You could try this (safer):

try:
    # http://effbot.org/zone/python-with-statement.htm
    # 'with' is safer to open a file
    with open('whatever.txt') as fh:
        # Do something with 'fh'
except IOError as e:
    print("({})".format(e))

The ouput would be:

([Errno 2] No such file or directory: 'whatever.txt')

Then, depending on the result, your program can just keep running from there or you can code to stop it if you want.




import os
path = /path/to/dir

root,dirs,files = os.walk(path).next()
if myfile in files:
   print "yes it exists"

This is helpful when checking for several files. Or you want to do a set intersection/ subtraction with an existing list.




import os
os.path.exists(path) # returns whether the path (dir or file) exists or not
os.path.isfile(path) # returns whether the file exists or not



Here's a 1 line Python command for the Linux command line environment. I find this VERY HANDY since I'm not such a hot Bash guy.

python -c "import os.path; print os.path.isfile('/path_to/file.xxx')"

I hope this is helpful.




If the file is for opening you could use one of the following techniques:

>>> with open('somefile', 'xt') as f: #Using the x-flag, Python3.3 and above
...     f.write('Hello\n')

>>> if not os.path.exists('somefile'): 
...     with open('somefile', 'wt') as f:
...         f.write("Hello\n")
... else:
...     print('File already exists!')



You can use the "OS" library of Python:

>>> import os
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.txt") 
True
>>> os.path.exists("C:\\Users\\####\\Desktop\\test.tx")
False



2017 / 12 / 22:

Although almost every possible way has been listed in (at least one of) the existing answers (e.g. Python 3.4 specific stuff was added), I'll try to group everything together.

Note: every piece of Python standard library code that I'm going to post, belongs to version 3.5.3 (doc quotes are version 3 specific).

Problem statement:

  1. Check file (arguable: also folder ("special" file) ?) existence
  2. Don't use try / except / else / finally blocks

Possible solutions:

  1. [Python]: os.path.exists(path) (also check other function family members like os.path.isfile, os.path.isdir, os.path.lexists for slightly different behaviors)

    os.path.exists(path)
    

    Return True if path refers to an existing path or an open file descriptor. Returns False for broken symbolic links. On some platforms, this function may return False if permission is not granted to execute os.stat() on the requested file, even if the path physically exists.

    All good, but if following the import tree:

    • os.path - posixpath.py (ntpath.py)

      • genericpath.py, line ~#20+

        def exists(path):
            """Test whether a path exists.  Returns False for broken symbolic links"""
            try:
                st = os.stat(path)
            except os.error:
                return False
            return True
        

    it's just a try/except block around [Python]: os.stat(path, *, dir_fd=None, follow_symlinks=True). So, your code is try/except free, but lower in the framestack there's (at least) one such block. This also applies to other funcs (including os.path.isfile).

    1.1. [Python]: pathlib.Path.is_file()

    • It's a fancier (and more pythonic) way of handling paths, but
    • Under the hood, it does exactly the same thing (pathlib.py, line ~#1330):

      def is_file(self):
          """
          Whether this path is a regular file (also True for symlinks pointing
          to regular files).
          """
          try:
              return S_ISREG(self.stat().st_mode)
          except OSError as e:
              if e.errno not in (ENOENT, ENOTDIR):
                  raise
              # Path doesn't exist or is a broken symlink
              # (see https://bitbucket.org/pitrou/pathlib/issue/12/)
              return False
      
  2. [Python]: With Statement Context Managers. Either:

    • Create one:

      class Swallow:  # Dummy example
          swallowed_exceptions = (FileNotFoundError,)
      
          def __enter__(self):
              print("Entering...")
      
          def __exit__(self, exc_type, exc_value, exc_traceback):
              print("Exiting:", exc_type, exc_value, exc_traceback)
              return exc_type in Swallow.swallowed_exceptions  # only swallow FileNotFoundError (not e.g. TypeError - if the user passes a wrong argument like None or float or ...)
      
      • And its usage - I'll replicate the isfile behavior (note that this is just for demonstrating purposes, do not attempt to write such code for production):

        import os
        import stat
        
        
        def isfile_seaman(path):  # Dummy func
            result = False
            with Swallow():
                result = stat.S_ISREG(os.stat(path).st_mode)
            return result
        
    • Use [Python]: contextlib.suppress(*exceptions) - which was specifically designed for selectively suppressing exceptions


    But, they seem to be wrappers over try/except/else/finally blocks, as [Python]: The with statement states:

    This allows common try...except...finally usage patterns to be encapsulated for convenient reuse.

  3. Filesystem traversal functions (and search the results for matching item(s))


    Since these iterate over folders, (in most of the cases) they are inefficient for our problem (there are exceptions, like non wildcarded globbing - as @ShadowRanger pointed out), so I'm not going to insist on them. Not to mention that in some cases, filename processing might be required.

  4. [Python]: os.access(path, mode, *, dir_fd=None, effective_ids=False, follow_symlinks=True) whose behavior is close to os.path.exists (actually it's wider, mainly because of the 2nd argument)

    • user permissions might restrict the file "visibility" as the doc states:

      ...test if the invoking user has the specified access to path. mode should be F_OK to test the existence of path...

    os.access("/tmp", os.F_OK)
    

    Since I also work in C, I use this method as well because under the hood, it calls native APIs (again, via "${PYTHON_SRC_DIR}/Modules/posixmodule.c"), but it also opens a gate for possible user errors, and it's not as Pythonic as other variants. So, as @AaronHall rightly pointed out, don't use it unless you know what you're doing:

    Note: calling native APIs is also possible via [Python]: ctypes — A foreign function library for Python, but in most cases it's more complicated.

    (Win specific): Since msvcr*(vcruntime*) exports a [MSDN]: _access, _waccess function family as well, here's an example:

    Python 3.5.3 (v3.5.3:1880cb95a742, Jan 16 2017, 16:02:32) [MSC v.1900 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os, ctypes
    >>> ctypes.CDLL("msvcrt")._waccess(u"C:\\Windows\\System32\\cmd.exe", os.F_OK)
    0
    >>> ctypes.CDLL("msvcrt")._waccess(u"C:\\Windows\\System32\\___cmd.exe", os.F_OK)
    -1
    

    Notes:

    • Although it's not a good practice, I'm using os.F_OK in the call, but that's just for clarity (its value is 0)
    • I'm using _waccess so that the same code works on Python3 and Python2 (in spite of unicode related differences between them)
    • Although this targets a very specific area, it was not mentioned in any of the previous answers


    The Lnx (Ubtu (16 x64)) counterpart as well:

    Python 3.5.2 (default, Nov 17 2016, 17:05:23)
    [GCC 5.4.0 20160609] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import os, ctypes
    >>> ctypes.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp", os.F_OK)
    0
    >>> ctypes.CDLL("/lib/x86_64-linux-gnu/libc.so.6").access(b"/tmp1", os.F_OK)
    -1
    

    Notes:

    • Instead hardcoding libc's path ("/lib/x86_64-linux-gnu/libc.so.6") which may (and most likely, will) vary across systems, None (or the empty string) can be passed to CDLL constructor (ctypes.CDLL(None).access(b"/tmp", os.F_OK)). According to [man]: DLOPEN(3):

      If filename is NULL, then the returned handle is for the main program. When given to dlsym(), this handle causes a search for a symbol in the main program, followed by all shared objects loaded at program startup, and then all shared objects loaded by dlopen() with the flag RTLD_GLOBAL.

      • Main (current) program (python) is linked against libc, so its symbols (including access) will be loaded
      • This has to be handled with care, since functions like main, Py_Main and (all the) others are available; calling them could have disastrous effects (on the current program)
      • This doesn't also apply to Win (but that's not such a big deal, since msvcrt.dll is located in "%SystemRoot%\System32" which is in %PATH% by default). I wanted to take things further and replicate this behavior on Win (and submit a patch), but as it turns out, [MSDN]: GetProcAddress function only "sees" exported symbols, so unless someone declares the functions in the main executable as __declspec(dllexport) (why on Earth the regular person would do that?), the main program is loadable but pretty much unusable
  5. Install some 3rd Party module with filesystem capabilities

    Most likely, will rely on one of the ways above (maybe with slight customizations).
    One example would be (again, Win specific) [GitHub]: Python for Windows (pywin32) Extensions, which is a Python wrapper over WINAPIs.

    But, since this is more like a workaround, I'm stopping here.

  6. Another (lame) workaround (gainarie) is (as I like to call it,) the sysadmin approach: use Python as a wrapper to execute shell commands

    • Win:

      (py35x64_test) e:\Work\Dev\\q000082831>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\System32\\cmd.exe\" > nul 2>&1'))"
      0
      
      (py35x64_test) e:\Work\Dev\\q000082831>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os; print(os.system('dir /b \"C:\\Windows\\System32\\cmd.exe.notexist\" > nul 2>&1'))"
      1
      
    • Lnx (Ubtu):

      [cfati@cfati-ubtu16x64-0:~]> python3 -c "import os; print(os.system('ls \"/tmp\" > /dev/null 2>&1'))"
      0
      [cfati@cfati-ubtu16x64-0:~]> python3 -c "import os; print(os.system('ls \"/tmp.notexist\" > /dev/null 2>&1'))"
      512
      

Bottom line:

  • Do use try / except / else / finally blocks, because they can prevent you running into a series of nasty problems. A counter-example that I can think of, is performance: such blocks are costly, so try not to place them in code that it's supposed to run hundreds of thousands times per second (but since (in most cases) it involves disk access, it won't be the case).

Final note(s):

  • I will try to keep it up to date, any suggestions are welcome, I will incorporate anything useful that will come up into the answer



Python 3.4+ has an object-oriented path module: pathlib. Using this new module, you can check whether a file exists like this:

import pathlib
p = pathlib.Path('path/to/file')
if p.is_file():  # or p.is_dir() to see if it is a directory
    # do stuff

You can (and usually should) still use a try/except block when opening files:

try:
    with p.open() as f:
        # do awesome stuff
except OSError:
    print('Well darn.')

The pathlib module has lots of cool stuff in it: convenient globbing, checking file's owner, easier path joining, etc. It's worth checking out. If you're on an older Python (version 2.6 or later), you can still install pathlib with pip:

# installs pathlib2 on older Python versions
# the original third-party module, pathlib, is no longer maintained.
pip install pathlib2

Then import it as follows:

# Older Python versions
import pathlib2 as pathlib



It doesn't seem like there's a meaningful functional difference between try/except and isfile(), so you should use which one makes sense.

If you want to read a file, if it exists, do

try:
    f = open(filepath)
except IOError:
    print 'Oh dear.'

But if you just wanted to rename a file if it exists, and therefore don't need to open it, do

if os.path.isfile(filepath):
    os.rename(filepath, filepath + '.old')

If you want to write to a file, if it doesn't exist, do

# python 2
if not os.path.isfile(filepath):
    f = open(filepath, 'w')

# python 3, x opens for exclusive creation, failing if the file already exists
try:
    f = open(filepath, 'wx')
except IOError:
    print 'file already exists'

If you need file locking, that's a different matter.




import os.path

if os.path.isfile(filepath):



Additionally, os.access():

if os.access("myfile", os.R_OK):
    with open("myfile") as fp:
        return fp.read()

Being R_OK, W_OK, and X_OK the flags to test for permissions (doc).




if os.path.isfile(path_to_file):
    try: 
        open(path_to_file)
            pass
    except IOError as e:
        print "Unable to open file"

Raising exceptions is considered to be an acceptable, and Pythonic, approach for flow control in your program. Consider handling missing files with IOErrors. In this situation, an IOError exception will be raised if the file exists but the user does not have read permissions.

SRC: http://www.pfinn.net/python-check-if-file-exists.html




I'm the author of a package that's been around for about 10 years, and it has a function that addresses this question directly. Basically, if you are on a non-Windows system, it uses Popen to access find. However, if you are on Windows, it replicates find with an efficient filesystem walker.

The code itself does not use a try block… except in determining the operating system and thus steering you to the "Unix"-style find or the hand-buillt find. Timing tests showed that the try was faster in determining the OS, so I did use one there (but nowhere else).

>>> import pox
>>> pox.find('*python*', type='file', root=pox.homedir(), recurse=False)
['/Users/mmckerns/.python']

And the doc…

>>> print pox.find.__doc__
find(patterns[,root,recurse,type]); Get path to a file or directory

    patterns: name or partial name string of items to search for
    root: path string of top-level directory to search
    recurse: if True, recurse down from root directory
    type: item filter; one of {None, file, dir, link, socket, block, char}
    verbose: if True, be a little verbose about the search

    On some OS, recursion can be specified by recursion depth (an integer).
    patterns can be specified with basic pattern matching. Additionally,
    multiple patterns can be specified by splitting patterns with a ';'
    For example:
        >>> find('pox*', root='..')
        ['/Users/foo/pox/pox', '/Users/foo/pox/scripts/pox_launcher.py']

        >>> find('*shutils*;*init*')
        ['/Users/foo/pox/pox/shutils.py', '/Users/foo/pox/pox/__init__.py']

>>>

The implementation, if you care to look, is here: https://github.com/uqfoundation/pox/blob/89f90fb308f285ca7a62eabe2c38acb87e89dad9/pox/shutils.py#L190




Links



Tags

python python   file