<a href=”https://github.com/you“><img style=”position: absolute; top: 0; right: 0; border: 0;” src=”https://camo.githubusercontent.com/365986a132ccd6a44c23a9169022c0b5c890c387/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f7265645f6161303030302e706e67” alt=”Fork me on GitHub” data-canonical-src=”https://s3.amazonaws.com/github/ribbons/forkme_right_red_aa0000.png“></a>

depfinder

https://travis-ci.org/ericdill/depfinder.svg?branch=master http://codecov.io/github/ericdill/depfinder/coverage.svg?branch=master

Using depfinder

$ depfinder -h
usage: depfinder [-h] [-y] [-V] [--no-remap] [-v] [-q] file_or_directory

Tool for inspecting the dependencies of your python project.

positional arguments:
  file_or_directory  Valid options are a single python file, a single jupyter
                     (ipython) notebook or a directory of files that include
                     python files

optional arguments:
  -h, --help         show this help message and exit
  -y, --yaml         Output in syntactically valid yaml when true. Defaults to
                     False
  -V, --version      Print out the version of depfinder and exit
  --no-remap         Do not remap the names of the imported libraries to their
                     proper conda name
  -v, --verbose      Enable debug level logging info from depfinder
  -q, --quiet        Turn off all logging from depfinder

Ok, great. That’s the help output. Not super helpful. What does the output of depfinder look like when we run it on the source code for depfinder?

$ depfinder depfinder
{'builtin': ['__future__',
             'argparse',
             'ast',
             'collections',
             'copy',
             'errno',
             'json',
             'logging',
             'os',
             'pprint',
             're',
             'subprocess',
             'sys'],
 'relative': ['_version', 'main'],
 'required': ['pyyaml', 'stdlib-list']}

So, what do these things mean? Well builtin are modules that are built in to the standard library. required are modules that are not from the standard library or from within the depfinder package and relative are modules that are imported from one module to another within the depfinder source code.

Also see this notebook

Description

Find all the unique imports in your library, automatically, because who likes do it by hand? depfinder uses the ast (Abstract Syntax Tree) module (and more ast docs) to find all ast.Try and ast.ImportFrom nodes. These ast.Import and ast.ImportFrom nodes are then grouped according to the following categories, in order of decreasing precedence:

  • relative

    The import is a relative import from within the same library

  • builtin

    The import is built into the standard library, as determined by scraping the official python docs for the builtins with stdlib-list

  • questionable

    The import occurs inside any combination of

    • ast.Try (ast.TryExcept on py27)
    • ast.FunctionDef
    • ast.ClassDef

    The module may be importable without these imports, but the it will likely not have full functionality.

  • required

    The import occurs at the top level of the module and will get executed when the module is imported. These imports must be accounted for in an environment, or the module will not be importable.

Installation

depfinder is on pypi. It is tested against Python 2.7, 3.3, 3.4 and 3.5.

pip install depfinder

It is available via conda.

conda install -c ericdill depfinder

It is also via github.

git clone git@github.com:ericdill/depfinder
cd depfinder
python setup.py install

or

pip install https://github.com/ericdill/depfinder/zipball/master#egg=depfinder

It has one dependency, stdlib_list, which is where I get the list of libraries built in to the standard library. stdlib-list can be installed via pip

pip install stdlib-list

or conda

conda install -c ericdill stdlib-list

API

depfinder.main.get_imported_libs(code)[source]

Given a code snippet, return a list of the imported libraries

Parameters:code (str) – The code to parse and look for imports
Returns:The ImportCatcher is the object in depfinder that contains all the information regarding which imports were found where. You will most likely be interested in calling the describe() function on this return value.
Return type:ImportCatcher

Examples

>>> depfinder.get_imported_libs('from foo import bar')
{'required': {'foo'}, 'questionable': set()}
>>> with open('depfinder.py') as f:
        code = f.read()
        imports = depfinder.get_imported_libs(code)
        print(imports.describe())
{'builtin': {'__future__', 'json', 'ast', 'os', 'sys', 'collections'},
 'required': {'stdlib_list'}}
depfinder.main.iterate_over_library(path_to_source_code)[source]

Helper function to recurse into a library and find imports in .py files.

This allows the user to apply filters on the user-side to exclude imports based on their file names. conda-skeletor makes heavy use of this function

Parameters:path_to_source_code (str) –
Yields:catchers (tuple) – Yields tuples of (module_name, full_path_to_module, ImportCatcher)

Return all imported modules in all .py files in path_to_source_code

Parameters:
  • path_to_source_code (str) –
  • remap (bool, optional) – Normalize the import names to be synonymous with their conda/pip names
Returns:

The list of all imported modules, sorted according to the keys listed in the docstring of depfinder.ImportCatcher.describe()

Return type:

dict

Examples

>>> depfinder.simple_import_search('/path/to/depfinder/source')
{'builtin': ['__future__',
             'ast',
             'collections',
             'json',
             'os',
             'shlex',
             'sys',
             'tempfile'],
 'required': ['depfinder',
              'nbformat',
              'pytest',
              'setuptools',
              'sphinx_rtd_theme',
              'stdlib_list',
              'test_with_code']}

IPython/Jupyter Notebook support

depfinder has support for v4 Jupyter notebooks.

depfinder.main.notebook_path_to_dependencies(path_to_notebook, remap=True)[source]

Helper function that turns a jupyter notebook into a list of dependencies

Parameters:
  • path_to_notebook (str) –
  • remap (bool, optional) – Normalize the import names to be synonymous with their conda/pip names
Returns:

Dict of dependencies keyed on

  • ‘builtin’ - libraries built in to python
  • ‘required’ - libraries that are found at the top level of your modules
  • ‘questionable’ - libraries that are found inside try/except blocks
  • ‘relative’ - libraries that are relative imports

Return type:

dict

Examples

>>> depfinder.notebook_path_to_dependencies('depfinder_usage.ipynb')
{'builtin': ['os', 'pprint'], 'required': ['depfinder']}