Improve dependency health of your Python projects with deptry

Effective dependency management is relatively straightforward when dealing with a small-scale project with a limited number of dependencies. However, complexity arises with the growth of a project, especially when it encompasses multiple interdependent internal projects and various external Python modules. This growth often introduces a web of dependency issues.

A practical solution for staying on top of dependency health is the tool ‘deptry’. This command-line utility serves as an effective linter that identifies dependency-related problems within a Python project. By integrating deptry into your repository’s CI pipeline, you can maintain a robust defense against several dependency issues.

deptry currently has 4 different types of checks that it performs for your project’s dependencies:

1. Missing Dependencies: Dependencies that are imported in code but not present in the project requirements (Code: DEP001)
2. Unused Dependencies: Dependencies that are in the project requirements but not used in the code (Code: DEP002)
3. Transitive Dependencies: Dependencies that are used in the code but aren’t part of the project’s explicit requirements and are instead available because they are dependencies of the project’s dependencies (Code: DEP003)
4. Use of development dependencies: If your code is using dependencies mentioned as development dependencies (Code: DEP004)

deptry supports the following types of projects:

• Projects that use Poetry and a corresponding pyproject.toml file
• Projects that use PDM and a corresponding pyproject.toml file
• Projects that use any package manager that strictly follows PEP 621 dependency specification
• Projects that use a requirements.txt file according to the pip standards

I will discuss the use of deptry for one of our poetry projects (using poetry 1.7.1). You can add deptry as a dev dependency for your poetry project as follows:

poetry add - group dev deptry

Usage:

deptry <directory1> <directory2>

When I ran deptry for my project and got 2,176 dependency issues:

Now that seems like too many issues and surely my project wasn’t in that bad of a state. Upon inspection, I realized most dependency issues arose because deptry did not detect some first-party Python modules in my project and some files that should have been excluded from this test. To remedy this, I add the following configurations to my pyproject.toml to avoid warnings/errors for the cases where they were unnecessary.

[tool.deptry]
known_first_party = ["foo", "bar"]
extend_exclude = ["tools/directory", "tools/dir2/file.py"]

First under [tool.deptry], I had to add some first-party modules that were not being detected correctly and thus deptry was throwing errors on things like:

from foo import ComputeClass

When foo was a first-party dependency.

I also added some subdirectories and files that should be excluded from dependency analysis because they had some development files that weren’t production code and were raising issues for the DEP004 check. By default deptry already excludes: [“venv”, “.venv”, “.direnv”, “tests”, “.git”, “setup.py”]. Therefore, I am simply extending the excluded list and adding some other directories/files. There is an option to replace this list completely.

After this, I re-ran deptry and got 278 dependency issues:

Investigate it further and there were several genuine issues in all 4 categories and a couple of issues that were supposed to be ignored.

Because I was starting with an existing poetry project and I had several transitive dependency issues. So I couldn’t just do poetry add <dependency> since the latest version of the dependency might have breaking changes. So I used the existing poetry.lock to get the versions of the transitive dependencies that were already working and explicitly added those versions in the pyproject.toml

Then, I added certain dependencies to be excluded on a per-rule basis. This was pytest for the dev dependency check and also another as an unused dependency which we needed to pin. Both of these are for project-specific reasons that I wouldn’t get into. But this was just to show how to add exclusion rules for deptry.

[tool.deptry.per_rule_ignores]
DEP004 = ["pytest"]
DEP002 = ["dependencyx"]

Re-ran deptry again and finally the green signal that every developer wants to see

All in all, I added 19 dependencies in my pyproject.toml that were previously transitive dependencies and removed 7 unused dependencies. Now with deptry integrated into the CI, we can be sure that these issues don’t slip past without notice.

For more information visit the deptry homepage. The documentation is short and simple.

Do you use another method to keep an eye on the health of your dependencies? If so, please share.