Python Dependency Management Made Simple

Here you will understand dependency management in Python, its importance and its tools. Let’s explore!

Are you tired of struggling with Python dependency management? I know I certainly was. Dependencies are external libraries and modules that our code needs to run properly. Managing dependencies can be a major headache when dealing with multiple projects or collaborating with other developers. Here are a few simple approaches to Python dependency management that have made my life much easier.  

Explore free Python courses

In this article, I will walk you through the basics of dependency management, show you some common tools and techniques for managing dependencies, and help you understand how to keep your Python projects organized and efficient. So, without further ado, let’s dive in! 

Explore Online Python Courses

We will be covering the following sections: 

• Quick Intro to Dependency Management in Python 
• Why is Dependency Management Important? 
• Tools for Dependency Management 
• Virtual Environments 
  • Virtualenv
  • Pip 
  • Conda 
  • Poetry 

Quick Intro to Dependency Management in Python 

Dependency management identifies, tracks, and resolves the dependencies our code needs to run. In other words, it ensures that all the external libraries and modules our code relies on are available and working properly. 

Why is Dependency Management Important? 

There are several reasons why dependency management is important. First and foremost, it helps ensure that our code works as intended. If a dependency is missing or not working properly, our code may fail to run or produce incorrect results. Additionally, dependency management helps us avoid conflicts between different versions of the same library or module, which can cause all sorts of issues.  

Tools for Dependency Management 

Thankfully, several tools and techniques are available to help us manage dependencies in Python. Here are a few of the most popular: 

Virtual Environments 

Virtual environments are a powerful tool for managing dependencies. A virtual environment is a self-contained Python environment isolated from your system’s global environment. This means you can install dependencies specific to a project without worrying about conflicts with other projects or system-level dependencies. I recommend using virtual environments as a first step in managing dependencies for any project.  

Why do we need Python Virtual Environments?

Isolation: The primary reason for using virtual environments is isolation. It’s important to have a separate environment for each project. Without virtual environments, the packages installed for one project may conflict with those installed for another. This can cause issues that are difficult to debug.

Reproducibility: Virtual environments also make reproducing your project’s environment easy. When you share your project with others, they can easily create a virtual environment with the same packages as yours.

Flexibility: Virtual environments allow you to experiment with different package versions. For example, you can create a virtual environment for testing the latest package versions without affecting the stable environment of your main project.

Security: Virtual environments provide an additional layer of security. If you accidentally install a malicious package, it will only affect the virtual environment and not your system’s Python environment.

Creating Python Virtual Environments

A tool called virtualenv simplifies dependency management by creating isolated environments for each project, which can be customized with specific dependencies without interfering with the system’s global environment. Let’s discuss this tool in detail, shall we? 

What is Virtualenv? 

Virtualenv is a Python tool that allows you to create isolated Python environments. These environments are independent of the global system environment and can contain their dependencies without interfering with the dependencies of other projects or the system itself. Virtualenv can be installed on all major operating systems, including Windows, macOS, and Linux. 

Installing Virtualenv 

To install Virtualenv, you must first install Python on your system. If you haven’t installed Python yet, you can download it from the official website and follow the installation instructions for your operating system. Once Python is installed, you can use the following command to install Virtualenv using pip, which is Python’s package manager: 

pip install virtualenv 
Creating a Virtual Environment 

Creating a Virtual Environment 

Once Virtualenv is installed, you can create a new virtual environment for your project by running the following command: 

virtualenv myenv 

This will create a new virtual environment called myenv in the current directory. You can choose any name you like for your virtual environment. 

Activating a Virtual Environment 

After creating a virtual environment, you need to activate it before installing any dependencies. To activate the virtual environment, run the following command: 

• On macOS or Linux: 

source myenv/bin/activate 

• On Windows: 

myenvScriptsactivate 

When the virtual environment is activated, you should see the name of the environment in the terminal prompt. This indicates that the virtual environment is active, and any packages installed using pip will be installed into this environment. 

Installing Dependencies 

With the virtual environment active, you can now install the dependencies for your project. To install a package using pip, run the following command: 

pip install package_name 

Replace package_name with the name of the package you want to install. This will install the package and all its dependencies into the virtual environment. 

Saving Dependencies 

It’s a good practice to keep track of the dependencies for your project, so you can easily recreate the environment on a different machine or share it with others. To save the dependencies for your project, you can use the following command: 

pip freeze > requirements.txt 

This will create a file called requirements.txt that lists all the packages installed in the virtual environment and their version numbers. 

Deactivating a Virtual Environment 

When you’re finished working on a project, you can deactivate the virtual environment by running the following command: 

deactivate 

This will deactivate the virtual environment and return you to the global system environment. 

Pip 

Pip is the default package manager for Python. It allows installing, upgrading, and removing packages from the Python Package Index (PyPI) and other sources. You can use pip to install packages globally or within a virtual environment.  

Installing pip 

Before you can use pip to manage dependencies, you need to install it. Fortunately, pip comes pre-installed with Python versions 2.7.9, Python 3.4 and later. If pip is not installed on your system, you can install it by running the following command: 

Python -m ensurepip –upgrade 

This will install pip and upgrade it to the latest version. 

Installing a Package 

To install a package using pip, you need to run the following command: 

pip install package_name 

Replace package_name with the name of the package you want to install. For example, to install the popular NumPy package, you would run the following command: 

pip install numpy 

pip will download the package from the Python Package Index (PyPI) and install it on your system. 

Installing a Specific Version of a Package 

Sometimes, you may need to install a specific version of a package. To do this, you can use the following command: 

pip install package_name==version_number 

Replace package_name with the package’s name and version_number with the specific version you want to install. For example, to install version 1.19.2 of NumPy, you would run the following command: 

pip install numpy==1.19.2 

Upgrading a Package 

To upgrade a package to the latest version, you can use the following command: 

pip install --upgrade package_name 

Replace package_name with the name of the package you want to upgrade. For example, to upgrade NumPy to the latest version, you would run the following command: 

pip install --upgrade numpy 

Removing a Package 

To remove a package, you can use the following command: 

pip uninstall package_name 

Replace package_name with the name of the package you want to remove. For example, to remove NumPy, you would run the following command: 

pip uninstall numpy 

Installing Packages from a Requirements File 

If you have a list of packages that you need to install, you can create a requirements file that contains a list of package names and versions. For example, your requirements file may look like this: 

numpy==1.19.2 

pandas==1.2.3 

matplotlib==3.3.4 k

To install all the packages listed in the requirements file, you can use the following command: 

pip install -r requirements.txt 

Replace requirements.txt with the name of your requirements file. 

Conda 

Conda is a package management system specifically for data science and scientific computing. It allows you to install, update, and manage packages in multiple languages (including Python) and environments. Conda is particularly useful for managing complex dependencies and creating reproducible environments. 

Installing Conda 

Before you can use Conda to manage dependencies, you need to install it. Conda is included in the Anaconda distribution, which provides a complete scientific computing environment for Python. To install Anaconda, you can download it from the official website.  

Creating a Conda Environment 

One of the benefits of using Conda is that it allows you to create isolated environments for each project. These environments can have their own Python version and package dependencies, making managing dependencies easy and avoiding conflicts between projects. 

To create a new environment using Conda, you can use the following command: 

conda create --name env_name python=3.9 

Replace env_name with the name you want to give your environment. You can also specify the Python version you want to use; in this case, we’re using Python 3.9. 

Activating and Deactivating an Environment 

To activate an environment, you can use the following command: 

conda activate env_name 

Replace env_name with the name of the environment you want to activate. Once activated, any packages you install will be installed in that environment. 

To deactivate an environment, you can use the following command: 

conda deactivate 

Installing a Package 

To install a package using Conda, you need to run the following command: 

conda install package_name 

Replace package_name with the name of the package you want to install.  

Installing a Specific Version of a Package 

To install a specific version of a package, you can use the following command: 

conda install package_name=version_number 

Replace package_name with the package’s name, and version_number with the specific version you want to install.  

Upgrading a Package 

To upgrade a package to the latest version, you can use the following command: 

conda update package_name 

Replace package_name with the name of the package you want to upgrade.  

Removing a Package 

To remove a package, you can use the following command: 

conda remove package_name 

Replace package_name with the name of the package you want to remove.  

Exporting and Importing an Environment 

Once you have installed all the packages you need in your environment, you can export it to share with other developers or deploy it to a server. To export an environment, you can use the following command: 

conda env export > environment.yml  

This will create a file called environment.yml that comprises a list of all the packages installed in the environment. To create an environment from this file, you can use the following command: 

conda env create -f environment.yml 

This will create a new environment with the same packages as the exported environment. 

Poetry 

Poetry is a relatively new tool for Python dependency management. It aims to simplify the process of managing dependencies and creating virtual environments while providing support for package building, publishing, and versioning. Poetry uses a lock file to ensure that all dependencies are installed with the correct versions, and it provides a simple command-line interface for managing packages.  

Installing Poetry 

Before you can use Poetry to manage dependencies, you need to install it. You can install Poetry by running the following command in your terminal: 

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - 

Alternatively, you can download the installer from the official website. 

Creating a Poetry Project 

To create a new Poetry project, you can use the following command: 

poetry new project_name 

Replace project_name with the name of your project. This will create a new project directory with a basic project structure. 

Adding Dependencies 

To add a dependency to your project, you can use the following command: 

poetry add package_name 

Replace package_name with the name of the package you want to add. Poetry will download the package from the Python Package Index (PyPI) and add it to your project’s pyproject.toml file. 

You can also specify a specific version of the package to install by using the following command: 

poetry add package_name==version_number 

Installing Dependencies 

To install all the dependencies for your project, you can use the following command: 

poetry install 

This will read the pyproject.toml file and install all the dependencies in your project’s virtual environment. 

Updating Dependencies 

To update all the dependencies for your project to their latest versions, you can use the following command: 

poetry update 

This will update all the dependencies in your project’s virtual environment to their latest versions. 

Removing Dependencies 

To remove a dependency from your project, you can use the following command: 

poetry remove package_name 

Replace package_name with the name of the package you want to remove. 

Creating a Lock File 

Poetry creates a lock file (poetry.lock) that stores the exact versions of all the dependencies for your project. This file ensures that all developers working on the project use the same versions of the dependencies. 

To create a lock file, you can use the following command: 

poetry lock 

This will read the pyproject.toml file and generate a lock file with the exact versions of all the dependencies. 

Endnotes 

Managing dependencies in Python can be daunting, but it’s essential to any software development project. By using the right tools and techniques, and following best practices, we can simplify and more manageable the process. This article has given you a good introduction to Python dependency management. Happy coding!   

Contributed By: Prerna Singh