Using Py Dotenv (python-dotenv) Package to Manage Env Variables

Before co-founding Configu, Ran was a full stack developer at Testim.io and previously served in an elite cybersecurity unit in the Israeli Defense Forces.

What Is Python Dotenv? 

Python Dotenv is an open-source software library that allows software developers to specify the environment in which their code runs. When working on a project, it is often necessary to have different settings for different environments. For example, you might want to use different database settings for development and production environments. Python Dotenv allows you to specify these settings in a file, rather than hardcoding them into your application.

The library reads from a .env file at the root of your project and loads them into os.environ. These environment variables can then be accessed in your code. This tool helps to manage and control the environment variables for your Python application.

Python Dotenv helps to keep your project’s configuration separate from your code, which is a recommended practice. This separation allows for a cleaner and more manageable codebase. 

Why Use Python Dotenv? 

There are several reasons why you should consider using Python Dotenv in your projects:

Separation of Configuration from Code

In a typical application, there are many settings that you might want to change based on the environment in which the code is running. These include things like database connection details, API keys, and other configuration settings.

By using Python Dotenv, you can keep these settings in a separate file, making your code cleaner and easier to manage. It also makes your code more secure, as sensitive data like API keys are not exposed in the codebase. This separation means that you can share your code without worrying about exposing sensitive data.

Ease of Use and Consistency

Python Dotenv is easy to use. All you need to do is create a .env file in the root directory of your project and start adding your environment variables. When your application starts, Python Dotenv will automatically load these variables into the environment.

This simplifies the management of your environment variables and ensures consistency across all environments. Whether you’re working on your local machine, a test server, or a production server, you can ensure that your application is running with the correct settings.

Transparency and Debugging

When an error occurs in your application, it can be challenging to figure out exactly what went wrong. With Python Dotenv, you can easily see the environment in which your code is running. This transparency makes it easier to reproduce bugs and understand the conditions under which they occur. It also allows you to easily change your environment settings for testing purposes, without having to modify your code.

Getting Started with Python Dotenv 

Installation

To begin using Python Dotenv, you first need to install it. Python makes this process simple with the pip package manager. Open your terminal or command prompt, and run the following command. Use pip3 if you are using Python 3.

pip install python-dotenv

This command fetches the python-dotenv package from the Python Package Index (PyPI) and installs it on your system. Once you’ve installed it, you’re ready to start using Python Dotenv in your projects. This is a third-party package, so you’ll need to import it in your Python files to use it.

Creating the .env File

Your .env file will contain all the environment variables that you want to use in your project.

To create a .env file, navigate to your project directory in the terminal and use the touch command as follows:

touch .env

This command creates an empty .env file in your project directory. Open this file using your preferred text editor, and add your environment variables in the form of KEY=VALUE pairs. For example, if you’re working on a web application, you might have a .env file that looks like this:

DEBUG=True
SECRET_KEY=your-secret-key
DATABASE_URL=postgres://user:password@localhost:5432/mydatabase

Loading Environment Variables with Python Dotenv

After you’ve set up your .env file, Python Dotenv allows you to load these environment variables into your Python project. This process involves using the load_dotenv function from the Python Dotenv package.

Here’s a simple example of how you might use load_dotenv in a Python script:

import os
from dotenv import load_dotenv

load_dotenv()

# Now you can access your environment variables using os.getenv
debug = os.getenv('DEBUG')
secret_key = os.getenv('SECRET_KEY')
database_url = os.getenv('DATABASE_URL')

print(f"Debug: {debug}\n Secret key: {secret_key}\n Database URL: {database_url}"

The output should look like this:

In this example, the load_dotenv function loads the environment variables from your .env file, and the os.getenv function retrieves these variables so that you can use them in your script.

Managing Variables in Different Environments

Python Dotenv allows you to manage variables across different environments. For instance, you might have different database configurations for your development, testing, and production environments. Instead of hardcoding these configurations into your Python scripts, you can define them in separate .env files and load the appropriate file based on the current environment.

To manage variables in different environments, you can use the find_dotenv function from the Python Dotenv package. This function searches for the .env file in your project directory and any parent directories, allowing you to specify different .env files for different environments. 

Here’s an example. For this code to work, the project folder should have two .env files: .env.development and .env.production.

import os
from dotenv import load_dotenv, find_dotenv

# Find the correct .env file based on the current environment
env_file = find_dotenv(f'.env.{os.getenv("ENV", "development")}')

# Load the environment variables from the .env file
load_dotenv(env_file)

# Now you can access your environment variables using os.getenv
debug = os.getenv('DEBUG')
secret_key = os.getenv('SECRET_KEY')
database_url = os.getenv('DATABASE_URL')

print(f"Debug: {debug}\n Secret key: {secret_key}\n Database URL: {database_url}"

In this example, the find_dotenv function tries to find a .env file that matches the current environment (which you can set using the ENV environment variable). This enables you to easily switch between different configurations by changing the ENV variable.

To switch to the production environment, use the command export ENV=production

Now when you run your program, production environment variables will be loaded:

Tips From the Expert

In my experience, here are tips that can help you better manage environment variables using Python Dotenv:

1. Set default values: Use os.getenv('KEY', 'default_value') to prevent errors from missing variables.
2. Use dotenv for local only: Load secrets into production via the environment, not the .env file.
3. Validate variables early: Check required variables on app startup to avoid runtime surprises.
4. Secure .env file permissions: Limit access to prevent unauthorized reads.
5. Leverage dotenv-linter: Automate the detection of common issues in .env files.
6. Group related settings logically: Name variables consistently across environments (e.g., DB_HOST, DB_PORT).

Ran Cohen

CTO & Co-Founder. Configu

Before co-founding Configu, Ran was a full stack developer at Testim.io and previously served in an elite cybersecurity unit in the Israeli Defense Forces.

Python Dotenv Best Practices 

Keep .env Out of Version Control

When you place sensitive information like API keys, database credentials, and secret keys in your .env file, you want to ensure that they remain confidential. Exposing these through version control systems like Git could lead to a security breach.

You’ll need to add your .env file to your .gitignore file. This way, your .env file will not be tracked by Git, and it won’t be pushed to your remote repository. You also need to double-check this by typing git status in your terminal to confirm that your .env file is not being tracked.

Remember, your .env file is for your local environment alone. For other environments, like staging and production, you should set these environment variables directly on the server. This way, you ensure the security of your sensitive data while maintaining the integrity of your codebase.

Ensure You Don’t Hard-Code Fallbacks

Hard-coding values in your application can lead to a number of problems, such as difficulty in debugging, increased complexity, and the risk of exposing sensitive data.

To avoid these problems, you should use the os.getenv() function. This function returns the value of the environment variable, if it exists. If it doesn’t exist, it will return None. This gives you the flexibility to handle missing environment variables in a way that suits your application, without hard-coding fallbacks.

Moreover, using os.getenv() allows you to maintain consistency across different environments. You can easily switch between different sets of environment variables by changing your .env file, without needing to modify your application code. This makes your code cleaner, easier to manage, and more secure.

Avoid Overly Complex .env Structures

You might be tempted to use nested structures in your .env  file to group related configuration variables together. However, this can lead to confusion and make the file harder to manage.

Instead, you should keep your .env  file as simple as possible. Use flat structures, with one environment variable per line. Each line should consist of a key and a value, separated by an equals sign. This makes your .env file easy to read and update.

You should use descriptive names for your environment variables. This makes it clear what each variable is used for, which can help you and other developers working on your codebase to quickly understand the configuration of your application.

Consider Security when Storing Secrets

You should be aware of the potential security risks involved in storing secrets in your .env  file. If an attacker gains access to your .env file, they could gain access to all your application’s secrets.

To mitigate this risk, you should ensure that your .env file is only accessible to the necessary personnel. This means setting appropriate permissions on your .env file to restrict who can read and write to it. In addition, consider using encryption for particularly sensitive secrets. While this adds complexity to your application, it can significantly enhance the security of your secrets.

As an alternative, you can store sensitive configuration information in a dedicated configuration management solution. This can help you manage multiple environments in Python while avoiding any exposure of sensitive details.

Related content: Read our guide to npm dotenv

Managing Python Environment Variables with Configu

Configu is a configuration management platform comprised of two main components:

Configu Orchestrator

As applications become more dynamic and distributed in microservices architectures, configurations are getting more fragmented. They are saved as raw text that is spread across multiple stores, databases, files, git repositories, and third-party tools (a typical company will have five to ten different stores).

The Configu Orchestrator, which is open-source software, is a powerful standalone tool designed to address this challenge by providing configuration orchestration along with Configuration-as-Code (CaC) approach.

Configu Cloud

Configu Cloud is the most innovative store purpose-built for configurations, including environment variables, secrets, and feature flags. It is built based on the Configu configuration-as-code (CaC) approach and can model configurations and wrap them with unique layers, providing collaboration capabilities, visibility into configuration workflows, and security and compliance standardization.

Unlike legacy tools, which treat configurations as unstructured data or key-value pairs, Configu is leading the way with a Configuration-as-Code approach. By modeling configurations, they are treated as first-class citizens in the developers’ code. This makes our solution more robust and reliable and also enables Configu to provide more capabilities, such as visualization, a testing framework, and security abilities.

Learn more about Configu