What Are Python Environment Variables?
Python environment variables are dynamic named values that can change the way running processes behave on a computer. They are part of the environment in which a process runs. For example, a running process can query the value of the TEMP environment variable to discover a suitable location to store temporary files, or the HOME or USERPROFILE variable to find the directory structure owned by the user running the process.
Environment variables provide a simple way to share configuration settings between multiple applications and processes in Linux. Python, like other languages, provides a straightforward way to interact with these environment variables via the os module.
In this article:
Common Use Cases for Environment Variables in Python
Here are a few ways you might use environment variables in your Python programs.
Application Configuration
Another common use of environment variables is application configuration. You might want to have different settings for your development, staging, and production environments. Rather than hardcoding these settings into your application, you can use environment variables to dynamically apply these configurations based on the current environment.
Learn more in our detailed guide to Python configuration file
Dynamic Resource Pointers
Environment variables can also be used as dynamic resource pointers. This means that you can use them to point to resources that your application needs to interact with, such as API URLs, service endpoints, or file paths. By using environment variables, you can easily change the resources your application is pointing to without having to modify your Python script.
Feature Flags and Toggles
Python environment variables can be used to implement feature flags or toggles within your application. Feature flags allow you to turn certain features of your application on or off without having to redeploy your whole application. This is particularly useful for testing new features in a production environment.
Related content: Read our guide to testing environment
Tips From the Expert
In my experience, here are tips that can help you better leverage environment variables in Python:
-
Implement caching for frequently accessed variables
Accessing environment variables repeatedly during runtime can incur a performance hit, especially in high-traffic systems. Cache frequently used variables at the start of your application to minimize access overhead. -
Sanitize and validate environment variable inputs
Environment variables are not inherently safe. Ensure your application validates the input format and content of environment variables before using them. For instance, verify if URLs are correctly formatted or if numeric variables fall within expected ranges. -
Use environment-specific configurations to streamline automation
For CI/CD pipelines or container orchestration systems, define different.envfiles or use specific tools (like Helm for Kubernetes) to dynamically inject variables based on the deployment environment, reducing manual intervention. -
Use multi-layered security for sensitive environment variables
Beyond storing secrets in environment variables, layer your security. Use Vault services (e.g., AWS Secrets Manager, HashiCorp Vault) combined with environment variables to further secure sensitive information, such as credentials or API keys. -
Leverage environment variables in microservices architecture
When using Python in microservices, use environment variables to inject service discovery information (like service endpoints or message broker URLs) into each service at runtime, ensuring flexible and scalable deployment without code changes.
Sensitive Information Storage
Environment variables are often used to store sensitive information. For instance, if you’re working on a web application that requires a database, you’ll need a username and password to access the database. Instead of hardcoding this information into your Python script (which is a significant security risk), you can store these values as environment variables.
Note: Environment variables also present a security risk, because they can be viewed by anyone with access to your environment. So it is preferred to store sensitive data using secrets.
How to Access Environment Variables from Python
Accessing environment variables from Python is straightforward. Python’s built-in os module provides a dictionary-like interface for querying environment variables. Here is an example of how you might access an environment variable:
import os
# Get the value of an environment variable
db_url = os.environ.get('DB_URL')
print(db_url)In this example, the os.environ.get('DB_URL') call fetches the value of the DB_URL environment variable, which provides the current URL of the database. If the DB_URL environment variable is not set, the os.environ.get() function will return None.
How to Set Environment Variables in Python
Setting environment variables in Python is also straightforward. You can set environment variables using the os.environ object. Here is an example:
import os
# Set an environment variable
os.environ['DB_URL'] = 'new-url'In this example, the os.environ['DB_URL'] = 'new-url' line sets the DB_URL environment variable to 'new-url'. You might do this when your database moves to a different place on the network. You, or others in your development team, can then use the os.environ.get() function to retrieve the updated value of the DB_URL environment variable.
5 Best Practices for Setting and Using Environment Variables in Python
To unlock the full potential of Python environment variables, it’s important to adopt best practices associated with their use. These variables are not just a way to store data; they represent a powerful mechanism for managing application configuration.
1. Avoid Hardcoding Sensitive Information
One of the foremost best practices associated with Python environment variables is avoiding the hardcoding of sensitive information directly into your application’s code. This practice is a common pitfall for many developers, particularly those new to the field, and can lead to severe security risks.
Hardcoding sensitive information like database credentials, API keys, or other secrets into your code can expose this information to anyone who has access to your codebase. This exposure can lead to security breaches, data leaks, and other serious consequences. By leveraging Python environment variables, you can store this sensitive information outside of your code.
However, it is important to note that environment variables are not a fully secure alternative to hard-coding sensitive information, because they can be viewed by anyone with access to your environment. To fully secure credentials and other sensitive data, store it as secrets with strong encryption.
2. Use Descriptive Names
Using descriptive names for your Python environment variables is another critical best practice. The names you choose for your variables should clearly indicate what data they hold, making your code easier to understand and maintain.
For example, a variable that stores your application’s database URL should be named something like 'DATABASE_URL'. This name is immediately clear and indicates exactly what data the variable holds. A less descriptive name, like 'DB_UL', may be unclear to other developers, making your code harder to understand and maintain.
3. Use .env Files for Local Development
Another best practice when working with Python environment variables is to use .env files for local development. These files allow you to define your environment variables in one place, making them easier to manage and update.
When you’re working on a Python project locally, you can define your environment variables in a .env file at the root of your project. This file should not be committed to your version control system, as it typically holds sensitive information. You can use a .gitignore file to avoid committing the real .env file to your repository.
Python has several libraries, like python-dotenv, that can load variables from a .env file into your environment. This practice makes your environment setup more straightforward and helps ensure that your application behaves consistently across different environments.
4. Provide Default Values
Providing default values for your Python environment variables is a best practice that can make your application more resilient and easier to use. Default values serve as a fallback if an environment variable is not set, allowing your application to continue functioning in a predictable manner.
In Python, you can provide a default value for an environment variable using the os.environ.get() method. This method takes two arguments: the name of the environment variable and the default value to return if the variable is not set.
5. Use Configuration Management Tools
Configuration management tools allow you to manage your application’s configuration in a centralized, organized manner, making it easier to manage complex applications with many configuration options.
These tools allow you to define your application’s configuration in code, including your Python environment variables. This practice, known as Infrastructure-as-Code (IaC), allows you to manage and version control your application’s configuration alongside your application’s code.
Using Python environment variables with configuration management tools provides several benefits. It makes your application’s configuration more visible and easier to manage, allows for automated and consistent environment setup, and makes your application easier to scale and deploy to different environments. Some configuration management tools also provide secrets functionality, allowing you to encrypt and securely store sensitive information.
Related content:
- GitHub Actions environment variables
- Docker environment variables
- Docker Compose environment variables
Managing Python Environment Variables with Configu
Configu is a configuration management platform comprised of two main components:
Configu Orchestrator
As applications become more dynamic and distributed in microservices architectures, configurations are getting more fragmented. They are saved as raw text that is spread across multiple stores, databases, files, git repositories, and third-party tools (a typical company will have five to ten different stores).
The Configu Orchestrator, which is open-source software, is a powerful standalone tool designed to address this challenge by providing configuration orchestration along with Configuration-as-Code (CaC) approach.
Configu Cloud
Configu Cloud is the most innovative store purpose-built for configurations, including environment variables, secrets, and feature flags. It is built based on the Configu configuration-as-code (CaC) approach and can model configurations and wrap them with unique layers, providing collaboration capabilities, visibility into configuration workflows, and security and compliance standardization.
Unlike legacy tools, which treat configurations as unstructured data or key-value pairs, Configu is leading the way with a Configuration-as-Code approach. By modeling configurations, they are treated as first-class citizens in the developers’ code. This makes our solution more robust and reliable and also enables Configu to provide more capabilities, such as visualization, a testing framework, and security abilities.
