Environment variables

The Peril of Hardcoding Credentials: A Real-World Scenario

The primary motivation for adopting secure storage practices stems from the severe risks associated with hardcoding credentials. Consider this real-world scenario: a developer is building an application that integrates with Mpesa Daraja. In a rush, or due to a lack of awareness, they hardcode the CONSUMER_KEY and CONSUMER_SECRET directly into their Python application:

# mpesa_client.py - INSECURE EXAMPLE
import requests
import base64
 
# DANGER: Hardcoded Mpesa Daraja API credentials
CONSUMER_KEY = "dfgHjK8lMnOpQ2rS1tUvWxYzA3bC4dE5"
CONSUMER_SECRET = "fGhIjKlM6nOpQrStUvW7xYzAbC8dE9fG"
 
def get_mpesa_access_token():
    """
    Obtains an Mpesa Daraja OAuth 2.0 access token using hardcoded credentials.
    """
    # Concatenate consumer key and secret with a colon
    credentials = f"{CONSUMER_KEY}:{CONSUMER_SECRET}"
    
    # Base64 encode the credentials
    encoded_credentials = base64.b64encode(credentials.encode('utf-8')).decode('utf-8')
 
    # Daraja API endpoint for authentication
    auth_url = "https://sandbox.safaricom.co.ke/oauth/v1/generate?grant_type=client_credentials"
 
    headers = {
        "Authorization": f"Basic {encoded_credentials}",
        "Content-Type": "application/json"
    }
 
    try:
        response = requests.get(auth_url, headers=headers)
        response.raise_for_status() # Raise an exception for HTTP errors
        token_data = response.json()
        access_token = token_data.get("access_token")
        expires_in = token_data.get("expires_in")
        
        if access_token:
            print("Got an M-Pesa access token.")  # never print the token itself
            print(f"Token expires in: {expires_in} seconds")
            return access_token
        else:
            print("Failed to retrieve access token from response.")
            return None
    except requests.exceptions.RequestException as e:
        print(f"Error during token retrieval: {e}")
        if response is not None:
            print(f"Response status code: {response.status_code}")
            print(f"Response body: {response.text}")
        return None
 
if __name__ == "__main__":
    token = get_mpesa_access_token()
    if token:
        print("Mpesa integration ready with token.")
    else:
        print("Mpesa integration failed: No token obtained.")

Execution Trace (Hypothetical - if credentials were valid and hardcoded):

1. The mpesa_client.py script is executed.
2. The Python interpreter loads the script, and the CONSUMER_KEY and CONSUMER_SECRET variables are immediately assigned their literal string values.
3. The get_mpesa_access_token() function is called.
4. Inside the function, the credentials string is formed by combining the key and secret: "dfgHjK8lMnOpQ2rS1tUvWxYzA3bC4dE5:fGhIjKlM6nOpQrStUvW7xYzAbC8dE9fG".
5. The base64.b64encode() function converts this string into a Base64-encoded byte string. Base64 is an encoding scheme that transforms binary data into an ASCII string format, making it safe for transmission in text-based protocols like HTTP. It is not encryption. The result is then decoded back to a UTF-8 string for the Authorization header.
6. A requests.get() call makes an HTTP GET request to the Daraja authentication endpoint. This request includes the Authorization header, which contains the Base64-encoded credentials.
7. The Daraja API validates these credentials. If they are correct, it returns a JSON response containing an access_token and an expires_in value.
8. The script then parses this JSON response and prints the obtained token.

💡 What is happening here: The application uses the hardcoded secret values to construct a special authorization string. This string is then encoded (not encrypted) and sent to the Daraja API to prove the application's identity and get an access token. The entire process relies on those initial, hardcoded secrets.

The Public GitHub Repository Catastrophe: Imagine this mpesa_client.py file is part of a larger project. The developer, perhaps intending to share their work or collaborate, pushes their code to a public GitHub repository.

• Immediate Exposure: As soon as the code is pushed, the CONSUMER_KEY and CONSUMER_SECRET are publicly visible to anyone who views the repository. They are now exposed to the entire internet.
• Version History: Even if the developer realizes their mistake and removes the credentials in a subsequent commit, the sensitive data remains permanently in the Git history. Anyone can clone the repository and use git log or git blame to find the commit where the credentials were first introduced, effectively retrieving the "deleted" secrets.
• Automated Scanners: Malicious actors and security researchers actively scan public GitHub repositories for hardcoded secrets. Tools are designed to detect patterns resembling API keys, database credentials, and other sensitive information. Once detected, these keys can be immediately exploited.
• Consequences: With the Mpesa Daraja CONSUMER_KEY and CONSUMER_SECRET, an attacker can generate their own access tokens and potentially:
  • Initiate unauthorized Mpesa transactions.
  • Query sensitive transaction data.
  • Cause financial losses to the application owner or their users.
  • Disrupt the service by exhausting API rate limits or making fraudulent requests.

This scenario underscores why hardcoding is an unacceptable practice for any sensitive data, especially in a collaborative or public development environment.

Solution for Development: Environment Variables and .env Files

For local development and testing, environment variables, often managed via .env files, offer a significant improvement over hardcoding. They provide a simple yet effective way to keep credentials out of source code.

Mechanism of .env Files and python-dotenv

When using .env files in a Python project, the python-dotenv library is commonly employed. This library helps load variables from .env files into the application's environment.

Step-by-step mechanism:

1. .env file creation: A developer creates a file named .env in the root directory of their project. This file contains key-value pairs for environment variables, such as DARAJA_CONSUMER_KEY="your_value".
2. .gitignore entry: The .env file is immediately added to the project's .gitignore file. This crucial step instructs Git to ignore the .env file, preventing it from being committed to the version control repository.
3. python-dotenv installation: The python-dotenv library is installed in the project's virtual environment using a package manager like pip (pip install python-dotenv).
4. Loading in code: At the entry point of the application, dotenv.load_dotenv() is called. This function reads the key-value pairs from the .env file and adds them to the os.environ dictionary, making them accessible as standard environment variables within the Python process.
5. Accessing variables: The application then retrieves the values using os.environ.get("VARIABLE_NAME"), just as it would for any other environment variable.

💡 What is happening here: This process ensures that sensitive values are never directly written into the code or committed to Git. Instead, they are stored in a local, ignored file, and a special library loads them into the application's memory when it starts, keeping them secure during development.

This approach ensures that the sensitive values are never part of the source code that gets committed to Git. Each developer can have their own .env file with their specific credentials (e.g., development keys), and the production environment will use different mechanisms (e.g., actual system environment variables or a secret management system).

Implementation Example: Mpesa Daraja Authentication with .env

Let's refactor the previous insecure example to use .env files for local development.

1. Create a .env file: In your project's root directory, create a file named .env with the following content (replace with your actual sandbox credentials):

# .env
DARAJA_CONSUMER_KEY="your_sandbox_consumer_key_here"
DARAJA_CONSUMER_SECRET="your_sandbox_consumer_secret_here"

2. Add .env to .gitignore: Ensure your .gitignore file contains the line:

# .gitignore
.env

This prevents the .env file from being committed to your Git repository.

3. Install python-dotenv:

pip install python-dotenv requests

4. Modify mpesa_client.py to load from .env:

# mpesa_client.py - SECURE FOR DEVELOPMENT EXAMPLE
import os
import requests
import base64
from dotenv import load_dotenv
 
# 1. Load environment variables from .env file
# This line should be called as early as possible in your application's lifecycle.
load_dotenv()
 
def get_mpesa_access_token():
    """
    Obtains an Mpesa Daraja OAuth 2.0 access token using credentials from environment variables.
    """
    # 2. Retrieve credentials from environment variables
    CONSUMER_KEY = os.environ.get("DARAJA_CONSUMER_KEY")
    CONSUMER_SECRET = os.environ.get("DARAJA_CONSUMER_SECRET")
 
    # 3. Validate that the environment variables are set
    if not CONSUMER_KEY or not CONSUMER_SECRET:
        print("Error: DARAJA_CONSUMER_KEY and DARAJA_CONSUMER_SECRET environment variables must be set.")
        print("Please ensure you have a .env file with these variables or they are set in your shell.")
        return None
 
    # Concatenate consumer key and secret with a colon
    credentials = f"{CONSUMER_KEY}:{CONSUMER_SECRET}"
    
    # Base64 encode the credentials
    encoded_credentials = base64.b64encode(credentials.encode('utf-8')).decode('utf-8')
 
    # Daraja API endpoint for authentication (using sandbox for development)
    auth_url = "https://sandbox.safaricom.co.ke/oauth/v1/generate?grant_type=client_credentials"
 
    headers = {
        "Authorization": f"Basic {encoded_credentials}",
        "Content-Type": "application/json"
    }
 
    try:
        response = requests.get(auth_url, headers=headers)
        response.raise_for_status() # Raise an exception for HTTP errors (4xx or 5xx)
        token_data = response.json()
        access_token = token_data.get("access_token")
        expires_in = token_data.get("expires_in")
        
        if access_token:
            print(f"Successfully obtained Mpesa Access Token.")
            # For security, avoid printing the raw token in logs in production.
            print(f"Token expires in: {expires_in} seconds")
            return access_token
        else:
            print("Failed to retrieve access token from response.")
            print(f"Response body: {response.text}")
            return None
    except requests.exceptions.RequestException as e:
        print(f"Error during token retrieval: {e}")
        if response is not None:
            print(f"Response status code: {response.status_code}")
            print(f"Response body: {response.text}")
        return None
 
if __name__ == "__main__":
    print("Attempting to get Mpesa Daraja Access Token...")
    token = get_mpesa_access_token()
    if token:
        print("Mpesa integration ready with token.")
    else:
        print("Mpesa integration failed: No token obtained.")

Line-by-Line Operational Explanation:

1. import os: This line imports the os module, which provides a way for Python programs to interact with the operating system, including accessing environment variables.
2. import requests: This imports the requests library, a popular tool for making HTTP requests to web services like the Mpesa Daraja API.
3. import base64: This imports the base64 module, used for encoding credentials into a Base64 format as required by the Daraja API for Basic Authentication.
4. from dotenv import load_dotenv: This specifically imports the load_dotenv function from the python-dotenv library.
5. load_dotenv(): This is the crucial step for local development. When this line executes, python-dotenv actively searches for a .env file in the current directory (or its parent directories). If found, it reads each KEY=VALUE pair from the file and adds KEY to the os.environ dictionary with its corresponding VALUE. This effectively makes the variables defined in the .env file available as standard system environment variables for the current Python process.
6. CONSUMER_KEY = os.environ.get("DARAJA_CONSUMER_KEY"): The os.environ.get() method is used to safely retrieve the value associated with the DARAJA_CONSUMER_KEY environment variable. Using .get() is safer than direct dictionary access (os.environ["KEY"]) because it returns None if the variable is not set, preventing a program-crashing KeyError.
7. CONSUMER_SECRET = os.environ.get("DARAJA_CONSUMER_SECRET"): This line similarly retrieves the DARAJA_CONSUMER_SECRET from the environment variables.
8. if not CONSUMER_KEY or not CONSUMER_SECRET:: This block performs essential validation. It checks if the environment variables were successfully loaded and are not empty. If either is None (meaning it wasn't found), the program prints a clear error message and exits gracefully. This prevents the application from proceeding with missing credentials, adhering to the "fail-fast" principle.
9. credentials = f"{CONSUMER_KEY}:{CONSUMER_SECRET}": The retrieved key and secret are concatenated (joined together) with a colon separator, as specifically required by the Daraja API for constructing the Basic Authentication string.
10. encoded_credentials = base64.b64encode(credentials.encode('utf-8')).decode('utf-8'): The concatenated string is first encoded into bytes using utf-8 and then Base64 encoded. Finally, it's decoded back to a string so it can be correctly included in the HTTP header.
11. auth_url = "https://sandbox.safaricom.co.ke/oauth/v1/generate?grant_type=client_credentials": This line defines the specific Daraja API endpoint used for obtaining an access token. Using the sandbox.safaricom.co.ke URL is appropriate for development and testing without affecting live financial transactions.
12. headers = {...}: This constructs the HTTP headers that will be sent with the request. It includes the Authorization header, which contains the Base64-encoded credentials prefixed by "Basic ", indicating the type of authentication.
13. response = requests.get(auth_url, headers=headers): This line makes the actual HTTP GET request to the Daraja API using the constructed URL and headers.
14. response.raise_for_status(): This is a critical error-handling step. It checks if the HTTP response status code indicates an error (any 4xx client error or 5xx server error). If an error status is found, it raises an HTTPError exception, which is then caught by the try...except block, allowing for controlled error handling.
15. token_data = response.json(): If the request is successful, this line parses the JSON response received from the Daraja API. This response typically contains the access_token and expires_in values.
16. access_token = token_data.get("access_token"): This line safely extracts the access_token from the parsed JSON data.
17. The remaining code handles printing success or failure messages and returns the obtained token, or None if an error occurred.

Advantages of .env files for Development:

• Security (Local): Keeps sensitive credentials out of the source code and version control, significantly mitigating the risk of accidental exposure in public repositories.
• Ease of Use: Simple to set up and manage for individual developers. Each developer can have their own .env file with their specific credentials (e.g., development keys) without affecting others.
• Flexibility: Allows easy switching between different sets of credentials (e.g., for different sandbox accounts) by simply modifying the .env file.
• Standard Practice: Widely adopted across various programming ecosystems, making it familiar to many developers.

Limitations of .env files (especially for Production):

• Not for Production: .env files are generally not suitable for production environments.
  • Scalability: Managing .env files across multiple servers or container instances becomes cumbersome, difficult to synchronize, and highly error-prone.
  • Security (Production): They are plain text files on the file system. If a server is compromised, these files are easily discoverable. They lack the robust encryption at rest, fine-grained access control, comprehensive auditing, and automated rotation capabilities inherent in dedicated secret management systems.
  • Deployment: Integrating .env files securely into automated deployment pipelines can be tricky and less secure than injecting environment variables directly or using dedicated secret managers.