Skip to main content

Secrets & environments

We use Doppler as the source of truth for secrets and environment-specific configuration. Python code declares the variables it needs with typed environment classes, and deployment tools decide which Doppler project or platform environment provides the values.

Doppler projects​

Doppler is organized around deployment ownership:

  • Each deployed app or service has its own project, such as database, datasmart, data-tools, automation-scales-api, automation-ms-sync, automation-ms-graph-proxy, or stoneware-v2.
  • The shared project owns values that are genuinely shared across deployments, such as SharePoint and Microsoft credentials.
  • The monolith-dev project is the local-development aggregate. It contains or references all variables needed for normal development so notebooks, scripts, and partially configured app environments can still run locally.

App-specific values should live in the app's Doppler project. Shared credentials should live in shared and be referenced into app projects or monolith-dev rather than copied by hand.

Python model​

Environment variables are declared in small EnvVars classes near the code that owns the capability:

from shared.utils.environment_variables import EnvVars, env_var


class DatabaseEnv(EnvVars):
    PGURL: str
    PROD_PGURL: str = env_var(fallback="PGURL")

Applications compose those capability classes into one app-local ProjectEnv in env.py:

from shared.db.env import DatabaseEnv
from shared.sharepoint.env import SharePointEnv
from shared.utils.environment_variables import ProjectEnv, RuntimeEnv


class DatasmartEnv(ProjectEnv, RuntimeEnv, DatabaseEnv, SharePointEnv):
    _doppler_project = "datasmart"


env = DatasmartEnv.current()

App code should import from its app-local environment:

from datasmart.env import env

pg_url = env.PGURL

Shared utilities should import or accept the capability class that owns their required variables:

from shared.sharepoint.env import SharePointEnv


class SharePointClient:
    def __init__(self, env: SharePointEnv | None = None):
        self.env = env or SharePointEnv.current(lazy=True)

For notebooks and ad-hoc development, shared.env remains available as a compatibility facade:

from shared.env import PGURL

New application code should prefer the app-local env object. New shared utility code should add or reuse an EnvVars capability class instead of adding unrelated variables to a central file.

Loading order​

In development, ProjectEnv.current() loads values in this effective precedence order:

  1. Process environment variables.
  2. The app's Doppler project, using the active config, usually dev.
  3. Shared development fallback projects, monolith-dev by default.
  4. The repository-root .env file.

MONO_PROJECT, MONO_DOPPLER_PROJECT, and MONO_DOPPLER_CONFIG are set by mono run for configured tools. A class can also pin its project with _doppler_project = "project-name".

In production, Doppler is not loaded by application code. Secrets should be injected by the platform. Dagster Cloud deployments can load from AWS Secrets Manager when BRIMSTONE_AWS_SECRETS_PATH is set; Railway and similar platforms provide process environment variables directly.

Adding or changing a secret​

  1. Decide ownership.
    • App-only secret: add it to the app's Doppler project.
    • Shared credential: add it to shared, then reference it from the app projects and monolith-dev.
    • Development-only aggregate value: add or reference it in monolith-dev.
  2. Add the value to every config that needs it, typically dev and prd.
  3. Add the variable to code.
    • App-specific variable: define it on the app-local ProjectEnv class.
    • Shared capability variable: define it in the owning EnvVars class, such as shared.sharepoint.env, shared.db.env, or shared.s3.env.
  4. Import the app-local env in app code, or the owning capability class in shared library code.
  5. Run the app or a focused import/test locally so missing variables fail before deployment.

Descriptor options​

Use env_var() when a variable needs behavior beyond "required string":

from typing import Optional

from shared.utils.environment_variables import EnvVars, env_var


class ServiceEnv(EnvVars):
    API_TOKEN: str
    PUBLIC_MODE: bool = False
    LEGACY_TOKEN: str = env_var(fallback="API_TOKEN")
    OPTIONAL_WEBHOOK: Optional[str] = env_var(optional=True)
    AWS_ACCESS_KEY_ID: Optional[str] = env_var(
        optional=True,
        set_in_process=True,
    )

Supported field types are str, bool, Optional[str], and Optional[bool]. Required variables are verified when the environment object is created unless they are marked lazy=True.