Installation.save() persists plaintext Databricks credentials (token/password/client_secret) when a Config field is included #410

Summary

Installation.save() serializes a dataclass to JSON/YAML and writes it to a durable WorkspaceFS file. When one of the dataclass fields is a databricks.sdk.core.Config (a documented, supported field type -- see the save() docstring), the blueprint marshals it through Config.as_dict(), which returns the SDK's raw internal dict. In that dict the sensitive=True attributes -- token (PAT), password (basic auth), client_secret (OAuth) -- are stored unmasked, so the plaintext secret is written straight into the workspace file.

The SDK itself treats these as sensitive: it marks them sensitive=True and masks them as *** in its own debug_string()/repr. But Config.as_dict()/to_dict() do not apply that masking, and the blueprint persists the unmasked result.

Root cause (source-traced)

src/databricks/labs/blueprint/installation.py:

• save() -> _marshal(type_ref, [], inst) (line ~500).
• _marshal dispatch: if isinstance(inst, databricks.sdk.core.Config): return self._marshal_databricks_config(inst).
• _marshal_databricks_config(inst):

current_client_config = self._current_client_config()   # self._ws.config.as_dict()
remote_file_config = inst.as_dict()                      # <-- raw dict incl. token/password
if current_client_config == remote_file_config:
    return None, True                                    # skip only if identical to live client
return remote_file_config, True                          # else WRITE it (secret included)

• _overwrite_content() -> upload() -> self._ws.workspace.upload(dst, raw, ...) -> durable WorkspaceFS.

SDK side (databricks-sdk config.py, verified on 0.33.0 and 0.139.0):

token: str = ConfigAttribute(env='DATABRICKS_TOKEN', auth='pat', sensitive=True)
password: str = ConfigAttribute(env='DATABRICKS_PASSWORD', auth='basic', sensitive=True)
client_secret: str = ConfigAttribute(env='DATABRICKS_CLIENT_SECRET', auth='oauth', sensitive=True)

class ConfigAttribute:
    def __set__(self, cfg, value):
        cfg._inner[self.name] = self.transform(value)   # transform defaults to str -> RAW value

class Config:
    def as_dict(self):
        return self._inner                               # <-- raw, unmasked

So as_dict()['token'] == the plaintext PAT.

Proof of concept (reproduced)

poc_token_leak.py builds a Config(host=..., token="dapi3fS7SECRET_TOKEN_value_42"), wraps it in a dataclass field, and save()s it through the real blueprint marshaling (MockInstallation overrides only the file sink, not the marshaling):

--- runtime.json ---
{'config': {'host': 'https://dbc-1234567890.cloud.databricks.com',
            'token': 'dapi3fS7SECRET_TOKEN_value_42',
            'http_timeout_seconds': 2.0, 'retry_timeout_seconds': 2, 'auth_type': 'pat'}}

[LEAK] plaintext PAT token present in persisted file: True

Also confirmed the basic-auth path:

basic-auth serialized config: {'host': ..., 'username': 'alice', 'password': 'P@ssw0rd-basic', ...}
[LEAK] password present: True

Repro:

python3 -m venv v && v/bin/pip install "databricks-sdk>=0.16.0,<1.0"
PYTHONPATH=<blueprint>/src v/bin/python poc_token_leak.py

(The SDK's Config() makes a best-effort /.well-known/databricks-config probe; set http_timeout_seconds=2, retry_timeout_seconds=2 to fail it fast offline -- the probe itself prints the token masked as ***, i.e. the SDK knows it's sensitive.)

Impact

A Databricks PAT is a long-lived bearer credential granting full access to that workspace. Persisting it in plaintext to a durable WorkspaceFS file that (a) outlives the process that created it and (b) is readable by other workspace users per ACLs, exposes workspace-admin access to anyone with read on that file (and to anyone the file is shared/synced/versioned to). Same for basic-auth password and OAuth client_secret.

Reachability / scope (stated honestly)

• Trigger: a caller save()s a dataclass (or nested list/dict) that contains a databricks.sdk.core.Config field. This is a documented supported field type (the save() docstring lists "The Databricks SDK configuration class: databricks.sdk.core.Config"), so it's reachable by design, not a corner case.
• _marshal_databricks_config compares against the current live client config and skips the write only when identical; when the persisted config differs (different host/token, or a config reconstructed from a DSN/env that carries a token), the raw secret is written.
• Consuming Databricks Labs products (ucx, quality-monitor, etc.) are the realistic callers; exact prevalence of "Config-as-persisted-field" varies per product.

Suggested fix

In _marshal_databricks_config, redact sensitive keys before persisting -- e.g. drop token/password/client_secret/google_credentials/azure_client_secret from the as_dict() output (or replace with None/a sentinel), since the SDK already flags them ConfigAttribute.sensitive. Persisting a connection (host, cloud, account_id) is fine; persisting the secret is the bug.

Duplicate check

No existing issue found for plaintext token/password/secret persistence in the serialized config (searched: token, secret, leak, credential, password, as_dict). Nearest are #245 "Logging context", #246 CLI auth UX, #58 "brute-forcing SerdeError with as_dict()" -- none about secret-in-file.

This report was produced with AI assistance (Claude, Anthropic); the PoC above was run and verified locally against the real databricks-sdk and blueprint source.