Skip to content

TypeScript configuration pipeline replacing process-environment.sh #1673

Description

@kmcginnes

Parent PRD

#1670

What to build

Create a startup-config.ts module that replaces the environment variable reading and config merging logic from process-environment.sh. This module runs at server startup before the Express app is created.

The pipeline:

  1. Load .env via dotenv (provides defaults like LOG_LEVEL=info)
  2. Read process.env (Docker ENV vars override dotenv values)
  3. If config.json exists at CONFIGURATION_FOLDER_PATH, parse it using the config.json parser (TypeScript config.json parser with lenient JSON support #1671) and overlay values onto the environment. Config.json wins over env vars for connection-related fields (preserving current shell script precedence).
  4. If NEPTUNE_NOTEBOOK is truthy, force PROXY_SERVER_HTTPS_CONNECTION=false
  5. Pass the resolved values to the existing parseEnvironmentValues() Zod pipeline

Key details:

  • GRAPH_EXP_HTTPS_CONNECTION and NEPTUNE_NOTEBOOK are currently written to .env but never read by any TypeScript code — they are dead variables. Do not replicate this behavior.
  • Only PROXY_SERVER_HTTPS_CONNECTION actually matters for the server from the values the shell script appends to .env.
  • The shipped .env file stays unchanged. We stop appending to it.

Refer to the "Configuration Pipeline" and "Precedence Rules" sections of #1670 for details.

Acceptance criteria

  • Loads dotenv defaults, then process.env, then config.json overlay
  • Config.json values override env vars for connection-related fields
  • NEPTUNE_NOTEBOOK=true forces PROXY_SERVER_HTTPS_CONNECTION=false
  • Does not write to .env or any other file
  • Dead variables (GRAPH_EXP_HTTPS_CONNECTION, NEPTUNE_NOTEBOOK in .env) are not replicated
  • Feeds resolved values to existing parseEnvironmentValues() Zod pipeline
  • Unit tests cover: precedence rules, NEPTUNE_NOTEBOOK override, boolean case insensitivity, missing config.json, config.json with overrides

Blocked by

User stories addressed

  • User story 1
  • User story 4
  • User story 8
  • User story 9

Activity

  1. added theissue type on Apr 16, 2026
  2. mjuarros commented on Sep 24, 2026

    @mjuarros
    Contributor

    Proposed refreshed scope (grounded in current main, 2026-09-24)

    The original description predates the proxy-server refactor and has drifted. Proposing this rewrite for review before implementation.

    Scope clarification

    This ticket replaces only the config-resolution responsibility of process-environment.sh. It does not delete the script, rewire docker-entrypoint.sh, or stop writing defaultConnection.json. The script remains the live container entrypoint dependency until #1674 (in-memory /defaultConnection) and #1675 (delete scripts + change Dockerfile ENTRYPOINT) land. After this ticket the shell scripts stay on disk and the container behaves identically — the only change is that the server also resolves config in TypeScript (harmless transient duplication until #1675).

    What to build

    A startup-config.ts module in packages/graph-explorer-proxy-server/src/ exposing a pure, testable function (e.g. resolveStartupEnv) that runs at startup, before the Express app is created, and feeds the existing parseEnvironmentValues() pipeline in env.ts.

    Pipeline:

    1. Read process.env (dotenv already populates it in node-server.ts from .env.local then .env).
    2. If config.json exists at CONFIGURATION_FOLDER_PATH, overlay its values. Config.json wins over env vars for connection-related fields, preserving current shell precedence. This step depends on the TypeScript config.json parser with lenient JSON support #1671 parser (see Dependencies).
    3. If NEPTUNE_NOTEBOOK is truthy, force PROXY_SERVER_HTTPS_CONNECTION=false.
    4. Return a plain env record passed to parseEnvironmentValues().

    Integration points that already exist on main (do not rebuild)

    • env.ts — parseEnvironmentValues() + EnvironmentValuesSchema (Zod), case-insensitive BooleanStringSchema. Note LOG_LEVEL defaults to debug, not info (the original spec was wrong on this).
    • node-server.ts — already loads dotenv and calls parseEnvironmentValues → resolveServerConfig. Insert the new module between those two steps.
    • server-config.ts, paths.ts — unchanged.

    Key details:

    • GRAPH_EXP_HTTPS_CONNECTION and NEPTUNE_NOTEBOOK-in-.env are dead variables never read by TypeScript. Do not replicate them.
    • Only PROXY_SERVER_HTTPS_CONNECTION matters to the server from what the shell script writes to .env.
    • Write nothing to disk. The shipped .env at packages/graph-explorer/.env stays unchanged; stop appending to it.

    Coordination — read before implementing

    Draft PR #2252 (opened 2026-09-23) already adds NEPTUNE_NOTEBOOK to EnvironmentValuesSchema and makes resolveServerConfig reject a NEPTUNE_NOTEBOOK=true + PROXY_SERVER_HTTPS_CONNECTION=true conflict, with new config-pipeline.test.ts / server-config.test.ts cases. Reconcile with that PR so the notebook-override logic lands in exactly one place rather than being duplicated here.

    Acceptance criteria

    Env-only core (no #1671 needed):

    • resolveStartupEnv layers process.env and applies documented precedence
    • NEPTUNE_NOTEBOOK truthy forces PROXY_SERVER_HTTPS_CONNECTION=false (coordinated with Name the NEPTUNE_NOTEBOOK/HTTPS conflict instead of blaming missing certs #2252, not duplicated)
    • Boolean parsing is case-insensitive (via existing BooleanStringSchema)
    • Writes to no file
    • Dead variables (GRAPH_EXP_HTTPS_CONNECTION, NEPTUNE_NOTEBOOK in .env) are not replicated
    • Resolved values feed parseEnvironmentValues(); node-server.ts wired accordingly
    • Container behavior is unchanged (shell entrypoint still runs; this is additive)

    Config.json overlay (requires #1671):

    • config.json values override env vars for connection-related fields
    • Unit tests cover: missing config.json, config.json with overrides, precedence

    Testing

    • New unit tests call startup-config.ts directly (fast, no shell-out), porting the precedence / NEPTUNE_NOTEBOOK / boolean cases from the 31-test process-environment.test.ts.
    • Keep process-environment.test.ts and config-pipeline.test.ts green in parallel — they still cover the live shell path until Delete shell scripts and update Dockerfile entrypoint #1675.

    Dependencies

    Suggested merge chain for the full script removal

    #1671 → #1673 → #1674 → #1675, with #1672 (SSL in TS) parallelizable and activating at #1675. process-environment.sh is deleted at #1675, not here.


    Draft for review — not yet applied to the issue body.

  3. kmcginnes commented on Sep 25, 2026

    @kmcginnes
    CollaboratorAuthor

    The "preserve current shell precedence" assumption here doesn't hold end to end. The shell and the server pick different winners when a setting comes from both config.json and -e.

    • docs/references/configuration.md says the JSON wins, and process-environment.sh does that when it writes .env. docker-entrypoint.sh then decides on certificates from .env.
    • The server loads .env with dotenv, which never overrides a variable that's already set. So at the server, the -e value wins.
    • A key missing from config.json becomes an empty value in the shell, which drops the env value. -e NEPTUNE_NOTEBOOK=true with a config.json that has no NEPTUNE_NOTEBOOK key loses the notebook preset.
    • A value already in .env loses to the image's own ENV lines, since dotenv skips a variable that's already set. A mounted .env with PROXY_SERVER_HTTP_PORT=8080 still serves on 80, and the same goes for GRAPH_EXP_ENV_ROOT_FOLDER and LOG_STYLE.

    What that looks like today:

    • config.json HTTPS true with -e PROXY_SERVER_HTTPS_CONNECTION=false: the container generates certificates, needs HOST or it exits, then serves HTTP on port 80.
    • config.json HTTPS false with -e PROXY_SERVER_HTTPS_CONNECTION=true: certificates are skipped, then the server exits with "certificate files are missing".

    Before building this step we need to pick one rule for every setting. Making the server follow the docs isn't free. Anyone who copied the documented config.json example (HTTPS true) and passes -e ...=false because a load balancer terminates TLS is on HTTP today and would move to HTTPS on 443 on upgrade.

    The scenario table in config-pipeline.test.ts from #2252 pins each of these cases as they behave now, so a rule change will show up as flipped rows.

  4. self-assigned this
    on Nov 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestinfrastructureIssues related to tooling or deployment

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions