bug: OTEL_EXPORTER_OTLP_ENDPOINT in the environment breaks trigger deploy (depot build)

[cloudbring]

Provide environment information

System:
  OS: macOS 15 (also reproduced on Linux, GitHub Actions ubuntu-24.04)
Binaries:
  Node: 22.x
  npm / pnpm / bun: any
trigger.dev CLI 4.5.4, @trigger.dev/sdk 4.5.4, @trigger.dev/build 4.5.4
@depot/cli 0.0.1-cli.2.80.0 (pinned by trigger.dev 4.5.4)

Describe the bug

In plain terms: OTEL_EXPORTER_OTLP_ENDPOINT is a normal, standard
environment variable that lots of projects already set — it tells OpenTelemetry
where to send traces. If that variable is set when you run trigger deploy, the
deploy fails while building the image. Remove the variable and the exact same
project deploys fine.

That's the bug: a value the user put in their own environment, for their own
unrelated tooling, breaks trigger.dev's build. A user-set environment variable
shouldn't be able to do that. The deploy should either ignore it or keep it out
of the build, not fall over on it. The error messages also point at "depot" and
look like an outage, so it's easy to lose a lot of time blaming the wrong thing
(we did) instead of realizing one of your own env vars is the cause.

Here's what actually happens under the hood.

If OTEL_EXPORTER_OTLP_ENDPOINT is set in the environment, trigger deploy
fails during the remote (depot) image build, before the build really starts.
The variable's value only changes which error you get:

• a valid URL (e.g. http://localhost:4318) →
  Error: cannot merge resource due to conflicting Schema URL
• any non-URL value (in our case a dotenvx encrypted:… blob) →
  failed to build resolver: passthrough: received empty target in Build()

Both surface as Error building image, with that line as the last log. Unset
the variable and the same project deploys fine on the same depot backend
seconds later. --local-build is unaffected.

We chased this for a while as flaky depot infrastructure (the empty target
error reads like a depot outage). It isn't — it's deterministic per-environment,
and it's the OTLP endpoint variable leaking into the depot build.

Root cause. The remote build runs the depot CLI via
packages/cli-v3/src/deploy/buildImage.ts:

const childProcess = depot(args, {
  cwd: options.cwd,
  env: {
    DEPOT_BUILD_ID: options.buildId,
    DEPOT_TOKEN: options.buildToken,
    DEPOT_PROJECT_ID: options.buildProjectId,
    DEPOT_NO_SUMMARY_LINK: "1",
    DEPOT_NO_UPDATE_NOTIFIER: "1",
  },
});

depot() (from @depot/cli) runs the binary through execa, whose default is
extendEnv: true. So that env object is merged onto the parent process.env
rather than replacing it — the depot CLI inherits every variable trigger deploy has, including OTEL_EXPORTER_OTLP_ENDPOINT. The depot CLI then reads it
for its own OTel export and dies during telemetry init (empty gRPC target for a
malformed value, resource Schema-URL conflict for a valid one) before it builds
anything.

The value gets into the environment easily. A common case is a .env file: the
CLI loads it with plain dotenv, so whatever is in it ends up in the
environment the depot child inherits. (Ours is encrypted with dotenvx, so the
value the CLI sees is the literal encrypted:… string — a non-URL — which is
why we hit the empty target variant rather than the Schema-URL one.) Setting
the variable directly in the shell does the same thing.

Reproduction repo

https://github.com/cloudbring/trigger-depot-otel-endpoint-repro

To reproduce

From the repo above (trivial single task, nothing project-specific):

npm install
npx trigger.dev@4.5.4 login
export TRIGGER_PROJECT_REF=proj_xxxxxxxx

# fails — .env sets OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
npm run deploy

# succeeds — same project, same code, variable removed
mv .env .env.off && npm run deploy

Or without the file:

OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 npx trigger.dev@4.5.4 deploy   # fails
npx trigger.dev@4.5.4 deploy                                                     # succeeds

Additional information

--local-build is a working workaround, and so is unsetting
OTEL_EXPORTER_OTLP_ENDPOINT for the deploy — the latter keeps you on remote
builds. The local path runs docker buildx (x("docker", ["buildx", …]) in the
same file) which inherits the same variable but tolerates it, so the hard
failure is specific to the depot CLI's telemetry init.

Since some ambient env (PATH, HOME, …) is needed for depot to run, dropping
extendEnv wholesale probably isn't the fix. Options that would work: strip or
neutralize the OTel exporter variables for the depot child (e.g. delete
OTEL_EXPORTER_OTLP_ENDPOINT / set OTEL_SDK_DISABLED=true in that env), or
have the depot CLI not fail the build when its own telemetry endpoint is bad.
Happy to open a PR for the CLI-side version if that's the direction you'd want.