Skip to content

fix: preserve JSON value types when building environment variables - #149

Open
husniadil wants to merge 1 commit into
securestart:mainfrom
husniadil:fix/json-value-stringification
Open

fix: preserve JSON value types when building environment variables#149
husniadil wants to merge 1 commit into
securestart:mainfrom
husniadil:fix/json-value-stringification

Conversation

@husniadil

Copy link
Copy Markdown

The problem

Secrets whose payload is JSON are turned into environment variables with
fmt.Sprintf("%v", v). That is Go's debug formatting, not JSON, so the value a
process receives is not the value that was stored:

Stored in the secret Reaches the process
1754110382 1.754110382e+09
["a","b"] [a b]
{"token":"secret"} map[token:secret]

The first row is the damaging one. JSON has no integer type, so every number
decodes as float64, and %v prints large float64 values in scientific
notation. Any large integer in a secret — a Unix timestamp, an account id, a
port — arrives corrupted
, and nothing reports an error.

Small integers are unaffected (5432 stays 5432), which is part of why this
has gone unnoticed: it looks fine until a value crosses the threshold where Go
switches to exponent form.

Why nothing caught it

The conversion cannot fail. %v accepts any value and always returns a string,
so a type it handles badly still looks like a successful fetch. The failure
surfaces later, in the process that consumes the variable, far from the cause.

The fix

internal/provider/value.go adds two helpers:

  • DecodeSecretJSON decodes with UseNumber(), so numbers keep the text they
    were written with instead of round-tripping through float64.
  • StringifyValue renders scalars literally and re-encodes arrays and objects
    as JSON, so a structured value stays parseable by the receiving process.

Scope

Four providers decode JSON payloads and are affected:

  • aws_secretsmanager
  • gcloud_secretmanager
  • azure_keyvault
  • bitwarden (in note format)

1password, infisical and bitwarden_sm build string-only maps from their
SDKs, so %v was already a no-op there. They move to the shared helper anyway
so the behaviour cannot drift apart later — those three lines are not bug fixes.

Behaviour that is deliberately unchanged: a payload that is not JSON still
falls back to a single <PROVIDER_ID>_SECRET variable. DecodeSecretJSON
rejects trailing content after a JSON object, which json.Unmarshal also
rejected but a bare json.Decoder would have accepted.

null becomes an empty string, matching the existing behaviour for a missing
value. Previously it produced the literal <nil>.

Verification

  • TestE2E_JSONValueTypes runs a mixed-type payload through LocalStack and
    AWS Secrets Manager end to end. Reverting the helper to the old behaviour
    makes it fail on exactly the three cases above:

    Secret 'SCOPES': expected '["read","write"]', got '[read write]'
    Secret 'NESTED': expected '{"token":"sk-secret"}', got 'map[token:sk-secret]'
    Secret 'EXPIRES_AT': expected '1754110382', got '1.754110382e+09'
    
  • Unit tests for both helpers, including the timestamp case and the
    non-JSON/trailing-content fallbacks.

  • The full -short ./tests/end2end/... suite passes locally with Docker
    (437s), unchanged from before.

  • go build ./... and go vet clean on every touched package.

internal/provider/vault and internal/provider/gcsm unit tests fail on
main today, before and after this branch. That is unrelated and is what
#148 addresses.

Docs

CONFIGURATION.md gains a Value Types table under Key Mappings. It sits there
rather than in each provider section because the conversion is shared by every
provider that parses JSON.

Secrets whose payload is JSON were rendered with fmt.Sprintf("%v"), which is
Go's debug formatting rather than JSON. The value a process received was then
not the value that was stored:

  1754110382          -> "1.754110382e+09"
  ["a","b"]           -> "[a b]"
  {"token":"secret"}  -> "map[token:secret]"

The first case is the damaging one. JSON has no integer type, so every number
decodes as float64, and %v prints large float64 values in scientific notation.
Any large integer in a secret — a Unix timestamp, an account id, a port —
arrived corrupted, with no error to indicate it.

Nothing surfaced this because the conversion cannot fail: %v accepts any value
and always produces a string, so a type it handles badly still looks like a
successful fetch. The failure appears later, in the process that receives the
value.

DecodeSecretJSON keeps numbers as json.Number so their original text survives,
and StringifyValue re-encodes arrays and objects as JSON so the receiving
process can parse them back. Scalars keep their literal form.

Four providers decode JSON payloads and are affected: aws_secretsmanager,
gcloud_secretmanager, azure_keyvault, and bitwarden in 'note' format.
1password, infisical and bitwarden_sm build string-only maps from their SDKs,
so %v was already a no-op there; they move to the shared helper so the
behaviour cannot drift apart later.

Payloads that are not JSON still fall back to being treated as a single value,
including the case of trailing content after a JSON object, which
json.Unmarshal rejected and a bare json.Decoder would not.

CONFIGURATION.md gains a Value Types table under Key Mappings, since the
conversion applies to every provider that parses JSON rather than to one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hmh2p2Bg6kmxxvzpFDW2WL
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant