fix: preserve JSON value types when building environment variables - #149
Open
husniadil wants to merge 1 commit into
Open
fix: preserve JSON value types when building environment variables#149husniadil wants to merge 1 commit into
husniadil wants to merge 1 commit into
Conversation
Secrets whose payload is JSON were rendered with fmt.Sprintf("%v"), which is
Go's debug formatting rather than JSON. The value a process received was then
not the value that was stored:
1754110382 -> "1.754110382e+09"
["a","b"] -> "[a b]"
{"token":"secret"} -> "map[token:secret]"
The first case is the damaging one. JSON has no integer type, so every number
decodes as float64, and %v prints large float64 values in scientific notation.
Any large integer in a secret — a Unix timestamp, an account id, a port —
arrived corrupted, with no error to indicate it.
Nothing surfaced this because the conversion cannot fail: %v accepts any value
and always produces a string, so a type it handles badly still looks like a
successful fetch. The failure appears later, in the process that receives the
value.
DecodeSecretJSON keeps numbers as json.Number so their original text survives,
and StringifyValue re-encodes arrays and objects as JSON so the receiving
process can parse them back. Scalars keep their literal form.
Four providers decode JSON payloads and are affected: aws_secretsmanager,
gcloud_secretmanager, azure_keyvault, and bitwarden in 'note' format.
1password, infisical and bitwarden_sm build string-only maps from their SDKs,
so %v was already a no-op there; they move to the shared helper so the
behaviour cannot drift apart later.
Payloads that are not JSON still fall back to being treated as a single value,
including the case of trailing content after a JSON object, which
json.Unmarshal rejected and a bare json.Decoder would not.
CONFIGURATION.md gains a Value Types table under Key Mappings, since the
conversion applies to every provider that parses JSON rather than to one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hmh2p2Bg6kmxxvzpFDW2WL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Secrets whose payload is JSON are turned into environment variables with
fmt.Sprintf("%v", v). That is Go's debug formatting, not JSON, so the value aprocess receives is not the value that was stored:
17541103821.754110382e+09["a","b"][a b]{"token":"secret"}map[token:secret]The first row is the damaging one. JSON has no integer type, so every number
decodes as
float64, and%vprints largefloat64values in scientificnotation. Any large integer in a secret — a Unix timestamp, an account id, a
port — arrives corrupted, and nothing reports an error.
Small integers are unaffected (
5432stays5432), which is part of why thishas gone unnoticed: it looks fine until a value crosses the threshold where Go
switches to exponent form.
Why nothing caught it
The conversion cannot fail.
%vaccepts any value and always returns a string,so a type it handles badly still looks like a successful fetch. The failure
surfaces later, in the process that consumes the variable, far from the cause.
The fix
internal/provider/value.goadds two helpers:DecodeSecretJSONdecodes withUseNumber(), so numbers keep the text theywere written with instead of round-tripping through
float64.StringifyValuerenders scalars literally and re-encodes arrays and objectsas JSON, so a structured value stays parseable by the receiving process.
Scope
Four providers decode JSON payloads and are affected:
aws_secretsmanagergcloud_secretmanagerazure_keyvaultbitwarden(innoteformat)1password,infisicalandbitwarden_smbuild string-only maps from theirSDKs, so
%vwas already a no-op there. They move to the shared helper anywayso the behaviour cannot drift apart later — those three lines are not bug fixes.
Behaviour that is deliberately unchanged: a payload that is not JSON still
falls back to a single
<PROVIDER_ID>_SECRETvariable.DecodeSecretJSONrejects trailing content after a JSON object, which
json.Unmarshalalsorejected but a bare
json.Decoderwould have accepted.nullbecomes an empty string, matching the existing behaviour for a missingvalue. Previously it produced the literal
<nil>.Verification
TestE2E_JSONValueTypesruns a mixed-type payload through LocalStack andAWS Secrets Manager end to end. Reverting the helper to the old behaviour
makes it fail on exactly the three cases above:
Unit tests for both helpers, including the timestamp case and the
non-JSON/trailing-content fallbacks.
The full
-short ./tests/end2end/...suite passes locally with Docker(437s), unchanged from before.
go build ./...andgo vetclean on every touched package.internal/provider/vaultandinternal/provider/gcsmunit tests fail onmaintoday, before and after this branch. That is unrelated and is what#148 addresses.
Docs
CONFIGURATION.mdgains a Value Types table under Key Mappings. It sits thererather than in each provider section because the conversion is shared by every
provider that parses JSON.