Repository navigation
fix(auth): expand policy variables in Resource and NotResource - #535
Open
LeeroyHannigan wants to merge 19 commits into
Open
LeeroyHannigan wants to merge 19 commits into
LeeroyHannigan wants to merge 19 commits into
Conversation
In a 2012-10-17 document, every ${key} in a Resource or NotResource
pattern is replaced by the request's value before ARN matching, with
${key, 'default'} falling back to its default and ${*}, ${?} and ${$}
producing literal characters. Text substituted from a value, a default
or an escape is matched literally; a wildcard written in the policy
still matches. A variable whose key the request does not carry, with no
default, makes the pattern match nothing. A 2008-10-17 document, or one
with no Version, matches the pattern as written, so PolicyDocument now
keeps an absent Version as None.
Condition values read the ${key, 'default'} form through the same module.
Measured with the IAM policy simulator. Closes #533.
… evaluator One test per measured row of issue #533, plus the escapes, the default form in a condition value, the 2008-10-17 and no-Version documents for NotResource, the permissions boundary and session policy paths, and explicit_deny_matches_arc. PolicyDocument::expands_policy_variables moves beside the expansion code so document.rs does not grow.
…in its field
Two rules of IAM's policy variable expansion, measured with the policy
simulator, that the Resource expansion did not follow:
- A multivalued condition key is never a policy variable. A Resource
pattern naming one matches no resource and a condition value naming one
is not expanded, with one value in the request or several. The shared
resolver used the first value, so an Allow on
table/${dynamodb:LeadingKeys} granted a table the caller chose.
- Only a colon written in the policy separates ARN fields. A colon inside a
substituted value is a byte of its field, so a region-field value of
us-east-1:123456789012 matches no table ARN, where it used to shift the
fields and match a stream ARN.
Tests pin both, variables in the region and account fields, an unresolved
variable before a wildcard, and a policy-text ? beside a variable.
LeeroyHannigan
requested review from
amrith,
c33howard,
jcshepherd,
pdf-amzn,
robinnsc and
yesyayen
as code owners
October 25, 2026 00:02
3 tasks done
Text substituted into a Resource pattern or a condition value from a
variable value or a default is matched as text: it is not expanded a
second time, and a `*` or `?` in it is not a wildcard. An Allow whose
NotResource pattern holds an unresolved variable grants every table.
The `${key, 'default'}` form, and the rule that a multivalued key is
never a variable, hold under every condition operator form, not only
bare StringEquals.
New evaluator tests pin each rule. A resource tag value of
`${aws:username}` names only the table `${aws:username}`, and the same
text reached through `${aws:UserAgent}` is compared as written in a
condition. The defaults `'*'` and `'b?'` match only `table/*` and
`table/b?`. `Allow NotResource table/${aws:PrincipalTag/team}` with no
tag grants `table/blue` and `table/red`. `${aws:PrincipalTag/team,'none'}`
without a space denies `table/none`. A `team` value of `bl*` denies only
`table/bl*`. The default form expands under ForAnyValue, ForAllValues,
IfExists and Null, and `${dynamodb:LeadingKeys}` stays text under
ForAnyValue, ForAllValues and IfExists.
The NotResource Allow, no-space default and `bl*` cases repeat rows the
IAM policy simulator measured; the other cases follow from those rules
and are labelled as not measured.
A condition on a key the service populates but the context cannot
resolve fails closed: a Deny applies and an Allow does not grant. A
`${key}` variable naming such a key in Resource or NotResource was read
as an absent key instead, so `Deny table/${key}` did not apply and
`Allow NotResource table/${key}` granted every table.
`resolve` now reports such a key on its own, without using a default,
and `resource_matches` fails closed when no pattern matched and one
names it. A pattern that matches, or one whose variable names an absent
key with no default, still decides alone. No key is listed in
`FAIL_CLOSED_GLOBAL_KEYS` in this build, so no decision changes today;
the comment on that list now names both places that read it. A variable
naming such a key inside a condition value is left as written, with or
without a default.
The tests mark `aws:PrincipalOrgID` unresolvable through the test
context. Deny on `table/${aws:PrincipalOrgID}`, with and without a
default, and Deny NotResource block GetItem on `table/T`. Allow on
`table/${aws:PrincipalOrgID, 'T'}` and Allow NotResource do not grant
it. `table/T` in the same list, or an absent key in the same pattern,
still decides.
Comments in the policy variable code and tests cited cells of a table in an issue comment (C1, D1 and so on), which a reader of the tree cannot look up. Each label is replaced by the case it stood for, with the PR #535 body table as the source: `dynamodb:LeadingKeys` with one value and with two, `aws:RequestedRegion` and `aws:PrincipalAccount` in the region and account fields, a region tag value `us-east-1:123456789012` and `team=blue:x`, and `table/${aws:PrincipalTag/team}*` with no tag beside `table/b?${aws:PrincipalTag/team}`. Comments only.
The header of resource_variables.rs said every test was a measured row,
while the `${?}` and `${$}` escapes, NotResource in a 2008-10-17 or
no-Version document, the condition default form and the boundary,
session policy and explicit_deny_matches_arc tests were never measured.
The Deny NotResource test with no tag said "not measured" although the
simulator measured that case, and the `${*}` heading said "measured"
for both its checks although only `table/x` was measured.
The header now names the tests that repeat a simulator case, the checks
inside them that go beyond the measured target or context, and the
tests that follow from the measured rules. The fields file header names
its two derived checks. Comments only.
The policy write paths checked that a `Version` member exists but not its value, and the value now decides whether Resource and NotResource variables expand. A document whose `Version` was `"2012-10-18"`, `"2012-10-17 "`, `20121017`, `null`, `["2012-10-17"]` or `"2012-10-17T"` was stored, and every variable in its Resource patterns then matched as literal text, with no error at write. `from_json_for_write` now refuses a `Version` that is not the string `2012-10-17` or `2008-10-17`, the two values in IAM's policy grammar, with `invalid Version (must be the string 2012-10-17 or 2008-10-17):` followed by the value. The check sits beside `expands_policy_variables` so document.rs does not grow. It runs at write only, so a stored document with another value keeps parsing. A missing `Version` is still refused by each write path before parsing. The user, group and role policy, permissions boundary and console writes reach the check; an AssumeRole session policy is still checked only for the member. The message IAM returns for such a document was not measured. The test writes each of the six values and expects the refusal, parses each as a stored document, and writes both accepted values. The security model manual and the backlog entry on `Version` are updated.
The requirements, the two auth component designs and the backlog said that policy variables are deferred and that Resource and Condition values are literal strings, which the expansion in policy/variables.rs now contradicts. REQ-ABAC-006 states the requirement as met for Resource, NotResource and condition values, and the non-goals table drops its policy variables row. The policy variables note in 05-component-auth.md describes the expansion and points to the security model manual, the Mode 3 limitation names only the condition-value gaps, and the v3 note no longer calls the support deferred. The backlog entry keeps the two condition-value gaps filed as #534.
Stored policies change decision in three ways when policy variables
expand in Resource and NotResource, and operators had no note of it: a
pattern with a variable now matches the table it names, a condition
value naming a multivalued key is no longer expanded to its first
value, and the `${key, 'default'}` form now expands in condition
values. The entry names each change, including an Allow that now
grants and a Deny NotResource that now spares the named table, the
write-time refusal of a Version other than 2012-10-17 or 2008-10-17,
who is affected, and how to find affected policies. No catalog
migration is required.
A condition value `${key, 'default'}` whose key the context reports
unresolvable keeps its text as written: resolve returns
Missing::Unresolvable whatever the default, and expand_condition_value
falls back to the raw text. No existing test put such a key in a
condition value.
The new case_three test puts `${aws:PrincipalOrgID, 'blue'}` in a
StringEquals Deny on aws:PrincipalTag/team, for a caller tagged
team=blue. With the key unresolvable the Deny does not apply. With the
key absent the default is used and the Deny applies. With the key
present its value is used, so `blue` denies and `red` does not.
resource_pattern_is_unresolvable starts with `expand &&`, so a
`2008-10-17` document, or one with no Version, matches a pattern naming
a key the context cannot resolve as written and does not fail closed.
Every test of the fail-closed rule used a `2012-10-17` document, so no
test covered the gate.
The new case_three test puts `Deny table/${aws:PrincipalOrgID}` in each
of the two documents beside `Allow dynamodb:*`, with the key
unresolvable: GetItem on table `T` is allowed, and GetItem on the table
named `${aws:PrincipalOrgID}`, which the text matches, is denied.
The v0.1.15 upgrade entry and the policy variables paragraph of
05-component-auth.md said that a variable whose key the request does
not carry makes the pattern match nothing. That holds only with no
default. With a default, the default is used:
`table/${aws:PrincipalTag/team, 'none'}` matches `table/none` when the
request has no `team` tag, as measured with the IAM policy simulator
for the PR #535 measurements table. Both places now say so.
Item 2 of the v0.1.15 entry said a condition value naming a multivalued
key is compared as written, so such a condition no longer matches. That
holds for a positive operator. Under a negated operator the comparison
with the text now succeeds where the comparison with the first value
failed: `Allow` if `StringNotEquals` on aws:PrincipalTag/team with the
value `${dynamodb:LeadingKeys}`, for a caller tagged team=blue whose
request has the leading key `blue`, was denied before and is allowed
now, and the same condition in a Deny now applies.
The item now names both operator kinds with that example, and the
check-before-upgrading sentence names an Allow that now grants and a
Deny that now applies, for a Resource pattern with a variable and for
such a negated condition.
- variable_values: the multivalued condition-value test also covers
Null. With dynamodb:LeadingKeys holding `false`, expanding
`${dynamodb:LeadingKeys}` to its first value would make
`Null aws:PrincipalTag/team` match for a caller with a team tag, and
the Deny would apply; the test expects Allow. The module header
already listed Null for both rules.
- case_three: the header says its expectations are ExtendDB rules with
no service measurement. The Version write test says its message is
ExtendDB's own and that whether IAM refuses those values at write was
not measured; validate_version and the backlog row say the same.
- condition.rs: the expand_policy_variables comment says "unresolved"
for a key the request does not carry, and names the key the context
cannot resolve, which is left as written with or without a default.
- Upgrade manual, v0.1.15: a permissions boundary write refuses another
Version with `Invalid permissions boundary document: ...`; a session
policy is checked only for its Version and Statement members; item 3
names the present-key case of `${key, 'default'}`; the search advice
names get-user-boundary, get-role-boundary and list-roles for
boundaries and trust policies.
Measured with the IAM policy simulator on 2026-10-28: with aws:TagKeys
as [x, y] or [x], table/${aws:TagKeys, 'x'} in Resource and
${aws:TagKeys, 'x'} as a StringEquals condition value both answer
implicitDeny; with aws:TagKeys absent both answer allowed. The code
used the default whenever the key was multivalued, present or not. Now
a present multivalued key makes the pattern match nothing and keeps a
condition value as written, and the default is used only when the
request does not carry the key. Two evaluator tests pin the six cells
on dynamodb:LeadingKeys; the security model and the module comment say
the rule.
LeeroyHannigan
added a commit
that referenced
this pull request
Oct 28, 2026
A condition value now expands only in a 2012-10-17 document and only under the string and ARN operators, Null and Bool read a variable value as false, and a variable naming a key the context cannot resolve makes the value match nothing, as an absent key with no default does. Two tests and several docs from #535 described the earlier comparison. the_default_form_expands_in_a_null_condition is renamed a_null_value_written_with_a_default_reads_as_false. Its two decisions still hold, because Null reads the text as false and needs the key present; the default is not used. The Null assertion of a_multivalued_key_in_a_condition_value_is_not_expanded_under_any_form now expects Deny: the text reads as false and team is present, the shape of measured cell N1a. In case_three.rs the test for an unresolvable key in a condition value is renamed to say the value matches nothing; its decisions are unchanged. The FAIL_CLOSED_GLOBAL_KEYS doc and the case_three.rs module doc say that a value naming such a key matches nothing. The variable_values.rs module doc says that Null expands neither form and that the default form was measured under StringEquals. REQ-ABAC-006 and the Policy Variables note and limit 4 of 05-component-auth.md no longer list the #534 gaps, and the backlog item for them is removed, because the Version rule and the escape expansion close them. The security model sentence on multivalued keys says a written default is used only when the request does not carry the key.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #533.
Summary
A policy variable in a statement's
ResourceorNotResourcewas compared as literal text, sotable/${aws:PrincipalTag/team}matched no table. ADenywritten that way denied nothing and anAllowgranted nothing. This change expands variables inResourceandNotResourcepatterns before matching, following IAM's rules as measured with the IAM policy simulator:2012-10-17document. A2008-10-17document, or one with noVersion, matches the pattern as written.PolicyDocument.versionis nowOption<String>, so an absentVersionis no longer read as2012-10-17; the evaluator is the field's only reader. Since the value now decides expansion, the user, group and role policy, permissions boundary and console writes refuse aVersionthat is not the string2012-10-17or2008-10-17, such as"2012-10-18","2012-10-17 "or the number20121017. They already required the member. A stored document with another value keeps parsing and matches its patterns as written. AnAssumeRolesession policy is still checked only for the member.*and?as wildcards. Text that comes from a variable value, a default or an escape is matched literally, and a tag value ofbl*matches only a table namedbl*.:written in the policy separates ARN fields. A:inside a value is a byte of its field: a region-field value ofus-east-1:123456789012matches no table ARN. Variables in the region and account fields expand.Resourcestatement does not apply and aNotResourcestatement applies to every resource (both measured, forAllowandDeny). An unresolved variable before a wildcard does not widen the pattern totable/*.FAIL_CLOSED_GLOBAL_KEYS) fails closed inResourceandNotResource, as such a key already does in a condition: when no pattern matches, aDenyapplies and anAllowdoes not grant, and a default written beside the key is not used. The list is empty in this build, so no decision changes today.dynamodb:LeadingKeysis never a variable, inResourceor in a condition value, with one value in the request or several. The condition path used the first value before.${key, 'default'}expands todefaultwhen the key is absent, inResourceand in condition values, with or without a space after the comma.${*},${?}and${$}stand for a literal character inResource.Patterns without
${take the samearn_matchpath as before. The expansion lives incrates/auth/src/policy/variables.rs;resource_matchesgains the document and the request context at its five call sites, including the permissions boundary and session policy checks andexplicit_deny_matches_arc. The one hunk incondition.rsmakesexpand_policy_variablesdelegate to the new module; it does not overlap the lines #471 or #522 change.Measurements
aws iam simulate-custom-policy, 2026-10-24, actiondynamodb:GetItem,Allow dynamodb:*on*beside the statement under test unless the row says otherwise. ARNs arearn:aws:dynamodb:us-east-1:123456789012:table/....table/${aws:PrincipalTag/team}team=bluetable/blueteam=bluetable/redteamtable/bluetable/${aws:PrincipalTag/team}*team=bluetable/blue-xteamtable/bluetable/${aws:username}aws:username=alicetable/aliceaws:username=alicetable/bobtable/${aws:PrincipalTag/team},Version2008-10-17team=bluetable/blueVersionteam=bluetable/blueNotResourcetable/${aws:PrincipalTag/team}team=bluetable/blueteam=bluetable/redteamtable/blueNotResourcetable/${aws:PrincipalTag/team}teamtable/blue,table/redtable/${aws:PrincipalTag/team, 'none'}teamtable/nonetable/${aws:PrincipalTag/team,'none'}teamtable/nonetable/${*}table/xtable/${aws:PrincipalTag/team}team=bl*table/blueteam=bl*table/bl*team=blue:xtable/blue:xtable/b?${aws:PrincipalTag/team}team=uetable/bluetable/${dynamodb:LeadingKeys}LeadingKeys=[blue, red]or[blue]table/blue,table/redtable/${dynamodb:LeadingKeys}LeadingKeys=[blue, red]table/blue*ifStringEquals aws:PrincipalTag/team: ${dynamodb:LeadingKeys}team=blue,LeadingKeys=[blue, red]or[blue]table/xarn:aws:dynamodb:${aws:RequestedRegion}:123456789012:table/blueRequestedRegion=us-east-1table/bluearn:aws:dynamodb:us-east-1:${aws:PrincipalAccount}:table/bluePrincipalAccount=123456789012table/bluearn:aws:dynamodb:${aws:PrincipalTag/region}:*:table/Tregion=us-east-1:123456789012table/T,table/A/stream/x:table/TTesting
${?}and${$}escapes, the2008-10-17and no-Versiondocuments forNotResource, the default form in a condition value, and the boundary, session policy andexplicit_deny_matches_arcpaths. Further evaluator tests pin rules that follow from the measured ones: a substituted value holding${is not expanded again, a*or?in a default is literal, and the default form and the multivalued rule hold underForAnyValue,ForAllValues,IfExistsandNull. The headers of the evaluator test files for policy variables say which expectations repeat a simulator case and which are derived, and the header ofcase_three.rssays its expectations are ExtendDB rules with no service measurement. Unit tests cover the parser, the token matcher and theVersionwrite check, whose message is ExtendDB's own.resource_variables.rs, 5 inresource_variables_fields.rs, 2 invariable_values.rs, 2 incase_three.rs), the version check removed (7), the default form removed (11),${*}read as a wildcard (2), the multivalued check removed (4), a value's:read as a field separator (1), a key the context cannot resolve read as absent (3), theVersionwrite check removed (1), the default used for such a key in a condition value (1), theVersiongate removed from the fail-closed rule forResource(1), and a multivalued key expanded to its first value underNullonly (1).case_three.rscover a key the service populates but ExtendDB cannot resolve; their expectations are an ExtendDB rule with no service measurement. A condition value${aws:PrincipalOrgID, 'blue'}naming such a key is compared as written and its default is not used, while the same value takes the default when the key is absent and the key's value when it is present. A2008-10-17or no-Versiondocument whoseResourcenames such a key matches the pattern as written and does not fail closed.resource_variables_fields.rspin the multivalued-key default rule above ondynamodb:LeadingKeys(present with two values, present with one, absent; inResourceand in a condition value). Control: the previous rule (the default used whenever the key is multivalued) fails exactly these two tests.cargo fmt --all -- --check,cargo clippy -p extenddb-auth --all-targets -- -D warnings, andcargo test -p extenddb-auth: 284 unit tests and 3 doc tests passed. Rust 1.88.0:cargo check -p extenddb-auth --all-targetsandcargo test -p extenddb-auth, 284 and 3 passed. On a SQLite server built from0da0c6d7(the commits after it change tests, comments and documentation, and the multivalued-key default rule, which the evaluator tests cover),test_abac.py,test_authz_resource_arn.py,test_auth_integration.py,test_fgac_leading_keys.pyandtest_resource_tag_migration.pypassed (225 tests), as did the console test for a user policy the service refuses. Against the same server, each refusedVersionsent through the user, group and role policy, boundary and console writes was refused, and a GetItem by a user taggedteam=bluewas denied onblueand allowed onredunderDeny Resource table/${aws:PrincipalTag/team}, and the reverse underDeny NotResource. CI runs the workspace tests.Behaviour change for existing policies
ResourceorNotResourcepattern with a variable in a2012-10-17document now matches the expanded value. A stored policy that relied on the literal text matching nothing now applies.dynamodb:LeadingKeysis no longer expanded to the key's first value; the text is compared as written. Under a positive operator such asStringEqualsthe condition no longer matches, and under a negated operator such asStringNotEqualsit now matches, so anAllowwith such a negated condition can now grant and aDenywith one can now apply.${key, 'default'}now expands to the request's value forkey, or todefaultwhen the key is absent, where the whole text was compared before.dynamodb:LeadingKeysoraws:TagKeyswith a default (${aws:TagKeys, 'x'}) uses the default only when the request does not carry the key. When the request carries it, with one value or several, aResourcepattern matches nothing and a condition value is compared as written. Measured with the IAM policy simulator on 2026-10-28:aws:TagKeysas[x, y],[x]and absent againsttable/${aws:TagKeys, 'x'}inResourceanswer implicitDeny, implicitDeny, allowed, and against${aws:TagKeys, 'x'}as aStringEqualsvalue onaws:PrincipalTag/team=xthe same three answers.Versionis not the string2012-10-17or2008-10-17is refused withInvalid policy document: policy parse error: invalid Version (must be the string 2012-10-17 or 2008-10-17): <value>, and a permissions boundary write withInvalid permissions boundary document: policy parse error: invalid Version .... Before, the document was stored and itsResourcevariables matched as literal text. IAM refuses such a document: Access Analyzer reportsINVALID_VERSION("The version 2012-10-18 is not valid. Use one of the following versions: 2008-10-17, 2012-10-17.") and the policy simulator answersInvalidInputfor it (measured 2026-10-28, two samples each for2012-10-18and2008-10-18); the message text is ExtendDB's own.Not changed here
Two gaps in condition values remain, both present on
mainbefore this change and filed as #534: a condition value in a2008-10-17or no-Versiondocument is still expanded, where IAM treats it as literal text, and${*}in a condition value is left as text, where IAM expands it to*.AssumeRolesession policy is checked only for itsVersionandStatementmembers and is parsed when a request uses the session, so it gets neither theVersionvalue check nor theSidand CIDR write checks.Checklist
cargo fmt,cargo clippycleandocs/manuals/10-security-model.md,docs/manuals/07-upgrade-manual.mdunder v0.1.15,docs/design/01-requirements.md,docs/design/05-component-auth.md,docs/design/05-component-auth-v3.md,docs/backlog.md)Versionother than2012-10-17or2008-10-17