Skip to content

Adopt ENSIP-15 normalization (replace IDNA-based Normalize) #51

Description

@yashgo0018

Summary

ens.Normalize, ens.NormaliseDomain, and everything that flows through them (NameHash, LabelHash, DNSEncode, Tld, DomainPart) currently use golang.org/x/net/idna with MapForLookup. That is not ENSIP-15. It is close enough for many ASCII names, but it diverges from the canonical ENS normalizer used by viem, the ENS app, and the on-chain Universal Resolver on a large fraction of the official validation suite.

Downstream code that calls ens.Normalize() or ens.NameHash() and assumes the result matches the rest of the ENS ecosystem may get wrong namehashes, accept invalid names, or reject valid ones.

Evidence

Compared ens.Normalize on master against viem (normalize from viem/ens, which delegates to @adraffy/ens-normalize) using the official suite from adraffy/ens-normalize.js validate/tests.json (38,613 vectors, suite v1.11.1, Unicode 17.0.0):

metric master (IDNA) with fix
Vectors tested 38,613 38,613
Same output as viem 18,285 18,310
Both reject (agreement) 804 20,303
Divergent 19,524 0
Matches official spec 49.4 % 100 %

Headline divergence categories on master:

category count
viem rejects (disallowed character); go-ens passes 11,069
viem rejects (illegal mixture); go-ens passes 4,602
different output (no error from either) 3,084
viem rejects (illegal placement); go-ens passes 232
viem rejects (underscore allowed only at start); go-ens passes 220
viem rejects (whole-script confusable); go-ens passes 130
go-ens rejects; viem passes 97
other ENSIP-15 rejection rules go-ens misses 90

Examples that matter in practice

Invalid names accepted by go-ens today (rejected by UR / viem):

  • Empty labels: .eth, foo..eth, foo.
  • Label extension: te--st.eth
  • Mid-label underscore: a_b.eth
  • Whole-script confusables: apple.дррӏе.аррӏе.aррӏе
  • Disallowed codepoints: fullwidth dots (), circled letters, etc.

Both accept, but different namehash (~3k cases):

  • Extended Arabic-Indic digits ۰۱۲۳۸۹ → IDNA keeps as-is; ENSIP-15 maps to Arabic-Indic ٠١٢٣٨٩ (not ASCII 012389)
  • Unicode hyphen lookalikes → ENSIP-15 collapses to ASCII -
  • Apostrophe / quote variants → different normalized forms
// master (IDNA): accepts, wrong hash vs viem / on-chain UR
hash, _ := ens.NameHash("a_b.eth")

// master: different hash than viem for the same visual name
hash, _ := ens.NameHash("۰۱۲۳۸۹.eth")  // ≠ viem's NameHash for the ENSIP-15 form

Proposed fix

Replace the IDNA-based normalizer with the canonical Go port maintained by the ENSIP-15 spec author:

  • Dependency: github.com/adraffy/go-ens-normalize (ensip15 package)
  • Same implementation viem uses under the hood (@adraffy/ens-normalize), Unicode 17.0.0, zero transitive runtime deps

API surface

function change
Normalize delegate to ensip15.Shared().Normalize
NormaliseDomain delegate to ensip15.Shared().Normalize (preserve *. wildcard prefix)
NormaliseDomainStrict unchanged — stays on IDNA StrictDomainName=true for DNS-strict checks; document that it is not ENSIP-15
NameHash, LabelHash, DNSEncode pick up correct behaviour automatically via Normalize

Additional wiring

  • DNSEncode: trim a single trailing dot before calling Normalize, so legacy foo.eth. convenience input still encodes even though ENSIP-15 rejects trailing-dot empty labels
  • Tests: update legacy IDNA-era fixtures; add normalize_ensip15_test.go for accept/reject categories and Arabic-Indic digit behaviour
  • CI regression guard (optional but recommended): ensip15_vectors_test.go runs the full 38,613-vector suite when compare-normalization/tests.json is present (skips gracefully if not fetched)

A working implementation exists on branch ensip15-normalize (commit 3b3fe35 on the fork); I can open the PR immediately after #49 merges.

Backwards compatibility

This is a behaviour-breaking fix for spec non-compliance:

  • Names that IDNA incorrectly accepted will start erroring under ENSIP-15 (empty labels, confusables, disallowed chars, etc.)
  • Names that IDNA normalized differently will produce different namehashes (~3k vector cases; Arabic-Indic digits are the most cited example)
  • Plain ASCII names like vitalik.eth are unaffected

References

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions