dyfields

One declarative form definition that doubles as render instructions, validation rules, and API documentation.

dyfields is a zero-dependency Go library (standard library only) for forms whose fields are not fixed: what the user picks decides what comes next, which fields become required, and which ones get locked.

It is general-purpose — not just for config forms. Registration forms, surveys, orders, and approval flows all fit.

go get github.com/BrobridgeOrg/dyfields

Requires Go 1.21+.

What it does

Goal	API
Add and remove fields to compose a form definition	`NewBuilder()` / `FromSchema()` / `AddField` / `RemoveField` / `MoveField`
Export the definition as JSON and load it back	`Schema.Marshal()` / `Parse()` / `Load()`
Check the definition itself for problems	`Compile()` / `ValidateSchema()`
Validate externally supplied values against it	`Compiled.Validate()` / `Compiled.Apply()`

Quick start

The code below is the Example() in example_test.go. It runs with the test suite, so it cannot rot.

b := dyfields.NewBuilder()
b.AddField(dyfields.String("cluster").Label("Cluster").Required())
b.AddField(dyfields.Enum("env").Label("Environment").Required().Default("dev").
    Options(dyfields.Opt("dev", "Development"), dyfields.Opt("prod", "Production")))

brokers := dyfields.ObjectList("brokers").Label("Broker").
    MinItems(1).MaxItems(16).Unique("host").ItemLabel("{host}:{port}")
brokers.AddField(dyfields.String("host").Format("hostname").Required())
brokers.AddField(dyfields.Int("port").Range(1, 65535).Default(9092))
brokers.AddField(dyfields.Bool("use_tls").Default(false))
brokers.AddField(dyfields.Secret("password").VisibleWhen(dyfields.IsTrue("use_tls")))
b.AddField(brokers)

c, err := b.Compile()          // the definition itself is checked here
if err != nil {
    panic(err)
}

doc := dyfields.ValueDocument{
    Values: map[string]any{
        "cluster": "orders",
        "brokers": []any{
            map[string]any{"$id": "b1", "host": "a.internal", "use_tls": true},
        },
    },
    Secrets: map[string]string{"brokers.b1.password": "s3cret"},
}

out, errs := c.Validate(doc)
fmt.Println("errors:", errs.Len())                 // errors: 0
fmt.Println("env:", out.Values["env"])             // env: dev (default filled in)

Now feed it something broken:

bad := dyfields.ValueDocument{
    Values: map[string]any{
        "brokers": []any{map[string]any{"$id": "b1", "host": "not a host"}},
    },
    Secrets: map[string]string{"brokers.b1.password": "s3cret"},
}
fixed, errs := c.Validate(bad)
for _, e := range errs {
    fmt.Printf("%s: %s: %s\n", e.Path, e.Code, e.Reason)
}
// brokers[0].host: format: must be a valid hostname
// cluster: required: Cluster is required
fmt.Println("secrets left:", len(fixed.Secrets))   // 0

Note the last line. use_tls was not supplied and defaults to false, so password is hidden and its secret is not carried forward. A field you cannot see holds no value — that rule runs through the whole library.

Using it from a service

The quick start builds a definition in code. In an application the definition is usually already JSON — in a file, or in a database column — and the job is to compile it once and check every request against it. This is Example_service() in example_test.go, so it too runs with the test suite.

// The definition, as it would come out of a file or a database column.
const schemaJSON = `{
  "schema_version": 1,
  "groups": [{
    "key": "delivery",
    "label": "Delivery",
    "fields": [
      {"name": "channel", "type": "enum", "label": "Channel", "required": true,
       "options": [{"value": "email"}, {"value": "webhook"}]},
      {"name": "address", "type": "string", "format": "email", "label": "Address",
       "required": true,
       "visible_when": {"field": "channel", "equals": "email"}},
      {"name": "endpoint", "type": "string", "format": "uri", "label": "Endpoint",
       "required": true,
       "visible_when": {"field": "channel", "equals": "webhook"}},
      {"name": "signing_token", "type": "secret", "label": "Signing token",
       "visible_when": {"field": "channel", "equals": "webhook"}}
    ]
  }]
}`

// Compile once. A *Compiled is immutable and safe to share across goroutines,
// so this belongs at start-up, not in the handler. A schema that does not
// compile is a bug in your own deployment, not a bad request.
c, err := dyfields.Load([]byte(schemaJSON))
if err != nil {
	log.Fatalf("bad schema: %v", err)
}

// What the service has stored for this tenant today.
stored := dyfields.ValueDocument{
	Values:  map[string]any{"channel": "email", "address": "ops@example.com"},
	Secrets: map[string]string{},
}

// Handing the current state to a browser. Redact first: it swaps every secret
// for the list of paths that have one set, which is exactly what the form
// needs to render "already configured" without ever shipping the value.
public := c.Redact(stored)
body, _ := json.Marshal(public)
fmt.Println("GET  /settings ->", string(body))

// An edit arrives. A patch is not a document: an absent secret means "leave
// it", "" also means "leave it" (an untouched password field submits exactly
// that), and null means "clear it".
patch, err := dyfields.ParsePatch([]byte(`{
  "values": {"channel": "webhook", "endpoint": "not a url"},
  "secrets": {"signing_token": "whsec_123"}
}`))
if err != nil {
	fmt.Println("PATCH /settings -> 400", err)
	return
}

// Ask what the edit would cost before making it. Switching the channel hides
// the email address, and a field you cannot see holds no value -- so the
// address is about to be dropped. This is the warning the UI shows.
if lost := c.ImpactOf(stored, patch); len(lost) > 0 {
	fmt.Println("would clear:", lost)
}

// Apply merges and validates in one step. Use it rather than Validate for
// edits: readonly is only enforceable here, because only Apply can see what
// the value used to be.
next, errs := c.Apply(stored, patch)
if errs.Len() > 0 {
	// Sorted so this example prints the same thing every run; a real handler
	// would hand the slice over as it is.
	sort.Slice(errs, func(i, j int) bool { return errs[i].Path < errs[j].Path })
	out, _ := json.Marshal(map[string]any{"errors": errs})
	fmt.Println("PATCH /settings -> 422", string(out))
}

// Fix the endpoint and try again.
patch.Values["endpoint"] = "https://hooks.example.com/ingest"
next, errs = c.Apply(stored, patch)
if errs.Len() > 0 {
	log.Fatalf("unexpected: %v", errs)
}

// next is what to persist, and it is the checker's own output rather than
// anything the handler assembled: defaults filled in, invisible fields
// removed, transient fields dropped.
stored = next
saved, _ := json.Marshal(stored.Values)
fmt.Println("stored values:", string(saved))
fmt.Println("stored secrets:", stored.Secrets)

It prints:

GET  /settings -> {"values":{"address":"ops@example.com","channel":"email"}}
would clear: [address]
PATCH /settings -> 422 {"errors":[{"path":"endpoint","field":"endpoint","group":"delivery","code":"format","reason":"must be an absolute URI"}]}
stored values: {"channel":"webhook","endpoint":"https://hooks.example.com/ingest"}
stored secrets: map[signing_token:whsec_123]

Four things in there are worth carrying into your own code:

Compile once, at start-up. *Compiled is immutable and safe to share across goroutines. A schema that fails to compile is a bug in your deployment, not a bad request, so it belongs in the path that can still refuse to start.
Redact() before anything leaves the process — a response body, a log line, an audit record. It keeps the values and replaces the secrets with the list of paths that have one set, which is what a form needs to render "already configured".
Use Apply() for edits, not Validate(). readonly can only be enforced against a previous value, so Validate() alone cannot see a locked field being changed. That is a documented boundary, not an oversight. Apply() also stays quiet about the transient fields — they are dropped from the payload by design, so they never reach the server, and a rule that reads one is the client's to check.
Persist what it returns. next is the checker's own output — defaults filled in, invisible fields removed, transient fields dropped — not something the handler assembled. Storing it is what stops the form and the server from disagreeing about what was submitted.

ImpactOf() is the one call with no equivalent elsewhere: it answers "what would this edit throw away" before the edit happens, which is the difference between a confirmation dialog and a support ticket.

Core concepts

Schema / Group / Field

Schema
└── Group (fields rendered together; always / collapsible / toggleable)
    └── Field (one input; an object_list field carries its own sub-fields)

A group has one of three modes:

Mode	Meaning
`always`	Always expanded
`collapsible`	Can be folded; purely visual, folding changes no values
`toggleable`	Carries a boolean switch; turning it off clears the whole group, secrets included

Fields added through Builder without naming a group land in the default group, "default".

ValueDocument: values and secrets are kept apart

type ValueDocument struct {
    Values  map[string]any    `json:"values"`
    Secrets map[string]string `json:"secrets,omitempty"`
}

Secrets is always a flat map[string]string. That makes "there are no secrets in the values tree" a one-glance invariant: Values can be logged, exported, or handed to a client wholesale without walking it to filter anything out.

Secret keys use $id paths: brokers.b1.password, sinks.s_k1.mappings.m_3.salt.

`mask`: hiding values on the way out

A secret is for a value the application never needs to read back. A national ID, a phone number, a customer's email address are not that: they have to be stored, displayed and edited — and they still have no business appearing in an API response, a log line or an audit record in full.

mask is the declaration for those. It changes nothing about what is validated or stored; it only says how Redact writes the field.

dyfields.String("national_id").Label("National ID").Mask(&dyfields.Mask{KeepTail: 4})

{"name": "national_id", "type": "string", "label": "National ID", "mask": {"keep_tail": 4}}

A mask is either a shorthand string or an object. "all" is the default and means the whole value; "omit" removes the key. Everything else is spelled out:

`mask`	`"0912345678"` becomes	For
`"all"`	`**********`	anything whose value is nobody's business downstream
`{"keep_tail": 4}`	`******5678`	matching a log line against the record the user is holding
`{"keep_head": 3, "keep_tail": 3}`	`091****678`	a value whose two ends both carry meaning
`{"keep_tail": 4, "char": "•"}`	`••••••5678`	when stars read badly next to the rest of the output
`{"keep_tail": 4, "width": 6}`	`******5678`	a fixed width, so the real length stays hidden too
`"omit"`	(the key is absent)	a field a caller has no business knowing exists

Key	Means
`keep_head`	how many characters survive at the front
`keep_tail`	how many survive at the end
`char`	what the hidden part is written with; exactly one character, default `*`
`width`	a fixed number of mask characters, whatever the input length

Four details are deliberate:

A value too short for the rule it declares is hidden outright. {"keep_tail": 4} on "6789" yields ****, not 6789. Keeping every character would publish the value under a mask, and a mask that quietly passes its input through is the one failure it must not have.
Characters, not bytes. keep_tail: 4 on "王小明的祕密" keeps four characters.
width is how you hide the length as well. Without it the mask is one character per hidden character, which is readable but publishes the length — and the length of a national ID or a password is itself worth hiding.
omit says nothing rather than saying "not for you". The others leave a visible admission that a value is there; omit removes the key. That is what an API hands a caller who has no business knowing the field exists at all.

An unknown key or an unknown shorthand is a parse error, and a mask that could not fire — on a secret, on a container — is a compile error (bad_mask). A masking rule that quietly does nothing reads, in the schema, as a promise that was never kept.

The value cannot come back in

Masking is only half of an API response. The other half is what happens when the client edits one field of the document it was handed and sends the whole thing back — including "national_id": "******6789". Storing that would destroy the real value, silently, with no error to notice.

There is no marker to catch it with, and that is on purpose: a mask is an ordinary string once Redact has written it, and anything that could recognise one could also be forged by a client sending that exact string. So the rule is not "refuse the mask", it is do not send what you did not change:

The client sends a patch. Diff builds it from the document the form was given and the one it holds now, so an untouched field is not in the request at all.
Apply keeps every key the request does not mention. That is the same rule partial updates already live by, and it is the whole of the protection.

Which makes Diff load-bearing rather than a convenience. A front end that posts whole documents back has no defence here, in this library or any other.

The update round trip

Put together, the two halves make the update path a loop that never has to think about masking again:

// GET /customers/{id} -- what the client is allowed to see.
json.NewEncoder(w).Encode(c.Redact(stored))

// PUT /customers/{id} -- only what the user changed.
patch, err := dyfields.ParsePatch(body)
next, errs := c.Apply(stored, patch)   // absent key == unchanged

Apply is what makes "the hidden field has no value in the request, so keep the old one" automatic: a patch says only what changed, and everything it does not mention — a masked field, an omit field the client never even saw, a secret the form showed as a placeholder — keeps what is stored. That is the same rule that already made partial updates work; masking just gave it a second job.

On the client side, Diff builds that patch:

patch := c.Diff(loaded, edited)   // loaded is what the GET returned

It compares the document the form was handed against the one it holds now and emits only the difference, so a field the user never touched is never in the request in the first place — and a masked value, being equal to what was loaded, drops out by construction rather than by a special case. It is exported from the JavaScript package too, which is where a form actually runs; see langs/javascript.

mask only applies to scalar fields, and declaring it on a secret is a compile error (bad_mask) — a secret is never in the value tree to begin with, so a mask there reads as a promise the schema cannot keep. To mask inside an object_list, declare it on the fields within; it fires for every entry.

Conditions

Four slots, each with its own meaning:

Slot	When it does not hold
`visible_when`	The field is hidden and its value is removed
`required_when`	The field becomes required
`readonly_when`	The field is locked (only `Apply` can enforce this — see below)
`valid_when`	Reports an error, with a code and reason you choose

The first three decide the shape of the document, and so may not read a transient field — a schema that tries is refused at compile time with transient_ref. A transient value never reaches the server, so the form and the server would answer the condition differently, and the server's answer is the one that gets persisted: a visible_when reading a transient field would strip its field's value out of the stored document on every unrelated update. valid_when is the exception, because it only reports a problem; Apply simply does not ask it. The rule is about what a condition reads, not where it is written — a transient field keeps its own required_when and its own valid_when.

Conditions are a closed set, deliberately not an expression language:

leaves:          equals / in / not_in / is_true / is_empty
field-to-field:  equals_field / not_equals_field / gt_field / gte_field / lt_field / lte_field
combinators:     all_of / any_of / not

Go constructors mirror them: Equals, In, NotIn, IsTrue, IsFalse, IsEmpty, IsNotEmpty, EqualsField, GtField…, AllOf, AnyOf, Not.

On the JSON side an unknown key is a hard error: a typo in a condition must never quietly become a condition that is always true.

Field types

Type	JSON shape	Notes
`string`	string	Supports `pattern` / `min_length` / `max_length` / `format`
`integer`	number	Fractions rejected
`number`	number	float64
`decimal`	string	Travels as a string so money does not lose cents to float; `MinDec` / `MaxDec`
`boolean`	bool
`secret`	—	Value lives in `Secrets`, never in `Values`
`enum`	any	Needs `options`; `Multiple()` for multi-select
`datetime`	string	RFC3339 by default; `format` may be `date` or `time`
`file`	string	Holds a file reference (path or ID); the library never touches contents
`list`	array	Array of scalars; constrain elements with `Items()`
`map`	object	`key_pattern` constrains keys, `Items()` constrains values
`object_list`	array of object	Array of objects, with its own sub-fields and scope
`json`	any	The escape hatch; checking stops at its boundary, unknown keys are not flagged

Built-in formats: email, uri, hostname, ipv4, ipv6, uuid, duration, bytesize, cron, regex, date, time, datetime. An unknown format is not an error — it degrades to a pure UI hint.

Constraints a type cannot honour are rejected at Compile(): max_length on an integer or min on a string will not compile, because "a typo that looks like it works" is the hardest kind of bug to find.

object_list: `$id` and scopes

Each entry in an array of objects needs a stable identity, so secrets stay aligned and errors point at the right row.

$id: charset [A-Za-z0-9_-]{1,64}. It is required whenever the entry contains a secret field anywhere below it, and the client generates it.
Secret keys use the $id path: companions.c_8f2a.id_number
Error paths use the index: companions[0].id_number
FieldError.ItemPath also carries the $id path, so the two line up.

Sub-fields may reference outwards (nesting is capped at MaxDepth = 5):

Prefix	Meaning
(none)	Same scope
`^.`	One scope out; stackable as `^^.`
`$.`	Root scope

References only go outwards. This is not merely a simplification: it makes cross-scope cycles structurally impossible, so the topological check only has to run within a single scope, and only on visible_when.

mappings := dyfields.ObjectList("mappings")
mappings.AddField(dyfields.String("column").
    VisibleWhen(dyfields.Equals("^.mode", "manual")))   // reads mode one scope out

API overview

Definitions

Compile(s Schema) (*Compiled, error)   // check the definition, build indexes
Parse(data []byte) (Schema, error)     // strict JSON (unknown keys are errors)
ParseReader(r io.Reader) (Schema, error)
Load(data []byte) (*Compiled, error)   // Parse + Compile
ValidateSchema(data []byte) SchemaErrors
Schema.Marshal() / MarshalIndent() ([]byte, error)

*Compiled is read-only and safe for concurrent use — compile once at startup and share it.

Values

(*Compiled) Validate(doc ValueDocument) (ValueDocument, FieldErrors)
(*Compiled) Visible(doc ValueDocument) VisibleSet
(*Compiled) Apply(current ValueDocument, patch Patch) (ValueDocument, FieldErrors)
(*Compiled) Diff(base, next ValueDocument) Patch
(*Compiled) ImpactOf(current ValueDocument, patch Patch) []string
(*Compiled) Redact(doc ValueDocument) PublicDocument

Validate returns a settled document: defaults filled in, values of hidden fields removed, transient fields dropped.
Visible computes visibility only, for live rendering on the client. It validates nothing and reports nothing.
Apply merges: an absent key means unchanged, so an update keeps the values the request does not mention. It is the only place readonly is enforced, and the one place that says nothing about a transient field — a server cannot check what it was never sent.
Diff is the inverse: it produces the patch that turns one document into another, so a client sends only what it changed. Apply(base, Diff(base, next)) reproduces next, except that a secret base holds and next does not is kept rather than cleared.
Redact replaces secrets with the list of paths that have one set and applies any mask the schema declares, so the result is safe to hand back to a client, write to a log, or keep as an audit record. It never changes the document you persist.

Order of validation

1. settle       recompute group/field visibility and fill defaults until it converges
2. (after that) remove the values and secrets of hidden fields
3. checkScope   per field: type, format, range, required, uniqueness, valid_when; recurse
4. checkStray   look for secrets that match no field at all

Steps 1 and 2 are separate on purpose. Deleting as you go would let a toggle carrying default: true wipe its own group in the first pass whenever the document omits that key.

Every error is collected, never short-circuited on the first one, so a client can mark every offending field at once.

Apply: patch semantics

Apply handles "the user changed a few fields". It takes its own Patch type:

type Patch struct {
    Values  map[string]any     `json:"values"`
    Secrets map[string]*string `json:"secrets,omitempty"`
}

Case	Meaning
The patch does not mention a field	Keep the current value
Value is `null`	Set to null (ordinary fields)
An object_list entry carries `"$deleted": true`	Delete that entry
An object_list entry is absent from the patch	Keep it — absence is not deletion
Secret is `""`	Unchanged (the client never has to send the real password back)
Secret is `null`	Clear it
Secret is any other string	Set it

Secrets use map[string]*string rather than map[string]string because unchanged / set / clear cannot be expressed with a sentinel string — whichever sentinel you pick will one day be somebody's actual password.

readonly is only enforced in Apply, because deciding "was this changed?" requires a baseline. That is a deliberate, documented assumption: a server that wants to stop tampering with read-only fields must go through Apply.

ImpactOf tells you, before anything is applied, which paths this patch would clear — turning off a toggle takes its whole group with it, for instance. Useful for an "are you sure?" prompt.

Errors

Two error types, each carrying a structured code:

type SchemaError struct { Path, Group, Code, Reason string }                        // the definition is wrong
type FieldError  struct { Path, Field, Group, ItemPath, Key, Code, Reason string }  // a value is wrong

SchemaErrors and FieldErrors both offer Len() and ErrorOrNil(); FieldErrors adds Has(code).

Schema codes: duplicate_name, duplicate_group, unknown_ref, cycle, missing_items, missing_fields, missing_toggle, depth_exceeded, bad_type, bad_pattern, bad_range, bad_options, bad_default, bad_unique, bad_condition, bad_item_label, not_comparable, container_required, bad_mask, transient_ref.

Field codes: required, type, format, pattern, range, length, enum, min_items, max_items, duplicate, unknown_field, readonly, missing_item_id, bad_item_id, key_pattern, invalid.

A duplicate message deliberately omits the value — unique may well cover a secret field.

Deliberately not done

An expression language. Conditions are a closed set. Logic beyond it belongs in your code, not in the schema.
Inward cross-scope references. Outward only, which buys the structural guarantee that no cycle can exist.
Handling file contents. A file field stores an identifier, nothing more.
Enforcing readonly in Validate. Without a baseline there is nothing to compare against; use Apply.
Unbounded nesting. MaxDepth = 5. Anything deeper wants a json field or a second form.

Other languages

Language	Location	Notes
Go	this repository	reference implementation, including the `Builder` DSL
JavaScript	`langs/javascript`	browser-side checker, zero dependencies, ES modules

The JavaScript port runs against the same testdata/ fixtures as the Go tests, so the two implementations cannot drift apart without a test going red. Its USAGE.md walks one form through the browser-side API in the order a UI meets it.

examples/ holds a React and a Vue front-end built on the JavaScript port: one schema, no per-field code, with the submitted payload shown as you type. Both are TypeScript, typed from the shipped declarations.

Testing

go test -race ./...
cd langs/javascript && npm test

License

See LICENSE.

Name		Name	Last commit message	Last commit date
docs		docs
examples		examples
langs/javascript		langs/javascript
testdata		testdata
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
apply.go		apply.go
builder.go		builder.go
compile.go		compile.go
condition.go		condition.go
diff.go		diff.go
diff_test.go		diff_test.go
doc.go		doc.go
dyfields_test.go		dyfields_test.go
example_test.go		example_test.go
go.mod		go.mod
mask_test.go		mask_test.go
parse.go		parse.go
pipeline_test.go		pipeline_test.go
registration_test.go		registration_test.go
semantics_test.go		semantics_test.go
types.go		types.go
validate.go		validate.go
value.go		value.go

Document	Contents
`docs/design.md`	Full design document: the reasoning and trade-offs behind each decision
`docs/example-walkthrough.md`	Scenario check: schema / payload / UI three-way correspondence
`docs/example-pipeline.md`	Scenario check: nested object arrays and editing paths
`langs/javascript/USAGE.md`	Walking one form through the browser API, from first render to second save

Navigation Menu

Uh oh!

Repository files navigation

dyfields

What it does

Quick start

Using it from a service

Core concepts

Schema / Group / Field

ValueDocument: values and secrets are kept apart

`mask`: hiding values on the way out

The value cannot come back in

The update round trip

Conditions

Field types

object_list: `$id` and scopes

API overview

Definitions

Values

Order of validation

Apply: patch semantics

Errors

Deliberately not done

Other languages

Further reading

Testing

License

About

Releases

Packages

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

dyfields

What it does

Quick start

Using it from a service

Core concepts

Schema / Group / Field

ValueDocument: values and secrets are kept apart

mask: hiding values on the way out

The value cannot come back in

The update round trip

Conditions

Field types

object_list: $id and scopes

API overview

Definitions

Values

Order of validation

Apply: patch semantics

Errors

Deliberately not done

Other languages

Further reading

Testing

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

`mask`: hiding values on the way out

object_list: `$id` and scopes