diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 5c9016028b0..dcfa970f755 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -712,6 +712,7 @@ peps/pep-0833.rst @dstufft peps/pep-0835.rst @ilevkivskyi peps/pep-0836.rst @savannahostrowski @Fidget-Spinner @brandtbucher peps/pep-0840.rst @jeremyhylton @gvanrossum +peps/pep-0841.rst @corona10 @sobolevn # ... peps/pep-2026.rst @hugovk # ... diff --git a/peps/pep-0841.rst b/peps/pep-0841.rst new file mode 100644 index 00000000000..91c3a093dcb --- /dev/null +++ b/peps/pep-0841.rst @@ -0,0 +1,314 @@ +PEP: 841 +Title: Adding Frozen Syntax to Make Immutable Types Optimizable +Author: Donghee Na , + Nikita Sobolev +Discussions-To: https://discuss.python.org/t/pep-841-adding-frozen-syntax-to-make-immutable-types-optimizable/108219 +Status: Draft +Type: Standards Track +Created: 16-Jul-2026 +Python-Version: 3.16 +Post-History: `17-Jul-2026 `__ + + +Abstract +======== + +This PEP proposes *frozen display* syntax: ``f{1, 2, 3}`` evaluates to a +:class:`frozenset`, and ``f{'a': 1}`` evaluates to a ``frozendict``. +Because immutability is guaranteed by the syntax itself rather than +inferred from usage, the compiler can treat frozen displays as first-class +citizens of its optimization pipeline: constant displays are folded into a +single ``LOAD_CONST`` with an exact result type at compile time and +cached in ``.pyc`` files. + + +Motivation +========== + +Python has display syntax for its mutable containers but none for its +immutable ones. Today an immutable set must be written as +``frozenset({1, 2, 3})`` and an immutable mapping as +``frozendict({'a': 1})``. Each of these: + +* builds a mutable set or dict, then copies it into the immutable type. +* looks up the name ``frozenset`` or ``frozendict`` at runtime on every + execution. +* cannot be easily optimized by the compiler, because either name may be + rebound and the call may have arbitrary side effects, or + value can be of unexpected type for the optimization, + value can be non uniquely referenced. + +CPython already hints at the opportunity: the peephole optimizer rewrites +a constant set display into a frozenset, but only as the right operand +of ``in``. Assign the same display to a variable and the optimization is +gone. The root cause is that the compiler can never prove immutability +of a ``set`` or ``dict`` display, so it must rebuild it on every +execution. A display whose *semantics* guarantee immutability removes +that barrier once and for all. + +One of the goals of this PEP is to increase the use of immutable containers +in CPython, preparing for a concurrent future in which free-threading and +subinterpreters become significantly more common. +Efficient creation of immutable data structures and convenient syntax would +encourage the use of ``frozendict`` and ``frozenset``. This would reduce bugs +caused by accidental mutation of shared mutable containers while also +improving performance. For example, subinterpreters already share +``frozenset`` objects, and there are plans to share ``frozendict`` objects as +well. + +Immutable container displays have been requested and discussed by the +community several times, most recently in the `frozenset and frozendict +comprehensions +`__ +thread on Discourse. + + +Rationale +========= + +Syntax, not a builtin call +-------------------------- + +Only syntax gives the compiler a semantic guarantee. A call to +``frozenset(...)`` can be shadowed, but a frozen display cannot. Every +optimization described below follows from this single property. + +Static analysis benefits in the same way. Today, tools +that don't perform semantic analysis must assume +that ``frozenset(...)`` refers to the builtin. A frozen display turns +that assumption into a syntactic guarantee, so analyzers can treat the +result as immutable with full confidence. This holds even for purely +syntactic tools that perform no name resolution. + +Linters and formatters can automatically rewrite ``frozenset({1, 2, 3})`` +and ``frozendict({1: 2})`` to be ``f{1, 2, 3}`` and ``f{1: 2}`` +on newer Python versions. + +Why ``f{...}`` +-------------- + +The ``f`` prefix reads as *frozen*, mirroring the familiar f-string +prefix convention. Sharing the letter with f-strings is not a problem: +strings are immutable too, so either way an ``f`` prefixed expression +evaluates to an immutable value. ``f{`` is a syntax error in all +current Python versions, so the syntax is fully backward compatible. The tokenizer +emits a single ``FLBRACE`` token for ``f{``, so ``f {1}`` (with a space) +remains an error and there is no ambiguity with the name ``f`` or with +f-strings. + + +Specification +============= + +Grammar +------- + +Our goal is to make new syntax and grammar identical +to existing ``set`` and ``dict`` syntax and grammar rules. + +New alternatives are added to ``atom``, mirroring ``set`` and ``dict`` +displays and comprehensions:: + + fset: FLBRACE star_named_expressions '}' + fsetcomp: FLBRACE star_named_expression for_if_clauses '}' + fdict: FLBRACE [double_starred_kvpairs] '}' + fdictcomp: FLBRACE kvpair for_if_clauses '}' + +* ``f{1, 2, 3}`` is a frozenset display. +* ``f{'a': 1, 'b': 2}`` is a frozendict display. +* ``f{}`` is an empty frozendict, mirroring ``{}``. +* Star unpacking follows the existing displays: ``f{*xs}`` is a + frozenset display (like ``{*xs}``) and ``f{**d}`` is a frozendict + display (like ``{**d}``). +* Comprehensions are supported: ``f{x for x in xs}`` is a frozenset + comprehension and ``f{k: v for k, v in items}`` is a frozendict + comprehension. +* Async comprehensions are also supported + in async contexts: ``f{x async for arange(5)}`` + is a frozenset async comprehension + and ``f{k: v assync for k, v in items}`` + is a frozendict async comprehension. +* :pep:`798` and unpacking in comprehensions is also supported. + ``f{*nums for nums in list_of_nums}`` is a frozenset + compehension with unpacking and ``f{**items for nums in list_of_items}`` + is a frozendict comprehension with unpacking. + +AST +--- + +Four new expression nodes are added: ``FrozenSet(elts)``, +``FrozenDict(keys, values)``, ``FrozenSetComp(elt, generators)``, and +``FrozenDictComp(key, value, generators)``, structurally identical to +their mutable counterparts. Distinct nodes (rather than a flag) let +every downstream +consumer (e.g. the symbol table, the AST optimizer, the code generator, +and third-party tools) dispatch on immutability directly. + +Semantics +--------- + +A frozenset display evaluates to exactly what ``frozenset({...})`` +returns. A frozendict display evaluates to exactly what +``frozendict({...})`` returns. Both result types are immutable and +hashable, which is what makes the compile-time treatment below sound. + +Bytecode +-------- + +Two new instructions are added: + +* ``BUILD_FROZENSET (count)`` works like ``BUILD_SET``, but the freshly + created, uniquely referenced set is frozen in place with no copy. +* ``BUILD_FROZENMAP (count)`` works like ``BUILD_MAP``, but creates a + ``frozendict``. + +``count`` is an ordinary oparg with the same format and meaning as in +the existing ``BUILD_SET`` and ``BUILD_MAP`` instructions. + +Displays that use star-unpacking or exceed the stack-use guideline fall +back to building the mutable container and freezing it in place. The +result is indistinguishable. Comprehensions take the same path, so +they need no new opcodes. + + +The optimization pipeline +========================= + +The central claim of this PEP is that frozen displays are not merely +convenient syntax: they give every stage of the compiler a guarantee it +can act on. The reference implementation already exercises the full +pipeline: + +1. **AST preprocessing.** ``FrozenSet`` and ``FrozenDict`` participate + in AST-level constant folding of their elements. + +2. **Code generation.** The common case compiles to a single + ``BUILD_FROZENSET`` / ``BUILD_FROZENDICT`` instruction with no name + lookup, no temporary copy, and an exact, statically known result type + (used by the compiler's type inference, e.g. to reject + ``f{1, 2}[0]`` at compile time). + +3. **Control flow graph (CFG) constant folding.** A display whose + keys and values are all constants is folded into a single + ``LOAD_CONST``, serialized into the ``.pyc`` by marshal, and shared + across all executions: zero per-execution construction cost. Unlike + the existing list/set folds, this is *unconditionally* valid, since + immutability comes from the language semantics, not from how the + value is used. A display with a non-constant element, e.g. + ``f{'key': ['list']}``, is still built at runtime. + +4. **Constant deduplication.** Frozen constants participate in + ``co_consts`` deduplication. For a frozendict the deduplication key + keeps the insertion order, so a display never loses its iteration + order to an equal display with a different key order. + +.. note:: + + This PEP deliberately makes no claims about JIT-level optimization: + the JIT project is currently on hold following the `Steering + Council's announcement + `__. + +The pipeline also opens future work that mutable displays can never +support: sharing folded frozen constants across code objects and +immortalizing them under free threading. + + +Backwards Compatibility +======================= + +``f{`` is a syntax error today, so no existing code changes meaning. +The changes visible to tooling are: a new ``FLBRACE`` token, four new +AST node types, two new opcodes, and a bytecode magic number bump. + + +How to Teach This +================= + +"Prefix a set or dict display with ``f`` to make it frozen". +Style guidance: prefer ``f{...}`` over +``frozenset({...})`` for literal values on Python 3.16+. +Constant frozen displays are free after the first execution. + + +Impact on the Standard Library +============================== + +A quick survey of the standard library (excluding tests) finds about +105 ``frozenset(...)`` and 65 ``frozendict(...)`` call sites, of which +about 46 and 22 respectively pass a literal display and could be +written as ``f{...}``. They spread across widely used modules such as +``typing``, ``dataclasses``, ``functools``, ``copy``, and +``traceback``. + +These numbers are only an estimate of the potential effect. This PEP +does not propose a mechanical rewrite of the standard library. + + +Reference Implementation +======================== + +A complete implementation, including the parser, AST, code generator, +and CFG constant folding, is available in the `fset_fdict branch +`__ of the author's +CPython fork. + + +Rejected Ideas +============== + +Alternative spellings +--------------------- + +Many spellings were considered. ``f`` was chosen simply because it is +the prefix that best evokes *frozen*: + +* Single letter prefixes: ``i{'key': 1}`` (immutable), ``z{'key': 1}``. + This was rejected because types are called ``frozen``, + ``f`` as a prefix reads the best. +* Multi letter prefixes: ``fr{'key': 1}``, ``fz{'key': 1}``, + ``frz{'key': 1}``. This was rejected because it just adds an extra letter + to write and read with no extra real value. +* Symbol prefixes: ``${'key': 1}``, ``+{'key': 1}``. + This was rejected because most symbols already have a meaning as operators. + We can't reuse them for this new purpose. + We also don't want to add new symbols like ``$`` + to keep them for something else in the future. +* Bracket variants: ``{{'key': 1}}``, ``|{'key': 1}|``, ``{|'key': 1|}``. + This was rejected because adding + a single token ``f{`` is easier than adding two tokens. + Writing ``f{`` is also easier than writing two different brackets. + ``{{}}`` is rejected because it is a valid syntax right now. +* Word prefixes: ``frozen{'key': 1}``, ``fdict{'key': 1}``, + ``frozendict {'key': 1}``. + This was rejected because it is rather verbose. + +We also rejected using ``F{`` as it is possible with fstrings. +This was rejected to keep the syntax as minimalistic as possible +and not to create extra ``F{`` token. + +Freezing methods +---------------- + +Methods such as ``{'key': 1}.freeze()`` or +``{'key': 1}.take_frozendict()`` are not real alternatives: they can be +added independently of this PEP. + +While such methods can be a great feature on its own, +making this the only way to create immutable containers is not an option: + +* Multiline expressions with such methods are really hard to read, + because you need to read the very last line to know the type of the object. +* It is quite verbose to write for common cases. +* It does not provide syntax guarantees for static analysis tools. +* It may have a different semantics when used + as ``a = {1: 2}; b(a.take_frozenset())``, + dependending on `how the internals would look like `_, + it might mean that ``a`` would be cleared. + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.