Skip to content

DEV.md

Latest commit

 

History

History
176 lines (117 loc) · 41.4 KB

DEV.md

File metadata and controls

176 lines (117 loc) · 41.4 KB
Raw

Development

Install from source

Requires Python ≥3.10 and Rust ≥1.98. In your Python environment:

git clone https://github.com/AnswerDotAI/basedpl.git
cd basedpl
pip install .

For a standalone executable without Python, run cargo install --path .. Cargo installs it in its bin directory, normally ~/.cargo/bin; put that directory on your PATH.

Commands

cargo test
cargo run -- -e '2×3+4'
cargo fastfmt
maturin develop
pytest -q
ship-rs-build

Rebuild with maturin develop after Rust changes before checking the installed extension. Cargo tests alone do not update the editable Python installation. Use cargo fastfmt, not cargo fmt.

Use lowercase j in complex literals throughout tests and examples, including adapted reference cases. Reserve uppercase J for explicit input-alias tests. Keep archived upstream source unchanged.

Write literal matrices in array notation, [10 20 30 ⋄ 40 50 60], not as a reshape, 2 3⍴10 20 30 40 50 60. Keep ⍴ where the example is about reshape.

The development profile uses optimization level 1 without LTO. Tests inherit these settings. Debug information, assertions, overflow checks and incremental compilation remain enabled.

CI uses the development profile for Rust and Python tests. Distribution wheels use the dist profile: optimization level 2, no LTO, 16 codegen units, no incremental compilation and stripped symbols.

Documentation

Documentation lives in nbs/. index.ipynb generates both the homepage and README.md. The glyph reference and the remaining guides are Quarto Markdown. Pages that document system functions are APL notebooks with ]help cells, as are tutorials where saved output helps. sidebar.yml lists pages explicitly; glyph pages are reached through glyphs.qmd and search.

magics.ipynb exports basedpl.notebooks; dyalog.ipynb exports basedpl.dyalog; j.ipynb exports basedpl.j. Edit those notebooks, then run nbdev-export. The other Python modules remain hand-written. Cargo owns the package version.

xml.ipynb, plot.ipynb and the index contain executed SVG examples. Update them with bapl-nb --save; do not run a full nbdev-docs build for routine edits.

bapl-nb nbs/                          # execute APL notebooks without changing files
bapl-nb nbs/getting-started.ipynb --save
bapl-nb nbs/index.ipynb --save        # update homepage examples
nbdev-readme                        # regenerate README.md
nbdev-test nbs/ --save                # execute Python notebooks
nbdev-test nbs/dyalog.ipynb --flags dyalog
nbdev-proc-nbs
cd _proc
quarto preview

Install development and documentation tools with pip install -e '.[dev]'. Rendering uses saved outputs. bapl-nb --save updates output only; execution counts are unchanged. The Rust documentation tests run APL examples in .qmd files. Set BASEDPL_PAGE to part of a page path to run only matching pages. pytest runs their Python examples. Use nbdev-test for Python notebook examples. Edit the homepage in nbs/index.ipynb, then regenerate the README.

Structure

  • lib/*.apl: Dyalog dfns adapted from April and the Dyalog dfns workspace. Reference cases load these shared definitions with •load; case-specific setup stays in each test. See lib/README.md for usage and provenance.
  • array.rs: ordinary Value atoms and immutable shared arrays; checked construction, recursive prototypes, axis offsets, direct/mapped result frames, cell descriptors, padded cell assembly and row-based display. Results print as source that reads back. Simple vectors and vectors of strings print as literal runs. Other vectors print in brackets, and a one-item vector ends with ;. A rank-0 array prints as ⊂x. Higher ranks use array notation, and one major cell ends with ⋄. An empty array other than ⍬ and "" prints as its shape reshaping its prototype, 0 3⍴0, and an empty record prints as ⍬:⍬. Inside brackets, an item with a space or a : between its units gets parentheses. An array with named axes prints as its keyed shape reshaping the array without names, ["k":2]⍴3 12. An array of rank 2 or more with keys prints as a key list for each axis applied to the array without keys, ["a" 1;"x" "y"]:[1 2 ⋄ 3 4], where a position stands for a missing key. Strings print in double quotes and characters in single quotes. Arrays retain function handles and cache lexical dependencies without owning lexical frames or Python objects.
  • number.rs: Integer/Exact/Float/Complex values, normalization, checked arithmetic, promotion, comparison and structural conversion. i64 and BigRational share one exact domain. Checked integer overflow falls back to BigRational; integral rational results return to i64 when they fit. Representation is private. Complex values use num_complex::Complex64, normalizing exactly zero imaginary parts to Float.
  • agreement.rs: positional broadcasting and key union through one output layout and two index maps. Scalar functions, Each, rank frames, native mathematical functions and explicit scalar axes use this path. Missing entries use prototype fill. Compact numeric kernels read the maps directly. Equal-shape and repeated-block mappings avoid coordinate calculations and expanded input copies. Contracted-axis maps require equal key sets and retain the first axis's order.
  • keyed.rs: axis-label construction, lookup and named updates. Each immutable Keys holds an optional name for each position, and a name-to-position hash. Present names are unique. align matches named positions by name and unnamed positions in order. Agreement, match, catenate and reordering all use it. A Keys with no names at all is dropped, so the axis is unkeyed. Fills and positions from an unkeyed part have no name. Records, JSON objects, CSV headers and system-function options need a name for every entry. Layout in array.rs owns dimensions and keys together. Its axis selection, concatenation and replacement operations describe structural results. Frames and cells carry layouts into assembly, which retains cell-axis labels shared by all result cells. Dot access is a binder rewrite to Pick in eval.rs. Plain assignment extends each missing name along a path, outermost first. An axis with no keys gains them: the new position has a key, and the existing positions have none. New vector entries start as empty records. New matrix cells take the prototype. The design is in meta/axiskey.md.
  • primitive.rs: primitive identities/glyphs, valences and array-level implementations. Replicate, products and assignment retain their own agreement rules. Allocation caps are separate from numeric-to-integer conversion.
  • number_theory.rs: segmented prime enumeration, Miller–Rabin primality and Brent/Pollard–rho factorisation; exact integer results and scalar-cell assembly.
  • system.rs: case-insensitive •name table for constant arrays and native functions. System functions use ordinary function nodes and application. primitive.rs contains single-character primitives.
  • regex.rs: •r compiles a Rust regex into a keyed vector of native functions. Functions share an Arc<Regex> through system::Call; dot access, composition and Python use ordinary function values. Positions count Unicode characters.
  • distribution.rs: statrs-backed probability distributions, plus closed-form logistic. Constructors return keyed native functions sharing one distribution. Sampling checks shape/allocation limits and cancellation; numeric methods preserve layouts through pervasion. Discrete draws and quantiles are exact integers.
  • Generators: •rand seed returns a record of roll and deal sharing one Xoshiro256PlusPlus behind a mutex. The generator algorithm is fixed. Range sampling and the distribution samplers come from rand and statrs, and new releases of either can change the draws. distributions.apl pins one sequence to detect that. sample accepts that record on its left and finds the stream through its roll function. Roll and deal take the generator as a parameter. Plain ? passes the thread-local generator.
  • csv.rs: •csv import and •tocsv export, keyed options, per-column numeric inference and lossless compact storage. The csv crate handles records and quoting.
  • data.rs: shared keyed-option decoding, numeric fill, •vfi numeric-field parsing and UTF-8/binary file I/O (•nget/•nput). Binary vectors use exact integers in 0…255. Writes validate before opening and use exclusive creation unless overwrite is explicit. System functions take options on the left. Options::new reads a keyed vector or a plain-text shorthand. The monadic form uses the defaults.
  • json.rs: •json parses and •tojson serializes, with keyed objects, exact integers and explicit null fill. The tagged worker protocol remains in protocol.rs.
  • polynomial.rs: coefficient/factored/exponent-table forms, Horner evaluation, companion-matrix roots through faer, and analytic polynomial gradients/VJPs.
  • selection.rs: temporary labels for selective assignment. The binder marks their data flow and permits only selection functions; masks still read real user bindings. Option<SelectionKind> distinguishes ordinary evaluation, whole-item selection and element selection. Nested labels retain their storage and paths. Prototype labels preserve empty-cell structure.
  • display.rs: boxed-array diagrams, function trees and session display settings. Returned values and ⍕ remain independent of these settings. bundle converts a MIME-keyed vector to the bundle sent to frontends. with_renderer adds a native _mime_ renderer to a record. svg wraps SVG text as a bundle. •svg and •plot share both.
  • syntax.rs: byte-spanned lexer and structural parser, character, string and numeric literals, definition kind/full span, and structural guards and default arguments. The parser collects nodes and separators first. The enclosing delimiter then gives the separators their meaning. At the top level and in braces, a line break or ⋄ ends a statement. Parentheses only group: a line break inside them is a space, and ⋄ and ; are errors. In brackets, spaces separate items, ; separates items that contain spaces, and ⋄ separates major cells. ; and ⋄ can't share one pair of brackets. Brackets round one item without ; only group it. Brackets are a record when any item is key:value: a : with only values before it. Literals separated only by spaces merge into one vector literal before units form. A unit is a run of nodes with no spaces between them. Assignment, pipes and guards also end units. The run before ← stays flat, so assignment can take its target from the end of it: the arrays applied to each other there, as in v[2], and any function of a modified assignment. A parenthesised group counts as an array only when its expression gives one. A selection writes into the first array of the applied run at its end, as mt in (3↑mt[i])←…. parse returns complete syntax, incomplete input, or a structural error without evaluating expressions.
  • eval.rs: persistent session, explicit right-to-left category-reduction stack, and shared primitive/operator/dfn/train calls. Structural resolution consumes one item. A unit resolves to one value by the same binder. One category table selects binding actions, priority and waiting states. An array next to an array applies: the binder turns v i into a call to ⌷ with ⊂i as its left argument. Application binds more loosely than a left argument and more tightly than a call, and it groups from the right. A bound left argument with no right argument makes a section. Binder::sections puts a placeholder argument at the right end, and the functions to its left compose onto it with Atop and ⊸, so 1+2×- is {1+2×-⍵}. With two arguments, the left one goes to the last function, so X (1-×) Y is 1-X×Y, and y 2× x is an error because 2⊸× is bound. Functions side by side form a train only when no left argument is present. Grammatical reduction returns application requests to the evaluator; it never calls functions itself. APL and Python share operand normalization and validation in Function::new. Reduced entities rebind against their right context when their category changes. Functions have immutable shared nodes; unfinished trains and bound left arguments exist only in the binder. No per-glyph arithmetic precedence. Output and the final result are separate from errors.
  • error.rs: retained source text, byte spans, inspectable error kinds, and readable Unicode-width diagnostics with separate call-site context. Tabs use four-column stops; other control characters are escaped.
  • cli.rs / main.rs: native command and REPL. The REPL evaluates the parsed result once, not during completeness checking.
  • inspection.rs: non-executing name/glyph inspection and cursor lookup. build.rs embeds the glyph pages in nbs/glyphs/ for glyph and syntax help. System functions take their help from BUILTINS in system.rs, which names a glyph page for a few of them. ]help output carries both plain text and Markdown. Session methods own lexical name listing, classification, source and erasure.
  • kernel.rs: native kernmini adapter. ThreadWorker runs the interpreter on its own thread, so async transport and interrupts remain responsive. Implicit display becomes Jupyter results, explicit output becomes stdout, and completion reuses the REPL glyph matcher plus user/system names. Inspection and whole-cell name?/name?? use the shared metadata path. The wheel installs its kernelspec from wheel/data/share/jupyter/kernels/basedpl/. History uses kernmini's default; subshells are not advertised.
  • j.rs: the J engine and J kernel, built with the python feature. Engine loads libj at run time, registers output and input callbacks through JSM, and runs profile.ijs. J sets its recursion limit from the stack of the thread that starts an engine. An engine therefore runs J only on that thread. The kernel starts its engine on a worker thread with a 64 MB stack. JSetM returns J's error flag, which an earlier failure leaves set. set clears the flag first with an empty sentence. The J kernel shares kernel::serve and kernel::next_count with the APL kernel. The wheel ships no J kernelspec. install_j_kernel writes one through kernmini's install_kernelspec.
  • editor.rs: Rustyline terminal adapter using the shared naming table. Only typed backtick names auto-expand; Tab is an explicit completion request. Bracketed paste/history/navigation cancel automatic expansion. Rustyline owns terminal modes, editing, in-memory history and the final newline on Ctrl-D. No history file or input rewriting in the interpreter/frontends.
  • symbols.rs: shared glyph, canonical name, monadic name, dyadic name and extra completion aliases. Used by editor.rs and exposed as basedpl.symbols for Python exports and notebook JavaScript completion. Also owns shortcut formatting from keyboard.json; Python symbol rows append this display suffix. Add or change names here, not in individual consumers.
  • python/basedpl/keyboard.json: shared Alt-chord map, embedded by symbols.rs and packaged for editor adapters. Keys are US characters after Shift, before Alt. Chords insert literals even in strings/comments.
  • protocol.rs: JSON-lines encoding over ordinary Rust values and sessions. No protocol types enter evaluation or arrays.
  • execution.rs: evaluation deadlines, a thread-safe interrupt flag and an optional poll hook that can request cancellation. Context lends primitive code the current execution control and source span. Cancellation unwinds through ordinary errors but bypasses APL guards.
  • xml.rs: XML element trees, serialization and •svg. Elements are keyed vectors with tag, attrs and children entries. •xml checks names and escapes & < > ". •svg adds a _mime_ field holding its renderer. •mime calls a keyed vector's _mime_ function through ordinary dispatch, with implicit echo disabled and cancellation still active. Implicit display falls back to text when a renderer fails. Interrupts and timeouts still propagate. Keys of the form _name_ are hooks that the language calls.
  • plot.rs: •plot returns a spec record holding data, the left settings and a native _mime_ renderer. The renderer validates the spec through data::Options, reads every default itself and draws SVG with plotters (SVG backend only, no font files). The data's structure supplies x positions, category labels and named series. All axes use one linear Coord with ticks from Axis::ticks. Log scales transform values before drawing. A vector or matrix of spec records draws a figure. Repeated specs span cells, and share unions axis ranges within column or row groups. Legends are opt-in. 'end' writes names beside each series' last point. Corners use plotters' boxed legend. The renderer draws data labels and end labels itself. place moves each one vertically clear of earlier labels.
  • worker.rs: sequential evaluations with a separate stdin reader for control messages. python/basedpl/worker.py owns process lifetime, deadlines and hard-kill fallback. No APL execution occurs on the reader thread.
  • reference.rs: independent fixture decoding and structural comparison, shared by Rust tests, the worker's case request and Python's private _check_reference. Every reference case gets a fresh session.
  • python.rs: optional PyO3 boundary. _Array and _Function hold native values; _Session holds the evaluator and runs each request on the calling thread. Requests and replies carry shared native values and retained diagnostics, never Python objects. python/basedpl/__init__.py provides arrays, conversions, the apl workspace and errors. functions.py constructs functions and exports word names from symbols.rs. The optional ipython.py adapter supplies Function help/source and completion inside apl strings. _cli.py forwards arguments to the Rust CLI.
  • tests/core.rs: storage, ownership, parser diagnostics, API behaviour and cross-call recovery checks. Use equiv_in! { &mut session; code => expected_apl, ... } for session workflows and fails_in(&mut session, kind, &[code, ...]) for errors in the same session. Expected expressions run in fresh sessions. Keep Rust constructors for foundational and representation checks. Self-contained language cases belong in tests/reference/core.apl. tests/cli.rs exercises the actual native process. Documentation examples share one session per APL code block. Unannotated code supplies setup; ⍝ introduces an independently evaluated expectation.
  • python/basedpl/reference.py: Source importers, Dyalog expectation capture, scan/review/activation, and Corpus inventory access. bqn(src) runs BQN through links/BQN/bqn.js with Node and returns what it prints, for checking BQN comparisons. Corpus searches and patches tests/reference/inventory/*.jsonl from a kernel. Default views omit large expectations; request fields explicitly. scripts/reference.py is a thin scan/review/activation CLI. python/basedpl/apltests.py reads/writes .apl records and appends reviewed inventory cases.
  • tests/reference.rs and tests/reference/*.apl: bAsedPL semantic cases plus acceptance tests from ngn, April, APLcart and Dyalog documentation. .apl files determine active coverage. Use ⍝⍝ section headings and short case descriptions for non-obvious checks. Optional ⍝ ⎕: lines assert explicit output. core.apl retains exact Rust array equality, including numeric domains. Implementation gaps belong only in pending JSONL records, never passing error tests. See tests/reference/README.md for format and workflow.

Value is a number, character, function or shared ArrayData. Array storage is Integer(Vec<i64>), Float(Vec<f64>) or Mixed(Vec<Value>). Atoms, unit arrays and singleton vectors remain distinct. Checked constructors preserve elements and select compact storage without numeric promotion. as_integers() and as_floats() expose borrowed slices. at() and elements() yield owned values without materializing the array. Shape, prototype, numeric-domain summary and lexical dependencies are cached in immutable Arc storage. Shape and tally return exact integers directly from dimensions. Nonempty arrays derive their prototype from the first item; empty construction requires a prototype. Prototype filling memoizes shared nested arrays and retains function handles. Function equality is identity; ordering is unsupported. Values and shared function nodes are Send + Sync.

Float arithmetic/comparison dispatches outside slice loops. Primitive numeric folds bypass scalar-array allocation and interpreter calls. Homogeneous float sum/product reductions use Rust 1.98 algebraic operations, including axis reductions; grouping and bitwise reproducibility are not promised. All scans use successive left accumulation. Seeded and unseeded forms share lane traversal. Reduction starts from its whole seed on the right; scan starts from its whole seed on the left. Each axis lane uses the same seed. Rank supplies separate seeds per cell. Generic reductions remain right-associated at every rank. Generic functions keep their call order and side effects. Exact arithmetic is unchanged. Finite-result checks remain at construction boundaries; division retains 0÷0=1. No fast-math flags, custom SIMD intrinsics, CPU-specific wheel flags, compensated summation or strict/fast modes are used.

Inverse dispatch carries an optional fixed argument: the pair's Boolean is true for a fixed left argument and false for a fixed right argument. Commute switches sides. Before, After and binding propagate that information. Inverse scans apply the operand's dyadic inverse to adjacent accumulators, using the seed for the first pair. They share forward scan's axis/seed validation. General forks, one-argument Before and arbitrary dfns require rules beyond this propagation and remain unsupported.

Vector-axis reduction is (f/⍤,)⍠axes Y through the general axis rule, so it assembles like any function along axes. The selected axes move last in the order given, so axis order decides the ravel. A seed binds to the reduction before composition with Ravel, so the general rule doesn't split it along the axes. Single-axis reductions retain their direct traversal and numeric kernels. An axis node on a fold passes its axis to the fold.

Layout also carries optional axis names. Agreement pairs equal names first, then remaining axes positionally, leaving differently named axes separate. Its existing index maps handle the resulting permutations and broadcasts. Layout construction retains names until result assembly drops all occurrences of collisions. Public name attachment rejects duplicates. Axis selections resolve strings against argument layouts before numeric-axis dispatch. ⍴ returns the shape keyed by the axis names, with no key for an unnamed axis. Reshape takes the result names from the keys of its left argument, and keeps position keys only when the shape is unchanged. ArrayData shares its element buffer through a separate Arc<Storage>; layout changes copy metadata only.

Python operator properties build native nodes when operands are complete. _Pending retains unfinished constructions and fills missing operands innermost-first; f.power.each(n) builds (f⍣n)¨. No source generation or Python evaluation callback enters the interpreter.

The numeric policy is documented in README.md. Primitives raise LIMIT for any generated array of more than MAX_GENERATED_ELEMENTS (one million) elements. serde_json is a frontend dependency; PyO3 remains optional. JSON requests decode directly to String, one per line, with no object envelope or custom escaping. Responses remain structured objects. Keep JSON and Python conversions separate: they implement different external contracts.

Lexical frames are a stack with non-owning parent indices, separate from dynamic call/handler state. Nested functions see live lexical bindings, not snapshots. Plain dfn name assignment is local. Modified and selective array assignment update the nearest existing lexical binding. Each update retains its resolved owner and original array across modifier calls. The write does not look up the name again. Arrays may contain functions, including active local captures. Returns and outer updates reject functions or arrays whose lexical dependencies would not survive. This includes empty prototypes. Tail calls retain lexical dependencies in argument arrays as well as the called function. Dfns can return functions directly. Public export walks the shared array/function graph once and rejects active frame references. Operator derivation retains operand values without running the body or creating an invocation frame. Frames pop on success and error; no collector is needed. Execute uses the same lexical capture rule for newly created definitions. Python calls retained functions through its one workspace, apl. Rust callers supply the session when calling a retained function. Recheck the lifetime argument before adding namespaces, nonlocal function assignment or escaping lexical closures. Dyalog reference runs and the scope boundary are recorded in meta/PRD.md gate L.

Bracket lists are structural syntax. Each item contributes one element, including array and function values. Items evaluate left to right, as statements do. [a b]← destructures by the assignment rules below. First ↑, Pick and complete atomic ⌷ indices return stored values through the shared call-result path, including functions. Array indices supply result frames; partial coordinates retain trailing cell axes. Empty coordinates preserve the argument. Array consumers and cell assembly store function results as elements. Python can construct Array([plus, times]) and retrieve callables with first, pick, .py or .np. •tojson and the worker protocol omit keyed-vector entries that hold functions. They reject any other function.

Agenda selector◶cases stores its operands in an ordinary composed-function node. Construction validates the nonempty function vector and constant selectors. Function selectors run once per call. Scalar indices use the shared position conversion, counting from 0. The selected branch receives the original arguments through shared function dispatch.

Catch-all and numbered guards checkpoint the installing frame's local binding map after the condition executes. Restoring it removes later introduced locals and restores previous local values, including modified assignments. Outer/global writes and output are not rolled back. This deliberately omits Dyalog's distinction between rebinding a local and modifying its existing binding. Assignments within the condition survive rollback. Guard handlers are popped before execution and unwind dynamically through ordinary calls. Cancellation and unsupported-feature errors are not caught. Ordinary and error guards may have an empty result; selecting one returns no value.

Empty Each calls its operand once, replacing only empty arguments with their prototypes. Function::call_prototype scopes prototype mode around the ordinary call path. Compositions, dfns, dops and helpers inherit the mode. Pick uses structural prototype selection throughout that call. Other errors and explicit output remain observable. The caller's mode is restored on success and error.

eval_with and call_with install per-evaluation EvalOptions. echo defaults to true; false suppresses implicit display at top level and inside execute, without suppressing explicit output or display commands. An optional output sink receives explicit/display events as they occur instead of collecting Evaluation.output; ordinary callers retain the capture interface. Clone the InterruptHandle to another thread or supply a timeout. Checks run at binding/call boundaries and inside long interpreted/primitive loops. Tight bounded float kernels keep their existing slice paths; native-library calls and individual BigInt operations are not preempted. Python's apl uses cooperative cancellation only. The separate process Worker.eval allows a grace period before killing an unresponsive process. Killing loses the session; requests are never replayed.

Session::call resolves a function expression and invokes it with one or two existing arrays through the ordinary APL call path. call_function_with accepts a retained node. Neither binds temporary argument names. Evaluation.function exposes an unshy exportable function result; set_function checks and binds a function. PyO3 _Session.request accepts code or a native function, native arguments/bindings, timeout and echo. The process worker separately decodes bindings and args in protocol.rs; exact JSON integers must not pass through f64. The process protocol does not export native function handles.

Python has one workspace, apl: a _Workspace created at import around the Rust evaluator. fn is apl.fn. Module-level builtin calls and the notebook magics use apl. IPython completion accepts apl or a method bound to it. Plain calls and 'explicit' calls request echo=False, and 'repl' calls request echo=True. Plain calls print explicit output and return an Array or an unshy Function; capturing calls return Result without printing. The capture mode is positional-only because keywords bind APL names. Errors follow the same print/capture rule. Keywords bind APL names, including native functions; apl.timeout sets a per-evaluation deadline. ]clear removes every name and restores the display settings the frontend started with. ]box reset restores those display settings. Array retains the native value losslessly. .py and .np make Pythonic copies; NumPy is optional and lazy. Python integers transfer through PyO3's BigInt support without decimal-string conversion. Array indexing uses the shared Rust selector behind ⌷, counting from 0. : takes a whole axis, and negative positions count from the end. Arithmetic and function construction use native nodes, never generated APL source.

Evaluation.output contains ordered Output { kind, data } events. data is a MIME bundle with text/plain; OutputSink receives the same events. Python Result.events exposes them and .output extracts text. _repr_mimebundle_() calls •mime through apl. The notebook runner saves display events as display_data and explicit output as streams. JSON value transfer carries array contents/axes. The worker omits keyed-vector entries that hold functions, such as _mime_ renderers. It rejects any other function. Rich output bundles cross that boundary separately.

.fn() parses once and retains a late-bound expression. Function nodes cache whether they contain late-bound operands. At a call boundary, resolution rebuilds affected nodes with the calling session's current bindings; unaffected nodes remain shared. A memo preserves shared function graphs. Reduction identities, inverse recognition and primitive fast paths then see ordinary functions. Cyclic name resolution errors at the depth limit. The binder still resolves ordinary APL names at execution time. Python word names select direct-call valence and currying; operators use the underlying APL function.

The module and apl share builtin attribute lookup in functions.py. The registry comes from Rust glyph metadata and system-function names; module functions are constructed lazily through __getattr__. Glyphs, canonical names and aliases are ambivalent; operation names select valence and curry. Operation names win overlaps, and glyph/operation names win unbulleted system-name collisions. Identifier spellings normalize hyphens, Python keywords and Unicode NFKC. __all__ and __dir__ expose the registry without creating every function. .fn() functions are ambivalent. Function.__call__ passes keyword arguments as a keyed left argument, and its positional arguments then form a vector as the right argument (⍬ when there are none). Calls without keyword arguments keep f(⍵) and f(⍺, ⍵).

printing.py renders native function parts as Python names and combinators. The PyO3 adapter exposes each node's immediate operands as shared handles. The printer propagates valence and groups Python expressions by precedence. Dfns, defined operators and late-bound source remain fn(...); printing performs no evaluation or name resolution.

Dfn return selection follows statement syntax, not display shyness. A final assignment returns its value silently; a non-assignment call returns immediately even when its result is silent. Empty bodies and exhausted guards return no value. Default ⍺ assignment skips its RHS when supplied and does not itself return a value. Each invocation shadows its caller's ⍺.

The binder returns either a value or a tail application at eligible return positions. Defined calls loop over tail applications and discard frames above the callee's highest lexical dependency. This dependency is cached with immutable function nodes, including function operands and train arms. Tests run 10,000 tail calls with one frame, or two when an outer lexical binding is retained. Installed handlers disable tail reuse. Non-tail evaluation and retained lexical frames have a 1,024-level limit. Tail call diagnostics retain the final tail site, not an unbounded history.

Flat binding/operator derivation and assignment chains use explicit vectors. A common evaluation-depth budget covers groups and all function representations; function construction separately limits graph depth, protecting recursive application and destruction. Neither limit is an execution-time sandbox. stacker extends the stack onto the heap at each call and binding level, so evaluation needs no large thread stack.

An assignment arrow drains its right-hand binding stack, assigns to the structural target suffix and resumes binding the prefix. Its RHS is never evaluated twice. Target recognition uses runtime categories at top level and dfn-local name rules inside definitions. Statement return selection distinguishes an assignment from an expression containing one. Modified destructuring calls its modifier left to right. Ordinary destructuring assigns right to left.

Numeric comparisons use fixed relative tolerance 1e-14 when approximate, exact rational comparison otherwise. Each tolerance-sensitive operation must ship with independent inside/outside-tolerance cases: comparisons, membership/index-of, match, unique/grouping, and floor/ceiling. Reuse the rounded pair 0.3 and 0.1+0.2 across applicable operations, with an outside-tolerance control, zero/negative boundaries, and exact/mixed counterparts. Keep structural Rust assertions exact; do not use the language's comparison as the test oracle. The concrete acceptance matrix is in meta/PRD.md §9.4.1. The reference corpus deliberately retains unsupported cases; enable them as their requirements are met.

Numeric semantics are Rust-owned. Explicit Python conversions copy exact integers through PyO3 and non-real values through PyComplex::from_doubles. Fraction components transfer as native integers. NumPy Boolean, integer and real arrays cross the boundary as one int64 or float64 buffer copy. Python casts them first. _Array.numeric fills compact storage from the buffer. _Array.buffer copies compact storage out for np.frombuffer. uint64 values above the int64 range raise ValueError. Complex, string and object arrays convert per element. No Python numeric objects enter the core. JSON remains a separate process boundary: exact integers use arbitrary-sized JSON integers with serde_json's arbitrary_precision feature. Fractions retain tagged decimal-string components, complex values a tagged numeric pair. Reference-interpreter cases compare equal numeric values across exact/float domains because Dyalog has no corresponding explicit exact domain. Rust core tests assert the exact representation, prototypes and compact buffers separately.

Complex arithmetic extends the existing scalar/array/operator dispatch, not a second evaluator. The lexer shares one real-component scanner between ordinary and ajb literals. Equality uses magnitude-based tolerance separately from real ordering; counts still require exact integrality and zero imaginary part. Approximate prototypes/identities normalize to Float. Complex division scales its denominator and direction scales its input to avoid avoidable squared-magnitude overflow/underflow. Powers, logs, circle functions and factorial/binomial extend this numeric layer. Complex components/results remain finite. Real Float values admit ±infinity but never NaN. Comparisons handle infinity before tolerance or exact-to-float conversion. The compact float path explicitly checks zero divisors. JSON uses signed infinity tags; Python uses floating infinities. Dyalog 20.0.53963.0 executions supply structured documentation-example expectations, except labelled bAsedPL policy differences.

The private _core._Session holds the evaluator in a mutex. Each request waits for the lock without holding the GIL, then evaluates on the calling thread with the GIL released. A second mutex holds the active request's interrupt handle, so interrupt() works from another thread. Each request owns a fresh cancellation flag. The request's poll hook takes the GIL at most every 10 ms to check Python signals. Ctrl-C sets the cancellation flag and raises KeyboardInterrupt with the captured output. Evaluation touches no Python objects. Arrays can be dropped on any thread. No unsafe Send implementation is used.

Brackets round one item without ; only group it, so [x] is x, and [x;] is a one-item vector. Only ⊂ encloses. A list of counts as Power's operand gives the result for each count. A count of ∞ runs until the state matches the previous one, through the same ≡ call as ⍣≡. Power keeps every state of the until form when its operand is a one-item list holding a function, as in f⍣[≡;]. Python's .history(p) builds that operand for a predicate, and the counts (×p)×⍳1+|p for a count.

⍠ builds an axis node from its operand, a list of numbers or names. Python's f[A] builds the same node. A primitive with its own axis meaning handles the node in call_axes, and a fold takes the axis directly. Every other function goes through cell_axes. It moves the selected axes of each argument last, applies rank, then moves the result axes back. selection, behind ⌷ and Python indexing, reads an atomic ∞ part as a whole axis and ¯∞ as a whole axis reversed. It counts negative positions from the end. When the first part is the only one and its elements are arrays, it does choose indexing. Pick, choose, reach and Agenda share the same position rule.

Session::members rewrites dot access before binding. After a value, x.name becomes "name"⊃x, and x.(I) or x.[I] becomes (I)⌷x or [I]⌷x. The rewritten node is a group. Assignment, adding keys and chaining reuse the ⊃ and ⌷ paths. Between functions, . stays inner product.

Workspace and builds

Prefer broad dependency ranges with a required lower bound, such as >=0.24.4, <1, rather than exact pins or Cargo's minor-constrained 0.x caret ranges. Resolve API changes when they arise; do not add compatibility layers preemptively.

The canonical version lives in Cargo.toml; Python uses dynamic = ["version"]. The crate produces an rlib, native executable, and optional basedpl._core extension. python enables PyO3; extension-module also enables PyO3's extension linking mode. Default Cargo builds have no Python dependency.

Rust 1.98 or later is required by the algebraic float methods. CI tests with stable Rust. Keep fastws-generated Cargo patches and .git/fastws-cargo-key under fastws control. Preserve the pyproject source/cache keys; do not commit the workspace-generated Cargo.lock or manually replace workspace configuration. meta/ is ignored planning material, never committed.

uv builds and maturin develop --release use the incremental release profile: LTO off, 16 codegen units, this package incremental. Distributed wheels use dist: full LTO, one codegen unit, incremental off, stripped output.

CI tests the native core/CLI/JSON process before installing the extension and running Python tests. Python tests cover boundary behavior and the real installed command, not a duplicate Rust semantic suite. A dist-profile CPython 3.13 wheel has also passed these checks in a clean temporary environment outside the workspace, with the source checkout and Rust absent from its import/command paths. The existing wheel/sdist and tagged publication flow remains unchanged.

tests/test_repl.py uses a real pseudo-terminal for symbol entry, ambiguity, bracketed paste, Ctrl-C recovery and Ctrl-D's final newline; piped process tests cannot exercise these paths. Rust editor tests cover matching and quoted/comment context. The editor's Enter callback records just the accepted replacement because Rustyline cannot combine replacement and submission in one command; the adapter applies it before history/evaluation and shows the glyph in its submission message. This does not reprocess an entire source string.

Do not repeat isolated wheel installs for routine feature changes. maturin develop plus the normal tests is the development default; reserve clean-install checks for packaging-sensitive changes or release preparation, using a small relevant subset of existing tests.

Release

Development tests and artifact checks precede release approval. Once Jeremy approves a release, confirm the clean tree and Cargo version, then use ship-release with no flags. For this maturin project fastship tags/pushes the current version, leaves publication to CI, then bumps Cargo and refreshes the editable installation. There is no changelog step. First publication requires Jeremy's PyPI trusted-publisher setup. Never commit, push, tag, or publish without approval.