Skip to content

Fixes for broken master, trust-based source selection, module split, feed discovery & caching, sectioned config, and release tooling#26

Merged
line0 merged 186 commits into
mainfrom
add-missing-semver-module
Jul 23, 2026
Merged

Fixes for broken master, trust-based source selection, module split, feed discovery & caching, sectioned config, and release tooling#26
line0 merged 186 commits into
mainfrom
add-missing-semver-module

Conversation

@line0

@line0 line0 commented May 23, 2026

Copy link
Copy Markdown
Contributor

PR description

Summary

This branch started as a set of fixes to unbreak master after the incomplete 0.7.0 refactor, but it has grown to carry the bulk of the 0.7.0 work itself. It is being merged as a single unit. The list below describes the delta against master, grouped by area rather than by commit (history was rebased/reworked as we iterated, so individual commits are not a reliable map of the end state).

At a high level, relative to master this branch:

  • Restores a working updater/loader (the original "broken master" fixes).
  • Adds a real SemanticVersion type, a computed-property primitive (Accessors), and a feed trust/security model (FeedTrust, Host, SSRF guard, fetch policy).
  • Replaces DepCtrl's hard dependencies on BadMutex, DownloadManager and PreciseTimer with internal, OS-API-based implementations, and drops the aegisub.re / aegisub.util dependencies.
  • Ports the newer ConfigHandler/ConfigView, FileOps and Logger from the sqlite branch.
  • Adds a CLI/CI toolchain (test runner, deploy, bundler) and a JSON-Schema-validated feed format.
  • Migrates all annotations to LuaCATS and namespaces the repo module paths.

It also introduces a few one-time, persisted config migrations.

Reviewability note

This is intentionally a large single PR. The genuine "unbreak master" fixes (SemanticVersion presence, script-type handling, the double-init regression, capitalize, late DownloadManager init, the multi-assignment corruption) are the smallest slice; everything else is new 0.7.0 surface. Reviewers short on time should concentrate on the Security and Migrations sections, which change what gets fetched/executed and mutate users' config files.


Changelog — 0.7.0

Added

Version handling

  • SemanticVersion module: parsing, formatting and comparison utilities (absent on master), plus an instantiable version value object — construct from a string, from numeric components, or fromPacked; instances compare with < / <= / ==, stringify, bump immutably, and test range satisfaction. Static helpers accept an instance anywhere they take a version.
  • npm-style version ranges for provides module aliases.

Core primitives

  • Accessors: metatable-backed computed properties. Accessors.property{get, set} in the class body plus Accessors.install(class) after it; setter-less properties are read-only, subclasses inherit parent properties, every property is recorded on class.__accessors, and readable properties appear in pairs(instance) via __pairs.

Trust & networking

  • FeedTrust class: centralized trust/block logic. Trusted / blocked / extra-feed lists with add + remove; block entries are objects carrying a match mode (prefix or exact) and a block reason; trust/block data can be sourced from the official feed.
  • Host class: DNS resolver binding and private/loopback host detection (isPrivate, memoized).
  • FeedInventory: breadth-first feed crawler, with every root and child gated through the fetch policy.
  • Persistent feed cache and an untrusted-feed fetch policy.
  • Trust-ranked package-source selection with provider fallback; interactive trust flows with a persisted per-package source choice.

Self-contained runtime

  • Internal NamedSemaphore / FileLock locking primitive that uses only OS APIs, as a replacement for the BadMutex dependency.
  • Internal DownloadManager alternative built on OS APIs or an installed libcurl, with stall detection and a concurrent-connection limit.
  • Bundled dkjson, provided as json when luajson is not installed; module import aliases.
  • Aegisub shim adds the ?user directory to the Lua/Moon search path.

FileOps / Common

  • FileOps.writeFile().
  • Common helper to detect running headless / in CLI.

Toolbox

  • Feed-management macro/UI.

Tooling

  • CLI/CI toolchain: test runner that works on automation-script tests and external repos, runs across multiple packages, and supports multi-file suites; a deploy command; a bundler script with CLI zip-archiver bindings.
  • JSON Schema for the DepCtrl feed plus a schema validator.

Changed

  • Repository module files namespaced under an l0/ directory to match their require paths. (Confirm downstream require paths before tagging.)
  • Records back their version with a SemanticVersion instance (@semanticVersion); record.version is now a packed-integer accessor over it, so existing packed-int reads/writes/comparisons are unchanged. The version is persisted to config as a major.minor.patch string.
  • Cheap no-arg getters are now read-only computed properties: Downloader:getProgressprogress, Lock:getStatestate, UnitTestSuite:getFailuresfailures, and Host isPrivate. (getVersionNumber / getVersionString remain instance methods.)
  • Updater status codes are now an UpdateStatus enum; scriptType is a ScriptType enum end-to-end.
  • Ported the newer ConfigHandler / ConfigView, FileOps and Logger from the sqlite branch.
  • DepCtrl uses its own internal BadMutex / DownloadManager / PreciseTimer replacements and no longer requires aegisub.re or aegisub.util.
  • No longer shadows packages installed to the Aegisub ?data automation directory.
  • Logger: assert passes all varargs through on success (chainable); new assertNotNil (fails only on nil); dump / dumpToString accept a maxDepth; new describeType; log/format calls guard against non-string templates; incremental progress-bar fill; RNG warmed up after seeding.
  • UnitTestSuite: manual _order no longer required; tests can be skipped; per-test durations measured with a monotonic wall clock instead of CPU time.
  • All annotations converted to LuaCATS / luals format; type annotations added throughout.
  • Consistent private-method convention across the codebase.
  • LF + final-newline enforced repo-wide for stable feed hash calculation.

Security

  • SSRF guard: feed and package downloads to private/loopback hosts are blocked (Host + DNS resolver).
  • Untrusted-feed fetch policy honored for both crawl roots and advertised children; a blocked root is never fetched.
  • Block-list entries validated at the mutation boundary (unknown match mode coerced to prefix) and deduplicated case-insensitively, matching case-insensitive block matching.
  • The official trust/block list is no longer cached when its feed fails to load, so an offline first run can pick it up later instead of being stuck on the fallback.
  • FileOps.remove() no longer recurses into directories by default, removing a silent whole-tree deletion path.

Fixed

Updater / loader

  • __depCtrlInit hooks were re-run on already-initialized modules, breaking modules that mutate exported state on first init (e.g. BadMutex).
  • Legacy boolean true/false script-type arguments were rejected after the ScriptType enum migration.
  • DownloadManager was instantiated too late, failing the update process.
  • Multi-assignment on a failed update could corrupt record fields.
  • Typo broke optional-module presence checks; broken module-loader skip lists.
  • performUpdate gated unmanaged-record handling on a nonexistent updateRecordType field, so unmanaged modules never registered a dummy ref/version and the dummy ref was never cleaned up on failure.
  • Up-to-date-but-not-updated path passed a namespace string instead of the record to loadModule, crashing on a string index.
  • Record.checkVersion ignored its precision argument.
  • Broken version-number display in version errors; update errors omitted whether a record was virtual/unmanaged; toolbox error-path typo.

SemanticVersion

  • Parsers accepted numeric segments over 255 and non-dot separators.
  • toString roundtrip raised on an invalid version string.
  • Malformed changelog version keys (from unvalidated feed data) crashed getChangelog after a successful install; such keys are now skipped.

FileOps

  • joinPath dropped every segment after an empty/separator-only one (joinPath("a", "", "b")"a").
  • move did not create destination directories recursively and emitted wrong error messages.
  • Downloader did not create the target directory tree recursively.
  • Inaccurate path-length-limit detection; mkdir reported success on failure; createTempDir / rmdir errors lacked detail.

Config

  • ConfigHandler cache was scanned by raw path instead of validated path, building duplicate handlers with divergent in-memory config for the same file.
  • ConfigView did not register/unregister itself with its handler on setFile / unsetFile, so whole-file refreshes skipped it and its writes were lost after another view replaced the backing tables.
  • Default-wrapping wiped populated sections on load.
  • Record type persisted the stale boolean unmanaged flag instead of recordType, round-tripping unmanaged records back to managed.
  • writeConfig was not actually called when persisting a new record.
  • Spurious deprecation notices on non-deprecated ConfigView methods.

Feed / trust / deployment

  • Nil dereference crash in feed template variable expansion.
  • Fetched-feed temp files were written outside the trimmed cache dir, so temp downloads accumulated unbounded.
  • Deployment ignored the delete flag on files.

Concurrency / logging

  • NamedSemaphore called sem_unlink on per-instance GC, letting an ephemeral handle remove a name still held by another owner and producing split-brain locks on POSIX; unlink is now deferred to Lua-state teardown. (Windows unaffected; verified on WSL with a regression test.)
  • Logger crashed instead of degrading when the log file could not be opened, and tried to clean logs before the log dir existed.

Common

  • Cyclic tables were wrongly reported equal when a sibling key differed; depth accounting was also corrupted.
  • itemsEqual tracked items as a boolean set, so duplicates matched incorrectly ({1,1} vs {1,1} → false); now count-matched as a multiset.

Platform / misc

  • Broken home-directory expansion on Linux.
  • package.path corruption when the existing path did not end in a semicolon.
  • capitalize() used unsupported string indexing and had never actually worked; the depctrl CLI used the wrong config filename; assorted wrong file names and log tags.

Removed

  • Hard dependencies on aegisub.re, aegisub.util, PreciseTimer, external BadMutex and external DownloadManager (replaced by internal equivalents — see Changed).

Migrations (one-time, persisted on first load)

  • Legacy boolean unmanaged flag → recordType; the obsolete flag is dropped from the config file.
  • Packed-integer record version → major.minor.patch string.

Both run once and persist. Note the implication for users who roll back to 0.6.x after a migrated load.

Dev / docs

  • AGENTS.md contribution conventions (annotation/comment style, file naming, MoonScript gotchas, public-API and test-exposure patterns, computed properties, changelog rules).
  • Expanded test coverage: FileOps, validateFullPath / validateNamespace, FileLock / NamedSemaphore, downloader integration, Toolbox plumbing, wider Updater coverage; tests split into per-file suites; record-registry cleanup after runs; mock-server readiness fix.
  • README update; cspell settings and dictionaries.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to restore a working “master” by reintroducing missing version-handling functionality, fixing several regressions in module loading and update/feed handling, and adding initial unit-test and line-ending consistency infrastructure.

Changes:

  • Added SemanticVersioning and migrated version parsing/formatting to use it across updater/feed paths.
  • Fixed/adjusted updater + module-loader behaviors related to initialization and update flows, and added a minimal DepCtrl unit test.
  • Updated the project update feed metadata, bumped versions, and enforced LF + final newline consistency.

Reviewed changes

Copilot reviewed 12 out of 15 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
modules/DependencyControl/Updater.moon Migrates updater version handling to SemanticVersioning and adjusts update logging/changelog display.
modules/DependencyControl/UpdateFeed.moon Adds legacy scriptType compatibility + new invalid scriptType messaging; moves DownloadManager init earlier.
modules/DependencyControl/UnitTestSuite.moon No functional change (newline normalization).
modules/DependencyControl/Tests.moon Adds a first unit test for Common.terms.capitalize.
modules/DependencyControl/SemanticVersioning.moon Introduces semantic version parsing/formatting/comparison utilities.
modules/DependencyControl/Record.moon Switches record version parsing to SemanticVersioning and adds compatibility helpers.
modules/DependencyControl/ModuleLoader.moon Updates init-hook guard logic and switches version formatting/parsing to SemanticVersioning.
modules/DependencyControl/FileOps.moon No functional change (newline normalization).
modules/DependencyControl/ConfigHandler.moon No functional change (newline normalization).
modules/DependencyControl/Common.moon Fixes capitalize() implementation to avoid unsupported string indexing.
modules/DependencyControl.moon Bumps DepCtrl version, exposes SemanticVersioning, and adds a Moonscript minimum-version guard.
macros/l0.DependencyControl.Toolbox.moon Bumps toolbox version; updates version formatting and script-type handling; registers DepCtrl test suite.
DependencyControl.json Updates feed versions/hashes and adds new files (including tests) to the module feed.
.vscode/settings.json Enforces final newline on save in VS Code.
.gitattributes Enforces LF line endings in git.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread modules/DependencyControl/UpdateFeed.moon Outdated
Comment thread modules/DependencyControl/UpdateFeed.moon Outdated
Comment thread modules/l0/DependencyControl/SemanticVersioning.moon Outdated
Comment thread modules/l0/DependencyControl/SemanticVersion.moon
Comment thread modules/DependencyControl/Updater.moon Outdated
Comment thread modules/l0/DependencyControl/ModuleLoader.moon
Comment thread modules/DependencyControl/ModuleLoader.moon Outdated
Comment thread DependencyControl.json Outdated
@line0
line0 force-pushed the add-missing-semver-module branch 2 times, most recently from 7f7ad69 to 9fee44a Compare May 23, 2026 22:10
@line0
line0 requested a review from Copilot May 23, 2026 22:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 15 changed files in this pull request and generated 5 comments.

Comment thread modules/DependencyControl/Updater.moon Outdated
Comment thread modules/DependencyControl/ModuleLoader.moon Outdated
Comment thread macros/l0.DependencyControl.Toolbox.moon Outdated
Comment thread macros/l0.DependencyControl.Toolbox.moon Outdated
Comment thread modules/DependencyControl/SemanticVersioning.moon Outdated
@line0
line0 force-pushed the add-missing-semver-module branch from 9fee44a to ad4a5ec Compare May 23, 2026 22:46
@line0
line0 requested a review from Copilot May 23, 2026 22:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 15 changed files in this pull request and generated 5 comments.

Comment thread modules/DependencyControl/ModuleLoader.moon Outdated
Comment thread modules/DependencyControl/ModuleLoader.moon Outdated
Comment thread modules/l0/DependencyControl/ModuleLoader.moon Outdated
Comment thread modules/DependencyControl/UpdateFeed.moon Outdated
Comment thread modules/l0/DependencyControl/Updater.moon Outdated
@line0
line0 force-pushed the add-missing-semver-module branch from ad4a5ec to 5bc3f45 Compare May 23, 2026 23:14
line0 added 4 commits May 24, 2026 08:14
nothing inside DepCtrl is supplying this parameter, so the only effect
this bug would have had, is removing the default guard against modules
trying to load themselves
- assert now passes through all varargs on success (allows chaining)
- new assertNotNil method - fails only on nil, unlike assert which also fails on false
- dump/dumpToString methods accept a maxDepth to cap recursive table output
- new static Logger.describeType method - returns Lua type name but renders MoonScript class instances as "ClassName object"
- fix: guard log/format calls against non-string message templates
line0 and others added 5 commits July 22, 2026 02:48
Encode and decode the config through l0.dkjson directly instead of a generic
`json` (one json lib per file), and write it with dkjson's Prettier-flavored
indentMode plus a trailing newline, so a hand-edited config file stays readable.
Decoding via dkjson's nil+error return also makes corrupt-file handling
consistent (a parse failure now always backs the file up), rather than
depending on the json lib's error convention.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Global Configuration" macro that edits DependencyControl's global
settings (updates/feeds/logging/paths) in one sectioned dialog, leaving the
three feed lists to Manage Feeds. A declarative field table drives the form and
sources every effective default from its owning class or the central
config-schema; integers use native intedit widgets, enums dropdowns, booleans
checkboxes, and paths edits. Values left at their default are unset so the
config file stays minimal, and crawlLimits is reassigned as a fresh object.
Adds a GlobalConfig test class and a Toolbox changelog entry.

Also refreshes the feed's file hashes to match the current sources.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@line0 line0 changed the title Multiple fixes for broken master Fixes for broken master, trust-based source selection, module split, feed discovery & caching, sectioned config, and release tooling Jul 22, 2026
line0 and others added 5 commits July 22, 2026 11:27
…records triggering downstream bugs

Triggered by a bad required module spec (`{"ffi", "json"}`) in the ILL.ILL module
The prettier `indentMode` encoder had three issues, all reachable now that
dkjson is exported (`json`/`dkjson`) to the whole ecosystem:

- Non-deterministic key order: the default case-insensitive sort left keys
  that fold to the same lowercase string (e.g. name/Name) ordered by table
  iteration, so identical objects could serialize to different bytes and
  churn config/feed files on every rewrite. Ties are now broken by the raw
  key, giving a total order.

- Reference cycles overflowed the stack instead of failing cleanly. The
  encoder now raises "reference cycle" — the same error upstream dkjson
  raises in every other mode — via a pre-flight ancestor walk.

- Output now ends with a single trailing newline, matching Prettier. The
  callers that appended "\n" by hand (ConfigHandler, UpdateFeed) drop it;
  those that omitted it (dumpExpanded, FileCache) gain it, so every file
  DepCtrl writes ends consistently.

Grow the prettier encoder tests from 4 round-trips to 31 mostly exact-output
cases: layout, key ordering, width-based array breaking, cycles, and the
trailing newline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ent config hive

After waiting on the updater lock, refreshRecord reloads each pending module
via loadConfig. ConfigView\load returns true even for a hive missing from the
file, so an uninstalled module was mistaken for an installed one and its
identity fields (moduleName/namespace) were blanked while scriptType stayed
Module — the follow-up loadModule then indexed package.loaded with a nil key
("table index is nil").

Treat a hive as a real install only when it carries a version, so a still-
virtual module with no hive stays virtual and installs normally. Import stays a
full overwrite (so a field an update dropped, e.g. a removed provides, clears)
except moduleName/namespace, which a corrupt hive must never blank.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…of its DepCtrl record no longer demotes it to unmanaged

hello ILL
@line0
line0 force-pushed the add-missing-semver-module branch 3 times, most recently from ade6961 to ff0fa9a Compare July 22, 2026 22:32
line0 added 2 commits July 23, 2026 01:59
open(2) and sem_open(3) are variadic but were cdef'd with fixed
prototypes and passed bare-number args. Harmless on x86-64, but the
arm64 Apple ABI passes varargs on the stack, so the mode/value
arrived as garbage — file locks failed to exclude and the first
sem_wait blocked forever. Declare them variadic and pass the integer
args as typed cdata.
@line0
line0 force-pushed the add-missing-semver-module branch 2 times, most recently from d020a1e to 2056bfe Compare July 23, 2026 00:32
@TypesettingTools TypesettingTools deleted a comment from github-actions Bot Jul 23, 2026
@TypesettingTools TypesettingTools deleted a comment from github-actions Bot Jul 23, 2026
@TypesettingTools TypesettingTools deleted a comment from github-actions Bot Jul 23, 2026
@TypesettingTools TypesettingTools deleted a comment from github-actions Bot Jul 23, 2026
@line0
line0 force-pushed the add-missing-semver-module branch from 2056bfe to 0e9bb28 Compare July 23, 2026 00:47
line0 added 3 commits July 23, 2026 03:06
- switch to maintained luarocks action which supports windows
- switch windows build toolchain to msvc
- run windows tests in pwsh
- install zlib via vcpkg on windows test runner
- adjust zlib name for lzlib
- caching for slower windows workflow
@line0
line0 force-pushed the add-missing-semver-module branch from 0e9bb28 to be2852d Compare July 23, 2026 01:08
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Test results — macos-latest ✅1032 · ubuntu-latest ✅1032 · windows-latest ✅1028

Summary

Tests 📝 Passed ✅ Failed ❌ Skipped ⏭️ Other ❓ Flaky 🍂 Duration ⏱️
3096 3092 0 4 0 0 1m 19s

🎉 No failed tests in this run.

Github Test Reporter by CTRF 💚

🔄 This comment has been updated

@line0
line0 merged commit 619842f into main Jul 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants