Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 78 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,8 @@ python/ PyPI package `trackparse` (Python 3.9+, zero deps, uv
rust/ planned crates.io port (README stub only)
site/ documentation site + playground (Astro + Starlight, GitHub Pages)
astro.config.mjs base /trackparse, sidebar
src/content/docs/ the pages (Markdown/MDX); reference/spec.md is generated from spec/SPEC.md
src/content/docs/ the pages (Markdown/MDX, JS + Python tabs on every example);
reference/spec.md is generated from spec/SPEC.md
src/components/ HeroDemo, Playground (runs js/ in the browser), Feature, Stats
src/lib/ trackview.ts (highlight + field rendering), icons.ts (Lucide SVG strings)
scripts/sync-spec.mjs copies spec/SPEC.md into the docs at build time
Expand Down Expand Up @@ -112,6 +113,10 @@ js/CHANGELOG.md, spec/CHANGELOG.md, python/CHANGELOG.md generated by release-p
disagree, fix both in the same change.
- Principle: **extract, don't delete.** Anything recognised goes into a field; anything not
recognised stays in the title verbatim. A rule that silently drops input is wrong.
- **The website and the READMEs ship with the change.** A PR that changes behaviour, API, options,
output fields, warnings, commands or published numbers updates the matching pages in
`site/src/content/docs` and the README sections in the same PR. See
[Documentation site and READMEs](#documentation-site-and-readmes).

## Workflow: add or change a rule

Expand All @@ -133,7 +138,10 @@ js/CHANGELOG.md, spec/CHANGELOG.md, python/CHANGELOG.md generated by release-p
output, property tests passing, `pnpm gen:check` clean. Same for Python (`uv run pytest`,
`uv run python scripts/conformance.py`), then `node scripts/compare-ports.mjs` must report 0
differences.
7. **Commit message.** Conventional commits; the message *is* the changelog entry (see
7. **Docs.** Update the site pages and README sections that describe the rule (see
[Documentation site and READMEs](#documentation-site-and-readmes)), with JavaScript and Python
examples whose outputs you ran, and `pnpm docs:build` green.
8. **Commit message.** Conventional commits; the message *is* the changelog entry (see
[Versioning and release](#versioning-and-release)). Any change to an existing `expected` value is
a behaviour change and must be a `feat` (or carry `!`), never a `fix`/`chore`. Don't touch
versions or changelogs by hand.
Expand Down Expand Up @@ -249,6 +257,71 @@ Format and matcher: [`spec/fixtures/README.md`](spec/fixtures/README.md). Schema
- `js/test/skip.json` is for port-specific gaps only, keyed by case id with a reason. It can only
shrink: a skip-listed case that passes fails the run. Shared "known wrong" belongs in `xfail`.

## Documentation site and READMEs

The site at <https://4matic.github.io/trackparse/> (`site/`) is the user documentation; the READMEs
are short entry points that link into it. Both must describe the code as it is on `main`.

**Keep them in sync.** Whenever a change touches one of these, update the listed places in the same
PR:

| Change | Update |
|---|---|
| a rule's behaviour (R1–R12) | the guide for that area (`guides/what-it-understands`, `artists`, `versions`, `junk-and-flags`, `modes`, `formatting`), the root README tables |
| tracklists (R13) | `guides/tracklists`, `guides/editing-tracklists`, `reference/tracklist`, `reference/warnings` |
| a function, option, field or type | `reference/parse`, `reference/options`, `reference/functions` or `reference/tracklist`; the API sections of `js/README.md` and `python/README.md` |
| a warning code | `reference/warnings` |
| fixture counts, file counts, the size budget | `components/Stats.astro`, `index.mdx`, `start/introduction`, `project/testing`, `project/ports`, the root README "At a glance" and "How it's tested" |
| a port, a command, CI or the release flow | `project/ports`, `project/contributing`, `project/testing` ("Run it yourself") |
| `spec/SPEC.md` | nothing by hand: `reference/spec` is generated from it at build time |

**Every code example is bilingual.** There are no per-language pages. Each example on the site is a
pair of synced tabs, with the JavaScript example first:

````mdx
<Tabs syncKey="lang">
<TabItem label="JavaScript">

```js
parse("Noisia - Stigma (2015)").year; // 2015
```

</TabItem>
<TabItem label="Python">

```python
parse("Noisia - Stigma (2015)").year # 2015
```

</TabItem>
</Tabs>
````

- Keep the `syncKey` and the labels exactly as above; they make one choice apply site-wide.
Pages with tabs are `.mdx` and import `Tabs, TabItem` from `@astrojs/starlight/components`.
- Write the Python version in Python terms: snake_case functions, options and attributes
(`format_track`, `known_artists=`, `full_title`), tuples, `None`/`True`, and results as the
dataclass `repr` (`Junk(raw='HD', kind='quality')`).
- Run every example and paste the real output; never write an output by hand. Language-specific
notes (typing, `to_dict()`, `TypeError`) go on the page they concern, not on a separate page.
- The playground and the landing demo import `js/` directly, so they follow the code automatically;
check them in a browser when the output shape changes.

**READMEs link to the site.** Each README section opens with a `**Docs:**` line that links the
matching site pages (`README.md`, `js/README.md`, `python/README.md`, `rust/README.md`,
`spec/fixtures/README.md`, `spec/generator/README.md`). Add one for a new section, and fix the links
when a page or heading moves. Use full `https://4matic.github.io/trackparse/...` URLs: npm and PyPI
render the package READMEs where relative links break.

**Moving or removing a page.** npm (`homepage`), PyPI (`[project.urls]`) and published READMEs point
at site URLs that can't be changed after a release. When a page moves or disappears, add a
`redirects` entry in `site/astro.config.mjs` so the old URL keeps working, then update the
links in the repo. The sidebar is in the same file.

**Check before pushing:** `pnpm docs:build` passes (it fails on broken MDX), and you looked at the
changed pages with `pnpm docs`. Site-only commits (`docs(site)`, `fix(site)`, `feat(site)`) don't
trigger a release; changes under `js/`, `python/` or `spec/` do, even `docs:` ones.

## Porting guide (Rust, others)

A port is done when it passes every fixture and every invariant. The Python port is done and is a
Expand Down Expand Up @@ -383,4 +456,6 @@ Version rules:
- [ ] `pnpm --filter trackparse size` and `check:package` pass
- [ ] R13 changes: `node scripts/corpus-check.mjs` passes when you have the corpus
- [ ] Conventional commits and PR title; `feat` if any fixture expectation changed
- [ ] README and site (`site/src/content/docs`) examples still match real output if the change affects them
- [ ] Site pages (`site/src/content/docs`) and READMEs updated in the same PR, examples in both
JavaScript and Python tabs with real output, `**Docs:**` links still pointing at existing
pages, `pnpm docs:build` passes
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ Every key is always present. When a guess was involved, `warnings` says which on

## At a glance

**Docs:** [What it understands](https://4matic.github.io/trackparse/guides/what-it-understands/)

| Number | What it means |
|---|---|
| **12,965** shared fixtures | 1,148 written by hand, 11,817 generated from combinations. 100% pass in JS and Python, nothing skipped. |
Expand Down Expand Up @@ -123,6 +125,8 @@ Python 3.9+, no dependencies. A Rust port is [on the way](#ports).

## Quick start

**Docs:** [Getting started](https://4matic.github.io/trackparse/start/getting-started/)

```js
import { parse, parseArtists, format } from "trackparse";

Expand Down Expand Up @@ -178,6 +182,8 @@ full Python API is in [`python/README.md`](python/README.md).

## What it understands

**Docs:** [What it understands](https://4matic.github.io/trackparse/guides/what-it-understands/) · [Artists and credits](https://4matic.github.io/trackparse/guides/artists/) · [Versions and remixes](https://4matic.github.io/trackparse/guides/versions/) · [Junk, flags and prefixes](https://4matic.github.io/trackparse/guides/junk-and-flags/)

These are real outputs of `parse(input)` with default options, unless noted.

| Input | Artists | Title | Versions / other |
Expand Down Expand Up @@ -213,6 +219,8 @@ These are real outputs of `parse(input)` with default options, unless noted.

## Tracklists

**Docs:** [Tracklists](https://4matic.github.io/trackparse/guides/tracklists/) · [Editing tracklists](https://4matic.github.io/trackparse/guides/editing-tracklists/) · [Tracklist API](https://4matic.github.io/trackparse/reference/tracklist/)

Mixes and podcasts come with a tracklist: a block of text, one track per line, usually with a start
time in front. `trackparse/tracklist` reads the whole block. Each line gets its kind, its time and a
full `ParsedTrack`. The list as a whole gets checks with line numbers, and edits rewrite only the
Expand Down Expand Up @@ -307,6 +315,8 @@ apply_edits(text, [{"op": "set-time", "line": 8, "seconds": 4262}]).changed

### Tracklist mode

**Docs:** [Tracklist mode](https://4matic.github.io/trackparse/guides/tracklists/#tracklist-mode)

Inside a tracklist the first ` - ` is the artist/title separator on practically every line. A single
track string doesn't get that benefit of the doubt, so `parse()` sometimes reads the part after the
dash as a version or a genre. `parseTracklist` parses each line with `mode: "tracklist"`, which
Expand All @@ -330,6 +340,8 @@ of the list uses spaced dashes.

## How it's tested

**Docs:** [How it's tested](https://4matic.github.io/trackparse/project/testing/)

The fixtures in [`spec/fixtures`](spec/fixtures) are the source of truth. The spec says what should
happen, the fixtures pin it down, and every port has to pass all of them. The JS and Python
packages both do: **12,965 of 12,965**, with empty skip lists, in CI on every push.
Expand Down Expand Up @@ -430,6 +442,8 @@ cd python && uv run pytest # the same fixtures and invariants in Python

## API

**Docs:** [parse and ParsedTrack](https://4matic.github.io/trackparse/reference/parse/) · [Other functions](https://4matic.github.io/trackparse/reference/functions/)

### `parse(input, options?)` → `ParsedTrack`

| Field | Type | |
Expand Down Expand Up @@ -479,6 +493,8 @@ The vocabulary is in [`spec/data`](spec/data).

### Options

**Docs:** [Options](https://4matic.github.io/trackparse/reference/options/)

| Option | Default | |
|---|---|---|
| `mode` | `"auto"` | `"clean"`, `"youtube"`, `"filename"`, `"tracklist"`, or detect from the string |
Expand All @@ -489,6 +505,8 @@ The vocabulary is in [`spec/data`](spec/data).

### Other functions

**Docs:** [Other functions](https://4matic.github.io/trackparse/reference/functions/) · [Formatting](https://4matic.github.io/trackparse/guides/formatting/)

| Function | |
|---|---|
| `parseArtists(input, options?)` | Split a bare artist string into credits |
Expand All @@ -511,6 +529,8 @@ parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versi

### Modes

**Docs:** [Modes](https://4matic.github.io/trackparse/guides/modes/)

| Mode | For | Adds |
|---|---|---|
| `clean` | Store metadata, tags | Only a spaced dash separates artist and title |
Expand All @@ -521,6 +541,8 @@ parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versi

### Tracklist API

**Docs:** [Tracklist API](https://4matic.github.io/trackparse/reference/tracklist/) · [Warnings](https://4matic.github.io/trackparse/reference/warnings/)

Import from `trackparse/tracklist` (Python: `trackparse.tracklist`). It's a separate entry, so code
that only parses single tracks never loads it.

Expand Down Expand Up @@ -606,6 +628,8 @@ the same keys as JS: `{"op": "shift", "seconds": 45, "from": 5}`.

## How it works

**Docs:** [How it works](https://4matic.github.io/trackparse/project/how-it-works/)

A fixed pipeline of hand-written scanners. There are no regexes with lookaround and no
backtracking, so it runs in linear time and ports to Python and Rust with the same output.

Expand Down Expand Up @@ -634,6 +658,8 @@ spaced dashes. The checks run last, over the finished entries.

## Ports

**Docs:** [Ports](https://4matic.github.io/trackparse/project/ports/)

| Language | Package | Status |
|---|---|---|
| JavaScript / TypeScript | [`trackparse`](https://www.npmjs.com/package/trackparse) on npm | reference implementation, 12,965 / 12,965 fixtures, tracklists as `trackparse/tracklist` |
Expand All @@ -644,6 +670,8 @@ Want another language? The porting checklist is in [AGENTS.md](AGENTS.md).

## Compared to other libraries

**Docs:** [Compared to others](https://4matic.github.io/trackparse/project/comparison/)

| | trackparse | [get-artist-title](https://github.com/goto-bus-stop/get-artist-title) | [metadata-filter](https://github.com/web-scrobbler/metadata-filter) | [track_parser](https://rubygems.org/gems/track_parser) |
|---|---|---|---|---|
| Output | structured object | `[artist, title]` | cleaned strings | artists, feat, remixer |
Expand All @@ -659,6 +687,8 @@ trackparse is for when you need to know who did what.

## Limitations

**Docs:** [Limitations and roadmap](https://4matic.github.io/trackparse/project/roadmap/)

Not there yet:

- Tracklists that give each track's length instead of its start time. Times are always read as
Expand All @@ -678,6 +708,8 @@ Planned next: the Rust port. The browser playground is already live at

## Contributing

**Docs:** [Contributing](https://4matic.github.io/trackparse/project/contributing/)

The documentation site lives in [`site/`](site) (Astro + Starlight, `pnpm docs` to run it
locally) and deploys to GitHub Pages on every push to `main`.

Expand Down
20 changes: 20 additions & 0 deletions js/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ Node 18 or later. No Node-specific APIs are used, so it also runs in browsers an

## Quick start

**Docs:** [Getting started](https://4matic.github.io/trackparse/start/getting-started/)

```js
import { parse } from "trackparse";
// const { parse } = require("trackparse");
Expand All @@ -51,12 +53,16 @@ track.mode; // 'youtube' (auto-detected from the junk gro

## API

**Docs:** [parse and ParsedTrack](https://4matic.github.io/trackparse/reference/parse/) · [Other functions](https://4matic.github.io/trackparse/reference/functions/)

### `parse(input, options?) → ParsedTrack`

Parses one track string. Every key of the result is always present.

### `parseArtists(input, options?) → Artist[]`

**Docs:** [parseArtists](https://4matic.github.io/trackparse/reference/functions/#parseartistsinput-options)

Splits an artist string (no title) into credits.

```js
Expand All @@ -71,6 +77,8 @@ parseArtists("Sub Focus, Wilkinson & Dimension feat. Kojo");

### `format(track, options?) → string`

**Docs:** [format](https://4matic.github.io/trackparse/reference/functions/#formattrack-options) · [Formatting](https://4matic.github.io/trackparse/guides/formatting/)

Renders a `ParsedTrack` back to `Artists - Title (versions) (prod. ...)`.

```js
Expand All @@ -92,6 +100,8 @@ format(t, { feat: "omit", versions: "none" }); // 'Wilkinson - Afterglow'

### `normalize(input) → string`

**Docs:** [normalize](https://4matic.github.io/trackparse/reference/functions/#normalizeinput)

The Unicode cleanup applied before parsing: NFC, zero-width spaces, BOM and soft hyphens removed, fullwidth
brackets, smart quotes and dash variants mapped to ASCII, whitespace collapsed. Idempotent.

Expand All @@ -102,6 +112,8 @@ normalize(" Artist​ — Title (Remix) “Quoted” ");

### `allArtists(track) → Artist[]`

**Docs:** [allArtists](https://4matic.github.io/trackparse/reference/functions/#allartiststrack)

Track credits plus every version's remixers, deduplicated.

```js
Expand All @@ -112,6 +124,8 @@ allArtists(parse("Camo & Krooked - Kallisto (Noisia Remix)", { knownArtists: ["C

### `createParser(options) → { parse, parseArtists }`

**Docs:** [createParser](https://4matic.github.io/trackparse/reference/functions/#createparseroptions)

Compiles options, including extra vocabulary, once. Per-call options are merged over the base.

```js
Expand All @@ -133,6 +147,8 @@ The spec version this build implements, e.g. `"0.3.0"`. Always equal to the pack

## Options

**Docs:** [Options](https://4matic.github.io/trackparse/reference/options/)

| Option | Default | Effect |
|---|---|---|
| `mode` | `"auto"` | `"clean"` (tags, store metadata), `"youtube"` (video titles), `"filename"`, `"tracklist"` (one line of a tracklist), or `"auto"` to detect. |
Expand Down Expand Up @@ -207,6 +223,8 @@ The full field reference, a gallery of examples and the comparison with other li

## Tracklists

**Docs:** [Tracklists](https://4matic.github.io/trackparse/guides/tracklists/) · [Editing tracklists](https://4matic.github.io/trackparse/guides/editing-tracklists/)

`trackparse/tracklist` parses a whole tracklist: one entry per line with its kind, start time and
a full `ParsedTrack`, plus checks with line numbers. Joining `raw + eol` over the entries gives the
input back byte for byte, whatever the line endings. It's a separate entry point (about 7.6 KiB
Expand Down Expand Up @@ -288,6 +306,8 @@ Field tables and the full list of rules: the [tracklists guide](https://4matic.g

## Limitations

**Docs:** [Limitations and roadmap](https://4matic.github.io/trackparse/project/roadmap/)

One track string per `parse()` call; tracklists go through `trackparse/tracklist`, which reads
start times only (no durations, trailing times or ranges). No label/catalog, key or BPM fields, and
no built-in artist database: pass `knownArtists` for names that contain a joiner.
Expand Down
Loading
Loading