diff --git a/AGENTS.md b/AGENTS.md index 9118db4..c44b096 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -79,7 +79,8 @@ python/ PyPI package `trackparse` (Python 3.9+, zero deps, uv rust/ planned crates.io port (README stub only) site/ documentation site + playground (Astro + Starlight, GitHub Pages) astro.config.mjs base /trackparse, sidebar - src/content/docs/ the pages (Markdown/MDX); reference/spec.md is generated from spec/SPEC.md + src/content/docs/ the pages (Markdown/MDX, JS + Python tabs on every example); + reference/spec.md is generated from spec/SPEC.md src/components/ HeroDemo, Playground (runs js/ in the browser), Feature, Stats src/lib/ trackview.ts (highlight + field rendering), icons.ts (Lucide SVG strings) scripts/sync-spec.mjs copies spec/SPEC.md into the docs at build time @@ -112,6 +113,10 @@ js/CHANGELOG.md, spec/CHANGELOG.md, python/CHANGELOG.md generated by release-p disagree, fix both in the same change. - Principle: **extract, don't delete.** Anything recognised goes into a field; anything not recognised stays in the title verbatim. A rule that silently drops input is wrong. +- **The website and the READMEs ship with the change.** A PR that changes behaviour, API, options, + output fields, warnings, commands or published numbers updates the matching pages in + `site/src/content/docs` and the README sections in the same PR. See + [Documentation site and READMEs](#documentation-site-and-readmes). ## Workflow: add or change a rule @@ -133,7 +138,10 @@ js/CHANGELOG.md, spec/CHANGELOG.md, python/CHANGELOG.md generated by release-p output, property tests passing, `pnpm gen:check` clean. Same for Python (`uv run pytest`, `uv run python scripts/conformance.py`), then `node scripts/compare-ports.mjs` must report 0 differences. -7. **Commit message.** Conventional commits; the message *is* the changelog entry (see +7. **Docs.** Update the site pages and README sections that describe the rule (see + [Documentation site and READMEs](#documentation-site-and-readmes)), with JavaScript and Python + examples whose outputs you ran, and `pnpm docs:build` green. +8. **Commit message.** Conventional commits; the message *is* the changelog entry (see [Versioning and release](#versioning-and-release)). Any change to an existing `expected` value is a behaviour change and must be a `feat` (or carry `!`), never a `fix`/`chore`. Don't touch versions or changelogs by hand. @@ -249,6 +257,71 @@ Format and matcher: [`spec/fixtures/README.md`](spec/fixtures/README.md). Schema - `js/test/skip.json` is for port-specific gaps only, keyed by case id with a reason. It can only shrink: a skip-listed case that passes fails the run. Shared "known wrong" belongs in `xfail`. +## Documentation site and READMEs + +The site at (`site/`) is the user documentation; the READMEs +are short entry points that link into it. Both must describe the code as it is on `main`. + +**Keep them in sync.** Whenever a change touches one of these, update the listed places in the same +PR: + +| Change | Update | +|---|---| +| a rule's behaviour (R1–R12) | the guide for that area (`guides/what-it-understands`, `artists`, `versions`, `junk-and-flags`, `modes`, `formatting`), the root README tables | +| tracklists (R13) | `guides/tracklists`, `guides/editing-tracklists`, `reference/tracklist`, `reference/warnings` | +| a function, option, field or type | `reference/parse`, `reference/options`, `reference/functions` or `reference/tracklist`; the API sections of `js/README.md` and `python/README.md` | +| a warning code | `reference/warnings` | +| fixture counts, file counts, the size budget | `components/Stats.astro`, `index.mdx`, `start/introduction`, `project/testing`, `project/ports`, the root README "At a glance" and "How it's tested" | +| a port, a command, CI or the release flow | `project/ports`, `project/contributing`, `project/testing` ("Run it yourself") | +| `spec/SPEC.md` | nothing by hand: `reference/spec` is generated from it at build time | + +**Every code example is bilingual.** There are no per-language pages. Each example on the site is a +pair of synced tabs, with the JavaScript example first: + +````mdx + + + +```js +parse("Noisia - Stigma (2015)").year; // 2015 +``` + + + + +```python +parse("Noisia - Stigma (2015)").year # 2015 +``` + + + +```` + +- Keep the `syncKey` and the labels exactly as above; they make one choice apply site-wide. + Pages with tabs are `.mdx` and import `Tabs, TabItem` from `@astrojs/starlight/components`. +- Write the Python version in Python terms: snake_case functions, options and attributes + (`format_track`, `known_artists=`, `full_title`), tuples, `None`/`True`, and results as the + dataclass `repr` (`Junk(raw='HD', kind='quality')`). +- Run every example and paste the real output; never write an output by hand. Language-specific + notes (typing, `to_dict()`, `TypeError`) go on the page they concern, not on a separate page. +- The playground and the landing demo import `js/` directly, so they follow the code automatically; + check them in a browser when the output shape changes. + +**READMEs link to the site.** Each README section opens with a `**Docs:**` line that links the +matching site pages (`README.md`, `js/README.md`, `python/README.md`, `rust/README.md`, +`spec/fixtures/README.md`, `spec/generator/README.md`). Add one for a new section, and fix the links +when a page or heading moves. Use full `https://4matic.github.io/trackparse/...` URLs: npm and PyPI +render the package READMEs where relative links break. + +**Moving or removing a page.** npm (`homepage`), PyPI (`[project.urls]`) and published READMEs point +at site URLs that can't be changed after a release. When a page moves or disappears, add a +`redirects` entry in `site/astro.config.mjs` so the old URL keeps working, then update the +links in the repo. The sidebar is in the same file. + +**Check before pushing:** `pnpm docs:build` passes (it fails on broken MDX), and you looked at the +changed pages with `pnpm docs`. Site-only commits (`docs(site)`, `fix(site)`, `feat(site)`) don't +trigger a release; changes under `js/`, `python/` or `spec/` do, even `docs:` ones. + ## Porting guide (Rust, others) A port is done when it passes every fixture and every invariant. The Python port is done and is a @@ -383,4 +456,6 @@ Version rules: - [ ] `pnpm --filter trackparse size` and `check:package` pass - [ ] R13 changes: `node scripts/corpus-check.mjs` passes when you have the corpus - [ ] Conventional commits and PR title; `feat` if any fixture expectation changed -- [ ] README and site (`site/src/content/docs`) examples still match real output if the change affects them +- [ ] Site pages (`site/src/content/docs`) and READMEs updated in the same PR, examples in both + JavaScript and Python tabs with real output, `**Docs:**` links still pointing at existing + pages, `pnpm docs:build` passes diff --git a/README.md b/README.md index 9cbe482..11b49a5 100644 --- a/README.md +++ b/README.md @@ -95,6 +95,8 @@ Every key is always present. When a guess was involved, `warnings` says which on ## At a glance +**Docs:** [What it understands](https://4matic.github.io/trackparse/guides/what-it-understands/) + | Number | What it means | |---|---| | **12,965** shared fixtures | 1,148 written by hand, 11,817 generated from combinations. 100% pass in JS and Python, nothing skipped. | @@ -123,6 +125,8 @@ Python 3.9+, no dependencies. A Rust port is [on the way](#ports). ## Quick start +**Docs:** [Getting started](https://4matic.github.io/trackparse/start/getting-started/) + ```js import { parse, parseArtists, format } from "trackparse"; @@ -178,6 +182,8 @@ full Python API is in [`python/README.md`](python/README.md). ## What it understands +**Docs:** [What it understands](https://4matic.github.io/trackparse/guides/what-it-understands/) · [Artists and credits](https://4matic.github.io/trackparse/guides/artists/) · [Versions and remixes](https://4matic.github.io/trackparse/guides/versions/) · [Junk, flags and prefixes](https://4matic.github.io/trackparse/guides/junk-and-flags/) + These are real outputs of `parse(input)` with default options, unless noted. | Input | Artists | Title | Versions / other | @@ -213,6 +219,8 @@ These are real outputs of `parse(input)` with default options, unless noted. ## Tracklists +**Docs:** [Tracklists](https://4matic.github.io/trackparse/guides/tracklists/) · [Editing tracklists](https://4matic.github.io/trackparse/guides/editing-tracklists/) · [Tracklist API](https://4matic.github.io/trackparse/reference/tracklist/) + Mixes and podcasts come with a tracklist: a block of text, one track per line, usually with a start time in front. `trackparse/tracklist` reads the whole block. Each line gets its kind, its time and a full `ParsedTrack`. The list as a whole gets checks with line numbers, and edits rewrite only the @@ -307,6 +315,8 @@ apply_edits(text, [{"op": "set-time", "line": 8, "seconds": 4262}]).changed ### Tracklist mode +**Docs:** [Tracklist mode](https://4matic.github.io/trackparse/guides/tracklists/#tracklist-mode) + Inside a tracklist the first ` - ` is the artist/title separator on practically every line. A single track string doesn't get that benefit of the doubt, so `parse()` sometimes reads the part after the dash as a version or a genre. `parseTracklist` parses each line with `mode: "tracklist"`, which @@ -330,6 +340,8 @@ of the list uses spaced dashes. ## How it's tested +**Docs:** [How it's tested](https://4matic.github.io/trackparse/project/testing/) + The fixtures in [`spec/fixtures`](spec/fixtures) are the source of truth. The spec says what should happen, the fixtures pin it down, and every port has to pass all of them. The JS and Python packages both do: **12,965 of 12,965**, with empty skip lists, in CI on every push. @@ -430,6 +442,8 @@ cd python && uv run pytest # the same fixtures and invariants in Python ## API +**Docs:** [parse and ParsedTrack](https://4matic.github.io/trackparse/reference/parse/) · [Other functions](https://4matic.github.io/trackparse/reference/functions/) + ### `parse(input, options?)` → `ParsedTrack` | Field | Type | | @@ -479,6 +493,8 @@ The vocabulary is in [`spec/data`](spec/data). ### Options +**Docs:** [Options](https://4matic.github.io/trackparse/reference/options/) + | Option | Default | | |---|---|---| | `mode` | `"auto"` | `"clean"`, `"youtube"`, `"filename"`, `"tracklist"`, or detect from the string | @@ -489,6 +505,8 @@ The vocabulary is in [`spec/data`](spec/data). ### Other functions +**Docs:** [Other functions](https://4matic.github.io/trackparse/reference/functions/) · [Formatting](https://4matic.github.io/trackparse/guides/formatting/) + | Function | | |---|---| | `parseArtists(input, options?)` | Split a bare artist string into credits | @@ -511,6 +529,8 @@ parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versi ### Modes +**Docs:** [Modes](https://4matic.github.io/trackparse/guides/modes/) + | Mode | For | Adds | |---|---|---| | `clean` | Store metadata, tags | Only a spaced dash separates artist and title | @@ -521,6 +541,8 @@ parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versi ### Tracklist API +**Docs:** [Tracklist API](https://4matic.github.io/trackparse/reference/tracklist/) · [Warnings](https://4matic.github.io/trackparse/reference/warnings/) + Import from `trackparse/tracklist` (Python: `trackparse.tracklist`). It's a separate entry, so code that only parses single tracks never loads it. @@ -606,6 +628,8 @@ the same keys as JS: `{"op": "shift", "seconds": 45, "from": 5}`. ## How it works +**Docs:** [How it works](https://4matic.github.io/trackparse/project/how-it-works/) + A fixed pipeline of hand-written scanners. There are no regexes with lookaround and no backtracking, so it runs in linear time and ports to Python and Rust with the same output. @@ -634,6 +658,8 @@ spaced dashes. The checks run last, over the finished entries. ## Ports +**Docs:** [Ports](https://4matic.github.io/trackparse/project/ports/) + | Language | Package | Status | |---|---|---| | JavaScript / TypeScript | [`trackparse`](https://www.npmjs.com/package/trackparse) on npm | reference implementation, 12,965 / 12,965 fixtures, tracklists as `trackparse/tracklist` | @@ -644,6 +670,8 @@ Want another language? The porting checklist is in [AGENTS.md](AGENTS.md). ## Compared to other libraries +**Docs:** [Compared to others](https://4matic.github.io/trackparse/project/comparison/) + | | trackparse | [get-artist-title](https://github.com/goto-bus-stop/get-artist-title) | [metadata-filter](https://github.com/web-scrobbler/metadata-filter) | [track_parser](https://rubygems.org/gems/track_parser) | |---|---|---|---|---| | Output | structured object | `[artist, title]` | cleaned strings | artists, feat, remixer | @@ -659,6 +687,8 @@ trackparse is for when you need to know who did what. ## Limitations +**Docs:** [Limitations and roadmap](https://4matic.github.io/trackparse/project/roadmap/) + Not there yet: - Tracklists that give each track's length instead of its start time. Times are always read as @@ -678,6 +708,8 @@ Planned next: the Rust port. The browser playground is already live at ## Contributing +**Docs:** [Contributing](https://4matic.github.io/trackparse/project/contributing/) + The documentation site lives in [`site/`](site) (Astro + Starlight, `pnpm docs` to run it locally) and deploys to GitHub Pages on every push to `main`. diff --git a/js/README.md b/js/README.md index 09c9f80..ef9b563 100644 --- a/js/README.md +++ b/js/README.md @@ -29,6 +29,8 @@ Node 18 or later. No Node-specific APIs are used, so it also runs in browsers an ## Quick start +**Docs:** [Getting started](https://4matic.github.io/trackparse/start/getting-started/) + ```js import { parse } from "trackparse"; // const { parse } = require("trackparse"); @@ -51,12 +53,16 @@ track.mode; // 'youtube' (auto-detected from the junk gro ## API +**Docs:** [parse and ParsedTrack](https://4matic.github.io/trackparse/reference/parse/) · [Other functions](https://4matic.github.io/trackparse/reference/functions/) + ### `parse(input, options?) → ParsedTrack` Parses one track string. Every key of the result is always present. ### `parseArtists(input, options?) → Artist[]` +**Docs:** [parseArtists](https://4matic.github.io/trackparse/reference/functions/#parseartistsinput-options) + Splits an artist string (no title) into credits. ```js @@ -71,6 +77,8 @@ parseArtists("Sub Focus, Wilkinson & Dimension feat. Kojo"); ### `format(track, options?) → string` +**Docs:** [format](https://4matic.github.io/trackparse/reference/functions/#formattrack-options) · [Formatting](https://4matic.github.io/trackparse/guides/formatting/) + Renders a `ParsedTrack` back to `Artists - Title (versions) (prod. ...)`. ```js @@ -92,6 +100,8 @@ format(t, { feat: "omit", versions: "none" }); // 'Wilkinson - Afterglow' ### `normalize(input) → string` +**Docs:** [normalize](https://4matic.github.io/trackparse/reference/functions/#normalizeinput) + The Unicode cleanup applied before parsing: NFC, zero-width spaces, BOM and soft hyphens removed, fullwidth brackets, smart quotes and dash variants mapped to ASCII, whitespace collapsed. Idempotent. @@ -102,6 +112,8 @@ normalize(" Artist​ — Title （Remix） “Quoted” "); ### `allArtists(track) → Artist[]` +**Docs:** [allArtists](https://4matic.github.io/trackparse/reference/functions/#allartiststrack) + Track credits plus every version's remixers, deduplicated. ```js @@ -112,6 +124,8 @@ allArtists(parse("Camo & Krooked - Kallisto (Noisia Remix)", { knownArtists: ["C ### `createParser(options) → { parse, parseArtists }` +**Docs:** [createParser](https://4matic.github.io/trackparse/reference/functions/#createparseroptions) + Compiles options, including extra vocabulary, once. Per-call options are merged over the base. ```js @@ -133,6 +147,8 @@ The spec version this build implements, e.g. `"0.3.0"`. Always equal to the pack ## Options +**Docs:** [Options](https://4matic.github.io/trackparse/reference/options/) + | Option | Default | Effect | |---|---|---| | `mode` | `"auto"` | `"clean"` (tags, store metadata), `"youtube"` (video titles), `"filename"`, `"tracklist"` (one line of a tracklist), or `"auto"` to detect. | @@ -207,6 +223,8 @@ The full field reference, a gallery of examples and the comparison with other li ## Tracklists +**Docs:** [Tracklists](https://4matic.github.io/trackparse/guides/tracklists/) · [Editing tracklists](https://4matic.github.io/trackparse/guides/editing-tracklists/) + `trackparse/tracklist` parses a whole tracklist: one entry per line with its kind, start time and a full `ParsedTrack`, plus checks with line numbers. Joining `raw + eol` over the entries gives the input back byte for byte, whatever the line endings. It's a separate entry point (about 7.6 KiB @@ -288,6 +306,8 @@ Field tables and the full list of rules: the [tracklists guide](https://4matic.g ## Limitations +**Docs:** [Limitations and roadmap](https://4matic.github.io/trackparse/project/roadmap/) + One track string per `parse()` call; tracklists go through `trackparse/tracklist`, which reads start times only (no durations, trailing times or ranges). No label/catalog, key or BPM fields, and no built-in artist database: pass `knownArtists` for names that contain a joiner. diff --git a/python/README.md b/python/README.md index 11a2597..523a694 100644 --- a/python/README.md +++ b/python/README.md @@ -5,7 +5,7 @@ [![python](https://img.shields.io/github/actions/workflow/status/4matic/trackparse/python.yml?branch=main&label=tests&logo=github)](https://github.com/4matic/trackparse/actions/workflows/python.yml) [![license](https://img.shields.io/badge/license-MIT-blue)](https://github.com/4matic/trackparse/blob/main/LICENSE) -**[Documentation](https://4matic.github.io/trackparse/)** · **[Python guide](https://4matic.github.io/trackparse/guides/python/)** · **[Playground](https://4matic.github.io/trackparse/playground/)** +**[Documentation](https://4matic.github.io/trackparse/)** · **[Getting started](https://4matic.github.io/trackparse/start/getting-started/)** · **[Playground](https://4matic.github.io/trackparse/playground/)** **Turn messy track names into structured data: who made it, who's featured, who remixed it, and what's just noise.** @@ -39,6 +39,8 @@ uv add trackparse ## Quick start +**Docs:** [Getting started](https://4matic.github.io/trackparse/start/getting-started/) + ```python from trackparse import format_track, parse @@ -110,6 +112,8 @@ print(json.dumps(track.to_dict(), indent=2)) ## API +**Docs:** [Other functions](https://4matic.github.io/trackparse/reference/functions/) + ```python parse(input, *, mode="auto", uploader=None, known_artists=(), split_and="auto", keywords=None) -> ParsedTrack parse_artists(input, *, ) -> list[Artist] @@ -123,6 +127,8 @@ SPEC_VERSION, __version__ ### `parse(input, **options)` +**Docs:** [parse and ParsedTrack](https://4matic.github.io/trackparse/reference/parse/) · [Options](https://4matic.github.io/trackparse/reference/options/) + | Option | Default | | |---|---|---| | `mode` | `"auto"` | `"clean"`, `"youtube"`, `"filename"`, `"tracklist"`, or `"auto"` to detect from the string | @@ -163,6 +169,8 @@ the defaults. Passing something that isn't a `str` raises `TypeError`. ### `ParsedTrack` +**Docs:** [ParsedTrack fields](https://4matic.github.io/trackparse/reference/parse/#parsedtrack) + | Attribute | Type | | |---|---|---| | `input` | `str` | The original string, untouched | @@ -205,6 +213,8 @@ Version types, junk kinds and warnings are listed in the ### `to_dict()` and `from_dict()` +**Docs:** [JSON and Python objects](https://4matic.github.io/trackparse/reference/parse/#json-and-python-objects) + Every result class has `to_dict()`, which returns plain dicts and lists matching [`parsed-track.schema.json`](https://github.com/4matic/trackparse/blob/main/spec/schema/parsed-track.schema.json) (`fullTitle`, `unknownArtist`, ...). With `json.dumps(track.to_dict(), separators=(",", ":"), @@ -217,6 +227,8 @@ ParsedTrack.from_dict(track.to_dict()) == track # True ### `parse_artists(input, **options)` +**Docs:** [parseArtists](https://4matic.github.io/trackparse/reference/functions/#parseartistsinput-options) + Splits a bare artist string. Takes the same options as `parse`. ```python @@ -228,6 +240,8 @@ from trackparse import parse_artists ### `format_track(track, **options)` +**Docs:** [format](https://4matic.github.io/trackparse/reference/functions/#formattrack-options) · [Formatting](https://4matic.github.io/trackparse/guides/formatting/) + Renders a track back to a string. | Option | Default | | @@ -257,6 +271,8 @@ format_track(track, position=True) ### `normalize(s)` +**Docs:** [normalize](https://4matic.github.io/trackparse/reference/functions/#normalizeinput) + The Unicode cleanup every function runs first: NFC, invisible characters, and dash, bracket and quote variants. @@ -269,6 +285,8 @@ normalize("Noisia​ – Stigma （VIP）") ### `all_artists(track)` +**Docs:** [allArtists](https://4matic.github.io/trackparse/reference/functions/#allartiststrack) + Credits plus every remixer, deduplicated. ```python @@ -280,6 +298,8 @@ from trackparse import all_artists ### `create_parser(**options)` +**Docs:** [createParser](https://4matic.github.io/trackparse/reference/functions/#createparseroptions) + Compiles the options once, for many calls. Per-call options are merged over the parser's: a value passed per call replaces the base one (`known_artists` included), `None` leaves it alone, and `keywords` dicts are merged. @@ -302,6 +322,8 @@ released together and always share one version number. ## Modes +**Docs:** [Modes](https://4matic.github.io/trackparse/guides/modes/) + | Mode | For | Adds | |---|---|---| | `clean` | Store metadata, tags | Only a spaced dash separates artist and title | @@ -325,6 +347,8 @@ parse("Shelter", mode="youtube", uploader="Porter Robinson").artists ## Tracklists +**Docs:** [Tracklists](https://4matic.github.io/trackparse/guides/tracklists/) · [Editing tracklists](https://4matic.github.io/trackparse/guides/editing-tracklists/) + `trackparse.tracklist` parses a whole tracklist, one entry per line, and checks it. Each entry keeps its line exactly as written plus its own line ending, so `"".join(e.raw + e.eol for e in tl.entries)` is always the input, CRLF or not. @@ -434,6 +458,8 @@ parse("Electrosoul System - Liquid Metal", mode="tracklist").title ## Typing +**Docs:** [Types](https://4matic.github.io/trackparse/reference/parse/#types) + The package ships `py.typed` and passes `mypy --strict`. `ParsedTrack`, `Artist`, `Version`, `Junk`, `Flags`, `Position` and `Timestamp` are frozen dataclasses with snake_case attributes, and so are the tracklist results in `trackparse.tracklist`. @@ -462,7 +488,7 @@ same spec rule IDs. Behaviour changes start in the spec, not here: see ## Links -- [Documentation](https://4matic.github.io/trackparse/), with a [Python guide](https://4matic.github.io/trackparse/guides/python/) and a +- [Documentation](https://4matic.github.io/trackparse/), with every example in Python and JavaScript, and a [playground](https://4matic.github.io/trackparse/playground/) that runs the parser in your browser - [Spec](https://github.com/4matic/trackparse/blob/main/spec/SPEC.md) - [Project README](https://github.com/4matic/trackparse#readme) diff --git a/python/pyproject.toml b/python/pyproject.toml index adbce67..9814506 100644 --- a/python/pyproject.toml +++ b/python/pyproject.toml @@ -45,7 +45,7 @@ dependencies = [] [project.urls] Homepage = "https://4matic.github.io/trackparse/" -Documentation = "https://4matic.github.io/trackparse/guides/python/" +Documentation = "https://4matic.github.io/trackparse/start/getting-started/" Repository = "https://github.com/4matic/trackparse" Issues = "https://github.com/4matic/trackparse/issues" Changelog = "https://github.com/4matic/trackparse/blob/main/python/CHANGELOG.md" diff --git a/rust/README.md b/rust/README.md index a3443ae..bf6f146 100644 --- a/rust/README.md +++ b/rust/README.md @@ -5,3 +5,5 @@ only on `unicode-normalization`, with `serde` behind a feature flag, and it will fixtures in [`../spec/fixtures`](../spec/fixtures). Until then, see the [spec](../spec/SPEC.md) and the [JS package](../js). + +**Docs:** [Ports](https://4matic.github.io/trackparse/project/ports/) · [Adding a language](https://4matic.github.io/trackparse/project/ports/#adding-a-language) diff --git a/site/astro.config.mjs b/site/astro.config.mjs index bb630df..8d30727 100644 --- a/site/astro.config.mjs +++ b/site/astro.config.mjs @@ -8,6 +8,8 @@ export default defineConfig({ site: "https://4matic.github.io", base: "/trackparse", trailingSlash: "always", + // The Python guide was folded into the other pages (every example has a Python tab). PyPI links here. + redirects: { "/guides/python/": "/trackparse/start/getting-started/" }, markdown: { smartypants: false }, // Starlight is a multi-page site. Prefetching every link as it scrolls into view, plus the // cross-document view transitions in custom.css, makes page changes instant without a JS router. @@ -57,7 +59,6 @@ export default defineConfig({ { label: "Formatting", slug: "guides/formatting" }, { label: "Tracklists", slug: "guides/tracklists" }, { label: "Editing tracklists", slug: "guides/editing-tracklists" }, - { label: "Python", slug: "guides/python" }, ], }, { diff --git a/site/src/content/docs/guides/artists.md b/site/src/content/docs/guides/artists.mdx similarity index 57% rename from site/src/content/docs/guides/artists.md rename to site/src/content/docs/guides/artists.mdx index 0a2227b..854a46b 100644 --- a/site/src/content/docs/guides/artists.md +++ b/site/src/content/docs/guides/artists.mdx @@ -3,25 +3,44 @@ title: Artists and credits description: How trackparse splits artist lists, finds features and producers, and when to pass knownArtists. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + Every credit in the output is an `Artist`: + + + ```ts { name: string; role: "primary" | "featured" | "producer" | "remixer"; joiner: string | null; source: "artist" | "title" | "version" } ``` + + + +```python +Artist(name: str, role: ArtistRole, joiner: Optional[str], source: ArtistSource) +# role: "primary" | "featured" | "producer" | "remixer"; source: "artist" | "title" | "version" +``` + + + + `role` says what the person did. `joiner` is the text that introduced the name, exactly as written (`"&"`, `","`, `"ft."`, `"prod. by"`), and `null` for the first name of a list. `source` says where the credit was found: left of the dash (`artist`), right of it (`title`), or inside a version (`version`). Remixers live inside `versions`, not in `artists`. Use [`allArtists(track)`](/trackparse/reference/functions/#allartiststrack) -when you want everyone in one list. +(`all_artists` in Python) when you want everyone in one list. ## Joiners `&`, `x`, `vs.`, `,`, `;`, `+`, `/` and `and` separate names. They're joiners, not roles: every name they produce is still primary. + + + ```js parse("Skrillex & Diplo x Justin Bieber - Where Are Ü Now").artists; // Skrillex (joiner null), Diplo (joiner '&'), Justin Bieber (joiner 'x'), all primary @@ -30,6 +49,20 @@ parse("Kanye West vs. Daft Punk - Stronger").artists.map((a) => a.name); // [ 'Kanye West', 'Daft Punk' ] ``` + + + +```python +parse("Skrillex & Diplo x Justin Bieber - Where Are Ü Now").artists +# Skrillex (joiner None), Diplo (joiner '&'), Justin Bieber (joiner 'x'), all primary + +[a.name for a in parse("Kanye West vs. Daft Punk - Stronger").artists] +# ['Kanye West', 'Daft Punk'] +``` + + + + Some things look like joiners and aren't: - An unspaced `/`, `&` or `+` never splits. `AC/DC` and `Drum&Bass` stay whole. @@ -43,6 +76,9 @@ Some things look like joiners and aren't: `and` is common inside band names, so by default it only splits after a comma has already split the same list: + + + ```js parse("Simon and Garfunkel - The Boxer").artists.map((a) => a.name); // [ 'Simon and Garfunkel' ] @@ -51,7 +87,21 @@ parseArtists("Crosby, Stills, Nash and Young").map((a) => a.name); // [ 'Crosby', 'Stills', 'Nash', 'Young' ] ``` -Set `splitAnd: "always"` or `"never"` to change that. + + + +```python +[a.name for a in parse("Simon and Garfunkel - The Boxer").artists] +# ['Simon and Garfunkel'] + +[a.name for a in parse_artists("Crosby, Stills, Nash and Young")] +# ['Crosby', 'Stills', 'Nash', 'Young'] +``` + + + + +Set `splitAnd` (Python: `split_and`) to `"always"` or `"never"` to change that. ## Features @@ -59,35 +109,84 @@ Set `splitAnd: "always"` or `"never"` to change that. only count at the start of a bracket, since "with" turns up in plenty of titles. All of these give the second name the role `featured`: + + + ```js parse("Noisia feat. Foreign Beggars - Shellshock"); // source 'artist' parse("Noisia - Shellshock ft. Foreign Beggars"); // source 'title' parse("Silk City - Electricity (with Dua Lipa)"); // source 'title' ``` + + + +```python +parse("Noisia feat. Foreign Beggars - Shellshock") # source 'artist' +parse("Noisia - Shellshock ft. Foreign Beggars") # source 'title' +parse("Silk City - Electricity (with Dua Lipa)") # source 'title' +``` + + + + If the same person is credited twice, they appear once: + + + ```js parse("Noisia feat. Foreign Beggars - Shellshock (feat. Foreign Beggars)").artists.map((a) => a.name); // [ 'Noisia', 'Foreign Beggars' ] ``` + + + +```python +[a.name for a in parse("Noisia feat. Foreign Beggars - Shellshock (feat. Foreign Beggars)").artists] +# ['Noisia', 'Foreign Beggars'] +``` + + + + Names are compared with full Unicode case folding and accents removed, so `Sigur Rós` and `SIGUR ROS` count as the same artist. ## Producers + + + ```js const t = parse("Future - Mask Off (prod. by Metro Boomin)"); t.artists[1]; // { name: 'Metro Boomin', role: 'producer', joiner: 'prod. by', source: 'title' } t.title; // 'Mask Off' ``` + + + +```python +t = parse("Future - Mask Off (prod. by Metro Boomin)") +t.artists[1] # Artist(name='Metro Boomin', role='producer', joiner='prod. by', source='title') +t.title # 'Mask Off' +``` + + + + ## knownArtists +*(Python: `known_artists`)* + trackparse has no artist database, and that's deliberate. A built-in list would be wrong for half its users and impossible to keep current. Instead you tell it the names you care about: + + + ```js parse("Chase & Status - Baddadan").artists.map((a) => a.name); // [ 'Chase', 'Status' ] @@ -96,14 +195,31 @@ parse("Chase & Status - Baddadan", { knownArtists: ["Chase & Status"] }).artists // [ 'Chase & Status' ] ``` + + + +```python +[a.name for a in parse("Chase & Status - Baddadan").artists] +# ['Chase', 'Status'] + +[a.name for a in parse("Chase & Status - Baddadan", known_artists=["Chase & Status"]).artists] +# ['Chase & Status'] +``` + + + + A known name is matched case- and accent-insensitively wherever a credit list starts. It also protects names that begin with digits from being read as a track number (`3 Doors Down`). If you -parse a lot of strings with the same names, put them in [`createParser`](/trackparse/reference/functions/#createparseroptions). +parse a lot of strings with the same names, put them in [`createParser`](/trackparse/reference/functions/#createparseroptions) (`create_parser`). ## Unknown artists `ID` (and `Unknown`, `Untitled`, `?`, `???`) means the track hasn't been announced yet. It doesn't become a name. -The track gets `flags.unknownArtist: true` instead, and the same goes for titles: +The track gets `flags.unknownArtist: true` (`flags.unknown_artist` in Python) instead, and the same goes for titles: + + + ```js const t = parse("ID - ID (ID Remix)"); @@ -112,3 +228,17 @@ t.flags.unknownArtist; // true t.flags.unknownTitle; // true t.versions[0].unknownArtist; // true ``` + + + + +```python +t = parse("ID - ID (ID Remix)") +t.artists # () +t.flags.unknown_artist # True +t.flags.unknown_title # True +t.versions[0].unknown_artist # True +``` + + + diff --git a/site/src/content/docs/guides/editing-tracklists.md b/site/src/content/docs/guides/editing-tracklists.mdx similarity index 72% rename from site/src/content/docs/guides/editing-tracklists.md rename to site/src/content/docs/guides/editing-tracklists.mdx index 1255f75..7bbd190 100644 --- a/site/src/content/docs/guides/editing-tracklists.md +++ b/site/src/content/docs/guides/editing-tracklists.mdx @@ -3,16 +3,34 @@ title: Editing tracklists description: Fix times, shift whole sections, sort, insert and delete lines, and touch nothing else. --- -`applyEdits(text, edits)` takes a tracklist as text and returns new text. It's built for editors: +import { Tabs, TabItem } from "@astrojs/starlight/components"; + +`applyEdits(text, edits)` (`apply_edits` in Python) takes a tracklist as text and returns new text. It's built for editors: you get back the lines it changed, and every other line keeps its bytes and its line ending. + + + ```js import { applyEdits, parseTracklist } from "trackparse/tracklist"; ``` + + + +```python +from trackparse.tracklist import apply_edits, parse_tracklist +``` + + + + Using the [list from the previous page](/trackparse/guides/tracklists/#a-first-list), apply the suggested fix, then push everything from line 5 on back by 45 seconds: + + + ```js const fix = parseTracklist(text).warnings[1].suggestion; // { line: 8, raw: '01:11:02', seconds: 4262 } @@ -33,6 +51,32 @@ result.text.split("\r\n").slice(4); // ] ``` + + + +```python +fix = parse_tracklist(text).warnings[1].suggestion # Suggestion(line=8, raw='01:11:02', seconds=4262) + +result = apply_edits(text, [ + {"op": "set-time", "line": fix.line, "seconds": fix.seconds}, + {"op": "shift", "seconds": 45, "from": 5}, +]) + +result.changed # (5, 6, 7, 8, 9) +result.warnings # (EditWarning(code='style-upgraded', edit=1, line=5),) +result.text.split("\r\n")[4:] +# ['01:00:16 Alix Perez - Fade Away', +# '01:03:55 Interview with Hybris', +# '01:08:29 Imanu - Sacrifice', +# '01:11:47 Current Value feat. Teddy Killerz - Payload', +# '01:16:21 Hybris - Spinning Top'] +``` + + + + +In Python the edits are plain dicts with the same keys, `"from"` included. + Lines 1 to 4 are untouched, CRLF included. Edits run in order, each on the text the previous one produced, with line numbers as that text has them. @@ -43,7 +87,7 @@ produced, with line numbers as that text has them. | `{ op: "set-time", line, seconds }` | Rewrite the time on that line | | `{ op: "shift", seconds, lines?, from? }` | Add `seconds` (negative to go back) to the listed lines, to every timed line from `from` on, or to all of them | | `{ op: "sort-by-time" }` | Reorder by time. Untimed lines move with the timed line above them, and lines before the first time stay on top | -| `{ op: "set-rest", line, rest }` | Replace everything after the time, for example with `format(track)` | +| `{ op: "set-rest", line, rest }` | Replace everything after the time, for example with `format(track)` (`format_track`) | | `{ op: "insert", after, text }` | Insert a line after line `after` (`0` puts it at the top). It gets the text's usual line ending | | `{ op: "delete", line }` | Remove a line | @@ -77,16 +121,3 @@ The property tests check these on random text and random edit lists: - An edit never touches a line it doesn't name. - `shift(+s)` followed by `shift(-s)` gives back the original text, unless something was clamped or upgraded on the way. - -## Python - -Same function, snake_case, with plain dicts for edits. The keys are the same as in JS: - -```python -from trackparse.tracklist import apply_edits - -r = apply_edits(text, [{"op": "set-time", "line": 8, "seconds": 4262}, - {"op": "shift", "seconds": 45, "from": 5}]) -r.changed # (5, 6, 7, 8, 9) -r.warnings # (EditWarning(code='style-upgraded', edit=1, line=5),) -``` diff --git a/site/src/content/docs/guides/formatting.md b/site/src/content/docs/guides/formatting.mdx similarity index 57% rename from site/src/content/docs/guides/formatting.md rename to site/src/content/docs/guides/formatting.mdx index 8767ee6..a990951 100644 --- a/site/src/content/docs/guides/formatting.md +++ b/site/src/content/docs/guides/formatting.mdx @@ -3,9 +3,15 @@ title: Formatting description: Turn a ParsedTrack back into a string, in the style you want. --- -`format(track, options?)` renders a parsed track back to a string. Use it to normalise a messy +import { Tabs, TabItem } from "@astrojs/starlight/components"; + +`format(track, options?)` (`format_track(track, **options)` in Python) renders a parsed track back +to a string. Use it to normalise a messy library to one style, or to build a display title without the junk. + + + ```js import { format, parse } from "trackparse"; @@ -17,6 +23,23 @@ format(t); // 'Wilkinson ft. Becky Hill & Tom Cane - Afterglow (Sub Focus VIP Remix)' ``` + + + +```python +from trackparse import format_track, parse + +t = parse( + "01. Wilkinson ft. Becky Hill & Tom Cane - Afterglow (Sub Focus VIP Remix) [Official Video]" +) + +format_track(t) +# 'Wilkinson ft. Becky Hill & Tom Cane - Afterglow (Sub Focus VIP Remix)' +``` + + + + Junk is never written back. Everything else is, by default, the way it was written. ## Options @@ -30,6 +53,9 @@ Junk is never written back. Everything else is, by default, the way it was writt | `producers` | write `(prod. by …)` credits | `true` | | `position` | write the track number back | `false` | + + + ```js format(t, { feat: "title" }); // 'Wilkinson - Afterglow (ft. Becky Hill & Tom Cane) (Sub Focus VIP Remix)' @@ -48,8 +74,31 @@ format(parse("Future - Mask Off (prod. by Metro Boomin)"), { producers: false }) // 'Future - Mask Off' ``` -In Python the function is `format_track`, with keyword arguments: -`format_track(t, feat="title", joiners="canonical")`. + + + +```python +format_track(t, feat="title") +# 'Wilkinson - Afterglow (ft. Becky Hill & Tom Cane) (Sub Focus VIP Remix)' + +format_track(t, feat="title", joiners="canonical") +# 'Wilkinson - Afterglow (feat. Becky Hill & Tom Cane) (Sub Focus VIP Remix)' + +format_track(t, feat="artist", feat_marker="featuring") +# 'Wilkinson featuring Becky Hill & Tom Cane - Afterglow (Sub Focus VIP Remix)' + +format_track(t, feat="omit") # 'Wilkinson - Afterglow (Sub Focus VIP Remix)' +format_track(t, versions="none") # 'Wilkinson ft. Becky Hill & Tom Cane - Afterglow' +format_track(t, position=True) # '01. Wilkinson ft. Becky Hill & Tom Cane - Afterglow (Sub Focus VIP Remix)' + +format_track(parse("Future - Mask Off (prod. by Metro Boomin)"), producers=False) +# 'Future - Mask Off' +``` + + + + +Python takes the options as keyword arguments, so `featMarker` is `feat_marker`. ## Round trip @@ -57,9 +106,22 @@ Parsing the formatted string gives back the same artist names and roles, the sam same version types and remixers, as long as the first parse raised no warnings. The spec makes this an invariant (R11.6), and the property tests check it on generated input: + + + ```js parse(format(parse(s))) // same artists, title and versions as parse(s) ``` + + + +```python +parse(format_track(parse(s))) # same artists, title and versions as parse(s) +``` + + + + The [playground](/trackparse/playground/) has a `format()` panel if you want to try the options on your own strings. diff --git a/site/src/content/docs/guides/junk-and-flags.md b/site/src/content/docs/guides/junk-and-flags.mdx similarity index 62% rename from site/src/content/docs/guides/junk-and-flags.md rename to site/src/content/docs/guides/junk-and-flags.mdx index e3da4c1..8cc2ee9 100644 --- a/site/src/content/docs/guides/junk-and-flags.md +++ b/site/src/content/docs/guides/junk-and-flags.mdx @@ -3,17 +3,34 @@ title: Junk, flags and prefixes description: Video tags, quality markers, labels and platform suffixes; explicit and clean flags; years, track numbers and cue times. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + ## Junk Junk is everything that describes the upload rather than the song. It leaves the title, and it goes into `junk` with a kind so you can decide what to do with it. + + + ```js const t = parse("Daft Punk - One More Time (Official Music Video) [HD]"); t.title; // 'One More Time' t.junk; // [ { raw: 'Official Music Video', kind: 'video' }, { raw: 'HD', kind: 'quality' } ] ``` + + + +```python +t = parse("Daft Punk - One More Time (Official Music Video) [HD]") +t.title # 'One More Time' +t.junk # (Junk(raw='Official Music Video', kind='video'), Junk(raw='HD', kind='quality')) +``` + + + + | Kind | Examples | |---|---| | `video` | `Official Video`, `Official Music Video`, `Visualizer` | @@ -30,43 +47,100 @@ The phrases are in [`junk.json`](https://github.com/4matic/trackparse/blob/main/ (plus `genres.json` for the genre kind) and [`platform-suffixes.json`](https://github.com/4matic/trackparse/blob/main/spec/data/platform-suffixes.json). Seeing junk is also one of the ways `mode: "auto"` decides a string came from YouTube. + + + ```js parse("Porter Robinson - Shelter - YouTube").junk; // [ { raw: 'YouTube', kind: 'platform' } ] parse("Elektronomia - Sky High [NCS Release]").junk; // [ { raw: 'NCS Release', kind: 'label' } ] ``` + + + +```python +parse("Porter Robinson - Shelter - YouTube").junk # (Junk(raw='YouTube', kind='platform'),) +parse("Elektronomia - Sky High [NCS Release]").junk # (Junk(raw='NCS Release', kind='label'),) +``` + + + + Catalogue numbers like `[HOSP123]` are not junk yet. They stay in the title. See the [roadmap](/trackparse/project/roadmap/). ## Flags + + + ```js parse("Eminem - Without Me (Explicit)").flags; // { explicit: true, clean: false, unknownArtist: false, unknownTitle: false } ``` + + + +```python +parse("Eminem - Without Me (Explicit)").flags +# Flags(explicit=True, clean=False, unknown_artist=False, unknown_title=False) +``` + + + + `explicit` and `clean` come from `(Explicit)`, `[Clean]` and the like. `unknownArtist` and -`unknownTitle` are set when the artist or title is a placeholder such as `ID`. +`unknownTitle` (`unknown_artist` and `unknown_title` in Python) are set when the artist or title is a placeholder such as `ID`. ## Year A year on its own in brackets, or as a dash suffix, goes into `year`: + + + ```js const t = parse("Noisia - Stigma (2015)"); t.title; // 'Stigma' t.year; // 2015 ``` + + + +```python +t = parse("Noisia - Stigma (2015)") +t.title # 'Stigma' +t.year # 2015 +``` + + + + A year inside a version (`Remastered 2015`) belongs to that version instead, in `versions[i].year`. ## Track numbers + + + ```js parse("01. Aphex Twin - Xtal").position; // { raw: '01', number: 1 } parse("2 Unlimited - No Limit").position; // null ``` + + + +```python +parse("01. Aphex Twin - Xtal").position # Position(raw='01', number=1) +parse("2 Unlimited - No Limit").position # None +``` + + + + `raw` keeps the digits as written, so you can tell `01` from `1`. These count as track numbers: - a number followed by `.` or `)`: `01. Title`, `1) Intro`; @@ -80,10 +154,24 @@ A plain number before a word is part of the name: `2 Unlimited`, `50 Cent`, `808 A time at the very start becomes `timestamp`, with seconds worked out for you: + + + ```js parse("[12:34] Noisia - Stigma").timestamp; // { raw: '12:34', seconds: 754 } parse("1:02:03 Noisia - Stigma").timestamp; // { raw: '1:02:03', seconds: 3723 } ``` + + + +```python +parse("[12:34] Noisia - Stigma").timestamp # Timestamp(raw='12:34', seconds=754) +parse("1:02:03 Noisia - Stigma").timestamp # Timestamp(raw='1:02:03', seconds=3723) +``` + + + + For whole tracklists with a time on every line, use [`parseTracklist`](/trackparse/guides/tracklists/) -instead. It understands many more time formats and checks the order. +(`parse_tracklist`) instead. It understands many more time formats and checks the order. diff --git a/site/src/content/docs/guides/modes.md b/site/src/content/docs/guides/modes.mdx similarity index 64% rename from site/src/content/docs/guides/modes.md rename to site/src/content/docs/guides/modes.mdx index 769cdb3..f173ccf 100644 --- a/site/src/content/docs/guides/modes.md +++ b/site/src/content/docs/guides/modes.mdx @@ -3,6 +3,8 @@ title: Modes description: clean, youtube, filename and tracklist modes, what each one adds, and how auto picks between them. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + The same string can mean different things depending on where it came from. `Blink-182-All The Small Things` is clearly artist and title on YouTube. In a music library, a tight dash like that is more likely part of a name. Modes tell trackparse how much guessing the @@ -25,6 +27,9 @@ The resolved mode is in `track.mode`, so you can always see which one ran. 2. Junk, a `|`, or a platform suffix like `- YouTube` means `youtube`. 3. Anything else is `clean`. + + + ```js parse("07_Burial_-_Archangel.mp3"); // mode 'filename', position 7, artist 'Burial', title 'Archangel' @@ -33,11 +38,28 @@ parse("Noisia - Stigma | Official Video"); // mode 'youtube', title 'Stigma', junk [ { raw: 'Official Video', kind: 'video' } ] ``` + + + +```python +parse("07_Burial_-_Archangel.mp3") +# mode 'filename', position 7, artist 'Burial', title 'Archangel' + +parse("Noisia - Stigma | Official Video") +# mode 'youtube', title 'Stigma', junk (Junk(raw='Official Video', kind='video'),) +``` + + + + ## youtube mode Turn it on explicitly when you know the strings are video titles. It takes guesses that `clean` won't, and every guess shows up in `warnings`: + + + ```js parse('"Numb" by Linkin Park', { mode: "youtube" }); // artists [ 'Linkin Park' ], title 'Numb', warnings [ 'bySplit' ] @@ -49,6 +71,23 @@ parse("Blink-182-All The Small Things", { mode: "youtube" }); // artists [ 'Blink-182' ], title 'All The Small Things', warnings [ 'unspacedDashSplit' ] ``` + + + +```python +parse('"Numb" by Linkin Park', mode="youtube") +# artists ['Linkin Park'], title 'Numb', warnings ('bySplit',) + +parse('Linkin Park: "Numb"', mode="youtube") +# artists ['Linkin Park'], title 'Numb', warnings ('quotedTitleSplit',) + +parse("Blink-182-All The Small Things", mode="youtube") +# artists ['Blink-182'], title 'All The Small Things', warnings ('unspacedDashSplit',) +``` + + + + In `clean` mode the last one comes back with no artist, the whole string as the title and a `noSeparator` warning. That's the safe answer when you can't be sure. @@ -58,19 +97,48 @@ Many music videos are titled with just the song name, and the artist is the chan channel as `uploader` and youtube mode uses it when there's no separator. Auto-generated suffixes like `- Topic` and `VEVO` are dropped: + + + ```js parse("Around the World", { mode: "youtube", uploader: "Daft Punk - Topic" }); // artists [ 'Daft Punk' ], title 'Around the World' ``` + + + +```python +parse("Around the World", mode="youtube", uploader="Daft Punk - Topic") +# artists ['Daft Punk'], title 'Around the World' +``` + + + + ## filename mode + + + ```js parse("07_Burial_-_Archangel.mp3"); // position 7, Burial / Archangel parse("311 - Amber.mp3").position; // { raw: '311', number: 311 } parse("311 - Amber").artists[0].name; // '311' ``` + + + +```python +parse("07_Burial_-_Archangel.mp3") # position 7, Burial / Archangel +parse("311 - Amber.mp3").position # Position(raw='311', number=311) +parse("311 - Amber").artists[0].name # '311' +``` + + + + The last two show why the mode matters. In a file name a leading number with a dash is a track number. Anywhere else, `311` could be a band. @@ -79,6 +147,9 @@ number. Anywhere else, `311` could be a band. Inside a tracklist almost every line is `Artist - Title`, so the first spaced dash can be trusted. A single string doesn't get that benefit of the doubt: + + + ```js parse("Electrosoul System - Liquid Metal"); // title 'Electrosoul System', junk [ { raw: 'Liquid Metal', kind: 'genre' } ] @@ -87,5 +158,19 @@ parse("Electrosoul System - Liquid Metal", { mode: "tracklist" }); // artists [ 'Electrosoul System' ], title 'Liquid Metal' ``` -[`parseTracklist`](/trackparse/guides/tracklists/) uses this mode for every line. You only need to + + + +```python +parse("Electrosoul System - Liquid Metal") +# title 'Electrosoul System', junk (Junk(raw='Liquid Metal', kind='genre'),) + +parse("Electrosoul System - Liquid Metal", mode="tracklist") +# artists ['Electrosoul System'], title 'Liquid Metal' +``` + + + + +[`parseTracklist`](/trackparse/guides/tracklists/) (`parse_tracklist`) uses this mode for every line. You only need to set it yourself when you parse tracklist lines one at a time. diff --git a/site/src/content/docs/guides/python.md b/site/src/content/docs/guides/python.md deleted file mode 100644 index 5758636..0000000 --- a/site/src/content/docs/guides/python.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -title: Python -description: The Python package. Same parser, same output as JavaScript, with snake_case functions and frozen dataclasses. ---- - -```sh -pip install trackparse # or: uv add trackparse -``` - -Python 3.9 to 3.14, no dependencies, fully typed. The Python package isn't a wrapper around the -JavaScript one. It's a separate implementation that passes the same 12,965 fixtures, and a script -in CI compares the complete output of both on every fixture input. The difference count is zero. - -## Single tracks - -```python -from trackparse import format_track, parse, parse_artists - -track = parse("Noisia feat. Foreign Beggars - Shellshock") -track.artists -# (Artist(name='Noisia', role='primary', joiner=None, source='artist'), -# Artist(name='Foreign Beggars', role='featured', joiner='feat.', source='artist')) - -parse("Queen - Bohemian Rhapsody (Live at Wembley 1986)").versions[0] -# Version(type='live', raw='Live at Wembley 1986', artists=(), modifiers=(), -# descriptor='at Wembley', year=1986, unknown_artist=False, delimiter='(') - -[f"{a.name}:{a.role}" for a in parse_artists("Sub Focus, Wilkinson & Dimension feat. Kojo")] -# ['Sub Focus:primary', 'Wilkinson:primary', 'Dimension:primary', 'Kojo:featured'] - -format_track(parse("Wilkinson ft. Becky Hill - Afterglow (Sub Focus Remix)"), feat="title", joiners="canonical") -# 'Wilkinson - Afterglow (feat. Becky Hill) (Sub Focus Remix)' -``` - -Options are keyword arguments in snake_case: - -```python -parse(s, mode="youtube", known_artists=["Chase & Status"], split_and="never", uploader="Daft Punk - Topic") -``` - -## Names - -| JavaScript | Python | -|---|---| -| `parse` | `parse` | -| `parseArtists` | `parse_artists` | -| `format` | `format_track` | -| `normalize` | `normalize` | -| `allArtists` | `all_artists` | -| `createParser` | `create_parser` | -| `parseTracklist` | `parse_tracklist` (in `trackparse.tracklist`) | -| `applyEdits` | `apply_edits` (in `trackparse.tracklist`) | -| `fullTitle`, `unknownArtist`, … | `full_title`, `unknown_artist`, … | - -## Results are dataclasses - -`ParsedTrack`, `Artist`, `Version`, `Junk`, `Flags`, `Position` and `Timestamp` are frozen -dataclasses with snake_case attributes. Sequences are tuples, so results are immutable and hashable. -You can put them in sets or use them as dict keys. - -`to_dict()` gives you the spec's JSON with camelCase keys: - -```python -import json - -json.dumps(track.to_dict(), separators=(",", ":"), ensure_ascii=False) -# the same bytes as JSON.stringify(parse(s)) in JavaScript -``` - -The single-track classes also have `from_dict()`, which goes the other way. - -## Tracklists - -```python -from trackparse.tracklist import apply_edits, parse_tracklist - -tl = parse_tracklist(text, audio_seconds=4800, marker_words=["drop"]) -tl.warnings[1].suggestion -# Suggestion(line=8, raw='01:11:02', seconds=4262) - -apply_edits(text, [{"op": "shift", "seconds": -3000, "lines": [2, 3]}]).warnings -# (EditWarning(code='time-clamped', edit=0, line=2),) -``` - -Edits are plain dicts with the same keys as in JS, including `"from"`. The tracklist results -(`Tracklist`, `TracklistEntry`, `Parts`, `TimeValue`, `Suggestion`, `Stats`, `EditResult`) are frozen -dataclasses too, with `to_dict()`. - -## Typing - -The package ships `py.typed` and passes `mypy --strict`. Option dicts have `TypedDict` types -(`ParseOptions`, `KeywordOptions`, `FormatOptions`), and the string enums are `Literal` aliases -(`VersionType`, `JunkKind`, `Warning`, `Mode`, `EntryKind`, `TracklistWarningCode`, …). - -Every function raises `TypeError` if the input isn't a `str`. Anything else, including bad options, -falls back to defaults rather than raising. diff --git a/site/src/content/docs/guides/tracklists.md b/site/src/content/docs/guides/tracklists.mdx similarity index 76% rename from site/src/content/docs/guides/tracklists.md rename to site/src/content/docs/guides/tracklists.mdx index 7fb01ee..dab3173 100644 --- a/site/src/content/docs/guides/tracklists.md +++ b/site/src/content/docs/guides/tracklists.mdx @@ -3,24 +3,38 @@ title: Tracklists description: Parse the tracklist of a mix or podcast line by line, with start times, line kinds and checks that point at the exact line. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + Mixes and podcasts come with a tracklist: a block of text, one track per line, usually with a start time in front. `trackparse/tracklist` reads the whole block. Every line gets its kind, its time and a full [`ParsedTrack`](/trackparse/reference/parse/). The list as a whole gets checks with line numbers. + + + ```js import { parseTracklist } from "trackparse/tracklist"; ``` + + + ```python from trackparse.tracklist import parse_tracklist ``` + + + It's a separate entry point, about 7.6 KiB gzipped, so code that only parses single tracks never loads it. ## A first list + + + ```js const text = [ "Hour 2", @@ -50,6 +64,41 @@ list.stats; // { lines: 9, tracks: 7, markers: 1, timed: 8, firstTime: 2900, lastTime: 4536 } ``` + + + +```python +text = "\r\n".join([ + "Hour 2", + "48:20 Noisia - Stigma", + "52:47 Mefjus & Kasra - Bandwagon (Phace Remix)", + "56:05 Black Sun Empire - ID", + "59:31 Alix Perez - Fade Away", + "01:03:10 Interview with Hybris", + "01:07:44 Imanu - Sacrifice", + "00:11:02 Current Value feat. Teddy Killerz - Payload", # typo: should be 01:11:02 + "01:15:36 Hybris - Spinning Top", +]) + +tl = parse_tracklist(text) + +[e.kind for e in tl.entries] +# ['header', 'track', 'track', 'track', 'track', 'marker', 'track', 'track', 'track'] + +bandwagon = tl.entries[2] +bandwagon.time # TimeValue(raw='52:47', seconds=3167) +bandwagon.parts # Parts(lead='', time='52:47', gap=' ', rest='Mefjus & Kasra - Bandwagon (Phace Remix)') +[a.name for a in bandwagon.track.artists] # ['Mefjus', 'Kasra'] +[f"{v.type} by {v.artists[0].name}" for v in bandwagon.track.versions] # ['remix by Phace'] +tl.entries[3].track.flags.unknown_title # True (the ID) + +tl.stats +# Stats(lines=9, tracks=7, markers=1, timed=8, first_time=2900, last_time=4536) +``` + + + + Paste it into the [playground](/trackparse/playground/#list=Hour%202%0A48%3A20%20Noisia%20-%20Stigma%0A52%3A47%20Mefjus%20%26%20Kasra%20-%20Bandwagon%20(Phace%20Remix)%0A56%3A05%20Black%20Sun%20Empire%20-%20ID%0A59%3A31%20Alix%20Perez%20-%20Fade%20Away%0A01%3A03%3A10%20Interview%20with%20Hybris%0A01%3A07%3A44%20Imanu%20-%20Sacrifice%0A00%3A11%3A02%20Current%20Value%20feat.%20Teddy%20Killerz%20-%20Payload%0A01%3A15%3A36%20Hybris%20-%20Spinning%20Top) to see the table, the warnings and the fix button. @@ -96,10 +145,13 @@ raises an `invalid-time` warning. | `blank` | empty or only whitespace | Only `track` and `marker` entries have a `track`. No line is ever dropped. Extra marker words go in -the `markerWords` option: with `["drop"]`, `03:00 Drop` becomes a marker instead of a track. +the `markerWords` option (`marker_words` in Python): with `["drop"]`, `03:00 Drop` becomes a marker instead of a track. ## Checks + + + ```js list.warnings; // [ @@ -109,6 +161,19 @@ list.warnings; // ] ``` + + + +```python +tl.warnings +# (TracklistWarning(code='no-start-at-zero', line=2, severity='info', suggestion=None), +# TracklistWarning(code='time-out-of-order', line=8, severity='warning', +# suggestion=Suggestion(line=8, raw='01:11:02', seconds=4262))) +``` + + + + `00:11:02` sits between `01:07:44` and `01:15:36`. Changing one digit gives exactly one time that fits, `01:11:02`, so the warning carries it as a suggestion. When several fixes would fit, or none does, `suggestion` is `null` and you only get the warning. diff --git a/site/src/content/docs/guides/versions.md b/site/src/content/docs/guides/versions.mdx similarity index 61% rename from site/src/content/docs/guides/versions.md rename to site/src/content/docs/guides/versions.mdx index 6105973..44fa622 100644 --- a/site/src/content/docs/guides/versions.md +++ b/site/src/content/docs/guides/versions.mdx @@ -3,10 +3,15 @@ title: Versions and remixes description: The 27 version types, how generic words like "Mix" and "Edit" get their meaning, and what each Version field holds. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + A version is anything that says which take of a song this is: a remix, a VIP, a radio edit, a live recording, a remaster, a sped-up TikTok cut. trackparse reads them from brackets and from dash suffixes, and puts each one in `versions`, left to right. + + + ```ts { type: VersionType; // 'remix', 'vip', 'radio', … @@ -20,7 +25,26 @@ suffixes, and puts each one in `versions`, left to right. } ``` -`title` never includes versions. `fullTitle` is the title with them put back, in their original + + + +```python +Version( + type: VersionType, # 'remix', 'vip', 'radio', … + raw: str, # the text as written, without brackets: 'Sub Focus VIP Remix' + artists: tuple[Artist, ...], # remixers, role 'remixer' + modifiers: tuple[str, ...], # other type words that didn't win: ('extended',) + descriptor: Optional[str], # leftover words: "Taylor's", 'at Wembley' + year: Optional[int], + unknown_artist: bool, # 'ID Remix' + delimiter: VersionDelimiter, # "(" | "[" | "{" | "-" +) +``` + + + + +`title` never includes versions. `fullTitle` (`full_title` in Python) is the title with them put back, in their original brackets. ## The 27 types @@ -47,16 +71,34 @@ The keywords behind them are in | `(Extended VIP Mix)` | `vip`, modifiers `[extended]` | the strongest word wins, the rest become modifiers | | `(Taylor's Version)` | `version`, descriptor `"Taylor's"` | nothing to borrow, so the words are kept as a descriptor | + + + ```js parse("Pendulum - Watercolour (Extended VIP Mix)").versions[0]; // { type: 'vip', raw: 'Extended VIP Mix', artists: [], modifiers: [ 'extended' ], // descriptor: null, year: null, unknownArtist: false, delimiter: '(' } ``` + + + +```python +parse("Pendulum - Watercolour (Extended VIP Mix)").versions[0] +# Version(type='vip', raw='Extended VIP Mix', artists=(), modifiers=('extended',), +# descriptor=None, year=None, unknown_artist=False, delimiter='(') +``` + + + + ## Several versions Two groups give two versions. So does a slash inside one group: + + + ```js parse("Avicii - Levels (Skrillex Remix) [Radio Edit]").versions.map((v) => v.type); // [ 'remix', 'radio' ] @@ -65,28 +107,60 @@ parse("Dimension - Offshore (Mefjus Remix / Noisia Remix)").versions.map((v) => // [ 'Mefjus', 'Noisia' ] ``` + + + +```python +[v.type for v in parse("Avicii - Levels (Skrillex Remix) [Radio Edit]").versions] +# ['remix', 'radio'] + +[v.artists[0].name for v in parse("Dimension - Offshore (Mefjus Remix / Noisia Remix)").versions] +# ['Mefjus', 'Noisia'] +``` + + + + ## Dash suffixes Streaming services love writing versions after a second dash. Those are read too, and the delimiter is `-`: + + + ```js const t = parse("The Beatles - Hey Jude - Remastered 2015"); t.title; // 'Hey Jude' t.versions[0]; // { type: 'remaster', raw: 'Remastered 2015', year: 2015, delimiter: '-', … } ``` + + + +```python +t = parse("The Beatles - Hey Jude - Remastered 2015") +t.title # 'Hey Jude' +t.versions[0] # Version(type='remaster', raw='Remastered 2015', ..., year=2015, ..., delimiter='-') +``` + + + + A dash suffix that isn't a version stays in the title. In a [tracklist](/trackparse/guides/tracklists/#tracklist-mode) the rule is stricter, because there the first dash is almost always the separator. ## Unknown remixers `(ID Remix)` is a remix by someone not yet announced. You get the version with -`unknownArtist: true` and no artists, rather than an artist called "ID". +`unknownArtist: true` (`unknown_artist`) and no artists, rather than an artist called "ID". ## Your own words -If your catalogue uses a word trackparse doesn't know, add it with `keywords.versionHeads`: +If your catalogue uses a word trackparse doesn't know, add it with `keywords.versionHeads` (`version_heads` in Python): + + + ```js import { createParser } from "trackparse"; @@ -96,4 +170,18 @@ parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versi // 'rework' ``` + + + +```python +from trackparse import create_parser + +parser = create_parser(keywords={"version_heads": {"rerub": "rework"}}) +parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versions[0].type +# 'rework' +``` + + + + Keywords only add to the built-in vocabulary. They can't remove words. See [Options](/trackparse/reference/options/#keywords). diff --git a/site/src/content/docs/reference/functions.md b/site/src/content/docs/reference/functions.md deleted file mode 100644 index 7c8007e..0000000 --- a/site/src/content/docs/reference/functions.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -title: Other functions -description: parseArtists, format, normalize, allArtists, createParser and SPEC_VERSION. ---- - -All of these are exported from `trackparse`. Python names are in brackets. - -## parseArtists(input, options?) - -*(Python: `parse_artists`)* - -Splits a bare artist string, with no title, into credits. - -```js -parseArtists("Sub Focus, Wilkinson & Dimension feat. Kojo").map((a) => `${a.name}:${a.role}`); -// [ 'Sub Focus:primary', 'Wilkinson:primary', 'Dimension:primary', 'Kojo:featured' ] -``` - -Takes `knownArtists`, `splitAnd` and `keywords`. Useful for the artist field of ID3 tags or a -streaming API, where the title is already separate. - -## format(track, options?) - -*(Python: `format_track`)* - -Renders a `ParsedTrack` back to a string. The options are `feat`, `featMarker`, `joiners`, -`versions`, `producers` and `position`. [Formatting](/trackparse/guides/formatting/) has examples -of each. - -## normalize(input) - -The Unicode cleanup every function runs first. It applies NFC, removes invisible characters -(zero-width spaces, BOMs, soft hyphens), turns dash, bracket and quote variants into their plain -ASCII forms, and collapses whitespace. - -```js -normalize("Noisia​ — “Stigma”"); // 'Noisia - "Stigma"' -``` - -You don't need to call it before `parse`. It's exported because it's handy for building lookup -keys. - -## allArtists(track) - -*(Python: `all_artists`)* - -Every credit of a track in one list: the artists, then every remixer, with duplicates removed. - -```js -allArtists(parse("Wilkinson ft. Becky Hill - Afterglow (Sub Focus Remix)")).map((a) => `${a.name}:${a.role}`); -// [ 'Wilkinson:primary', 'Becky Hill:featured', 'Sub Focus:remixer' ] -``` - -## createParser(options) - -*(Python: `create_parser`)* - -Compiles options and keywords once and returns an object with `parse` and `parseArtists`. Per-call -options are merged over the compiled ones. Use it when you parse many strings with the same -`knownArtists` or `keywords`. - -```js -import { createParser } from "trackparse"; - -const parser = createParser({ - knownArtists: ["Chase & Status", "Camo & Krooked"], - keywords: { versionHeads: { rerub: "rework" } }, -}); - -parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versions[0].type; -// 'rework' -``` - -## SPEC_VERSION - -The version of the [spec](/trackparse/reference/spec/) the package implements, as a string. The -packages and the spec are released together, so it's the same as the package version. diff --git a/site/src/content/docs/reference/functions.mdx b/site/src/content/docs/reference/functions.mdx new file mode 100644 index 0000000..49999cf --- /dev/null +++ b/site/src/content/docs/reference/functions.mdx @@ -0,0 +1,158 @@ +--- +title: Other functions +description: parseArtists, format, normalize, allArtists, createParser and SPEC_VERSION. +--- + +import { Tabs, TabItem } from "@astrojs/starlight/components"; + +All of these are exported from `trackparse`. The Python package has the same functions in +snake_case: + +| JavaScript | Python | +|---|---| +| `parse` | `parse` | +| `parseArtists` | `parse_artists` | +| `format` | `format_track` | +| `normalize` | `normalize` | +| `allArtists` | `all_artists` | +| `createParser` | `create_parser` | +| `SPEC_VERSION` | `SPEC_VERSION` | +| `parseTracklist`, `applyEdits` (from `trackparse/tracklist`) | `parse_tracklist`, `apply_edits` (from `trackparse.tracklist`) | + +Options are keyword arguments in Python, and fields are snake_case attributes (`full_title`, +`unknown_artist`). Every function raises `TypeError` if the input isn't a `str`; anything else, +including bad options, falls back to defaults. + +## parseArtists(input, options?) + +*(Python: `parse_artists`)* + +Splits a bare artist string, with no title, into credits. + + + + +```js +parseArtists("Sub Focus, Wilkinson & Dimension feat. Kojo").map((a) => `${a.name}:${a.role}`); +// [ 'Sub Focus:primary', 'Wilkinson:primary', 'Dimension:primary', 'Kojo:featured' ] +``` + + + + +```python +[f"{a.name}:{a.role}" for a in parse_artists("Sub Focus, Wilkinson & Dimension feat. Kojo")] +# ['Sub Focus:primary', 'Wilkinson:primary', 'Dimension:primary', 'Kojo:featured'] +``` + + + + +Takes `knownArtists`, `splitAnd` and `keywords` (`known_artists`, `split_and`, `keywords`). Useful for the artist field of ID3 tags or a +streaming API, where the title is already separate. + +## format(track, options?) + +*(Python: `format_track`)* + +Renders a `ParsedTrack` back to a string. The options are `feat`, `featMarker` (`feat_marker`), +`joiners`, `versions`, `producers` and `position`. [Formatting](/trackparse/guides/formatting/) has examples +of each. + +## normalize(input) + +The Unicode cleanup every function runs first. It applies NFC, removes invisible characters +(zero-width spaces, BOMs, soft hyphens), turns dash, bracket and quote variants into their plain +ASCII forms, and collapses whitespace. + + + + +```js +normalize("Noisia​ — “Stigma”"); // 'Noisia - "Stigma"' +``` + + + + +```python +normalize("Noisia\u200b — “Stigma”") # 'Noisia - "Stigma"' +``` + + + + +You don't need to call it before `parse`. It's exported because it's handy for building lookup +keys. + +## allArtists(track) + +*(Python: `all_artists`)* + +Every credit of a track in one list: the artists, then every remixer, with duplicates removed. + + + + +```js +allArtists(parse("Wilkinson ft. Becky Hill - Afterglow (Sub Focus Remix)")).map((a) => `${a.name}:${a.role}`); +// [ 'Wilkinson:primary', 'Becky Hill:featured', 'Sub Focus:remixer' ] +``` + + + + +```python +[f"{a.name}:{a.role}" for a in all_artists(parse("Wilkinson ft. Becky Hill - Afterglow (Sub Focus Remix)"))] +# ['Wilkinson:primary', 'Becky Hill:featured', 'Sub Focus:remixer'] +``` + + + + +## createParser(options) + +*(Python: `create_parser`)* + +Compiles options and keywords once and returns an object with `parse` and `parseArtists` +(`parse_artists`). Per-call +options are merged over the compiled ones. Use it when you parse many strings with the same +`knownArtists` or `keywords`. + + + + +```js +import { createParser } from "trackparse"; + +const parser = createParser({ + knownArtists: ["Chase & Status", "Camo & Krooked"], + keywords: { versionHeads: { rerub: "rework" } }, +}); + +parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versions[0].type; +// 'rework' +``` + + + + +```python +from trackparse import create_parser + +parser = create_parser( + known_artists=["Chase & Status", "Camo & Krooked"], + keywords={"version_heads": {"rerub": "rework"}}, +) + +parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)").versions[0].type +# 'rework' +``` + + + + +## SPEC_VERSION + +The version of the [spec](/trackparse/reference/spec/) the package implements, as a string. The +packages and the spec are released together, so it's the same as the package version. diff --git a/site/src/content/docs/reference/options.md b/site/src/content/docs/reference/options.md deleted file mode 100644 index a5247f4..0000000 --- a/site/src/content/docs/reference/options.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -title: Options -description: Every option of parse, parseArtists and createParser, including custom keywords. ---- - -```ts -interface ParseOptions { - mode?: "auto" | "clean" | "youtube" | "filename" | "tracklist"; - knownArtists?: string[]; - uploader?: string; - splitAnd?: "auto" | "always" | "never"; - keywords?: KeywordOptions; -} -``` - -| Option | Default | | -|---|---|---| -| `mode` | `"auto"` | Which [mode](/trackparse/guides/modes/) to use. `auto` picks `filename`, `youtube` or `clean` from the string | -| `knownArtists` | `[]` | Names that are never split, like `Chase & Status` or `Earth, Wind & Fire`. Matched case- and accent-insensitively | -| `uploader` | – | The YouTube channel. In youtube mode it becomes the artist when there's no separator. `- Topic` and `VEVO` are dropped | -| `splitAnd` | `"auto"` | `auto` splits on `and` only after a comma (`A, B and C`), so `Simon and Garfunkel` stays whole. `always` and `never` do what they say | -| `keywords` | – | Extra vocabulary, see below | - -Python takes the same options as keyword arguments: `mode`, `known_artists`, `uploader`, -`split_and`, `keywords`. - -Options never cause an error. An unknown mode, a non-array `knownArtists` or a keyword of the wrong -type is ignored and the default is used. - -## keywords - -`keywords` adds words to the built-in vocabulary. It can't take any away. - -```ts -interface KeywordOptions { - versionHeads?: Record; // { rerub: "rework" } - descriptors?: string[]; // extra style words, like the built-in "D'n'B" - genres?: string[]; // extra genre names (junk kind 'genre') - junk?: Record; // { "live stream": "other" } - featMarkers?: string[]; // extra feat markers -} -``` - -Keys are compared the same way as the built-in words: ASCII letters are lowercased and one trailing -`. , ! ? : ;` is ignored. - -```js -import { createParser } from "trackparse"; - -const parser = createParser({ - knownArtists: ["Chase & Status", "Camo & Krooked"], - keywords: { versionHeads: { rerub: "rework" } }, -}); - -const t = parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)"); -t.artists.map((a) => a.name); // [ 'Chase & Status', 'Camo & Krooked' ] -t.versions[0].type; // 'rework' -``` - -## Tracklist options - -`parseTracklist` and `applyEdits` take `mode` (default `"tracklist"`), `knownArtists`, `splitAnd` -and `keywords`, plus two of their own: - -| Option | Default | | -|---|---|---| -| `audioSeconds` | – | Length of the audio. Turns on the `time-after-audio-end` check | -| `markerWords` | `[]` | Extra marker words. With `["drop"]`, `03:00 Drop` is a marker, not a track | - -See the [Tracklist API](/trackparse/reference/tracklist/). diff --git a/site/src/content/docs/reference/options.mdx b/site/src/content/docs/reference/options.mdx new file mode 100644 index 0000000..9df55c0 --- /dev/null +++ b/site/src/content/docs/reference/options.mdx @@ -0,0 +1,131 @@ +--- +title: Options +description: Every option of parse, parseArtists and createParser, including custom keywords. +--- + +import { Tabs, TabItem } from "@astrojs/starlight/components"; + + + + +```ts +interface ParseOptions { + mode?: "auto" | "clean" | "youtube" | "filename" | "tracklist"; + knownArtists?: string[]; + uploader?: string; + splitAnd?: "auto" | "always" | "never"; + keywords?: KeywordOptions; +} +``` + + + + +```python +parse( + input, + *, + mode: ModeOption = "auto", # "auto" | "clean" | "youtube" | "filename" | "tracklist" + known_artists: Sequence[str] = (), + uploader: Optional[str] = None, + split_and: str = "auto", # "auto" | "always" | "never" + keywords: Optional[KeywordOptions] = None, +) +``` + + + + +| Option (Python) | Default | | +|---|---|---| +| `mode` | `"auto"` | Which [mode](/trackparse/guides/modes/) to use. `auto` picks `filename`, `youtube` or `clean` from the string | +| `knownArtists` (`known_artists`) | `[]` | Names that are never split, like `Chase & Status` or `Earth, Wind & Fire`. Matched case- and accent-insensitively | +| `uploader` | – | The YouTube channel. In youtube mode it becomes the artist when there's no separator. `- Topic` and `VEVO` are dropped | +| `splitAnd` (`split_and`) | `"auto"` | `auto` splits on `and` only after a comma (`A, B and C`), so `Simon and Garfunkel` stays whole. `always` and `never` do what they say | +| `keywords` | – | Extra vocabulary, see below | + +Options never cause an error. An unknown mode, a `knownArtists` that isn't a list or a keyword of the wrong +type is ignored and the default is used. + +## keywords + +`keywords` adds words to the built-in vocabulary. It can't take any away. In Python its keys are +snake_case too: `version_heads`, `feat_markers`. + + + + +```ts +interface KeywordOptions { + versionHeads?: Record; // { rerub: "rework" } + descriptors?: string[]; // extra style words, like the built-in "D'n'B" + genres?: string[]; // extra genre names (junk kind 'genre') + junk?: Record; // { "live stream": "other" } + featMarkers?: string[]; // extra feat markers +} +``` + + + + +```python +class KeywordOptions(TypedDict, total=False): + version_heads: dict[str, VersionType] # {"rerub": "rework"} + descriptors: list[str] # extra style words, like the built-in "D'n'B" + genres: list[str] # extra genre names (junk kind 'genre') + junk: dict[str, JunkKind] # {"live stream": "other"} + feat_markers: list[str] # extra feat markers +``` + + + + +Keys are compared the same way as the built-in words: ASCII letters are lowercased and one trailing +`. , ! ? : ;` is ignored. + + + + +```js +import { createParser } from "trackparse"; + +const parser = createParser({ + knownArtists: ["Chase & Status", "Camo & Krooked"], + keywords: { versionHeads: { rerub: "rework" } }, +}); + +const t = parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)"); +t.artists.map((a) => a.name); // [ 'Chase & Status', 'Camo & Krooked' ] +t.versions[0].type; // 'rework' +``` + + + + +```python +from trackparse import create_parser + +parser = create_parser( + known_artists=["Chase & Status", "Camo & Krooked"], + keywords={"version_heads": {"rerub": "rework"}}, +) + +t = parser.parse("Chase & Status x Camo & Krooked - Track (Alix Perez Rerub)") +[a.name for a in t.artists] # ['Chase & Status', 'Camo & Krooked'] +t.versions[0].type # 'rework' +``` + + + + +## Tracklist options + +`parseTracklist` and `applyEdits` (`parse_tracklist`, `apply_edits`) take `mode` (default +`"tracklist"`), `knownArtists`, `splitAnd` and `keywords`, plus two of their own: + +| Option | Default | | +|---|---|---| +| `audioSeconds` (`audio_seconds`) | – | Length of the audio. Turns on the `time-after-audio-end` check | +| `markerWords` (`marker_words`) | `[]` | Extra marker words. With `["drop"]`, `03:00 Drop` is a marker, not a track | + +See the [Tracklist API](/trackparse/reference/tracklist/). diff --git a/site/src/content/docs/reference/parse.md b/site/src/content/docs/reference/parse.mdx similarity index 58% rename from site/src/content/docs/reference/parse.md rename to site/src/content/docs/reference/parse.mdx index ce98195..8adc2de 100644 --- a/site/src/content/docs/reference/parse.md +++ b/site/src/content/docs/reference/parse.mdx @@ -3,16 +3,33 @@ title: parse and ParsedTrack description: The parse function and every field of the ParsedTrack it returns. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + + + + ```ts parse(input: string, options?: ParseOptions): ParsedTrack ``` -`parse` never throws. Any string gives a `ParsedTrack`, and unknown or malformed options fall back -to their defaults. The options are on the [Options](/trackparse/reference/options/) page. + + + +```python +parse(input: str, *, mode="auto", known_artists=(), uploader=None, split_and="auto", keywords=None) -> ParsedTrack +``` + + + + +`parse` never throws on a string. Any string gives a `ParsedTrack`, and unknown or malformed options +fall back to their defaults. (Python raises `TypeError` if `input` isn't a `str` at all.) The options are on the [Options](/trackparse/reference/options/) page. ## ParsedTrack -Every key is always present. +Every key is always present. Python has the same fields as attributes of a frozen dataclass, in +snake_case (`full_title`, `flags.unknown_artist`), with tuples instead of arrays and `None` instead +of `null`. | Field | Type | | |---|---|---| @@ -31,10 +48,23 @@ Every key is always present. ## Artist + + + ```ts { name: string; role: ArtistRole; joiner: string | null; source: ArtistSource } ``` + + + +```python +Artist(name: str, role: ArtistRole, joiner: Optional[str], source: ArtistSource) +``` + + + + | Field | Values | |---|---| | `role` | `primary`, `featured`, `remixer`, `producer` | @@ -45,6 +75,9 @@ Every key is always present. ## Version + + + ```ts { type: VersionType; @@ -58,6 +91,25 @@ Every key is always present. } ``` + + + +```python +Version( + type: VersionType, + raw: str, + artists: tuple[Artist, ...], + modifiers: tuple[str, ...], + descriptor: Optional[str], + year: Optional[int], + unknown_artist: bool, + delimiter: VersionDelimiter, +) +``` + + + + | Field | | |---|---| | `type` | one of `remix` `bootleg` `vip` `edit` `flip` `refix` `rework` `mashup` `blend` `dub` `mix` `extended` `radio` `club` `original` `instrumental` `acapella` `live` `acoustic` `remaster` `demo` `reprise` `cover` `version` `spedUp` `slowed` `nightcore` | @@ -77,13 +129,48 @@ Every key is always present. in [Junk, flags and prefixes](/trackparse/guides/junk-and-flags/#junk). The vocabulary is in [`spec/data`](https://github.com/4matic/trackparse/tree/main/spec/data). +## JSON and Python objects + +In JavaScript the result is a plain object, so `JSON.stringify(track)` is the spec's JSON. + +In Python, `ParsedTrack`, `Artist`, `Version`, `Junk`, `Flags`, `Position` and `Timestamp` are +frozen dataclasses. Sequences are tuples, so results are immutable and hashable: you can put them in +sets or use them as dict keys. `to_dict()` gives the spec's JSON with its camelCase keys, and +`from_dict()` goes the other way: + +```python +import json + +json.dumps(track.to_dict(), separators=(",", ":"), ensure_ascii=False) +# the same bytes as JSON.stringify(parse(s)) in JavaScript + +ParsedTrack.from_dict(track.to_dict()) == track # True +``` + ## Types + + + All of these are exported from `trackparse` as TypeScript types: `ParsedTrack`, `Artist`, `ArtistRole`, `ArtistSource`, `Version`, `VersionType`, `VersionDelimiter`, `Junk`, `JunkKind`, `Flags`, `Position`, `Timestamp`, `Warning`, `Mode`, `ModeOption`, `ParseOptions`, `KeywordOptions`, `FormatOptions`. + + + +The package ships `py.typed` and passes `mypy --strict`. These are exported from `trackparse`: + +- dataclasses: `ParsedTrack`, `Artist`, `Version`, `Junk`, `Flags`, `Position`, `Timestamp`; +- `Literal` aliases: `ArtistRole`, `ArtistSource`, `VersionType`, `VersionDelimiter`, `JunkKind`, + `Warning`, `Mode`, `ModeOption`; +- `TypedDict`s for options: `ParseOptions`, `KeywordOptions`, `FormatOptions`; +- `Parser`, the class `create_parser` returns. + + + + The JSON shape is also published as a JSON Schema, [`parsed-track.schema.json`](https://github.com/4matic/trackparse/blob/main/spec/schema/parsed-track.schema.json). diff --git a/site/src/content/docs/reference/tracklist.md b/site/src/content/docs/reference/tracklist.mdx similarity index 70% rename from site/src/content/docs/reference/tracklist.md rename to site/src/content/docs/reference/tracklist.mdx index 206127d..55360dc 100644 --- a/site/src/content/docs/reference/tracklist.md +++ b/site/src/content/docs/reference/tracklist.mdx @@ -3,22 +3,38 @@ title: Tracklist API description: parseTracklist, applyEdits and every field they return. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + + + + ```js import { applyEdits, parseTracklist } from "trackparse/tracklist"; // CommonJS: const { applyEdits, parseTracklist } = require("trackparse/tracklist"); ``` + + + ```python from trackparse.tracklist import apply_edits, parse_tracklist ``` -Neither function throws on string input. The guides [Tracklists](/trackparse/guides/tracklists/) + + + +Neither function throws on string input. In Python the results are frozen dataclasses with +snake_case attributes (`spec_version`, `first_time`) and a `to_dict()` that returns the spec's JSON; +edits are plain dicts with the same keys as in JavaScript, `"from"` included. The guides [Tracklists](/trackparse/guides/tracklists/) and [Editing tracklists](/trackparse/guides/editing-tracklists/) walk through examples. ## parseTracklist(text, options?) Returns a `Tracklist`. Every key is always present. + + + ```ts { specVersion: string; @@ -28,6 +44,21 @@ Returns a `Tracklist`. Every key is always present. } ``` + + + +```python +Tracklist( + spec_version: str, + entries: tuple[TracklistEntry, ...], + warnings: tuple[TracklistWarning, ...], + stats: Stats, # lines, tracks, markers, timed, first_time, last_time +) +``` + + + + ### TracklistEntry One per line, in order. @@ -44,10 +75,24 @@ One per line, in order. ### TracklistWarning + + + ```ts { code: TracklistWarningCode; line: number; severity: "info" | "warning"; suggestion: { line, raw, seconds } | null } ``` + + + +```python +TracklistWarning(code: TracklistWarningCode, line: int, severity: Severity, suggestion: Optional[Suggestion]) +# Suggestion(line: int, raw: str, seconds: float) +``` + + + + Sorted by line. The codes are listed under [Warnings](/trackparse/reference/warnings/#tracklist-warnings). Warnings from each line's own `ParsedTrack` stay on `entries[i].track.warnings` and aren't repeated here. @@ -63,7 +108,7 @@ here. ## applyEdits(text, edits, options?) -Returns `{ text, changed, warnings }`. +Returns `{ text, changed, warnings }` (Python: an `EditResult`). | Field | | |---|---| @@ -105,12 +150,28 @@ them. | `audioSeconds` | – | Length of the audio. Enables `time-after-audio-end` | | `markerWords` | `[]` | Extra marker words | -Python uses keyword arguments (`audio_seconds=`, `marker_words=`). +Python uses keyword arguments: `mode`, `known_artists`, `split_and`, `keywords`, `audio_seconds`, +`marker_words`. ## Types + + + Exported from `trackparse/tracklist`: `Tracklist`, `TracklistEntry`, `EntryKind`, `EntryParts`, `Eol`, `TracklistTime`, `TimeSuggestion`, `TracklistStats`, `TracklistWarning`, `TracklistWarningCode`, `TracklistOptions`, `Edit`, `EditResult`, `EditWarning`, -`EditWarningCode`. The JSON Schema is +`EditWarningCode`. + + + + +Exported from `trackparse.tracklist`: the dataclasses `Tracklist`, `TracklistEntry`, `Parts`, +`TimeValue`, `Suggestion`, `Stats`, `TracklistWarning`, `EditResult`, `EditWarning`, and the +`Literal` aliases `EntryKind`, `Eol`, `Severity`, `TracklistWarningCode`, `EditWarningCode`. + + + + +The JSON Schema is [`tracklist.schema.json`](https://github.com/4matic/trackparse/blob/main/spec/schema/tracklist.schema.json). diff --git a/site/src/content/docs/start/introduction.md b/site/src/content/docs/start/introduction.mdx similarity index 88% rename from site/src/content/docs/start/introduction.md rename to site/src/content/docs/start/introduction.mdx index 0ca5570..252673f 100644 --- a/site/src/content/docs/start/introduction.md +++ b/site/src/content/docs/start/introduction.mdx @@ -3,6 +3,8 @@ title: Introduction description: What trackparse is, what it gives you back, and the one rule behind it. --- +import { Tabs, TabItem } from "@astrojs/starlight/components"; + trackparse reads the names of music tracks the way people actually write them: store metadata, YouTube titles, file names, scrobbles, the tracklist under a mix. It gives you back an object with the artists and their roles, the clean title, every remix or version with its credits, and the noise @@ -36,12 +38,27 @@ The same rule shows up in smaller ways: ## What you get back + + + ```js import { parse } from "trackparse"; parse("Queen - Bohemian Rhapsody (Live at Wembley 1986)"); ``` + + + +```python +from trackparse import parse + +parse("Queen - Bohemian Rhapsody (Live at Wembley 1986)").to_dict() +``` + + + + ```json { "input": "Queen - Bohemian Rhapsody (Live at Wembley 1986)", @@ -70,7 +87,9 @@ parse("Queen - Bohemian Rhapsody (Live at Wembley 1986)"); } ``` -The [`parse` reference](/trackparse/reference/parse/) explains every field. +Both give the same JSON. In Python the result is a frozen dataclass with snake_case attributes, and +`to_dict()` turns it into the JSON above. The [`parse` reference](/trackparse/reference/parse/) +explains every field. ## Spec first diff --git a/site/src/styles/custom.css b/site/src/styles/custom.css index aa7a5ec..b44ee60 100644 --- a/site/src/styles/custom.css +++ b/site/src/styles/custom.css @@ -156,3 +156,12 @@ header.header { navigation: none; } } + +/* Inline code is one box: a short span moves to the next line whole instead of splitting + (`mypy --strict`), and a span longer than the line wraps inside its own rounded box. */ +.sl-markdown-content :not(pre) > code { + display: inline-block; + max-width: 100%; + overflow-wrap: anywhere; + vertical-align: baseline; +} diff --git a/spec/fixtures/README.md b/spec/fixtures/README.md index 1737a63..1a1a298 100644 --- a/spec/fixtures/README.md +++ b/spec/fixtures/README.md @@ -3,6 +3,8 @@ The shared test corpus. **Every port runs every case here**. The fixtures, not any one implementation, define correct behaviour. +**Docs:** [How it's tested](https://4matic.github.io/trackparse/project/testing/) · [Contributing](https://4matic.github.io/trackparse/project/contributing/) + ``` fixtures/ handwritten/*.json curated cases, grouped by topic; edit by hand diff --git a/spec/generator/README.md b/spec/generator/README.md index 9e043c1..d4221ab 100644 --- a/spec/generator/README.md +++ b/spec/generator/README.md @@ -5,6 +5,8 @@ Combinatorial fixture generator for the trackparse spec. It writes size at 7.8 MB). Every port runs these files next to the handwritten fixtures. +**Docs:** [How it's tested: generated cases](https://4matic.github.io/trackparse/project/testing/#generated-11817-cases) + ## Oracle principle The generator is **not a parser**. Each case is a *composition* of hand-written **atoms** (an