Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,17 @@
# Changelog

## 1.14.0

- **A home map has Current storylines.** When `lab.toml` names active storylines, one of the library's home
maps has a section `<h2 id="current-storylines">Current storylines</h2>` whose rows are the active storyline
pages and nothing else, so every lab shows its storylines in the same place. `lab check` refuses a library
with no such section or with two, the section on a map that is not a home map, an active storyline with no
row there, a row listed twice, and a row there that is not an active storyline. Rows are read as folio reads
them: the first link of an item in a `rows` list, up to the next `h2` or `h3`. lab-set-up writes the section,
and lab-work keeps it when a storyline is added or retired.
- Upgrading: run `lab init`. A lab that names its storylines also moves their rows under Current storylines
(`folio map move <id> --to <map>#current-storylines`) before `lab check` passes.

## 1.13.0

- **A scope page is a storyline.** A storyline is one argument the lab is building from its findings, for one
Expand Down
13 changes: 11 additions & 2 deletions docs/model.md
Original file line number Diff line number Diff line change
Expand Up @@ -402,7 +402,8 @@ moves from included to excluded. Adding, merging or retiring a storyline is dire
you. A question's or an experiment's So far gives the current finding, not the story of the bugs on the way:
the replaced experiments and the reasons they were replaced keep that story.

A lab opts in by naming the pages in `lab.toml` (section 9). Then `lab check` holds each scope table:
A lab opts in by naming the pages in `lab.toml` (section 9), and its library's home map lists them under one
section, Current storylines. Then `lab check` holds that section and each scope table:
included lists only finished work that is not void, replaced or dropped, open only work still to do, nothing
parked or dropped is in either, and a goal, a question or a step sits in one row; a storyline page committed
before a result about something it lists under included or open is noted. `lab now` lists each storyline's
Expand Down Expand Up @@ -962,7 +963,15 @@ a question or a step has one row on a page: where another row needs to mention i
without its link. A question written in one row and a step under it written in another is allowed: the open
question with its one finished step included is the usual case. A decision may be cited in any row. In
`lab now` and in the note on a stale page, a goal or a question written under included or open stands for
every step under it. A lab without `[storylines]` has none of this.
every step under it.

While a storyline is active, one of the library's home maps (`home.maps` in `folio.yaml`) has a section headed
`<h2 id="current-storylines">Current storylines</h2>`, and its rows are the active storyline pages and nothing
else. A row is what folio counts as one: the first link, to `/content/<path>/`, of an item of a list whose classes
include `rows`, from the heading to the next `h2` or `h3`. A link in prose or in a row's reason is not a row. It
is the one place those pages have on a map; the overview and a retired storyline have theirs under other
headings. The library has one such section, and no map that is not a home map has one. A lab without
`[storylines]` has none of this.

**Worked-out states.** A goal is open until `reached: true` makes it done or `dropped:` drops it. A
question is waiting while the decision it came from is open, then open, then done with its answer; it
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ build-backend = "hatchling.build"
[project]
dynamic = ["readme"]
name = "lab-kit-cli"
version = "1.13.0"
version = "1.14.0"
description = "A research lab's record, its method, and the command that keeps both."
license = "MIT"
authors = [{ name = "Piergiuseppe Mallozzi" }]
Expand Down
15 changes: 9 additions & 6 deletions skills/lab-set-up/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,15 @@ You set the lab up in one discussion with the operator, so that everything it se
`views:`. Set it up with folio's own skill if it is not there.
12. If the lab will argue its findings to an audience (a paper, a talk, a report), ask the operator which
storylines it aims at now, for whom and where (it is direction, and may stay empty). A storyline is the
argument such an output is written from, not the output itself. Write the overview page (for example `entries/where-the-results-stand`) with folio's
write skill: the words on one running example, the workflow, the related work, and empty sections for the
findings and the criticisms. For each storyline, write `entries/storyline-<slug>`: its thesis, then the scope table
(target audience and venues, core thesis, experiments included, excluded or superseded, open), with the
last three cells marked `data-scope="included"`, `"excluded"` and `"open"`. Name them in `lab.toml` under
`[storylines]` (`overview`, `active`). No card names a storyline (method rule 8).
argument such an output is written from, not the output itself. Write the overview page (for example
`entries/where-the-results-stand`) with folio's write skill: the words on one running example, the
workflow, the related work, and empty sections for the findings and the criticisms. For each storyline,
write `entries/storyline-<slug>`: its thesis, then the scope table (target audience and venues, core
thesis, experiments included, excluded or superseded, open), with the last three cells marked
`data-scope="included"`, `"excluded"` and `"open"`. Name them in `lab.toml` under `[storylines]`
(`overview`, `active`), and give each storyline page its row on a home map under the heading Current
storylines (`folio map add <map> <id> --section "Current storylines"`, which writes
`<h2 id="current-storylines">`); nothing else goes under that heading. No card names a storyline (method rule 8).
13. Commit. Git is what holds the record: nothing is held before the first commit, and
`lab check --strict` says so.

Expand Down
8 changes: 7 additions & 1 deletion skills/lab-work/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,14 @@ row, move it to included, or to excluded if it is void or no longer serves the a
that fails makes the experiment void: take it off every included row. `lab check` refuses what a row may
not list, and notes a storyline committed before a result about something it lists.

A home map has the section Current storylines (`<h2 id="current-storylines">`): one row per active storyline
page, and nothing else. `lab check` holds that its rows and `active` name the same pages.

Adding, splitting, merging or retiring a storyline is direction: bring it to the operator (the lab-talk skill),
then change the pages and `[storylines]`. A retired storyline's page leaves `active` and stays in the library.
then change the pages, `[storylines]` and the Current storylines rows. A new storyline page gets its row with
`folio map add <map> <id> --section "Current storylines"`; a page that already has a place moves there with
`folio map move <id> --to <map>#current-storylines`. A retired storyline's page leaves `active` and stays in the
library: move its row under another heading (`folio map move <id> --to <map>#<heading>`).

## When to open a decision

Expand Down
2 changes: 1 addition & 1 deletion src/lab_kit/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
"""lab-kit: a research lab's record, its method, and the command that keeps both."""
__version__ = "1.13.0"
__version__ = "1.14.0"
88 changes: 87 additions & 1 deletion src/lab_kit/storylines.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"""
from __future__ import annotations

import os, re
import glob, os, re
from html.parser import HTMLParser

from . import record as R
Expand Down Expand Up @@ -65,6 +65,70 @@ def handle_data(self, data):
self.title += data


SECTION = "current-storylines"
PAGE_LINK = re.compile(r"^/content/(.+?)/?(?:index\.html)?(?:[#?].*)?$")


class _Section(HTMLParser):
"""The rows of a map's "Current storylines" section, as folio places them: from `<h2 id="current-storylines">`
to the next `h2` or `h3` or the end of `main`, each row the first link of an `<li>` in a list whose classes
include `rows`. A link in prose, in a row's reason or in another list is not a row."""

def __init__(self):
super().__init__(convert_charrefs=True)
self.found, self.links, self._in, self._lists, self._li = 0, [], False, [], False

def handle_starttag(self, tag, attrs):
a = dict(attrs)
if tag in ("h2", "h3"):
self._in = tag == "h2" and a.get("id") == SECTION
self.found += self._in
elif not self._in:
return
elif tag in ("ul", "ol"):
self._lists.append("rows" in (a.get("class") or "").split())
elif tag == "li":
self._li = bool(self._lists and self._lists[-1])
elif tag == "a" and self._li:
self._li = False # only the row's first link places a page
m = PAGE_LINK.match(a.get("href") or "")
if m:
self.links.append(m.group(1).strip("/"))

def handle_endtag(self, tag):
if tag == "main":
self._in = False
elif self._in and tag in ("ul", "ol") and self._lists:
self._lists.pop()
elif tag == "li":
self._li = False


def home_maps(lab):
"""The map files the library's charter lists under `home.maps`."""
import yaml
docs = os.path.join(lab.root, lab.cfg.get("library", "docs"))
try:
charter = yaml.safe_load(open(os.path.join(docs, "folio.yaml"))) or {}
except (OSError, yaml.YAMLError):
return []
home = charter.get("home") if isinstance(charter, dict) else None
names = (home.get("maps") if isinstance(home, dict) else None) or []
return [os.path.join(docs, "content", "maps", f"{n}.html") for n in names if isinstance(n, str)]


def section(lab):
"""[(map file, [page paths], on a home map)] for each map of the library with a "Current storylines" section."""
home = {os.path.normpath(f) for f in home_maps(lab)}
out = []
for f in sorted(glob.glob(os.path.join(lab.root, lab.cfg.get("library", "docs"), "content", "maps", "*.html")),
key=lambda f: (os.path.normpath(f) not in home, f)):
s = _Section()
s.feed(R.read(f))
out += [(f, s.links, os.path.normpath(f) in home)] * s.found
return out


def page_file(lab, path):
return os.path.join(lab.root, lab.cfg.get("library", "docs"), "content", path.strip("/"), "index.html")

Expand Down Expand Up @@ -180,6 +244,28 @@ def check(lab, bad, notes):
for a, b in (("included", "excluded"), ("included", "open"), ("open", "excluded")):
for i in sorted(i for i in set(rows[a]) & set(rows[b]) if lab.nodes.get(i, {}).get("type") != "decision"): # a decision may be cited wherever it explains a row
bad("storylines", where, f"lists {i} under both {a} and {b}: a step or a question has one row; where another row needs it, name it in words, without its link")
found = section(lab)
for f, _, home in found:
if not home:
bad("storylines", lab.rel(f), "has a \"Current storylines\" section but is not a home map (`home.maps` in "
"folio.yaml): the section goes on a home map, where its rows are the pages' places")
found = [x for x in found if x[2]]
if not found and active:
bad("storylines", "lab.toml", f"no home map of the library has a section `<h2 id=\"{SECTION}\">Current storylines</h2>`: "
"one row for each active storyline")
for f, _, _ in found[1:]:
bad("storylines", lab.rel(f), f"has a second \"Current storylines\" section; the library has one, on {lab.rel(found[0][0])}")
if found:
f, listed, _ = found[0]
for a in active:
if a not in listed:
bad("storylines", lab.rel(f), f"\"Current storylines\" has no row for {a}, which `[storylines]` lists as active")
for x in sorted(set(listed)):
if x not in active:
bad("storylines", lab.rel(f), f"\"Current storylines\" lists {x}, which is not an active storyline: "
"add it to `active`, or move its row under another heading (`folio map move`)")
elif listed.count(x) > 1:
bad("storylines", lab.rel(f), f"\"Current storylines\" lists {x} more than once")
for q, step, at in stale(lab):
notes.append(f"storylines: {q} was last committed before the result of {step} ({at}): bring the storyline's argument up to date with it")

Expand Down
44 changes: 43 additions & 1 deletion tests/test_storylines.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,19 @@ def page(root, path, title="Storyline A: The square-root law", included=(), excl
open(f"{d}/index.html", "w").write(PAGE.format(title=title, included=links(*included), excluded=links(*excluded), open=links(*open_)))


def storylines(root, overview="entries/where-the-results-stand", active=("entries/storyline-a",)):
def storylines(root, overview="entries/where-the-results-stand", active=("entries/storyline-a",), listed=None):
text = open(f"{root}/lab.toml").read().split("[storylines]")[0]
open(f"{root}/lab.toml", "w").write(text + f'[storylines]\noverview = "{overview}"\nactive = [{", ".join(chr(34) + a + chr(34) for a in active)}]\n')
home_map(root, active if listed is None else listed)


def home_map(root, listed, name="library", home=("library",), extra=""):
os.makedirs(f"{root}/docs/content/maps", exist_ok=True)
open(f"{root}/docs/folio.yaml", "w").write("name: lab\nhome:\n maps: [" + ", ".join(home) + "]\n")
rows = "".join(f'<li><a href="/content/{a}/">{a}</a> <span class="why">why</span></li>' for a in listed)
open(f"{root}/docs/content/maps/{name}.html", "w").write(
f'<html><body><main><h2 id="current-storylines">Current storylines</h2><ul class="rows">{rows}</ul>{extra}'
f'<h2 id="what-the-lab-proves">What the lab proves</h2><ul class="rows"><li><a href="/content/entries/where-the-results-stand/">Overview</a></li></ul></main></body></html>')


def two_experiments(root):
Expand Down Expand Up @@ -187,3 +197,35 @@ def test_a_table_inside_a_scope_cell_is_read_as_part_of_it(tmp_path):
open(f"{d}/index.html", "w").write(f'<html><body><table><tr><td data-scope="included"><table><tr><td>{links(first, void)}</td></tr></table></td>'
f'<td data-scope="excluded"></td><td data-scope="open">{links(later)}</td></tr></table></body></html>')
assert f"lists {void} under included, which is void" in lab(root, "check", "--no-rederive", ok=False)


def test_the_home_map_lists_the_active_storylines_under_current_storylines(tmp_path):
root = str(tmp_path)
first, void, later = two_experiments(root)
page(root, "entries/storyline-a", included=(first,), excluded=(void,), open_=(later,))
page(root, "entries/storyline-b", included=(first,), excluded=(void,), open_=(later,))
storylines(root, active=("entries/storyline-a", "entries/storyline-b"), listed=("entries/storyline-a", "entries/old-storyline"))
out = lab(root, "check", "--no-rederive", ok=False)
assert '"Current storylines" has no row for entries/storyline-b' in out
assert '"Current storylines" lists entries/old-storyline, which is not an active storyline' in out
assert "where-the-results-stand, which" not in out # a row under another heading is not read
# only a row's first link, in a `rows` list, before the next h2 or h3, is a row; other forms of a page's link count
extra = ('<p>See <a href="/content/entries/prose/">a page</a>.</p><ul><li><a href="/content/entries/pointer/">x</a></li></ul>'
'<ul class="rows"><li><a href="/content/entries/storyline-b/index.html#top">B</a> <span class="why">vs '
'<a href="/content/entries/why/">why</a></span></li></ul><h3 id="later">Later</h3><ul class="rows">'
'<li><a href="/content/entries/under-h3/">c</a></li></ul>')
home_map(root, ("entries/storyline-a",), extra=extra)
assert "storylines:" not in lab(root, "check", "--no-rederive")
home_map(root, ("entries/storyline-a", "entries/storyline-a", "entries/storyline-b"))
assert '"Current storylines" lists entries/storyline-a more than once' in lab(root, "check", "--no-rederive", ok=False)
home_map(root, ("entries/storyline-a", "entries/storyline-b"))
home_map(root, ("entries/storyline-a",), name="another", home=("library", "another"))
assert 'has a second "Current storylines" section' in lab(root, "check", "--no-rederive", ok=False)
home_map(root, ("entries/storyline-a",), name="another", home=("library",))
assert "is not a home map" in lab(root, "check", "--no-rederive", ok=False)
os.remove(f"{root}/docs/content/maps/another.html")
os.remove(f"{root}/docs/content/maps/library.html")
assert 'no home map of the library has a section `<h2 id="current-storylines">' in lab(root, "check", "--no-rederive", ok=False)
storylines(root, active=())
os.remove(f"{root}/docs/content/maps/library.html")
assert "storylines:" not in lab(root, "check", "--no-rederive") # no active storyline needs no section
Loading