Fixture/snapshot tests for split, merge, overlap, and the analyzer

.github/workflows/test.yml

@@ -0,0 +1,18 @@
+name: Tests
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+      - run: npm ci
+      - run: npm test

SESSION_LOG.md

@@ -25,10 +25,19 @@
 
 **Backlog cleanup:** `feature/enhanced-chunking` turned out to be already deleted (backlog item was stale); pruned leftover local tracking refs for the merged `chore/remove-dead-css` + `feature/marketing-shell-parity` branches. Branch list is now just main + the three chunk-quality branches.
 
+## 2026-07-25 (cont. 2) — Fixture/snapshot test suite (backlog item since v3.1) DONE
+
+Will approved the fresh-scope recommendation. `feature/fixture-tests` (stacked on v3.4), 30 tests green in ~0.6s via `npm test` (node:test, zero new deps):
+- **Named exports added** at the bottom of `api/chunk.js` for the test surface (runtime unaffected — Vercel only uses the default handler; verified API still healthy afterward).
+- `tests/chunking.test.js` — recursiveSplit (limit, word preservation, paragraph boundaries, force-split), mergeSmallChunks, addOverlap (prefix content, recorded count, the exact 29-word clamp case from Chuck's page, no-op paths), initialism, Flesch-Kincaid ordering.
+- `tests/analyzer.test.js` — one isolated scenario per flag (incl. single-word-heading skip, UWM initialism anchoring, near-duplicate counted once per pair, overlap-prefix exclusion), per-chunk-average score math (85/B case), determinism, empty→null. Gotcha found: synthetic filler words are accidentally polysyllabic → FK grade >12, so the "simple prose" case is hand-written.
+- `tests/fixture-snapshot.test.js` — two committed HTML fixtures run through the full offline pipeline (extractSourceMetadata → buildChunks → enhanceChunks → analyzeChunks, tokenizer 'estimate' so snapshots don't churn with gpt-tokenizer versions); full-result deep-equal vs committed `*.snapshot.json`; regenerate intentionally with `UPDATE_SNAPSHOTS=1 npm test`. `nested-sections.html` is the regression fixture for the v3.3 extraction fix (parent must not swallow child `<section>` content; explicit asserts + snapshot).
+- `.github/workflows/test.yml` — npm ci + npm test on PRs and main pushes, Node 22.
+- Note: `node --test tests/` (directory form) fails silently on Node 22.22 — script uses the `'tests/*.test.js'` glob.
+
 **Open at day's end:**
-1. **Merge chain awaiting Will:** #10 (v3.2, base main) → #11 (v3.3, retarget to main after #10) → #12 (v3.4, retarget after #11). Merging main auto-deploys.
-2. Fixture/snapshot tests (split/merge/overlap/analyzer) — deferred since v3.1, still the one real backlog item.
-3. Phase 4 optional: template query generator, `?queries=` Ontologizer deep-link, compare mode, README rubric docs.
+1. **Merge chain awaiting Will:** #10 (v3.2, base main) → #11 (v3.3) → #12 (v3.4) → #13 (tests) — retarget each to main as its base lands. Merging to main auto-deploys.
+2. Phase 4 optional: template query generator, `?queries=` Ontologizer deep-link, compare mode, README rubric docs.
 
 ## 2026-07-24 — Chunk Quality plan (from Chuck's feedback) + repo moved out of archive
 

api/chunk.js

@@ -1095,3 +1095,19 @@ function slugify(text) {
     .replace(/\s+/g, '-')
     .slice(0, 64);
 }
+
+// Named exports for the test suite (tests/). Vercel only uses the default
+// handler; these change nothing at runtime.
+export {
+  CHUNK_SIZES,
+  countWords,
+  recursiveSplit,
+  mergeSmallChunks,
+  addOverlap,
+  buildChunks,
+  enhanceChunks,
+  analyzeChunks,
+  extractSourceMetadata,
+  initialism,
+  fleschKincaidGrade,
+};

package.json

@@ -5,6 +5,7 @@
   "type": "module",
   "main": "api/chunk.js",
   "scripts": {
+    "test": "node --test 'tests/*.test.js'",
     "serve": "python3 -m http.server 8080",
     "demo": "open public/index.html"
   },

tasks/todo.md

@@ -157,4 +157,4 @@ Branch: `feature/chunk-quality-v3.4`
 
 **Deviations from plan, all reasoned:** score model became per-chunk-average instead of flat deductions (proportionality); thin-section got an absolute 100w floor; anchor terms gained initialisms; answer-buried requires ≥2 heading content words; the near-duplicate flag surfaced the deferred nested-content extraction bug, fixed at the root in `extractContentPieces`/`stopCondition`.
 
-**Remaining backlog (not in this plan):** fixture/snapshot tests for split/merge/overlap/analyzer (deferred since v3.1); Phase 4 optional items (template query generator, `?queries=` deep-link from Ontologizer, compare mode, README rubric docs).
+**Remaining backlog (not in this plan):** ~~fixture/snapshot tests for split/merge/overlap/analyzer~~ **DONE 2026-07-25** (PR #13: 30 node:test tests, 2 HTML fixtures + committed snapshots, nested-section regression coverage, CI workflow); Phase 4 optional items (template query generator, `?queries=` deep-link from Ontologizer, compare mode, README rubric docs).

tests/analyzer.test.js

@@ -0,0 +1,226 @@
+// analyzeChunks() scenario tests: one isolated case per flag, scoring math,
+// and determinism. analyzeChunks mutates its input, so every scenario builds
+// fresh chunk objects.
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import { analyzeChunks } from '../api/chunk.js';
+
+const OPTIONS = { chunkSize: 'medium', strategy: 'heading', overlap: 35 };
+
+const SOURCE = {
+  url: 'https://example.com/rockets',
+  title: 'Rocket Engine Maintenance Guide - Acme Rocket Supply',
+  site: 'Acme Rocket Supply',
+  jsonld: [{ '@type': 'Organization', name: 'Acme Rocket Supply' }],
+};
+
+// Unique-vocabulary filler: short, simple sentences whose words never repeat
+// across seeds, so scenarios can't collide via the near-duplicate check.
+function filler(seed, sentences) {
+  return Array.from(
+    { length: sentences },
+    (_, i) => `${seed}${i}ax ${seed}${i}bx ${seed}${i}cx runs fine.`
+  ).join(' ');
+}
+
+function mkChunk(index, title, text, extra = {}) {
+  return {
+    big_chunk_index: index,
+    title,
+    level: 2,
+    fragment: `#chunk-${index}`,
+    small_chunks: [{ text, chunk_index: 1 }],
+    ...extra,
+  };
+}
+
+// ~105 words, anchored, heading echoed in the first sentence, no pronoun
+// opener, low reading grade — carries no non-info flags.
+function goodChunk(index, seed = 'alpha') {
+  return mkChunk(
+    index,
+    'Rocket Engine Maintenance',
+    `Rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler(seed, 19)}`
+  );
+}
+
+function nonInfoCodes(chunk) {
+  return chunk.flags.filter((f) => f.severity !== 'info').map((f) => f.code).sort();
+}
+
+test('clean page scores 100/A with no fixes', () => {
+  const chunks = [goodChunk(1, 'alpha'), goodChunk(2, 'bravo')];
+  const result = analyzeChunks(chunks, SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunks[0]), []);
+  assert.deepEqual(nonInfoCodes(chunks[1]), []);
+  assert.equal(result.score, 100);
+  assert.equal(result.grade, 'A');
+  assert.deepEqual(result.top_fixes, []);
+});
+
+test('dangling-reference: pronoun opener', () => {
+  const chunk = mkChunk(
+    1,
+    'Rocket Engine Maintenance',
+    `Our rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('charlie', 19)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['dangling-reference']);
+  assert.match(chunk.flags.find((f) => f.code === 'dangling-reference').message, /"Our"/);
+});
+
+test('thin-section: under the 100-word floor', () => {
+  const chunk = mkChunk(
+    1,
+    'Rocket Engine Maintenance',
+    `Rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('delta', 6)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['thin-section']);
+});
+
+test('oversized-section: over the medium max with no subheading', () => {
+  const chunk = mkChunk(
+    1,
+    'Rocket Engine Maintenance',
+    `Rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('echo', 110)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['oversized-section']);
+});
+
+test('generic-heading flags the stoplist and skips answer-buried', () => {
+  const chunk = mkChunk(
+    1,
+    'Overview',
+    `Rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('foxtrot', 19)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['generic-heading']);
+});
+
+test('answer-buried: first sentence ignores a multiword heading', () => {
+  const chunk = mkChunk(
+    1,
+    'Pricing And Contracts',
+    `Acme Rocket Supply ships worldwide from three depots. ${filler('golf', 19)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['answer-buried']);
+});
+
+test('answer-buried skips single-word headings', () => {
+  const chunk = mkChunk(
+    1,
+    'History',
+    `Acme Rocket Supply ships worldwide from three depots. ${filler('hotel', 19)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), []);
+});
+
+test('no-entity-anchor: section never names any page entity', () => {
+  const chunk = mkChunk(
+    1,
+    'Shipping Options',
+    `Shipping options include ground and air delivery for every order. ${filler('india', 19)}`
+  );
+  analyzeChunks([chunk], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), ['no-entity-anchor']);
+});
+
+test('no-entity-anchor accepts the initialism (UWM anchors the full name)', () => {
+  const source = {
+    url: 'https://example.edu/',
+    title: 'Admissions - University of Wisconsin-Milwaukee',
+    site: 'University of Wisconsin-Milwaukee',
+    jsonld: [],
+  };
+  const chunk = mkChunk(
+    1,
+    'Application Deadlines',
+    `Application deadlines at UWM fall in January and August. ${filler('juliet', 19)}`
+  );
+  analyzeChunks([chunk], source, OPTIONS);
+  assert.deepEqual(nonInfoCodes(chunk), []);
+});
+
+test('near-duplicate: identical chunks flag each other, counted once as a pair', () => {
+  const text = `Rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('kilo', 19)}`;
+  const a = mkChunk(1, 'Rocket Engine Maintenance', text);
+  const b = mkChunk(2, 'Rocket Engine Maintenance', text);
+  const result = analyzeChunks([a, b], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(a), ['near-duplicate']);
+  assert.deepEqual(nonInfoCodes(b), ['near-duplicate']);
+  const fix = result.top_fixes.find((f) => f.code === 'near-duplicate');
+  assert.equal(fix.count, 1); // one pair, not two instances
+});
+
+test('overlap prefixes are excluded from duplicate and thin checks', () => {
+  const a = goodChunk(1, 'lima');
+  const aWords = a.small_chunks[0].text.trim().split(/\s+/);
+  const prefix = aWords.slice(-60).join(' ');
+  const b = mkChunk(2, 'Launch Day Checklist', '', {
+    small_chunks: [{
+      text: `${prefix} The launch day checklist covers Acme Rocket Supply steps. ${filler('mike', 20)}`,
+      chunk_index: 1,
+      overlap_word_count: 60,
+    }],
+  });
+  analyzeChunks([a, b], SOURCE, OPTIONS);
+  assert.deepEqual(nonInfoCodes(a), []);
+  assert.deepEqual(nonInfoCodes(b), []);
+});
+
+test('readability: dense prose gets an info flag, simple prose does not', () => {
+  const dense = mkChunk(
+    1,
+    'Rocket Engine Maintenance',
+    'Rocket engine maintenance necessitates extraordinarily comprehensive organizational preparation incorporating interdependent technological capabilities alongside institutional considerations regarding Acme Rocket Supply operational infrastructure modernization initiatives representing considerable administrative complexity throughout implementation phases requiring deliberate prioritization frameworks additionally encompassing evaluation methodologies ' +
+      Array.from({ length: 12 }, (_, i) => `furthermore incorporating supplementary considerations regarding notwithstanding organizational deliberations ${i}`).join(' ')
+  );
+  // Genuinely plain prose — the synthetic filler() words are multi-syllable
+  // by accident, so a hand-written low-grade text is used here instead.
+  const simple = mkChunk(
+    2,
+    'Rocket Engine Maintenance',
+    'Rocket engine maintenance keeps each Acme Rocket Supply craft safe. We check the pump. We check the seals. We log each part. The crew signs off. Then we test fire the stand. It takes one day.'
+  );
+  analyzeChunks([dense, simple], SOURCE, OPTIONS);
+  assert.ok(dense.flags.some((f) => f.code === 'readability' && f.severity === 'info'));
+  assert.ok(!simple.flags.some((f) => f.code === 'readability'));
+});
+
+test('score is the per-chunk average and top fixes report point gains', () => {
+  const clean = goodChunk(1, 'oscar');
+  const flagged = mkChunk(
+    2,
+    'Rocket Engine Maintenance',
+    `Our rocket engine maintenance keeps every Acme Rocket Supply vehicle safe. ${filler('papa', 19)}`
+  );
+  const result = analyzeChunks([clean, flagged], SOURCE, OPTIONS);
+  assert.equal(clean.chunk_score, 100);
+  assert.equal(flagged.chunk_score, 70); // dangling-reference weighs 30
+  assert.equal(result.score, 85);
+  assert.equal(result.grade, 'B');
+  assert.deepEqual(result.top_fixes.map((f) => [f.code, f.count, f.points]), [
+    ['dangling-reference', 1, 15],
+  ]);
+});
+
+test('analysis is deterministic: identical input, identical output', () => {
+  const build = () => [
+    goodChunk(1, 'quebec'),
+    mkChunk(2, 'Overview', `Acme Rocket Supply ships worldwide from three depots. ${filler('romeo', 6)}`),
+  ];
+  const first = build();
+  const second = build();
+  const r1 = analyzeChunks(first, SOURCE, OPTIONS);
+  const r2 = analyzeChunks(second, SOURCE, OPTIONS);
+  assert.deepEqual(r1, r2);
+  assert.deepEqual(first, second); // flags + chunk_score attached identically
+});
+
+test('empty chunk list returns null', () => {
+  assert.equal(analyzeChunks([], SOURCE, OPTIONS), null);
+});