Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions csv-ci-case-study.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="A reproducible public CSV check found 10 exact duplicate rows before a generated dataset was published. See the evidence, proposed fix, and CI boundary.">
<link rel="canonical" href="https://softpeanut.github.io/csv-preflight/csv-ci-case-study.html">
<title>Finding Duplicate CSV Rows Before Publication — CSV Preflight</title>
<link rel="icon" href="logo.svg" type="image/svg+xml">
<link rel="stylesheet" href="styles.css">
<style>
.article{max-width:780px;margin:0 auto;padding:3rem 1.25rem 5rem}.article h1{font-size:clamp(2rem,7vw,3.7rem);line-height:1.04}.article h2{margin-top:2.5rem}.article p,.article li{line-height:1.75}.article .lede{font-size:1.2rem;color:#495164}.article .facts{display:grid;grid-template-columns:repeat(auto-fit,minmax(150px,1fr));gap:12px;margin:2rem 0}.article .fact{padding:1rem;border:1px solid #d7dce5;border-radius:.6rem;background:#f7f8fa}.article .fact strong{display:block;font-size:1.6rem}.article .note{padding:1rem 1.2rem;border-left:4px solid #3659d9;background:#eef2ff}.article nav{margin-bottom:2rem}
</style>
</head>
<body>
<main class="article">
<nav><a href="./">Try CSV Preflight</a> · <a href="validate-csv-github-actions.html">GitHub Actions guide</a> · <a href="ci-setup-terms.html">$99 setup terms</a></nav>
<article>
<span class="eyebrow">PUBLIC REPRODUCTION · 2026-08-27</span>
<h1>Ten duplicate rows caught before publication</h1>
<p class="lede">A generated open-source parts database provided a real test of the free Action: inspect the output, report exact duplicates, and block publication without guessing which records to rewrite.</p>

<div class="facts">
<div class="fact"><strong>1.1 MB</strong>public CSV snapshot</div>
<div class="fact"><strong>1,583</strong>data rows checked</div>
<div class="fact"><strong>10</strong>exact duplicates reported</div>
<div class="fact"><strong>0</strong>issues after exact-row dedupe</div>
</div>

<h2>What was checked</h2>
<p>The public <a href="https://github.com/CDFER/jlcpcb-parts-database">CDFER/jlcpcb-parts-database</a> project generates a CSV and publishes it through GitHub Pages. Running the released CSV Preflight implementation against the live snapshot found 10 data rows that were byte-for-byte equivalent after CSV parsing. The report identified their row numbers; it did not delete or merge them.</p>

<h2>The narrow proposed fix</h2>
<p><a href="https://github.com/CDFER/jlcpcb-parts-database/pull/11">Public pull request #11</a> adds a full-row <code>drop_duplicates()</code> call after the generator merges its data, then runs the Action before the Pages artifact is uploaded. On the reproduced snapshot, this reduced 1,583 rows to 1,573 and the validator reported no remaining structural issue.</p>

<div class="note"><strong>Status boundary:</strong> the pull request is an open contribution, not a customer result or an accepted upstream change. The repository's full multi-gigabyte generation process was not run locally, and the PR says so.</div>

<h2>Why exact duplicates are the safe first boundary</h2>
<p>Rows that merely share a part number or description may still represent distinct records. Automatically merging near-duplicates would introduce a business rule that a structural validator cannot justify. This check reports exact repeated rows; the owning project decides whether and where deduplication belongs.</p>

<h2>Reusable workflow pattern</h2>
<ol>
<li>Generate or update the CSV.</li>
<li>Run the Action using an immutable commit SHA.</li>
<li>Keep the issue report as an artifact even when validation fails.</li>
<li>Upload or publish data only after the check passes.</li>
</ol>

<p>The complete copy-ready workflow is in the <a href="validate-csv-github-actions.html">GitHub Actions guide</a>. It validates one generic UTF-8 CSV up to 10 MiB on the runner, requires no API key, and does not upload the input. Passing checks CSV structure only; it does not guarantee an importer accepts the file.</p>

<section class="pro">
<div><span class="eyebrow">FIXED SCOPE · USD 99</span><h2>Want this wired into one repository?</h2><p>One public repository or sanitized minimal reproduction, one CSV path, one workflow, diagnostic artifacts, a usage note, and one revision. Scope is confirmed in writing before payment.</p></div>
<a href="https://github.com/softpeanut/csv-preflight/issues/new?template=ci-setup.yml" target="_blank" rel="noopener">Check setup fit ↗</a>
</section>
</article>
</main>
</body>
</html>
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>CSV Preflight — local CSV and Shopify product import checker</title><meta name="description" content="Check CSV structure locally, with an optional Shopify product CSV preset for documented headers and values. No upload or signup."><link rel="canonical" href="https://softpeanut.github.io/csv-preflight/"><meta property="og:title" content="CSV Preflight — local CSV and Shopify product import checker"><meta property="og:description" content="Check generic CSV structure or a bounded set of documented Shopify product CSV rules without uploading your data."><meta property="og:url" content="https://softpeanut.github.io/csv-preflight/"><meta property="og:type" content="website"><meta name="twitter:card" content="summary"><link rel="icon" href="logo.svg" type="image/svg+xml"><link rel="stylesheet" href="styles.css"><script type="application/ld+json">{"@context":"https://schema.org","@type":"SoftwareApplication","name":"CSV Preflight","url":"https://softpeanut.github.io/csv-preflight/","applicationCategory":"DeveloperApplication","operatingSystem":"Any","description":"Check generic CSV structure or a bounded set of documented Shopify product CSV rules locally without uploading files.","isAccessibleForFree":true,"offers":{"@type":"Offer","price":"0","priceCurrency":"USD"}}</script></head>
<body><main><p><a href="article.html">Read: CSV Is a State Machine</a> · <a href="case-study.html">Checkable example</a> · <a href="ko.html" lang="ko">한국어 서비스</a></p><header><span class="eyebrow">LOCAL-ONLY CSV CHECK</span><h1>Catch import failures<br>before your importer does.</h1><p>Drop or paste CSV. Nothing leaves your browser.</p></header>
<body><main><p><a href="article.html">Read: CSV Is a State Machine</a> · <a href="case-study.html">Checkable example</a> · <a href="csv-ci-case-study.html">Public CI case study</a> · <a href="ko.html" lang="ko">한국어 서비스</a></p><header><span class="eyebrow">LOCAL-ONLY CSV CHECK</span><h1>Catch import failures<br>before your importer does.</h1><p>Drop or paste CSV. Nothing leaves your browser.</p></header>
<aside class="tip"><div><strong>Did the free checker save you time?</strong><span>An optional Lightning tip buys no cleanup, support, feature, or import guarantee. It is separate from the fixed-scope service.</span></div><a href="lightning:softpeanut@stacker.news">Tip 2+ sats · softpeanut@stacker.news</a></aside>
<section id="drop" class="card drop"><input id="file" type="file" accept=".csv,text/csv"><label for="file"><strong>Drop a CSV here</strong><span>or choose a file</span></label><div class="or">OR PASTE</div><textarea id="input" placeholder="email,name&#10;ada@example.com,Ada"></textarea><div class="profile"><label for="profile">Validation profile</label><select id="profile"><option value="generic">Generic CSV structure</option><option value="shopify:create">Shopify — new products</option><option value="shopify:update">Shopify — update products</option></select><small>Shopify preset reviewed against the <a href="https://help.shopify.com/en/manual/products/import-export/using-csv" target="_blank" rel="noopener">official product CSV guide</a> on 2026-08-27. It checks a bounded set of documented rules and does not guarantee import success.</small></div><button id="check">Run preflight</button></section>
<section id="results" class="card" hidden><div class="summary"><div><small>ENCODING</small><strong id="encoding">Pasted text</strong></div><div><small>DELIMITER</small><strong id="delimiter">Comma</strong></div><div><small>ROWS</small><strong id="rows">0</strong></div><div><small>ISSUES</small><strong id="count">0</strong></div></div><h2>Preflight report</h2><ul id="issues"></ul><div class="actions"><button id="csvDownload">Download cleaned CSV</button><button id="reportDownload" class="secondary">Download error report</button></div></section>
Expand Down
15 changes: 15 additions & 0 deletions offer.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ const koreanTemplate = readFileSync(
const robots = readFileSync(new URL("./robots.txt", import.meta.url), "utf8");
const sitemap = readFileSync(new URL("./sitemap.xml", import.meta.url), "utf8");
const caseStudy = readFileSync(new URL("./case-study.html", import.meta.url), "utf8");
const csvCiCaseStudy = readFileSync(new URL("./csv-ci-case-study.html", import.meta.url), "utf8");
const caseInput = readFileSync(new URL("./examples/messy-contacts.csv", import.meta.url), "utf8");
const caseOutput = readFileSync(new URL("./examples/normalized-contacts.csv", import.meta.url), "utf8");
const caseReport = readFileSync(new URL("./examples/preflight-errors.csv", import.meta.url), "utf8");
Expand Down Expand Up @@ -206,6 +207,7 @@ test("publishes truthful search metadata for every public page", () => {
"https://softpeanut.github.io/csv-preflight/ko.html",
"https://softpeanut.github.io/csv-preflight/article.html",
"https://softpeanut.github.io/csv-preflight/case-study.html",
"https://softpeanut.github.io/csv-preflight/csv-ci-case-study.html",
"https://softpeanut.github.io/csv-preflight/pro-terms.html",
"https://softpeanut.github.io/csv-preflight/pro-terms-ko.html",
"https://softpeanut.github.io/csv-preflight/shopify-csv-guide-ko.html",
Expand All @@ -230,3 +232,16 @@ test("case study downloads are exact outputs of the public analyzer", () => {
assert.match(caseStudy, /agreed before work starts/);
assert.doesNotMatch(caseStudy, /customer|client result|guaranteed/i);
});

test("public CI case study preserves evidence and sales boundaries", () => {
assert.match(page, /csv-ci-case-study\.html/);
assert.match(githubActionGuide, /csv-ci-case-study\.html/);
assert.match(csvCiCaseStudy, /1,583/);
assert.match(csvCiCaseStudy, /<strong>10<\/strong>exact duplicates reported/);
assert.match(csvCiCaseStudy, /1,573/);
assert.match(csvCiCaseStudy, /CDFER\/jlcpcb-parts-database\/pull\/11/);
assert.match(csvCiCaseStudy, /open contribution, not a customer result or an accepted upstream change/);
assert.match(csvCiCaseStudy, /full multi-gigabyte generation process was not run locally/);
assert.match(csvCiCaseStudy, /template=ci-setup\.yml/);
assert.doesNotMatch(csvCiCaseStudy, /customer success|production-proven|guaranteed|adopted upstream/i);
});
4 changes: 4 additions & 0 deletions sitemap.xml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@
<loc>https://softpeanut.github.io/csv-preflight/case-study.html</loc>
<lastmod>2026-08-27</lastmod>
</url>
<url>
<loc>https://softpeanut.github.io/csv-preflight/csv-ci-case-study.html</loc>
<lastmod>2026-08-27</lastmod>
</url>
<url>
<loc>https://softpeanut.github.io/csv-preflight/pro-terms.html</loc>
<lastmod>2026-08-27</lastmod>
Expand Down
2 changes: 1 addition & 1 deletion validate-csv-github-actions.html
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ <h2>Use schema checks separately</h2>

<p>Structural validation cannot decide whether a price is negative, a customer ID exists, or a date belongs to the required timezone. Add importer-specific schema and business-rule checks after structural parsing. Keep their reports separate so reviewers can distinguish malformed CSV from valid CSV containing invalid domain values.</p>

<p>Start with the <a href="./">browser checker</a>, inspect the <a href="https://github.com/softpeanut/csv-preflight-action">standalone Action source and tests</a>, or use the public <a href="https://github.com/softpeanut/csv-preflight-action/releases/tag/v1.0.0">v1.0.0 Action release</a>. For offline batch and team use, the <a href="pro-terms.html">published Pro terms</a> describe the completed product. If you want this exact one-file workflow configured against a public or sanitized repository, the separate <a href="ci-setup-terms.html">$99 fixed-scope setup terms</a> define every deliverable and exclusion before payment.</p>
<p>Start with the <a href="./">browser checker</a>, inspect the <a href="https://github.com/softpeanut/csv-preflight-action">standalone Action source and tests</a>, review the <a href="csv-ci-case-study.html">public 1,583-row reproduction</a>, or use the public <a href="https://github.com/softpeanut/csv-preflight-action/releases/tag/v1.0.0">v1.0.0 Action release</a>. For offline batch and team use, the <a href="pro-terms.html">published Pro terms</a> describe the completed product. If you want this exact one-file workflow configured against a public or sanitized repository, the separate <a href="ci-setup-terms.html">$99 fixed-scope setup terms</a> define every deliverable and exclusion before payment.</p>
</article>
</main>
</body>
Expand Down
Loading