Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/generate-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,15 @@ jobs:
run: pip install opendocs

- name: Generate documentation
# Inputs are passed through env rather than interpolated into the
# script body: `${{ }}` substitution happens before the shell runs, so
# a crafted input would otherwise execute as shell code.
env:
INPUT_README: ${{ github.event.inputs.readme_path || './README.md' }}
INPUT_FORMATS: ${{ github.event.inputs.formats || 'word,pdf,pptx' }}
run: |
README="${{ github.event.inputs.readme_path || './README.md' }}"
FORMATS="${{ github.event.inputs.formats || 'word,pdf,pptx' }}"
README="$INPUT_README"
FORMATS="$INPUT_FORMATS"

# Convert comma-separated to individual --format flags
for FMT in $(echo "$FORMATS" | tr ',' '\n'); do
Expand Down
26 changes: 26 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,16 @@ on:
- "src/**/*.py"
- "tests/**/*.py"
- "pyproject.toml"
- "README.md"
- ".github/workflows/quality.yml"
pull_request:
branches: [main]
paths:
- "src/**/*.py"
- "tests/**/*.py"
- "pyproject.toml"
- "README.md"
- ".github/workflows/quality.yml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down Expand Up @@ -41,3 +45,25 @@ jobs:

- name: Format check (ruff format)
run: ruff format --check src/ tests/

docs:
name: Documentation lint
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
cache: pip

- name: Install package
run: pip install -e .

# Dogfooding: OpenDocs gates its own README with the linter it ships.
# Link checking is left off so a third-party outage cannot fail the build.
- name: Lint README
run: opendocs lint ./README.md --local
5 changes: 4 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,8 @@ jobs:
- name: Run tests
run: pytest tests/ -v --tb=short
env:
# Prevent any accidental network calls during tests
# Disable diagram rendering so the suite makes no network calls and
# never triggers an `npx @mermaid-js/mermaid-cli` download (which
# pulls Chromium and dominated the run time).
OPENDOCS_MERMAID_BACKEND: none
PYTHONDONTWRITEBYTECODE: "1"
168 changes: 164 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ OpenDocs (by [ioteverythin](https://www.ioteverythin.com/)) takes a GitHub repos
| Knowledge Wiki | Markdown folder | Available |
| LLM Summaries | Stakeholder views | Available |

### What's New in v0.5.0
### What's New in v0.9.0

- **Interactive Knowledge Graph** -- Explorable HTML graph (vis.js) with search, filtering, god-node analysis, community clusters, provenance labels, and surprising connections
- **Graph JSON Export** -- Persistent `graph.json` with nodes, edges, communities, provenance, god nodes, surprising connections, and suggested questions. Query weeks later without re-processing
Expand All @@ -64,7 +64,7 @@ OpenDocs (by [ioteverythin](https://www.ioteverythin.com/)) takes a GitHub repos

### 1. Pipeline (Deterministic + LLM)

The core pipeline parses Markdown/Notebooks and generates all 11 output formats:
The core pipeline parses Markdown/Notebooks and generates all 12 selectable output formats:

- **Basic mode** -- Pure Markdown AST parsing, no LLM required. Fast, free, predictable.
- **LLM mode** -- Uses any supported LLM provider to extract entities, build knowledge graphs, and generate executive summaries + stakeholder views (CTO, Investor, Developer).
Expand Down Expand Up @@ -129,6 +129,166 @@ opendocs generate ./README.md --local --mode llm --provider anthropic
opendocs themes
```

### Incremental Builds

Generated artifacts are cached by content, so re-running the pipeline only
rebuilds what actually changed. This matters most for `opendocs watch`, which
otherwise regenerates every format on every save.

```bash
opendocs generate ./README.md --local # cached automatically
opendocs generate ./README.md --local --no-cache # force a full rebuild
opendocs generate ./README.md --local --cache-dir /tmp/od-cache

opendocs cache # show location, entry count, and size
opendocs cache --clear # empty it
```

The cache key covers the document content, the knowledge graph, the theme, the
template variables, the output format, and the installed opendocs version, so
changing any of them rebuilds rather than reusing. A cache hit reproduces the
stored artifact byte for byte; the one thing it does not refresh is the
generation timestamp, which is what makes identical inputs produce identical
output.

### What Changed Since the Last Release

`opendocs diff` compares two versions of your documentation and reports what
moved — new and removed sections, concepts, and relationships — then renders
it as release notes. Deterministic and offline.

```bash
# Two files
opendocs diff old/README.md new/README.md

# Two git revisions of the same file
opendocs diff v0.8.0 HEAD --git . --path README.md

# Two previously exported graphs
opendocs diff v1_graph.json v2_graph.json

# Draft release notes
opendocs diff v0.8.0 HEAD --git . --format markdown -o RELEASE_NOTES.md

# Machine-readable, including which formats are worth regenerating
opendocs diff old.md new.md --format json

# CI gate: fail if documentation drifted
opendocs diff committed.md generated.md --fail-on-change
```

The summary also reports which output formats are worth regenerating, so a
docs pipeline can rebuild only what the change actually affects.

### Linting Documentation in CI

`opendocs lint` checks documentation quality and exits non-zero when it
regresses, so a pull request can fail on a broken README the same way it fails
on a broken test:

```bash
opendocs lint ./README.md --local # errors fail the build
opendocs lint ./README.md --local --fail-on warning # be stricter
opendocs lint ./README.md --local --fail-on never # report only
opendocs lint ./README.md --local --json # machine-readable
opendocs lint https://github.com/owner/repo --check-links
```

All rules run offline; `--check-links` is the only one that uses the network.

| Rule | Severity | Catches |
|------|----------|---------|
| `no-title` | error | No level-1 heading to use as a title |
| `no-description` | error | No prose at all, only headings and code |
| `placeholder` | error | Unreplaced template text (`your-project-name`, `CHANGEME`) |
| `dead-link` | error | 4xx links (`--check-links`) |
| `missing-installation` / `missing-usage` / `missing-license` | warning | Conventional section absent |
| `thin-content` | warning | Fewer than 50 words of prose |
| `todo-marker` | warning | `TODO` / `FIXME` / `TBD` left in published docs |
| `ragged-table` | warning | Table rows that do not match the header, so it silently renders as plain text |
| `image-no-alt` | warning | Images without alt text |
| `heading-jump` | info | Heading levels skipping (H2 to H4) |
| `duplicate-heading` | info | The same heading repeated at one level |
| `unlabelled-code` | info | Fenced code with no language annotation |

Use it in a workflow:

```yaml
- name: Lint documentation
run: |
pip install opendocs
opendocs lint ./README.md --local
```

Exit codes: `0` clean, `1` findings at or above `--fail-on`, `2` the source
could not be read.

### Querying a Graph Later

`opendocs generate` writes a `graph.json` alongside the documents. `opendocs
query` answers questions about it without re-processing the source — no LLM,
no API key, no network:

```bash
# Overview: top entities and suggested questions
opendocs query output/myproject_graph.json

# Structural queries
opendocs query graph.json --stats
opendocs query graph.json --list-types
opendocs query graph.json --search redis
opendocs query graph.json --entity "PostgreSQL"
opendocs query graph.json --dependents "PostgreSQL" # what would be affected
opendocs query graph.json --dependencies "API Gateway" # what it relies on
opendocs query graph.json --neighbors "Redis"
opendocs query graph.json --path "API Gateway" "S3" # how two things connect
opendocs query graph.json --type database
opendocs query graph.json --community 2
opendocs query graph.json --provenance AMBIGUOUS # review low-confidence finds
opendocs query graph.json --god-nodes

# Plain-English questions, answered from graph structure
opendocs query graph.json "what depends on Redis?"
opendocs query graph.json "how are the API and S3 connected?"
opendocs query graph.json "which databases are used?"
```

Add `--json` to any query for machine-readable output, and `--limit N` to
control how many rows come back.

Exit codes make it scriptable: `0` success, `1` the query matched nothing,
`2` the graph file could not be read.

### Offline / Air-Gapped Use

The interactive knowledge graph normally loads vis.js from a CDN. To produce a
page that renders with no network access at all:

```bash
opendocs generate ./README.md --local --embed-graph-assets
```

The library is downloaded once and cached (`~/.cache/opendocs`), so subsequent
runs need no network. On a fully air-gapped machine, supply it yourself:

```bash
export OPENDOCS_VIS_NETWORK_JS=/path/to/vis-network.min.js
opendocs generate ./README.md --local --embed-graph-assets
```

Diagram rendering can be disabled the same way, which also makes builds fully
offline and much faster:

```bash
export OPENDOCS_MERMAID_BACKEND=none # auto | mmdc | ink | none
```

| Variable | Purpose |
|----------|---------|
| `OPENDOCS_MERMAID_BACKEND` | Diagram backend: `auto` (default), `mmdc`, `ink`, or `none` to disable |
| `OPENDOCS_VIS_NETWORK_JS` | Path to a local vis-network bundle for `--embed-graph-assets` |
| `OPENDOCS_CACHE_DIR` | Override the asset cache location (default `~/.cache/opendocs`) |

### Jupyter Notebook Ingestion

Generate polished reports from research notebooks and data-science projects:
Expand Down Expand Up @@ -315,8 +475,8 @@ pipeline.run(

## Features

- **15 Output Formats** -- Word, PDF, PPTX, Blog Post, Jira Tickets, Changelog, LaTeX Paper, One-Pager PDF, Social Cards, FAQ, Architecture Diagrams, Interactive Graph, Graph JSON, Knowledge Wiki
- **Interactive Knowledge Graph** -- Self-contained HTML visualization (vis.js) with search, legend, community clusters, god nodes, surprising connections, provenance bar, and suggested questions
- **15 Output Formats** -- Word, PDF, PPTX, Blog Post, Jira Tickets, Changelog, LaTeX Paper, One-Pager PDF, Social Cards, FAQ, Architecture Diagrams, Mindmap, Interactive Graph, Graph JSON, Knowledge Wiki
- **Interactive Knowledge Graph** -- Single-file HTML visualization with search, legend, community clusters, god nodes, surprising connections, provenance bar, and suggested questions. Loads vis.js from a CDN by default; pass `--embed-graph-assets` to inline it and get a page that renders fully offline
- **Graph JSON Export** -- Persistent queryable `graph.json` with nodes, edges, communities, provenance labels, god nodes, surprising connections, and suggested questions
- **Knowledge Wiki** -- Wikipedia-style inter-linked Markdown articles (one per community) with navigable index and full entity catalog
- **Community Detection** -- Label propagation algorithm groups entities into clusters by edge density (zero external dependencies)
Expand Down
6 changes: 6 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,12 @@ Issues = "https://github.com/ioteverythin/OpenDocs/issues"
where = ["src"]
exclude = ["opendocs.agents", "opendocs.agents.*"]

# The docagent templates are loaded at runtime by
# opendocs.docagent.templates.get_template(); without this they are dropped
# from the wheel and every lookup raises FileNotFoundError once installed.
[tool.setuptools.package-data]
"opendocs.docagent.templates" = ["*.md"]

[tool.ruff]
line-length = 120
target-version = "py310"
Expand Down
23 changes: 22 additions & 1 deletion src/opendocs/agents/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,27 @@
"""Agentic documentation layer for opendocs.

This package implements the Planner → Executor → Critic architecture
.. warning::

**Status: architectural scaffolding, not a working feature.** This package
is excluded from the built wheel (see ``pyproject.toml``) and is not
reachable from the CLI. Several key methods are stubs that return empty or
pass-through results:

- ``diff.DiffAgent._compute_diff`` returns an empty ``DiffSummary``
regardless of the refs it is given, so the whole diff chain reports no
changes even when a repository has them.
- ``diff.RegenerationAgent._regenerate`` returns the requested format list
without regenerating anything.
- ``diff.ImpactAgent._compute_impact`` is implemented, but indexes entities
by a ``source_file`` / ``file_path`` / ``path`` property that nothing in
the extractor ever sets, so its file-to-entity map is always empty.

Do not expose these through user-facing commands until those are
implemented — they would silently succeed while doing nothing. For working
change detection, see :mod:`opendocs.core.doc_diff`, which powers
``opendocs diff``.

This package sketches the Planner → Executor → Critic architecture
that differentiates opendocs from static doc generators like Mintlify,
GitBook AI, DocuWriter, and repo-diagram tools.

Expand Down
Loading
Loading