Skip to content

Generate pages for nested files, not just top-level ones #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
m-col merged 9 commits into from
Aug 30, 2026
Merged

Generate pages for nested files, not just top-level ones #33

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
cb92015
Expose a genuinely flattened file listing (blobs)
claude Aug 30, 2026
a175f52
Actually generate file pages for nested files, not just top-level ones
claude Aug 30, 2026
3764d23
Drop comment
claude Aug 30, 2026
e2f66dd
Move flattenFiles into Repositories.hs
claude Aug 30, 2026
1f58d3c
Drop tree_recursive - it never actually recursed
claude Aug 30, 2026
6725451
Reinstate tree_recursive, properly recursive this time
claude Aug 30, 2026
6332411
Add a per-tree page category, and rename file -> blob
claude Aug 30, 2026
949e1a8
Fix test.sh and rename special templates to foreach.<scope>.html
claude Aug 30, 2026
24b1cda
Rename TreeFile to TreeEntry and tree_recursive to entries, add trees…
claude Aug 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 30 additions & 17 deletions DOCUMENTATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ new to you, it may be enough to skim through some of the examples in the
otherwise the ginger docs can be very helpful to see what is supported.

Templates are a folder containing a number of ginger template files. There are
4 "scopes", each making available a unique set of variables storing information
5 "scopes", each making available a unique set of variables storing information
about the git repositories. Each template file has access to a single one of
these scopes. The structure of the template folder determines the scopes of the
files contained therein.
Expand All @@ -82,8 +82,9 @@ To illustrate, this is the expected structure:
repo/
inside_this_folder.html
two_names_are_special.html
file.html
commit.html
foreach.blob.html
foreach.commit.html
foreach.tree.html

The top-level folder, here `template`, is that which is specified in the config
file.
Expand All @@ -98,9 +99,14 @@ The special folder "repo" has access to the *repo scope*, which exposes
information pertaining to a single git repository. The template files contained
within this folder are parsed and output once per git repository.

The exceptions to this are the two special template files with the names
"file.html" and "commit.html". These have access to the *file scope* and
*commit scope* respectively, and are parsed and output once per file or commit.
The exceptions to this are the three special template files with the names
"foreach.blob.html", "foreach.commit.html" and "foreach.tree.html" - named to
make clear that, unlike everything else in "repo/", these are parsed and
output once *per* blob, commit or tree found anywhere in the repository, at
any depth, rather than once per repository. These have access to the *blob
scope*, *commit scope* and *tree scope* respectively (a submodule reference
is currently treated as a blob whose content is its target commit hash,
rather than as its own scope).

The resulting folder structure found in `output` will look like this (if
`repos` only contains gitja):
Expand All @@ -112,14 +118,17 @@ The resulting folder structure found in `output` will look like this (if
gitja/
inside_this_folder.html
two_names_are_special.html
file/
blob/
LICENSE.html
Makefile.html
...
commit/
0a18f38bb5c398bd192a6268281fc6abefaedd63.html
0a7601059956d9c4d395f5d08e8cf48a515d080f.html
...
tree/
src.html
...
...

### Static files
Expand All @@ -145,16 +154,20 @@ The variables available within each scope are listed here for reference:
| | name | The repository name, taken from its folder name. |
| | description | The repository's description (see below). |
| | commits | A list of the repository's commits. |
| | tree | A list of the top-level folder's contents. |
| | tree\_recursive | A list of *all* of the repository's contents. |
| | tree | A list of the repository root tree's entries. |
| | entries | A flat list of *every* entry (blob or tree) at any depth in the repository, like `git ls-tree -r -t`. |
| | blobs | A flat list of *every* blob (file) in the repository. |
| | trees | A flat list of *every* tree (directory) at any depth in the repository. |
| | tags | A list of the refs corresponding to tags. |
| | branches | A list of the refs corresponding to branches. |
| | readme | The repository's readme file, if it has one. |
| | license | The repository's license file, if it has one. |
| File | | *In addition to the variables from the Repo scope...* |
| | file | A single file. |
| Blob | | *In addition to the variables from the Repo scope...* |
| | blob | A single blob (file). |
| Commit | | *In addition to the variables from the Repo scope...* |
| | commit | A single commit. |
| Tree | | *In addition to the variables from the Repo scope...* |
| | tree | A single tree (directory) - shadows the repo-scope `tree`, since a tree page is itself scoped to one tree. |

As in [Jinja](https://jinja.palletsprojects.com), a list can be accessed with
indexing, and attributes can be accessed using a dot notation. For example, a
Expand All @@ -170,17 +183,17 @@ Here is the reference of attributes available on the variables that have them:
| | description | The repository's description (see below). |
| | head | The current git commit. |
| | updated | The time when the current commit was committed. |
| file | path | The path the file relative to the repository root. |
| | name | The name of the file. |
| | href | The name of the HTML file for this file. |
| | contents | The file's contents. |
| blob/tree | path | The path relative to the repository root. |
| | name | The name of the blob or tree. |
| | href | The name of the HTML file for this blob or tree. |
| | contents | The blob's contents (n/a for a tree). |
| | mode | Directory, Plain, Executable, Symlink or Submodule. |
| | mode\_octal | Mode in octal form e.g. "00644" for plain files. |
| | mode\_symbolic | Mode in symbolic form e.g. ""-rw-r--r--" for plain files.|
| | is\_directory | A boolean, useful for ginger conditionals. |
| | is\_binary | A boolean, tells you if the contents can be rendered. |
| | tree | A list of a directory's direct contents. |
| | tree\_recursive | A list of *all* of a directory's contents. |
| | tree | This tree's direct contents (n/a for a blob). |
| | entries | A flat list of *every* entry (blob or tree) at any depth under this tree (n/a for a blob). |
| ref | name | The tag or branch name. |
| | commit | The commit pointed to by the tag or branch. |
| commit | id | The SHA of the given commit. |
Expand Down
15 changes: 9 additions & 6 deletions src/Env.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ data Env = Env
{ envConfig :: Config
, envIndexTemplates :: [Template]
, envCommitTemplate :: Maybe Template
, envFileTemplate :: Maybe Template
, envBlobTemplate :: Maybe Template
, envTreeTemplate :: Maybe Template
, envRepoTemplates :: [Template]
, envOutput :: Path Abs Dir
, envRepos :: [Path Abs Dir]
Expand Down Expand Up @@ -104,16 +105,17 @@ loadEnv quiet force config = do

-- Load files from template directory
indexT <- collectTemplates files
commitT <- findTemplate "commit.html" filesRepo
fileT <- findTemplate "file.html" filesRepo
commitT <- findTemplate "foreach.commit.html" filesRepo
blobT <- findTemplate "foreach.blob.html" filesRepo
treeT <- findTemplate "foreach.tree.html" filesRepo
repoT <-
collectTemplates
. filter (flip notElem ["commit.html", "file.html"] . toFilePath . filename)
. filter (flip notElem ["foreach.commit.html", "foreach.blob.html", "foreach.tree.html"] . toFilePath . filename)
$ filesRepo

-- Exit early if we didn't find any templates
when
( all null [indexT, repoT] && all isNothing [commitT, fileT]
( all null [indexT, repoT] && all isNothing [commitT, blobT, treeT]
)
$ die "No templates were found."

Expand All @@ -123,7 +125,8 @@ loadEnv quiet force config = do
{ envConfig = config
, envIndexTemplates = indexT
, envCommitTemplate = commitT
, envFileTemplate = fileT
, envBlobTemplate = blobT
, envTreeTemplate = treeT
, envRepoTemplates = repoT
, envOutput = output
, envRepos = repos
Expand Down
64 changes: 45 additions & 19 deletions src/Repositories.hs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,9 +132,11 @@
withRunInIO \runInIO -> do
-- Create the destination folders --
commitDir <- (directory </>) <$> parseRelDir "commit"
fileDir <- (directory </>) <$> parseRelDir "file"
blobDir <- (directory </>) <$> parseRelDir "blob"
treeDir <- (directory </>) <$> parseRelDir "tree"
ensureDir commitDir
ensureDir fileDir
ensureDir blobDir
ensureDir treeDir

-- Check which commits are new since the last run --
newCommits <- getUpdates commitDir commits
Expand All @@ -149,12 +151,21 @@
whenJust (envCommitTemplate env) \commitT -> do
mapM_ (gen force commitT "commit" commitDir commitHref) newCommits

whenJust (envFileTemplate env) \fileT -> do
whenJust (envBlobTemplate env) \blobT -> do
let allBlobs = concatMap flattenFiles tree
if force
then mapM_ (gen True fileT "file" fileDir fileHref) tree
then mapM_ (gen True blobT "blob" blobDir blobHref) allBlobs
else
let updatedFiles = getUpdatedFiles tree newCommits
in mapM_ (gen True fileT "file" fileDir fileHref) updatedFiles
let updatedBlobs = getUpdatedFiles allBlobs newCommits
in mapM_ (gen True blobT "blob" blobDir blobHref) updatedBlobs

whenJust (envTreeTemplate env) \treeT -> do
-- A tree's own path never appears in a commit's diff (only
-- the blobs within it do), so unlike blobs, staleness can't
-- be judged by newCommits/getUpdatedFiles - always
-- regenerate every tree page.
let allTrees = concatMap flattenTrees tree
mapM_ (gen True treeT "tree" treeDir treeHref) allTrees

-- Copy any static files/folders into the output directory --
envRepoCopyStatics env directory
Expand All @@ -174,7 +185,7 @@
Path Rel Dir ->
T.Text ->
[Commit] ->
[TreeFile] ->
[TreeEntry] ->
HashMap.HashMap T.Text (GVal RunRepo)
package env repos name description commits tree =
HashMap.fromList
Expand All @@ -183,16 +194,18 @@
, ("name", toGVal . T.pack . init . toFilePath $ name)
, ("description", toGVal description)
, ("commits", toGVal commits)
, ("tree", toGVal . filter (notElem FP.pathSeparator . T.unpack . treeFilePath) $ tree)
, ("tree_recursive", toGVal tree)
, ("tree", toGVal tree)
, ("entries", toGVal . concatMap flattenTree $ tree)
, ("blobs", toGVal . concatMap flattenFiles $ tree)
, ("trees", toGVal . concatMap flattenTrees $ tree)
, ("readme", toGVal . findFile "readme" $ tree)
, ("license", toGVal . findFile "license" $ tree)
]
where
-- Find a file in the tree starting with the specified prefix. The prefix is looked
-- for on the full path, so will only find files in the top level directory.
findFile :: T.Text -> [TreeFile] -> Maybe TreeFile
findFile prefix = find (T.isPrefixOf prefix . T.toLower . treeFilePath)
findFile :: T.Text -> [TreeEntry] -> Maybe TreeEntry
findFile prefix = find (T.isPrefixOf prefix . T.toLower . treeEntryPath)

{-
Collect commit history up to a head.
Expand Down Expand Up @@ -275,7 +288,7 @@
bs <- curry B.packCStringLen content (fromIntegral contentLen)
let bs' = B.cons (fromIntegral lineOrigin) bs
(cur : _, rest) <- splitAt 1 <$> readIORef ioref
let (curHunk : _, restHunks) = splitAt 1 . diffHunks $ cur

Check warning on line 291 in src/Repositories.hs

View workflow job for this annotation

GitHub Actions / build-and-test 

Pattern match(es) are non-exhaustive
let updated =
cur
{ diffHunks =
Expand All @@ -298,17 +311,17 @@
Collect tree information for the given commit. Recurses on directories to list their
contents.
-}
getTree :: Git.CommitOid LgRepo -> ReaderT LgRepo IO [TreeFile]
getTree :: Git.CommitOid LgRepo -> ReaderT LgRepo IO [TreeEntry]
getTree = getTree' "" 0 . Git.commitTree <=< Git.lookupCommit
where
getTree' :: Git.TreeFilePath -> Int -> Git.TreeOid LgRepo -> ReaderT LgRepo IO [TreeFile]
getTree' :: Git.TreeFilePath -> Int -> Git.TreeOid LgRepo -> ReaderT LgRepo IO [TreeEntry]
getTree' parent count toid = do
one <- Git.lookupTree toid
entries <- Git.listTreeEntries one
let entries' = fmap (prependParent parent) entries
contents <- mapM (\x -> getEntryContents x (count + 1)) entries'
modes <- mapM (getEntryModes . snd) entries'
return $ zipWith3 TreeFile (fmap treePaths entries') contents modes
return $ zipWith3 TreeEntry (fmap treePaths entries') contents modes

prependParent ::
Git.TreeFilePath ->
Expand All @@ -317,7 +330,7 @@
prependParent "" pathentry = pathentry
prependParent parent (path, entry) = (mconcat [parent, "/", path], entry)

getEntryContents :: (Git.TreeFilePath, Git.TreeEntry LgRepo) -> Int -> ReaderT LgRepo IO TreeFileContents
getEntryContents :: (Git.TreeFilePath, Git.TreeEntry LgRepo) -> Int -> ReaderT LgRepo IO TreeEntryContents
getEntryContents (_, Git.BlobEntry oid _) _ = getBlobContents oid
getEntryContents (path, Git.TreeEntry oid) count = FolderContents <$> getTree' path count oid
getEntryContents (_, Git.CommitEntry oid) _ = return . FileContents . B.fromString . show . untag $ oid
Expand Down Expand Up @@ -372,10 +385,20 @@
(return [])
((x :) <$> go xs)

getUpdatedFiles :: [TreeFile] -> [Commit] -> [TreeFile]
flattenFiles :: TreeEntry -> [TreeEntry]
flattenFiles treeentry = case treeEntryContents treeentry of
FolderContents files -> concatMap flattenFiles files
_ -> [treeentry]

flattenTrees :: TreeEntry -> [TreeEntry]
flattenTrees treeentry = case treeEntryContents treeentry of
FolderContents files -> treeentry : concatMap flattenTrees files
_ -> []

getUpdatedFiles :: [TreeEntry] -> [Commit] -> [TreeEntry]
getUpdatedFiles [] _ = []
getUpdatedFiles _ [] = []
getUpdatedFiles files commits = filter ((`elem` updated) . treeFilePath) files
getUpdatedFiles files commits = filter ((`elem` updated) . treeEntryPath) files
where
updated :: [T.Text]
updated = fmap (bsToText . diffNewFile) . concatMap commitDiffs $ commits
Expand Down Expand Up @@ -425,8 +448,11 @@
commitHref :: Commit -> FilePath
commitHref = (++ ".html") . commitHash

fileHref :: TreeFile -> FilePath
fileHref = T.unpack . treePathToHref
blobHref :: TreeEntry -> FilePath
blobHref = T.unpack . treePathToHref

treeHref :: TreeEntry -> FilePath
treeHref = T.unpack . treePathToHref

{-
With a dictionary of preloaded values and a function to access additional data, create a
Expand Down
Loading
Loading