Skip to content

Remove duplicate metagenomics withName blocks in modules.config (closes #1173) - #1185

Open
Golchic wants to merge 3 commits into
nf-core:devfrom
Golchic:fix/dedup-metagenomics-modules-config
Open

Golchic wants to merge 3 commits into
nf-core:devfrom
Golchic:fix/dedup-metagenomics-modules-config

Conversation

@Golchic

@Golchic Golchic commented Aug 31, 2026

Copy link
Copy Markdown

Closes #1173. Approach as agreed with @TCLamnidis in the issue.

Changes

An older set of metagenomics withName blocks lived in the AUTHENTICATION
section of conf/modules.config, using the outdated
metagenomics/profiling/… + metagenomics/postprocessing/… publishDir
layout. They duplicated the blocks in the // METAGENOMIC SCREENING
section, which use the canonical <section>/<tool>/{data,stats}/ layout.
Nextflow's last-definition-wins meant the screening-section blocks were
already the effective config, so the old ones were dead weight.

  • Removed the 10 outdated duplicates: BBMAP_BBDUK, MALT_RUN,
    CAT_CAT_MALT, KRAKEN2_KRAKEN2, KRAKENUNIQ_PRELOADEDKRAKENUNIQ,
    METAPHLAN_METAPHLAN, MEGAN_RMA2INFO, AMPS, TAXPASTA_MERGE,
    TAXPASTA_STANDARDISE.
  • Moved the directives that only existed on the old blocks onto the
    screening-section blocks, ordered per docs/development/code_conventions.md
    (tag → ext.args → ext.prefix → publishDir):
    • KRAKEN2_KRAKEN2: tag
    • KRAKENUNIQ_PRELOADEDKRAKENUNIQ: tag + ext.prefix
  • Removed a byte-identical duplicate of
    .*MERGE_LIBRARIES:SAMTOOLS_MERGE_LIBRARIES in the LIBRARY MERGE section.

MALTEXTRACT is left untouched — it has no screening-section counterpart
(not a duplicate), so relocating/retargeting it is out of scope here. Happy
to do that as a follow-up if wanted.

Points for review

  • The moved KRAKEN2_KRAKEN2 tag ({ "${meta.sample_id}|single_end_mode_${meta.single_end}" })
    does not include ${meta.reference}, which code_conventions.md asks for
    on reference-specific processes. I moved it verbatim per "retain and move"
    rather than redesign it (the single_end_mode grouping looks intentional
    for metagenomic screening). Happy to align it to the
    ${meta.reference}|${meta.sample_id}_* format if you'd prefer.
  • No CHANGELOG.md entry — dev doesn't appear to track per-PR changelog
    entries during the DSL2 work (last touched 2023). Glad to add one.
  • docs/output.md unchanged: effective publishDir paths are the same (the
    screening-section blocks that already won are untouched).
  • Didn't run nf-core pipelines lint locally; verified structurally
    (no remaining duplicate withName selectors, braces balanced, directive
    order per code conventions).

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests! — config-only cleanup, covered by existing pipeline tests
  • Make sure your code lints (nf-core pipelines lint). — not run locally, see above
  • Ensure the test suite passes — via CI
  • CHANGELOG.md is updated. — see above

Closes nf-core#1173.

An older set of metagenomics `withName` blocks (filed under the
AUTHENTICATION section, using the outdated `metagenomics/profiling/…`
and `metagenomics/postprocessing/…` publishDir layout) duplicated the
blocks in the `// METAGENOMIC SCREENING` section, which use the
canonical `<section>/<tool>/{data,stats}/` layout. Nextflow's
last-definition-wins meant the screening-section blocks were already the
effective config, so the older ones were dead weight.

- Removed the 10 outdated duplicates: BBMAP_BBDUK, MALT_RUN,
  CAT_CAT_MALT, KRAKEN2_KRAKEN2, KRAKENUNIQ_PRELOADEDKRAKENUNIQ,
  METAPHLAN_METAPHLAN, MEGAN_RMA2INFO, AMPS, TAXPASTA_MERGE,
  TAXPASTA_STANDARDISE.
- Moved the `tag` (KRAKEN2_KRAKEN2, KRAKENUNIQ_PRELOADEDKRAKENUNIQ) and
  `ext.prefix` (KRAKENUNIQ_PRELOADEDKRAKENUNIQ) directives that only
  existed on the old blocks onto the screening-section blocks, ordered
  per docs/development/code_conventions.md (tag, ext.args, ext.prefix,
  publishDir).
- Removed a byte-identical duplicate of
  `.*MERGE_LIBRARIES:SAMTOOLS_MERGE_LIBRARIES` in the LIBRARY MERGE
  section.

MALTEXTRACT is left in place: it is not duplicated (it has no
screening-section counterpart), so relocating/retargeting it is out of
scope here.
@ilight1542
ilight1542 self-requested a review September 25, 2026 08:22

@ilight1542 ilight1542 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks totally fine to me, ran through some test datasets and the output directory structure is A-OK 👍

Comment thread conf/modules.config
]
}

withName: MALT_RUN {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i double checked and multi-ref is not an issue here, since MALT is run in groups of single stranded/double stranded datasets.

totally fine to leave implementation as it is and delete duplicate :)

Comment thread conf/modules.config
]
}

withName: BBMAP_BBDUK {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, remove old version of publish directories structure

@ilight1542
ilight1542 self-requested a review September 25, 2026 09:11
Comment thread conf/modules.config

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will also suggest the mentioned adjustment to get the maltextract postprocessing in line with current publish directory standards

Comment thread conf/modules.config Outdated

@ilight1542 ilight1542 Sep 25, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
path: { "${params.outdir}/metagenomics/maltextract/stats/" },

Comment thread conf/modules.config Outdated

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pattern: 'results/*',

@ilight1542 ilight1542 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor changes for maltextract

Comment thread conf/modules.config Outdated
@@ -1034,44 +962,6 @@ process {

@ilight1542 ilight1542 Sep 25, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

@ilight1542 ilight1542 Sep 25, 2026 •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will save everything for maltextract (still within the results subdirectory, couldnt figure that out), in-line with current standards :)

Per review from @ilight1542: move MALTEXTRACT results under
metagenomics/maltextract/stats/ (matching the <section>/<tool>/{data,stats}/
convention used by the rest of the METAGENOMIC SCREENING blocks), and widen
the publish pattern to results/* to match.
Per @ilight1542: with the pattern widened to results/*, the saveAs rename
to ${meta.id} would prevent saving everything under the results/
subdirectory. Removing it so the full results/ contents are published.
@Golchic

Golchic commented Sep 28, 2026

Copy link
Copy Markdown
Author

Thanks for testing this out and for the detailed suggestions! Fixed all three — publishDir now points to metagenomics/maltextract/stats/, pattern widened to results/*, and dropped the saveAs so everything under results/ actually gets published. Let me know if anything else needs tweaking 🙂

@Golchic
Golchic requested a review from ilight1542 September 28, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants