Conversation
Closes nf-core#1173. An older set of metagenomics `withName` blocks (filed under the AUTHENTICATION section, using the outdated `metagenomics/profiling/…` and `metagenomics/postprocessing/…` publishDir layout) duplicated the blocks in the `// METAGENOMIC SCREENING` section, which use the canonical `<section>/<tool>/{data,stats}/` layout. Nextflow's last-definition-wins meant the screening-section blocks were already the effective config, so the older ones were dead weight. - Removed the 10 outdated duplicates: BBMAP_BBDUK, MALT_RUN, CAT_CAT_MALT, KRAKEN2_KRAKEN2, KRAKENUNIQ_PRELOADEDKRAKENUNIQ, METAPHLAN_METAPHLAN, MEGAN_RMA2INFO, AMPS, TAXPASTA_MERGE, TAXPASTA_STANDARDISE. - Moved the `tag` (KRAKEN2_KRAKEN2, KRAKENUNIQ_PRELOADEDKRAKENUNIQ) and `ext.prefix` (KRAKENUNIQ_PRELOADEDKRAKENUNIQ) directives that only existed on the old blocks onto the screening-section blocks, ordered per docs/development/code_conventions.md (tag, ext.args, ext.prefix, publishDir). - Removed a byte-identical duplicate of `.*MERGE_LIBRARIES:SAMTOOLS_MERGE_LIBRARIES` in the LIBRARY MERGE section. MALTEXTRACT is left in place: it is not duplicated (it has no screening-section counterpart), so relocating/retargeting it is out of scope here.
ilight1542
left a comment
There was a problem hiding this comment.
Looks totally fine to me, ran through some test datasets and the output directory structure is A-OK 👍
| ] | ||
| } | ||
|
|
||
| withName: MALT_RUN { |
There was a problem hiding this comment.
i double checked and multi-ref is not an issue here, since MALT is run in groups of single stranded/double stranded datasets.
totally fine to leave implementation as it is and delete duplicate :)
| ] | ||
| } | ||
|
|
||
| withName: BBMAP_BBDUK { |
There was a problem hiding this comment.
correct, remove old version of publish directories structure
There was a problem hiding this comment.
I will also suggest the mentioned adjustment to get the maltextract postprocessing in line with current publish directory standards
There was a problem hiding this comment.
| path: { "${params.outdir}/metagenomics/maltextract/stats/" }, |
There was a problem hiding this comment.
| pattern: 'results/*', |
| @@ -1034,44 +962,6 @@ process { | |||
There was a problem hiding this comment.
this will save everything for maltextract (still within the results subdirectory, couldnt figure that out), in-line with current standards :)
Per review from @ilight1542: move MALTEXTRACT results under metagenomics/maltextract/stats/ (matching the <section>/<tool>/{data,stats}/ convention used by the rest of the METAGENOMIC SCREENING blocks), and widen the publish pattern to results/* to match.
Per @ilight1542: with the pattern widened to results/*, the saveAs rename to ${meta.id} would prevent saving everything under the results/ subdirectory. Removing it so the full results/ contents are published.
|
Thanks for testing this out and for the detailed suggestions! Fixed all three — publishDir now points to metagenomics/maltextract/stats/, pattern widened to results/*, and dropped the saveAs so everything under results/ actually gets published. Let me know if anything else needs tweaking 🙂 |
Closes #1173. Approach as agreed with @TCLamnidis in the issue.
Changes
An older set of metagenomics
withNameblocks lived in theAUTHENTICATIONsection of
conf/modules.config, using the outdatedmetagenomics/profiling/…+metagenomics/postprocessing/…publishDirlayout. They duplicated the blocks in the
// METAGENOMIC SCREENINGsection, which use the canonical
<section>/<tool>/{data,stats}/layout.Nextflow's last-definition-wins meant the screening-section blocks were
already the effective config, so the old ones were dead weight.
BBMAP_BBDUK,MALT_RUN,CAT_CAT_MALT,KRAKEN2_KRAKEN2,KRAKENUNIQ_PRELOADEDKRAKENUNIQ,METAPHLAN_METAPHLAN,MEGAN_RMA2INFO,AMPS,TAXPASTA_MERGE,TAXPASTA_STANDARDISE.screening-section blocks, ordered per
docs/development/code_conventions.md(
tag→ext.args→ext.prefix→publishDir):KRAKEN2_KRAKEN2:tagKRAKENUNIQ_PRELOADEDKRAKENUNIQ:tag+ext.prefix.*MERGE_LIBRARIES:SAMTOOLS_MERGE_LIBRARIESin theLIBRARY MERGEsection.MALTEXTRACTis left untouched — it has no screening-section counterpart(not a duplicate), so relocating/retargeting it is out of scope here. Happy
to do that as a follow-up if wanted.
Points for review
KRAKEN2_KRAKEN2tag({ "${meta.sample_id}|single_end_mode_${meta.single_end}" })does not include
${meta.reference}, whichcode_conventions.mdasks foron reference-specific processes. I moved it verbatim per "retain and move"
rather than redesign it (the
single_end_modegrouping looks intentionalfor metagenomic screening). Happy to align it to the
${meta.reference}|${meta.sample_id}_*format if you'd prefer.CHANGELOG.mdentry —devdoesn't appear to track per-PR changelogentries during the DSL2 work (last touched 2023). Glad to add one.
docs/output.mdunchanged: effective publishDir paths are the same (thescreening-section blocks that already won are untouched).
nf-core pipelines lintlocally; verified structurally(no remaining duplicate
withNameselectors, braces balanced, directiveorder per code conventions).
PR checklist
nf-core pipelines lint). — not run locally, see aboveCHANGELOG.mdis updated. — see above