Skip to content
iLiviusPublic

About

Unified short-read workflow for multi-marker amplicon (16S, ITS, 18S, gyrB, rpoB, COI) and shotgun taxonomic profiling. Snakemake, conda-based, reproducible.

Topics

Resources

Code of conduct

Contributing

Stars

8 stars

Watchers

1 watching

Forks

Repository files navigation

MetaFlux

______  ___    _____       _______________
___   |/  /______  /______ ___  ____/__  /___  _____  __
__  /|_/ /_  _ \  __/  __ `/_  /_   __  /_  / / /_  |/_/
_  /  / / /  __/ /_ / /_/ /_  __/   _  / / /_/ /__>  <
/_/  /_/  \___/\__/ \__,_/ /_/      /_/  \__,_/ /_/|_|

v2.6.1

Unified short-read multi-marker amplicon and shotgun taxonomic profiling workflow.

Snakemake License: MIT DOI CI

Metro overview of the MetaFlux workflow: six coloured amplicon marker lines and two shotgun profiler lines start at the config and meet at read tracking and MultiQC. COI is classified with SINTAX against MIDORI2 and then goes through two screens that can be switched off. Dashed branches show the optional 16S phylogeny and per-taxon reads; hollow stops are steps you can switch on or off in the config.

The detailed map, step by step, is on the documentation home page.

📖 Documentation

Full documentation: iLivius.github.io/MetaFlux

Installation environments, reference databases, Kraken2 index
Quick start first run, start to finish
Amplicon mode the DADA2 pipeline, step by step
Markers 16S · ITS · 18S · gyrB · rpoB · COI, one page each
Phylogeny (16S) optional ASV tree, exported unrooted, for Faith's PD and UniFrac in R
Shotgun mode Kraken2 + Bracken or sylph profiling
Configuration reference every key, default and trade-off
Output files what each run writes, and what reads it
Troubleshooting the failures worth knowing about in advance
Test datasets the public data each marker and shotgun mode are tested on, and the benchmark data

This README is deliberately short: it covers what MetaFlux is and how to get a first run going. Everything else — parameter reference, per-marker guidance, output semantics and the methodological caveats — lives on the documentation site.

Synopsis

MetaFlux is a Snakemake workflow that takes raw paired-end short reads through to a taxonomically annotated abundance table. It runs in one of two modes, selected by a single config key:

  • amplicon — a DADA2-based pipeline for multi-marker metabarcoding (16S rRNA, ITS, 18S rRNA, gyrB, rpoB, COI): PhiX removal, primer trimming, exact ASV inference, optional marker-region extraction, length filtering, and taxonomic assignment against SILVA (16S), UNITE (ITS), PR2 (18S), DD7RZ8 (gyrB), FROGS (rpoB), or MIDORI2 (COI).
  • shotgun — a Kraken2 + Bracken pipeline (or sylph, chosen with shotgun.profiler) for shotgun metagenomic taxonomic profiling: decontamination, quality trimming, read classification, abundance re-estimation, and OTU-table construction.

Both modes run from the same command and the same config file, and produce a single MultiQC report.

                 ┌──────────────┬───────────────┐
   config: mode  │   amplicon   │    shotgun    │
                 └──────┬───────┴───────┬───────┘
        raw paired-end short reads (R1/R2)
                        │               │
        bowtie2 PhiX*   │               │  BBDuk PhiX*  +  BBMap host*
        Cutadapt primer │               │  fastp trim
        DADA2 ASVs      │               │  Kraken2 classify
        Metaxa2 / ITSx* │               │  Bracken abundance
        length filter   │               │  OTU table from Bracken
        multi-marker    │               │  KrakenTools per-taxon reads*
        taxonomy        │               │  or sylph profile + sylph table†
                        ▼               ▼
              ASV table + taxonomy   OTU table + reports
                        └───────┬───────┘
                     read tracking + MultiQC

   * optional steps, toggled in config
   † shotgun.profiler: sylph replaces Kraken2 onwards; both runs the two side by side

What the workflow adds over running these tools by hand: truncLen chosen from the run's own quality profile under a merge-overlap constraint; expected amplicon length measured from the reference database with the supplied primers rather than hard-coded; orientation-aware primer trimming; per-rule CPU and memory in one place; and one reproducible entry point for both modes. See Rationale.

MetaFlux belongs to the BioFlux family of pipelines.

Quick Start

# 1. Clone
git clone https://github.com/iLivius/MetaFlux.git
cd MetaFlux

# 2. Create the launcher environment (Snakemake + conda support)
conda create -n snakemake -c conda-forge -c bioconda snakemake
conda activate snakemake

# 3. Edit config/config.yaml — set `mode`, input/output paths, and references
#    (amplicon: primer FASTAs; shotgun: a Kraken2 database, or sylph databases)

# 4. Dry-run to check the plan, then execute
snakemake --sdm conda --cores 16 --configfile config/config.yaml --dry-run
snakemake --sdm conda --cores 16 --configfile config/config.yaml

Only Snakemake (≥ 9) is installed by hand. --sdm conda (software-deployment-method) tells Snakemake to build each rule's own conda environment on first run, so every other tool is provisioned automatically.

Always dry-run first: it validates the config, resolves sample names and reference paths, and prints the plan without executing anything.

The shipped config/config.yaml is a template whose /path/to/... entries are placeholders. A common pattern is to copy it to config/config.local.yaml (git-ignored) and run with --configfile config/config.local.yaml, leaving the template untouched. Full details in the configuration reference.

Reference databases

Amplicon reference databases download automatically the first time they are needed and are cached under refdb/. Shotgun mode needs, supplied by hand, a Kraken2 index (shotgun.profiler: kraken2, the default), sylph databases with their lineage tables (sylph), or both (both).

Database Mode Provisioning
PhiX both auto-fetched (NCBI) for amplicon; BBDuk's bundled copy for shotgun
SILVA 144 (toGenus + species + SINTAX; 138.2 selectable) amplicon (16S) auto-fetched (md5-checked) and converted on first run
UNITE (general + UCHIME + SINTAX) amplicon (ITS) auto-fetched on first run
PR2 (DADA2 + UTAX) + SILVA-Euk amplicon (18S) auto-fetched on first run
DD7RZ8 (DADA2 trainset) amplicon (gyrB) auto-fetched on first run
FROGS RefSeq (converted to DADA2) amplicon (rpoB) auto-fetched and converted on first run
MIDORI2 (SINTAX) + bacterial COI homologues (GTDB R232) amplicon (COI) auto-fetched (checksummed) and converted on first run
Kraken2 / Bracken index shotgun (kraken2 or both) user-supplied
sylph databases + sylph-tax lineage tables shotgun (sylph or both) user-supplied
Host genome(s) shotgun (optional) user-supplied, path or URL

Choosing and downloading a Kraken2 index, and the memory it needs, are covered in reference databases; the sylph databases and their lineage tables in sylph.

Citation

Publications that use MetaFlux should cite the Zenodo record:

Antonielli, L. (2026). MetaFlux: a unified short-read multi-marker amplicon and shotgun taxonomic profiling workflow. Zenodo. https://doi.org/10.5281/zenodo.22876451

That DOI always resolves to the newest release. To cite the exact version you ran, use its own DOI instead — v2.6.1 is 10.5281/zenodo.23898251 — and give the version number in the reference, because results can depend on it. CITATION.cff carries the same information in a form GitHub shows under "Cite this repository".

MetaFlux is a wrapper around published tools and reference databases, and those do the actual work — cite them too. The full reference list is on the citation page.

Contributing, support and changes

  • CHANGELOG.md says what changed in each release, what runs again in an output folder made with the previous release, and what to edit in an old config.
  • SUPPORT.md says where to ask a question or report a problem; CONTRIBUTING.md how to report a bug, propose a change, run the checks and write code and comments the way this repository does. CODE_OF_CONDUCT.md applies to every interaction here.
  • Every push runs the checks of .github/workflows/ci.yml: the unit tests under tests/, a dry run of every marker, method, SILVA release and shotgun profiler on the tiny data under .test/ (its README says what the stand-in databases are), and a strict build of the documentation.

Acknowledgements

Developed at the AIT Austrian Institute of Technology. MetaFlux consolidates and modernises methods refined across many amplicon and metagenomics collaborations, and is part of the BioFlux family of workflows.

MetaFlux was developed in the context of MICROBE — MICRObiome Biobanking (RI) Enabler, a Horizon Europe research-infrastructure project coordinated by AIT (2023–2027) that develops the methods, and the routes of access, for preserving microbiomes with their composition and function intact. This project has received funding from the European Union's Horizon Europe research and innovation programme under grant agreement No. 101094353.

A large part of MetaFlux — merging the amplicon and shotgun workflows into one, the marker packs, two full code audits, this documentation, and the phylogeny module with its validation — was built in working sessions with Claude Code. Anthropic accepted MetaFlux into their Open Source Program and provided Claude Max for it, generously and with no strings attached; that support is what made the pace and the scope of this work possible, and it is acknowledged here with real gratitude. The deal on this side has been that every line is still read, understood and tested by a human before it goes in — which is why the code and these pages read the way they do.

License

MetaFlux is released under the MIT License. Third-party tools invoked by the workflow are distributed under their own licenses (a mix of MIT, BSD, and GPL/LGPL); see each tool's repository.

About

Unified short-read workflow for multi-marker amplicon (16S, ITS, 18S, gyrB, rpoB, COI) and shotgun taxonomic profiling. Snakemake, conda-based, reproducible.

Topics

Resources

Code of conduct

Contributing

Stars

8 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages