demux_plus
pipes/WDL/workflows/demux_plus.wdl

WORKFLOW demux_plus

File Path pipes/WDL/workflows/demux_plus.wdl
WDL Version 1.0
Type workflow

Imports

Namespace Path
demux ../tasks/tasks_demux.wdl
metagenomics ../tasks/tasks_metagenomics.wdl
read_utils ../tasks/tasks_read_utils.wdl
taxon_filter ../tasks/tasks_taxon_filter.wdl
assembly ../tasks/tasks_assembly.wdl
reports ../tasks/tasks_reports.wdl

Workflow: demux_plus

Picard-based demultiplexing and basecalling from a tarball of a raw BCL directory, followed by basic metagenomics and QC metrics. Intended for automatic triggering post upload on DNAnexus.

Author: Broad Viral Genomics
viral-ngs@broadinstitute.org

Inputs

Name Type Description Default
spikein_db File - -
trim_clip_db File - -
bmtaggerDbs Array[File]? - -
blastDbs Array[File]? - -
bwaDbs Array[File]? - -
instrument_model_user_specified String? - -
flowcell_tgz File - -
samplesheet File? - -
runinfo File? - -
sequencingCenter String? - -
barcode_columns_to_rev_comp Array[String]? - -
flowcell String? - -
minMismatchDelta Int? - -
maxNoCalls Int? - -
readStructure String? - -
minimumQuality Int? - -
threads Int? - -
runStartDate String? - -
maxRecordsInRam Int? - -
numberOfNegativeControls Int? - -
tileLimit Int? - -
firstTile Int? - -
machine_mem_gb Int? - -
machine_mem_gb Int? - -
query_chunk_size Int? - -
spades_min_contig_len Int? - -
spades_options String? - -
machine_mem_gb Int? - -
title String? - -
comment String? - -
template String? - -
tag String? - -
ignore_analysis_files String? - -
ignore_sample_names String? - -
sample_names File? - -
exclude_modules Array[String]? - -
module_to_use Array[String]? - -
output_data_format String? - -
config File? - -
config_yaml String? - -
title String? - -
comment String? - -
template String? - -
tag String? - -
ignore_analysis_files String? - -
ignore_sample_names String? - -
sample_names File? - -
exclude_modules Array[String]? - -
module_to_use Array[String]? - -
output_data_format String? - -
config File? - -
config_yaml String? - -
krakenuniq_db_tar_lz4 File - -
krona_taxonomy_db_tgz File - -
60 optional inputs with default values

Outputs

Name Type Expression
raw_reads_unaligned_bams Array[File] illumina_demux.raw_reads_unaligned_bams
cleaned_reads_unaligned_bams Array[File] deplete.cleaned_bam
contigs_fastas Array[File] spades.contigs_fasta
read_counts_raw Array[Int] deplete.depletion_read_count_pre
read_counts_depleted Array[Int] deplete.depletion_read_count_post
read_counts_prespades_subsample Array[Int] spades.subsample_read_count
demux_metrics File illumina_demux.metrics
demux_commonBarcodes File illumina_demux.commonBarcodes
demux_outlierBarcodes File illumina_demux.outlierBarcodes
instrument_model_inferred String select_first(flatten([[instrument_model_user_specified], [illumina_demux.run_info['sequencer_model']]]))
multiqc_report_raw File multiqc_raw.multiqc_report
multiqc_report_cleaned File multiqc_cleaned.multiqc_report
spikein_counts File spike_summary.count_summary
metagenomics_krona File krakenuniq.krona_report_merged_html
metagenomics_summary File metag_summary_report.krakenuniq_aggregate_taxlevel_summary
krakenuniq_classified_reads Array[File] krakenuniq.krakenuniq_classified_reads
krakenuniq_summary_reports Array[File] krakenuniq.krakenuniq_summary_reports
krakenuniq_krona_by_sample Array[File] krakenuniq.krona_report_html
demux_viral_core_version String illumina_demux.viralngs_version
krakenuniq_viral_classify_version String krakenuniq.viralngs_version
deplete_viral_classify_version String deplete.viralngs_version[0]
spades_viral_assemble_version String spades.viralngs_version[0]

Calls

This workflow calls the following tasks or subworkflows:

CALL TASKS illumina_demux ↗

No explicit input mappings

CALL TASKS spikein ↗ → align_and_count

Input Mappings (2)
Input Value
reads_bam raw_reads
ref_db spikein_db

CALL TASKS deplete ↗ → deplete_taxa

Input Mappings (4)
Input Value
raw_reads_unmapped_bam raw_reads
bmtaggerDbs bmtaggerDbs
blastDbs blastDbs
bwaDbs bwaDbs

CALL TASKS spades ↗ → assemble

Input Mappings (3)
Input Value
reads_unmapped_bam deplete.cleaned_bam
trim_clip_db trim_clip_db
always_succeed true

CALL TASKS multiqc_raw ↗ → MultiQC

Input Mappings (2)
Input Value
input_files illumina_demux.raw_reads_fastqc_zip
file_name "multiqc-raw.html"

CALL TASKS multiqc_cleaned ↗ → MultiQC

Input Mappings (2)
Input Value
input_files deplete.cleaned_fastqc_zip
file_name "multiqc-cleaned.html"

CALL TASKS krakenuniq ↗

Input Mappings (1)
Input Value
reads_unmapped_bam illumina_demux.raw_reads_unaligned_bams

CALL TASKS spike_summary ↗ → align_and_count_summary

Input Mappings (1)
Input Value
counts_txt spikein.report

CALL TASKS metag_summary_report ↗ → aggregate_metagenomics_reports

Input Mappings (1)
Input Value
kraken_summary_reports krakenuniq.krakenuniq_summary_reports

Images

Container images used by tasks in this workflow:

🐳 Parameterized Image
⚙️ Parameterized

Configured via input:
docker

Used by 1 task:
  • illumina_demux
🐳 ~{docker}

~{docker}

Used by 5 tasks:
  • multiqc_raw
  • multiqc_cleaned
  • spike_summary
  • metag_summary_report
  • spikein
🐳 Parameterized Image
⚙️ Parameterized

Configured via input:
docker

Used by 1 task:
  • krakenuniq
🐳 Parameterized Image
⚙️ Parameterized

Configured via input:
docker

Used by 1 task:
  • deplete
🐳 Parameterized Image
⚙️ Parameterized

Configured via input:
docker

Used by 1 task:
  • spades
← Back to Index

demux_plus - Workflow Graph

🖱️ Scroll to zoom • Drag to pan • Double-click to reset • ESC to close

demux_plus - WDL Source Code

version 1.0

import "../tasks/tasks_demux.wdl" as demux
import "../tasks/tasks_metagenomics.wdl" as metagenomics
import "../tasks/tasks_read_utils.wdl" as read_utils
import "../tasks/tasks_taxon_filter.wdl" as taxon_filter
import "../tasks/tasks_assembly.wdl" as assembly
import "../tasks/tasks_reports.wdl" as reports

workflow demux_plus {
    meta {
        description: "Picard-based demultiplexing and basecalling from a tarball of a raw BCL directory, followed by basic metagenomics and QC metrics. Intended for automatic triggering post upload on DNAnexus."
        author: "Broad Viral Genomics"
        email:  "viral-ngs@broadinstitute.org"
        allowNestedInputs: true
    }

    input {
        File spikein_db
        File trim_clip_db
        Array[File]? bmtaggerDbs  # .tar.gz, .tgz, .tar.bz2, .tar.lz4, .fasta, or .fasta.gz
        Array[File]? blastDbs  # .tar.gz, .tgz, .tar.bz2, .tar.lz4, .fasta, or .fasta.gz
        Array[File]? bwaDbs
        String?      instrument_model_user_specified
    }

    call demux.illumina_demux as illumina_demux

    scatter(raw_reads in illumina_demux.raw_reads_unaligned_bams) {
        call reports.align_and_count as spikein {
            input:
                reads_bam = raw_reads,
                ref_db    = spikein_db
        }
        call taxon_filter.deplete_taxa as deplete {
            input:
                raw_reads_unmapped_bam = raw_reads,
                bmtaggerDbs            = bmtaggerDbs,
                blastDbs               = blastDbs,
                bwaDbs                 = bwaDbs
        }
        call assembly.assemble as spades {
            input:
                reads_unmapped_bam = deplete.cleaned_bam,
                trim_clip_db       = trim_clip_db,
                always_succeed     = true
        }
    }

    call reports.MultiQC as multiqc_raw {
        input:
            input_files = illumina_demux.raw_reads_fastqc_zip,
            file_name   = "multiqc-raw.html"
    }

    call reports.MultiQC as multiqc_cleaned {
        input:
            input_files = deplete.cleaned_fastqc_zip,
            file_name   = "multiqc-cleaned.html"
    }

    call metagenomics.krakenuniq as krakenuniq {
        input:
            reads_unmapped_bam = illumina_demux.raw_reads_unaligned_bams
    }

    call reports.align_and_count_summary as spike_summary {
        input:
            counts_txt = spikein.report
    }

    call reports.aggregate_metagenomics_reports as metag_summary_report {
        input:
            kraken_summary_reports = krakenuniq.krakenuniq_summary_reports
    }

    output {
        Array[File] raw_reads_unaligned_bams          = illumina_demux.raw_reads_unaligned_bams
        Array[File] cleaned_reads_unaligned_bams      = deplete.cleaned_bam
        Array[File] contigs_fastas                    = spades.contigs_fasta
        
        Array[Int]  read_counts_raw                   = deplete.depletion_read_count_pre
        Array[Int]  read_counts_depleted              = deplete.depletion_read_count_post
        Array[Int]  read_counts_prespades_subsample   = spades.subsample_read_count
        
        File        demux_metrics                     = illumina_demux.metrics
        File        demux_commonBarcodes              = illumina_demux.commonBarcodes
        File        demux_outlierBarcodes             = illumina_demux.outlierBarcodes
        String      instrument_model_inferred         = select_first(flatten([[instrument_model_user_specified],[illumina_demux.run_info['sequencer_model']]]))
        
        File        multiqc_report_raw                = multiqc_raw.multiqc_report
        File        multiqc_report_cleaned            = multiqc_cleaned.multiqc_report
        File        spikein_counts                    = spike_summary.count_summary
        File        metagenomics_krona                = krakenuniq.krona_report_merged_html
        File        metagenomics_summary              = metag_summary_report.krakenuniq_aggregate_taxlevel_summary
        Array[File] krakenuniq_classified_reads       = krakenuniq.krakenuniq_classified_reads
        Array[File] krakenuniq_summary_reports        = krakenuniq.krakenuniq_summary_reports
        Array[File] krakenuniq_krona_by_sample        = krakenuniq.krona_report_html
        
        String      demux_viral_core_version          = illumina_demux.viralngs_version
        String      krakenuniq_viral_classify_version = krakenuniq.viralngs_version
        String      deplete_viral_classify_version    = deplete.viralngs_version[0]
        String      spades_viral_assemble_version     = spades.viralngs_version[0]
    }
}