Bactopia Tool - merlin

MinmER assisted species-specific bactopia tool seLectIoN, or Merlin, uses distances based on the RefSeq sketch downloaded by bactopia datasets to automatically run species-specific tools.

Currently Merlin knows 16 spells for which cover the following:

Genus/Species	Tools
Escherichia	ECTyper
Haemophilus	hicap, HpsuisSero
Klebsiella	Kleborate
Legionella	legsta
Listeria	LisSero
Mycobacterium	TBProfiler
Neisseria	meningotype, ngmaster
Salmonella	SeqSero2, SISTR
Staphylococcus	AgrVATE, spaTyper, staphopia-sccmec
Streptococcus	emmtyper, SsuisSero

Merlin is avialable as an independent Bactopia Tool, or in the Bactopia with the --ask_merlin parameter. Even better, if you want to force Merlin to execute all species-specific tools (no matter the distance), you can use --full_merlin. Then all the spells will be unleashed!

Example Usage

bactopia --wf merlin \
  --bactopia /path/to/your/bactopia/results \ 
  --include includes.txt  

Output Overview

Below is the default output structure for the merlin tool. Where possible the file descriptions below were modified from a tools description.

merlin/
├── <SAMPLE_NAME>
│   ├── agrvate
│   │   ├── <SAMPLE_NAME>-agr_gp.tab
│   │   ├── <SAMPLE_NAME>-blastn_log.txt
│   │   └── <SAMPLE_NAME>-summary.tab
│   ├── ectyper
│   │   ├── <SAMPLE_NAME>.tsv
│   │   └── blast_output_alleles.txt
│   ├── emmtyper
│   │   └── <SAMPLE_NAME>.tsv
│   ├── hicap
│   │   └── <SAMPLE_NAME>.tsv
│   ├── hpsuissero
│   │   └── <SAMPLE_NAME>_serotyping_res.tsv
│   ├── kleborate
│   │   └── <SAMPLE_NAME>.results.txt
│   ├── legsta
│   │   └── <SAMPLE_NAME>.tsv
│   ├── lissero
│   │   └── <SAMPLE_NAME>.tsv
│   ├── logs
│   │   └── <BACTOPIA_TOOL>
│   │       ├── nf-<BACTOPIA_TOOL>.{begin,err,log,out,run,sh,trace}
│   │       └── versions.yml
│   ├── mashdist
│   │   └── <SAMPLE_NAME>-dist.txt
│   ├── meningotype
│   │   └── <SAMPLE_NAME>.tsv
│   ├── ngmaster
│   │   └── <SAMPLE_NAME>.tsv
│   ├── seqsero2
│   │   ├── <SAMPLE_NAME>_log.txt
│   │   ├── <SAMPLE_NAME>_result.tsv
│   │   └── <SAMPLE_NAME>_result.txt
│   ├── sistr
│   │   ├── <SAMPLE_NAME>-allele.fasta.gz
│   │   ├── <SAMPLE_NAME>-allele.json.gz
│   │   ├── <SAMPLE_NAME>-cgmlst.csv
│   │   └── <SAMPLE_NAME>.tsv
│   ├── spatyper
│   │   └── <SAMPLE_NAME>.tsv
│   ├── ssuissero
│   │   └── <SAMPLE_NAME>_serotyping_res.tsv
│   ├── staphopiasccmec
│   │   └── <SAMPLE_NAME>.tsv
│   └── tbprofiler
│       ├── <SAMPLE_NAME>.results.csv
│       ├── <SAMPLE_NAME>.results.json
│       ├── <SAMPLE_NAME>.results.txt
│       ├── bam
│       │   └── <SAMPLE_NAME>.bam
│       └── vcf
│           └── <SAMPLE_NAME>.targets.csq.vcf.gz
├── logs
│   ├── csvtk_concat
│   │   └── <BACTOPIA_TOOL>
│   │       ├── nf-csvtk_concat.{begin,err,log,out,run,sh,trace}
│   │       └── versions.yml
│   └── custom_dumpsoftwareversions
│       ├── nf-custom_dumpsoftwareversions.{begin,err,log,out,run,sh,trace}
│       └── versions.yml
├── nf-reports
│   ├── merlin-dag.dot
│   ├── merlin-report.html
│   ├── merlin-timeline.html
│   └── merlin-trace.txt
├── agrvate.tsv
├── ectyper.tsv
├── emmtyper.tsv
├── hicap.tsv
├── hpsuissero.tsv
├── kleborate.tsv
├── legsta.tsv
├── lissero.tsv
├── meningotype.tsv
├── ngmaster.tsv
├── seqsero2.tsv
├── sistr.tsv
├── software_versions.yml
├── software_versions_mqc.yml
├── spatyper.tsv
├── ssuissero.tsv
└── staphopiasccmec.tsv

Directory structure might be different

merlin is available as a standalone Bactopia Tool, as well as from the main Bactopia workflow (e.g. through Staphopia or Merlin). If executed from Bactopia, the merlin directory structure might be different, but the output descriptions below still apply.

Results

Top Level

Below are results that are in the base directory.

Filename	Description
agrvate.tsv	A merged TSV file with AgrVATE results from all samples
ectyper.tsv	A merged TSV file with ECTyper results from all samples
emmtyper.tsv	A merged TSV file with emmtyper results from all samples
hicap.tsv	A merged TSV file with hicap results from all samples
hpsuissero.tsv	A merged TSV file with HpsuisSero results from all samples
kleborate.tsv	A merged TSV file with Kleborate results from all samples
legsta.tsv	A merged TSV file with legsta results from all samples
lissero.tsv	A merged TSV file with LisSero results from all samples
meningotype.tsv	A merged TSV file with meningotype results from all samples
ngmaster.tsv	A merged TSV file with ngmaster results from all samples
seqsero2.tsv	A merged TSV file with seqsero2 results from all samples
sistr.tsv	A merged TSV file with SISTR results from all samples
spatyper.tsv	A merged TSV file with spaTyper results from all samples
ssuissero.tsv	A merged TSV file with SsuisSero results from all samples
staphopiasccmec.tsv	A merged TSV file with staphopia-sccmec results from all samples

AgrVATE

Below is a description of the per-sample results from AgrVATE.

Extension	Description
-agr_gp.tab	Detailed report for agr kmer matches
-blastn_log.txt	Log files from programs called by AgrVATE
-summary.tab	A final summary report for agr typing

ECTyper

Below is a description of the per-sample results from ECTyper.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with ECTyper result, see ECTyper - Report format for details
blast_output_alleles.txt	Allele report generated from BLAST results

emmtyper

Below is a description of the per-sample results from emmtyper.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with emmtyper result, see emmtyper - Result format for details

hicap

Below is a description of the per-sample results from hicap.

Filename	Description
<SAMPLE_NAME>.gbk	GenBank file and cap locus annotations
<SAMPLE_NAME>.svg	Visualization of annotated cap locus
<SAMPLE_NAME>.tsv	A tab-delimited file with hicap results

HpsuisSero

Below is a description of the per-sample results from HpsuisSero.

Filename	Description
<SAMPLE_NAME>_serotyping_res.tsv	A tab-delimited file with HpsuisSero result

Kleborate

Below is a description of the per-sample results from Kleborate.

Filename	Description
<SAMPLE_NAME>.results.tsv	A tab-delimited file with Kleborate result, see Kleborate - Example output for more details

legsta

Below is a description of the per-sample results from legsta.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with legsta result, see legsta - Output for more details

LisSero

Below is a description of the per-sample results from LisSero.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with LisSero results

Mash

Below is a description of the per-sample results from Mash.

Filename	Description
<SAMPLE_NAME>-dist.txt	A tab-delimited file with mash dist results

meningotype

Below is a description of the per-sample results from meningotype .

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with meningotype result

ngmaster

Below is a description of the per-sample results from ngmaster.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with ngmaster results

SeqSero2

Below is a description of the per-sample results from SeqSero2.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with SeqSero2 results
<SAMPLE_NAME>.txt	A text file with key-value pairs of SeqSero2 results

SISTR

Below is a description of the per-sample results from SISTR.

Filename	Description
<SAMPLE_NAME>-allele.fasta.gz	A FASTA file of the cgMLST allele search results
<SAMPLE_NAME>-allele.json.gz	JSON formated cgMLST allele search results, see SISTR - cgMLST search results for more details
<SAMPLE_NAME>-cgmlst.csv	A comma-delimited summary of the cgMLST allele search results
<SAMPLE_NAME>.tsv	A tab-delimited file with SISTR results, see SISTR - Primary results for more details

spaTyper

Below is a description of the per-sample results from spaTyper.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with spaTyper results

SsuisSero

Below is a description of the per-sample results from SsuisSero.

Filename	Description
<SAMPLE_NAME>_serotyping_res.tsv	A tab-delimited file with SsuisSero results

staphopia-sccmec

Below is a description of the per-sample results from staphopia-sccmec.

Filename	Description
<SAMPLE_NAME>.tsv	A tab-delimited file with staphopia-sccmec results

TBProfiler

Below is a description of the per-sample results from TBProfiler.

Filename	Description
<SAMPLE_NAME>.results.csv	A CSV formated TBProfiler result file of resistance and strain type
<SAMPLE_NAME>.results.json	A JSON formated TBProfiler result file of resistance and strain type
<SAMPLE_NAME>.results.txt	A text file with TBProfiler results
<SAMPLE_NAME>.bam	BAM file with alignment details
<SAMPLE_NAME>.targets.csq.vcf.gz	VCF with variant info again refernce genomes

Audit Trail

Below are files that can assist you in understanding which parameters and program versions were used.

Logs

Each process that is executed will have a logs folder containing helpful files for you to review if the need ever arises.

Filename	Description
nf-<PROCESS_NAME>.begin	An empty file used to designate the process started
nf-<PROCESS_NAME>.err	Contains STDERR outputs from the process
nf-<PROCESS_NAME>.log	Contains both STDERR and STDOUT outputs from the process
nf-<PROCESS_NAME>.out	Contains STDOUT outputs from the process
nf-<PROCESS_NAME>.run	The script Nextflow uses to stage/unstage files and queue processes based on given profile
nf-<PROCESS_NAME>.sh	The script executed by bash for the process
nf-<PROCESS_NAME>.trace	The Nextflow Trace report for the process
versions.yml	A YAML formatted file with program versions

Nextflow Reports

These Nextflow reports provide great a great summary of your run. These can be used to optimize resource usage and estimate expected costs if using cloud platforms.

Filename	Description
merlin-dag.dot	The Nextflow DAG visualisation
merlin-report.html	The Nextflow Execution Report
merlin-timeline.html	The Nextflow Timeline Report
merlin-trace.txt	The Nextflow Trace report

Program Versions

At the end of each run, each of the versions.yml files are merged into the files below.

Filename	Description
software_versions.yml	A complete list of programs and versions used by each process
software_versions_mqc.yml	A complete list of programs and versions formatted for MultiQC

Parameters

Required Parameters

Define where the pipeline should find input data and save output data.

Parameter	Description	Default
--bactopia	The path to bactopia results to use as inputs

Filtering Parameters

Use these parameters to specify which samples to include or exclude.

Parameter	Description	Default
--include	A text file containing sample names (one per line) to include from the analysis
--exclude	A text file containing sample names (one per line) to exclude from the analysis

mashdist Parameters

Parameter	Description	Default
--mash_sketch	The reference sequence as a Mash Sketch (.msh file)
--mash_seed	Seed to provide to the hash function	42
--mash_table	Table output (fields will be blank if they do not meet the p-value threshold)	False
--mash_m	Minimum copies of each k-mer required to pass noise filter for reads	1
--mash_w	Probability threshold for warning about low k-mer size.	0.01
--max_p	Maximum p-value to report.	1.0
--max_dist	Maximum distance to report.	1.0
--merlin_dist	Maximum distance to report when using Merlin .	0.1
--full_merlin	Go full Merlin and run all species-specific tools, no matter the Mash distance	False
--use_fastqs	Query with FASTQs instead of the assemblies	False

AgrVATE Parameters

Parameter	Description	Default
--typing_only	agr typing only. Skips agr operon extraction and frameshift detection	False

ECTyper Parameters

Parameter	Description	Default
--opid	Percent identity required for an O antigen allele match	90
--opcov	Minumum percent coverage required for an O antigen allele match	90
--hpid	Percent identity required for an H antigen allele match	95
--hpcov	Minumum percent coverage required for an H antigen allele match	50
--verify	Enable E. coli species verification	False
--print_alleles	Prints the allele sequences if enabled as the final column	False

emmtyper Parameters

Parameter	Description	Default
--emmtyper_wf	Workflow for emmtyper to use.	blast
--cluster_distance	Distance between cluster of matches to consider as different clusters	500
--percid	Minimal percent identity of sequence	95
--culling_limit	Total hits to return in a position	5
--mismatch	Threshold for number of mismatch to allow in BLAST hit	5
--align_diff	Threshold for difference between alignment length and subject length in BLAST	5
--gap	Threshold gap to allow in BLAST hit	2
--min_perfect	Minimum size of perfect match at 3 primer end	15
--min_good	Minimum size where there must be 2 matches for each mismatch	15
--max_size	Maximum size of PCR product	2000

hicap Parameters

Parameter	Description	Default
--database_dir	Directory containing locus database
--model_fp	Path to prodigal model
--full_sequence	Write the full input sequence out to the genbank file rather than just the region surrounding and including the locus	False
--hicap_debug	hicap will print debug messages	False
--gene_coverage	Minimum percentage coverage to consider a single gene complete	0.8
--gene_identity	Minimum percentage identity to consider a single gene complete	0.7
--broken_gene_length	Minimum length to consider a broken gene	60
--broken_gene_identity	Minimum percentage identity to consider a broken gene	0.8

HpsuisSero Parameters

Parameter	Description	Default

Kleborate Parameters

Parameter	Description	Default
--skip_resistance	Turn off resistance genes screening	False
--skip_kaptive	Turn off Kaptive screening of K and O loci	False
--min_identity	Minimum alignment percent identity for main results	90.0
--kleborate_min_coverage	Minimum alignment percent coverage for main results	80.0
--min_spurious_identity	Minimum alignment percent identity for spurious results	80.0
--min_spurious_coverage	Minimum alignment percent coverage for spurious results	40.0
--min_kaptive_confidence	Minimum Kaptive confidence to call K/O loci - confidence levels below this will be reported as unknown	Good
--force_index	Rebuild the BLAST index at the start of execution	False

legsta Parameters

Parameter	Description	Default
--noheader	Don't print header row	False

LisSero Parameters

Parameter	Description	Default
--min_id	Minimum percent identity to accept a match	95.0
--min_cov	Minimum coverage of the gene to accept a match	95.0

meningotype Parameters

You can use these parameters to fine-tune your meningotype analysis

Parameter	Description	Default
--finetype	perform porA and fetA fine typing	False
--porB	perform porB sequence typing (NEIS2020)	False
--bast	perform Bexsero antigen sequence typing (BAST)	False
--mlst	perform MLST	False
--all	perform MLST, porA, fetA, porB, BAST typing	False

ngmaster Parameters

Parameter	Description	Default
--csv	output comma-separated format (CSV) rather than tab-separated	False

SeqSero2 Parameters

Parameter	Description	Default
--run_mode	Workflow to run. 'a' allele mode, or 'k' k-mer mode	k
--input_type	Input format to analyze. 'assembly' or 'fastq'	assembly
--bwa_mode	Algorithms for bwa mapping for allele mode	mem

SISTR Parameters

Parameter	Description	Default
--full_cgmlst	Use the full set of cgMLST alleles which can include highly similar alleles	False

spaTyper Parameters

Parameter	Description	Default
--repeats	List of spa repeats
--repeat_order	List spa types and order of repeats
--do_enrich	Do PCR product enrichment	False

staphopia-sccmec Parameters

Parameter	Description	Default
--hamming	Report the results as hamming distances	False

SsuisSero Parameters

Parameter	Description	Default

TBProfiler Parameters

Parameter	Description	Default
--call_whole_genome	Call whole genome	False
--mapper	Mapping tool to use. If you are using nanopore data it will default to minimap2	bwa
--caller	Variant calling tool to use	freebayes
--calling_params	Extra variant caller options in quotes
--tb_min_depth	Minimum depth required to call variant	10
--tb_af	Minimum allele frequency to call variants	0.1
--tb_reporting_af	Minimum allele frequency to use variants for prediction	0.1
--coverage_fraction_threshold	Cutoff used to calculate fraction of region covered by <= this value	0
--suspect	Use the suspect suite of tools to add ML predictions	False
--no_flagstat	Don't collect flagstats	False
--no_delly	Don't run delly	False

Optional Parameters

These optional parameters can be useful in certain settings.

Parameter	Description	Default
--outdir	Base directory to write results to	./
--run_name	Name of the directory to hold results	bactopia
--skip_compression	Ouput files will not be compressed	False
--keep_all_files	Keeps all analysis files created	False

Max Job Request Parameters

Set the top limit for requested resources for any single job.

Parameter	Description	Default
--max_retry	Maximum times to retry a process before allowing it to fail.	3
--max_cpus	Maximum number of CPUs that can be requested for any single job.	4
--max_memory	Maximum amount of memory (in GB) that can be requested for any single job.	32
--max_time	Maximum amount of time (in minutes) that can be requested for any single job.	120
--max_downloads	Maximum number of samples to download at a time	3

Nextflow Configuration Parameters

Parameters to fine-tune your Nextflow setup.

Parameter	Description	Default
--nfconfig	A Nextflow compatible config file for custom profiles, loaded last and will overwrite existing variables if set.
--publish_dir_mode	Method used to save pipeline results to output directory.	copy
--infodir	Directory to keep pipeline Nextflow logs and reports.	${params.outdir}/pipeline_info
--force	Nextflow will overwrite existing output files.	False
--cleanup_workdir	After Bactopia is successfully executed, the work directory will be deleted.	False

Nextflow Profile Parameters

Parameters to fine-tune your Nextflow setup.

Parameter	Description	Default
--condadir	Directory to Nextflow should use for Conda environments
--registry	Docker registry to pull containers from.	dockerhub
--singularity_cache	Directory where remote Singularity images are stored.
--singularity_pull_docker_container	Instead of directly downloading Singularity images for use with Singularity, force the workflow to pull and convert Docker containers instead.
--force_rebuild	Force overwrite of existing pre-built environments.	False
--queue	Comma-separated name of the queue(s) to be used by a job scheduler (e.g. AWS Batch or SLURM)	general,high-memory
--cluster_opts	Additional options to pass to the executor. (e.g. SLURM: '--account=my_acct_name'
--disable_scratch	All intermediate files created on worker nodes of will be transferred to the head node.	False

Helpful Parameters

Uncommonly used parameters that might be useful.

Parameter	Description	Default
--monochrome_logs	Do not use coloured log outputs.
--nfdir	Print directory Nextflow has pulled Bactopia to
--sleep_time	The amount of time (seconds) Nextflow will wait after setting up datasets before execution.	5
--validate_params	Boolean whether to validate parameters against the schema at runtime	True
--help	Display help text.
--wf	Specify which workflow or Bactopia Tool to execute	bactopia
--list_wfs	List the available workflows and Bactopia Tools to use with '--wf'
--show_hidden_params	Show all params when using --help
--help_all	An alias for --help --show_hidden_params
--version	Display version text.

Citations

If you use Bactopia and merlin in your analysis, please cite the following.

• Bactopia
  Petit III RA, Read TD Bactopia - a flexible pipeline for complete analysis of bacterial genomes. mSystems 5 (2020)

• AgrVATE
  Raghuram V. AgrVATE: Rapid identification of Staphylococcus aureus agr locus type and agr operon variants. (GitHub)

• ECTyper
  Laing C, Bessonov K, Sung S, La Rose C ECTyper - In silico prediction of Escherichia coli serotype (GitHub)

• emmtyper
  Tan A, Seemann T, Lacey D, Davies M, Mcintyre L, Frost H, Williamson D, Gonçalves da Silva A emmtyper - emm Automatic Isolate Labeller (GitHub)

• hicap
  Watts SC, Holt KE hicap: in silico serotyping of the Haemophilus influenzae capsule locus. Journal of Clinical Microbiology JCM.00190-19 (2019)

• HpsuisSero
  Lui J HpsuisSero: Rapid Haemophilus parasuis serotyping (GitHub)

• Kleborate
  Lam MMC, Wick RR, Watts, SC, Cerdeira LT, Wyres KL, Holt KE A genomic surveillance framework and genotyping tool for Klebsiella pneumoniae and its related species complex. Nat Commun 12, 4188 (2021)

• legsta
  Seemann T legsta: In silico Legionella pneumophila Sequence Based Typing (GitHub)

• LisSero
  Kwong J, Zhang J, Seeman T, Horan, K, Gonçalves da Silva A LisSero - In silico serotype prediction for Listeria monocytogenes (GitHub)

• Mash
  Ondov BD, Treangen TJ, Melsted P, Mallonee AB, Bergman NH, Koren S, Phillippy AM Mash: fast genome and metagenome distance estimation using MinHash. Genome Biol 17, 132 (2016)

• meningotype
  Kwong JC, Gonçalves da Silva A, Stinear TP, Howden BP, & Seemann T meningotype: in silico typing for Neisseria meningitidis. (GitHub)

• ngmaster
  Kwong J, Gonçalves da Silva A, Schultz M, Seeman T ngmaster - In silico multi-antigen sequence typing for Neisseria gonorrhoeae (NG-MAST) (GitHub)

• SeqSero2
  Zhang S, Den-Bakker HC, Li S, Dinsmore BA, Lane C, Lauer AC, Fields PI, Deng X. SeqSero2: rapid and improved Salmonella serotype determination using whole genome sequencing data. Appl Environ Microbiology 85(23):e01746-19 (2019)

• SISTR
  Yoshida CE, Kruczkiewicz P, Laing CR, Lingohr EJ, Gannon VPJ, Nash JHE, Taboada EN The Salmonella In Silico Typing Resource (SISTR): An Open Web-Accessible Tool for Rapidly Typing and Subtyping Draft Salmonella Genome Assemblies. PloS One, 11(1), e0147101. (2016)

• spaTyper
  Sanchez-Herrero JF, and Sullivan M spaTyper: Staphylococcal protein A (spa) characterization pipeline. Zenodo. (2020)

• SsuisSero
  Lui J SsuisSero: Rapid Streptococcus suis serotyping (GitHub)

• staphopia-sccmec
  Petit III RA, Read TD Staphylococcus aureus viewed from the perspective of 40,000+ genomes. PeerJ 6, e5261 (2018)

• TBProfiler
  Phelan JE, O’Sullivan DM, Machado D, Ramos J, Oppong YEA, Campino S, O’Grady J, McNerney R, Hibberd ML, Viveiros M, Huggett JF, Clark TG Integrating informatics tools and portable sequencing technology for rapid detection of resistance to anti-tuberculous drugs. Genome Med 11, 41 (2019)