Skip to content
sortmernaPublic

About

SortMeRNA: next-generation sequence filtering and alignment tool

Topics

Resources

Stars

304 stars

Watchers

11 watching

Forks

Repository files navigation

sortmerna

SortMeRNA is a local sequence alignment tool for filtering, mapping and clustering.

The core algorithm is based on approximate seeds and allows for sensitive analysis of NGS reads. The main application of SortMeRNA is filtering rRNA from metatranscriptomic data. SortMeRNA takes as input files of reads (fasta, fastq, fasta.gz, fastq.gz) and one or multiple rRNA database file(s), and sorts apart aligned and rejected reads into two files. SortMeRNA works with Illumina, Ion Torrent and PacBio data, and can produce SAM and BLAST-like alignments.

SortMeRNA is also available through the nf-core RNA-Seq pipeline v.3.26.0.

Table of Contents

What's new in 8.0.0

Using CXXOPTS library for options processing

The options processing and display improved and enhanced:

  • Dashes and underscores are interchangeable ('--paired-out' == '--paired_out').
  • A value may follow the name or be attached with '=' ('--threads 4' == '--threads=4').
  • Options summary: '-h' lists the options
  • Full options info: '--help' prints the full reference
  • Single option help: 'sortmerna -h' explains a single option e.g. 'sortmerna index -h'.
  • Option names may be given with one or two leading dashes ('-ref' == '--ref') as before

Fixes issue 466

What's new in 7.1.0

C API library (experimental)

Adding a C API library (details below). The main functionality is unchanged. The library is static self-contained, and allows for programmatic access to the main Sortmerna functionality.

Getting Started

SortMeRNA 6 is C++17 compliant. It uses CMake as the build system, and can be run/built on Linux, Mac, and on AMD64 and ARM64 architectures. Support for native Windows was dropped in version 6.0. WSL can be used to run Sortmerna on Windows.

Using Conda package

Install conda - official docs

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh

Sortmerna conda packages are hosted on conda-forge.

conda config --add channels conda-forge

conda search sortmerna

 Name              Version Build                 Channel     Subdir  
──────────────────────────────────────────────────────────────────────
 sortmerna 8.0.2   py311h19982a8_0 (+  4 builds) conda-forge linux-64
 sortmerna 8.0.1   py311h19982a8_0 (+  4 builds) conda-forge linux-64
 sortmerna 8.0.0   py311h19982a8_0 (+  4 builds) conda-forge linux-64
 sortmerna 7.1.0   py311h19982a8_0 (+  4 builds) conda-forge linux-64
 sortmerna 7.0.0   py310hea66570_0 (+  9 builds) conda-forge linux-64
 sortmerna 6.0.2   py310hea66570_0 (+  4 builds) conda-forge linux-64
 sortmerna 6.0.1   py310hea66570_0 (+  4 builds) conda-forge linux-64
 sortmerna 6.0.0   py310hea66570_0 (+  4 builds) conda-forge linux-64
 sortmerna 5.0.0   py310hea66570_0 (+  4 builds) conda-forge linux-64

# create a new environment and install SortMeRNA in it
conda create --name sortmerna_run
conda activate sortmerna_run
conda install sortmerna
which sortmerna
  /home/biocodz/miniconda3/envs/sortmerna_env/bin/sortmerna

# check version
sortmerna --version

# help
sortmerna -h  # list available commands
sortmerna --help  # view full help
sortmerna index -h  # view the selected command help (--index)
sortmerna paired-in -h
sortmerna paired_in -h

Using GitHub release binaries on Linux

Visit Sortmerna GitHub Releases

Linux distribution is a Shell script with the embedded installation archive.

Issue the following bash commands:

pushd ~

# get the distro
wget https://github.com/sortmerna/sortmerna/releases/download/v8.0.2/sortmerna-8.0.2-Linux.sh

# view the installer usage
bash sortmerna-8.0.2-Linux.sh --help
    Options: [defaults in brackets after descriptions]
      --help            print this message
      --version         print cmake installer version
      --prefix=dir      directory in which to install
      --include-subdir  include the sortmerna-8.0.2-Linux subdirectory
      --exclude-subdir  exclude the sortmerna-8.0.2-Linux subdirectory
      --skip-license    accept license

# run the installer
bash sortmerna-8.0.2-Linux.sh --skip-license
  sortmerna Installer Version: 8.0.2, Copyright (c) Clarity Genomics
  This is a self-extracting archive.
  The archive will be extracted to: $HOME/sortmerna
  
  Using target directory: /home/biocodz/sortmerna
  Extracting, please wait...
  
  Unpacking finished successfully

# check the installed binaries
ls -lrt /home/biocodz/sortmerna/bin/
sortmerna

# set PATH
export PATH=$HOME/sortmerna/bin:$PATH

# test the installation
sortmerna --version
  SortMeRNA version 8.0.2
  ...

# view help
sortmerna -h

Running

  • The only required options are --ref and --reads
  • Options (any) can be specified usig a single dash e.g. -ref and -reads
  • Dashes and underscores in option names are interchangeable e.g. --paired-out and --paired_out
  • A value can follow the option name or be attached with = e.g. --threads 4 and --threads=4
  • sortmerna -h lists the options, sortmerna --help prints the full reference, and sortmerna <option> -h explains a single option e.g. sortmerna index -h
  • Both plain fasta/fastq and archived fasta.gz/fastq.gz files are accepted
  • file extensions .fastq, .fastq.gz, .fq, .fq.gz, .fasta, ... are optional. The format and compression are automatically recognized
  • Relative paths are accepted

for example

# single reference and single reads file
sortmerna --ref REF_PATH --reads READS_PATH

# for multiple references use multiple '--ref'
sortmerna --ref REF_PATH_1 --ref REF_PATH_2 --ref REF_PATH_3 --reads READS_PATH

# for paired reads use '--reads' twice
sortmerna --ref REF_PATH_1 --ref REF_PATH_2 --ref REF_PATH_3 --reads READS_PATH_1 --reads READS_PATH_2

More examples can be found in test.jinja and run.py

Execution trace

Here is a sample execution trace.

IMPORTANT

  • Progressing execution trace showing the number of reads processed so far indicates a normally running program.
  • Non-progressing trace means a problem. Please, kill the process (no waiting for two days), and file an issue here
  • please, provide the execution trace when filing issues.

Sample execution statistics are provided to give an idea on what the execution time might be.

Read feed modes (gzipped FASTQ)

Since 5.0.0 SortMeRNA processes both indexed and flat files in parallel rather than physically splitting them. SortMeRNA automatically defaults to indexed in-memory feed (INDEXED_GZ) that uses rapidgzip to assign each worker thread its own byte range — no split files are written to disk.

The legacy --readfeed 1 on-disk splitting still works but is deprecated. The previously-documented run.py split pre-splitting workflow is no longer necessary and not advised.

Building from sources

Build instructions

C library API

Experimental: the C API is new, and it may change in later releases in ways that require callers to be updated.

A reentrant C library (libsmr_api.a, header include/smr_api.h) runs SortMeRNA inside another program, without a subprocess:

  • smr_run aligns reads files and returns per-read results as well as the usual reports.
  • smr_run_seqs aligns in-memory sequences.
  • smr_index_load / smr_run_seqs_with_index / smr_index_free load the index once and align many in-memory batches against it.
  • smr_index_load_seqs loads the reference sequences from memory instead of a file.

Reuse one workdir so that the index is built only once. The in-memory functions report every hit, with an e-value per read that does not depend on how reads are batched, so their e-values differ from the CLI's.

The build installs the library next to the executable, so python3 setup.py sortmerna leaves lib/libsmr_api.a and include/smr_api.h under the install prefix (dist by default), and the packages contain them too. The archive is self-contained — RocksDB, Parasail, alp and zlib are merged into it — so an application links only:

cc app.c -I dist/include dist/lib/libsmr_api.a -lstdc++ -lpthread -ldl -lm

See C Library API for the reference.

User Manual

See Sortmerna Read The Docs project.

In case you need PDF, any modern browser can print web pages to PDF.

Databases

The reference databases are maintained in the sortmerna-database repository, which documents database sources, preparation, clustering parameters, and benchmark measurements of sensitivity, specificity, runtime, and memory usage.

Download the default (recommended), sensitive, or fast database from the SILVA 138.2 / Rfam 15.1 release.

Each archive contains the reference FASTA and family map. Build indexes with your installed SortMeRNA version.

Taxonomies

The folder data/rRNA_databases/silva_ids_acc_tax.tar.gz contains SILVA taxonomy strings (extracted from XML file generated by ARB) for each of the reference sequences in the representative databases. The format of the files is three tab-separated columns, the first being the reference sequence ID, the second being the accession number and the final column is the taxonomy.

Citation

If you use SortMeRNA, please cite: Kopylova E., Noé L. and Touzet H., "SortMeRNA: Fast and accurate filtering of ribosomal RNAs in metatranscriptomic data", Bioinformatics (2012), doi: 10.1093/bioinformatics/bts611.

Contributors

See AUTHORS for a list of contributors to this project.

Maintained by the team at Clarity Genomics Inc and Centre de Recherche en Informatique, Signal et Automatique de Lille.

Third-party dependencies

Refer to 3rdparty.jinja for the full list of bundled libraries. In 8.x these include: Zlib, RocksDB, Rapidgzip (indexed_bzip2), BBHash, Parasail, ALP, CPPOPTS.

Support

For questions and comments, feel free to file an issue, or start a discussion.

About

SortMeRNA: next-generation sequence filtering and alignment tool

Topics

Resources

Stars

304 stars

Watchers

11 watching

Forks

Releases

Used by

Contributors

Languages