Repository navigation
Description
Summary
TransDecoder.Predict writes its checkpoint directory into the LongOrfs directory it receives as a staged input.
That directory belongs to the TRANSDECODER_LONGORF task, so two things follow, and both are reproducible from a clean run:
- The task never resumes. Because it modifies its own input, a following
-resumere-executes it even when nothing has changed. - The re-execution then fails. Nextflow wipes the task directory before re-running, but not the LongOrfs directory the symlink points at, so the checkpoints outlive the outputs.
TransDecoder.Predictsees a complete checkpoint set, skips every step including the ones that create the outputs, and exits 0 having written nothing.
The failure surfaces as Missing output file(s) '*.transdecoder.pep' with Command exit status: 0, which points at the module rather than at the checkpoints, and it is permanent: every retry repeats it.
Reproduction
With nf-core/metatdenovo's smallest test profile, about two minutes per run:
nextflow run nf-core/metatdenovo -profile test,docker --outdir out --orf_caller transdecoder
# [SUCCESS] completed=63 failed=0 cached=0
nextflow run nf-core/metatdenovo -profile test,docker --outdir out --orf_caller transdecoder -resume
# TRANSDECODER_LONGORF CACHED
# TRANSDECODER_PREDICT FAILED
# [FAILED] completed=1 failed=1 cached=39Nothing changed between the two runs.
TRANSDECODER_LONGORF is cached as expected; TRANSDECODER_PREDICT is not, and its re-execution fails.
From the failing task's .command.err, every step is skipped, including the final copies:
-- Skipping CMD: cp .../longest_orfs.cds.best_candidates.gff3.revised_starts.gff3
.../megahit_assembly.contigs.fa.transdecoder.gff3,
checkpoint [.../megahit_assembly.contigs.fa.transdecoder_dir/__checkpoints_TDpredict/copy_to_final_outputfile.974da7....ok] exists.
transdecoder is finished.
and in the task directory the checkpoints resolve outside it:
megahit_assembly.contigs.fa.transdecoder_dir ->
.../work/6a/36d125.../megahit.transdecoder/megahit_assembly.contigs.fa.transdecoder_dir
Versions: transdecoder 5.7.1--pl5321hdfd78af_0, Nextflow 26.04.6, local executor with Docker.
Why the module cannot just pass this through
This is how the tool is meant to work -- TransDecoder.Predict's own help says --output_dir | -O <string> same output directory from the TransDecoder.LongOrfs, so writing into the LongOrfs directory is by design.
What is specific to the module is that the directory is a staged input rather than a working copy, so the tool's writes land in another task's directory and Nextflow's caching sees an input that changes after use.
Possible fixes
In rough order of preference:
-
Give Predict a private, writable view of the LongOrfs directory.
Building a directory of symlinks to each file inside the staged one costs no extra disk: reads follow the symlinks, and everythingPredictcreates -- the checkpoint directory and its intermediates -- is local to the task.
Worth verifying first thatPredictonly ever adds files there and never overwrites a LongOrfs output. -
Copy the directory (
cp -rL) into the task directory before running.
Simplest and certainly correct, but it duplicates the LongOrfs output, which is large for metagenome assemblies --longest_orfs.cdsalone can reach several GB. -
At minimum, remove a stale
__checkpoints_TDpredictbefore running, so a re-executed task redoes the work instead of failing.
One line, and it fixes the hard failure, but it writes into another task's directory and leaves the self-invalidation in place, so it is a fallback rather than a fix.
Independently of which is chosen, the module could check that its declared outputs exist and fail with a message naming the checkpoints, instead of exiting 0 and leaving Nextflow to report a missing file.
Impact
Any interrupted run, any errorStrategy 'retry', and any -resume can hit this.
With a retry policy it converts a transient failure into a permanent one, and on large assemblies the wasted work is substantial: LongOrfs plus Predict over a multi-sample metagenome assembly is hours.
Reported with help from Claude Code.
Activity
transdecoder/predict: fix module never resuming and failing on re-exe…
Add sativaepang/looplace module
Add sativaepang/looplace module
Add sativaepang/looplace module