Error in Build RPGG for my own datasets

[BrendaLee1]

Hi,
I use 35 hyplotypes to build RPGG for my VNTR sets, I find that the program always reports the following error when I run /soft/danbing-tk-1.1//script/preMBE.py /35Hyplto/genome.bam.tsv pan.tr.mbe.v0.bed:
beds[hi,m] = np.loadtxt(f"{g}/tmp1.{h}.bed", dtype=object, usecols=[0,1,2,6], ndmin=2)
ValueError: shape mismatch: value array of shape (101497,4) could not be broadcast to indexing result of shape (37920,4)
I checked f"{g}/tmp1.{h}.bed" file, it's seems that every hyplotype file have different number of mapping locations, what should I do to solve this problem, any suggestion will be appreciated.

Best wishes
Lee

[joyeuxnoel8]

Hi Lee,

Could you trying using the latest commit? The preMBE.py is obsolete and not used in the current pipeline.

Cheers,
-Tony

[BrendaLee1]

Hi Tony,
I tried both danbing-tk-1.3.2 and danbing-tk-1.0, I find that danbing-tk-1.0 can run successfully 'snakemake -p -s ../pipeline/GoodPanGenomeGraph.snakefile -j 4 --rerun-incomplete --output-wait 3', but danbing-tk-1.3.2 always have the following message:
module load gcc

/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((0+1)),$((0+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00514.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00514.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.1.tr.fasta

if [ 0 == "1" ]; then
samtools fasta -@2 -n |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG005 14.rawPB -fai /dev/stdin -o HG00514.rawIL -p 1 -cth 45 -rth 0.5
fi

/bin/bash: line 3: module: command not found
[Thu Jul 20 11:54:39 2023]
Error in rule GenRawGenomeGraph:
jobid: 11
output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/o utput/HG00733.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.graph.kmers
shell:

set -eu
ulimit -c 20000
cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/
module load gcc

/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((2+1)),$((2+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00733.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00733.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.1.tr.fasta

if [ 0 == "1" ]; then
samtools fasta -@2 -n |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG007 33.rawPB -fai /dev/stdin -o HG00733.rawIL -p 1 -cth 45 -rth 0.5
fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

/bin/bash: line 3: module: command not found
[Thu Jul 20 11:54:39 2023]
Error in rule GenRawGenomeGraph:
jobid: 10
output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/o utput/HG00514.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.rawPB.graph.kmers
shell:

set -eu
ulimit -c 20000
cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/
module load gcc

/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((0+1)),$((0+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00514.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00514.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.1.tr.fasta

if [ 0 == "1" ]; then
samtools fasta -@2 -n |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG005 14.rawPB -fai /dev/stdin -o HG00514.rawIL -p 1 -cth 45 -rth 0.5
fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

It's seems that lack of module make the command failed. Because our platform do not support this software, so I delete all the module load command in file GoodPanGenomeGraph.snakefile.

The pan.tr.kmers, pan.ntr.kmers and pan.graph.kmers successfully generated but error message occur when run rule GenSerializedGraphAndIndex, error message below:

Error in rule GenSerializedGraphAndIndex:
jobid: 9
output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/pan.graph.umap, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/pan.kmerDBi.umap, /rd1/laixh/soft/danbi
shell:

cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/
ulimit -c 20000

/rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools serialize /rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan
/rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools ksi pan.tr.kmers >/rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan.tr.ksi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

**I tried command line "/rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools serialize /rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan", this command returns Segmentation fault (core dumped).
Do you have any idea to fix this problem?

best wishes,
Lee

[joyeuxnoel8]

Hi Lee,

Thanks for reporting this error. I just removed the strict dependency for module. It will now invoke module only when your system supports it. Let me know if this works.

Thanks!
-Tony

[BrendaLee1]

Hi,
Thanks for your reply, but the error message still occur:
Error in rule GenRawGenomeGraph:
jobid: 11
output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.graph.kmers
shell:

set -eu
ulimit -c 20000
cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/
type module &>/dev/null
if [ $? == 0 ]; then
module load gcc
fi

/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((2+1)),$((2+2)) /rd1/laixh/soft/danbing-tk-1.3.2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00733.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.1.tr.fasta

if [ 0 == "1" ]; then
samtools fasta -@2 -n |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin |
/rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG00733.rawPB -fai /dev/stdin -o HG00733.rawIL -p 1 -cth 45 -rth 0.5
fi

    (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Best wishes,
Lee

[joyeuxnoel8]

Fixed again! Forgot to make sure return code is zero. Hope it works now.