Skip to content

Binny error: Vizbin-contig-names-points file empty (FIX proposed below)

IMP version: a35c654

ESB server: litcrit

IMP path: /mnt/internal/tools/IMP3

Sample: NOMIS sediment - GL_53_Down

Log files: /mnt/esb-storage-01/NOMIS/test/IMP/MG/GL_53_Down/run1/logs

Sample Size:

ls -lh Preprocessing/
total 180G
-rw-rw-r-- 1 susheel.busi susheel.busi  14G May 31 13:40 mg.r1.fq.gz
lrwxrwxrwx 1 susheel.busi susheel.busi   30 May 31 12:53 mg.r1.preprocessed.fq -> mg.r1.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi  63G May 31 12:51 mg.r1.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi  13G May 31 13:36 mg.r1.trimmed.fq.gz
-rw-rw-r-- 1 susheel.busi susheel.busi  15G May 31 13:42 mg.r2.fq.gz
lrwxrwxrwx 1 susheel.busi susheel.busi   30 May 31 12:53 mg.r2.preprocessed.fq -> mg.r2.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi  62G May 31 12:53 mg.r2.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi  13G May 31 13:38 mg.r2.trimmed.fq.gz
lrwxrwxrwx 1 susheel.busi susheel.busi   30 May 31 12:53 mg.se.preprocessed.fq -> mg.se.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi 1.9G May 31 12:53 mg.se.trimmed.PhiX_filtered.fq
-rw-rw-r-- 1 susheel.busi susheel.busi 378M May 31 12:53 mg.se.trimmed.fq.gz
-rw-rw-r-- 1 susheel.busi susheel.busi 371M May 31 12:53 mg.se1.trimmed.fq.gz
-rw-rw-r-- 1 susheel.busi susheel.busi 7.4M May 31 12:53 mg.se2.trimmed.fq.gz
  • Issue persists across IRIS and litcrit.
  • Setting the number of CPUS from 64 to 96 on litcrit, still has the issue.
  • Java machine ran out of memory.
  • Potential solution? --> java -jar -Xmx3g VizBin-dist.jar. See https://github.com/claczny/VizBin/wiki 
2021-07-28 08:16:36,414 DEBUG [main] (Main.java:45) - Running command line...
2021-07-28 08:16:36,421 DEBUG [main] (ProcessInput.java:60) - Init of ProcessInput
2021-07-28 08:16:36,440 DEBUG [Thread-0] (ObjectWithProperties.java:33) - Property changed: FINISHED. Old: true New: false
2021-07-28 08:16:36,441 DEBUG [Thread-0] (ProcessInput.java:105) - /tmp
2021-07-28 08:16:36,475 DEBUG [Thread-0] (ProcessInput.java:114) - Loading data from file.
Contig length treshold: 1000
2021-07-28 08:16:36,476 DEBUG [Thread-0] (ProcessInput.java:75) - Loading fasta file: Binning/binny/mg.assembly.merged.cut.fa
2021-07-28 08:16:36,605 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] Loading fasta file: Binning/binny/mg.assembly.merged.cut.fa (5)
2021-07-28 08:17:06,776 DEBUG [Thread-0] (ProcessInput.java:75) - DataSet loaded (1139281 sequences)
2021-07-28 08:17:06,777 DEBUG [Thread-0] (ProcessInput.java:75) - Creating kmers (k=5, merge = true)
2021-07-28 08:17:06,777 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] DataSet loaded (1139281 sequences) (5)
2021-07-28 08:17:06,778 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] Creating kmers (k=5, merge = true) (5)
2021-07-28 08:17:06,779 DEBUG [Thread-0] (DataSetUtils.java:72) - Creating usedVal for k=5
2021-07-28 08:18:17,735  INFO [Thread-0] (DataSetUtils.java:66) - Total number of kmers: -2125057372.
1449 (0.0%) kmers ignored since containing unknown letters.
2021-07-28 08:18:17,735 DEBUG [Thread-0] (ProcessInput.java:75) - Normalizing vectors...
2021-07-28 08:18:17,736 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] Normalizing vectors... (10)
2021-07-28 08:19:59,122 DEBUG [Thread-0] (ProcessInput.java:75) - Clr normalization...
2021-07-28 08:19:59,127 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] Clr normalization... (15)
2021-07-28 08:22:32,173 DEBUG [Thread-0] (ProcessInput.java:75) - Running PCA... (Mtj)
2021-07-28 08:22:32,175 DEBUG [AWT-EventQueue-0] (ProcessInput.java:91) - [PROGRESS BAR] Running PCA... (Mtj) (20)
Jul 28, 2021 8:22:35 AM com.github.fommil.netlib.LAPACK <clinit>
WARNING: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
Jul 28, 2021 8:22:35 AM com.github.fommil.jni.JniLoader liberalLoad
INFO: successfully loaded /tmp/jniloader106444219514783734netlib-native_ref-linux-x86_64.so
2021-07-28 08:26:32,208  INFO [Thread-0] (ProcessInput.java:238) - Error! Java machine ran out of memmory.
Check input file size, or increase java heap size.
Application will now restart.
java.lang.OutOfMemoryError: Java heap space
        at no.uib.cipr.matrix.AbstractDenseMatrix.<init>(AbstractDenseMatrix.java:59)
        at no.uib.cipr.matrix.DenseMatrix.<init>(DenseMatrix.java:167)
        at lu.uni.lcsb.vizbin.pca.PrincipleComponentAnalysisMtj.setup(PrincipleComponentAnalysisMtj.java:40)
        at lu.uni.lcsb.vizbin.DataSetUtils.computePca(DataSetUtils.java:246)
        at lu.uni.lcsb.vizbin.ProcessInput$2.run(ProcessInput.java:141)
2021-07-28 08:26:32,218 ERROR [Thread-0] (ProcessInput.java:252) - Jar: /mnt/internal/tools/IMP3/workflow/bin/VizBin-dist.jar
2021-07-28 08:26:32,218 ERROR [Thread-0] (ProcessInput.java:260) - Command: [/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-1.el7_9.x86_64/jre/bin/java, -jar, /mnt/internal/tools/IMP3/workflow/bin/VizBin-dist.jar]
Edited by Susheel Busi