LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jim251

I am developing a new process that needs to load > 3 million records into a SQL 2012 server. At around 2.8 million records loaded I get the error below. Looking at Google.com, many of the comments are pointing me towards a Java memory issue. My next step is to try modifying the dburl: parameter to increase the java heap size to 6G. So far the syntax is not working for me: "java -Xmx6g <myprogram>". What is the proper method for passing this param? Here is my existing DBURL: jdbc:microsoft:sqlserver://cplbdb-dt-01d

Full error is below:

Installing log (04/09/2014 10:37:57) ...
*** glibc detected *** java: free(): invalid pointer: 0x00002aaabc01a858 ***======= Backtrace: =========/lib64/libc.so.6[0x3b5047230f]/lib64/libc.so.6(cfree+0x4b)[0x3b5047276b]/opt/home/brain/brain45/lib/linux-x86-64/libLaeExprOps.so(_ZN6Salmon4LineD0Ev+0xc3)[0x2aaab2adb253][0x2aaaab1139df]======= Memory map: ========40000000-40009000 r-xp 00000000 fd:02 1869892 /opt/home/brain/jdk1.6.0_25/bin/java40108000-4010a000 rwxp 00008000 fd:02 1869892 /opt/home/brain/jdk1.6.0_25/bin/java4010a000-4010b000 ---p 4010a000 00:00 04010b000-4020b000 rwxp 4010b000 00:00 040dbe000-40dc1000 ---p 40dbe000 00:00 040dc1000-40ebf000 rwxp 40dc1000 00:00 040f21000-40f22000 ---p 40f21000 00:00 040f22000-41022000 rwxp 40f22000 00:00 041022000-41023000 ---p 41022000 00:00 041023000-41123000 rwxp 41023000 00:00 041123000-41124000 ---p 41123000 00:00 041124000-41224000 rwxp 41124000 00:00 041224000-41225000 ---p 41224000 00:00 041225000-41325000 rwxp 41225000 00:00 041325000-41328000 ---p 41325000 00:00 041328000-41426000 rwxp 41328000 00:00 041426000-41429000 ---p 41426000 00:00 041429000-41527000 rwxp 41429000 00:00 041863000-41866000 ---p 41863000 00:00 041866000-41964000 rwxp 41866000 00:00 041de6000-41de9000 ---p 41de6000 00:00 041de9000-41ee7000 rwxp 41de9000 00:00 041ee7000-41eea000 ---p 41ee7000 00:00 041eea000-41fe8000 rwxp 41eea000 00:00 041fe8000-41feb000 ---p 41fe8000 00:00 041feb000-420e9000 rwxp 41feb000 00:00 0420e9000-420ea000 ---p 420e9000 00:00 0420ea000-421ea000 rwxp 420ea000 00:00 0421ea000-421eb000 ---p 421ea000 00:00 0421eb000-42beb000 rwxp 421eb000 00:00 042beb000-42bec000 ---p 42beb000 00:00 042bec000-435ec000 rwxp 42bec000 00:00 04e17d000-4e870000 rwxp 4e17d000 00:00 0 [heap]bae00000-bc2c0000 rwxp bae00000 00:00 0bc2c0000-c0000000 rwxp bc2c0000 00:00 0c0000000-c9600000 rwxp c0000000 00:00 0c9600000-d4e00000 ---p c9600000 00:00 0d4e00000-eaab0000 rwxp d4e00000 00:00 0eaab0000-ed4a0000 rwxp eaab0000 00:00 0ed4a0000-f51a0000 ---p ed4a0000 00:00 0f51a0000-100000000 rwxp f51a0000 00:00 0300c600000-300c6e6000 r-xp 00000000 fd:00 1331852 /usr/lib64/libstdc++.so.6.0.8300c6e6000-300c8e5000 ---p 000e6000 fd:00 1331852 /usr/lib64/libstdc++.so.6.0.8300c8e5000-300c8eb000 r-xp 000e5000 fd:00 1331852 /usr/lib64/libstdc++.so.6.0.8300c8eb000-300c8ee000 rwxp 000eb000 fd:00 1331852 /usr/lib64/libstdc++.so.6.0.8300c8ee000-300c900000 rwxp 300c8ee000 00:00 03b50000000-3b5001c000 r-xp 00000000 fd:00 1444092 /lib64/ld-2.5.so3b5021b000-3b5021c000 r-xp 0001b000 fd:00 1444092 /lib64/ld-2.5.so3b5021c000-3b5021d000 rwxp 0001c000 fd:00 1444092 /lib64/ld-2.5.so3b50400000-3b5054e000 r-xp 00000000 fd:00 1444094 /lib64/libc-2.5.so3b5054e000-3b5074d000 ---p 0014e000 fd:00 1444094 /lib64/libc-2.5.so3b5074d000-3b50751000 r-xp 0014d000 fd:00 1444094 /lib64/libc-2.5.so3b50751000-3b50752000 rwxp 00151000 fd:00 1444094 /lib64/libc-2.5.so3b50752000-3b50757000 rwxp 3b50752000 00:00 03b50800000-3b50882000 r-xp 00000000 fd:00 1444099 /lib64/libm-2.5.so3b50882000-3b50a81000 ---p 00082000 fd:00 1444099 /lib64/libm-2.5.so3b50a81000-3b50a82000 r-xp 00081000 fd:00 1444099 /lib64/libm-2.5.so3b50a82000-3b50a83000 rwxp 00082000 fd:00 1444099 /lib64/libm-2.5.so3b50c00000-3b50c02000 r-xp 00000000 fd:00 1444097 /lib64/libdl-2.5.so3b50c02000-3b50e02000 ---p 00002000 fd:00 1444097 /lib64/libdl-2.5.so3b50e02000-3b50e03000 r-xp 00002000 fd:00 1444097 /lib64/libdl-2.5.so3b50e03000-3b50e04000 rwxp 00003000 fd:00 1444097 /lib64/libdl-2.5.so3b51000000-3b51016000 r-xp 00000000 fd:00 1444115 /lib64/libpthread-2.5.so3b51016000-3b51215000 ---p 00016000 fd:00 1444115 /lib64/libpthread-2.5.so3b51215000-3b51216000 r-xp 00015000 fd:00 1444115 /lib64/libpthread-2.5.so3b51216000-3b51217000 rwxp 00016000 fd:00 1444115 /lib64/libpthread-2.5.so3b51217000-3b5121b000 rwxp 3b51217000 00:00 03b51800000-3b51807000 r-xp 00000000 fd:00 1444159 /lib64/librt-2.5.so3b51807000-3b51a07000 ---p 00007000 fd:00 1444159 /lib64/librt-2.5.so3b51a07000-3b51a08000 r-xp 00007000 fd:00 1444159 /lib64/librt-2.5.so3b51a08000-3b51a09000 rwxp 00008000 fd:00 1444159 /lib64/librt-2.5.so3b53c00000-3b53c15000 r-xp 00000000 fd:00 1444156 /lib64/libnsl-2.5.so3b53c15000-3b53e14000 ---p 00015000 fd:00 1444156 /lib64/libnsl-2.5.so3b53e14000-3b53e15000 r-xp 00014000 fd:00 1444156 /lib64/libnsl-2.5.so3b53e15000-3b53e16000 rwxp 00015000 fd:00 1444156 /lib64/libnsl-2.5.so3b53e16000-3b53e18000 rwxp 3b53e16000 00:00 03b5ba00000-3b5ba11000 r-xp 00000000 fd:00 1444170 /lib64/libresolv-2.5.so3b5ba11000-3b5bc11000 ---p 00011000 fd:00 1444170 /lib64/libresolv-2.5.so3b5bc11000-3b5bc12000 r-xp 00011000 fd:00 1444170 /lib64/libresolv-2.5.so3b5bc12000-3b5bc13000 rwxp 00012000 fd:00 1444170 /lib64/libresolv-2.5.so3b5bc13000-3b5bc15000 rwxp 3b5bc13000 00:00 03b62200000-3b6220d000 r-xp 00000000 fd:00 1444113 /lib64/libgcc_s-4.1.2-20080825.so.13b6220d000-3b6240d000 ---p 0000d000 fd:00 1444113 /lib64/libgcc_s-4.1.2-20080825.so.13b6240d000-3b6240e000 rwxp 0000d000 fd:00 1444113 /lib64/libgcc_s-4.1.2-20080825.so.12aaaaaaab000-2aaaaaaad000 r-xs 0000d000 fd:02 2820383 /opt/home/brain/brain45/lib/java/ext/msutil.jar2aaaaaaad000-2aaaaaaaf000 r-xs 0000f000 fd:02 2820384 /opt/home/brain/brain45/lib/java/ext/mssqlserver.jar2aaaaaaaf000-2aaaaaab5000 r-xs 00041000 fd:02 2820385 /opt/home/brain/brain45/lib/java/ext/msbase.jar2aaaaaab5000-2aaaaaab8000 r-xs 00036000 fd:02 2820382 /opt/home/brain/brain45/lib/java/ext/sqljdbc.jar2aaaaaabf000-2aaaaaacc000 r-xp 00000000 fd:02 1870534 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libverify.so2aaaaaacc000-2aaaaabcb000 ---p 0000d000 fd:02 1870534 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libverify.so2aaaaabcb000-2aaaaabce000 rwxp 0000c000 fd:02 1870534 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libverify.so2aaaaabce000-2aaaaabf7000 r-xp 00000000 fd:02 1870552 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libjava.so2aaaaabf7000-2aaaaacf6000 ---p 00029000 fd:02 1870552 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libjava.so2aaaaacf6000-2aaaaacfd000 rwxp 00028000 fd:02 1870552 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libjava.so2aaaaacfd000-2aaaaacfe000 r-xp 2aaaaacfd000 00:00 02aaaaacfe000-2aaaaacff000 rwxp 2aaaaacfe000 00:00 02aaaaacff000-2aaaaad07000 rwxs 00000000 fd:00 1081355 /tmp/hsperfdata_brain/87382aaaaad12000-2aaaaad1c000 r-xp 00000000 fd:00 1442147 /lib64/libnss_files-2.5.so2aaaaad1c000-2aaaaaf1b000 ---p 0000a000 fd:00 1442147 /lib64/libnss_files-2.5.so2aaaaaf1b000-2aaaaaf1c000 r-xp 00009000 fd:00 1442147 /lib64/libnss_files-2.5.so2aaaaaf1c000-2aaaaaf1d000 rwxp 0000a000 fd:00 1442147 /lib64/libnss_files-2.5.so2aaaaaf1d000-2aaaaaf2b000 r-xp 00000000 fd:02 1870539 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libzip.so2aaaaaf2b000-2aaaab02d000 ---p 0000e000 fd:02 1870539 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libzip.so2aaaab02d000-2aaaab030000 rwxp 00010000 fd:02 1870539 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libzip.so2aaaab030000-2aaaab2a1000 rwxp 2aaaab030000 00:00 02aaaab2a1000-2aaaae031000 rwxp 2aaaab2a1000 00:00 02aaaae031000-2aaaae03b000 rwxp 2aaaae031000 00:00 02aaaae03b000-2aaaae0f1000 rwxp 2aaaae03b000 00:00 02aaaae0f1000-2aaaae0fc000 rwxp 2aaaae0f1000 00:00 02aaaae0fc000-2aaaae11a000 rwxp 2aaaae0fc000 00:00 02aaaae11a000-2aaaae165000 rwxp 2aaaae11a000 00:00 02aaaae165000-2aaaae1c1000 ---p 2aaaae165000 00:00 02aaaae1c1000-2aaaae26f000 rwxp 2aaaae1c1000 00:00 02aaaae26f000-2aaaae285000 rwxp 2aaaae26f000 00:00 02aaaae285000-2aaaae2c3000 ---p 2aaaae285000 00:00 02aaaae2c3000-2aaaae31a000 rwxp 2aaaae2c3000 00:00 02aaaae31a000-2aaaae366000 rwxp 2aaaae31a000 00:00 02aaaae366000-2aaaae3c2000 ---p 2aaaae366000 00:00 02aaaae3c2000-2aaaae471000 rwxp 2aaaae3c2000 00:00 02aaaae471000-2aaaae47c000 rwxp 2aaaae471000 00:00 02aaaae47c000-2aaaae49a000 rwxp 2aaaae47c000 00:00 02aaaae49a000-2aaaae4c2000 rwxp 2aaaae49a000 00:00 02aaaae4c2000-2aaaae65a000 r-xs 0302a000 fd:02 1873486 /opt/home/brain/jdk1.6.0_25/jre/lib/rt.jar2aaaae65a000-2aaaaeb93000 rwxp 2aaaae65a000 00:00 02aaaaeb93000-2aaab2165000 r-xp 00000000 fd:00 1331672 /usr/lib/locale/locale-archive2aaab2165000-2aaab2174000 r-xs 0017a000 fd:02 2820386 /opt/home/brain/brain45/lib/java/ext/classes12.jar2aaab2174000-2aaab217d000 r-xs 0006d000 fd:02 263886 /opt/home/brain/brain45/lib/java/log4j.jar2aaab217d000-2aaab2190000 r-xs 000e0000 fd:02 263884 /opt/home/brain/brain45/lib/java/lsCore.jar2aaab2190000-2aaab219d000 r-xs 000b4000 fd:02 263888 /opt/home/brain/brain45/lib/java/lae.jar2aaab219d000-2aaab2211000 r-xp 00000000 fd:02 263896 /opt/home/brain/brain45/lib/linux-x86-64/libNodeJava.so2aaab2211000-2aaab2310000 ---p 00074000 fd:02 263896 /opt/home/brain/brain45/lib/linux-x86-64/libNodeJava.so2aaab2310000-2aaab2316000 rwxp 00073000 fd:02 263896 /opt/home/brain/brain45/lib/linux-x86-64/libNodeJava.so2aaab2316000-2aaab25d2000 r-xp 00000000 fd:02 263933 /opt/home/brain/brain45/lib/linux-x86-64/libjigsaw11.so2aaab25d2000-2aaab26d2000 ---p 002bc000 fd:02 263933 /opt/home/brain/brain45/lib/linux-x86-64/libjigsaw11.so2aaab26d2000-2aaab26f5000 rwxp 002bc000 fd:02 263933 /opt/home/brain/brain45/lib/linux-x86-64/libjigsaw11.so2aaab26f5000-2aaab26f7000 rwxp 2aaab26f5000 00:00 02aaab26f7000-2aaab27bb000 r-xp 00000000 fd:02 263931 /opt/home/brain/brain45/lib/linux-x86-64/libjssh11.so2aaab27bb000-2aaab28ba000 ---p 000c4000 fd:02 263931 /opt/home/brain/brain45/lib/linux-x86-64/libjssh11.so2aaab28ba000-2aaab28c2000 rwxp 000c3000 fd:02 263931 /opt/home/brain/brain45/lib/linux-x86-64/libjssh11.so2aaab28c2000-2aaab28c3000 rwxp 2aaab28c2000 00:00 02aaab28c3000-2aaab2b4d000 r-xp 00000000 fd:02 263932 /opt/home/brain/brain45/lib/linux-x86-64/libLaeExprOps.so2aaab2b4d000-2aaab2c4d000 ---p 0028a000 fd:02 263932 /opt/home/brain/brain45/lib/linux-x86-64/libLaeExprOps.so2aaab2c4d000-2aaab2c6a000 rwxp 0028a000 fd:02 263932 /opt/home/brain/brain45/lib/linux-x86-64/libLaeExprOps.so2aaab2c6a000-2aaab2c6b000 rwxp 2aaab2c6a000 00:00 02aaab2c6b000-2aaab2ea3000 r-xp 00000000 fd:02 263930 /opt/home/brain/brain45/lib/linux-x86-64/libLaeNode.so2aaab2ea3000-2aaab2fa3000 ---p 00238000 fd:02 263930 /opt/home/brain/brain45/lib/linux-x86-64/libLaeNode.so2aaab2fa3000-2aaab2fb9000 rwxp 00238000 fd:02 263930 /opt/home/brain/brain45/lib/linux-x86-64/libLaeNode.so2aaab2fb9000-2aaab2fba000 rwxp 2aaab2fb9000 00:00 02aaab2fcd000-2aaab397c000 r-xp 00000000 fd:02 297844 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicudata.so.362aaab397c000-2aaab3a7b000 ---p 009af000 fd:02 297844 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicudata.so.362aaab3a7b000-2aaab3a7c000 rwxp 009ae000 fd:02 297844 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicudata.so.362aaab3a7c000-2aaab3be9000 r-xp 00000000 fd:02 297846 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicui18n.so.362aaab3be9000-2aaab3ce8000 ---p 0016d000 fd:02 297846 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicui18n.so.362aaab3ce8000-2aaab3cf4000 rwxp 0016c000 fd:02 297846 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicui18n.so.362aaab3cf4000-2aaab3cf5000 rwxp 2aaab3cf4000 00:00 02aaab3cf5000-2aaab3e51000 r-xp 00000000 fd:02 297845 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicuuc.so.362aaab3e51000-2aaab3f51000 ---p 0015c000 fd:02 297845 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicuuc.so.362aaab3f51000-2aaab3f5f000 rwxp 0015c000 fd:02 297845 /opt/home/brain/brain45/platform/linux-x86-64/os/lib/libicuuc.so.362aaab3f5f000-2aaab3f61000 rwxp 2aaab3f5f000 00:00 02aaab4000000-2aaab49ed000 rwxp 2aaab4000000 00:00 02aaab49ed000-2aaab8000000 ---p 2aaab49ed000 00:00 02aaab8000000-2aaab8013000 r-xp 00000000 fd:02 1870528 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnet.so2aaab8013000-2aaab8114000 ---p 00013000 fd:02 1870528 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnet.so2aaab8114000-2aaab8117000 rwxp 00014000 fd:02 1870528 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnet.so2aaab8117000-2aaab811e000 r-xp 00000000 fd:02 1870529 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnio.so2aaab811e000-2aaab821d000 ---p 00007000 fd:02 1870529 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnio.so2aaab821d000-2aaab821f000 rwxp 00006000 fd:02 1870529 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/libnio.so2aaab8337000-2aaab833f000 r-xs 00115000 fd:02 1870645 /opt/home/brain/jdk1.6.0_25/jre/lib/resources.jar2aaab844a000-2aaab844e000 r-xp 00000000 fd:00 1442145 /lib64/libnss_dns-2.5.so2aaab844e000-2aaab864d000 ---p 00004000 fd:00 1442145 /lib64/libnss_dns-2.5.so2aaab864d000-2aaab864e000 r-xp 00003000 fd:00 1442145 /lib64/libnss_dns-2.5.so2aaab864e000-2aaab864f000 rwxp 00004000 fd:00 1442145 /lib64/libnss_dns-2.5.so2aaabc000000-2aaabc3a3000 rwxp 2aaabc000000 00:00 02aaabc3a3000-2aaac0000000 ---p 2aaabc3a3000 00:00 02aaac0000000-2aaac01e8000 rwxp 2aaac0000000 00:00 02aaac01e8000-2aaac4000000 ---p 2aaac01e8000 00:00 02b8b1da3a000-2b8b1da3c000 rwxp 2b8b1da3a000 00:00 02b8b1da4f000-2b8b1da56000 r-xp 00000000 fd:02 1870524 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/jli/libjli.so2b8b1da56000-2b8b1db57000 ---p 00007000 fd:02 1870524 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/jli/libjli.so2b8b1db57000-2b8b1db59000 rwxp 00008000 fd:02 1870524 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/jli/libjli.so2b8b1db59000-2b8b1db5c000 rwxp 2b8b1db59000 00:00 02b8b1db5c000-2b8b1e476000 r-xp 00000000 fd:02 1870559 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/server/libjvm.so2b8b1e476000-2b8b1e578000 ---p 0091a000 fd:02 1870559 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/server/libjvm.so2b8b1e578000-2b8b1e72d000 rwxp 0091c000 fd:02 1870559 /opt/home/brain/jdk1.6.0_25/jre/lib/amd64/server/libjvm.so2b8b1e72d000-2b8b1e767000 rwxp 2b8b1e72d000 00:00 07fff8d05a000-7fff8d070000 rwxp 7ffffffe9000 00:00 0 [stack]ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0 [vdso]

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

For clarification, here are a few questions:
- Exactly which node are you using? (From the question I'm guessing either the JDBC Store, or possibly the JDBC Execute nodes)
- What is the value of the CommitFrequency paramameter? (If it is not set, you might try 1000 or 10,000 and doing this is likely to improve performance)
- If you are using the JDBC Store node, what is the value in LoadMethod? (optimized is significantly faster, but uses memory differently)

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jim251

Here are my responses...

- Exactly which node are you using? (From the question I'm guessing either the JDBC Store, or possibly the JDBC Execute nodes)
JDBC Store (Core) Lavastorm Version 4.5.2

- What is the value of the CommitFrequency paramameter? (If it is not set, you might try 1000 or 10,000 and doing this is likely to improve performance)
Blank/default or 1 are the only conditions we were able to run with this parameter. Tried 1000 and got this error:

Exited with non zero status

Installing log (04/09/2014 19:49:32) ...
2014-04-09 19:49:33; Level: 4; Type: 0; Desc: "prepare statement error: java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for JDBC]Can't start a cloned connection while in manual transaction mode."; File: "DBLoader.java"; Line: 94; Context: "nodeMain"
Uninstalling log (04/09/2014 19:49:33) ...

- If you are using the JDBC Store node, what is the value in LoadMethod? (optimized is significantly faster, but uses memory differently)
Blank/Default. I haven't tried any variation of this yet. What would you recommend?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jim251

Get this error when running with the LoadMethod = Optimized:

Exited with non zero status

Installing log (04/09/2014 19:58:45) ...
2014-04-09 19:58:46; Level: 4; Type: 0; Desc: "prepare statement error: java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for JDBC]Can't start a cloned connection while in manual transaction mode."; File: "DBLoader.java"; Line: 94; Context: "nodeMain"
Uninstalling log (04/09/2014 19:58:46) ...

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rabbott

Hi All,

Issues of this nature should be reported to Lavastorm Support, rather than on the Lavastorm user community forums. There's a sticky note posted here on this topic: Technical Support.

The issue referred to above will be fixed in the next LAE version releases, 4.6.2 and 5.0, both expected sometime this month. Here is the release note:

"The JDBC Query, JDBC Store, and JDBC Execute nodes were based on a legacy Java node architecture which heavily relied on Java JNI. These nodes have been refactored onto the current Java node infrastructure which is a pure Java implementation to fix a number of existing issues with the JNI implementation. The functionality of these nodes has not changed."

As for a workaround, the one other customer that has encountered this problem also had a large amount of data pushing through the node. What they did was to divide the data stream across multiple nodes, so that say 500K records get pushed through each, and maybe cascade the nodes so that they execute independently using clocks. I believe whatever solution was devised did alleviate the issue for a time, but it later resurfaced. The precise conditions under which the error occurs are still unknown. The occurrence is intermittent. One other thing would be to make sure you're working with the latest JDBC driver for your database.

- Bob

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: xathras

Hi jim251. At gogo we experienced this issue

As a work around, make the use of your cores and run a hash split, running nodes in parallel, look at the chunk sizes you're passing through, the commit frequency etc.
If you can where possible use native OCI (oracle), ODBC drivers etc. However, I recognize like gogo, sometimes that isn't possible.

We spent some extensive time working with Lavastorm on this and we know the fix will resolve it for you

Wayne