LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dwong80

Has anybody seen this error message before? We are running LAE v3.

"Exception java.lang.OutOfMemoryError: requested 8160 bytes for GrET* in /BUILD_AREA/jdk1.5.0_11/hotspot/src/share/vm/utilities/growableArray.cpp. Out of swap space?"

We seem to have enough disk space which I ssume is the "swap space" it is talking about.

I notice that we have a bunch of processes running. I wonder if they are hung?

brain 92 4 0 05:25:50 ? 73:35 java -classpath /opt/lavastorm/lib/java:/opt/lavastorm/lib/brain.jar:. -Djava.e
brain 6265 1 0 Jul 28 ? 0:52 /opt/lavastorm/db/../bin/Solaris/lssort
brain 6266 1 0 Jul 28 ? 0:40 /opt/lavastorm/db/../bin/Solaris/join
brain 641 1 0 Jun 03 ? 393:49 /opt/lavastorm/jdk1.5.0_11/bin/java -Djava.endorsed.dirs=/opt/lavastorm/jakarta
brain 4 29999 0 05:25:01 ? 0:00 /bin/bash /opt/lavastorm/brain/brx/ad_ncim
brain 1934 1903 0 17:27:21 pts/1 0:00 ps -efa
brain 24950 1 0 Jun 18 ? 1:40 /opt/lavastorm/db/../bin/Solaris/filter
brain 29999 190 0 05:25:01 ? 0:00 sh -c /opt/lavastorm/brain/brx/ad_ncim >/opt/lavastorm/brain/brx/ad_ncim.log 2>

What is the recommendation here? The process seems to be stuck.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: timonk

In the vast majority of cases where a LAE graph encounters memory issue, the cause is related to the use of Lookup/Join/XREF type nodes trying to deal with large and/or poorly constrained data sets.

Lookups, XREF's and Joins operate by building lists of the data they are working with directly into memory. Lookup and XREF especially are not recommended for large data sets because of this.

If the statements being used to join/cross reference the data are not precise enough, you can end up with truly enormous cartesian products of your initial data, resulting in far larger resultset sizes than the initial inputs would suggest (I have seen 250k records become hundreds of millions). This can quickly consume the available memory.

Is your graph utilizing such operations?

Regards
Timon Koufopoulos
MDA Support.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

Because this is a "swap" error, I would check your unix configuration.

SWAP space on a *nix box is usually a separate disk partition, and it's not entirely uncommon for that partition to first be setup too small. You may need to check with your aministrator to see if the "swap partition" is configured big enough for your installation.

Yes, it was probably triggered by a Join that is trying to produce too many records, but if you know that the Join is well-behaved, then I'd check the swap partition size.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

As far as the older processes.. if you KNOW that nobody is still running one of those nodes, then it's likely that they are stuck. In a lot of cases, you should never see a "date" next to a node for the start time, especially not something over a month old.

Your process #92 here is your brain server.. you don't want to kill that one, but that Join/Filter/Sort can probably be killed.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dwong80

Hi there.

Before we read your reply, we killed process #92 to try to get things running again. We see a new process again.

brain 2048 0.5 3.2817152250272 ? S 18:01:00 61:08
java -classpath /opt/lavastorm/lib/java:/opt/lavastorm/lib/brain.jar:. -Djava.ext.dirs=/opt/lavastorm/lib/java/ext:/opt/lavastorm/jdk1.5.0_11/jre/lib/ext -server -Xms512m -Xmx1024m com.lavastorm.brain.server.Server

So it appears to be running again? However, now we are getting i/o stream errors. Could this be a result of killing process #92?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dwong80

Problem solved by killing process and re-starting process on Solaris