LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: gmullin

I'm trying to determine how long a particular node is running. It's not my graph and I don't know what user is running it, but here's what I got (user and server name removed).

Here's my top command on the server:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21754 lavadmin 20 0 204m 3436 1824 S 14 0.0 566:10.08 nodeLookup

So I now look up that process using these commands:

user1@servername:/home/user1> ps -ef | grep 21754
lavadmin 21754 27416 13 Dec01 ? 09:28:37 /opt/lavastorm/lae/4.6.1/db/../bin/linux-x86-64/nodeLookup

user1@servername:/home/user1> ls -ld /proc/21754
dr-xr-xr-x 8 lavadmin lavastor 0 Dec 1 15:24 /proc/21754

Here's the current time on the server as I post this:
Thu Dec 4 12:59:00 CET 2014

So am I right in saying that we have a single Lookup node running for almost 3 days? Or has it been restarted at some point?

Can a node hang on the server somewhere and end up in the process list for a long time?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

There are times when the BRE client loses connectivity, and a running node is orphaned such that it will just sit on the server (usually doing nothing) because it is waiting to try to tell BRE that it finished.
It is also possible with the WHILE operator to construct a node that never terminates.

Generally, it is safe to issue the command kill -9 21754 to stop the process.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

As of LAE 4.6.1 there is better traceability. Using the process ID you can find the more information about who is running the node and the node that is running. To me this is a very compelling reason to upgrade.

Here is what the 4.6.1 release notes say:

The temporary files (log files, BRD files, prop files, etc) that are produced when a node runs now include the node handle in the filename.

The files are still stored (as in previous versions) under the directory:
<tmpDir>/<username>/<graphName>/<runName>/

In LAE 4.6.1, the filenames now have the format:
temp.<Hostname>.<pid>.<rand>.<timestamp>.<nodeHand le>.<suffix>
Where <rand> is a randomly generated string to differentiate files generated with the same values for the other filename components.

This filename format change means that given a PID for a node, a system administrator can search the tmp directory to locate temp files corresponding to that node (based on a creation/modification date and time etc), and from there will have information about the user who ran the node, the name of the graph containing the node, the run which was used to run the node, and the handle of the node within that graph. With this information, if the graph is available, the node itself can be found using the "Find" option in BRE, searching using the handle of the node. Issue 4107. Support Calls CS-4012, CS-3876, CS-3598, CS-4188 and CS-4352.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: gmullin

Thanks guys, this was massively helpful.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: snbkumar

Hi Jones,

We have lae.version=4.5.4.0.0. Is there a way to get the timestamp in the log?

Currently we have:
Node <NODE1> (5175f55745880dae) Started.
Node <NODE1> (5175f55745880dae) Node Processing.
Node <NODE2> (5175f55706550152) Started.
Node <NODE2> (5175f55706550152) Node Processing.
Node <NODE3> (5175f55700f54f29) Started.
Node <NODE3> (5175f55700f54f29) Node Processing.
Node <NODE4> (5175f55710b25b62) Started.
Node <NODE4> (5175f55710b25b62) Node Processing.
Node <NODE2> (5175f55706550152) Node Completed.
Node <NODE1> (5175f55745880dae) Node Completed.

And we would like to have:
Node <NODE1> (5175f55745880dae) Started at 20150209 02:01:00.
Node <NODE1> (5175f55745880dae) Node Processing at 20150209 02:02:00.
Node <NODE2> (5175f55706550152) Started at 20150209 02:01:00.
Node <NODE2> (5175f55706550152) Node Processing at 20150209 02:02:00.
Node <NODE3> (5175f55700f54f29) Started at 20150209 02:03:00.
Node <NODE3> (5175f55700f54f29) Node Processing at 20150209 02:04:00.
Node <NODE4> (5175f55710b25b62) Started at 20150209 02:05:00.
Node <NODE4> (5175f55710b25b62) Node Processing at 20150209 02:06:00.
Node <NODE2> (5175f55706550152) Node Completed at 20150209 02:07:00.
Node <NODE1> (5175f55745880dae) Node Completed at 20150209 02:08:00.

Thanks,
Bharath

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

There is another log file MyGraphName.log.brd

If you can find that, there is a column in there for all start/end times.