LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: burke

I have a simple graph which runs perfectly fine in BRE, however running as a BRX it consistently fails.
[code:1]
Node Multiple_BRD_Files (496b769f7ece1aa5) errored: Error while starting execution of node 'Multiple_BRD_Files'
caused by: Waiting for outputs.
[/code:1]

The graph uses a directory list node to find all files in the directory.

A filter to extract file name info (files are named in a structured manner which includes a date for the data they contain).

Another filter to build a list of file names which meet the date requirements for data requested.

Then the Multiple_BRD_Files node to read and output the data based on above filters.

The output is then sent to an Output BRD File node.

This runs perfectly and as expected from within the BRE yet when run on the server as a BRX it fails as per above message, any ideas?

Also to me it seems (see output below) this graph when run as a BRX does not wait for the preceding nodes to compete before running the following node as it does in the BRE.

[code:1]
[brain@abnsolava01 scripted]$ ./E2E_run_YYZ_Daily_v2.sh
Graph YYZ_daily Started.
Node Directory_List (496b769f7cf24042) Started.
Node Directory_List (496b769f7cf24042) Node Processing.
Node Extract_File_Name_Date (496b769f2adb7024) Started.
Node Extract_File_Name_Date (496b769f2adb7024) Node Processing.
Node Directory_List (496b769f7cf24042) Node Completed.
Node Multiple_BRD_Files (496b769f7ece1aa5) Started.
Node Multiple_BRD_Files (496b769f7ece1aa5) errored: Error while starting execution of node 'Multiple_BRD_Files'
caused by: Waiting for outputs.
Graph YYZ_daily errored: Error in execution
caused by: Still executing.
Graph Completed with Errors.
Graph YYZ_daily errored: Error handling message
caused by: Error handling null message: Error in execution: Still executing.
State File: file:/data2/tmp/brain/YYZ_daily/temp.abnsolava01.26794.0.1231870199516.brs
BRAIN Controller exit status: 6
There may be no data or data with errors.
[/code:1]

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ltolleson

Try to add a clock between each node to ensure they are running in sequence. It is likely that your server is setup to run the BRX in "Streaming" mode in which case it could potentially start the a Filter node before the other nodes complete and run these nodes in parallel.

The default setting for the ls.brain.controller.scheduler property, which controls this functionality, is "streaming". I believe this could be why this is happening. To change this property to "non-streaming" change the value of ls.brain.controller.scheduler in the ls_brain.prop file located in the conf directory under the BRAIN Server install. There are examples of the appropriate value located in that file. If this property is changed the BRAIN Server will need to be re-started.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: burke

Thanks for the quick reply.

In checking the config setting for "ls.brain.controller.scheduler" I see it is "default" or as you noted 'streaming'.

Using the clocks did work however I would prefer to make the change in the config.

I am wondering what impact this config change may/could potentially have on other graphs which are run in the BRE?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ltolleson

Since you are reading a list of files from a directory I can assume the number of records is quite small. With only a small number of records, clocking the nodes will have little impact on the performance of your graph and will only be affecting this one graph.

However, if this property is changed to "non-streaming" mode it will affect every graph that is run on that server. Streaming mode by it's very nature performs better than Non Streaming since it allows a node to begin execution as soon as it has data available and before it's predecessor is complete. This is not true for all nodes (ie the sort node), but it true for many of the node types.

I would recommend for now clocking these few nodes and leaving the server in "Streaming" mode.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: burke

Thanks again for the quick reply.

I will use the clock for this graph, and forgo setting the server configuration.

This particular graph is quite simple whereas most of our others are quite complex and involved.

Thanks for your help!