LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only

  • 1.  BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 02:03

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Alphabetwolf

    Hi LavaStorm Community,

    I've got a question concerning BRS files - specifically those created by the Execute BRX node.
    Now, I've edited the python script of the Execute BRX node to save the BRS file for its execution via this simple line:

    args.extend(['-u', username, '-p', password, '-brdlog', '"%s"' % brdlogFile])
if doProps:
	args.extend(['-P', 'ls.prop.config="%s"' % propFile, '"%s"' % brx ])
else:
	args.extend(['-s', propFile, '"%s"' % brx ])
	
#
# custom section start
# - sets the runName for this run
# - also adds an extra argument to the execution, causing it to write the
# - BRS file to the output folder provided (if a BRSOutputFolder was provided)
#
self.node.logHigh(brsOutputFolder)
self.node.logHigh('"%sBRS/%s.brs"' % (brsOutputFolder, runName))
args.extend(['-r', '"%s"' % runName])
if brsOutputFolder != "":
	args.extend(['-dump', '"%sBRS/%s.brs"' % (brsOutputFolder, runName)])
#
# custom section end
#

cmd = string.join(args, ' ')
self.node.logInfo(cmd)
process = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
exitCode = 0;
    This works fine and creates a .brs file whenever the Execute BRX node runs. Now, however when I import the .brs file to inspect it I noticed that while most nodes work fine and can have their output inspected as expected, some of them do not seem to have any output linked on their output. As an example, for a healthy node it may have 
    output:out1=file:/temp/lavastorm/ssapp0886/tmp/...../temp.ssapp0886.1514.0.1470120274000.brd
    , while another may have 
    output:out1=cstream:ssapp0886:36407
    instead, which does not let me inspect the output of the node.

    Would anyone on this forum have an idea as to what is causing this, and how I can ensure that all my nodes have their output properly recorded in the .brs files?


    Any help is greatly appreciated.


  • 2.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 06:50

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    You are correct in your findings.. there's no output temp file there.
    CSTREAM means that the nodes were connected thru a TCP/IP pipe.. direct memory access between the nodes. No temp file is written to disk.

    This mode is called STREAMING, and can significantly improve performance.
    The positive side is that no time is spent waiting on Disk IO - the data goes directly from one node to the next. It also requires zero space on your temp drive.
    The one downside, as you've discovered, is that you can't inspect the temp data after a run finishes.

    When looking at a completed graph in BRE, you'll see that a pair of nodes has executed, but their input/output pins are white instead of green... If you look on the Status Tab, you'll see the same "cstream" indicator. This means that the connection between the nodes was streamed.

    You can turn this behavior on and off at a system level by changing this parameter:
    ls.brain.controller.scheduler = "default"
    # "debug", "non-streaming", "streaming", "default"

    You should be able to control streaming thru ExecuteBRX the way you did with args.extend() I think you would use "-P" to add such parameters.

    My memory recalls that there may be some way to turn it on and off at the individual node level (inside a graph), but we'll have to search for that.


  • 3.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 18:46

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Alphabetwolf

    Thanks that is exactly what I'm looking for. I made the required change to the code by adding the ls.brain.controller.scheduler parameter, but it doesn't seem to be working. I'm probably just missing a small detail.

    This is my updated code:
    args.extend(['-u', username, '-p', password, '-brdlog', '"%s"' % brdlogFile])
if doProps:
	args.extend(['-P', 'ls.prop.config="%s"' % propFile, '"%s"' % brx ])
else:
	args.extend(['-s', propFile, '"%s"' % brx ])
	
#
# custom section start
# - sets the runName for this run
# - also adds an extra argument to the execution, causing it to write the
# - BRS file to the output folder provided (if a BRSOutputFolder was provided)
#
self.node.logHigh(brsOutputFolder)
self.node.logHigh('"%sBRS/%s.brs"' % (brsOutputFolder, runName))
args.extend(['-r', '"%s"' % runName])
if brsOutputFolder != "":
	args.extend(['-dump', '"%sBRS/%s.brs"' % (brsOutputFolder, runName)])
	# set the scheduler to default instead of streaming so that all of the nodes
	# have output data stored against them
	args.extend(['-P', 'ls.brain.controller.scheduler="default"'])
#
# custom section end
#

cmd = string.join(args, ' ')
self.node.logInfo(cmd)
process = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
exitCode = 0;


  • 4.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 21:06

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    Try formatting it this way:
    p_ls_brain_controller_scheduler


  • 5.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 22:02

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Alphabetwolf

    I tried that formatting now and it isn't making a difference.

    # set the scheduler to default instead of streaming so that all of the nodes
# have output data stored against them
args.extend(['-P', 'p_ls_brain_controller_scheduler="default"'])


  • 6.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-02-2016 23:14

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    Set it to "non-streaming"


  • 7.  RE: BRS Files Created from Execute BRX Node

    Employee
    Posted 08-03-2016 00:09

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Alphabetwolf

    That did the trick! Thanks a lot for your help in this. I didn't think default wouldn't work, because by default the graphs had always been outputting their BRS as non-streaming, but apparently non-streaming is what I was looking for.

    Final code for anyone that stumbles across this thread later with the same question:
    # set the scheduler to non-streaming instead of streaming so that all of the nodes
# have output data stored against them
args.extend(['-P', 'ls.brain.controller.scheduler="non-streaming"'])