LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sassy1

I have a very complicated graph and need to break it into 3 parts: an "Extract" graph that stages the data, an "Enrichment" graph that processes the data, and an "Output" graph that deals with the 4 outputs (going to .csvs).

Support suggested the following: "If you wish to run, say, 3 graphs in a specific order, you would use the CLOCK connections to link the Execute BRX nodes together, to ensure they will run in the order you wish. And each .brx you execute can in-and-of-itself have an Execute BRX node that will start yet another level of sub-graph."

Does anyone have an example of a graph calling a graph or is there any documentation on this?

Another solution suggested was "You would do so (in general) through a shell script that executed the graphs in order and setup whatever parameters each graph required to run correctly. " Are there examples of this? I haven't worked with parameters yet so I need some guidance on how to code/use these in shell scripts passing to graphs.


Any help would be very much appreciated.

Thanks!

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: alyon

See attachment.

It was provided on this forum before.
It shows an example of first graph calling second graph by passing parameters to it and using execute brx node.


Cheers
Anshul
Attachments:
ExecuteBrxExample.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

Also, it is not clear to me, do the graphs need to be scheduled to run consecutively? If you just break a graph into three parts, you want to do a proper handoff of the data from one graph to the other using a staging area and check pointed files (files stored in known named locations, using the BRD File and Output BRD. But if you are running the graphs manually, you don't need to mess with clocks, executeBRX, or shell scripts. Those options are if you want to have a scheduled or automated way to run one after the other. It wasn't clear to me in your initial post if you need that....

Cheers
Rich

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sassy1

Thank you Anshul & Rich.

I'll look at the example Anshul provides and hopefully this will get me what I need.

Hello Rich - yes they do want this to be an automated run of the graph. They want it put into crontab to be run automatically at a scheduled time once a month. The graph got so large that we thought breaking it down into parts would be appropriate, however, all the parts need to be completely re-run every month. If you have any suggestions or other options, please let me know.

Right now it runs fine manually, but to automate it is obviously a more complicated process. One option I suggested is to parameterize the graph for the inputs, but leave it in a single graph running in agressive mode. That way, you can avoid the hand off of data to separate graphs. Would you advise that or to break these up? Please let me know.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

Ok, so if we are attempting to "chain" the graphs, another way to do this is using the scripts I worked with Curt on, that can schedule consecutive graph executions. You will still want to create the BRD output and input files in each of the graphs to "hand off" the data from one graph to the other, but you could avoid doing the Execute BRX step, and opt to use the cron-related scripts that we worked on to run the graphs in sequence. In order to do that, you want to run the graphs all on the same cron script command (I think it was runGraphs or something like that), which can run multiple graphs, one after the other. If you attempt to run them as separate cron jobs, you have the co-ordination problem of one graph needs to finish, and you may not know when it finishes...we worked on making sure your script behaves properly for you, for just that reason.

Cheers
Rich

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sassy1

Thanks Rich. Curt left the company a few months ago. A lot of what you did with him was either not well documented and/or passed along to me. I did find the RunGraphs shell script and can try going from there.

Not sure if it's alright to ask in this thread, but I'm struggling with parameters. If I have a node with staged data (instead of getting the data from the database), how can I pass in a start_date and end_date for this staged data node? I currently have hard-coded dates in these fields. To get this graph automated, I need to understand how to get this done. I searched the forum, but didn't find anything related to passing parameters to staged data nodes. Are there examples of this? Please direct me if possible.

Thank you - as always