LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dhrobertson

Hi guys,

I am starting to doubt myself here so please correct me if I am wrong....

My client wants to be able to schedule graphs to run automatically. I have explained to them that in order to be able to do this, firstly I need to be able to test that the graph(s) which I have developed on my desktop edition of Lavastorm, will run successfully via the server. I need to check runtimes, that the paths are correct relative to the server etc. etc ie. I need to be able to connect to the server on port 7721 and run the graph end-to-end to double check it all works. once I am happy with that, I need to create an executable of the brg by creating an LXA file at which point this can be uploaded to the server and scheduled via whatever scheduler they use along with any parameters the graph may need. that is my understanding of how I get a graph into an automated production environment. or at least that is the way I have done it using logistics manager in a windows environment... this client is using linux which I have not used before so hence I am unsure here...

They are wanting me to just hand over the brg's I have created on my desktop and they will go ahead and schedule these via their scheduler software....

I have explained many times it does not work like this and you can't just schedule a brg and it will work, but I am starting to doubt myself.... please can someone confirm that I am not being stupid and that my way is the correct way of getting a graph into production and just passing them a brg is not going to work??? or am I actually wrong in this assumption?

thank you,

regards

Douglas

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

You're not entirely crazy. <grin>

The term "Whatever scheduler" is your first issue. The BRG would not be used directly by their scheduler.. it has to be converted to a BRX or a LXA.

- To run graphs from a command line, (this would include cron, or ctrlM, or other linux program) you need to use the program laeController which requires building a BRX.
- To run graphs in the Lavastorm Automation Services, you need to build a LXA.

In either case, consideration must be made if you need external graph parameters. Those have to be setup so as to feed the graph properly.

Yes, your graph (BRG) has to be properly configured and tested so as to name the input and output tables and files, handle date processing, etc. I don't know that YOU would need access from BRE to the LAE server yourself, but without such access, diagnosing issues is going to be a severe case of trial and error.

Now, I do have a customer or two for whom I create and test BRGs on their test server, and then I hand them the BRGs. They then examine my code for security issues, and then create the BRX with their copy of BRE. Those graphs have various system and database parameters in place such that if you run the BRX on the test machine, it connects to the test database, but if you use the same BRX on the production machine, the parameters there direct you to the production database, etc.

It takes a bit of time and trial-and-error to get all the settings in place so that a scheduling system is running the way you wish, but after you go thru that, adding a new graph to the schedule is not significant. At my main client, I can add a new (working/tested) graph to the scheduler within 3-5 minutes.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dhrobertson

Thank you Stony,

That makes sense. I was assuming that an LXA is used whether using the automation services or scheduler so you have clarified that one. They need BRX's.
My concern is I don't believe they have any experience in Lavastorm at all and thus if I hand them a brg I'm not entire convinced they would know what to do with it.
Certainly if I have not been able to test the graph via the server myself and they don't know how to do it either, as you said, it is complete hit and miss.
I have highlighted my concerns about all this so that is really all I can do, but I just don't want them turning around and suggesting I have handed them a turd due to the fact I have not had the ability to test it on the server at all.
At least I think I'm clear on what the requirements are for being able to schedule via a scheduler.
are there any documents out there with examples of the command lines for a scheduled job like a cron job for example, with a few parameters. just to get a feel of what it would look like in the scheduler?

thanks

Doug

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

I would ask, how much do you know about the server environment? If you have "enough" information, then you could design the graph with hard-coded names, etc, and you wouldn't need parameters. But, that is often not the case, you want to enable a bit of variability into the graph, such as changing dates, server passwords, etc. It's the things that need to (or want to) change that need to be parameterized.

At the absolute simplest:
To send parameters to the graph, there are two methods:
or
I prefer the first form because it allows less typing on the command line itself.

Inside your graph, to use either form, you use {{*myParm*}} - note that in linux you add "p_" but not inside the graph. Like with all parameters spelling and capitalization is important.

For my customers, I wrap the call to laeController into a generic script and then pass parameters to the script which are then passed to the controller. I also do a good bit of logging and error capture/notification. Setting all that up would be more difficult if you don't have server access.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: dhrobertson

Thank you Stoney,

I currently don't know anything about the environment which is not helpful. what I do know is that the majority of the parameters I am using are dealing with filepaths. the reason I have done this is that possible the location of the source files changes or destination files. all source and destinations files are not local to the lavastorm server... they are on various Windows server shares.

An additional question I have around this is that when I run one of my graphs manually, it is obviously running under my user context which is an active directory account which has access to the server share locations. so, when the final output node run which at the moment are just creating excel outputs, they work because my user account has access.

When this is automated and the graph still need to write to these locations... it will not be running under my user context any more. When the username and password is specified in the run statements mentioned above, will the username entered have to have access to the server share as well in order for the node to write to it? would I need to add user account which has access to the share into the lavastorm users file in order for it to work.

I'm really struggling with all this due to my lack of Linux experience. I feel like I may need to sit with one of the lava team and work through some of my graphs to see how they can be automated as I feel I wont be able to work this out on my own...