LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: spalluru

Hi ,

We are planning to use lavastorm servers as shared platoform for many teams. Wanted to know is there any way
we can limit (put soft/hard limits) on CPU consumption by each of the job run so that it can facilitate fair sharing of CPU among
all the jobs running on the lavastorm server

-Sreekanth

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

LAE provides a Thread Pooling feature which allows you to control resource consumption by various groups of users. For example, you could create a pool of 4 threads dedicated to the user account employed to execute automated runs and another pool of 4 threads which can be used by users performing ad-hoc runs via the client application. This feature is documented here: http://www.lavastorm.com/assets/Thre...figuration.pdf

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: spalluru

Originally posted by: mmarinelli
					

LAE provides a Thread Pooling feature which allows you to control resource consumption by various groups of users.  For example, you could create a pool of 4 threads dedicated to the user account employed to execute automated runs and another pool of 4 threads which can be used by users performing ad-hoc runs via the client application.  This feature is documented here: http://www.lavastorm.com/assets/Thre...figuration.pdf

			

Hi ,

Thanks for pointing to the doc. In our case we sort of planning to run all graphs (BRX) with single lavastorm application user using laeController if so, I feel this doc will not be applicable to us please confirm ?
. I have few questions on threads created by lavastorm while running the graph

1) Assume 2 BRX/graphs are running in parallel, how are threads allocated to these jobs ? assume ls.brain.controller.farm has below value set
ls.brain.controller.farm=( "localhost:${ls.brain.server.port}:20" )

2) With above setting will two jobs will spawn 20 threads in parallel total 40 ( 20+20) and all jobs will competing for available CPUs?
3) or only 20 threads (as per farm setting) will be shared across two jobs and total threads spawned on system by lavastorm OS user will be only 20 ?
4) Or each node present in graph will try to spawn 20 threads (upper limit) to complete its job ?

5) Is there any way I can monitor how many threads were used by each graph ? is this is logged into any of the files?

-Sreekanth

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

Hi Sreekanth,

Yes, generally most people run all graphs under the same application user. But I don't see an issue with creating multiple users for batch jobs unless maybe there is something in your environment I'm not aware of. This would be a good way of managing the thread pooling.

Also, you can override parameters for individual graph runs using the -P option. So when you execute the laeController command add -P 'ls.brain.controller.farm = ( "localhost:7721:9" )' (changing the 9 to the number you want and using the correct port number which is usually 7721)

So to answer your specific numbered questions:
1) Each job will get its own 20 threads to use as it pleases (unless limited by thread pooling)
2) Two jobs will spawn in parallel 40 (20+20) competing for available CPUs. But remember also that depending on how the graph is built, a job may only be able to use a few of the 20 threads at any one time.
3) There is no sharing, you'd have to use thread pooling to control sharing.
4) Exactly, each graph will be able to run up to 20 threads depending on the graph construction
5) The best answer I am aware of is that you can use the "-brdlog <logfilename>.brd" option on the command line with laeController. This causes the controller to create an output log in the BRD format that is available for analysis by an LAE graph. You'll be able to read this log file, and figure out the start and end times for each node. Then the strategy that I've seen work is to check each 5 minute increment and calculate the number of nodes running at that instant in time. (With this information, it seems like there should be a good strategy that you could use to calculate the peak number of threads used by a run, but nothing is coming to mind yet)

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

Over night I had an idea how to process the BRD Log to find the number of nodes being run. Attached is the results.

To use, go to the run parameters to specify the directory and a pattern for the BRD file(s) to process. Also specify the size of the time units to use in the reports.
Attachments:
AnalyzeBrdLog.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: xathras

Hi Sreekanth, like you I shared this concern and with the help of Lavastorm I implemented Thread Pooling. However, it doesn't directly resolve the problem, you can limit the resources and handlers available. But you cannot enforce Lavastorm specific threads can only consume 50% of the CPU per say. My research brought me to a tool called cpulimit. Take a look here and here for more information. If you have Lavastorm installed under a service user account on Linux then you could throttle at the Linux service user account: http://www.cyberciti.biz/faq/cpu-usa...ter-for-linux/ and http://cpulimit.sourceforge.net/. You can also play with usage limits, http://stackoverflow.com/questions/4...-user-in-linux. But be careful is my recommendation here - particular on file usage limits if you have some large graphs the open files might be high