LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: svimes

Lets say I have Excel01.xls --> Excel08.xls in a directory

I use a Directory node to pick up the eight filenames
To get the data I can connect an excel node to the directory node, add seven more outputs to give me eight, put "FileName" in the FileName Expr field and F5.

However, subsequent nodes will only start once all data from all excels 01 - 08 are loaded. Is there any way to parallel process this so smaller files can carry on once loaded without waiting for the others? I know I can split the result from the directory node into eight and attach eight execl nodes but it seems a bit clumsy.

Thanks

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: andycooper

Hi,

Do you need eight separate outputs for what you are trying to achieve?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: svimes

Andy - Not necessarily but I want to start processing the data for a given excel workbook once it has loaded, without it having to wait for the others to complete their loads. The initial set of nodes that will be applied to all excel spraedsheets will be the same libarary composite node.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: andycooper

Are you using the desktop or server edition of LAE and what version?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: svimes

LAE 4.6 server version

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: andycooper

Basically, before we can decide what to do with each file we either need to know in advance how big each file is (which I assume isn't known?) or we need to run all files into LAE to evaluate then. If you know in advance that say sheets 1-4 are big and 5-8 are small then we could simply separate them out and use two nodes. Will the size of data you are working with reduce later on in the graph? If not then are you just shifting the problem later on in the process. It's not easy to comment 100% without understanding the size of the data and what ultimately you are trying to achieve.

The Hash Split node allows you to improve the speed of processing the data but I think you still will have the same issue as you'd need to apply this to each sheet with large data volumes when it works best with a single source.

It might be easier to drop me an e-mail with more details and I can look at it offline.

Andy