LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mwillett

Hi there,

I have 50 files, all with the same key to allow me to join them, but with different data held in them.

I would like to create one table with all the data.

Obviously I can join these one by one (49 times in total), but is there a faster way to join these that creating 49 look up nodes?

Mike

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: xathras

Hi Mike,

Are they CSV files or similar?
What you could do is the following:

1. Place all the files in a directory and have them in the same format
2. Add a Directory Node to look for the files.
3. Pin the Output node of the Directory Node to a CSV Node. You may think that the CSV Node doesn't have an input but the it will allow it.

You can use the same concept for Multiple BRD Files.

That way all your data is joined. Assuming that all the fields/columns are the same just different data sets

Wayne

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mwillett

Thanks Wayne but unfortunatel\y the challenge is that each file has different headers and a varying number of headers, depending on the file.

I also have another potential challenge in the join in that, in some cases, different files have some (only some not all) columns that are the same.

I expect to do this now is looking more like a manual process rather than automated.

Mike

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: xathras

Mike,

Can you give me an example. See if i can come up with something for you.

Wayne

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

I sent Mike a sample graph.. I'll work thru it with him and then post the solution here.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Gaetan.Patry

Can you share what the solution was?
More specifically, how do you configure the Directory List Node to consolidate all the files listed in its Output in one single CSV file?
Thanks!

Originally posted by: stonysmith
					

I sent Mike a sample graph.. I'll work thru it with him and then post the solution here.
			

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

If you simply add a directory list and then connect that to a Delimited File node, you can read multiple CSVs in a single pass. My solution mentioned above is unnecessary if all the CSVs have the same header.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Gaetan.Patry

I am new to this application (will have formal training in 2 weeks) so I am not that familiar with configuring nodes.

When I try what you suggest, I get an error message saying "Property ls.brain.node.convert,filename Expr not set"
It seem to point to the fact that I don't have anything populated in the FilenameExpr box.

Can you guide me on how I should populate the fields of the Delimited File node, using an example? I tried with the help guide but it did not really clarify what I need to do with FilenameExpr and other parameters.

Thanks!

Originally posted by: stonysmith
					

If you simply add a directory list and then connect that to a Delimited File node, you can read multiple CSVs in a single pass.  My solution mentioned above is unnecessary if all the CSVs have the same header.
			

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Tim Meagher

Hi,

Within the FilenameExpr you simply need to specify the name of the field in the input data which contains the filename to read.
If this is coming from a Directory List node, then the field name should just be 'FileName'

As a general note, if you are having difficulty understanding what a parameter is for, you can hover over the label for the parameter, and help will appear in a tool-tip informing you as to the purpose of the parameter.

To understand more about the node you are trying to use, simply select the node and hit F1, or within the node editor, click "Node Help" at the top right of the dialog.

Regards,
Tim.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Gaetan.Patry

That worked fine.
Many thanks for the quick reply! Well Appreciated!