LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Bryt Park

Hi

I am trying to piece together certain fields from multiple independent source files.

However, there are no common field/s among these sources, therefore Join would probably not work.

All these source files contain ONLY ONE record, they are produced by multiple complicated VBA programs.

I have tried Cat node, but it won't put these field into one single record, please kindly advise.

Source A:
Field_A_1, Field_A_2, Field_A_3

Source B:
Field_B_1, Field_B_2, Field_B_3

Source C:
Field_C_1, Field_C_2, Field_C_3

Expected Result:
Field_A_1, Field_B_2, Field_C_3

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mgajdosik

Use join node or lookup node and use 1 and 1 as keys instead of fields... Only if you have 1 record in each input... otherwise you will get cartesian product.

node:Join_Inner
bretype:core::Join Inner
editor:sortkey=567421ad4ded589d
input:@45781ca80c2802d0/=
input:@45781ca971fe502f/=
output:@45781cad02e051b0/=
prop:LeftInputKey=<<EOX
1
EOX
prop:RightInputKey=<<EOX
1
EOX
editor:XY=680,80
node:Sort_2
bretype:::Sort
editor:shadow=457823031055565e
input:@40fd2c743ebf4304/=
output:@40fd2c746a2a3b47/=
end:Sort_2

node:Bypass_3
bretype:::Bypass
editor:shadow=457823031db44468
input:@45782010749f0c96/=
input:@457820124fef7658/=
output:@40fd2c7436717256/=
end:Bypass_3

node:Join
bretype:::Join
editor:shadow=457823031dc50173
input:@40fd2c745b6d7704/=
input:@40fd2c74504921cd/=
output:@40fd2c7430f76546/=
end:Join

node:Bypass
bretype:::Bypass
editor:shadow=4578230322105ec6
input:@45782010749f0c96/=
input:@457820124fef7658/=
output:@40fd2c7436717256/=
end:Bypass

node:Sort
bretype:::Sort
editor:shadow=457823032cda57c6
input:@40fd2c743ebf4304/=
output:@40fd2c746a2a3b47/=
end:Sort

end:Join_Inner

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

Going all technical...

This only works when both input files are smaller than the available RAM memory.
If you have a 50gb file on the left and use a 1=1 join, the node will blow up with a memory allocation error.

Most people know about not putting too large of a file into the RIGHT pin of a LOOKUP, but they often don’t know that JOIN will blow up also.

JOIN tries to hold all of the left input records in memory until it knows that they have been matched to the record on the right…. in the case of a 1=1 join, that means it tries to hold ALL of the records.. so you’re limited to the size of physical memory on the LEFT.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mgajdosik

AS mentioned my solution is only for 1 input record for each input. If you want to concatenate records horizontally and you have multiple rows and just want to merge these together, what you can do is to add to each record unique Id using filter node in which you would have following syntax

id = execCount
emit *, id

then you can combine the records based on Id, assuming you want to achieve that 1st row from 1st input should be aligned with 1st row from 2nd input etc.

Marek