LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only

  • 1.  Field Delimiter Node issues with semicolon and quotes

    Employee
    Posted 03-14-2016 03:34

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: laurrieta

    Hi

    I'm having issues when converting a flat file: after reading multiple notes and doing the changes, the result is not the expected one. In Source Data.txt, the fileds are delimited by ';' and double quotes. You can see also the results with Field Delimiter node (records are broken but the least, but not the fields and quotes are still there), and CSV (it's ok, except the record is in one column). In both cases, these nodes are taken the data source from Get FTP node, so the constraint is that process should be fully automated with no manual intervention in the source file (TextPad and similars are not an option)...

    I'll appreciate your help to sort it out ...
    Kind regards
    Attachments:
    CSV Output.bmp
    Field Delimiter Output.bmp


  • 2.  RE: Field Delimiter Node issues with semicolon and quotes

    Employee
    Posted 03-22-2016 03:08

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: awilliams1024

    Hi laurrieta

    here is one way to separate out the fields using the Experimental Split String node.
    In the example the static data node is emulating the output from your Get FTP node. The quote characters are being removed before the split as it appears there are no embedded quotes in the data.
    Copy and paste the following code into the BRE canvas.

    Regards,
    Adrian

    node:Output_from_Get_FTP_node
bretype:core::Static Data
editor:Label=Output from Get FTP node
editor:sortkey=56f10a250af331a2
output:@40fe6c55598828e5/=
prop:StaticData=<<EOX
a;b;dummy
\"001\";\"29022016\";
\"002\";\"29022016\";
\"003\";\"29022016\";
\"004\";\"29022016\";
EOX
editor:XY=190,130
end:Output_from_Get_FTP_node

node:Remove_the_Quotes
bretype:core::Filter
editor:Label=Remove the Quotes
editor:sortkey=56f10eb424fa0be6
input:@40fd2c74167f1ca2/=Output_from_Get_FTP_node.40fe6c55598828e5
output:@40fd2c7420761db6/=
prop:Script=<<EOX

theData = replace('a;b;dummy', "\"","")

emit theData

EOX
editor:XY=410,130
end:Remove_the_Quotes

node:Split_String
bretype:lal1::Split String
editor:sortkey=56f113bc53cb7e39
input:@55830d2f13603161/=Remove_the_Quotes.40fd2c7420761db6
output:@55831c714d7c687c/=
inclockxy:Inclock=20,30
outclockxy:Outclock=400,30
prop:Delimited Field=theData
prop:Delimiter=;
prop:Field Names=a,b,dummy
editor:XY=510,130
node:Filter_2
bretype:::Filter
editor:shadow=558a53423fa40b62
input:@40fd2c74167f1ca2/=
output:@40fd2c7420761db6/=
end:Filter_2

node:Pivot__Data_To_Names
bretype:::Pivot - Data To Names
editor:shadow=558a534250592991
input:@4ca9ea0e77f858a8/=
output:@4ca9ea197eed4a1a/=
node:GroupByPath
bretype:::GroupByPath
editor:shadow=558a5342037d6b2d
input:@4ccfdae429e85cb5/=
output:@4ccfdae4585258de/=
node:DataToNamesWithGroupBy
bretype:::DataToNamesWithGroupBy
editor:shadow=558a534221300fa2
input:@4cceefe2650d45e1/=
input:@4cceefe45c596005/=
output:@4cceeff7142d3549/=
end:DataToNamesWithGroupBy

node:Sort
bretype:::Sort
editor:shadow=558a53424bcf15ed
input:@40fd2c743ebf4304/=
output:@40fd2c746a2a3b47/=
end:Sort

node:Agg
bretype:::Agg
editor:shadow=558a53423c147acd
input:@40fd2c7427456e5b/=
output:@40fd2c744c862db0/=
output:@4cd003f34b1437c4/=
end:Agg

node:Bypass
bretype:::Bypass
editor:shadow=558a53422f017577
input:@4b467f7e129d45c1/=
input:@4b467f830ffe047b/=
output:@40fd2c7436717256/=
end:Bypass

node:Agg_2
bretype:::Agg
editor:shadow=558a53421c4f0ecb
input:@40fd2c7427456e5b/=
output:@40fd2c744c862db0/=
end:Agg_2

end:GroupByPath

node:NoGroupByPath
bretype:::NoGroupByPath
editor:shadow=558a5342413d0f62
input:@4ccfdae429e85cb5/=
output:@4ccfdae4585258de/=
node:DataToNamesNoGroupBy
bretype:::DataToNamesNoGroupBy
editor:shadow=558a53422af362f1
input:@4cceefe2650d45e1/=
input:@4cceefe45c596005/=
output:@4cceeff7142d3549/=
end:DataToNamesNoGroupBy

node:Agg
bretype:::Agg
editor:shadow=558a534204d243d1
input:@40fd2c7427456e5b/=
output:@40fd2c744c862db0/=
end:Agg

end:NoGroupByPath

node:Bypass
bretype:::Bypass
editor:shadow=558a5342526457a8
input:@4ccfdb241245012a/=
input:@4ccfdb25174c47e7/=
output:@40fd2c7436717256/=
end:Bypass

end:Pivot__Data_To_Names

node:Filter
bretype:::Filter
editor:shadow=558a534239d66e8d
input:@40fd2c74167f1ca2/=
output:@40fd2c7420761db6/=
end:Filter

end:Split_String


  • 3.  RE: Field Delimiter Node issues with semicolon and quotes

    Employee
    Posted 04-04-2016 07:13

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: laurrieta

    ok, good enough.

    Thanks a lot.