LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jpstory

Hi All

I am trying to import a csv file via CSV Node, the problem is the file does not have field headers, so I have to provide
I understand that I could simply type the headers in the FieldNames field under the Optional tab.

However for some reason I have to do it in other way:

I read all the csv files with a directory node, and then add the field headers (double quoted and separated with comma) as an additional field called MyFieldName into the output of Direcotry node.

Then I connect the Directory Node output to a CSV Node, at this point, the MyFieldName field would be available as an input variable to be used in the CSV Node's ( at the FileNames field under Optinal tab).

The problem is, CSV Node treat MyFieldName as a variable rather than a list of string to be used as field headers.

Is there any way to get CSV Node to treat MyFiledName as a list rather than one single word/variable? Please kindly help, thanks!

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: AdamParker

Try using the Delimited Data node from the lal1 library (if you have access to it). I think it has the ability to take field names from an input field.

I don't believe the CSV File node supports what you are trying to accomplish above.

Edit: Grammar

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jpstory

Hi

Thanks for looking at this. Just to clarify, you meant you don't belive the CSV File node support what I am trying to accomplish, correct? (your last sentence is a bit confusing with the double negative)

I have tried Delimited Date node, it does not work. It return similar error message as CSV Data node does.

Any other thoughts would be appreciated!

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: AdamParker

Sorry about that, I edited the comment to clarify.

What is the issue with putting the list of fields in the FieldNames parameter?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jpstory

the variable ("MyFieldName" ) I put in the FieldNames parameter is read by Delimted Data node as a string rather then a list.

Here's the graph, you need to mannually create a Sample.csv file with data like this:
"Report Date" 2016-10-06 "Report Run Date" 2016-10-07
Name Gender Age ID
Name 1 M 25 Empoyee 1
Name 2 F 55 Empoyee 2
Name 3 M 47 Empoyee 3
Name 4 F 35 Empoyee 4






node:Delimited_File_3
bretype:core::Delimited File
editor:sortkey=588112691fd225bf_2
input:58811269563f55af/=Lookup_3.40fd2c7445835585
output:@41e6c6cd11e613c6/=
prop:FieldDelimiter=,
prop:FieldNames=MyFieldName
prop:FilenameExpr=<<EOX
'FileName'
EOX
prop:FilenameOutputField=SourceFile
prop:RecordDelimiter=\n
editor:XY=570,870
end:Delimited_File_3

node:Lookup_3
bretype:core::Lookup
editor:sortkey=58810f02240075b5_2
input:@40fd2c746abc6dc7/=Directory_List_3.40fd2c747c2b1c0a
editor:bend=0=110|820
editor:bend=0=470|820
input:@40fd2c74486e4494/=Filter_5.40fd2c7420761db6
output:@40fd2c7445835585/=
prop:InputKey=<<EOX
0
EOX
prop:LookupKey=<<EOX
0
EOX
prop:Script=<<EOX
emit 1:*
emit 'MyFieldName'

EOX
editor:XY=500,870
end:Lookup_3

node:Filter_5
bretype:core::Filter
editor:sortkey=5881105075a96f60_2
input:@40fd2c74167f1ca2/=Head_5.42fbf39e14aa2417
output:@40fd2c7420761db6/=
prop:Script=<<EOX

emit 'record' as MyFieldName

EOX
editor:XY=410,880
end:Filter_5

node:Head_5
bretype:core::Head
editor:sortkey=58810ed9310e5ed2_4
input:@42fbf39f1af2376d/=CSV_File_5.58810d597725247e
output:@42fbf39e14aa2417/=
prop:RecordCount=1
editor:XY=330,880
end:Head_5

node:CSV_File_5
bretype:core::CSV File
editor:sortkey=58810d366658128d_4
input:58810d364ec4614a/=Head_2.42fbf39e14aa2417
output:@41e6c6cd11e613c6/=
output:58810d597725247e/out2=
prop:FilenameExpr=<<EOX
FileName
EOX
prop:FilenameOutputField=SourceFile
editor:XY=250,870
end:CSV_File_5

node:Head_2
bretype:core::Head
editor:sortkey=58810ed9310e5ed2_2
input:@42fbf39f1af2376d/=Directory_List_3.40fd2c747c2b1c0a
output:@42fbf39e14aa2417/=
prop:RecordCount=1
editor:XY=170,870
end:Head_2

node:Directory_List_3
bretype:core::Directory List
editor:sortkey=58810cfb5d3c5458_2
output:@40fd2c747c2b1c0a/=
prop:CaseInsensitive=false
prop:DirectoryName=C\My Documents\LavaStorm\Test
prop:Pattern=Sample*.csv
prop:Recurse=true
editor:XY=80,870
end:Directory_List_3

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

One method of stripping off the first line in multiple source files would be to leverage the R language capabilities of Lavstorm provided by the Power R node. Note a separate Node Pack license is required to execute this node.

An example data flow is below.

Power_R_Skip_Initial_Record_in_CSV_Files_v0.1--share.brg


The data flow also includes a general purpose delimited file import node also implemented using the Power R node that provides some functionality not available in the standard CSV File node.

Regards,
Adrian

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

If you do not have access to the Power R node you could also try using the Open Source R node. The data flow below includes an example node that provides the same functionality as the example above.

R_Node_Skip_Initial_Record_in_CSV_Files_v0.1--share.brg


The R Node installer can be obtained from the Lavastorm Libraries download page here: http://www.lavastorm.com/current-rel...orm-libraries/


Please see the R Node installer Release notes (also available at the above location) for details of how to set up and run the R node.
Please ensure you obtain the correct version of the R Node installer that aligns with the 64-bit/32bit architecture of your machine.
Rserve must be running on the R server and accessible from the Lavastorm application for this node to operate.


Regards,
Adrian