Data360 Analyze

Good evening,

I'm coming to the Community with another obstacle I'm facing at the moment. I have a data flow which will output several files (pipe delimited TXT files). The name of the files is dynamic, but I can pick it up on import (which I do) so when I output I can use the correct file name. The problem is that using this method the FileName and File columns are part of the output. This results in a structural validation error in the service I'm using to later insert the data.

Objective

1. Capture the file name (column "File" in below screen capture of data viewer) so I can use it later on output to properly name my output file.
2. Not include the "FileName" and "File" columns in final output of TXT file so it will pass structural validations later. 

Any suggestions or ideas on a possible solution are greatly appreciated. Stay safe!

Thanks,
Josh

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

When you say you are using pipe delimited txt files, am I right in presuming you are writing out the data using Output CSV/Delimited node? If so there is a Field Order section in that node where you can rearrange the order of the fields and also you can remove any fields that you don't want. In this case if FileName on the input only serves the purpose of telling the node what the output file should be called you could remove it in this section and it would not be on your output. 

Another option is to write out the file to some generic file name (with FileName removed from the fields), then using a transform node directly after you have written out the file (clock it to your output node), you would use os.move(GenericFile,NewFileName) command.

Thanks Gerry! I can't believe I didn't think of this approach.

Josh