Data360 Analyze

 View Only

  • 1.  Dynamic Output Filename

    Posted 06-14-2021 07:37

    Hey

    There's a few posts on the community board around writing output file with a dynamic name. The recommendation is to write out the file to some generic file name and then using a transform node directly after you have written out the file (clock it to your output node), you would use os.move(GenericFile,NewFileName) command.

    Is there any improved way of doing this so a file does not have to first be written before it can be renamed? If not is this on the roadmap?

    thanks

    Scott



  • 2.  RE: Dynamic Output Filename

    Employee
    Posted 06-15-2021 02:55

    The majority of 'Output *' nodes require the 'File' property to be specified as a Literal value, though the Output Raw node and the Output CSV/Delimited node ('Filename' property) can obtain the value from an input field by using the "... (From Field )" source option on the property.

    You can introduce some dynamic aspects to a filename by including textual property substitution in the literal value of the File property. For instance, if you are generating a daily file you could include the ExecutionDate_PathSafe Execution property: 

    Which would generate a file with a name similar to this: Transactions_20210615.xlsx see the 'Advanced topics' > 'Using derived property values' > 'Execution properties' topic in your Help documentation or the online documentation for the latest version of the product here:

    https://d3sa-preview.infogixsaas.com/docs/dist/help/Default.htm#d-advanced-topics/substituting-execution-properties.htm

    If you need to construct the filename using some data flow logic then you could use the Execute Data Flow node in a 'parent' /'driver' data flow to run a 'child' data flow that performs the actual data processing, with the generated filename being passed to the child data flow as a Run property. See the 'Node help' > 'Interfaces and Adapters' > 'Execute Data Flow' topic in your Help documentation or the online documentation, here:

    https://d3sa-preview.infogixsaas.com/docs/dist/help/Default.htm#e-node-help/Interfaces_and_Adapters/execute-data-flow.htm

    Using the Execute Data Flow node approach allows you to specify multiple attributes that can be passed into the child data flow as Run properties and is a powerful mechanism, but adds complexity. The rename mechanism is simpler to achieve for most use cases.

    I do not believe there is any planned enhancements at this time to allow other Output nodes to source the File property value from an input field.