Data360 Analyze

currently I am using python to create a few directories.. what I would like to do is add another column to the data and put the path in it

currently:

FILES=self.inputs[0].read()
PTH=os.path.dirname(FILES["APV_CSV"])+"/"
if not os.path.exists(PTH+"Clean"):
   os.makedirs(PTH+"Clean")
   os.chmod(PTH+"Clean",0o777)

self.outputs[0].write(FILES)

id like to do something like this:

FILES=self.inputs[0].read()
FILES.append("PATH",str)
PTH=os.path.dirname(FILES["APV_CSV"])+"/"
FILES["PATH"]=PTH

am I close? lol

Thanks!

Yes, very close.  The example above looks like Brainscript but the question is for Python.  I'm assuming you'd like to do this in Analyze.  This is the common syntax in Analyze for all the nodes that allow you to modify Python (e.g. Transform, Generate Data, Split, Filter, Aggregate, Joins, Lookups, etc...)

First add the output column in the Configure Fields section and then simply populate it with either a value or a variable in the Process Records section. 

ConfigureFields

out1.path = unicode

ProcessRecords

out1.path = "mypath"

Attached files

Example Creating and Populating Output Pins.PNG

this is in a Python Node in Analyze... and the script is Python code.

Any particular reason you are using the "Python Node" vs "Generate Data" or the Transform node?  The syntax for that node is slightly different as it's typically used for legacy dataflows/graphs.  The new nodes are much simpler to use. 

Regardless, here is the syntax for the legacy "Python Node" (make sure you are using tabs for indention)

To create the output column on the output pin use this:

pathName = "mypath"                                       #Example variable
self.som = self.newMetadata()                          #Create output metadata
self.som.append("Path","string")                       #Create a new field called Path
self.outputs[0].metadata = self.som                  #Add metadata to the output pin

To populate use this: 
outSchemaRec = self.outputs[0].newRecord() #Create a new output record
outSchemaRec[0] = pathName                        #Set the value of the new field
self.outputs[0].write(outSchemaRec)               #Write the output record

I am trying to use the python node because its already being used to create the sub folders.. and the path and such are ready to go. Although yeah, i could just create the field in a transform node before this node...

but i just realized, its useless. The export nodes dont allow you to use a field name for the file, but the import does?

is there a way to set a parameter dynamically? 

The typical, and simplest method, to accomplish this is to use a 2nd node to rename the file. Below is an alternate method that technically works, but is subject to change in future releases using a default run time property and a JEXL command. 

FileName:

c:/MyTemp/TEST_{{^RunDate^}}.xlsx

or

c:/MyTemp/TEST_{{!jexl!parameters.get("RunDate").replace("-","").replace(":","").substring(0,4)!}}.xlsx

To insert a run property or see a list, simply click on the context menu for the property.  e.g. 

The preferred, and supported, method to parameterize the name is to use the path safe version of the RunDate property reference:  {{^RunDate_PathSafe^}}  to insert the date without needing to strip out the element delimiter characters in the date e.g.

C:/MyTemp/TEST_{{^RunDate_PathSafe^}}.xlsx

Which results in a file named Test_20200110.xlsx

Note the Run* run properties were first introduced in Analyze v.3.5.1

See the following topic in your Help documentation for further details:

Advanced topics > Using derived property values > Run property substitution

This information is also available online for the latest version of the product here:

https://d3sa-preview.infogixsaas.com/docs/dist/help/Default.htm#d-advanced-topics/substituting-run-properties.htm 

SO there is still no way to  SET a param in code? it would make things so much easier....

ok, renaming was using move(a,b) whats the new code to rename a file?

whats the best way to do it now (so that the rename is performed after the file appears)

thanks!

Use the techniques described in this article to rename a file. You can create a Run Dependency so that the Transform node only runs once the output connector node has finished running. 

For more information on Run Dependencies see the following topic in your Help documentation:

Advanced topics > Creating run dependencies

The information is also available online for the latest version of the product here:

https://d3sa-preview.infogixsaas.com/docs/dist/help/Default.htm#d-advanced-topics/creating-run-dependencies.htm 

Nice! As always thank you Adrian! you should have a special tag/image "GURU" lol