List of Contributions

Scott PERRY

Contact Details

My Content

1 to 20 of 50+ total

RE: D360 Analyse - Load JSON Lines file using JSON Node

Posted By Scott PERRY 11-28-2023 03:33
Found In Egroup: Data360 Analyze \ view thread
Hi Ernest, thanks for the detailed reply. I will give it a try using the transform node and let you know. thanks again! ------------------------------ . ------------------------------

S3 Bucket - specify the end point url host name

Posted By Scott PERRY 11-24-2023 06:52
Found In Egroup: Data360 Analyze \ view thread
Hi All We are currently running a version of D360 Analyse that does not allow me to specify the end point url and it is defaulted to amazonaws.com. I understand that once we upgrade to latest version this will be possible but in the meantime is there a workaround way way of overriding it? Perhaps ...

D360 Analyse - Load JSON Lines file using JSON Node

Posted By Scott PERRY 11-23-2023 14:43
Found In Egroup: Data360 Analyze \ view thread
Hi All Hoping you can help with the following issue. I am trying to load a JSON Lines file using the D360 Analyse JSON node. It loads the data but where there is a value for a key that is a list of values it splits each value in that list into it's own row and then any data on the records (for other ...

RE: Python Version

Posted By Scott PERRY 07-04-2023 06:49
Found In Egroup: Data360 Analyze \ view thread
Hi, Please can you advise if Python 3 is available in D360 Analyse yet and from which D360 Analyse version? Also what version of python is in Data360 Analyze v3.8.4.1? thanks Scott ------------------------------ . ------------------------------

Execute Data Flow Node queries

Posted By Scott PERRY 11-08-2022 04:33
Found In Egroup: Data360 Analyze \ view thread
Hi Team I have a few questions on the "Execute Data Flow" Node as follows: When I run one of my Data Flows it takes 38 mins. When I run the same data flow via Execute Data Flow Node it ran for over 24 hours and still has not finished (the Run Parameters are exactly the same, it gets 3/4 of the ...

RE: How to load fields from csv file, using a CSV/Delimited input node, that contain an apostrophe

Posted By Scott PERRY 06-16-2022 09:35
Found In Egroup: Data360 Analyze \ view thread
Hi Adrian I don't see SubstituteInvalidCharachters as an option on the CSV/Delim node. (v3.6.3.6520).  Is there a way for me to manually define it or other approach? Thanks Scott

How to load fields from csv file, using a CSV/Delimited input node, that contain an apostrophe

Posted By Scott PERRY 06-06-2022 10:37
Found In Egroup: Data360 Analyze \ view thread
Hi Support team,  Please can you advise how to load fields from csv file, using a CSV/Delimited input node, that contain an apostrophe? example field name - "Participant's LEI". It can load the data if I specify all the field names in the "Field Names" option in the node but remove the apostrophe ...

Directory Node - Exclude sub directories

Posted By Scott PERRY 03-29-2022 04:26
Found In Egroup: Data360 Analyze \ view thread
Hi I am using the Directory Node - "Exclude sub directories" functionality I pass a filepath of "MyRoot/Archive/" and I have a set of Standard exclusion sub directories of Test Archive I would expect the following folders to be excluded MyRoot/Archive/Test MyRoo ...

Writing to Excel Template

Posted By Scott PERRY 03-25-2022 09:58
Found In Egroup: Data360 Analyze \ view thread
Hi We are using the Output Excel Node using the TemplateInputFile Parameter to write our Excel output to a template. The template has 57 columns some of which are "grouped" together. There is no other data in the file. To write a few million records across a couple of these files is taking ...

Run dependencies - Trigger node execution from optional unconnected nodes

Posted By Scott PERRY 02-25-2022 03:58
Found In Egroup: Data360 Analyze \ view thread
Hi, Let's say I have 3 nodes called A, B and C. None of them are connected to each other. I am able to create a run dependency between C and A, and C and B, such that both A and B must complete before C will run. However, my scenario is that Node C is dependent on the completion of either ...

RE: Switch doesnt work as expected... Alternative?

Posted By Scott PERRY 02-23-2022 14:18
Found In Egroup: LAE \ view thread
Hi, I have a similar issue to the above but for nodes rather than for files. Let's say I have 3 nodes called A, B and C. Node C is dependent on the completion of either node A or node B. So either A or B will complete at runtime but never both as they are parallel flows in the data flow and ...

RE: schedule same data flow in parallel with different runtime properties

Posted By Scott PERRY 02-23-2022 10:13
Found In Egroup: Data360 Analyze \ view thread
thanks John. Yeah that would work it's just I need a way of doing it without using the "Execute DataFlow" node. I guess there must be some internal Id for a run instance. For example when i look at the run data for a scheduled instance that is running and/or completed, the name I click on to open the ...

RE: schedule same data flow in parallel with different runtime properties

Posted By Scott PERRY 02-23-2022 09:30
Found In Egroup: Data360 Analyze \ view thread
Hi John thanks for the prompt reply. Yes i have used the "Execute DataFlow" for other purposes and it's very nice. The issue i have that i am trying to solve here is that I want to create a unique temporary folder for storing some temp files per scheduled instance of a graph but that unique ...

RE: schedule same data flow in parallel with different runtime properties

Posted By Scott PERRY 02-23-2022 08:49
Found In Egroup: Data360 Analyze \ view thread
Hi Adrian Is there a unique runtime D360 variable that I can use that is unique to a particular scheduled instance of the same graph? I thought of the "Run Time" parameter but that wouldn't be unique if two instances of the same graph were scheduled at the same time. Kind regards Sco ...

RE: Python Version

Posted By Scott PERRY 02-18-2022 09:18
Found In Egroup: Data360 Analyze \ view thread
Hi Adrian Do you have a date by when you will make Python 3 optionally available in D360? thanks Scott

RE: Run dependencies

Posted By Scott PERRY 02-08-2022 07:49
Found In Egroup: Data360 Analyze \ view thread
Thanks Adrian. Just to clarify a couple of points: 1) Regarding competing of later nodes with earlier nodes within the same dataflow (where they are dependent on the previous node completing so can't be run in parallel), is it more efficient to use a run dependency clock between nodes or is it effectively ...

Run dependencies

Posted By Scott PERRY 02-06-2022 08:48
Found In Egroup: Data360 Analyze \ view thread
Hi Infogix, In my graph I have a series of nodes that are connected to each other and each node takes a long time to run. When I run the graph (all nodes at once) then the later nodes start running and effectively waiting for the earlier nodes to complete. My question is whether that is the most ...

RE: Inporting Multable excel Worksheets

Posted By Scott PERRY 12-01-2021 09:04
Found In Egroup: LAE \ view thread
Hi Just to add that if there was a way to put an if condition in the workbook xml so that for each sheet it will check if cell A1 and B1 are populated then set headerRow="1" else If cell A2 and B2 are populated then set headerRow="2" else If cell A3 and B3 are populated then set headerRow="3" else ...

Execute Data Flow Node - Permissions

Posted By Scott PERRY 12-01-2021 08:20
Found In Egroup: Data360 Analyze \ view thread
Hi Infogix I have a graph that contains an "Execute Data Flow" node. When I run the graph I get the following error message which seems to be permissions related as those in my team with developer read/write permissions do not have this issue when they run the graph. I have read and execute permissions ...

RE: Inporting Multable excel Worksheets

Posted By Scott PERRY 12-01-2021 08:07
Found In Egroup: LAE \ view thread
Hi Infogix I am using the MetaDataOutputPin to load multiple worksheets dynamically from an excel file into as many BRD files as there are worksheets. This works well except for when there is a header. E.g. if there is a header value in cell A1 of a worksheet only column A is outputted to that worksheet's ...