List of Contributions

Peter Sykes

Contact Details

My Content

1 to 20 of 25 total

RE: How to do custom http paging.

Posted By Peter Sykes 12-09-2024 16:55
Found In Egroup: Data360 Analyze \ view thread
OK, this works. ------------------------------ Peter Sykes Data Governance & Architecture Vontobel Holding AG Zurich ------------------------------

How to do custom http paging.

Posted By Peter Sykes 12-09-2024 13:14
Found In Egroup: Data360 Analyze \ view thread
Hello All, Phoning a friend. I am trying to download data from the bitbucket REST with the http node. I see that the response tells me that I have not downloaded the last page, and I can see from from the documentation that I need to write a custom paging script. So I form my URLs and fire them ...

RE: XML Files Containing Both Data and Images

Posted By Peter Sykes 11-28-2024 03:20
Found In Egroup: Data360 Analyze \ view thread
There are many many rabbit holes which one can go down with this topic. However, my opinion is that the best approach for XML and binary data would be to base64 encode a binary element. With this approach it is simply standard XML being transmitted. ------------------------------ Peter Sykes Data Governance ...

RE: Pivot Names to Data Node Help

Posted By Peter Sykes 09-05-2024 10:28
Found In Egroup: Data360 Analyze \ view thread
Hey, not sure if you got that sorted out. Maybe post a sample line with an example of what you'd like. ------------------------------ Peter Sykes Data Governance & Architecture Vontobel Holding AG Zurich ------------------------------

RE: End of month

Posted By Peter Sykes 09-05-2024 10:24
Found In Egroup: Data360 Analyze \ view thread
Try this, unfortunately python 2.7 doesn't have a timedelta with months. Test it a bit though. ------------------------------ Peter Sykes Data Governance & Architecture Vontobel Holding AG Zurich ------------------------------

RE: New Fixed Format node does not work correctly

Posted By Peter Sykes 05-31-2024 10:23
Found In Egroup: Data360 Analyze \ view thread
Perhaps notepad++ regular expression search and replace? (/n to :) ------------------------------ Peter Sykes Data Governance & Architecture Vontobel Holding AG Zurich ------------------------------

RE: Dbt support

Posted By Peter Sykes 05-10-2024 10:56
Found In Egroup: Data360 Analyze \ view thread
In the meanwhile, if you have enough developers, you can take the compiled SQL found per object in the manifest.json file. Depending on how you have set it up, this might be also available via http if you are using Azure. We are able to fill in the table to table transformations missing by de default ...

RE: Edit/Change Field API Name

Posted By Peter Sykes 04-18-2024 08:56
Found In Egroup: Data360 Govern \ view thread
Certainly this is the way it has been for some time. It would be great to have the option to just do it. Maybe you want to post that to the ideas forum. ------------------------------ Peter Sykes Data Governance & Architecture Vontobel Holding AG Zurich ------------------------------

RE: Check field type and Convert to int dynamically in a transform node?

Posted By Peter Sykes 02-09-2024 03:15
Found In Egroup: Data360 Analyze \ view thread
Something I have some stuff which converts to unicode. Process records: for fieldName in in1: out1[fieldName] = unicode Configurefields: for fieldName in in1: if in1[fieldName] is not Null: out1[fieldName] = unicode(in1[fieldName], "utf-8") You can do something similar for integers. ...

RE: Field name with spaces

Posted By Peter Sykes 02-02-2024 03:03
Found In Egroup: Data360 Analyze \ view thread
Once field names become interesting, e.g. the output from JSON with dots etc, I find it easier to use the dictionary notation. (Not sure if its really called that). Its a few more chars to type, but seems bombproof. Configure fields: out1['Feed Identifier'] = unicode Process Records: out1['Feed Identifier']= ...

RE: Proper Syntax to call Oracle Stored Procedure using DB Execute

Posted By Peter Sykes 10-17-2023 04:50
Found In Egroup: Data360 Analyze \ view thread
There are some options, but with a JDBC execute node you can just paste in a valid anonymous PL/SQL e.g. from SQL developer to run it. You don't seem to require any parameters so something like this: Also the JDBC execute node needs an input, even if just dummy data from a create data node. sql query: ...

RE: Flattening rows

Posted By Peter Sykes 09-13-2023 09:41
Found In Egroup: Data360 Analyze \ view thread
Nice. ------------------------------ Peter Sykes Vontobel Holding AG Zurich ------------------------------

RE: D360 - Excel SOLVER Equivalent

Posted By Peter Sykes 08-17-2023 02:26
Found In Egroup: Data360 Analyze \ view thread
Hi Andrew, This is certainly possible, but even a back of the envelope implementation is a bit of work. As you know, what you are trying to do is linear optimization. You would have two approaches in general: a library based approach or a native one one. For the library based approaches - there ...

RE: Execute Stored Procedure oddity

Posted By Peter Sykes 07-12-2023 03:26
Found In Egroup: Data360 Analyze \ view thread
If this is Oracle, then there are some strange cache behavior, even when you have nocache set in the sequence definition, especially when different users are used to insert rows. The default sequence cache for oracle is 20. In a developer session, you will probably always have the sequences cached ...

RE: Changing query

Posted By Peter Sykes 05-30-2023 01:58
Found In Egroup: Data360 Analyze \ view thread
Hi Henrik, I have been able to use the snowflake version of an anonymous pl/sql block in the jdbc execute node. (needed as the exception handling is not helpful in Analyze). I would try if Oracle pl/sql works in the same way, (begin.. xzy... end). If so, there are many examples out there with running ...

RE: Calculating max dates with transform node

Posted By Peter Sykes 03-28-2023 03:09
Found In Egroup: Data360 Analyze \ view thread
It is probably possible to do all at once, but I would probably just aggregate the data (using an an aggregate node), get the max date, and do a lookup with the original stream and the output of the aggregate to "join" and then filter/split the rest of the values. This does take some more components, ...

RE: Removing spaces from string in column

Posted By Peter Sykes 03-28-2023 03:05
Found In Egroup: Data360 Analyze \ view thread
Something like this would be easier to comment on if you include a couple records of sample data. I think you might want to change the line fields['S&E'] = str2 to be out1['S&E'] = str2 If you don't expect an array then I probably wouldn't personally use "in". I don't think its doing what you expect ...

RE: Use Oracle Sequence and add to incoming data

Posted By Peter Sykes 03-26-2023 16:49
Found In Egroup: Data360 Analyze \ view thread
If your organization's coding standards allow for it, then the easiest thing to do is to set the column default value to the sequence.NEXTVAL you want to use, and ignore the entire column in the insert statement. This works with JDBC store, I think it should work with DB Store. ------------------------------ ...

RE: Fetch XML from Database

Posted By Peter Sykes 02-10-2023 07:54
Found In Egroup: Data360 Analyze \ view thread
It should be more or less the same (use the same DB Query parameters now in the more or less the same fields on the JDBC node). Depending on how friendly the tnsnames has been set up in your organization that will work. If that is not so nice, e.g. if you use ldap, then you will need to use the same ...

RE: Fetch XML from Database

Posted By Peter Sykes 02-10-2023 03:15
Found In Egroup: Data360 Analyze \ view thread
I reproduced your problem with the Db Query node. I think this is a very low level issue due to the CLOB datatype and OCI. I would stay away and use the JDBC interface to get the data / as below with the same SQL it works for me. If you can-t you need to raise a ticket with support. ------ ...