List of Contributions

Henrik B

Contact Details

My Content

1 to 20 of 37 total

Incomplete Time Series

Posted By Henrik B 06-18-2024 05:43
Found In Egroup: Data360 Analyze \ view thread
Hello, i have the following problem: Assume i have the data set below, i would like to get a complete timeseries for the data coming in, so 12 more rows of data. Sometimes my time series in hours comes in incomplete. Is it possible to fill the time series for a specific day ? It could just be ...

RE: Row Hash optimization, Data comparison/validation test

Posted By Henrik B 05-30-2024 04:56
Found In Egroup: Data360 Analyze \ view thread
Thanks for the help Adrian, the solution works. I did a test with 3 different data sets(rows > 10^6), it turned out that my solution i first posted was 18.2% faster than your version(i took a mean of the 3 test). ------------------------------ Henrik B E.ON Sverige --------------------------- ...

RE: Row Hash optimization, Data comparison/validation test

Posted By Henrik B 05-30-2024 02:20
Found In Egroup: Data360 Analyze \ view thread
Thanks for the answer Adrian. The problem with the new script compared to the old is that, i dont get unique values for the Rowhash output, all values are the same. In the old script, i get unique values for each row, for RowHash in my test dataset. Modified script: ------------------------------ ...

Row Hash optimization, Data comparison/validation test

Posted By Henrik B 05-29-2024 04:58
Found In Egroup: Data360 Analyze \ view thread
Hello, this is a great solution im using to generate a hash value for an entire row with N number of columns and N amount of data. I use this for data comparison/Data validation test to see if two datasets are exactly the same, so i just run two datasets into the transform node below and then i make ...

RE: Create a Temporary List or Set in Transform Node

Posted By Henrik B 04-22-2024 04:38
Found In Egroup: Data360 Analyze \ view thread
Hi, for me it would be a lot easier to understand. If you send pictures of a sample data input, and expected output. Like below: ------------------------------ Henrik B E.ON Sverige ------------------------------

concatenating N amount of columns and values comma separated

Posted By Henrik B 04-22-2024 03:55
Found In Egroup: Data360 Analyze \ view thread
Hello, assume we have the input data as per below: Image 1 Assume i want the input data represented as an output as below: Image 2 Would this be possible to do for N amount of columns and N amount of data values, making it a generic solution without hardcoding each column name? It ...

RE: Data Flows that are "Locked for Editing"

Posted By Henrik B 04-08-2024 05:15
Found In Egroup: Data360 Analyze \ view thread
Hi, as far as i know you need to be signed in as an admin user and do the following: ------------------------------ Henrik B E.ON Sverige ------------------------------

Batch insert query optimization

Posted By Henrik B 03-22-2024 06:39
Found In Egroup: Data360 Analyze \ view thread
Hello, i want to use the dbexecute and not the db store node to do a batch insert of rows. So assume the following scenario: In this case i only have two rows, i want to make into one single insert statement as per below: This method is very costly, if i want to insert a big number of rows, especially ...

Calculate Fields Node

Posted By Henrik B 02-22-2024 01:24
Found In Egroup: Data360 Analyze \ view thread
Hello everyone, a quick question. Do i always have to create new output columns while i use calculate fields, it would be nice to use it for conditionals? As an example, would it be possible to replicate the following from a transform node in Calculate fields, to have a conditional without creating ...

RE: Field name with spaces

Posted By Henrik B 02-05-2024 04:29
Found In Egroup: Data360 Analyze \ view thread
Hm i would change the name in modify fields, adding a space. Or you can maybe do it from a select query, select column1 as column 1 maybe ------------------------------ Henrik B E.ON Sverige ------------------------------

RE: How to Sum 3 column value but can contain NULL

Posted By Henrik B 01-22-2024 01:45
Found In Egroup: Data360 Analyze \ view thread
Can't you replace the nulls' with zeroes? so it becomes a valid object to perform int operations on? And if you want to perform string operations i would replace the null's with an empty string. ------------------------------ Henrik B E.ON Sverige ------------------------------

RE: Date to date delta count for time series

Posted By Henrik B 10-11-2023 02:15
Found In Egroup: Data360 Analyze \ view thread
I found that one solution is, solving the logic with SQL and then extracting that to a D360 Workflow. ------------------------------ Henrik B E.ON Sverige ------------------------------

RE: Server Performance Report

Posted By Henrik B 10-10-2023 06:52
Found In Egroup: Data360 Analyze \ view thread
Sidetracking a bit but, The lookup node can have a big impact on the ETL and server performance, it can bottleneck. "The Lookup node is recommended for use with a small data set on the right input as the entire right data set is loaded into memory", so in those cases its better with a join/merge, ...

Date to date delta count for time series

Posted By Henrik B 10-10-2023 05:38
Found In Egroup: Data360 Analyze \ view thread
Hello, assume i have 3 dates and a count column per each date, is there a convenient way to calculate the delta for each 2 adjacent dates? starting from the oldest date: (In reality it will be around 30-60 dates) What i have: |Date| |count| 2023-01-02 1 2023-01-03 8 2023-01-04 9 ...

RE: Find special character and replace it and everything after it

Posted By Henrik B 09-29-2023 04:34
Found In Egroup: Data360 Analyze \ view thread
You could also solve it with regexp in transform node: import re x = re.search(r'(.*)\|',in1.ProtocolNumber) if x: out1.ProtocolNumber = x.group(1) ------------------------------ Henrik B E.ON Sverige ------------------------------

RE: Flattening rows

Posted By Henrik B 09-13-2023 02:29
Found In Egroup: Data360 Analyze \ view thread
something like this? But adapting the script in the aggregate node to exclude nulls? ------------------------------ Henrik B E.ON Sverige ------------------------------

Lookup node & performance

Posted By Henrik B 09-05-2023 02:13
Found In Egroup: Data360 Analyze \ view thread
What would be the consequences of running millions of rows and a big amount of columns for lookup nodes in 360 often? In the documentation it states that "The Lookup node is recommended for use with a small data set on the right input as the entire right data set is loaded into memory" My point ...

RE: Delete excel file from specific path after each run of the data flow.

Posted By Henrik B 08-22-2023 02:01
Found In Egroup: Data360 Analyze \ view thread
In the first step, i'm using a directory list node as an input to the above. ------------------------------ Henrik B E.ON Sverige ------------------------------

RE: Disabled nodes

Posted By Henrik B 07-04-2023 02:31
Found In Egroup: Data360 Analyze \ view thread
So in other words, there is nothing negative with having n amount of disabled nodes in scheduled graphs besides, it doesn't look structured? ------------------------------ Henrik B E.ON Sverige ------------------------------

Disabled nodes

Posted By Henrik B 06-30-2023 04:25
Found In Egroup: Data360 Analyze \ view thread
Hello, I'm wondering if disabled nodes has any affect on the performance on scheduled graphs? Assume i have a scheduled graph that runs every day, but it also contains 100 disabled nodes. Is it preferable to not have any disabled nodes in a scheduled graphs, and if so why? Best regards ...