Data360 Analyze

The Calculate Fields node is not really suitable for this type of treatment as there are multiple steps, each of which need the output of the previous step to create the final result. The Transform node is more appropriate and provides a more transparent view of the processing being performed on the data. Both nodes are Java based so there is no intrinsic benefit to using a Calculate Fields node and may actually provide poorer performance due to the need to write out each intermediate step.

You could improve the performance of the code in the Transform node by replacing the lines in the ProcessRecords script that concatenate the field string values,  replacing the current line in the script with the following which uses the join() method:

------------------------------
Adrian Williams
Precisely Software Inc.
------------------------------

Thanks for the answer Adrian.

The problem with the new script compared to the old is that, i dont get unique values for the Rowhash output, all values are the same.

In the old script, i get unique values for each row, for RowHash in my test dataset.

Modified script:

The Calculate Fields node is not really suitable for this type of treatment as there are multiple steps, each of which need the output of the previous step to create the final result. The Transform node is more appropriate and provides a more transparent view of the processing being performed on the data. Both nodes are Java based so there is no intrinsic benefit to using a Calculate Fields node and may actually provide poorer performance due to the need to write out each intermediate step.

You could improve the performance of the code in the Transform node by replacing the lines in the ProcessRecords script that concatenate the field string values,  replacing the current line in the script with the following which uses the join() method:

------------------------------
Adrian Williams
Precisely Software Inc.

Ah, my apologies, there was an element missing in the original logic that I have also corrected in my original message above. The new statement updates the logic to hash the values of the fields and now produces the same results as your code.

Thanks for the answer Adrian.

The problem with the new script compared to the old is that, i dont get unique values for the Rowhash output, all values are the same.

In the old script, i get unique values for each row, for RowHash in my test dataset.

Modified script:

The Calculate Fields node is not really suitable for this type of treatment as there are multiple steps, each of which need the output of the previous step to create the final result. The Transform node is more appropriate and provides a more transparent view of the processing being performed on the data. Both nodes are Java based so there is no intrinsic benefit to using a Calculate Fields node and may actually provide poorer performance due to the need to write out each intermediate step.

You could improve the performance of the code in the Transform node by replacing the lines in the ProcessRecords script that concatenate the field string values,  replacing the current line in the script with the following which uses the join() method:

------------------------------
Adrian Williams
Precisely Software Inc.