LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

For more information, please see the Node Training Overview.

Session 7

For this session, we are moving into mostly prototype nodes; Next session, stay tuned for 4.5 goodness.

Nodes:

Extract Pattern, Levenshtein Distance Match, Soundex Match,
Transpose Digit Match, Substring Lookup

As always, I look forward to talking with you.

Cheers
Rich
Attachments:
Training Session 7.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sgolding@mda-data.com

Hi Team,

I'm trying to understand the 'Transposed Digit Match' node by running through some simple examples however I can't explain the outputs. Could you please run through the following example. Basically 2 inputs with the same record except the last 2 digits have been switched.

INPUT 1
rowID:int,num:string
1,"1234"

INPUT 2
rowID:int,num:string
1,"1245"

NODE CONFIGURATION
threshold: 3
main field: num

EXPECTED OUTPUT
output "main"
rowID:int, num:string, test_index:long
1, "1234", 1
output "lookup"
rowID:int, num:string, test_index:long
1, "1243", 1

ACTUAL OUTPUT
null both outputs

Is this correct or a bug?

Thanks,

Stephen.
Attachments:
transpose_sample.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

If you want to look for transposed digits in the numbers 1234 and 1243, the existing node will work. However, neither of the two examples you provided actually tests for this case. In the zip file you attached, both of the data sources contain 1234, so there would be no transposed digit match for identical inputs. In the example you provided in within your post, you're comparing 1234 and 1245. These would not produce a match, either, as there are no transposed digits in the source data (as there would be with 1234/1243 or 1245/1254).

If you can provide another example where you are not getting the expected results, it may be helpful to track down any bugs in the node if they exist.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sgolding@mda-data.com

Sorry, classic example of human error.

Both examples should have been inputs "1234" and "1243".

Please refer the attached. They're still not matching.

Best regards,

Stephen.
Attachments:
transpose_sample_2.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

These values are not matching because the groupBy parameter is set to incorporate the first 3 characters of each value when calculating matches. This parameter is provided to optimize matching by only searching within sets which share a common key, greatly reducing the computational load of the matching.

If you set the groupBy expression to something else, for example to only the first digit of each number, or instead to something constant like "true", you will see the behavior you are expecting.

I noticed that while the documentation for this parameter indicates that the default behavior is to use the first 1 digit of each number as a key, the actual parameter code uses the first 3 digits, which is misleading. The next update of this library will correct this discrepancy.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: sgolding@mda-data.com

Thank you for your help and clear explanation.

I understand where I went wrong in my configuration.

Regards,

Stephen.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: kevin_chuah

A quick check for transposition can be to check if the difference between the numbers are divisible by 9. ie 1921 versus 1291 (got this from bank teller school)

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ryeh

For quick reference, here's the same SoundEx example using the released Fuzzy X-Ref node: https://infogix.zendesk.com/hc/en-us/community/posts/360051801653