Data360 Analyze

 View Only
Back to discussions
Expand all | Collapse all

Dictionary | Delimited | List Comparison

  • 1.  Dictionary | Delimited | List Comparison

    Posted 06-29-2022 20:41
    Edited by Nitin Chawla 06-29-2022 20:44
    Hello!

    I need to implement a logic to perform validity check between two columns.

    Table A (500K records)

    Column #1    ​                                     Column #2    
    ASXX; ASDFG                                 A1234567
    AWXS                                              A1234568
    CSER                                              A1234569
    DF66; DF77; D78S; XYZA              A1234570
    A124; B456; C123; D567               A1234571

    Note: Number of delimited values in column #1 is not fixed.

    Table B (Look up table) having 2000 records
    Column #1
    ASXX
    ASDFG                                
    AWXS                                             
    CSER                                             
    DF66
    DF77
    D78S
    XYZA             
    A124
    B456
    C123
    D567              
    YYYY

    Current, logic implemented is using a loop which is taking time by picking the first row in table A, 1st two elements (ASXX; ASDFG) and checking with lookup table.

    Is there a way I can implement logic via Dictionary in Analyze ? We can set Column #1 from Table A and B as dictionary and do a comparison. Can anyone please help to provide some references, if this logic can be implemented in Analyze.

    ​If there is any other approach which I can explore as a combination of CSV node and Transform node, please suggest that as well. ​

    ------------------------------
    Thanks and Regards,
    Nitin
    ------------------------------