LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mglynn

I have data set values which will often update based upon the file being accessed. In these circumstances I need all entries with the Primary ID deleted. Eg.

PRI_ID SERV UPD
001 1234 01/01/17
001 1234 02/01/17
002 1235 01/01/17
003 1236 01/01/17
003 1236 02/01/17
004 1237 01/01/17
005 1238 01/01/17
005 1238 02/01/17
006 1239 01/01/17

In these situations I want to remove any file which has a duplicated PRI_ID. This would leave me with only 002, 004 & 006.

What is the best (or any) method for achieving this?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi,

You do not say how the PRI_ID relates to the file to be removed. However, you can use the Duplicate Detection node to separate the records with only a single occurrence of PRI_ID in the data set from those that have multiple occurrences. Alternatively, you could use an Agg Ex node to generate the count of occurrences and then use that as the selection criteria to identify the relevant records. Once you have segmented the records you could then us the deleteFile() function to remove the appropriate file.

The attached data flow shows the use of the Duplicate Detection node and the Agg Ex node in this use case.

Removing_both_duplicate_values.brg


Regards,
Adrian