Assure DQ

 View Only
  • 1.  High volume Duplicate check

    Posted 03-25-2020 13:38

    If I was to perform a duplicate check to see if a record came in for the past 30 days, what is the best way to code this if my volume is 1 million a day? would saving it under control data efficient?  

    OR

    Is this better to be performed in Data360 DQ+ product?



  • 2.  RE: High volume Duplicate check

    Employee
    Posted 04-01-2020 13:17

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    In Assure, you grab the history of the control you're running and check against the last 30 days to see if there are duplicates (you should set your retention to 30 so you don't check records older than 30 days).

    In DQ+, you put your History of 30 days into a Data Store, and your current data into a Data Store and compare the two (again, set retention to 30 days).

    We think DQ+ would perform faster than Assure would for this.