Data360 Analyze

 View Only
  • 1.  Calculating max dates with transform node

    Posted 03-27-2023 06:46

    Hello,

    assume i have the following data:

    date:

    2023

    2022

    2021

    2022 

    2023

    and so on...

    I would like to only have an output, for those rows that contains the max(date)

    i have tried out1.something = max(in1.date) but it doesn't work

    More or less i want to create a flag for those rows that are max dates

    if in1.date == max(in1.date)

         out1.flag = 1

    else:

        out1.flag = 0

    Best regards

    H



    ------------------------------
    Henrik B
    E.ON Sverige
    ------------------------------


  • 2.  RE: Calculating max dates with transform node

    Posted 03-28-2023 03:10

    It is probably possible to do all at once, but I would probably just aggregate the data (using an an aggregate node), get the max date, and do a lookup with the original stream and the output of the aggregate to "join" and then filter/split the rest of the values. This does take some more components, but will be easier to see where the next problem is when it goes wrong.



    ------------------------------
    Peter Sykes
    Vontobel Holding AG
    ZUERICH
    ------------------------------



  • 3.  RE: Calculating max dates with transform node

    Employee
    Posted 03-28-2023 09:56

    You could use the Sort node and configure it to sort descending on the date field. 

    Then use a Transform node to filter the records with the maximum date value. The maximum value will be in the first record so you can capture this value in an if statement where the condition is selecting the first record in the input data.



    ------------------------------
    Adrian Williams
    Precisely Software Inc.
    ------------------------------



  • 4.  RE: Calculating max dates with transform node

    Posted 03-29-2023 02:35

    Great solution.

    What if, i want to find the 3 largest values in my data set? Not only the Max value.



    ------------------------------
    Henrik B
    E.ON Sverige
    ------------------------------



  • 5.  RE: Calculating max dates with transform node

    Employee
    Posted 03-29-2023 05:31
    Edited by Adrian Williams 03-30-2023 04:43

    I think I would move towards using an approach similar to Peter's.

    Modify the Sort node to only output a single record for each date value by setting the 'Unique' property to True:

    Then use a Head node to output the first N=3 records:

    Then use a Join node to perform an Inner join on the original data set using the top N date values as the match key

    Note you need to discard the right fields else you will have duplicate field names on the output (and they don't provide any additional value anyway).



    ------------------------------
    Adrian Williams
    Precisely Software Inc.
    ------------------------------