Data360 Analyze

 View Only
Back to discussions
Expand all | Collapse all

Excel Output Limits - Creating multiple excel outputs until output is complete

  • 1.  Excel Output Limits - Creating multiple excel outputs until output is complete

    Employee
    Posted 01-10-2022 03:45

    I currently have an issue where i have an end result of ~3 million rows of data which changes depending on what day it is.

    I am trying to output the 3 million rows of data to an excel output, however, there is a limit of ~1 million rows on excel.

    What I am trying to achieve is a setup where if I feed in 3 million rows, Data360 will create multiple excel outputs files until there are no more rows to process.

    Sadly i am not familiar with Python to achieve this so any help would be superb.



  • 2.  RE: Excel Output Limits - Creating multiple excel outputs until output is complete

    Employee
    Posted 01-18-2022 05:15

    You could use a Transform node that has been configured with additional outputs and then steer each input record to the appropriate output by examining the current execution count of the node and comparing it with a threshold value:

    The Max_Records value would need to be set to the required value (~1M).

     

     

    Attached files

    Partition_Records - 18 Jan 2022.lna