Data360 Analyze

Welcome to the Data360 Analyze community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Here are some useful links where you can find more information:

Product Announcements  Product Documentation  Ideas Portal

Discussions

Members

Resources

Events

 View Only

Reduce to List Functionality using the Aggregate node

  • 1.  Reduce to List Functionality using the Aggregate node

    Employee
    Posted 09-01-2017 07:41

    Hi,

    A couple of months ago a user inquired about the availability of 'reduce to list' functionality in Dataverse. At the time a solution was presented that used the Agg Ex node to implement the functionality. The Aggregate node was included in the Dataverse 3.1.5 release and so it is timely to provide the community with an implementation of the 'reduce to list' functionality using the new Aggregate node.

    The Reduce_to_List node provides functionality similar to the previous node. The Aggregate node leverages the Python scripting capability in Dataverse so there are some differences in its configuration compared with the previous node.

     

    The 'Group_By' property accepts an input field or a comma-separated list of field names. As is the case with other Python based nodes, the format used to specify the input fields is different to that used in earlier nodes:

     

    The input field can be specified using the format fields.fieldName provided the field name is a valid Python variable name that does not conflict with the name of any built-in Python variables or variables from imported modules. When the node property's context menu is used to insert an input field reference, this will be the usual format for the inserted field name. If the input field name includes space characters the inserted field format will use the format fields['field Name'] - obviously you can manually enter fields using either of these formats. In addition, you can also specify the fields manually using the following formats:

    in1.fieldName

    in1['field Name']

     

    When specifying multiple fields, they must all be included in the same line, e.g.:

     

    or use the back-slash character '\' to signify the continuation of a single line:

     

     

    The above field reference formats can also be used when configuring the 'ItemField' property. However, this property accepts only one field reference, e.g.:

     

    The 'Delimiter' property enables you to optionally specify the delimiter character(s) used to delimit the items in the output list. The default is the *' character.

    Other optional properties allow you to specify whether the input data is to be sorted using the Group_By field/ fields, the name of the output field containing the item list (by default = 'ItemField_List') and the initial value for the list (which defaults to be the same as the delimiter).

    You can also specify a predicate expression in the 'PredicateExp' property which is used to determine whether the ItemField value in a particular record is to be included in the list. This expression must evaluate to a Boolean result. The expression must be a valid Python statement, e.g.

     

    The example data flow below provides a use case for the node. Note, the implementation in the example is a standard node rather than a library node. To enable you to easily re-use this functionality in your projects you can import the Reduce_to_List library node. The node can then be found in the Aggregation and Transformation category of the node palette.

    Note you will need a Dataverse 3.1.5 installed to import and use this node.

    Regards,

    Adrian

    Attached files

    Reduce_to_List_using_Aggregate--share_315.lna
    Reduce_to_List.lna