LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only

  • 1.  To find specific columns in the Raw Data.

    Employee
    Posted 03-21-2016 07:22

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Jagdev

    Hi Experts,

    I need your help to get rid of one issue. The data set we received might or might not contains below mandatory columns in them. Is it possible to check the availability of these columns in the data set and if not find/available then add them with "0" value.

    Example of column names mentioned below.

    - ABC
    - DEF
    - GHI
    - JKL

    Regards,
    Jagdev


  • 2.  RE: To find specific columns in the Raw Data.

    Employee
    Posted 03-21-2016 07:47

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    Create a static data node with just the column headers, no data rows:

    ABC,DEF,GHI,JKL

    Then use a CAT node to add that "empty" data to your input stream, and turn on the CAT option for ConcatenateType=Union

    That way, even if the columns are missing from the input, the CAT will add them.

    Note: you may need to set the data types to match the incoming data, something like this:

    ABC:string,DEF:long,GHI:double,JKL:string


  • 3.  RE: To find specific columns in the Raw Data.

    Employee
    Posted 03-21-2016 08:55

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Jagdev

    Hi Stonysmith,

    This is fantastic!

    The concept is simple and clean. I have a single doubt. Is it possible instead of displaying null value in the newly added column, can we have the value "0". I need this columns for calculation purpose at the end.

    Regards,
    Jagdev


  • 4.  RE: To find specific columns in the Raw Data.

    Employee
    Posted 03-21-2016 09:12

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    Not with the CAT node.

    The simplest way to deal with the nulls is a Filter node...


    emit *
override emit ABC=ABC.ifNull("-Null-")
override emit DEF=DEF.ifNull(long(0))
override emit GHI=GHI.ifNull(double(0))
override emit KLM=KLM.ifNull("-Null-")


  • 5.  RE: To find specific columns in the Raw Data.

    Employee
    Posted 03-23-2016 03:46

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Jagdev

    Hi Stonysmith,

    I tweak the code and it is working now. Thanks for all your help on this. Please see the below updated code.

    emit *
override emit 'ABC'.ifNull(double(0)) as "ABC"
override emit 'DEF'.ifNull(double(0)) as "DEF"
override emit 'GHI'.ifNull(double(0)) as "GHI"
override emit 'JKL'.ifNull(double(0)) as "JKL"
    Regards,
    Jagdev
    Attachments:
    Validation1.brg