LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only
  • 1.  Sum across cilumns

    Employee
    Posted 04-02-2013 10:09

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: svimes

    Hi i have 18 columns all with the field name "*_FEE_*" and i column which is an ID field. I would like to emit, for each row , the sum of the columns with "*_FEE_*".

    Thanks


  • 2.  RE: Sum across cilumns

    Employee
    Posted 04-03-2013 04:58

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Tim Meagher

    Hi,

    There is no simple BRAINscript operator that allows you to do this type of operation.
    However, you can get the entire list of input fields, and check to see for each of the fields if the field name matches a specified pattern.
    If the field matches the pattern, than you can then add the field value to a count variable, and at the end output this variable.

    A (heavily commented) example of how to do this is below:

    #The pattern against which to match the fields.
    #If you want to make this node relatively generic, you could provide this as a 
    #parameter, then use parameter substitution in the Script
    fieldPattern = ".*_FEE_.*"
    
    
    
    
    #Get the names of all of the input fields
    fields = inputFields(1)
    
    
    #initialize the sum variable
    sumVal = 0
    
    
    #Loop over all input fields
    fieldNum = 0
    while fieldNum < fields.len() {
        fieldName = fields.getItem(fieldNum)
        
        #Check if this field matches the pattern
        #If so, get the field value and add to the sum
        if (regexIsMatch(fieldName, fieldPattern)) then {
            sumVal = sumVal + field(fieldName)
        }
        fieldNum = fieldNum + 1
    }
    
    
    #Output the sum & the id field
    emit id
    emit sumVal as "SumVal"

    While this will do what you want, it is a pretty inefficient way to perform the operation.
    In the example above, the field names will be checked on each execution iteration (i.e. for each input record).
    It would make more sense to check the field names, and determine over which fields you want to sum on the first iteration, and then for all subsequent iterations, you can just sum the fields you know match the pattern.

    An optimized example (again heavily commented) is shown below:

    #The pattern against which to match the fields.
    #If you want to make this node relatively generic, you could provide this as a 
    #parameter, then use parameter substitution in the Script
    fieldPattern = ".*_FEE_.*"
    
    
    #Only look at the field names and determine which fields you want to sum on the first pass
    #This just saves on execution time in subsequent passes
    if firstExec then {
        #Get the names of all of the input fields
        fields = inputFields(1)
        
        #Setup a list to store the names of the fields you want to sum
        sumFields = list()
        fieldNum = 0
        
        #Loop over all input fields
        while fieldNum < fields.len() {
            fieldName = fields.getItem(fieldNum)
            
            #Check if this field matches the pattern
            #If so, add the field name to the list of fields over which to sum
            if (regexIsMatch(fieldName, fieldPattern)) then {
                sumFields = sumFields.append(fieldName)
            }
            fieldNum = fieldNum + 1
        }
    }
    
    
    #This part will be executed for each input record...
    #Initialize the sum variable
    sumVal = 0
    fieldNum = 0 
    
    
    #Loop over all of the fields to sum, and add the values
    while fieldNum < sumFields.len() {
        sumVal = sumVal + field(sumFields.getItem(fieldNum))
        fieldNum = fieldNum + 1
    }
    
    
    #Output the sum & the id field
    emit id
    emit sumVal as "SumVal"
    Regards,
    Tim.


  • 3.  RE: Sum across cilumns

    Employee
    Posted 04-04-2013 04:52

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: svimes

    Thanks Tim

    Why do you need to add the full stops for the field pattern?
    also when i run the second version it errors:

    ERROR processing data:
    value of variable 'sumVal' does not agree with compiled type: long vs. integer
    2013-04-04 12:13:49.000; WARN: value of variable 'sumVal' does not agree with compiled type: long vs. integer
    id: 0 chain: 0 group: 0
    cppDetail: context: BrainScriptVariable::getValue() ..\expert\BrainScriptVariable.cpp@53

    I have converted all the data types to long


  • 4.  RE: Sum across cilumns

    Employee
    Posted 04-04-2013 05:06

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Tim Meagher

    Hey,

    The reason you need the full stops is that this is a regular expression rather than a glob-style pattern.
    In a regular expression, the '.' means any character, and the '*' means that the preceeding character can appear any number of times (including 0 times).
    Without the '.', you will get an error saying there is "nothing to repeat", since the regular expression would then be intepreted as "repeat <nothing> 0 or more times" ... which doesn't really make sense.

    Regarding the error you are having, the problem is that with the definition of sumVal.
    The line:
    sumVal=0
    Initializes the sumVal variable, however, it also assumes that it is an integer type.
    If any of your fields are of a long type, then the line:

    sumVal = sumVal + field(sumFields.getItem(fieldNum))
    Will fail with the error you are seeing.

    In order to get this to work (which will also work correctly even if all inputs are int types), simply change the declaration of sumVal to be:
    sumVal= long(0)
    Tim.