LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

I have a large amount of unsorted data and want to find some statistics about the values in one of the columns. I would prefer not to have to do a sort on that column since it only has very few possible values. For example, I need to accumulate the revenue by market. What is the simplest way to do this?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

You want to use the Accum node that is in Basic.brg, that is what it is designed for.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

I was going to suggest the same, but isn't this node deprecated and as such no longer supported?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

A quick and filthy way to do this would be to perform a hash split on the key field and then peform sort-less aggregates on the resulting, homogeneous (with respect to the key column) data. This assumes that you know the number of distinct values for the key column up front, and that they will not change, so it's not the best approach for a production environment. Sort of thinking out loud here...

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

I should clarify. Generally, I recommend using core, and not using basic. This is one of the very few bis of functionality that remain in basic that isn't currently easier in core. There is an easy way to do it using dictionaries, but the performance would not be acceptable, and we will be re-working this in some up-coming release of BRAIN. So, for now, use accum.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

If performance isn't a concern, you can create a BRAINscript/dict solution using an agg. The problem with this solution is, the dict has to get re-created each iteration through, which could get expensive, depending on the number of values.

You would peg the GroupBy to 1, and here is the BRAINscript code for the agg node:
key = 'type'
value = id
if firstExec then
{
d = dict()
l = list()
}

if not d.inDict(key) then
{
l = l.append(key)
d = d.dictStore(key, value)
}
else
d = d.dictStore(key, value + d.dictRetrieve(key))

output 1 {
emit key, value
where False
}

if lastInGroup then {
i = 0
e = len(l)
while (i < e) {
key = l[i]
value = d.dictRetrieve(key)
do output 1 {
emit key, value
}
i = i + 1
}
}

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

Thanks for the responses. This helped me know what my options are. I do recommend upgrading that Accum node to BrainScript soon.