LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

I am attempting to use an Agg node to help parse an input file and combine some data in a header with the rest of the data it applies to. It works in Expert but I've had trouble making a BrainScript version.

Attached is a sample input using core::Static Data. It is processed with a basic::agg node and also a core::Agg node. The core::Agg node uses a List to keep a bunch more data around than is necessary to output. I had to go this rought because I was unable to declare a variable in a normal way that kept its value from one record to the next without using the groupString() function.


Attachments:
aggProblem.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

I'm unclear as to what you're looking for here, as both of your nodes seem to produce the same (correct) output. Is this an optimization issue? Please elaborate. Thanks.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

Exactly, this is an optimization issue. The Expert code version does not have to deal with Lists and will probably execute much faster then the BrainScript version.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: tczotter

You're over complicating this.

It is very simple.

MIN = MIN.trim()
data_line = MIN.strlen() == 10
if data_line then MOU = MOU.trim().long()
else tag = FULL_LINE.substr(5,FULL_LINE.strlen() - 10)
emit tag, MIN, MOU
where data_line

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: tczotter

Forgot to mention:

You don't need an agg node here.

In Brainscript, we have variables that can hold values across rows, which is what I did with the "tag" value in my previous post.

I only update it on certain rows. Otherwise it stays the same.

That script works in a filter or agg, but filter will be faster and does not need a Group By expression.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ejones

Thanks for this. I didn't realize it was so easy to do. Interesting, but it makes sense, that filters would run faster than aggs.