LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ltolleson

Many times we find there are numerous fields in our graphs that need to be removed from the data stream. Normally we would have to exclude those files one-by-one. There are two soon to be documented features in BrainScript that now allow fields to be excluded from the output by using either wildcard (*) or pattern (regexp) values. The attached graph gives examples of each of these new functions and how they can be used in your graphs.

If you have any questions or comments about this functionality please reply to this thread.

Attachments:
Exclude_Pattern_Wildcard.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ltolleson

I just realized that the wildcard and pattern functions work with the emit statement as well.

Example

emit wildcard "ACCT_*"

This example says output all fields that start with the string "ACCT_"

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jhoitt

I have several hundred CSV files containing a column heading GST/HST.
The filter node can't handle the slash.
Your post really helped because I could not find how to handle this in the BRAIN Script Help or the filter node help.

I used the wildcard to get around this issue.

emit wildcard "GST*"

The field comes over as GST/HST and then I must use single quotes 'GST/HST' to use the field in other nodes.

Anybody with a better way please let me know.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rabbott

What version of BRE are you using? With 4.1.5 I was unable to reproduce the issue you described. I used a Static Data node and changed the string column "color" to "GST/HST", ran it through a Filter node with:

emit 'GST/HST'

and it worked fine. I then output the Static Data node to a .csv file (Output Delimited node), input using the CSV File node, and ran through the same filter and again it worked.

It appears that there may have been some confusion over the use of the single quotes around the field (column) name. If the field name is alphanumeric and unambiguous, it can be used without the single quotes (a.k.a. "tics"). Also a field name can be referenced without single quotes if it consists of an identifier, or a list of identifiers connected by "." . Field names can be prefixed with input#: without single quotes as well.

The 'GST/HST' field name contains the front slash, which is a special character denoting a command option. It needs to be interpreted literally and so must be "escaped" somehow. The single quotes do this.

Attached is the graph I used to check some things. Perhaps it might provide some clariifcation.

Cheers,

- Bob
Attachments:
GST-HST.zip

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: jhoitt

I am using BRE v4.1.5.0 Build 268.

I think I caused the problem myself in my CSV File input node by specifying the column name GST/HST in the list of field names. I could have just specified a column heading that didn't include escape characters.

Here is my original code just in case you want to recreate the same situation.

My original CSV Input node.

node:CSV_File
bretype:core::CSV File
editor:sortkey=4b6993d031166bf4_2
input:4b6993d010a7405c/=Get_Billing_files__Jan.40fd2c747c2b1c0a
output:@41e6c6cd11e613c6/=
prop:FieldNames=Location,Province,Terminal,Card,Se rial #,Date,Time,Status,Reference #,Unit Cost,# of Cards,Gross,Discount %,Net,GST/HST,Total Due,Customer
prop:FilenameExpr=<<EOX
FileName
EOX
prop:QuotedDoubleQuotes=Literal Quote if no EscapeCharacters
editor:XY=140,60
end:CSV_File



My Filter node using the wildcard
node:Filter_15
bretype:core::Filter
editor:sortkey=4b73174d555b2164_13
input:@40fd2c74167f1ca2/=CSV_File.41e6c6cd11e613c6
output:@40fd2c7420761db6/=
prop:Script=<<EOX
UseThisFieldToAggregate = "Agg"
emit Card.trim() as Card
emit 'Serial #'.trim() as SerialNumber
emit Date.trim() as Date
emit Time.trim() as Time
emit Status.trim() as Status
emit 'Reference #'.trim() as ReferenceNumber
emit 'Unit Cost'.trim() as UnitCost
emit '# of Cards'.trim() as NumberOfCards
emit Gross.trim() as Gross
emit 'Discount %'.trim() as DiscountPercent
emit Net.trim() as Net
emit wildcard "GST*"
emit 'Total Due'.trim() as TotalDue
emit UseThisFieldToAggregate
EOX
editor:XY=260,60
end:Filter_15