Data360 Analyze

Is there a node that can look for specific keywords across my whole data set not just 1 field. as an added complication the keyword maybe found with in the field or set of text/numbers not as a word on its own.. 

eg in a data set of 330,000 rows and 57 fields I am looking for the word 'bank'  it could also be found in more than one of the fields in a single record. 

There is no built-in node to find text across multiple fields but you could use some code in a Transform node.

Requires Analyze v.3.6.x

Attached files

Replace Text on all Fields - 24 Jan 2020.lna

Thanks - but I require the records flagged rather than text replaced within the field. If the keyword is found the just bring the record back. how would I do this rather than replace?

The logic in the original node does not modify the contents of the input data. The node scans each string/unicode type field in an input record. If a match is found in a field in a record then a record is output containing the input record number, the field that contained the matching text and the all of the fields in original record. For example searching for "A" in a case-sensitive manner outputs two records for the fourth input record:

If you want to only output one record for each input record regardless of whether one or more fields contained the search text then you could comment out the lines that explicitly write the record (see the middle node in the Find_Text_V2 data flow):

The original node (the top node in the updated data flow) and the middle node only output records where a match was found. If you wanted to output all input records regardless of whether a match was found you could change two of the 'else' statements in the code to write the input field values when a match was not found.

The output of the node would then be the following when searching for "A" case-sensitive (see the bottom node in the updated data flow):

The 'recordNumber' and 'matchedField' fields would be NULL for records where none of the searched fields containted the text. If required, you could then use a Split node to segregate the sets of records using a NULL value in one of those fields as the criterion.

Attached files

Find_Text_V2 - 5 May 2021.lna

That work! Many thanks 

Great! I now realise that somehow the data flow attached to my first reply was replacing text - the wrong file was attached - my bad. Apologies for any confusion caused.

hello - on the flip side of this is there a way to exclude the matches so the node will only bring the data back where no hits were found? 

I now need to use the node where a word can match for example 'cork' and the follow up with exclude those that contain 'Ireland' within that data set.  so either a spit result or only bring back if no match

The nodes in the example data flow (Find_text_V2) above output two fields with match-related information:

You could use a first instance of the node to identify records containing 'cork then use a Split node configured to output records where either the recordNumber (or the matchField) field was not Null. Then connect a Modify Fields node that is configured to remove the match-related fields:

Another Find Text field can then be connected to match 'Ireland' and another Split node that is configured to select the records that did not match (i.e. where the matchField is Null):

The node's 'true' output will then contain the records that did not contain the second search value. See an updated example that searches for 'A' and then excludes 'econd'.

Attached files

Find_Text_V3 - 20 May 2021.lna

Thanks this almost works but then I'm losing where the cork was found in the original search by removing the match field. 

I need to know the records that contain cork and what field in matches with but remove all with Ireland. Should I then use a join to add the match field back for the cork?

Thanks this almost works but then I'm losing where the cork was found in the original search by removing the match field. 

I need to know the records that contain cork and what field in matches with but remove all with Ireland. Should I then use a join to add the match field back for the cork?

Use the Modify Fields node after the first Find Text field to rename the fields rather than deleting them:

The modified fields will be passed through the second Find Text node