Data360 Analyze

I've run the Data Analyzer node to analyze the output data from the Static Data node.  The data is just the sample one provided by Dataverse, and which is clearly set as an integer type from Static Data node but found by Data Analyzer as bint type.  I'm not sure how Dataverse created this data or why it finds it to be different at the end of the process.  How can I correct this?

Nothing to correct. bint is "binary integer".. meaning that the data is stored in an internal format which is quicker to process.

Where are you seeing the bint? in the data viewer, or by looking in th BRD file itself?

From the "analysis" output pin of data analyzer node.  I'm using this data as a filter node to change the metadata.  Unfortunately the metadata cannot use the bint type, and something I need to define for NewTypeField in Change Metadata Node.

I assume your use case is similar to that shown below.

The 'bint' data type from the 'analysis' pin of the Data Analyzer node can be converted by using the following script in the downstream Transform node:

tmpType = if('discovered type' == "bint") then "int" else 'discovered type'

emit 'field name' as "OldName",
   str(null) as "NewName",
  tmpType as "NewType"

 I would be interested to understand why your preferred solution is to use the Change Metadata node instead of using the output directly from the Data Analyzer node's 'converted data' pin. The data type on the converted data pin is valid and can be used by downstream nodes - i.e. no conversion is actually required in this case.

Also, have you looked at the Data Converter node which provides automatic data type detection functionality and is provides better performance for larger data sets compared with the Data Analyzer node.

Using the Filter + Change Metadata node because I was trying to filter out unused fields that are sitting in my dataset.

I wasn't trying to convert any data from one type to another.  Would you still recommend Data Converter node?

I'm interested to know why Dataverse nodes would even list out an unrecognised bint data type if other nodes can't recognise the type unless its called int... Maybe I have misunderstood something.

I got my node to work so thank you very much.

The DataAnalyzer node is reporting the 'backoffice" data type for the field.. It makes sense not to confuse the issue.  In the background, there is a (slight) difference between integer and bint, so the node is technically correct, but it doesn't matter as far as you're concerned.  You will always refer to it as "int".

If you want to drop some columns, then you can simply set up a Transform node this way:

emit *
exclude myField1
exclude myField2
exclude myField3

You don't need to reference the data types to do this.