LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: aop

Hi!

Has someone tried to modify the data analyzer node to handle e.g. additional number formats? I'm not familiar with python and currently I cannot really use the data analyzer (for automatic conversion of data types) due to the limited conversion possibilities in the officially supported version. I'm mostly missing double conversion for number formats with a comma as decimal separator (most common format in europe). Would be nice to have the analyzer to understand most common thousand separators as well (whitespace, comma, dot).

In addition I would like the data analyzer to understand date formats like DD.MM.CCYY or D.M.CCYY. I guess this is indeed something I could easily do by just adding these formats to the list inside the current node.

Defining data types is currently quite time consuming process in BRE compared to some other competing products so if someone has used some other approach to this subject I would be very interested to hear! I have also thought about creating a custom node with e.g. a date field list as a first paramater and a date format as manual input as a second paramater and then having something similar for number fields but I think I would need python or java for this. This kind of setting would ease up the data type definiton process enough as I quite often know in advance which fields I'd like to convert. It's just the large number of fields to be converted and the need to work with many different kinds of input data every day that's causing the trouble and some extensive copy&paste operations now. If someone happens to have a python/java example that would perform some operation to a specified set of fields I would be really interested to see so I could probably get started from there!

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: rboccuzzi

There is an enhancement request to expose the formats tested for dates and times in a node parameter for a user to customize. You are correct though, you could easily copy a version of the Data Analyzer into a library of your own, and make a few modifications to it. You definitely don't want to modify it in place in core. I recommend changing the name of the node too, to something like "My Analyzer", so people know it isn't the same one as in core.

The code in the analyzer does just go through an ORDERED list of formats for the date and time formats, so you could just add or reorder that list to give new formats or new priorities (when it can't tell because values don't fall outside of range, it just chooses from the highest priority in those lists).

As for your other topic, detecting numbers based on different separators that would be a little more involved; you can't natively do it in Python, so it would be a bit of string manipulation and testing, it would be a more significant modification of the Data Analyzer. It's a good request though, and I have created an enhancement request (PR4396) to include that functionality in the future.

As for Python node examples, I would suggest looking at the Python Filter, it probably is easier and you don't really need the straight up Python node; it just makes it slightly easier to get started. Additionally, there are plenty of examples of Python nodes in core itself. Again, I don't recommend you changing anything in there, but you can look around for examples of how to do something in Python.

Cheers
Rich