Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.
Originally posted by: Tim MeagherHey,
The problem here is that the Delimited File node does not handle unicode data.
It always tries to read the data in the LAE's character set.
The LAE has a configurable character set, which is used for the string data fields and metadata.
However, this is required to be a single-byte character set (by default this is windows-1252).
The Delimited File node doesn't actually do any character set manipulation or character encoding/decoding.
Therefore, the data that is on the output of the node is actually UTF-8 data, which is why if you change the display preferences in the BRD Viewer, it displays correctly.
However, since this is in a string field, none of the other nodes are able to handle this data as they try to decode it in the LAE's character set.
When you try and convert this to unicode, you will run into errors for this reason.
This limitations with both the Delimited File node, and the LAE's character set are logged in our bug tracking system and will be addressed in a future release.
This, however, requires a large development effort to overhaul all of the node infrastructure that expects all string fields to be encoded using a single byte character set, along with the various pieces of the software that are required to process output and input metadata.
In the meantime, I have put together a node which I think should help you address the problem of acquiring delimited unicode data in the LAE.
I have posted the node in the thread:
http://community.lavastorm.com/threa...g-unicode-data
Once the data is in a correctly encoded unicode data type, it should work correctly with the Output Excel node.
Note that this node is not an officially Lavastorm released or supported node, therefore support will only be provided through the forums until this node or something similar can be integrated into the core product.
Hope this helps,
Tim.