LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: islandz

I have a small set of records (750) that I have loaded into LS. Throughout the entire workflow, the accented characters appear with no issues. However, the output from the Delimited File output node results in what looks like escape characters, rather than the correct accented characters (ie should be ��� but output shows �\xe9�). As a test, I also created an Excel output and the characters show fine.

I opened the output text file in both EmEditor and Notepad++ and both show the escape characters.

Can you please assist with how I can get the correct accented characters in the text output?

Thank you

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: prasmussen

Islandz,

There are a few methods to do so; however, the attached node is probably the simplest. Give it a try and let us know if it works for you.
Attachments:
OutputDelimitedUTF.txt

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: islandz

Morning,

This seems to have worked to output to a text file with the correct accented characters. Another (related) issue I noticed, after this post, is that when I exported to a dbf (via an R-Node, provided by a Lavastorm Employee), also had an issue with corrupted characters. I have attached the code for the R-Node. Is there a way to basically combine these two (the node you provided and the R-Node) so we can still output dbf files? (We need dbf files since we have a legacy system that only accepts dbf input.)

Thank you

R_Node_dbf.txt

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: prasmussen

The R node you sent ran just fine and created UTF8 characters in the dbf output. In my case, I used Greek specific characters such as "ΠΦ". There are R libraries for converting specific character sets other than UTF8 if these don't work. Also, please confirm the data going into the node is unicode.

Can you post a sample of the data and I'll try to recreate.

Make sure the package is installed and available in the library: e.g.
install.packages("foreign")

I needed to modify my path, but your's may be fine if you installed in the LAE directory: e.g.
library("foreign", lib.loc="C:/Program Files/R/R-3.3.2/library")
try1 <- paste("{{^filepath^}}", ".dbf", sep = "")
write.dbf(in1, try1, factor2char = TRUE)

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: islandz

Here is a sample subset of the data. The input is actually an Excel file. It has worked in the past, so am not sure what changed to cause this issue.

Sample_Data.zip

Thank you

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: prasmussen

This worked just fine with non-ASCII characters captured correctly in the output DBF file. On your Excel node, please make sure the Default Type field on the Optional tab is set to Unicode. The default is Unicode but you might have changed it at some point.