Data360 Analyze

 View Only

  • 1.  Replace weird characters

    Employee
    Posted 04-02-2019 04:29

    This post has been created here on behalf of user annunziato lo bosco from an original post on the Lavastorm zendesk community portal.

     

    Hello,

    I have the following problem:
    I have a column, 'junk ", as a string, which contains characters like Ã, ä, à ©, ¶ which I would like to replace with characters like ö, Å..If I use the Replace Text node, it works, but since this node has have been deprecated and will not be supported in a future release, I would like to use the Transform node.
    When I use the Transform, in process record out1.junk = field.junk.replace ('Ã', 'ö'), it doesn't work.
    I used too the node Modify field, to convert the string in unicode and then the Transform node, but it doesn't work.
    Please could you support to solve this issue?
    Thanks.


  • 2.  RE: Replace weird characters

    Employee
    Posted 04-02-2019 04:46

    The Python scripting language used by the Transform node is Python 2. You need to declare the strings in your replace() function as unicode rather than as str objects. e.g. using:

    out1.junk = fields.junk.replace(u'ä', u'ä')

    Note also the input field reference in the above statement is 'fields' instead of 'field'

    There is a useful description of some of the issues that can occur when encoding characters here:

    https://www.i18nqa.com/debug/utf8-debug.html 

     

    Also, see the attached example data flow (requires Data3Sixty v.3.2.7 or above)

     

    Regards,

    Adrian

     

    Attached files

    Replace_Weird_Characters_Example - 2 Apr 2019.lna

     



  • 3.  RE: Replace weird characters

    Employee
    Posted 04-02-2019 05:28

    You could also use the Calculate Fields node instead of the Transform node. The downside is that you need to create separate intermediate fields for each text replacement performed.

    You can use the Modify Fields node to remove the extraneous fields:

    Regards,

    Adrian

     



  • 4.  RE: Replace weird characters

    Posted 04-03-2019 05:45

    Hello Adrian,

    thanks a lot, all the solutions wok fine.

    Regards, Annunziato.



Global message icon