LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only
Back to discussions
Expand all | Collapse all

Node Release: Encoded File

  • 1.  Node Release: Encoded File

    Employee
    Posted 11-16-2010 15:09

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: mmarinelli

    Node Name: Encoded File
    Node Category: Acquisition
    Use Case: Use this node to deal with files which are not ASCII encoded
    Tested on Release: 4.1.7
    External Dependencies: None
    Practical Usage Example: An example use of this node has been provided in the prototype examples graph which is released in conjunction with the prototype library. Please see that graph for a executable use case.
    Node Documentation:

    Functional Description:
    Reads encoded file(s) into the graph by converting to Unicode using the Python codecs module.See http://docs.python.org/library/codec...dard-encodings for details.Warning: If the encoding chosen is wrong the node may hang and not produceany output.Tested with utf-8 and utf-16.For an EBCDIC USA file use cp037 (untested at this writing)
    Parameters:
    • EnableEncoderDiscovery: If set to true, the node will attempt to determine the source file encoding by inspecting the Byte Order Mark (BOM) at the beginning of the file. If no encoding can be detected this way, the DefaultEncoding parameter value will be used.
    • FilenameExpr: Identifies which data to use for the filename, if filenames are processed from an added input.
    • DefaultEncoding: Encoding which is used to decode file if EnableEncoderDiscovery is set to false or if it is set to true but no encoding can be determined from the file contents
    • FieldNames: Comma-delimited list of field names. If omitted, the first row of the first input file will be used for the data header.

      Note: While this node can read Unicode data, it can only use ASCII field names. If non-ASCII characters are found within the field names in the header, the node will error.
    • FieldDelimiter: Delimiter character between field values. May either be a standard character or a Python-style Unicode escape (e.g. \u0256).
    • EncodingErrorBehavior: Rule to use for recovering from encoding errors.
      * 'strict': raise an exception in case of an encoding error
      * 'replace': replace malformed data with a suitable replacement marker, such as '?' or '\ufffd'
      * 'ignore': ignore malformed data and continue without further notice
      * 'xmlcharrefreplace': replace with the appropriate XML character reference (for encoding only)
      * 'backslashreplace': replace with backslashed escape sequences (for encoding only)
    Known Limitations: This node is a temporary solution for the inability of some native acquisition nodes to handle Unicode sources. Once the core nodes support Unicode, this node will be obsolete.