Data360 Analyze

 View Only

  • 1.  Data Streaming Input sources

    Employee
    Posted 05-23-2019 07:15

    Hi Everyone,

    Please let me know if D3S Analyze supports real-time data streaming (Apache Kafka) as an input source or do we have to store the real-time data into HDFS or hadoop and then source data using one of the in-built nodes for HDFS within Analyze to source data?

    Best,

    Rakshit



  • 2.  RE: Data Streaming Input sources

    Employee
    Posted 06-05-2019 02:02

    Data3Sixty Analyze does not currently provide a pre-built input connector for real-time data streaming sources such as Apache Kafka. However you may be able to leverage the Kafka Java client as part of a custom node using Analyze's Java node, or use the Kafka Python client in conjunction with Analyze's Python node.

    If you have a Kafka HDFS Sink Connector that is able to export the contents of a Kafka topic to an HDFS file then an Analyze HDFS Download node could be used to read the contents of the file. 



  • 3.  RE: Data Streaming Input sources

    Employee
    Posted 01-13-2020 08:12

    The Data360 DQ+ product supports streaming data using Kafka. See the DQ+ documentation for further details:

    https://doc.infogixsaas.com/dqplus/Default.htm#h-nodes/nodes-streaming.htm