Data360 DQ+

 View Only
  • 1.  HIVE_URL variable

    Posted 08-13-2020 11:18

    What would  ${HIVE_URL}  look like after it gets populated?  what kind of jdbc connection is being used? 

    Our current setup is we have YARN and HDFS config files from our hadoop team and use a kerberos user that exist in our hadoop environment for D360 to talk to our hadoop systems.   

    Does the database channel from the attached image apply to D360?  or is it prepopulated with dummy fields? like the url, username (sagacity) and password. I assume the username is a default from the product since we have never setup a "sagacity" user in our D360 settings nor in our Hadoop environments.       



  • 2.  RE: HIVE_URL variable

    Employee
    Posted 08-31-2020 07:26

    Hi JL... We are looking into getting this answer for you!  I apologize for the delay.  



  • 3.  RE: HIVE_URL variable

    Employee
    Posted 08-31-2020 07:49

    JL, in the comment above you mention an attached image, but I didn't see on in the post.



  • 4.  RE: HIVE_URL variable

    Posted 08-31-2020 08:11

    Hi Jeff,  Please find attached image. Basically, wanted to find out if these are being used or just some dummy fields based on how we connect to our Hive tables.   

    Attached files

    hive url.pdf

     



  • 5.  RE: HIVE_URL variable

    Employee
    Posted 08-31-2020 10:26

    Hi JL,

    Please find answers below, you have asked regarding HIVE URL and type.

     

    1. What would  ${HIVE_URL}  look like after it gets populated?

      The Hive URL look like below once get populated.
      jdbc:hive2://HOST1:2181,HOST2:2181,HOST3:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2-hive2
      In order to connect to Hive (Kerberos Enabled environment) we change it to below.
      jdbc:hive2://ip-172-29-68-84.ec2.internal:2181,ip-172-29-68-206.ec2.internal:2181,ip-172-29-68-114.ec2.internal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;principal=hive/ip-172-29-68-206.ec2.internal@EC2.INTERNAL
      The value of URL can also be obtain from Hive service from Hadoop admin console. As shown in screenshot below,




    2. what kind of jdbc connection is being used?

      We are using Type4 jdbc driver.


    3. Does the database channel from the attached image apply to D360?  or is it prepopulated with dummy fields? like the url, username (sagacity) and password. I assume the username is a default from the product since we have never setup a "sagacity" user in our D360 settings nor in our Hadoop environments.

      However, we still didn't see any image attached previously by you. But to answer your question "YES" whenever we create Datastore and point it to database, the dummy fields automatically populate as shown in below picture.

      In DQ+ 3.3 the username and password are dummy (you don't need to use actual service ID/password). But make sure you have replaced the URL with it's actual values as mentioned on above query #1, as well as you have to follow the Fix Pack- 3 instructions Data3Sixty DQ+ 3.3 Fixpack 3.



    To connect with Hive in DQ+4.3 or later version, please follow below instructions.

    Connecting to HIVE as a data source in DQ+ 4.3 or later versions.

     



  • 6.  RE: HIVE_URL variable

    Posted 09-01-2020 07:06

    Thanks Jarrar.  I found it. It was being set as a property variable under the environment settings of Data360 DQ+. And the full URL was there to be found. Thank you for your help.